Item Response Analysis of the Positive and Negative Syndrome Scale
Item Response Analysis of the Positive and Negative Syndrome Scale
Background: Statistical models based on item response theory were used to examine (a) the performance of individual Positive and Negative Syndrome Scale (PANSS) items and their options, (b) the effectiveness of various subscales to discriminate among individual differences in symptom severity, and (c) the appropriateness of cutoff scores recently recommended by Andreasen and her colleagues (2005) to establish symptom remission.
Methods: Option characteristic curves were estimated using a nonparametric item response model to examine the probability of endorsing each of 7 options within each of 30 PANSS items as a function of standardized, overall symptom severity. Our data were baseline PANSS scores from 9205 patients with schizophrenia or schizoaffective disorder who were enrolled between 1995 and 2003 in either a large, naturalistic, observational study or else in 1 of 12 randomized, double-blind, clinical trials comparing olanzapine to other antipsychotic drugs.
Results: Our analyses show that the majority of items forming the Positive and Negative subscales of the PANSS perform very well. We also identified key areas for improvement or revision in items and options within the General Psychopathology subscale. The Positive and Negative subscale scores are not only more discriminating of individual differences in symptom severity than the General Psychopathology subscale score, but are also more efficient on average than the 30-item total score. Of the 8 items recently recommended to establish symptom remission, 1 performed markedly different from the 7 others and should either be deleted or rescored requiring that patients achieve a lower score of 2 (rather than 3) to signal remission.
Conclusion: This first item response analysis of the PANSS supports its sound psychometric properties; most PANSS items were either very good or good at assessing overall severity of illness. These analyses did identify some items which might be further improved for measuring individual severity differences or for defining remission thresholds. Findings also suggest that the Positive and Negative subscales are more sensitive to change than the PANSS total score and, thus, may constitute a "mini PANSS" that may be more reliable, require shorter administration and training time, and possibly reduce sample sizes needed for future research.
The Positive and Negative Syndrome Scale (PANSS) is the most widely used measure of symptom severity in schizophrenia. This 30-item scale is typically administered by trained clinicians who evaluate patients' current severity level on each symptom (item) by endorsing 1 of 7 options (weights) numbered 1 through 7. The PANSS has demonstrated high internal reliability, good construct validity, and excellent sensitivity to change in both short term and long term trials. However, despite extensive psychometric research, it is unclear how individual PANSS items differ in their usefulness in assessing the severity of schizophrenia.
Indeed, studies examining the psychometric properties of the PANSS have focused, without exception, on estimates of scale reliability, validity, and factor analysis using methods from classical test theory and have typically identified 5 underlying factors. Approaches based on classical test theory rely primarily on omnibus statistics that average over levels of individual variation and offer no means to gauge the quality of individual items or options across different levels of symptom severity. In contrast, methods based on item response theory (IRT) provide significant improvements over classical techniques, as they model the relation between item responses and symptom severity directly, quantifying how the performance of individual items and options (severity levels 1 to 7) change as a function of overall, standardized, symptom severity. IRT analyses provide unique and relevant information concerning (a) how well a set of item options assess the entire continuum of symptom severity, (b) whether weights assigned to individual item options are appropriate for measuring a particular trait or symptom, and (c) how well individual items or subscales are connected to the underlying construct and discriminate among individual differences in symptom severity (see the publication by Santor and Ramsay for an overview).
IRT methods are ideal for examining the performance of options within items that are to be used to define remission of psychopathology. Andreasen and her colleagues published guidelines recommending that schizophrenia remission be defined as achieving option scores less than or equal to 3 on each of only 8 PANSS items: Delusions, Unusual Thought Content, Hallucinatory Behavior, Conceptual Disorganization, Mannerisms and Posturing, Blunted Affect, Social Withdrawal, and Lack of Spontaneity. Setting equal remission thresholds (≤3) for all 8 items suggests that the level of symptom severity corresponding to "3 or less" is generally equal for all 8 items. If the region of symptom severity at which Options 1, 2, and 3 are most likely to be endorsed differs across items, then some items are more influential than others in determining whether or not remission has been met. Alternately, if the threshold of "3 or less" corresponds to higher severity for an item (i.e., is more easily achieved as symptom severity improves), then either that item is redundant (since it is more likely to be reached first) or else its threshold should be revised downward. For example, the remission threshold for that item could be set at 2 rather than at 3. IRT analyses examine the manner in which individual item options (and cutoff scores) are related to overall symptom severity which is central to evaluating the appropriateness of proposed cut-point scores (thresholds) determining illness remission.
The 3 primary purposes of our study are (a) to examine and characterize the performance of individual items from the PANSS at both the option (severity) and item (symptom) levels with the goal of identifying areas for improvement, (b) to examine the ability of various subscales to discriminate among individual difference in illness severity, which might then identify a better measure of change, and (c) to evaluate the appropriateness of items and options proposed for determining when symptoms remission has been achieved.
Abstract and Background
Abstract
Background: Statistical models based on item response theory were used to examine (a) the performance of individual Positive and Negative Syndrome Scale (PANSS) items and their options, (b) the effectiveness of various subscales to discriminate among individual differences in symptom severity, and (c) the appropriateness of cutoff scores recently recommended by Andreasen and her colleagues (2005) to establish symptom remission.
Methods: Option characteristic curves were estimated using a nonparametric item response model to examine the probability of endorsing each of 7 options within each of 30 PANSS items as a function of standardized, overall symptom severity. Our data were baseline PANSS scores from 9205 patients with schizophrenia or schizoaffective disorder who were enrolled between 1995 and 2003 in either a large, naturalistic, observational study or else in 1 of 12 randomized, double-blind, clinical trials comparing olanzapine to other antipsychotic drugs.
Results: Our analyses show that the majority of items forming the Positive and Negative subscales of the PANSS perform very well. We also identified key areas for improvement or revision in items and options within the General Psychopathology subscale. The Positive and Negative subscale scores are not only more discriminating of individual differences in symptom severity than the General Psychopathology subscale score, but are also more efficient on average than the 30-item total score. Of the 8 items recently recommended to establish symptom remission, 1 performed markedly different from the 7 others and should either be deleted or rescored requiring that patients achieve a lower score of 2 (rather than 3) to signal remission.
Conclusion: This first item response analysis of the PANSS supports its sound psychometric properties; most PANSS items were either very good or good at assessing overall severity of illness. These analyses did identify some items which might be further improved for measuring individual severity differences or for defining remission thresholds. Findings also suggest that the Positive and Negative subscales are more sensitive to change than the PANSS total score and, thus, may constitute a "mini PANSS" that may be more reliable, require shorter administration and training time, and possibly reduce sample sizes needed for future research.
Background
The Positive and Negative Syndrome Scale (PANSS) is the most widely used measure of symptom severity in schizophrenia. This 30-item scale is typically administered by trained clinicians who evaluate patients' current severity level on each symptom (item) by endorsing 1 of 7 options (weights) numbered 1 through 7. The PANSS has demonstrated high internal reliability, good construct validity, and excellent sensitivity to change in both short term and long term trials. However, despite extensive psychometric research, it is unclear how individual PANSS items differ in their usefulness in assessing the severity of schizophrenia.
Indeed, studies examining the psychometric properties of the PANSS have focused, without exception, on estimates of scale reliability, validity, and factor analysis using methods from classical test theory and have typically identified 5 underlying factors. Approaches based on classical test theory rely primarily on omnibus statistics that average over levels of individual variation and offer no means to gauge the quality of individual items or options across different levels of symptom severity. In contrast, methods based on item response theory (IRT) provide significant improvements over classical techniques, as they model the relation between item responses and symptom severity directly, quantifying how the performance of individual items and options (severity levels 1 to 7) change as a function of overall, standardized, symptom severity. IRT analyses provide unique and relevant information concerning (a) how well a set of item options assess the entire continuum of symptom severity, (b) whether weights assigned to individual item options are appropriate for measuring a particular trait or symptom, and (c) how well individual items or subscales are connected to the underlying construct and discriminate among individual differences in symptom severity (see the publication by Santor and Ramsay for an overview).
IRT methods are ideal for examining the performance of options within items that are to be used to define remission of psychopathology. Andreasen and her colleagues published guidelines recommending that schizophrenia remission be defined as achieving option scores less than or equal to 3 on each of only 8 PANSS items: Delusions, Unusual Thought Content, Hallucinatory Behavior, Conceptual Disorganization, Mannerisms and Posturing, Blunted Affect, Social Withdrawal, and Lack of Spontaneity. Setting equal remission thresholds (≤3) for all 8 items suggests that the level of symptom severity corresponding to "3 or less" is generally equal for all 8 items. If the region of symptom severity at which Options 1, 2, and 3 are most likely to be endorsed differs across items, then some items are more influential than others in determining whether or not remission has been met. Alternately, if the threshold of "3 or less" corresponds to higher severity for an item (i.e., is more easily achieved as symptom severity improves), then either that item is redundant (since it is more likely to be reached first) or else its threshold should be revised downward. For example, the remission threshold for that item could be set at 2 rather than at 3. IRT analyses examine the manner in which individual item options (and cutoff scores) are related to overall symptom severity which is central to evaluating the appropriateness of proposed cut-point scores (thresholds) determining illness remission.
The 3 primary purposes of our study are (a) to examine and characterize the performance of individual items from the PANSS at both the option (severity) and item (symptom) levels with the goal of identifying areas for improvement, (b) to examine the ability of various subscales to discriminate among individual difference in illness severity, which might then identify a better measure of change, and (c) to evaluate the appropriateness of items and options proposed for determining when symptoms remission has been achieved.
Source...