Abstract
Keywords
Introduction
Peer support and social support are prominent aspects of treatment for people with psychoactive substance–use disorder (PSUD).
1
For example, therapeutic communities have long been a standard residential program approach for people with primary PSUDs.
2
They are characterized by
Peer support as a construct is placed within, and is part of, the broader construct of social support. 7 Social support is the perception that an individual is situated in a supportive network of relationships. 7 It is a multifaceted construct which includes the perception that one is cared for by others as well as the realistic expectation that one will receive tangible assistance from other people when needed. 8 Although social support is conceptualized and operationalized in a number of ways, there is strong evidence that health maintenance and recovery from illness can be influenced significantly by a person's access to supportive others. 9 Social support is linked to self-efficacy and outcome expectancy 10 both of which may have a positive impact on recovery.
Social support and peer support have been separated into many categories. Three kinds of peer support have been described by Schaefer et al: emotional, tangible, and informational support. 11 Schaefer et al differentiate between different aspects of peer support and perceived peer support wherein a social network is defined by the amount or number of social relationships one has. Peer support is defined as a person's impression of the benefits of those social relationships. 11 More recently, Salzer and Shear delineated four kinds of peer support: emotional, instrumental, informational, and affiliational support. 12 In spite of its intuitive appeal and its prominence in treatment programs, there is little evidence that deliberate and systematic research has been undertaken to understand the theoretical underpinnings of peer support and its role in treatment and recovery from PSUD. It appears that peer support variables are a better indication of health and well-being than numbers of social networks. 4 However, given the current body of substance-abuse treatment literature, there are no known norms for analyzing peer support including measuring and operationalizing the construct. Relationships between peer support and theoretical models in broad terms are not known. Therefore, the goal of this research is to use item response theory (IRT) to evaluate a scale designed to measure the peer support for adults treated for SUD and to suggest a possible tie-in of the construct of peer support to the literature on the topic.
Materials and Methods
Purpose
The purpose of this research is to use Rasch analysis to assess a 13-item peer support scale used in a group of adults treated for primary PSUD. The scale is designed to measure the relationship between adults treated for SUDs and their peers. One benefit of providing a measure of peer support to treatment communities is that it will be available for practitioners and researchers to design and evaluate treatment programs.
Participants
Study participants were adults who were treated and discharged from a primary in-patient substance-abuse treatment program located in the Midwest United States. Survey data were collected by independent university-based researchers and included participants who were discharged from treatment from the years 2004–2009. The researchers obtained a list of all the adults who completed treatment successively during the previous year and who provided consent to be part of this posttreatment outcomes study. The sampling frame included only successful treatment discharges in the previous year. The criteria for treatment success comport with prevailing professional standards in the substance-abuse treatment field.
Participants were contacted by telephone and asked to complete a questionnaire of 230 items. The questions contained items in several domains such as friends, family, employment, criminal activity, and relapse – all of which are related to the success of treatment. Part of this questionnaire is the 13-item peer relation scale. The list of potential participants contained names, telephone numbers, emergency contact information, as well as other contact information such as spouses, children, cell phone numbers, place of employment, etc. Great effort was made to contact participants; however, some participants had telephone numbers that were disconnected and not reconnected or they did not provide an updated telephone number and were thus lost to follow-up. These participants were not included in the analysis. Only patients who provided written, informed consent at the time of admission were contacted. The university institutional review board reviewed the research protocol and consent process.
The treatment facility used standard methods in assessing incoming adults and in making level-of-care placements. The American Psychiatric Association's
Data analysis
IRT has many applications one of which is the Rasch model. The Rasch model can provide information for understanding if the total score of an instrument is accurate enough to characterize an individual test taker.
13
In addition, Rasch analysis can be a dynamic form of analysis that provides evidence for why certain responses to an item may be invalid.
13
The data were analyzed using Winsteps software with a one-parameter IRT Rasch model.
13
The Rasch model used for this analysis is the polytomous
where,
Rasch models are successfully employed in well-defined groups of individuals who are responding to assessment items of a latent trait or characteristic. In these instances, the items are scored by integers in ordered categories with an increase in the level of magnitude of the latent trait or characteristic.14–16 The beginning point for estimating the latent trait in Rasch models is the sum of the items.
Results
The descriptive statistics are provided in Table 1. There were 408 participants with a mean age of 36.3 (standard deviation 10.88). The sample was mainly male (65.0%) with 35.0% females. The majority of sample was White (83.1%) while 14.2% were African-American and 1.6% were Latino. Alcohol abuse was reported by 59.1% of the sample. Cocaine was reported by 21.0% of the sample, with opiates and marijuana less frequently reported with 9.4% and 8.9%, respectively.
Demographic characteristics
Valid n = 408. Mean age = 36.3, standard deviation 10.88.
Table 2 shows the peer support scale, response categories, and response frequencies. Several items were reverse coded in order to identify participants who may not have carefully read and answered the questions. All missing values were excluded from the analysis. Rasch analysis skips responses with missing values and eliminates them from the analysis.
Response frequency by category of the 13-item social support scale.
Person and item reliability and separation examination
Table 3 shows reliability testing with separation and reliability coefficients. The person reliability is 0.50 and the Cronbach's alpha person raw score reliability is 0.92. The person reliability score is low while the Cronbach's alpha person raw score is high. The probability of a correct response by an individual to items that are scaled by difficulty is person reliability. 13 The low person reliability score could indicate the need to lengthen the test.
Person and item reliability.
Person raw score-to-measure correlation = 0.65; Cronbach alpha person raw score reliability = 0.92.
Item raw score-to-measure correlation = -0.96; Log-likelihood chi-square: 7,136.17 with 4,011 d.f., P = 0000.
There were 408 measured extreme and nonextreme people in the model. The 37 extreme people were removed and the reliability changed very little (0.50 and 0.51, respectively), ruling out ceiling and floor effects. The item reliability of 0.98 is high, which demonstrates model reliability. The real separation is 6.69, indicating a fair degree of item discrimination; strongly indicating items are placed reliably on the Rasch ruler of about six levels of importance. The real separation statistic shows that the level of the instrument response choices have equal discernment (ie, distance between 1 and 2 is similar to the distance between 3 and 4, etc).
Item difficulty statistics
Table 4 shows all the questions in the 13-item scale. The questions were scored with successive ordinal integers 1 through 5. Table 4 gives item difficulty statistics in misfit order. The Item Number column shows the arrangement of the question in the data. The column headed Total Score is the sum of the scored responses to the corresponding item by survey participants and the column Count gives the completeness for each item that ranges from 387 to 355. The next column displays the statistic for the measure for each item and this is the Rasch estimate of item difficulty or model parameters expressed as logits. Item difficulty shows how items are organized on the scale by how possible it is for the items to be chosen. 13
13-item social support scale, item difficulty statistics in misfit order including infit and outfit statistics.
1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = almost always.
Unidimensionality and fit statistics
Table 4 also provides the infit and outfit statistics. The infit and outfit statistics are representative of how well the data conform to the IRT model.
17
The outfit statistics are based on “conventional sum of squared standardized residuals and infit statistics are based on information-weighted sum squared standardized residuals”.
17
The outfit and infit statistics provide information used to identify “(1) items that are not part of the same dimension; (2) the item is subject to misunderstanding; and (3) the likelihood that a response is guesswork or a person possesses special familiarity or expertise”.
17
The mean-square (MNSQ) and the
Item difficulty statistics
The last pair of statistics measuring randomness of items in the scale in terms of the entire model is under the heading Exact Match in Table 4. The OBS% is the percentage of data points that are within 0.5 score points of their expected values. The EXP% is the percentage of data points that are anticipated to be within 0.5 score points of their expected values. If OBS% is less than the EXP%, then data are more random than what the model predicts and if OBS% is greater than EXP%, then the data are more predictable than what the model predicts 16 The OBS% statistics are greater than the EXP% statistics with the exception of the following items 1, 7, and 13. These items show a small degree of more randomness than the model predicts. Given the magnitude of the infit and outfit statistics in the preceding columns, the finding is appropriate. Additionally, deviation from the predicted randomness is slight except for item 13. The model average OBS% is less than the model average EXP% (63.0, 60.7) and the standard deviations of the averages are proportionally small (19.5 and 17.8, respectively). This indicates that the model comports to the level of predictability inherent in the model.
Table 5 provides the observed average statistic. These statistics are the average of the measures produced in each category by the model and the average measure is anticipated to increase with category value. The following items 1, 3, 4, 5, 6, 7, 8, 9, and 12 are ordered categories because the observed average of items increases with the category value. Items 2 and 13 have one or two categories that are disordered. Ordered items imply that respondents found the response categories to be meaningful, explicit, and understandable. 15 It should be noted that the average ability scores for items 2, 3, and 4 tend to be close.
Observed average by category.
Equals disordered category.
Discussion
Given the prominence of therapeutic communities as a mode of primary treatment, it is important to both the academic and treatment communities to have a standard way to measure peer support. The scale presented here can be useful for this purpose.
The psychometric properties of the instrument showed that the person reliability score is low (0.50); however, increasing the length of the test could potentially enhance the person reliability score. Another noteworthy finding is that it is clear from the infit and outfit coefficients that item 13 exhibits a high degree of variation from the model prediction than is commonly accepted. Similarly, the
Another improvement that could be made to the scale would be to collapse categories 2 and 3 or to collapse categories 3 and 4. This would eliminate three of the four disordered categories and potentially enhance the usability of the scale.
Additionally, participants had some difficulty differentiating between the choices
Conclusion
Despite these improvements, our results give treatment providers and researchers an opportunity to use a psychometrically sound instrument to determine treatment success in terms of peer support. Treatment providers may be better able to create or enhance treatment programs and they may be better able to assess posttreatment abstinence based on peer support. Measures of peer support will also enable treatment provided to better design treatment approaches because providers will be better able to screen, track, and measure outcomes of care. Some advantages to this instrument is that it is short, easy to administer, and can easily be used as a screening tool. The questions are based on the actual experiences of the patients and the results of the scale can be useful to treatment providers by creating a feedback loop. Because we know that the scale is psychometrically valid, it makes for a good research instrument as well. For example, from a research perspective, the case can be made for cardinality. The data gathered can be used in an array of quantitative research.
Author Contributions
Principle investigator of the study: JC. Designed the study and helped draft the manuscript: JC. Helped draft the manuscript: KM. Both authors approved of the final manuscript.