Sage Journals: Discover world-class research

Abstract

Background:

Musculoskeletal disorders are the second leading cause of disability worldwide.

Objective:

Examine experiences of chiropractic patients in the United States with chronic low back or neck pain.

Method:

Observational study of 1853 chronic low back pain and neck pain patients (74% female) who completed an online questionnaire at the 3-month follow-up that included Consumer Assessment of Healthcare Providers and Systems (CAHPS) items assessing their experiences with care.

Results:

We found similar reports of communication for the chiropractic sample and patients in the 2016 CAHPS National Database, but 85% in the database versus 79% in the chiropractic sample gave the most positive response to the time spent with provider item. More patients in the CAHPS database rated their provider at the top of the scale (8 percentage points). More chiropractic patients reported always getting answers to questions the same day (16 percentage points) and always being seen within 15 minutes of their appointment time (29 percentage points).

Conclusions:

The positive experiences of patients with chronic back and neck pain are supportive of their use of chiropractic care.

Keywords

patient experience chronic pain chiropractic CAHPS

Introduction

Musculoskeletal disorders are among the most prevalent health problems and the second leading cause of disability worldwide (1). Low back pain prevalence for adults in the United States is about 20% (2). Mafi et al (3) found that in contrast to national guidelines, “management of routine back pain increasingly has relied on advanced diagnostic imaging, referrals to other physicians, and use of narcotics, with a concomitant decrease in nonsteroidal anti-inflammatory drug (NSAID) or acetaminophen use and no change in physical therapy referrals” (p. 1580). They concluded that treatment of back pain represented an area of potential health-care cost savings in the future.

More than 50% of US adults have sought care from a chiropractor and about 30% of those with spinal pain in the United States have used chiropractic care (4). Spinal manipulation is recommended by the American College of Physicians as a noninvasive treatment of low back pain (5). A recent study found that chiropractic care for patients with chronic low back pain or neck pain was associated with significant 3-month improvements in all PROMIS-29 v2.0 health-related quality of life measures except emotional distress (6).

High levels of patient satisfaction with chiropractic treatment have been consistently reported (7 –10). For example, the average score on the 14-item chiropractic satisfaction questionnaire (administered using a 7-category response scale (very poor, poor, fair, good, very good, excellent, the best) in a sample of 486 patients of 44 chiropractors was in-between excellent and the best (8). Another study found that satisfaction of chiropractic patients with chronic low back pain was higher than that of patients of family physicians (11). But the sample consisted of only 71 chiropractic patients and 35 family medicine patients and used an ad hoc measure of satisfaction with ceiling effects (eg, 100% of the chiropractic patients agreed that the chiropractor felt their pain was real). A more recent study of 5422 members of the Gallup panel found that the majority perceived chiropractic care to be effective in treating back and neck pain and that chiropractors were trustworthy (4).

A robust comparison of chiropractic patient and medical patient care experiences requires use of a standardized measure. The Consumer Assessment of Healthcare Providers and Systems (CAHPS) project has advanced scientific understanding of the patient experience of care by developing standardized surveys that are in wide use throughout the United States. The CAHPS survey items represent what consumers value and for which they are the best source of information. The CAHPS Clinician & Group Survey is used extensively to assess ambulatory care delivered by provider groups and individual health-care providers (12,13).

We conducted an observational study of a sample of chronic low back pain and neck pain patients to evaluate their perceptions of the chiropractic care received. We administered CAHPS Clinician & Group Survey 3.0 items and supplemented them with other items appropriate for chiropractic care. This study provides information on the experiences of a national sample of chiropractic patients in the United States with chronic pain. These data were collected as part of a project to evaluate the appropriateness of manipulation and mobilization for chronic low back pain and neck pain.

Methods

We used multistage systematic stratified sampling with 4 levels: regions/states, sites (ie, metropolitan areas), providers/clinics, and patients (14). We recruited chiropractic practices in 6 states from major geographical regions of the United States: San Diego, California; Tampa, Florida; Minneapolis, Minnesota; Seneca Falls/Upstate, New York; Portland, Oregon; and Dallas, Texas.

We sought to recruit 20 or more chiropractic providers/clinics per site and to reflect the national proportions of provider gender, years of experience, and patient load as shown in the 2015 Practice Analysis Report from the National Board of Chiropractic Examiners (15). Our aim was to recruit 30% female practitioners, 30% with 5 to 15 years of experience and the rest with more than 15 years of experience, and equal proportions of those treating 25 to 74 patients per week versus 75 or more patients per week. We excluded providers who had more than half their patients with open personal injury/workers compensation litigation, because treatment patterns for these patients differs (eg, less radiographic use) from that of other patients (16). We also excluded providers who do not use manual manipulation or mobilization (ie, instrument-assisted-only practice). We used multiple approaches to recruit providers including announcements in journals, attending chiropractic conferences, social media, e-mail, snowball sampling, and key informants (17).

In addition to posters and fliers notifying patients about the study, the front desk staff at each clinic was asked to offer a prescreening questionnaire available to every patient who visited the clinic during a 4-week period and to keep a daily tally of all patients seen by participating chiropractors. This prescreening questionnaire was self-administered on an iPad and used to determine whether patients met the study inclusion/exclusion criteria: at least 21 years of age, could speak English well enough to complete the remaining questionnaires, not presently involved in ongoing personal injury/workers compensation litigation, and have now or ever had chronic low back or neck pain. Patients who met these criteria were invited to be in the study, and if they agreed, they were asked to provide their e-mail addresses and a phone number. All patients who provided e-mail addresses received an electronically delivered $5 USD gift card.

Patients invited to the study were e-mailed a longer screening questionnaire to determine whether they met the study criteria for chronic low back pain and chronic neck pain (ie, reported pain for at least 3 months prior to seeing the chiropractor and/or stated that their pain was chronic). If they were eligible for the study, patients were then consented and asked additional questions. Those not eligible and those who were eligible and started this screening questionnaire but did not finish it received a $5 USD gift card. Those eligible who consented and went on to complete the remaining questions on this survey received a $20 USD gift card and were then invited to complete subsequent surveys including a baseline and 3-month follow-up questionnaire. Participants received a $25 USD gift card for completing the baseline questionnaire and $25 USD gift card for completing the 3-month follow-up questionnaire.

Patient perceptions of care were assessed at the 3-month follow-up. Prior to the start of the longitudinal study, we conducted 6 focus groups (2 in Los Angeles, 2 in Chicago, and 2 in Boston) with patients to identify key aspects of experiences with chiropractic care. Based on focus group input and the literature (18), we selected items in the CAHPS Clinician & Group Survey 3.0 relevant to chiropractic care (3 access to care items, 4 communication items, and 1 global rating of the provider item). We supplemented these items with 2 additional access to care items, 5 additional communication items, 1 global rating of office appearance item, 4 items assessing office assistants, 1 item on insurance coverage, and 3 items assessing perceived outcomes of care. Prior to the main study data collection, we conducted 13 cognitive interviews to ensure the patient experience items were understood by patients, followed by a pilot study with 55 patients. The items administered in the longitudinal study are shown in the Appendix.

The study was approved by the RAND Corporation Human Subjects Protection Committee (#2013-0763) and was registered as an observational study on ClinicalTrials.gov (ID: NCT03162952).

Analysis Plan

All items were transformed linearly to a 0 to 100 possible range, with a higher score representing more positive experience with care. We created 7 patient-reported measures from the 25 items: access to care (5 items), communication (9 items), administrative assistant (4 items), overall ratings (2 items), office appearance (1 item), perceived outcomes (3 items), and insurance met expectations (1 item). We estimated internal consistency reliability (coefficient α) for the 5 multi-item scales (19) and clinic-level intraclass correlations for the 5 scales and 2 single-item measures (20). Nunnally (21) suggested reliability thresholds of 0.70 and 0.90 for group-level and individual-level comparisons, respectively. We applied the Spearman-Brown prophecy formula (22,23) to the intraclass correlation to estimate the sample sizes per clinic needed to achieve 0.70 and 0.90 reliability. Further, to examine potential selection bias we estimated correlations of the 7 patient experience measures with years seeing a chiropractor for pain, years seeing the chiropractor seen in this study for pain, number of visits to this chiropractor overall, and number of visits to this chiropractor in the last 6 months.

We compared responses to CAHPS items in the sample to those of 137 416 adult patients from 656 practice sites (370 Midwest, 145 West, 139 Northeast, 2 South; 257 hospital/health systems, 232 provider/physicians, 145 university/academic medical centers, 8 community health centers, 14 other) in the 2016 CAHPS Clinician and Group Database (24). California (n = 29 355), Minnesota (n = 24 699), Michigan (n = 21 819), and Massachusetts (n = 19 969) were the states with the most patients. The modal number of patients in the database were sampled from family practice (n = 39 078); between 3158 and 6538 patients were included from surgery, obstetrics/gynecology, ophthalmology, cardiology, and orthopedics specialties.

The CAHPS items were administered using a 6-month reporting window in the database while we used a 3-month reporting window in the chiropractic sample to cover the time between baseline and the 3-month follow-up assessment. In addition, we paid study participants to complete study questionnaires but participants in the CAHPS database were not paid. We computed 2 group (chiropractic sample vs CAHPS database) z tests of the significance of differences (P < .05, 2 tailed) in the proportions of patients picking the most positive (“top-box”) response to each of the parallel items.

Finally, we report descriptive statistics for responses to the 3 items assessing chiropractic patients perceived outcomes of care.

All analyses were conducted using SAS 9.4 (TS1M3). Clinic-level reliability was estimated using a SAS macro (25).

Results

A total of 2646 (94%) of the 2829 patients eligible for the study consented to be in it; 2024 (76%) of the 2646 completed a baseline questionnaire; 1835 (91%) of these completed the 3-month follow-up survey that includes the patient experience items. Table 1 summarizes the demographic characteristics of these 1835 patients. The average age of the end point sample was 49, 74% were female, and the majority had a college degree, were non-Hispanic white, worked full time, and had an annual income of $60 000 or more. The demographic characteristics of those who completed the 3-month survey was very similar to that of the baseline sample (results available upon request). The average score on the Patient-Reported Outcomes Measurement Information System v2.0 physical health summary scale reported by the sample on the 3-month survey was about a third of a standard deviation worse than the US general population (6).

Table 1.

Demographic Characteristics of the Sample.^a

Age	Mean = 49 (range: 21-95) (%)
Age 50+	50
Female (%)	74
Education
Less than high school	0.3
High school/general education diploma	7
Some college	37
Bachelor’s degree or higher	56
Race/ethnicity
Hispanic	5
Non-Hispanic
White	88
Asian	3
African-American	2
American Indian/Pacific Islander/Other	2
Working full time	59
Gross income (USD)
Income < $10 000	2
$10 000 ≥ income > $60 000	37
$60 000 ≥ income > $100 000	30
Income ≥ $100 000	32

^an = 1835.

Table 2 provides means, standard deviations, and reliability estimates for the patient experience measures. Means scores (0-100 possible range) ranged from 81 (insurance met expectations) to 95 (administrative assistant). Internal consistency reliabilities for the 5 multi-item scales ranged from 0.60 (administrative assistant) to 0.86 (communication). Four of these reliabilities met the 0.70 threshold for satisfactory reliability for group comparisons (21). Intraclass correlations for the 125 clinics in the sample ranged from 0.012 (perceived outcomes) to 0.101 (administrative assistant). The estimated number of patient responses per clinic needed to achieve 0.70 and 0.90 reliabilities, respectively, is access (33, 128), communication (33, 125), administrative assistant (21, 80), overall ratings (68, 260), office appearance (32, 122), perceived outcomes (197, 758), and insurance met expectations (117, 451).

Table 2.

Descriptive Statistics and Reliability Estimates for Patient Experience Measures.^a

Measure (Number of Items)	Mean	SD	α	Clinic-Level Intraclass Correlation
Access (5)	87	18	0.71	0.066
Communication (9)	86	17	0.86	0.067
Administrative assistant (4)	95	12	0.60	0.101
Global ratings (2)	89	14	0.71	0.033
Office appearance (1)	90	13	NA	0.069
Perceived outcomes (3)	82	17	0.74	0.012
Insurance met expectations (1)	81	31	NA	0.020

Abbreviation: NA, not applicable for single items.

^aMeasures are scored on 0 to 100 possible range with a higher score indicating more positive perceptions of care.

As seen in Table 3, correlations among the patient experience measures ranged from 0.03 (insurance met expectations with communication) to 0.64 (global ratings of care and perceived outcomes of care). The perceived outcomes of care scale had a significant association with every other measure except the question about whether insurance met expectations. The insurance met expectations item had the smallest correlations with other measures (r’s ranging from 0.03 to 0.14).

Table 3.

Product-moment Correlations Among Patient Experience Measures.^a

Measure	Communication	Administrative Assistant	Global Ratings	Office Appearance	Perceived Outcomes	Insurance Expectations Met
Access	0.59	0.34	0.40	0.27	0.33	0.07
Communication	1.00	0.36	0.62	0.39	0.50	0.03
Administrative assistant		1.00	0.32	0.27	0.24	0.08
Global ratings			1.00	0.50	0.64	0.05
Office appearance				1.00	0.28	0.14
Perceived outcomes					1.00	0.05

^aAll correlations significant at P < .0001 except for correlations of insurance expectations met with access and administrative assistant (P < .05) and correlations of insurance expectations met with communication, global ratings, and perceived outcomes (P > .05).

The 2-item global ratings of care scale were significantly positively associated with the total number of visits with the study chiropractor (r = 0.05; P = .0208), length of time the patient had been getting chiropractic care for pain (r = 0.07; P = .0022), and length of time the patient had been seeing the study chiropractor (r = 0.09; P < .0001). There were only 3 other significant associations with one of the chiropractor history variables: positive associations between access to care and length of time the patient received chiropractic care for pain (r = 0.06; P = .0141), perceptions of the administrative assistant and total visits with the study chiropractor (r = 0.05; P = .0420), and office appearance with how long the patient had been seeing the study chiropractor (r = .08; P = .0013).

Corresponding CAHPS items for patients in this sample compared to the 2016 CAHPS database are given in Table 4. Responses to the corresponding communication items were very similar, but those in the chiropractic sample were more likely to give the most positive response to the time spent with provider item (6 percentage points; z = 7.14; P < .001) and provider listens carefully to you (3 percentage points; z = 3.92; P < .001) than those in the CAHPS database. In addition, those in the CAHPS database sample were more likely than those in the chiropractic sample to rate their provider a 10, the most positive response (8 percentage points; z = 7.05; P < .001). The chiropractic patients reported somewhat better access to care than the medical patients. More of the chiropractic patients reported getting an appointment for urgent care always (3 percentage points; z = 2.76; P = .006) and getting answers to their questions after hours as soon as they needed (4 percentage points; z = 3.51; P < .001). A substantial greater percentage of chiropractic patients reported always getting answers to question the same day (16 percentage points; z = 22.72; P < .001) and always being seen within 15 minutes of their appointment time (29 percentage points; z = 25.15; P < .001).

Table 4.

CAHPS Item Responses for Chiropractic Sample and 2016 CAHPS Database.^a

Item	Chiropractic Sample (n = 1835; %)	CAHPS Database (n = 137 416; %)
Communication
How often did this chiropractor (provider) explain things in a way that was easy to understand?
Never	2	1
Sometimes	2	2
Usually	10	10
Always	86	86
How often did this chiropractor (provider) listen carefully to you?
Never	1	1
Sometimes	3	2
Usually	10	8
Always	85	88^b
How often did this chiropractor (provider) show respect for what you had to say?
Never	1	1
Sometimes	1	2
Usually	6	6
Always	92	91
How often did this chiropractor (provider) spend enough time with you?
Never	1	1
Sometimes	3	3
Usually	17	11
Always	79	85^b
Global rating
What number would you use to rate this chiropractor (provider)?
0-6	3	4
7-8	17	13
9	25	19
10	55	63^b
Access
When you contacted this chiropractor’s (provider’s) office to get an appointment for care you needed right away, how often did you get an appointment as soon as you needed?
Never	2	3
Sometimes	5	7
Usually	21	21
Always	72	69^c
When you contacted this chiropractor’s (provider’s) office during regular office hours, how often did you get an answer to your question that same day?
Never	0.5	5
Sometimes	3	9
Usually	10	25
Always	87	61^b
When you contacted this chiropractor’s (provider’s) office after regular office hours, how often did you get an answer to your question as soon as you needed?
Never	0	7
Sometimes	13	9
Usually	21	22
Always	66	62^b
How often did you see this chiropractor (provider) within 15 minutes of your appointment time?
Never	6	10
Sometimes	6	14
Usually	19	36
Always	69	40^b

Abbreviation: CAHPS, Consumer Assessment of Healthcare Providers and Systems.

^aThe CAHPS items were administered using a 6-month reporting window in the database while we used a 3-month reporting window in the chiropractic sample to cover the time between baseline and the 3-month follow-up assessment.

^bz test of significance of differences in percentages in most positive response category: P < .001.

^cz test of significance of differences in percentages in most positive response category: P < .01.

Table 5 shows frequencies for the perceived outcomes items. Seven of 10 of the chiropractic patients felt that the treatment made them feel much better (71%) and helped them a lot (73%). At least some improvement in pain over the last 3 months was reported by 86% of the patients, with 22% indicating a lot of improvement.

Table 5.

Perceptions of the Outcomes of Chiropractic Care.

Question	Percent
In the last 3 months, on average, how did the treatment from the chiropractor make you feel?
Much worse	0.2
A little worse	0.5
No change	2
A little better	26
Much better	71
In the last 3 months, how much did the treatment from the chiropractor help you?
Not at all	1
A little bit	5
Somewhat	21
A lot	73
Compared to how you felt 3 months ago, how much improvement in pain have you had?
No improvement	3
A little bit of improvement	12
Some improvement	26
Quite a bit of improvement	38
A lot of improvement	22

Discussion

The 25-patient experience items administered in the study were a combination of adaptation of 8 items in the CAHPS Clinician & Group Survey 3.0 and 17 items targeted at chiropractic care for chronic neck and low back pain. We found strong support for the reliability of the measures we used (5 multi-item scales and 2 single items). We found from 21 patients per clinic (administrative assistant) to 197 patients per clinic (perceived outcomes) would be needed to obtain 0.70 reliability at the clinic level. This is consistent with the number of completed surveys recommended per physician group for the CAHPS Clinician & Group Survey 3.0: 50 if there is a single physician, 100 if 2 physicians, and 150 if 3 physicians: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/cg/survey3.0/adult-eng-cg30-2351a.pdf. The perceived outcomes scale requires the largest number of completes because it varies the least across clinics and has the smallest intraclass correlation.

Because we included some CAHPS items (adapted to chiropractic) in our study, we were able to compare chiropractic experiences with experiences of a large sample of patients receiving ambulatory medical care. Although a small study of chiropractic and family medicine patients with chronic low back pain reported substantially higher levels of patient satisfaction among the chiropractic patients (8), we found similar patient experiences with communication in our sample of chiropractic patients with chronic low back or neck pain compared to medical patients on corresponding CAHPS survey items. Chiropractic patients reported more positive experiences with access to care than the medical patients. Hence, the current research adds to our understanding of the relative perceptions of chiropractic versus traditional medical care. However, the CAHPS database does not have information on chronic conditions so we are unable to subset it to patients with chronic low back pain or neck pain.

The CAHPS survey was designed to assess the experiences of patients with traditional medical care and uses a 6-month recall interval. Not all the CAHPS items could be administered because some were not applicable to chiropractic. In addition, we administered the items using a 3-month recall interval to correspond to the interval between baseline and follow-up in our study. Further, the chiropractic patients in this study were paid to complete a questionnaire that included the patient experience survey items but patients in the CAHPS database were not paid.

Because not all those eligible for the study participated in it, there is a possibility of selection bias (eg, those who participated tended to have received chiropractic care longer and, therefore, to have positive care experiences). But we found only 6 of 28 product–moment correlations between the 7 patient experience measures and the 4 utilization of care measures were statistically significant (P < .05). The largest correlation (r = 0.09) indicated a trivia association between greater utilization and more positive perceptions of care, suggesting that selection bias may not be large. In addition, selection bias could also apply to the CAHPS database. Some of the patients invariably received treatment other than chiropractic during the study. Thus, any changes in health may have been affected by that. But it is unlikely that receipt of this other care would have impacted their perceptions of chiropractic care, the focus of this article.

The results of this study contribute to the literature by providing evidence that experiences with chiropractic care are generally positive among patients with chronic back or neck pain. The study findings provide empirical verification of why some chronic pain patients utilize chiropractic care on a regular basis. It supports the use of chiropractic care as one option for improving functioning and well-being of patients with chronic low back pain or neck pain (26).

Anhang Price et al (27) conducted a systematic review of the literature and concluded that most studies indicated either positive or null associations between patient experiences and best practice clinical processes, lower hospital readmissions, and desirable clinical outcomes. Future research is needed to examine the associations of patient reports about care and expert ratings of the appropriateness of chiropractic care (28).

Footnotes

Authors' Note

Lara G Hilton is also affiliated with Deloitte Consulting LLP,Los Angeles,CA,USA.

Lan Coulter is also affiliated with UCLA School of Dentistry,Los Angeles,CA,USA.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: The authors disclose support from NIH National Center for Complementary and Integrative Health Grant No. 1U19AT007912-01.

ORCID iD

Ron D Hays,PhD

References

Vos

Flaxman

Naghavi

Lozano

Michaud

Ezzati

, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2163–96.

Shmagel

Foley

Ibrahim

. Epidemiology of chronic low back pain in US adults: national health and nutrition examination survey 2009-2010. Arthritis Care Res. 2016;68:1688–94.

Mafi

McCarthy

Davis

Landon

. Worsening trends in the management and treatment of back pain. JAMA Intern Med. 2013;173:1573–81.

Weeks

Goertz

Meeker

Marchiori

. Public perceptions of doctors of chiropractic: results of a national survey and examination of variation according to respondents’ likelihood to use chiropractic, experience with chiropractic, and chiropractic supply in local health care markets. J Manipulative Physiol Ther. 2015;38:533–44.

Qaseem

Wilt

McLean

Forciea

; Clinical Guidelines Committee of the American College of Physicians. Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2017;166:514–30.

Hays

Spritzer

Sherbourne

Ryan

Coulter

. Group and individual change in health-related quality of life in chiropractic patients with chronic low back or neck pain. Spine. 2018.

Alcantara

Ohm

Alcantara

. The use of PROMIS and the RAND VSQ9 in chiropractic patients receiving care with the Webster technique. Complement Ther Clin Practice. 2016;23:110–6.

Coulter

Hays

Danielson

. The chiropractic satisfaction questionnaire. Top Clin Chiropr. 1994;1:40–3.

Emary

Brown

Cameron

Pessoa

Bolton

. Management of back pain-related disorders in a community with limited access to health care services: a description of integration of chiropractors as service providers. J Manipulative Physiol Ther. 2017;40:635–42.

10.

MacPherson

Newbronner

Chamberlain

Hopton

. Patients’ experiences and expectations of chiropractic care: a national cross-sectional survey. Chiropr Man Therap. 2015;23:3.

11.

Nyiendo

Haas

Goodwin

. Patient characteristics, practice activities, and one-month outcomes for chronic, recurrent low-back pain treated by chiropractors and family medicine physicians: a practice-based feasibility study. J Manipulative Physiol Ther. 2000;23:239–45.

12.

Dyer

Sorra

Smith

Cleary

Hays

. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) clinical and group adult visit survey. Med Care. 2012;50:S28–34.

13.

Quigley

Martino

Brown

Hays

. Evaluating the content of the communication items in the CAHPS® clinician and group survey and supplemental items with what high-performing physicians say they do. Patient. 2013;6:169–77.

14.

Herman

Hilton

Sorbero

Rutter

Hays

Hilton

, et al. Characteristics of chiropractic patients being treated for chronic low back and chronic neck pain. J Manipulative Physiol Ther. 2018;41:445–55.

15.

Christensen

Hyland

Goertz

Kollasch

. Practice Analysis of Chiropractic 2015. Greeley, CO: National Board of Chiropractic Examiners; 2015.

16.

Blanchette

Cassidy

Rivard

Dionne

. Chiropractors’ characteristics associated with their number of workers’ compensation patients. J Can Chiropr Assoc. 2015;59:202–15.

17.

Coulter

Whitley

Ryan

, et al. Researching the appropriateness of care in the complementary and integrative health professions Part 4: putting the practice into EBP in chiropractic: recruiting clinics and patients. J Manipulative Physiol Ther. IN PRESS.

18.

Coulter

Hurwitz

Spritzer

Genovese

Hays

. A chiropractic supplemental item set for the consumer assessment of health plans study. Top Clin Chiropr. 2000;7:50–6.

19.

Cronbach

. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.

20.

Shrout

Fleiss

. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.

21.

Nunnally

. Psychometric Theory. 2nd ed. New York, NY: McGraw-Hill; 1978.

22.

Brown

. Some experimental results in the correlation of mental abilities. Br J Psychol. 1910;3:209–34.

23.

Spearman

. Correlation calculated from faulty data. Br J Psychol. 1910;3:271–95. 1910;3:296-322.

24.

CAHPS Database: 2016 CAHPS Clinician & Group Survey Database 2016 Chartbook: What Patients Say About Their Health Care Providers and Medical Practices. Agency for Healthcare Research and Quality. July 2017, https://cahpsdatabase.ahrq.gov/files/2016CAHPSClinicianGroupChartbook.pdf (accessed April 16, 2019).

25.

Hays

Wang

Sonksen

. General reliability and intraclass correlation program (GRIP). In: SAS Conference Proceedings: Western Users of SAS Software; September 27-29, 1995; Long Beach, CA; 220–223. Cary, NC: SAS Institute Inc; 1995.

26.

Goertz

Long

Vining

Pohlman

Kane

Corber

. Assessment of chiropractic treatment for active duty, U.S. military personnel with low back pain: study protocol for a randomized controlled trial. Trials. 2016;17:70.

27.

Anhang Price

Elliott

Zaslavsky

Hays

Lehrman

Rybowski

, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 2014;71:522–54.

28.

Coulter

Crawford

Hurwitz

Vernon

Khorsan

Suttorp Booth

, et al. Manipulation and mobilization for treating chronic low back pain: a systematic review and meta-analysis. Spine. 2018;18:866–79.