Abstract
Keywords
What is already known?
This article contributes to the class of health literacy studies concerned with how to engage vulnerable participants in ways that are ethical and empowering (Kilbourne, Switzer, Hyman, Crowley-Matoka, & Fine, 2006; Zion, Gillam, & Loff, 2000). It draws on frameworks based on critical methodologies, community-based approaches, and usability studies.
What this paper adds?
The methods presented in this article extend the user-centered paradigm by designing a study in which participants are tasked with assessing health information resources rather than a study that assesses participants’ comprehension of these materials. By focusing on document features rather than participant skills, these methods can be applied to studies that engage participants who might experience similar challenges when practicing health literacy such as English-language learners or adults with certain intellectual disabilities. Furthermore, this work not only promotes health literacy within communities whose members experience health inequities but also supports participant self-determination.
Introduction
Health literacy studies that involve members of vulnerable communities require special considerations at every stage of the research process in order to safeguard the integrity of the findings and the dignity of the participants themselves. From identifying a health disparity, to defining vulnerable communities, to designing an adequate research study (Kilbourne, Switzer, Hyman, Crowley-Matoka, & Fine, 2006), these special measures not only minimize the effects of sampling bias but also help to protect participants by minimizing the risk they potentially encounter as a result of participating in a research study (Zion, Gillam, & Loff, 2000). Collecting data from vulnerable participants, for instance, brings up a number of problematic situations and serious implications. These include negotiating the inherent power dynamics of the research context and balancing the needs of data collection with the need to preserve the participants’ sense of agency throughout the course of their participation. It is not enough to resolve these issues in a manner that minimizes risk for vulnerable participants. Methods based on critical and community-based approaches suggest that these challenges be resolved in ways that also empower and emancipate participants (Denzin & Lincoln, 2008). This article outlines a pilot study developed to help design a methodologically sound research protocol that prioritizes participants over the data they provide, elevating their role in the research process from data source to research collaborator. Some initial findings of the pilot study were previously presented as a conference poster (Morales, 2016). This article presents a more in-depth discussion of the methods used and their justifications.
The pilot study consists of case studies based on user-centered studies, community-based participatory research, and research with vulnerable populations to determine in what ways health literacy research makes participants vulnerable and how the research design process can best address these vulnerabilities. The main health literacy study, which the pilot study seeks to inform, aims to investigate how adult beginning through intermediate readers evaluate consumer health information.
Prior to engaging in a full research study, Van Teijlingen and Hundley (2002) suggest that researchers conduct pilot studies in order to develop adequate research instruments, design a research protocol, and identify potential logistical problems. Conducting a pilot study is, therefore, essential to the design of a research protocol that is not only mindful of bias but that also mitigates the risks potentially experienced by participants, protects their rights, and empowers them by preserving their agency over how to even engage with the research process. The ultimate goal of the pilot study presented in this article was to inform the design of an equitable research relationship between the researcher and the participant—one that actively promotes the agency of the research participant.
Background
Using health information in order to make informed health decisions—a common way of defining health literacy—requires other forms of literacy including reading text and interpreting quantitative information such as food labels and dosage instructions (Berkman, Davis, & McCormack, 2010; Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011; National Center for Education Statistics, 2016; Zarcadoolas, 2011). Evaluating information has become a major aspect of health literacy in recent years. For example, the National Institutes of Health (see https://ods.od.nih.gov/Health_Information/How_To_Evaluate_Health_Information_on_the_Internet_Questions_and_Answers.aspx), the National Library of Medicine (see https://www.nlm.nih.gov/medlineplus/webeval/webeval.html), and the National Network of Libraries of Medicine (see http://nnlm.gov/pnr/hip/criteria.html) each disseminate guidelines to help individuals judge the quality of health information found online. Despite these resources, research shows that consumer health information is very difficult to read (Berkman et al., 2011). How people evaluate the usefulness of health information might be related to how difficult it is to read.
Several recent initiatives have been focusing on the reading difficulty of health information and its impact on health literacy (Zarcadoolas, 2011). This is a considerable challenge since the population of the United States varies so widely in terms of literacy and, therefore not surprisingly, health literacy as well. Not only do literacy levels by race and age reveal a great disparity, but adults without health insurance, as well as those who receive Medicare or Medicaid also had lower average health literacy than adults with other types of insurance according to the most recent survey of adult literacy and health literacy (National Center for Education Statistics, 2003). The pilot study described in this article is part of a larger study that investigates whether and to what extent health information supports adults whose reading and health literacy levels may be below the national average.
Conducting health literacy research with adult beginning readers requires careful planning and execution. For adults who find reading challenging, participating in a research study that requires them to read documents about health topics might feel burdensome or overwhelming. Additional factors that might contribute to these adults’ degree of vulnerability when participating in the study include their daily schedules, their income level, as well as preexisting and undertreated health conditions such as poor vision. Recognizing the potential undue burden of participation for adult beginning readers in this type of research, a pilot study was conducted to accomplish two main objectives. The first was to test the research instruments. The second was to create a constructive space in which participants feel confident in their participation and in their evaluation of health information despite any personal challenges or limitations. It is important to note that this protocol was not designed to evaluate the health literacy skills of the participants but rather to develop a usability testing method of health information from a user-centered perspective. It is crucial that participants not be treated as objects under study. Instead, they need to be treated as knowledgeable judges of information. This is particularly important because it is their judgments about health information that are the primary variable in this study not their ability to read or comprehend health information.
Literature Review
Several areas of research and conceptual frameworks inform this pilot study design. Evaluating the usability of health information is well served by research on readability as well as by methods commonly used in participatory action research and community informatics. Guiding this study design is a critical methodology that focuses on community-based research as an alternative to traditional approaches, which prioritizes the perspective of the researcher. The following section discusses important concepts and key findings from the research traditions that inform the pilot study at hand.
Evaluating Health Information
Evaluating information has become a major aspect of health literacy in recent years (Sørensen et al., 2012). Research grounded in cognitive approaches to studying how individuals evaluate information, including health information, has found great variability in how people judge credibility, assess information quality, and indeed among the criteria they use to evaluate information (Arazy & Kopak, 2011; Hilligoss & Rieh, 2008; Hjørland, 2011; Yong, Stvilia, & Mon, 2012). In fact, a recent health literacy study found that individuals tend to rely more heavily on their own set of evaluation criteria rather than on criteria endorsed by quality guidelines (Diviani, van den Putte, Meppelink, & van Weert, 2016). Some of the nonendorsed criteria adults in that study used include style, tone, specificity, and presence of medical jargon. These findings are consistent with usability studies of health information (Eysenbach & Kohler, 2002; Marshall & Williams, 2006). Evidently, people pay attention to certain cues present in the text and use these cues in their information evaluation process. Because reading is no doubt implicated in the evaluation of health information, it is important to consider linguistic features when investigating how members of these communities evaluate health information.
That judgments vary with respect to credibility and quality of information has been well established according to the literature. What is less understood is what factors account for this variation. Studies based on the cognitive approach locate knowledge, experience, skills, and actions in the mind of the individual. It is, therefore, difficult for them to explain how social structures and community membership shape values, which may in turn affect the way members of different communities evaluate health information. Recently, health literacy research has moved beyond behavioral models to investigate whether social factors such as civic engagement, language use, employment, and marital status affect health literacy (Rikard, Thompson, McKinney, & Beauchamp, 2016; Sørensen et al., 2012; Zarcadoolas, Pleasant, & Greer, 2005). Accordingly, the main study of which this pilot study forms part investigates how members of a social and linguistic community evaluate health information.
Reading Research
The cognitive approach has also dominated research on reading and readability. This body of work conceptualizes reading as a set of skills or abilities involved in decoding meaning, defined as comprehension (Duke & Carlisle, 2010). Vocabulary and sentence length have long been used to measure the reading skill and health literacy level of individuals as well as the reading difficulty of health information (Agency for Healthcare Research and Quality, 2016; Gemoets, Rosemblat, Tse, & Logan, 2004; Zarcadoolas, 2011). Making meaning, however, is also a social process in which socially constructed values, beliefs, and practices affect the way individuals interpret and make sense of the world (Gee, 2015). This social practice approach to reading, as opposed to a cognitive or skills-based view, moves the focus away from asking how
Critical Methodologies and Participatory Action Research
A critical approach to community-based research requires the traditional roles of participant and researcher be mutually shared by the researcher, community organizations, and community members during different parts of the research process (Israel, Schulz, Parker, & Becker, 1998). In this paradigm, the researcher is not the only expert and, in fact, is often not an expert at all in issues that affect the community and its members (Smith, 1999). An important benefit of this more equitable distribution of roles is that it preserves a sense of agency over the terms of their own participation for otherwise disenfranchised members of the community. A critical approach to research requires the implementation of research methods that explicitly acknowledge and emphasize participants’ roles as experts of their lived experience while at the same time understating their roles as participants who are under observation.
As the term implies, participatory action research allows participants a more active role in the research process by inviting them to self-reflect and interpret their own roles in certain social practices (Kemmis & McTaggart, 2005). In the context of this health literacy study, such an approach focuses a critical and analytical lens on community participants’ own judgments about health information resources available to them. By supporting self-determination and self-reflection, participatory action research is aligned with the emancipatory aims of the critical methodologies framework described above.
Usability and Participatory Design in Community Informatics
The basic premise of participatory design is a user-centered approach to the design of information resources and sociotechnical systems, recognizing the impact these have on the everyday life of community members (Carroll & Rosson, 2007). To design and build systems that are usable and useful for members of a specific community, user evaluations are extremely important (Maguire, 2001; Redish, 2007). Usability studies are further expanding reading research by introducing user perspectives that are evidently related to reading as a social practice such as use, perceptions, and expectations (Leroy, Kauchak, & Mouradi, 2013; Sluis, Broek, Glassey, Dijk, & Jong, 2014). Usability methods are also a viable way of empowering vulnerable participants by recognizing adult beginning readers’ capacity to assess the usability of health information. Instead of focusing on the skills or abilities of the participants, the usability study discussed here investigates the usefulness of health information documents from the participant’s perspective.
Despite guidelines that stress the importance of easy-to-read health information (U.S. National Library of Medicine, 2016), adult beginning readers are underrepresented in the content creation process of health information resources resulting in a lack of user evaluation data from these users’ perspectives. To address the lack of usability data for these information resources, a qualitative research study was designed to investigate how adult beginning readers evaluate health information resources by using a case study method based on usability testing and participatory design principles.
Pilot Study Methods
The pilot study for which this health information evaluation case study was developed involved three participants, all of whom were clients of Diaspora Community Services, a nonprofit community outreach organization that operates a health clinic in the East Flatbush neighborhood of Brooklyn, New York. Once the clinic recruited participants and scheduled the research sessions, the study was administered in the same way for all three participants. First, the researcher introduced herself and explained her research interest in consumer health information behavior. The researcher then introduced the case study, explained the informed consent form, handed out the gift card, and answered any questions. After this introduction, the researcher gave the participant the evaluation task instruction sheet, the colored pens, one brochure, and one fact sheet. The researcher then verbally explained the task instructions, answered any questions, and let the participant begin. Once the participant finished evaluating both documents, the researcher asked the participant to verbally walk her through the documents and explain the different red and green marks on the page. Once this interview was complete, the researcher then asked the participant to fill out the short survey questionnaire. Because the survey questionnaire included questions about the participant’s level of education and language fluency, it was administered last in order to avoid participant bias during the evaluation task.
The following section describes the research design of the pilot study including the study participants and the design and selection of research instruments. This section also discusses key participant characteristics and outlines how these informed the research design. As a community-based research study, it is important to also consider the role of community organizations in this work; to this end, this section also includes a discussion of the collaboration between the researcher and the health community organization that facilitated this work.
Research Design
The method of data collection for this pilot study consisted of an exploratory case study (Baxter & Jack, 2008) including (1) a short demographic survey questionnaire, (2) an information evaluation task, and (3) a follow-up interview. All three activities were conducted in a single session in order to accommodate participants’ time constraints as well as to ensure that memory would not negatively affect the participants’ recollections about why they marked the documents the way they did.
Because of the unique challenges of conducting research that requires adult early readers to read health information, the study design attempted to minimize the participants’ perception that they are being tested on their reading abilities or medical knowledge. This type of study must also minimize the effects of the inherent power dynamic in the researcher/participant relationship and between the research site and the participant. The rest of this section describes the challenges and affordances of developing a study with a third-party collaborator and participants who are adult early readers.
Collaborating With a Community Organization
With sufficient planning, clarity, transparency, and flexibility on the part of the primary research team, the challenges of working with a community-based research partner, a community health clinic in this case, can often be minimized and resolved. In order to allow the clinic staff to recruit and schedule participants for immediate inclusion in the study, the researcher dedicated certain days to be “on call” near the location of the clinic. On those days, the researcher was available to conduct a case study the moment the clinic called with a confirmed participant appointment. For this pilot study, three participants were successfully recruited over the course of 2 weeks.
To ensure a successful collaboration with a third party, the terms of the collaboration must be clear to all from the beginning. For this study, the community health clinic agreed to (1) provide a letter of collaboration for the institutional review board application, (2) recruit participants for the study, (3) provide transportation for the participants to the clinic in order to participate in the study, and (4) provide a conference room in which to conduct the study itself. The primary research team agreed to (1) develop a study using consumer health materials that are relevant to the clinic’s patient population, (2) pay participants US$25 in the form of a gift card for their time, (3) be flexible in terms of availability and work around the participants’ own schedule, and (4) deliver a report based on the findings to the clinic director. To secure this level of collaboration with the community health clinic, the researcher first wrote to a member of the parent organization’s board of directors with information about the proposed study and requesting a meeting to discuss a possible collaboration. With the board member’s help, the researcher was then able to confirm an in-person meeting with the organization’s interim director to discuss the project, determine how the case studies would be carried out (whether one on one or in groups), and brainstorm topic ideas for the selection of research tools. It was during this meeting, for instance, that the director suggested the researcher use information about diabetes with participants, recognizing this health topic as an important one for the clinic’s patient population. After this meeting, the director then engaged the clinical director, who subsequently organized a meeting with the clinic staff and the researcher to discuss participant recruitment, participant compensation, and the distribution of research results.
There are several advantages to collaborating with a community organization. First, since the collaborating clinic already had an established relationship with the community of interest to the researcher, the clinic led recruitment efforts. Because of their long-standing relationship with and intimate personal knowledge of community members, the clinic was able to identify participants who met the study criteria, specifically, adults who are beginning through intermediate readers. Additionally, because the community health clinic served as an intermediary, the researcher could ensure the anonymity of participants in this pilot study. Since the clinic handled all recruitment efforts, the researcher did not need to record any of the participants’ names or telephone numbers. Second, collaborating with a clinic with close ties to the community helped the researcher select the health information materials for the evaluation task. The clinic director suggested the topic and source of materials after considering the health issues most relevant to the clinic patients and the types of information sources that tend to be available to them. Third, the community health clinic was able and willing to provide a meeting room where the researcher could conduct the study. The ability to meet the study participants on familiar ground helped to balance the researcher/participant power dynamic. That said, however, the power dynamics inherent in conducting research in a clinic, which also already possesses its power dynamics vis-à-vis the community it serves, can never be fully neutralized. Since the preexisting relationship between the clinic staff and the participants might unduly influence their responses during the study, it is important for researchers working on site to clearly identify themselves as not being members of the clinic staff and to disclose to participants the relationship between the research team, their home institution, and the clinic. This should be done to avoid conflating the roles of the researcher and the clinic staff for study participants who might see clinic staff as having some authority as providers of important health services for them and their families.
Participant Characteristics
In order to recruit adult beginning through intermediate readers to evaluate consumer health information, the study used purposive sampling by focusing on the patient population served by Diaspora Community Services in Brooklyn, New York. This clinic serves a community with a high incidence of poverty, unemployment, and low levels of education, including individuals receiving Medicaid, and who are primarily immigrants or individuals of African American, West Indian, or Latino descent (Diaspora Community Services, 2016). These community characteristics are important in this study considering the relationship between health literacy and insurance coverage (Sentell, 2012) as well as the link between poverty, low literacy levels, and negative health outcomes (Berkman et al., 2011; Bostock & Steptoe, 2012).
Because of the level of poverty in the community, an ethical calculation of the appropriate, noncoercive amount of participant compensation is critical. The level of poverty also means many adults have no reliable method of transportation, and compensation might need to include reimbursement for public transit. Poverty in this community also means many adults have chronic health conditions (City of New York, 2015). Some of the symptoms of these health conditions or treatment side effects, such as vision problems or clouding of consciousness, may contribute to a participant’s difficulty reading (Lipton, 2016). The demands these participants face in their daily lives mean they have limited time, energy, and flexibility, in addition to managing chronic health problems, all of which might potentially lower their level of commitment to the research study. Due to these challenges, researchers should allow ample time for participant recruitment and data collection, as these will likely take longer than working with a convenience sample of participants who are easier to recruit, such as university students.
Perhaps the most obvious challenge for a study focusing on reading health information is that participants have low levels of educational attainment. Since many adults in this community do not have a high school diploma (Diaspora Community Services, 2016), all research materials, including the informed consent form, study descriptions, instructions, interview questions, survey questions, and other data collection materials, must be written in clear, concise language that is easy to understand. Cognitive load has been found to affect an individual’s ability to recall information, and cognitive ability has been suggested to mitigate the effects of low literacy levels in adults engaged in a health literacy study (Wilson, Wolf, Curtis, Clayman, Cameron, Eigen, & Makoul, 2010). To avoid overwhelming or embarrassing participants, which may result in incomplete or otherwise unusable data, it is important to keep instructions and survey questionnaires short and to read the materials aloud as well as offer to write down the answers for the participants should they prefer it.
Instruments
This pilot study made use of four research instruments: one diabetes brochure, one diabetes fact sheet, a short survey questionnaire, and a semistructured interview. Because of the participant characteristics discussed above, the selection and design of these instruments required careful and deliberate planning. The following section discusses the selection, development, and use of these instruments during this pilot study in more detail.
Selecting Health Materials
When selecting health materials for participants to evaluate in this pilot study, participant characteristics help to limit and focus the sample of documents to be evaluated (Miles, Huberman, & Saldana, 2013). Adults with low levels of health literacy rely more on print-based media, rather than digital resources, as a source of health information (U.S. Department of Health and Human Services, 2008). The case studies that were part of this research focused exclusively on print media including one 6-page brochure published and distributed by the city health department and one 1-page fact sheet related to diabetes management and prevention created and distributed by a clinic. These materials were similar to those found at the clinic waiting room and were approved by the clinic director as materials that are similar to those distributed throughout the community it serves.
Research has found that prior knowledge and knowledge of a specific topic or domain affects reading comprehension (Ozuru, Dempsey, & McNamara, 2009; Tarchi, 2010). To minimize the effects of low topic knowledge in this study, and therefore help ensure participants would focus on their evaluation of the materials rather than their level of comprehension of the materials, the researcher selected diabetes as the topic of the documents in consultation with the clinic director. According to the clinic director, potential participants drawn from the clinic’s client roster were likely to be familiar with the topic of diabetes because of its prevalence among adults in the United States (American Diabetes Association, 2016) and among the clinic patient population specifically.
Designing the Evaluation Task
The evaluation task at the heart of this pilot case study was modeled after usability testing methods. Usability testing is a method of user interface and product design in which a designer observes an individual interacting with a prototype or finished product in order to identify obstacles she encounters when using the system or product in order to accomplish a specified task (Lewis, 2006). Because the goal of usability testing is not to assess participants but rather to evaluate a product in order to improve its design and functionality (Lewis, 2006), this method was a useful model for the health information evaluation task in this pilot study.
As with the selection of appropriate research materials, the researcher must consider participant characteristics carefully when designing the evaluation task. The participants of interest to this study might have limited access to technology and, additionally, might have limited—if any—experience with usability testing as described above. This might cause them anxiety (Dumas & Fox, 2009) or to misperceive the evaluation task as an evaluation of their own skills and aptitude. To avoid this, the researcher introduced the task as a health media product in need of user evaluation. Instead of using a numerical scale or a thumbs up/down rating system, however, the participants were instructed to mark up the documents with colored pens to indicate “good,” “helpful,” or “useful” document features (green pens) and “bad,” “confusing,” or “not useful” document features (red pens). In this way, the researcher hoped to minimize the participants’ perception that they were the focus of the evaluation and, instead, attempted to confer authority on them as judges of information quality. The potential here for flipping the researcher/participant power dynamic is significant, since the participant has some form of control over the research materials and the power and means to mark them up as they wish, in much the same way as authority figures like editors and teachers often do.
Semistructured interviews
The interview protocol was designed based on usability testing methods that ask participants to identify aspects of the health information materials that are useful, easy to use, and satisfying (Marcus, 2011). Once the participant had finished reading and marking up both documents to her satisfaction, the researcher then asked the participant to walk her through the documents, pointing out features of each document the participant found useful or not. The researcher took detailed notes during the interviews and made additional markings in black ink on the health documents themselves. During the interviews, the researcher paid special attention to portions of the documents the participant had marked with either red or green pens, noticing interesting markings such as participants adding their own written content to the documents. Questions based on the markings included “What do you mean by these minus (−) signs in red ink next to this section of the brochure?” “Why did you circle this in green/red?” and “Why did you draw a line/arrow from this section to this section of the brochure?” The researcher was also interested in whether certain features such as vocabulary, statistics, and the use of charts were problematic or helpful and therefore asked specific questions about these, such as the individual’s perception of the different statistics cited in the documents, her level of familiarity with “gestational diabetes,” “prediabetes,” and “A1C,” and her perception of the charts and graphics used.
Survey Questionnaire
The primary goal of including a short survey questionnaire in this study was to determine the participants’ levels of educational attainment and writing skills to confirm they fit the pilot study criteria. The researcher decided to keep the survey questionnaire short in order to avoid overtaxing the participants. To this end, the survey questionnaire consisted of seven demographic questions, none of which asked for any identifying information. The questions were short and direct and were printed on a single sheet of paper in 14-point Arial font to make them easier to read. The instructions included on the survey questionnaire consisted of only two sentences, one of which stated that participants had the right to decline to answer any or all of the questions for any reason and without risk of penalty. The researcher read these instructions aloud to each participant and offered to read the survey questions and record the answers for them if they so desired.
Pilot Study Findings
This case study method used in this pilot study has several benefits. The rich data collected include the identification of specific linguistic features of health documents that affect the way an adult early reader evaluates consumer health information such as design features, participants’ prior knowledge of the topic, the potential use of the documents, terminology, and the interpretation of statistical figures and bulleted lists. Additionally, this method provides a manner of data verification by including a semistructured interview that immediately follows the participant’s completion of the evaluation task. This is a form of member checking, a common validation technique in qualitative work (Lincoln & Guba, 1985).
A beneficial side effect of this method is that it recognizes the authority of the participant as ultimate user and judge of health information. Two of the participants in this pilot study demonstrated some domain expertise by identifying pieces of information they felt were missing from the documents or that seemed confusing based on information they had either seen on television, learned from friends and family who are diagnosed with diabetes (secondhand knowledge), or otherwise previously known. For instance, when reading the section of the 6-page brochure discussing symptoms associated with having diabetes before a diagnosis has been made, Sharon 1 added the phrase, “weight gain,” as an additional bullet item and mentioned during the interview that she knows from past experience that diabetes also causes weight gain, so she “knows it’s also a symptom.” Another participant, Diana, also added a bullet item, this time to the 1-page fact sheet’s section on managing diabetes: She added the word “pain” and explained that she has seen television commercials in which managing foot pain is also part of diabetes care. Particularly because the participants in this study are often members of marginalized groups, this method of data collection is a potentially empowering method, allowing them to take on the role of expert, editor, consultant, or judge of information quality.
This pilot case study highlighted specific challenges the main study design needs to resolve. While acknowledging the benefits of collaborating with a third party who has access to qualified participants, a more neutral site might more effectively neutralize the power dynamics between the clinic, the participants, and the researcher. Another important challenge is the perceived difficulty of the task for some participants due to their level of interest, motivation, or reading experience. Although this issue cannot be completely avoided, the researcher must be particularly sensitive to the participant’s level of effort during the task. The task’s perceived level of difficulty can diminish participant motivation and, therefore, hinder task completion. A participant who perceives the task particularly difficult or who perceives a document especially difficult to read, for instance, might skip over entire sections of the document. To identify when this might be the case and to minimize participant anxiety or the risk of task abandonment, the researcher should be present during the evaluation task, closely observe the participant without intruding on their task performance, and offer to help read the document aloud if necessary (Dumas & Fox, 2009). A related issue to motivation and task completion is a participant’s preexisting health conditions, such as problems with vision, which might affect how they read. Not everyone can afford eye care, and problems with vision are not always apparent. One participant in this pilot case study, for instance, self-disclosed having problems with her eyesight, leading her to mark in red ink entire sections of the documents that were printed in a font size too small for her to see.
A key implication of this pilot study is the need for a short, but formal, participant assessment of their experience with the research method itself. This assessment should be administered at the conclusion of the study itself and would serve to gather insight into the participants’ own experience with the study. This should be in the form of a short open-ended interview question that asks the participant to describe her experience in her own words, stating which aspects of the study she most liked and disliked, and why. Though this could also be administered as a short survey questionnaire, it might be difficult to avoid asking leading questions or priming participants with the researcher’s own language.
The insights gathered from this pilot study will be used to develop future work that includes a larger and more varied set of documents, as well as a larger number of study participants. A related semiexperimental study will include bilingual participants, some of whom will be English-language learners, and a parallel corpus of bilingual consumer health information documents. Finally, the researcher acknowledges the value of sharing the research materials used in this pilot case study as well as the available resources on methods for assessing document readability and information evaluation with the larger research and professional communities. To this end, the instruments used in this pilot study as well as the protocol for the case study methods proposed herein will be made available for research purposes on the website of the (site name and URL to be added after peer review).
Conclusion
This article presents a case study method developed for a pilot study about the consumer health information evaluation practices of adults who are beginning through intermediate readers. This method extends the user-centered paradigm by designing a study that, modeled on usability testing methods, asks users to assess a set of consumer health information documents. Because the method focuses on document features, it can also be used to examine the evaluation practices of other adults who might experience similar challenges when reading these same documents, such as English-language learners or adults with certain intellectual disabilities. Developing sound methods founded on a clear methodology for user-based studies involving adult developing readers has several important implications. These include a deeper understanding of the unique experiences, challenges, and ways of knowing of marginalized and vulnerable groups, as well as supporting empowerment tactics for members of these same communities, which can have transformative effects. By engaging members of marginalized communities as judges of health information quality, this kind of research supports community members’ self-determination while promoting health literacy.