Revisiting Validity in Research With Multilingual Learners: Moving From a Monolingual to a Multilingual Lens on Research Quality

Abstract

A critical revisiting of research methods with multilingual learners raises issues of validity that are often unaddressed in current discussions of research quality. Though validity is conceptualized and assessed differently across research traditions, validity broadly describes the production of findings that are credible and trustworthy. The historically monolingual lens of research methodologies has led to conceptions of validity that are inadequate for ensuring that reported findings “fit the data” in research with multilingual learners. In this article, I describe a retrospective inquiry to critically reassess how current paradigms of validity might be reconceptualized in research with multilingual learners. Through a reexamination of research practices from two of my own studies, this article considers three ways that current monolingual paradigms of validity should be expanded in research with multilingual learners to work toward multilingual conceptualizations of research quality. This article then offers guidance to aid other researchers in taking up issues of validity in the design and reporting of research with multilingual learners.

Conducting research with multilingual learners raises issues of validity that are often unaddressed by current discussions of research quality. Questions of what constitutes data and how researchers treat data in the research process are central to the endeavor of understanding the language, literacy, and learning processes of multilingual learners (Chapelle, 2011; Wolfson, 1986). Though validity is conceptualized and assessed differently across research traditions, validity broadly describes the production of findings that are credible and trustworthy (Miles et al., 2019). Though useful, this simple definition of validity conceals a more complex history of what has counted as valid research and for whom (Cian, 2021; Johnson & Saville-Troike, 1992).

In this article, I describe a retrospective inquiry into research practices to examine how current paradigms of validity might be reconceptualized in research with multilingual learners. The aim of this inquiry is to engage in critical reassessments of research practices to move toward more equitable and responsive understandings of validity. By reexamining research decisions from my own studies, I present evidence for the need to expand current understandings of validity to consider issues that are overlooked in monolingual conceptualizations of research quality. I then offer guidance grounded in my own research experiences to aid other researchers in taking up issues of validity in the design and reporting of research with multilingual learners.

Problematizing Validity in Research with Multilingual Learners

Validity has its roots in positivist and primarily quantitative ideas of quality that assume that research can be objective and reveal underlying truth, often through the accurate measurement of phenomena. A driving force in attention to validity has been the need for the findings and interpretations of research to be considered credible to justify the authority of scholarly knowledge (Davis, 1992). The historical roots of validity extended from the work of quantitative researchers and served to reinforce claims of the veracity of quantitative measures and assessments (Cian, 2021; Cronbach & Meehl, 1955; Davis, 1992). This narrow attention to establishing the credibility of measurement scales and related assessments is indicative of the myopic focus of early conceptualizations of validity.

The concept of validity has evolved over time, particularly in response to poststructural and critical critiques of its positivist and quantitative origins (Cian, 2021). These changes have broadened how validity is defined and assessed, brought attention to the influence of the researcher on the act and outcomes of research, and to the explanatory coherence of reported findings in relation to contextual factors (Johnson & Saville-Troike, 1992). The increased historical influence of poststructuralism raised particular challenges to quantitative and positivist notions of validity by undermining and subverting the epistemological foundations of these traditions, and directly interrogating “the power relations between knower and known, including those in emancipatory knowledge production” (Lenzo, 1995, p. 20).

Researchers applying Critical Race Theory and other critical lenses have extended and complicated ideas about power, truth, neutrality, and voice in both quantitative and qualitative conceptions of validity. At the broadest level, these conversations are concerned with the ways that numbers are not neutral, categories and units of analysis are not naturally occurring, data does not speak for itself, all research practices must be interrogated for ways they serve prejudiced and assimilationist ends, and intended and unintended uses of research outcomes must be considered (Garcia et al., 2018; Gillborn et al., 2018). This extends to questioning whether positivist and quantitative research approaches and their attendant conceptualizations of validity are “incongruent” with critical, emancipatory, and justice-oriented aims (Sablan, 2019, p. 185).

Sustained critiques of validity simultaneously draw attention to the complexity and high stakes involved in how research quality is conceptualized. Miles et al. (2019) aptly capture the trap validity poses in describing the risk that the conclusions drawn from data “can be evocative, illuminating—and wrong,” making it possible that, “the story, well told as it is, does not fit the data” (p. 289). The acknowledgment that the researcher plays a central role in constructing and interpreting data means that the researcher must make efforts to critically question the research process. This includes whether (and to what extent) their decisions may alter, misconstrue, or otherwise transform what was seen, observed, or collected in ways that may misrepresent the people or phenomena that are the focus of the research.

Approaches to establishing validity in the research process have evolved from a focus on narrow constructs aligned with scales and assessments to a broader range of practices that reflect more diverse epistemologies and understandings of what it means to “know” (Cian, 2021; Golafshani, 2003; Winter, 2000). Reflecting typical early approaches to validity, Cronbach and Meehl (1955) described four types of validity, including predictive validity, concurrent validity, content validity, and construct validity, with each tied to specific aspects of quantitative measurement tools. Though attention to these and other largely positivist constructs persist in research today, a broader range of practices and concepts are used and accepted by researchers to establish validity, ranging from the triangulation of data across sources and types to member checks to the provision of sufficient “thick description” to allow the reader to make judgments of research quality and transferability (Davis, 1992; Cian, 2021; Lather, 1986).

Yet increases in the complexity and nuance of how validity is understood have not consistently yielded validity practices that satisfactorily meet the needs of researchers or address the underlying issues arising from its positivist origins. Roulston and Bhattacharya (2018) describe the “difficult, contradictory, and fluid discourses and praxis in qualitative research” that include validity practices (p. 252). Shortcomings in how validity is constructed and applied persist not only from the continued influence of positivism but also from persistent blind spots and neglected issues in research practices more broadly. Among these are the broader monolingual and English-centric biases in educational research, including a legacy of theories, research practices, and pedagogical practices grounded in monolingual worldviews and educational institutions, particularly in the United States.

Multilingual children are learning two or more languages during childhood and are developing practices and competencies to navigate multiple language communities. The experience of being multilingual often also includes the construction and navigation of intersectional identities across dimensions of language, immigration status, race, culture, nationality, and other categories (Compton-Lilly et al., 2017). Applying monolingual approaches to understanding multilingual children can make invisible the lived experiences of these children, including the interrelated roles of language, race, and culture and the impacts of differing cultural models of language use across school, home, and community contexts (Flores & Rosa, 2015; Gee, 2012; Wagner, 2024). This is a critical absence because even subtle differences in research practices have widely different implications for what is seen, and for how researchers think about how multilingualism matters to literacy and learning.

Though translanguaging, transnationalism, and other constructs that complicate and expand views of multilingual learners have become more common in literacy research (e.g., Pontier & Riera, 2022; Valencia Mazzanti, 2022), these shifts often address what is studied in research and not how research is conducted by researchers. It is not sufficient for researchers to consider only the topics of research on multilingual learners. It is likewise necessary for research practices to align with and be responsive to the ways that languaging and literacy are social, cultural, and linguistic practices, and the implications of these realities for conducting research with multilingual learners (Bakhtin, 1981; García & Kleifgen, 2020; Gee, 2012).

In educational research, this is reinforced by monolingual biases in school settings where teachers and school systems, particularly in the United States, have historically adopted monolingual paradigms in the approaches used to teach children, even when children are multilingual (Flores & García, 2017; Valdés, 2020). Current conversations on literacy, including those around the science of reading, often perpetuate these monolingual and English-centric views of what counts as literacy and how reading and writing are done (Goldenberg, 2020; Share, 2021). This not only ignores a wide range of research on literacy but also problematically leads to instruction that neglects the many ways that literacy processes differ for non-English speakers and multilingual learners.

The combined impacts of historically monolingual lenses in research and school practices raise significant questions about the general credibility and trustworthiness of research findings on multilingual children, particularly when conducted in school settings. Opening research processes to debate and reconsideration is key to assessing to whom and how research is held accountable, and to moving toward more just and valid research on multilingual learners (Bacon, 2017; Flores & Rosa, 2015; Wagner, 2024). Conceptions of validity that are inadequate for ensuring that research “fits the data” collected from multilingual learners raise the risk of misleading findings or erroneous reports (Miles et al., 2019), and missed understandings of critical factors in the lives of multilingual learners. To work toward research on multilingual learners that is valid, researchers must negotiate these complex intersections of language, power, politics, and ethics (Lenzo, 1995).

Inquiry as a Tool for Critically Reassessing Research Practices

This article draws on my own experiences as a researcher of multilingual learners. Through critical inquiry (Cochran-Smith & Lytle, 2009; Lather, 1986), I look back on key moments in the research processes of two studies I conducted with multilingual learners to identify and explore issues related to the validity of research processes with multilingual learners. Lather (1986) broadly describes critical inquiry as “empirical research that advances emancipatory knowledge” (pp. 261–262). Critical inquiry is responsive to the experiences and needs of oppressed or marginalized peoples and is rooted in aims to develop understandings of the world that include and speak to the people who are the subject of research. Critical inquiry emphasizes reflection that questions and interrogates power relations and takes up emancipatory aims, including the explicit linking of deliberation to action in the social world (Lather, 1986).

The process of inquiry into practice enables researchers to take up a critical and emancipatory stance to interrogate the practices, assumptions, and beliefs that comprise the research process (Cochran-Smith & Lytle, 2009; Garrison, 2017; Wagner, 2023b). The inquiry process is grounded in Dewey's (1910, 1938) conception of practical inquiry and is guided by the emancipatory aims of critical theories (Lather, 1986). This enables systematic, intentional, and self-critical assessments of researchers’ actions and choices in the context of practice-based work. In this instance, the inquiry process is taken up to support “making problematic…the ways knowledge is constructed, evaluated, and used” Cochran-Smith & Lytle, 2009, p. 18) with specific attention to validity.

Other researchers have made similar efforts to retrospectively examine their own research practices, including in relation to issues affecting multilingual learners. Crumpler et al. (2011) turn a reflexive lens on researcher positionality, representation, and ideas of difference to critically reconsider how their own complicity and shortcomings in the research process enabled deficit ideas about multilingual learners to persist. Rogers and colleagues (Rogers & Labadie, 2018; Rogers et al., 2016) critically reassess ethics around the balancing of privacy, voice, and assent in research with children. In doing so, Rogers and colleagues reconsider moments in research that are often overlooked as administrative tasks as processes to be revisited and negotiated with children. In this study, I extend these efforts and take up the call of Crumpler et al. (2011) for “seeking out new research tools to help us perform research differently—tools for troubling regulative research practices” (p. 82).

Critical Inquiry Process

The critical inquiry that is the focus of this article examines ways that specific research processes hindered or supported the production of valid research findings in two of my own prior studies of multilingual learners. The data used in the critical inquiry process include the formal data sources from the two focal studies, which I describe later in this section, along with field notes and memos from the research process, reviewer feedback on manuscripts prepared from these studies, and my own memoing and responses during the publication process. In the critical inquiry process, I gave particular attention to researcher field notes and memos as these provided insights into issues and critical decisions in the design, implementation, and reporting of the research studies, including decisions that directly impacted the validity of the studies.

The inquiry process draws on Dewey's (1910, 1938) conception of practical inquiry and a more specific process grounded in this approach described by Garrison (2017) that occurs in four general phases. In the first phase, I undertook a retrospective review of data sources to identify “critical moments” when research processes either fell short in producing valid data and representations of multilingual children and families or enabled unexpected progress in breaking through issues of validity (Pennycook, 2004; Thomson et al., 2002). This process focused on documentations of my own thinking on the research process, recorded in field notes and memos, along with the other data sources from these studies. This process was open and broad, aimed to identify and bring into focus research decisions or aspects of research design that actively attended to or neglected aspects of multilingualism and validity.

In the next phase, I attempted to understand the issues and problems identified in the first phase by searching for information and possible explanations. This is a generative phase in the inquiry process intended to support open-ended consideration of assumptions that underlaid key research decisions and their possible impacts on validity. This relied on memoing as a retrospective process that encourages withdrawing or distancing oneself from the research under consideration to open space for critical reevaluations (Cox, 2013). This phase included making connections to and drawing on existing research and conversations in the field to support and extend the interrogation of these ideas.

The third phase focused on the synthesis of ideas generated in the prior phases. This process drew on the analytic tools of thematic analysis (Braun & Clarke, 2006). First, a coding process was used to identify recurring issues and topics in the prior stages of the critical inquiry process. Recursive readings and juxtaposition of the codes and underlying data extracts facilitated the identification of prospective themes, which were triangulated across events, studies, and data sources and revised through repeated cycles of analysis. In the last phase, I identified possible alternatives to research practices that were problematized in this critical reexamination, and that aim to move beyond normative ways of thinking and acting in the conducting of research. These are presented together with the themes in the sections that follow.

Focal Studies for Critical Inquiry

In the first of the two focal studies I revisit in this critical inquiry, I explored how multilingual learners in prekindergarten understood and constructed literate identities (Wagner, 2018, 2019, 2022). Literate identities describe the ways that children construct the self as a reader and writer across contexts and time, and for multilingual children, in ways that draw on their lived experiences as multilinguals (Wagner, 2024). I conducted the study at an elementary school in a northeastern city in the United States. The participants were ten multilingual learners from two prekindergarten classrooms, including a monolingual English classroom and a sheltered English immersion classroom where English and Spanish were used for instruction. The children spoke English and Spanish, Cape Verdean Creole, Haitian Creole, Portuguese, or Vietnamese. I visited the children's classrooms 3–4 days per week for a 5-month period and interacted with them in both English and Spanish.

In the second study, I examined the development of children's literate identities (Wagner, 2021) and the relationships between language contexts and literate identities (Wagner, 2023a). I conducted the study in a bilingual Chinese-English family literacy program at an elementary school in a northeastern city in the United States. The program was attended by ten children and their caregivers who spoke one or more Chinese languages, including Mandarin, Cantonese, Fuzhouese, Hainan, and/or Fujian, as their primary home language. The children participated in the program for 2 years across prekindergarten and kindergarten. The program provided English and Mandarin literacy instruction for the children and family members. I led all English language activities and a co-teacher led all Chinese language activities. The co-teacher was a Mandarin and English speaker who was a teacher and member of the community.

Both studies centered children as primary sources of data while also drawing on data from families, caretakers, and teachers. Though some details of the data collection methods varied across the studies, the types of data collected included recordings of literacy activities and children's play, multimodal interviews with children, and interviews with parents, caretakers, and teachers. Data was collected across different contexts and settings, and whenever possible data collection was done in multiple languages to open spaces for the multilingual practices of children and their families, including in Spanish, Mandarin, and Cantonese.

Researcher Positionality

At this point, it is appropriate to share more information on my own positionality as a researcher. I study multilingual learners from an identity perspective that adopts critical and poststructural lenses on childhood and literacy to disrupt narrow conceptions of how children engage in, take up, and make sense of literacy learning (Wagner, 2024; Wagner et al., 2024). I do not share cultural and linguistic histories with most of the children and families in my research studies, though I strive to develop collaborative partnerships with members of the communities with whom I conduct research, often as research assistants, co-teachers, and co-investigators. I work to decenter, question, and critique my own assumptions through critical engagements with these partners, and to center the ways that multilingual learners make meaning of their own literacies and learning.

My interactions with the children in the focal studies focused on informal interactions, leading English literacy activities, and facilitating children's own development of a sense of who they were as a reader and a writer. I have limited proficiency in Spanish and do not speak any Chinese languages. I am a native English speaker and do not identify as multilingual because my Spanish language use is limited and not a routine part of my life. With Spanish-speaking children, I engaged in some conversational interactions and book readings in both Spanish and English. Though some children welcomed interactions with me in Spanish, other children preferred to interact with me in English. In the context of both studies, my position largely mirrored the role of an English-speaking teacher.

Reconceptualizing Validity Through a Multilingual Lens

This section explores themes that repeatedly arose through the critical inquiry process and became salient throughlines of this critical reexamination. These issues are specific to research on language and literacy with multilingual learners, and present threats to validity if left unaddressed by researchers. These are: (1) locality of language practices; (2) porousness of language spaces; and (3) agency in languaging. Along with an explanation of each issue, I include specific examples from my own research to illustrate how each issue may arise in the research process. In the section that follows, I offer suggestions for addressing these issues in the design and reporting of research.

The Locality of Language Practices

Language, culture, and community are interrelated in complex ways. Languages are used by people in ways that are specific to various macro and micro communities, and draw on language variations, multidialecticism, and usages that are informed by the cultural and social contexts of local, regional, and transnational communities (Bacon, 2017; Gee, 2012). The locality of language practices describes how languages are used by people in ways that are specific to these various macro and micro communities. If researchers adopt an essentialized view of language that does not account for locality in their research procedures, they risk misinterpreting observed language practices.

Avoiding risks of misinterpreting language use requires researchers to have adequate knowledge of community language practices or to work closely with members of language communities. Without this knowledge, researchers may misinterpret local language practices that differ from “standardized” norms, including translanguaging practices that bring together multiple languages in ways that can be locally specific (García & Kleifgen, 2020). Even in cases where there is a shared language between researcher and participants, researchers may be both a cultural insider and outsider. The use of a common language may conceal differences in cultural and linguistic backgrounds that inform meaning and lead to unintended (or unrecognized) misunderstandings that undermine the validity of interpretations (Lee, 2017).

For example, variations in the meaning of the Spanish word “ahorita” arose in interpreting a child's statement in the first focal study, with interpretations of this word's meaning differing from “now” to “in a while” offered by research assistants who were speakers of Spanish from Puerto Rico and the Dominican Republic. Even within the context of their own understandings of the word's meaning, these Spanish speakers expressed uncertainty in determining how another speaker’s usage of the term should be interpreted. Such variation in a word's meaning within a single language, such as Spanish, is illustrative of the differences in language use across communities, and the ways these variations can impact the work of researchers who make erroneous assumptions about the uniformity of language.

Issues of misinterpretation of language practices are often salient when translation is used in the research process, and perhaps are of even greater concern because the researcher may be unable to critically question translations when they are not familiar with or experts in local language practices. The translation is often a cross-cultural undertaking, even when the translator and participants share a language (Lee, 2017). As Afreen (2022) observes, “the translation process is indexical of translator identity” (p. 22), making it possible, if not likely, that the translator may impose their identity on a participant's words and meaning.

Outside translators who are removed from the communities under study may work from notions of “standard” languages that do not align with language use in practice, or may not account for the variability and range of local language practices (Bacon, 2017; Lee, 2017). Without knowledge of the cultural significance of shifts across languages and language varieties in local communities, translators may not be able to produce translations that maintain either formal equivalence, which “allows the reader to understand the source language context,” or functional equivalence, which make it “so that the text functions similarly to the way the original functions in its source culture” (Afreen, 2022 p. 10)

In the second of the two focal studies, I adopted a community-based approach to translation that relied on local language speakers as translators (Wagner, 2021, 2023a). This approach arose from the stance that community members are better equipped to correctly capture meaning, particularly around changes in local language practices and language varieties, than outside translators. Members of the local community were used as translators to center community language knowledge as expertise. This was combined with member checks, or getting feedback from research participants on translations of language data, which are a common approach to increase validity in postpositivist paradigms of research (Cian, 2021; Lee, 2017).

One of the primary translators in the second study was the co-teacher of the family literacy program, who was a member of the local community and a fluent Mandarin speaker. The co-teacher conducted the family interviews so that participants could use Mandarin or other Chinese language varieties. The co-teacher recognized when participants shifted between Mandarin and other language varieties, and though she was sometimes able to understand these language varieties, this was not always the case. This required us to consult other community members in the translation processes and to seek clarifications from participants.

In some cases, the co-teacher reported that she sensed an underlying tone or connotation in the meaning of a participant's language shifts. In her explanation of these interactions, the co-teacher described how she tried to clarify if what the participant meant in a regional language variety was captured by a Mandarin word. In some cases, the participant indicated that the co-teacher's selected word was insufficient and further clarification was sought, highlighting the need for language awareness and negotiation in this process. The co-teacher was able to negotiate clarifications when she was unaware of the exact meaning of a language shift only because she was cognizant of and attuned to attempts by participants to move across languages in intentional ways that were meaningful. Attending to the nuances and choices of participants’ movements across languages helps to build credibility and confidence that the translations used in the study capture participants’ intended meanings.

Because novel or atypical research processes inherently question accepted norms in the research field, such procedures often encounter resistance within the research community. The manuscripts written on this study included a description of the community-based approach to the translation process, similar to the description included above. During the peer review process of one manuscript on this study, an anonymous reviewer identified “errors in the English translations” (comments from the anonymous reviewer) and provided revised translations that they requested be used in the manuscript. For example, the reviewer included requests to change “less jumpy” to “less active” in a parent's description of her child's behavior in Mandarin. Though such changes can appear to be minor, there can be significant differences in the connotations of these word choices. While the parent's choice of “less jumpy” connotes a reduction in uneasiness or anxiousness, the reviewer's rewriting of the words to “less active” indicates only a reduction in physical movement that alters the tone and erases part of the meaning embedded in the parent's description.

This exchange illustrates the ways that outsider “expert” knowledge is often privileged over insider community knowledge and imposed on the communities who are studied. This notably occurred even when the reviewer was aware of the rationale and approach to translation used in the study. When community translations are changed by “expert” revisions, this increases the likelihood that such changes undermine the voice and intentions of the participants or function to “correct” participants’ language to “standard” forms (Bacon, 2017). This is particularly problematic when an “expert” without access to the complete data or knowledge of the context of the research event refutes a translation that was made by a person with knowledge of local language practices and which was verified through a member-check with the participant. Altering translations in these circumstances jeopardizes the ability of researchers to construct credible and trustworthy conclusions from what become compromised and unreliable translations of data.

In this specific case, a re-explanation of the community-based approach to the translation process and the raising of the issues described here proved sufficient to advance the manuscript in the peer review process. Whether or how often similar events might occur with other reviewers or editors remains unclear, as does whether some authors would preemptively make these kinds of changes to facilitate a manuscript's advancing in the peer review process. Such conflicts are also overlaid by the power dynamics in higher education more broadly, including pressures to publish for tenure and promotion. Exchanges such as this one bring to the surface the messy reality of how research decisions affecting validity are made, and the ways norms and historical practices for validity are reinforced in ways that extend beyond the immediate context of research studies.

Porousness of Language Spaces

Multilingual learners transverse and navigate languages and physical spaces in ways that complexify efforts to research their lived experiences, learning, and language practices (García & Kleifgen, 2020; Gee, 2012). Boundaries of space and place are particularly salient in research on children in the early childhood and elementary grades because of the ways children straddle home, community, childcare, and school spaces, and are learning what it means to carry and negotiate their languages, cultures, and identities with them across these spaces (Compton-Lilly et al., 2017; Wagner, 2023a).

In space-based research, researchers often focus on a single context for language use or look at ways language patterns differ across spaces, often by comparing school and home as discrete spaces (Gee, 2012). A product of space-based research is the creation of boundaries that often segment children's language resources in ways that do not align with their language practices, and that may be unrecognized by the language user (Granados, 2015). The porousness of language spaces describes the ways that languages are often not confined by space-based boundaries, and how these boundaries often do not map to language-in-use. This reflects a heteroglossic view of language that draws attention both to the multiplicity of languages and language varieties and to languaging practices that are flexible and hybridized (Bakhtin, 1981).

Anzaldúa (1987) captures the complexity of multilingual learners’ relationships with spaces through the concept of “borderlands,” which draws attention to the ways multilinguals encounter, cross, and wrestle with borders of culture, language, and spaces. Anzaldúa explores the ways multilinguals traverse borderland spaces and the accompanying hybridity and tensions of language practices and identities they entail, and how they negotiate “a vague and undetermined place created by the emotional residue of an unnatural boundary” (Anzaldúa, 1987, p. 3). Such discomfort with the notions of borders and boundaries reflects the “ambiguity, duality, and intersectionality” (Yazan et al., 2019, p. 134) of language practices and identities across spaces, and the ways that borders, whether physical or imagined, are not rigid, but porous.

In one example of how children often struggle to work within the artificial boundaries created by space-based rules for language use, Raina, a 4-year-old who spoke English and Cape Verdean Creole and was a participant in the first study, repeatedly used her languages in ways that pushed against the constraints of the classroom space she was in. Raina switched to Creole in conversations with me and when working with the teacher, even though she was aware neither I nor her teacher understood Creole. Raina attempted to explain these language crossings: “I kind of, my brain, I talk Creole sometimes and English sometimes.” In her response is evident the tension between her “brain” and the way she understands it processes and uses language, and the language rules and norms defined and enforced in school spaces. For a researcher interpreting language practices like Raina's, accounting for the ways that the spatial boundaries of school are arbitrary and porous to her meaning-making about language is necessary to produce interpretations that reflect her emerging understandings of language-in-use. Tendencies to instead characterize multilinguals with terms like “confused” ignore the questioning needed to understand the ways Raina and other children are making meaning about language, and risk mischaracterizing children's language experiences.

Raina is not unique in both the tension she feels in spaces that attempt to restrict her language practices, or in the ways she traverses her languages and creates spaces as linguistic borderlands. Dong, a 4-year-old Mandarin and English speaker, participated in the second study. During an English language reading of the book Frederick by Leo Lionni, Dong similarly worked across the linguistic borders constructed in this classroom space:

Teacher: “And Frederick said I send you the rays of the sun. And as he spoke of the sun the four little mice began to feel warmer. Was it Fredericks voice? Was it magic?” (Points to the illustration of the sun rays.) What color is this?

Child: Yellow.

Teacher: Yea, yellow like what?

Dong: Yellow.

Teacher: Like the sun, right? He's telling them about the sun and it's making them all feel warm.

Dong: Yellow, yellow, yellow. 黃的是月亮 也是黃的 玉米也是黃的 一樣的顏色 [The yellow one is the moon. It's also yellow. The corn is yellow. They are the same color.]

The English language of the text, English questioning by the teacher, and English responses by other children all established this as an English language space in this moment. Though Dong initially participated in English by repeating the word “yellow,” his visible wrestling with the tensions of this monolingual space and the limitations it placed on his ability to express his thinking led him to cross its borders to speak in Mandarin. Much like Raina above, Dong's language practices encounter and then cross the implicit rules or norms of language practices of this space. In both examples, the children's language practices resist the language rules others have tried to tie to the physical boundaries of these classroom spaces. When researchers impose these kinds of spaced-based rules or make assumptions tying language use to spatial boundaries, they risk presenting simplified or partial conclusions about language practices and reinforcing the false coupling of language and space. Recognizing and grappling with the complex interplay of space and language, and the ways these contexts and places are experienced by multilingual children, is needed to yield interpretations that reflect and honor multilingual ways languages are carried and used across spaces.

Agency in Languaging

Human activity is socially situated, and human interactions are constantly constructed and negotiated with others and with society at large. The agency is the “socioculturally mediated capacity to act” (Ahearn, 2001, p. 112) and describes the ways individuals make decisions and act within the social and structural conditions of the environment. Agency in languaging describes the specific languaging choices of individuals across social contexts. Fluidity in power relations, social structures, and interpersonal dynamics can shift language users’ agency and the ways they engage in language practices with others across differing contexts (Morita, 2006).

Yara, a Spanish and English-speaking child in the first study, is illustrative of the ways that even young children assert identities as language users and work to actively shape how they and others use language. Yara expressed clear preferences to read in Spanish over English, often making statements like “I like Spanish more better.” When I asked her if she also liked reading books in English, Yara replied, “No … I like in Spanish.” After I began reading one English book with Yara, she tried to shift the language of reading from English to Spanish by interrupting to ask, “Can I read it?” She could not yet independently decode, but when she started telling the story she used Spanish. When I later brought Yara a bilingual book, she told me, “I can help you with Spanish” to try to convince me that we could read it. Yara's willingness to assert her language preferences and identities in interactions with a researcher assures that aspects of her multilingualism are made visible, but this is not the case with all children.

The realities of agency raise questions about what and how researchers come to see research participants. Labov (1972) introduced the concept of the “observer's paradox” to describe how the researcher cannot know how their own presence (or absence) changes the speech of those being observed. This is fundamentally an issue of validity as it raises questions of whether and how the researcher can accurately interpret observed practices. Considering the agency of language users extends this idea a step further by requiring the researcher to consider the “concealed self,” or the ways a person may actively conceal practices or identities that they do not want to be part of their public identities in certain social contexts. This accounts for not only the presence of the researchers but also the way that context shapes what language practices a child may choose to use and make visible across contexts in the presence of others. This can include agentive decisions by multilingual learners to conceal language practices and identities in school or childcare settings, particularly when these communities are constructed around monolingual norms.

In one example of how children can act agentively to shape perceptions of their language practices, two Vietnamese-speaking children in the first study limited their speaking of Vietnamese to play spaces where they believed they were alone or out of earshot of adults. One of these two children, Jackie, was willing to talk about her use of Vietnamese with her family, including reading Vietnamese books with her mother and sister at home. However, she obfuscated about her language use in school, repeatedly declined to acknowledge her use of Vietnamese with her classmate, shrugging her shoulders or responding “No” when asked if she spoke or read in Vietnamese in school. In contrast, she repeatedly affirmed her ability to read in English. She even asserted an ability to speak Spanish, which was more common among her peers, and demonstrated this claim by counting, “Uno dos tres.” Jackie's case highlights how children can make persistent and ongoing efforts to control what is seen or known about their language use, and to shape perceptions about their language practices and identities.

Children's agentive choices around language can extend to more overt attempts to alter how they are perceived as language users. David, a 4-year-old Cantonese and English speaker in the second study, pushed back on teacher-imposed language choices to assert an alternate language identity while reading a Chinese book with his mother that was assigned by the teacher:

David: Why did you get this book?

Mother: 依個係老師比㗎。唔係我攞嘅。老師話要睇中文。[This book was given to me by the teacher. I didn’t choose to take this book. The teacher says we need to read in Chinese.]

David: Read it.

Mother: 狼婆婆 [Grandma Wolf.]

David: Then just say I’m ​not​ Chinese. Did you say I’m- say that I’m not Chinese, okay?

Mother: ((laughs)) You are Chinese.

Though David was Chinese (both of his parents had immigrated to the United States from China) and could speak Cantonese, his demand that his mother “say I’m not Chinese” was an attempt to refute and conceal his multilingual and multicultural identity in this context. Responses like David's to the use of specific languages show the ways that children actively position themselves as users or non-users of certain languages across contexts, and make agentive choices to conceal or reveal parts of their self to others.

As is evident across these examples, children can act agentively to support or subvert the language identities and practices imposed by others, including parents and teachers (Revis, 2019). Children's languaging practices can likewise be regulated and influenced by school contexts and norms, peer groups, and other aspects of their identities, including race, ethnicity, and immigration history (Soltani & Tran, 2022). Accounting for the ways that context and community inform the language actions and choices of multilingual learners is necessary to construct explanations of research data that are not only “well told” but that is also valid. This includes consideration of what the researcher has seen, and what may be unseen by the researcher, much as David's multilingualism might have been had his assertions that he was “not Chinese” not been captured by an unattended video camera. Considering how complete or partial a view the researcher has of a child and what is revealed in how a child acts to shape their identity as a language user are necessary to weighing the completeness or fullness of data on multilingual children.

Implications for Researchers of Multilingual Learners

Critically reexamining research practices and opening accepted research norms to debate is necessary if research is to be held accountable to the lives and lived experiences of multilingual children and their families. A critical thoughtfulness to approaching research design can further attempts to construct valid data that reflect how multilingual learners negotiate language, literacy, and learning. Explicit consideration of validity issues at the start of the research design process can help researchers to think through appropriate procedures for specifying, collecting, and analyzing data, including the contexts for research and plans for involving participants in the research process (Cian, 2021; Holliday, 2004).

Recognizing the locality of language practices begins with learning about and valuing community language practices and expertise. This requires researchers to consider the varied and intersecting linguistic communities that shape languages used by the participants in a research study. Developing meaningful and equitable partnerships with research partners can facilitate more authentic research processes that remain close to community practices and norms, and can include using community members as research assistants, co-teachers, or co-investigators in research studies.

Critical issues of language and community often arise in the act of translation, making this a key site for reconsideration in the research process. The inclusion of language biographies of translators so that they can be critically compared with those of research participants is one approach to allow for critical consideration of the translation process (Lee, 2017). Even when language biographies or positionality statements are provided, it is important to interrogate the ways these may reveal or conceal the complexity of language proficiency and language-in-use, and to critically assess what these mean for the credibility of findings. Another alternative is to use community-based approaches to translation, like those described earlier. This can be combined with member checks and other practices that include and value the language knowledge of community members in the translation process (Cian, 2021; Lee, 2017).

Whether the research design, researcher responses, and analysis of data reify the notion that languages are bound to specific spaces or acknowledge and are open to the ways that languages cross spaces, like Dong and Raina do in the above examples, impacts the wholeness and truthfulness of a study's findings and characterizations of language-in-use. Designing research studies that span spaces and contexts, or that create contexts that are designed to welcome hybridity and duality in language practices, is one approach to wrestling with the heteroglossic nature of children's language practices.

When research is confined to a single context or other spaces in participants’ lives are not accessible to researchers, researchers should critically engage with and question observed language practices that are monoglossic or fall within single patterns of language use. Creating opportunities for participants to reflect on language practices and data, considering alternate explanations that do not simplify language practices through space-based explanations, or conducting member checks to validate conclusions that may rely on space-based assumptions of language-in-use are some ways researchers may critically readdress validity to counter space-based paradigms of language and literacy.

To construct and collect data that represents multilingual learners and the ways they use language, researchers must also consider how they are supporting children to be agentive about languaging. This requires providing contexts for children to make languaging choices that are authentic and represent children's lived experiences and contexts for languaging, such as by including researchers who are members of or familiar with community languaging practices in interviews and other research protocols. This can be further supported by planning for forms of data collection that provide multiple and varied opportunities for languaging across contexts, including across the languaging contexts children experience in and out of school.

Encouraging children to reveal aspects of their concealed self requires the building of trust, which requires researchers to design studies to include adequate time to build meaningful relationships with participants in authentic languaging contexts. Allowing readers to assess the relationship between the researcher and participants can further researcher claims of validity, and requires providing sufficient information about the language histories of both the participants and researchers. This notably includes research assistants or others who may assist in various aspects of studies. This can be done by providing language positionality statements, and by acknowledging both the limitations and strengths of the researchers’ positions, including to local language practices and communities.

Though this section has so far addressed implications for research, shortcomings in how validity is taken up in research on multilingual children likewise have implications for pedagogy, assessments, and other classroom practices that affect multilingual children. In ways that are direct and indirect, research findings inform and shape the professional preparation and training of teachers, the development of curriculum, and educational policymaking. Research findings that lack validity can lead to spurious decisions in these areas, and shaky foundations that undermine the learning of multilingual children. As teachers make in-the-moment decisions in the classroom, the themes presented here can likewise compromise the pedagogical reasoning teachers use with multilingual children. In these ways, concerns about validity are neither marginal nor for researchers alone, but are germane to both educational researchers and practitioners invested in the betterment of research and schooling for multilingual children.

Conclusion

Whether researchers value and give attention to validity can serve as a barometer of research quality. Though it is important for researchers to be responsive to norms for research quality, researchers must also interrogate our collective understandings of validity to further define areas in need of examination, enable the more thoughtful review and evaluation of research, and prompt discussion on methodological questions that are important yet remain under-discussed in a broader research context that continues to center a monolingual view of research practices. These conversations are critical to extending and normalizing research practices that account for and center the realities of multilingual learners.

This article raises questions about research methods to consider how research practices can undermine or support the production of knowledge in ways that are equitable and responsive to multilingual children, families, and communities. Over the past several decades educational researchers have taken up calls to better understand the language and literacy practices of multilingual learners, yet there remains a need to critically assess the underlying monolingual norms that persist in research methods. By sharing my own process of critical reflection, I invite researchers to consider how we, as education and literacy researchers, can make our research methods more equitable and responsive by asking critical questions about the assumptions and biases that underlie our research norms and practices.