Abstract
Keywords
Introduction
In part due to misunderstandings about what qualitative psychological research is and the marginalisation of qualitative methods within the field of psychology (e.g., Riley et al., 2019), qualitative scholars have made significant efforts in producing guidance for qualitative researchers to improve the quality of qualitative research. It is recognized that identifying appropriate quality criteria for any qualitative project can be overwhelming for novices, journal reviewers and teachers alike. The aim of this paper is to build on previous explorations of quality criteria (e.g. Yardley, 2000) while reviewing and providing broad outline quality criteria, before turning to four widely used and taught approaches (acknowledging that this is necessarily limited). Issues that are particularly relevant to these approaches are then considered. The goal of this paper is therefore to help researchers consider the main strategies relevant to their work (or work they are reviewing). In attempts to establish standards in qualitative research, conflicting sets of criteria have been developed, reflecting the deep divisions among qualitative researchers (Hammersley, 2024). Some researchers have attempted to show how quantitative criteria can be applied to qualitative research by providing checklists that can operate as universal standards in assessing qualitative work (Cesario et al., 2002; O’Brien et al., 2014; Popay & Rogers, 1998). There is a danger in developing criteria for qualitative research that attempts to mimic quantitative studies with their very different approaches and aims. For example, the use of coding frames with established inter-rater reliability as a measure of coding quality and the accuracy or reliability of coding. Avoidance of ‘bias’ in qualitative research, reliance on large and representative samples or adherence to a realist interpretation of data are deemed illogical and meaningless within a qualitative paradigm, where meaning and knowledge are understood as situated and contextual (Braun & Clarke, 2021a).
Some of the most recognizable checklists (OREC, CASP) used as standards for qualitative research are regarded as overly prescriptive, reducing space for creativity and increasing the homogenization of qualitative reporting (Shaw, 2019, p. 741) and that such criteria are inappropriate for judging studies across the diverse approaches and the multiple interpretative practices represented in qualitative research (Johnson et al., 2020; Kitto et al., 2008). They contend that a strict emphasis on checklists restrains the diversity and multiplicity of practices within qualitative research, reducing it to a set of technical procedures (Barbour, 2001; Riley et al., 2019). These scholars rightly argue that criteria should be aligned with the varying epistemological positions of methodological approaches which hold different assumptions (e.g., phenomenology, pragmatism or constructionism).
The plurality of methodological approaches in qualitative research means there is variation in their theoretical, ontological and epistemological assumptions: their understanding of investigations and their methodological focus which serve as a blueprint for various qualitative research designs. On these grounds, researchers have questioned the feasibility and desirability of establishing general criteria, contending that the methods or techniques to use are practical in nature - choices about how a study can be assessed cannot be predetermined or made in advance of any individual study. Given this variety of approaches, it has been argued that traditional quality criteria or generic markers should be reformulated, and qualitative research should be evaluated against methodological standards of evaluation which would signal a high-quality study in specific areas, i.e., phenomenological, grounded theory, discursive and thematic analysis (Cresswell & Poth, 2018; O’Reilly et al., 2021). In response to such calls, authors have provided more refined quality markers relevant to the most widely used and taught methodological approaches in Psychology, including Reflexive Thematic Analysis (RTA) (Braun & Clarke, 2021b), Interpretative Phenomenological Analysis (IPA) (Nizza et al., 2021), Critical Discursive Psychology (CDP) (Edley, 2001a) and Discursive Psychology (DP) (O’Reilly et al., 2021) and Constructivist Grounded Theory (Charmaz & Thornberg, 2021). It should be noted that some of the quality criteria that are favoured in a specific approach may also be good practice in others, so criteria presented here are those that are particularly associated with that approach, even though they may also be applicable elsewhere. We therefore suggest following Symon and Cassell (2012) who propose that qualitative researchers should draw on the elements of quality that they deem most appropriate for their specific study with its specific aims.
While there is agreement in the application of standards when producing or assessing qualitative work based on the above methodologies, there is a need for criteria that are applicable for
In the current paper, we have provided a comprehensive set of markers and guidelines for quality standards that are organised into both general qualitative criteria and methodological approach-specific criteria. There are a vast number of guides to quality in qualitative research, so what does this paper offer to the novice researcher? Firstly, this paper provides comprehensive guidance which ensures that researchers have access to a holistic framework that covers fundamental principles as well as nuanced, context-specific guidelines relevant to their chosen qualitative approach in one place. This helps researchers tailor their methodological decisions to their specific requirements and allows them to compare and contrast between and within general and specific criteria. Secondly, the integration of general and approach specific criteria cultivates a deeper understanding of both the philosophical underpinnings and practical intricacies of qualitative inquiry, empowering researchers to navigate the methodological landscape with critical acumen and intellectual humility. We do this even by recognising that methodological criteria can be contrasting and competing even within methodologies. Finally, the inclusion of new criteria concerning impact and ethics on AI represents a cutting-edge advancement in methodological considerations which equips researchers with the necessary framework to navigate tangible impacts on practice or policy and the complex intersection of qualitative research and AI technologies.
In this paper, we begin by introducing the suggested general criteria which should be understood as a set of flexible guidelines that consider ways in which to demonstrate quality in qualitative studies. The general criteria presented are: transparency, rigor and richness, sensitivity to context, transferability and impact, and ethics that apply to qualitative research more generally and acknowledge the subjectively shaped products that are a result of the social context of the researcher and of those researched. See Yardley (2000) for general guidance on quality in qualitative research in health psychology, where similar criteria are featured (sensitivity to context; commitment and rigour; transparency and coherence; impact and importance). These criteria are generic and should serve as prompts, complemented with the later criteria which are specific to each adopted methodological approach or data collection method. We then move to present the “approach specific criteria” which we apply to four interpretive qualitative approaches: Reflexive Thematic Analysis, Interpretative Phenomenological Analysis, Critical Discursive Psychology/Discursive Psychology and Constructivist Grounded Theory. Although this article has been written with the intention that it would be used by the international qualitative community, all authors work within UK university contexts. In these settings, and in the teaching of qualitative methods in psychology, there is an emphasis on interpretivist research approaches which are rooted in constructionist or interpretivist epistemologies (Gibson & Sullivan, 2018). We have therefore focused on those methodologies that are most widely taught and engaged within UK academic settings. Finally, to illustrate the quality markers in these approaches and their similarities and differences, we present two tables with the essential qualities of the approaches to present the reader with more specific criteria that pertain to specific steps of the research process. The Table 1 is a summary of the specific criteria, whereas the Table 2 provides more detailed explanations of the criteria listed in the first table (see Appendices below).
General Quality Criteria
In this section, we present the general criteria that are essential for ensuring thoroughness in data collection, analysis and reporting of qualitative research. These criteria are:
Transparency
Transparency in research involves being candid and open about the research process. It is a practice in which researchers provide honest details about how the research was done, including our analytical and interpretative choices visible in a way that allows the reader to evaluate them (Moravcsik, 2019). These choices need to be aligned with the approach and analytic position employed. Researchers should ensure procedural rigor by being explicit in describing the way research was conducted through a detailing of how participants were recruited; how rapport and trust were developed; describing how data were collected, recorded, coded, and analysed; notes on interactions with collaborators and participants; and sharing accounts of how errors or participant refusals were dealt with (Kitto et al., 2008; Tuval-Mashiach, 2017). The researcher should therefore disclose every aspect of the data collection process, and the guidelines used to analyse data, by presenting excerpts of the textual data which affords the reader to discern their analysis and/or make available a record of the data to other analysts or the level of detail in transcription (Yardley, 2000). Some authors refer to this as an audit trail of notes and research content (e.g., interview guide, transcripts, audiotapes) that documents how the research was done (Seale, 1999). These can also be recorded in a diary, reflexive journal or memos and codebooks (when codebook methods are used). This information should be included throughout the account of the research, (e.g., described in the Method section but most evidence is included in the Appendices), or in content presented in case additional information about the research is required. This process also obliges the researcher to disclose the challenges and unexpected developments in the study, as well as revealing how the research focus transformed over time (Tracy, 2010).
Consistency and Coherence
A valuable way of thinking about issues of transparency in qualitative research is to ensure methodological consistency and coherence. Once a methodological approach is selected, a consistent approach needs to be adopted, as specific approaches have distinct features on a number of levels, including the type of questions raised, data collection, analysis and presentation of results (Holloway & Todres, 2003). Researchers should not simply “pick and choose from any qualitative strategy possible, and expect it to work out” (Mayan, 2009, p. 19). To illustrate this, if a study uses a phenomenological approach, there should be a focus on participants’ lived experiences in the findings section (Chenail & Duffy, 2011). This consistency must be demonstrated and reflected in the choices of research questions, the epistemological and ontological positions, procedures of data collection, analysis and knowledge claim. The fit between these research components contributes to coherence. This, however, should not be done in a rigid way of adhering to fixed methods or fall into the trap of ‘methodolatry’, a pervasive preoccupation with methodological concerns at the expense of other considerations (Chamberlain, 2000) that can lead to losing creativity and substance (Holloway & Todres, 2003). There are ways however which can be used to show flexibility and consistency at the same time. For example, some researchers claim that different methodological approaches within the qualitative research paradigm can be adopted. One useful way is by combining qualitative methodologies through triangulation of data. Triangulation can, for example, be achieved through the combination of interviews with focus groups or interviews with visual methods or other data collection methods. However, while combining methods can contribute to a more thorough exploration of complex phenomena, each method has its own assumptions in terms of theoretical frameworks we bring to research. In order to avoid blurring methods, researchers should consider providing information regarding data integration, data collection procedures and the insights obtained across data sets that are consistent with the chosen approach (Lambert et al., 2008).
Reflexivity
This refers to the awareness of the researcher’s influence in the (co)construction of knowledge. By engaging in reflexivity, the researcher accounts for
Interpersonal reflexivity concerns the relationships surrounding the research process impact the context, the individuals involved and the findings (Olmos-Vega et al., 2022). This involves engaging in reflections on the relations between the researcher and participant, understanding research as a co-production of research and an analysis of power dynamics between the researcher and the researched (Finlay, 2002). Researcher-participant relationships are not fixed and universal, however, researchers need to acknowledge the different social positions they hold relative to participants and the power that comes with these positions. Appreciating power positions in research is valuable in managing its influence on participants and data.
From a
Being reflexive is not limited to a specific section of a manuscript, rather researchers need to address steps on reflexivity across the manuscript (Haynes, 2012). Initially, developing a reflexivity plan that includes tools and strategies to facilitate reflexivity throughout the research process. In the method section, researchers should provide a rationale for choices made in the selection criteria, not only based on research aims but also considering the researchers’ positionality. Reflexivity informs positionality (Holmes, 2020). A good positionality statement (usually written in the Method section, after the Analytic Strategy subsection), should articulate the researchers’ viewpoints including the philosophical, personal, and theoretical beliefs and perspectives through which they view the research process.
The above elements of transparency have long been established as key quality criteria in qualitative research, nevertheless, often space to discuss or evidence these practices has been limited. However, as a result of broader mainstream concerns about the replicability, reproducibility and robustness of research, the open science movement has led to an increased focus on transparency in the wider research community. For qualitative researchers, the perceived imposition of open science practices designed primarily for research undertaken from quantitative, experimental, and positivist approaches may seem understandably incompatible and inappropriate (for example, the notion that open access to data is a necessity for all research fails to acknowledge legitimate sensitivities around qualitative data). However, if open science is understood as a set of flexible practices, then due to its increased focus on transparency, it represents an opportunity for qualitative researchers already familiar with reflecting on and articulating the reasoning for their own research practices.
Branney et al. (2022) describe a number of ways in which qualitative researchers can engage with and implement open science practices in ways compatible with a qualitative research paradigm. These include establishing a framework that acknowledges the input of all research contributors, using pre-registration as a tool to facilitate documentation of reflective practices, and reflecting carefully on what aspects of data can and cannot be publicly archived with appropriate permissions. By adopting these approaches, qualitative researchers can potentially engage meaningfully with open science practices through the evidencing of transparency (Branney et al., 2022).
Rigor and Richness
Rigor is a pivotal aspect of qualitative research, yet one on which there is little agreement due to variations in strategies across inquiry approaches, research goals and data collection methods. Rigor is essential throughout the research process: demonstrating thoroughness in data collection, analysis and reporting. This can be achieved by making informed decisions regarding suitable samples and contexts that effectively address specific issues (Tracy, 2010). Determining an appropriate sample size in qualitative research is not a straightforward matter. The size of a qualitative sample should strike a balance — it needs to be large enough to facilitate a nuanced and comprehensive understanding of the phenomenon under study, yet small enough to enable in-depth, case-oriented analysis (Vasileiou et al., 2018). If data collected from each person are useable and unique, meaningful input is collected from fewer participants, whereas they may increase the number of observations of the same group of participants in order to better engage with the complexities of the data (Roy et al., 2015). The scope of the study, the characteristics of the topic (e.g., complexity, accessibility), the quality of data and the study design influence choices in sampling (Morse, 2000). Depth and extent are interconnected, with depth depending on the questions asked and the diverse angles of vision sought. Yet, this needs to relate to the goals of the project: if the aims are to understand broad variations within specific phenomena, then there could be scope for an increase in sampling and the criteria of selection. If the goal is to understand the intricate dynamics of reality construction, then smaller homogenous sampling is the way to go. Each methodological approach will have a few guidelines for sampling decisions based on the type of project. However, the aim should be to attain enough data to support significant claims and use appropriate procedures in terms of interview practices, field notes and analytic procedures (Tracy, 2010).
Closely related to the sampling choices, is the practice and quality of data collection and analysis procedures. The structure of questions in qualitative interviewing and the type of questions influence the richness of data generated (Vasileiou et al., 2018). For example, open-ended questions, the order in which they are posed to participants, then the evaluation of fieldnotes and the practices to ensure transcript accuracy and the appropriateness of the interview all influence the quality of the data. Rigor in data analysis involves the capacity to provide evidence demonstrating that the findings are firmly rooted in data, thereby enhancing understanding. To showcase this attribute, qualitative researchers must clarify how they derive their results, relying on vivid examples from the data, including quotes, images, and text (Levitt et al., 2017). This method enables the reader to evaluate the accuracy of the analysis. For instance, phenomenological analyses often emphasise in-depth analysis of single cases (see below for criteria on specific approaches), whereas reflexive thematic analysis and grounded theory focus on identifying contradictions in data and seeking insights into emerging complexities that can result in data that grows both in depth and breadth. Nevertheless, the integration of expressive, imaginative, and creatively engaging writing can additionally enhance this evaluative process (Freeman, 2007). From a broader epistemological position, a constructionist researcher may rely on thorough self-reflection to ensure that the results are firmly rooted in the data. Alternatively, researchers guided by post-positivist principles may utilize interrater reliability calculations to enhance the reliability of their analysis. Critical researchers, on the other hand, might participate in a co-analysis process with participants, facilitating the co-construction of meanings that align with their life contexts (Levitt et al., 2017).
Triangulation is effective in enhancing research findings by drawing on diverse sources as well as integrating various methodological approaches. This ‘pluralistic approach’ involves combining more than one analytical method to enhance the multi-layered analysis (Frost et al., 2010). This refers to involving multiple qualitative approaches for analysing data, which enriches the understanding of the phenomenon being researched by offering a range of interpretations and ways of understanding how meaning is derived from the data. By combining multiple methods, researchers can explore structural, linguistic and contextual aspects of accounts which would enrich the analysis at multiple levels (Frost, 2011). This form of triangulation aims to achieve ‘complementarity’ by using different methods, with the purpose of reducing biases and assumptions inherent in a single method to provide insight into the complexity of the social world (Frost, 2011). Working with multiple methods would include ensuring that there is a clear rationale for theories and methods being used and that the researcher shows reflexivity and documents the research process.
Sensitivity to Context
In qualitative research, context is the domain in which subjectivities are developed and where societal constructs are interacted with (Cornish, 2004). These interactions and sequences unfold in historical and social domains (Locke & Budds, 2020), and identities are fluid depending on the context in which they are employed (Edley, 2001b). Therefore, high-quality qualitative research takes into consideration the study’s sociocultural contexts and how different societal contexts produce not only diverse constructs and meanings but also different accounts of turn-taking when speaking (Chauhan & Sehgal, 2022).
Thus, aligning the methodological approach with the social world is crucial for high-quality research, further emphasizing transparency in the research process. This includes considering and explicating research questions and participant selection, while also acknowledging that social norms shape and are shaped by the research context. Context-sensitive research acknowledges the impact of historical factors, apparatuses, power dynamics, and sensitivity. In high-quality qualitative research, analysts consider and describe contexts, aiming for diversity and strong justification for plurality and context-richness (Chauhan & Sehgal, 2022; Levitt et al., 2017).
A practical consideration when assessing context sensitivity is to examine what is considered or framed as ‘common sense’ alongside the hegemonic constructions of everyday life in the context the research is taking place (Donoghue, 2018). An example of this is the move from a reluctance to a willingness in men to talk about their emotions regarding testicular cancer (Seymour-Smith, 2008, 2013) Therefore, a researcher examining data relating to masculinity would need to contextualise them not only in relation to the cultural context but also the socio-historical era and the constructions that were hegemonic in pop culture and the media around that time.
The researcher also affects the context of the study through their own understandings, identity and insider experience. As a result, another quality indicator regarding context-sensitive research design is that the researcher has immersed themself in the context in which the research takes place (Levitt et al., 2017). This could be demonstrated through detailed descriptions that sufficiently inform the reader about the context and consider the extent to which the findings can be transferred to other social or power domains and exchanges (Levitt et al., 2021).
Transferability and Impact
Unlike quantitative research, qualitative research is typically not generalisable to the wider population (see Smith, 2018 for discussions about this). This is because qualitative research often involves smaller, homogeneous groups or explores specific issues with findings relevant to the selected participants in a particular context (Coyle, 2021). However, there are other ways to consider the relevance of our qualitative findings, such as transferability, considering how findings might apply to other contexts, settings, or populations. For example, Smith (2018) discuss how research findings from one context might overlap with other situations and could also be used to create ethnodrama (the practice of transforming research into a playscript to convey a sense of the findings to an audience) or docudrama (a televised or film production which utilises events that have happened) which draw upon research data. Additionally, concepts of generalization and theoretical generalization in qualitative research involve generating new concepts and theories that can be applied elsewhere (Smith, 2018). Goodman (2008) argues that discursive analysis can reveal how a particular discursive strategy produces the same interactional result across different settings, illustrating how existing prejudice can be used to justify further prejudice in various contexts.
Impact is perhaps one of the latest ‘buzz words’ within research and is a key criterion for assessing the real-world application of research. In the UK, the Research Exercise Framework specifically assesses impact as a means of assessing the excellence of research and is linked to the distribution of public funding to support research in UK higher education institutions. However, the ‘measurement’ of impact is not universal and differs across countries. More generally, impact is becoming increasingly important when applying for research funding. Qualitative research has great potential to achieve impact. Impact is not assessed by academic measures such as citations, but rather by the wider impact beyond academia that research can have on society, public policy and services, health, economy, or environment (Galdas, 2017). Typically, to demonstrate impact, there needs to be some way of evidencing a change occurring as a result of the research. An example of this could be whether a weight loss programme results in achieving weight loss for the attendees over a period of time.
Ethics
Ethical integrity is a crucial aspect of conducting high-quality qualitative research. In psychology, researchers are expected to follow institutional guidelines and legislative contexts to ensure their studies are ethical and conducted in a responsible manner. For instance, in the UK, qualitative researchers adhere to the British Psychological Society (BPS) Code of Human Research Ethics (Oates et al., 2021), which sets out the general ethical principles of the field. Similarly, a USA-oriented code of ethics is the Ethical Principles of Psychologists and Code of Conduct by the American Psychological Association (2017).
Yet, several challenges may hinder adhering to ethical guidelines in qualitative research. The emergence of new societal apparatuses can introduce novel ethical challenges, leading to an ever-changing landscape of ethical issues and debates. The continuous enrichment of research topics in qualitative research contributes to the evolution of ethical considerations. Technology, especially artificial intelligence (AI), is a domain profoundly impacted by these challenges, as the accelerating pace of technological developments constantly introduces new ethical dilemmas (Floridi, 2019).
AI presents a range of potential future concerns for qualitative research due to its rapid technological advancement, novelty, and social context (Floridi, 2019; McCradden et al., 2020). Examples of applied research in this domain include AI and human collaboration in vehicle management (Xu et al., 2023) and healthcare (McCradden et al., 2020). Qualitative research involving AI and human participants prompts questions about data protection, privacy, reliance on AI-generated predictions, the validity of computer outputs, commercialization of data, sharing data with private institutions, and confidentiality (McCradden et al., 2020). The unpredictable challenges posed by AI, coupled with the lack of up-to-date regulation, highlight the need for ongoing consideration (e.g., Liu et al., 2022). As suggested by Tasioulas (2019), legislative, institutional, and collective social morality often struggle to keep pace with the accelerating rate of technological developments. Consequently, individuals are frequently called upon to make decisions based on their own moral judgments. Such issues can result in a plethora of implications. AI often fails to adhere to appropriate ethical standards due to questionable data collection practices, such as targeted advertising on social media, which raises privacy and exploitation concerns by leveraging user data without clear consent (Tasioulas, 2019). These challenges are accompanied by algorithmic bias stemming from training data that reflects societal biases and prejudices (Tasioulas, 2019).
Such ethical dilemmas inherent in AI extend to qualitative research. It is crucial that qualitative research addresses the ethical implications of AI by striking a balance between the collective good and individual safety. This entails confronting potential concerns such as data management, consent, and transparency, and emphasising principles of responsibility and privacy (Bouhouita-Guermech et al., 2023). In navigating the ethical implications of qualitative research, researchers should take into consideration that AI serves to facilitate rather than replace research endeavours (Anis & French, 2023). As such, qualitative researchers should prioritise informed consent, assess the effectiveness, benefits, and risks of using AI in qualitative research, ensure safety and security, and maintain transparency (Bouhouita-Guermech et al., 2023).
The necessity of accountability mechanisms and consideration of ethical implications to address such challenges could be addressed through the implementation of open science principles. Implementing open science principles can encourage diverse participation and accurate representation of diverse populations’ needs. For example, AI research in healthcare or social justice (e.g., the glass ceiling) could be driven by the integration of diverse data sources (Christou, 2023; Ciechanowski, 2020; Tasioulas, 2019). Addressing these issues can be facilitated through the engagement of key stakeholders. For further exploration of AI and ethics in qualitative research, readers are encouraged to consult Anis & French (2023); Bouhouita-Guermech et al. (2023).
Another potential solution to such challenges is participatory science, a key aspect of open science. Participatory science involves collaboration between scientists and non-scientists to combat potential bias in AI applications. Through the active involvement of participants, especially those from underrepresented groups, biases can be identified and addressed. This process leads to more inclusive and representative datasets, ultimately enhancing the relevance of research questions to underrepresented populations (Norori et al., 2021).
Therefore, researchers working in sensitive or novel fields should develop moral sensitivity (Heggestad et al., 2013) and adopt a value-driven approach to assessing the best possible outcome for all the parties involved (Van der Burg & Brom, 2000). Indeed, the BPS Code of Human Research Ethics (Oates et al., 2021) suggests that when new ethical issues emerge that society does not cover, or when ethics are understood as tick-box exercises, the ethical quality of research can be compromised. As a result, the code of ethics emphasises the importance of “Respect for the autonomy, privacy and dignity of individuals, groups and communities. Scientific integrity. Social responsibility. Maximising benefit and minimising harm” (Oates et al., 2021, p.6).
This means our personal morality, collective morality and ethical guidelines need to be updated regularly. The standards of our discipline are therefore often open to debate, with consideration for the context sensitivity of ethics (Van der Burg & Brom, 2000). In such cases, ethics should be understood beyond narrow institutional definitions and tick-box bureaucratic exercises. High quality ethics require the academic community to not simply adhere to ethical guidelines but also view themselves as collectively responsible for upholding the ethical standards of the profession (Heggestad et al., 2013; Van der Burg & Brom, 2000).
Whilst such initiatives might initially appear time-consuming, high-quality ethical research has many benefits. For example, interviews have historically raised several ethical issues often discussed in contemporary psychology, such as unintentional harm, power imbalances and challenges around confidentiality and issues around anonymity. However, overlooking the fact that ethically conducted interviews can be beneficial to participants is common. Interviews provide participants with an opportunity to narrate their experiences, prioritise their own constructions, express emotions, and process their experiences in a cathartic manner (Corbin & Morse, 2003; Pilbeam et al., 2022). For instance, Wolgemuth et al. (2015) found that participants in their study benefited from non-judgmental conversations, opportunities for self-reflection, and using the interview as a means to connect with and advocate for social groups of their choice.
Such research contexts, however, require the interviewer to be reflexively aware of the fluidity of power in knowledge-production (Råheim et al., 2016). As a result, a potential way to evaluate power-related issues is to consider whether the participant should benefit more from the interview than the interviewer (DiCicco-Bloom & Crabtree, 2006). Moreover, the study should present more benefits than risks to the participant; therefore, the researcher should consider socio-political, power-related and emotional risks that could potentially emerge from the research process (Corbin & Morse, 2003).
Ensuring ethical research in qualitative psychology is crucial amid the evolving social landscape, even in the absence of clear guidelines for emerging topics. Researchers can navigate these challenges by embracing moral sensitivities that align with the socio-political context; making value-driven decisions, and fostering open communication with both society and the academic community. These approaches contribute to high-standard, ethical research, and offer substantial benefits for the study, researchers, participants, and the broader field of qualitative psychological research.
Specific Criteria for Qualitative Approaches
In this section, we outline four commonly used qualitative research approaches that align with constructionist or interpretivist epistemologies: Reflexive Thematic Analysis (RTA), Interpretative Phenomenological Analysis (IPA), Critical Discursive Psychology/Discursive Psychology (CDP/DP) and Constructivist Grounded Theory (CGT). Firstly, we provide context for the different versions within a broader approach and then move to the specific approaches mentioned above by presenting their quality criteria. While some specific criteria overlap with the general criteria mentioned above and criteria within approaches, certain general criteria vary in the depth to which they are addressed across distinct approaches.
Reflexive Thematic Analysis
‘Thematic analysis’ encompasses various data analysis approaches that share commonalities but differ in specific procedures and underlying ontological and epistemological assumptions. Thematic analysis is widely used for identifying and analysing patterns in qualitative data and is a
This paper focuses on
Explicate Assumptions Underpinning Approach and Ensure Consistency Between Work Presented and Approach Claimed
As noted above, there exist multiple approaches to thematic analysis and ‘thematic analysis’ should not be presented as a single orientation to qualitative data analysis. Given the diversity of philosophical positions which can underpin different thematic analysis approaches, it is important that researchers are clear about the specific thematic analysis approach they are taking. Whilst some thematic analysis approaches can be used from a range of different paradigmatic positions (King & Brooks, 2018), other approaches are more constrained.
The thematic analysis approach we are focusing on here - Reflexive TA - is explicitly located within a paradigm which understands ‘qualitative research’ as encompassing both procedure and philosophy (rather than simply referring to non-numerical data or the use of particular tools and techniques) (Braun & Clarke, 2019). It should be made clear what theory is being drawn on to inform the use of RTA and there should be methodological integrity - the overall research design, theoretical position claimed, language and concepts used, and analysis presented should all be coherent, and in keeping with RTA’s assumptions and procedural practices (as distinct from other thematic analysis approaches) (Braun & Clarke, 2021b).
Quality assurance strategies employed should similarly be consistent with RTA. Specifically, coding reliability and data saturation measures are not appropriate in RTA, researcher subjectivity is seen as a resource and coding as an inherently subjective process. Coding quality stems from depth of engagement with data, and meaning is constructed by the researcher through situated, reflexive interpretation. Quality assurance processes suggesting that consensus is an appropriate measure of coding quality (coding reliability) or that there are a finite number of themes and ‘correct’ meanings residing independently within the data (data saturation) are not therefore consistent with the values and assumptions of RTA (Clarke & Braun, 2021b).
Discuss Practice Reflexively (Avoid Generic Procedural Descriptions)
Evidencing rigor and a systematic approach to data analysis are appropriate quality assurance strategies but should be undertaken in an appropriately reflexive (and contextually located) way (Braun et al., 2022; Terry et al., 2017). Guidance on undertaking RTA describes six (recursive) phases, which are explicitly intended as tools to facilitate a process rather than as steps to simply apply (Braun et al., 2022; Terry et al., 2017). The analytic procedures used should be clearly outlined but not simply as generic procedures. To acknowledge the researcher’s central role in knowledge production there should be a clear outline of how the procedural steps involved in RTA were engaged with, including an appropriately reflexive discussion of practice and decision-making.
Appropriate Conceptualisation of Themes (Avoid Topic Summaries)
Different approaches to thematic analysis conceptualise themes in various ways. In some thematic analysis approaches, themes represent topic or domain summaries and simply summarise (perhaps divergent) views on a particular issue. The aim of RTA is not to simply describe and reduce data, and data topics should not be presented as themes (Braun & Clarke, 2021b). Theme development in RTA requires considerable analytic work involving a comprehensive and inclusive coding process. Data is interpreted and made sense of, not simply paraphrased or generated from a few vivid examples. There should be a good balance between analytical narrative and illustrative extracts. The analysis presented should be convincing, capturing cohesive patterns of shared meaning across a dataset. Themes in RTA should be distinct, each underpinned by a core central organising concept that underpins the theme (Clarke & Braun, 2018).
Interpretative Phenomenological Analysis
Phenomenology is a philosophical movement focused on the study of experience (Willis et al., 2016). Within psychology, the two main strands in this movement are descriptive phenomenology and interpretative phenomenology. Descriptive or ‘eidetic’ phenomenology is guided by the work of Husserl and aims to describe the ‘essence’ or the general characteristics of a phenomenon rather than the individual’s experiences (Giorgi, 2008). The objective is to describe phenomena as they manifest within consciousness. In order to study experience, it is essential to identify commonalities in the experiences of participants, so that a description is achieved. Essences are believed to represent the true nature of the phenomenon being studied. Such assumption posits that essences generated through phenomenological research yield a single correct interpretation of the experiences of individuals involved, reflecting a foundationalist approach in inquiry (Lopez & Willis, 2004). This perspective views reality as objective and not dependent on history or context. Giorgi’s contribution in specifying procedures for collecting and analysing lived experiences resembles a distillation process, whereby the researcher filters out essential elements to reach a precise description of the phenomenon (Wertz et al., 2011). Descriptive phenomenology does not seek to explain the experience or attribute meanings to it, instead, it focuses solely on the account of the actual experience. In employing this approach, researchers are involved in a phenomenological reduction and bracketing, whereby the researcher must set aside any preconceptions, presumptions, or biases regarding the phenomenon being studied, enabling it to be revealed in its distinct and subjective lived experience by the participant (Colaizzi, 1978).
In contrast,
Rather than what participants are consciously aware of, the focus is on what individuals experience (Solomon, 1987). Analysis should delve into uncovering meanings that may not be readily apparent to participants but are discernible within the narratives they generate. Echoing Heidegger’s notion of the
In interpretative phenomenology, analysis involves making sense of participants’ experiences to give meaning to the account and understand it. By engaging in a hermeneutic circle (the double hermeneutic) of recovering meaning, the researcher becomes deeply involved in the process of understanding the participant’s experience and unfolding different levels of meaning. In a further layer of interpretation, the researcher contextualises participants’ experiences by reflecting on the social and cultural context and the meaning imbued within their accounts (Willig, 2012). Another important assumption underlying the interpretative phenomenological approach is that researchers’ presuppositions and knowledge are valuable guides to research and make the inquiry meaningful. Instead, they emphasise that it is impossible to eliminate the background of understandings that have led to pursuing a particular research topic (Lopez & Willis, 2004). Another key difference is that the interpretive approach does not rely on a theoretical orientation or conceptual framework to direct the research. Instead, the theoretical approach is applied as needed and used to inform decisions about the research process (such as the research question, sampling and subjects). It can also serve as an orienting framework by the researcher to clarify assumptions about the study and to interpret findings. Consequently, the researcher should explain how the framework is utilized in data generation and interpretation.
Across the phenomenological psychology literature, debates rage on as to which approach best does phenomenology ‘justice’ (Matua et al., 2015). Descriptive phenomenology is most used in U.S. contexts, whereas interpretivist approaches have been widely applied in UK health and social research settings. One example of a popular interpretivist approach in psychology, and the one we will focus on, is
Constructing a Compelling, Unfolding Narrative
While this criterion is presented under IPA, it can relate to other approaches within qualitative research too. Findings should convey a ‘story’ or coherent narrative that has a sense of progression both within individual themes and across developed themes. Within individual themes, analytical claims are supported with illustrative quotations from participants that best support the claims underpinning the narrative. Researchers can develop a good analytical narrative by selecting the most appropriate quotes and the right order to put them in, with a clear alternation of a description of findings, quotes and interpretation, woven to explain the overall sense of the theme. Cumulatively, themes should tell a rich and coherent story that is supported by quotes from the data (Nizza et al., 2021).
Developing an Experiential And/Or Existential Account
As IPA is concerned with the examination of subjective experience, the concern of the analysis is in unravelling the meaning of major things that have happened in participants’ lives: “the experiential significance of the thing that’s happening” (Smith, 2019, p. 168). Following in-depth interpretative work, meaning is explored and narrated at the existential level, which involves answering questions of identity, continuity, and individual agency (Smith, 2019). Through this process, IPA analysts aim to provide meaning at the highest level by demonstrating the consequences that experiences have on a person’s sense of self and an individual’s existence.
Close Analytic Reading of Participants’ Words
The double hermeneutic is a central feature of Interpretative Phenomenological Analysis. This involves the researcher making sense of how the participant is making sense of their experience. This process is reflected in analysis through detective work by making present not only the obvious but also what is latent and hidden within the data (Nizza et al., 2021). This is achieved by closely analyzing and interpreting specific words and accounts made by participants, the tone and the imagery they connote. Findings need to present the close analytical reading of the words participants use to illustrate their experiences. In this way, the researcher increases transparency and trustworthiness, and the reader can comprehend the hermeneutic circle of the researcher’s making sense of the participants’ meaning-making.
Attending to Convergence and Divergence
IPA’s distinctive hallmark is its commitment to an idiographic analysis (Nizza et al., 2021). In a case-by-case approach, the experiential analysis intends to capture the personal experiences of each individual participant. The idiographic analysis is achieved by providing detailed analysis at the individual, participant level. Then, multiple cases can be compared in order to identify points of similarity and difference. Such analytical reflections of commonalities and particularities in data need to be presented in writing to highlight the interconnections and nuanced experiences of individuals.
Critical Discursive Psychology and Discursive Psychology
Discourse analysis is an umbrella term which covers a range of analytic approaches including some of those most used within psychology such as Critical Discursive Psychology (CDP) (Wetherell & Edley, 2008) and Discursive Psychology (DP) (Edwards & Potter, 1992; Wiggins, 2017). In contrast to many qualitative approaches, the focus of these discursive approaches is on language as a site where we constitute knowledge. The methodological criteria for CDP and DP both converge and differ. Both approaches take a relativist stance in that they postulate that things in the world are inseparable from our representations of them (Wiggins, 2017). In terms of data, CDP typically analyses a wide range of empirical materials from newspaper reports to interviews. In contrast, DP typically focuses on ‘naturalistic data’ due to a number of problems associated with interviews (see Potter & Hepburn, 2005). Furthermore, each discursive approach situates itself with a slightly different focus on macro and micro aspects of data analysis. For example, Schegloff (1997) argued that a technical analysis should privilege the categorisations that interactants make relevant and ground our analyses in what we see displayed in the data rather than imposing researcher’s concerns or interpretations onto the data. In contrast, CDP researchers propose that this technical analysis misses the wider thread and cultural and rhetorical influences that interactants draw upon (Billig, 1998; Wetherell, 1998).
The three main markers of quality are developing an appropriate research question, action orientation focused analyses and linking to related research. Developing an appropriate research questions is, of course, applicable to all qualitative approaches but the last two markers are specific to discursive research of the variety focused upon here. Other forms of discourse analysis, for example, Critical Discourse Analysis (CDA) (Fairclough, 2013) also treat language as a social practice but favour a more macro analysis to focus on how societal power relations are established and reinforced through language use and does not focus on the minutiae of action orientation.
Developing an Appropriate Research Question
Potter and Wetherell (1987) argue that research questions need to give priority to discourse and be related to construction and function: “how is discourse put together, and what is gained by this construction” (p. 161). In addition, the research question must fit with the analytic approach employed. However, discursive research is often inductive and as inclusive as possible. For example, initial discursive research questions may stem from our interest in specific forms of interactions. Yet, we might spot something of analytical interest in data that was not part of the original focus. DP explores how concepts related to psychology are constructed, made relevant and unfold while interacting. Therefore, a good quality research project and thus question should explore preferences, issues around accountability, how beliefs are invoked to justify certain actions and how identities are employed and change in everyday interactions (Wiggins, 2017). CDP, much like DP, can explore issues relating to negotiation and identity yet also explore the wider available cultural concepts. Formulating a discourse-oriented research question requires an alignment of theory with practice (Potter, 2003; Wiggins, 2017), whereas practice here implies the action orientation of participants’ discursive productions. A practical tip to assess a good research question would be that it specifies the action orientation element with words such as “negotiate” or “construct” and focus on the context of the research (Wiggins, 2017).
Another practical consideration is that discursive research questions should be tackled as their own entity and not as a by-product of exploring inner mental states. As such the research question should approach ‘talk’ and ‘text’ by emphasising the action orientation element and not as a medium for cognitive processes. Therefore, the focus should be in talk and written text. It is additionally important to explore not only what talk does in the context it is produced in but also what is achieved through constructions. A good research question should align with theory and contribute to an analytic argument or explicate new discursive constructions. Thus, the action orientation element that underpins the research question should have a functional purpose regarding what the interactions accomplish and their rhetorical purpose (Potter & Wetherell, 1987).
Action Orientation Focused Analysis
Analysis should be grounded in the epistemological framework of the analytic approach employed. Action orientation is fundamental to discourse analysis. This refers to the constructivist notion that talk (or text) is not merely communication, but is the site where things happen, agreements or disagreements occur, accusations and requests are made, and identities are forged. It is therefore important that the analysis focuses on what is being accomplished in the text. This means that using discourse analysis to attempt to show what participants are ‘really thinking’ or ‘actually mean’ is contrary to what DA is about. This is not to say that discourse analysts are not interested in talk about psychological states, in fact, this can be a key area of interest for discourse analysts, but to stay consistent with Edwards and Potter’s
Antaki et al. (2003) emphasize this focusing on what is
Linking to Related Research
When analysing data, discursive researchers typically draw upon other discourse and conversation analytic research to support their analyses. There is a need to be wary about linking to other topic related research due to the different epistemologies employed - we are not comparing like with like. Care also needs to be taken in the conclusion on the same grounds. Highlight instead what insight the discursive approach affords.
Constructivist Grounded Theory
The constructivist approach to Grounded Theory Methodology (GTM) is a repositioning (Charmaz, 2009) of the methodology originally developed by Glaser and Strauss (1967). Both Glaser and Strauss (alone and with Juliet Corbin) developed their own iterations of GTM with distinct philosophical underpinnings and approaches, and more recently researchers have proposed additional variants of GTM; Charmaz’s (2014) constructivist version is popular in a number of disciplines including psychology (Mills et al., 2006). Although we focus on the constructivist approach here, this represents only one version of GTM. Interested readers are encouraged to consult a recent review of quality in the Glaserian approach by Vander Linden and Palmieri (2021) and the latest edition of the textbook outlining the Straussian version (Corbin & Strauss, 2015) as well as the many papers and books on these versions of GTM.
Broadly, Glaser’s version of GTM focuses on the emergence of theory through careful application of GTM analysis techniques, views researchers as more objective (e.g., Glaser, 2002), and has been described as post-positivist (e.g., Urcia, 2021). Strauss’ approach to GTM is more interpretivist, acknowledging multiple perspectives (Urcia, 2021) and applying a broad range of potential data analysis techniques, such as axial coding and the conditional/consequential matrix (e.g., Strauss & Corbin, 1994). As there are different iterations of GTM it is important to carefully consider which to use and to avoid a ‘mash-up’ of incompatible techniques and assumptions, a potential danger for novice researchers (Breckenridge, 2012). Charmaz’s approach adopts methodological strategies from both Glaser and Strauss’ versions of GTM but adopts its own epistemology, within a social constructionist paradigm (Wertz et al., 2011).
An explicitly constructivist approach aims to produce an abstract understanding of a phenomenon (rather than an explanation, as in other versions of GTM); focuses on actions, meanings and constructions; and acknowledges both subjectivity (of participants and researchers) and context (Charmaz, 2014; Charmaz & Thornberg, 2021). The notion of the subjectivity of researchers is one that has been criticised heavily by Glaser, notably in his 2002 paper critiquing Charmaz’s constructivist version of GTM. For Glaser, “the data is what it is” (Glaser, 2002), and applying any theoretical framework a priori, including epistemological perspectives such as constructionism, is a form of data forcing 1 . Glaser did not completely reject the notion of researcher subjectivity, but his version of GTM relied upon the idea that a grounded theory will be “an abstraction from time, place and people” and that the researcher’s perspective is a source of bias “to weave into the constant comparative analysis” (Glaser, 2002, p. 3).
In addition to researcher subjectivity, in constructivist GTM a further emphasis is on processes, defined by Charmaz as “unfolding temporal sequences that may have identifiable markers with clear beginnings and endings and benchmarks in between” (Charmaz, 2014, p. 17).
A constructivist grounded theory research approach is open and curious; codes and categories are treated as provisional, able to be revised and rejected; we play with ideas and data, considering different theoretical explanations and checking them against the data; learning to tolerate ambiguity is important (Charmaz & Thornberg, 2021).
Charmaz (2008; 2014) proposes four quality criteria for constructivist GTM: credibility, originality, resonance and usefulness. Other forms of GTM have their own quality criteria such as Glaser’s (1978) criteria of fit, relevance, work and modifiability which draw upon the original criteria of fit, relevance and work set out in
Additionally, Charmaz and Thornberg (2021) propose a set of guidelines for constructivist grounded theorists, although they caution that this is not intended to be a prescriptive recipe for conducting constructivist GTM. Charmaz (2014) notes that quality criteria will depend on who develops them, for what purpose, and that they will differ for different disciplines and contexts.
Credibility
To demonstrate credibility, data should be rich, relevant, and extensive enough to allow us to ask incisive questions, make comparisons and develop thorough analysis. To gather rich data, we need to be open to the empirical world and willing to understand the experiences of different people. Data should also be sufficient to allow comparisons, create robust categories and convince readers of their significance (Charmaz & Thornberg, 2021). Charmaz suggests considering various aspects that can help to develop credibility of both analysis and argument: developing “intimate familiarity” (p. 337, 2014) with our research setting; constructing categories that reflect a wide range of observations; demonstrating a clear logical link between data and argument; and making systematic comparisons.
Originality
Charmaz and Thornberg (2021) suggest a number of routes to demonstrating originality including reconceptualising existing problems in new ways and providing new insights. We can also ask whether our analysis provides new conceptual rendering of data and consider the potential social and theoretical relevance of the theory, and its ability to “challenge, extend, or refine current ideas, concepts and practices” (Charmaz, 2014, p. 337). A useful way to do this would be to compare the final theory with existing research and literature, a part of the comparative method (Charmaz, 2012) that is scarcely used (Barbour, 2001, cited in Charmaz, 2012). Coding with gerunds for action and process can help researchers to view concepts and ideas from new angles. For an example of coding the same data for themes and topics and with gerunds see Charmaz (2012).
Quality constructivist GTM studies move beyond taken for granted understandings so that they are not simply descriptive, in the sense that researchers take concepts as given and do not take them apart to investigate how they are constituted, a problem with much research that claims to be GTM (Tweed & Charmaz, 2012). The aim of grounded theory is not to describe data but to
Finally,
Concluding Comments
Assessing quality in qualitative research is critical for evaluating the methodological quality of a research design. This guide is especially (but not only) relevant to novice researchers seeking to identify the main quality criteria of each methodological approach and identify the differences among each methodological approach and their criteria. Now that there is a large, and growing, body of quality criteria articles, we consider the most valuable contribution we can make is to show the qualitative researcher a synergy of both types that provides relative utility for novices in conducting research. General criteria are generic and should serve as prompts which should be complemented with the criteria which are specific to each adopted methodological approach or method of inquiry. We encourage researchers to use both general and specific guidance and use these all in a flexible manner. We acknowledge that not all criteria can be summarised in a single paper. We therefore encourage researchers to gain deeper insights into the criteria of each methodology through engaging with the relevant literature that has been signposted throughout this paper. For example, for a systematic comparison of the approaches discussed here and other approaches that are based on similar or distinct traditions and philosophical assumptions, you can refer to literature which provides an overview on these debates (Braun & Clarke, 2021b; Nizza et al., 2021; Wertz et al., 2011; Wiggins, 2017; Charmaz, 2009). While it is not always possible for researchers to strive for all criteria, it is essential that research is done with integrity and that researchers take responsibility for all aspects of the research. We hope the article will stimulate further thinking regarding methodological criteria that can be contrasting and competing even within methodologies.
Footnotes
Declaration of Conflicting Interests
Funding
ORCID iDs
