Sage Journals: Discover world-class research

Abstract

With qualitative methods being increasingly used in health science fields, numerous grids proposing criteria to evaluate the quality of this type of research have been produced. Expert evaluators deem that there is a lack of consensual tools to evaluate qualitative research. Based on the review of 133 quality criteria grids for qualitative research in health sciences, the authors present the results of a computerized lexicometric analysis, which confirms the variety of intra- and inter-grid constructions, including within the same field. This variety is linked to the authors’ paradigmatic references underlying the criteria proposed. These references seem to be built intuitively, reflecting internal representations of qualitative research, thus making the grids and their criteria hard to compare. Consequently, the consensus on the definitions and the number of criteria becomes problematic. The paradigmatic and theoretical references of the grids should be specified so that users could better assess their contributions and limitations.

Keywords

qualitative research quality evaluation criteria guidelines paradigms Alceste® lexicometric analysis

Introduction

The use of qualitative research methods in the social and human sciences considerably increased as of the 1990s (Chamberlain, Stephens, & Lyons, 1997; Flick, 2007; Miles & Huberman, 1994; Murray, 2000; Silverman, 1993; Smith, Harré, & Van Langenhove, 1995). This dissemination quickly reached the health field through nursing, medicine, and public health. This renewed interest in qualitative research promoted the publication of numerous “quality criteria” grids/guidelines (we use the words criteria grids, guidelines, and checklists as synonyms) to evaluate the quality of a qualitative research, project, or article (e.g., in scientific journals such as the British Medical Journal, Psychology and Health, Family Practice, Sociology of Health and Illness, etc.). This abundance of guidelines, however, resulted in the use of numerous terms and grids, which failed to contribute to the clarification of the main consensual criteria to evaluate qualitative research. This leads to a certain lack of recognition of this work in publications and for fundraising purposes, the solicited reviewers declaring that they are ill-informed or do not have the tools to evaluate qualitative research (Green & Britten, 1998; Greenhalgh & Taylor, 1997; Mays & Pope, 2000; Popay, Rogers, & Williams, 1998).

As part of an ongoing research (this document represents part of a larger research funded by the Swiss National Science Foundation (www.snf.ch), 2011-2014: “Quality of Qualitative Research in the Health Sciences: Which Evaluation Criteria?”), we examined 133 evaluation grids in five major health science fields: “medicine,” “public health,” “nursing,” “psychiatry and psychology,” and “research methodology.” To better understand their variability and its origins, we conducted a computer-assisted lexicometric analysis, our premise being that the diversity of the grids should not only be linked to the various disciplinary fields but is also intrinsic to various qualitative research conceptions among grid designers. In this article, we start by presenting a brief status report on the health science research field, in terms of quality criteria. We then present our methodology. In the third part, we present our main results. Finally, we discuss the inherent limitations of the various research conceptions highlighted within the different disciplines they represent.

Problem: A Shift in the Quarrels From the Last Century?

In a context of an increase in the publication of qualitative studies both in books and recognized scientific journals, debates on the relevance of developing (or not) specific criteria for qualitative methodologies have evolved continuously (Devers, 1999; Dixon-Woods, Shaw, Agarwal, & Smith, 2004; Mays & Pope, 2000; Yardley, 2008). Certain authors deem that the validity and reliability concepts stemming from quantitative methodologies are suitable to reach a high level of rigor in qualitative research (Miles & Huberman, 1994; Morse, Barrett, Mayan, Olson, & Spiers, 2002; Porter, 2002); however, another group of authors finds that applying the “reliability” and “validity” criteria directly is unacceptable for qualitative studies. This last group argues for differentiated criteria for qualitative work based on specific philosophical and methodological foundations. They argue their position by insisting on the structured thought needed in all paradigm transfer, especially to apply the same criteria to social and human sciences as those applied to natural sciences (Cicourel, 1964; Garfinkel, 1967; Hamberg, Johansson, Lindgren, & Westman, 1994; Schutz, 1962; Van den Hoonaard et al., 2004). Somewhere in the middle, a third group of authors argues that evaluation criteria can be transferred from quantitative to qualitative methods, as long as they are adapted (Pope & Mays, 2006).

In following these debates, we notice that the discrepancies regarding the quality criteria seem to reflect different conceptions of qualitative research (Santiago-Delefosse, 2007; Schneider, 1998). Yet, these discrepancies may explain why it is so hard to build consensus that will allow for the evaluation of qualitative research quality within the scientific community, especially because they remain obscure in literature on quality criteria grids for qualitative research.

Some work highlights the variability of existing grids (Cohen & Crabtree, 2008; Flick, 2007; Mays & Pope, 2000; Popay et al., 1998) and the difficulty in comparing them. The inconsistency of the number and objectives of the criteria used, the absence of consensual and explicit definitions, and the absence of construction consistency between the grids of one disciplinary field were identified in a previous publication (Santiago-Delefosse, Bruchez, Gavin, & Stephen, 2015). Therefore, research conceptions underlying these grids may be questioned as well as the epistemological and paradigmatic positions that guided their designers. This questioning is still highly relevant as the lack of consensus regarding a certain number of criteria, even minimal, is an argument to which article and fundraising evaluators refer and which motivates the rejection of projects.

Given these facts, our research focused on analyzing the links between the health science disciplinary fields and the major paradigms guiding grid designers in determining the criteria used to evaluate qualitative research. We used Kuhn’s (1962) general definition, which states that a paradigm is a set of disciplinary convictions shared by the scientific community at a given time during its history. Any science is stabilized by sharing the same paradigm, which marks its field and investigation procedures. This paradigm is developed by groups of socially dominating researchers who control the transmission of knowledge through research funds, positions, and publications. This definition captures the epistemological and paradigmatic choice that is made in all research conducted, whether this choice is deliberate or the outcome of reflection, or much more implicit.

Method

Make-Up of the Corpus

In total, 133 grids (checklists or guidelines) were examined in the various health fields, from the MEDLINE, PsychInfo, CINAHL, PeruniL, ScienceDirect, and Web of Sciences databases, using the keywords “health,” “qualitative research,” “assessment,” “appraising,” “quality criteria,” “peer review process,” “guideline,” “evaluation,” and “standards.” Data were used according to the following inclusion criteria: (a) grids published for health scientists (medicine, nursing, health psychology, health sociology and anthropology, public health, etc.), (b) guidelines that were sufficiently developed and commented, and (c) work published by health journals, institutes, and organizations whose scientific quality has been validated.

Whether they are taken from scientific journals (instructions to authors), articles for authors and researchers to improve their qualitative work or as an evaluation support tool, or methodology manuals, most of the grids examined are presented as lists of criteria to meet to produce “good qualitative research” that is rigorous and reliable. Most of these grids are accompanied by a text explaining the criteria (definitions and uses) and/or a theoretical or methodological text that is sufficiently detailed to support the quality criteria analysis.

Most of the grids examined come from publications dating from 1993 to 2011, which correspond to the increase in debates on the quality of qualitative research in the 1990s. Most grids come from medical journals (45), nursing literature (39), research methodology journals/literature (20), public health literature (19), and psychiatry/psychology literature (10). Most evaluation grids were published in medical and nursing literature.

We submitted our corpus to two analyses: a written analysis with inter-evaluator agreement that allowed us to determine coding categories for our second analysis and a computer-assisted lexicometric analysis (Alceste®).

Thematic Analysis of the Content and Theoretical and Methodological Foundations of the Analysis Categories

Using Alceste® requires prior coding of the text to show the software which variables must be identified in the text. To do this and identify the variables that would be most relevant to our questioning, we conducted a first thematic analysis of the grid content. This first content analysis helps to identify the “contextual variables” specific to each text (i.e., author, discipline, and paradigm; “paradigm” here refers to a system of fundamental beliefs belonging to the researchers and used as the basis for their theoretical and methodological choices in the conception of the grids). The variables are then coded to make them compatible with the computerized analysis. The content analysis was conducted independently by three researchers. The results were triangulated and discussed collectively to come to an inter-evaluator agreement (Braun & Clarke, 2006; Robert & Bouillaguet, 2002).

The reference framework used for this content analysis was the one used for the work conducted by Denzin, Guba, and Lincoln. Indeed, besides their heuristic scope, they are the most often cited in literature on the importance of paradigms and epistemologies in qualitative methods (Denzin & Lincoln, 1994/2011, Santiago-Delefosse & Rouan, 2001).

In 1994, Denzin and Lincoln proposed a classification that included two major paradigms subdivided into four epistemological research positions: the positivist/post-positivist paradigm (including the positivist or post-positivist epistemological approaches) and the interpretivist/constructionist paradigm (including the critical or constructivist epistemological approaches). Notice that in 2011, they added a “participative” epistemological position to the interpretivist/constructivist paradigm. A certain number of criteria differentiate these epistemological and paradigmatic approaches. Among these criteria, three major axes caught our attention as they can be operationalized and help to compare the criteria in the grids: (a) the form and nature of reality (following the preexisting or co-constructed one), (b) the researcher’s type of position regarding research (outside observer vs. participant involved in the co-construction), and (c) the use of the research (discovery of inherent truths vs. work on subjectivity and the impact of its construction on the world, work on the co-construction of realities, work on dominating forces and empowerment). We added a fourth axis regarding the presence of ethical questions, which also indicate the researcher’s position toward the research (absent when the research is considered an objective report on what exists vs. always present when the researcher, his or her values and the context are considered a part of the research at all times).

We analyzed our corpus by focusing on these four major axes, which help to report on the conceptions underlying the research in the 133 grids examined. As a result, we identified five paradigmatic and epistemological approaches: The first category was defined as “post-positivism”; the second one, “post-positivism/interpretivism” bringing together grids that combine both approaches; the third category entitled “constructivism/interpretivism” was built upon a similar basis; the fourth one was “critical constructivism”; and the fifth one was the “post-modern” approach including grids with criteria dealing with radical relativism/constructivism (considering, for example, that qualitative research is too singular to be evaluated). No grid was identified in the “positivist” epistemological approach.

Presentation of Alceste® and Relevance of the Lexicometric Analysis Based on Our Hypothesis

We used an analysis produced by Alceste® to process the contents of this corpus. Alceste® is textual or statistical data analysis software, originally designed by Max Reinert, of the CNRS, in France (Reinert, 1990, 1993). It has been used in the human and social sciences since the 1990s. It counts words in a frequential manner and helps to obtain analysis units that are based on formal criteria. It uses an inductive and recursive approach and helps to identify co-occurrences, or word associations in a sentence, by processing the words according to their similarities and differences. Technically, Alceste® breaks the corpus into fragments that are relatively similar in size, referred to as “context units” (e.c.u. the smallest statistical units created by the software, based on a compromise between the syntactic form [proper punctuation] and the statistical constraints [these units must be of comparable size]). These same fragments are then reclassified statistically and split into classes whose specific vocabulary is as different as possible. These classes are called “lexical worlds” and are considered to present an “image” of the representations contained in the text, the ideas and the corpus’ major themes (Garric & Capdevielle-Mougnibas, 2009). Second, it defines a link to what software developers refer to as “contextual variables” (data elements known by the researchers and used as instructions for the software so that the lexical analysis can be conducted with these variables in mind). These variables are used to identify the texts and are related to their content. The researchers introduce them according to their relevance to the research questions. The classes identified by the software must then be examined and linked to the co-occurrences to give them meaning and explain their differences (Aubert-Lotarski & Capdevielle-Mougnibas, 2002). Notice that the software’s results are based on frequencies rather than weighting, which could be considered a limitation.

We deemed it relevant to use this software because of its textual classification abilities, given that we hypothesized that the differences between the grids were not solely linked to disciplinary fields in the health sciences, but represented paradigmatic and epistemological positions within certain disciplines, connecting certain grids of disciplines that differed from those from which they stemmed. If this hypothesis was validated, we would obtain a classification of grids that would be more connected to paradigmatic/epistemological positions than to disciplinary fields. This does not mean that certain disciplinary fields cannot be consistent among one another (i.e., the older and more solid ones).

The following variables were used for the lexicometric analysis conducted as part of this work:

The five health disciplinary fields in which grids were published: medicine, nursing, public health, psychiatry/psychology, and research methodology.

The five paradigmatic and epistemological approaches as identified in the thematic analysis of the content: post-positivism, post-positivism/interpretivism, constructivism/interpretivism, critical constructivism and post-modernism.

Table 1 presents the “contextual variables” used and adapted for Alceste® coding and stemming from our written content analysis.

Table 1.

Contextual Variables Used for Alceste® Coding.

Five paradigmatic and epistemological approaches
Paradigmatic position	Post-positivism		Constructivism, critical constructivism, and post-modernism
Epistemological position	Post-positivism	Post-positivism/interpretivism	Constructivism/interpretivism	Critical constructivism	Post-modernism
Alceste® coding	pp1	pp2	c1	c2	c3
	The acknowledgement of an external reality to be discovered using both qualitative and quantitative methods that are capable of “representing” or “approximating” this reality. The introduction of qualitative methods corresponds to “a way to capture” this reality “as best as possible”. The researcher and his or her topic are considered independent.	Mixed grids mixing criteria stemming both from a post-positivist and interpretivist approach (hermeneutics, psychoanalytic . . . ). It designates an approach according to which there exists one dynamic and evolutionary reality to be discovered through the discourses.	Grids using a knowledge approach according to which there apparently exists multiple realities. Our image of reality is a construction of the human spirit, which must be interpreted in a social, cultural and temporal context. According to this approach, the researcher cannot be separated from his or her subject and interacts with it.	Refers to a constructivism approach that adds a critical dimension whose goal is to bring about a reform of knowledge-building in a sociocultural context.	Grids that highlight social constructivism according to which our understanding of the world not only reflects it but also organizes and orders it. Possibly knowing what “the reality” is questioned, and brings us closer to a form of relativism, where the researcher’s values and judgments change his or her subject.
Five health disciplinary fields
Disciplinary fields	Medicine	Nursing	Public health	Psychiatry/psychology	Research methodology
Alceste® coding	Med	Nurs	Pubh	Psy	Meth

Note. No grid examined corresponded to a pure positivist epistemological approach (which is normal for grids targeting the evolution of qualitative research).

Results

During the written analysis and according to the contextual variables used, we obtained a distribution of grids by “disciplinary field” and “paradigmatic and epistemological approach” (Table 2).

Table 2.

Distribution of the Grids by Disciplinary Field and Paradigmatic and Epistemological Position.

	Alceste® Codes
Disciplines (number of grids)	pp1	pp2	c1	c2	c3
Medicine (45)	4	23	11	7	0
Nursing (39)	1	9	14	11	4
Research methodology (20)	4	10	1	4	1
Psychology/psychiatry (10)	0	4	3	3	0
Public health (19)	4	11	2	2	0
Total = 133	13	57	31	27	5

Note. No grid examined corresponded to a pure positivist epistemological approach (which is normal for grids targeting the evaluation of qualitative research). pp1 = post-positivism; pp2 = post-positivism/interpretivism; c1 = constructivism/interpretivism; c2 = critical constructivism; c3 = post-modernism.

There are only a few post-positivist (pp1) grids in the corpus. They generally propose similar criteria from a positivist perspective: There are some in medicine, research methodology and public health, one in nursing, but none in psychology/psychiatry. The post-positivist/interpretivist approach (pp2) is the most important in our corpus (57/133) and concerns grids that include criteria close to the positivist field and more interpretative criteria: They are mostly found in medical journals. Constructivist/interpretivist grids (c1) are not as numerous (31/133) and propose criteria specific to qualitative research, integrating contextualized constructivism and the researcher’s interpretative position: They are mostly found in the fields of nursing and medicine. Grids classified in the critical constructivism category (c2) are slightly less numerous (27/133) and include criteria specific to qualitative research, with a constructivist approach: They are found in all disciplines, predominantly in nursing. Finally, there is a small group of five grids that is harder to classify in the previous categories, with a profile that is often explicitly geared toward a post-modern approach (c3) and published mainly in nursing.

We note that the grids are mainly split into two large groups. The “post-positivist/interpretivist (pp2)” group represents half of the corpus (57/133), and then the sum of the two “constructivist (interpretivist and critical)” also represents half of the corpus (58/133). Consequently, the issue regarding the discrepancies between the grids may stem from the difference between these two large groups, especially in the definition of the two “interpretivist” positions.

Lexicometric Analysis and Distribution of the Corpus Among Six Classes

Once Alceste® processed the corpus, the latter was split into six distinct classes. Figure 1 details the class distribution and indicates the five most significant terms (χ²) given by the software for each of the six classes as well as the contextual variables related to them: discipline and paradigmatic approach. To make it easier to read the analysis, we also numbered the quadrants of the factorial design (A, B, C, D).

Figure 1.

Distribution of the six classes and contextual variables.

The factorial design (Figure 1) shows the interrelations between the classes and their respective positions in the quadrants. Certain classes, namely, Classes 1 and 5 in Quadrant C, and Classes 2 and 6 in Quadrant B, are closely linked to one another, almost interwoven. As for Classes 3 and 4 (Quadrants A and D), they are more well-defined and independent. We thus notice that the classes do not necessarily cover the disciplines, as there are two psychology/psychiatry groups with different approaches, as well as two nursing grids (Classes 2 and 4) or public health grids (Classes 4 and 6) characterized by different paradigmatic approaches. We also notice that Class 4 includes grids from three disciplines: medicine, public health, and nursing.

The factorial design also indicates a bipartite distribution between the grids underlying approaches that are essentially constructivist (c1) and focused on the researcher’s qualities as an integral part of the research in quadrants of C and D (Classes 1, 4 and 5), and the grids with more post-positivist approaches that highlight the “good research practices” needed to ensure the research’s validity and objectivity, in Quadrants A and B (Classes 2, 3, and 6).

Dendrogram of the Six Classes and Significant Terms

The examination of the significant terms (χ²) present in each class can explain the specificities in the factorial design and identify those of each class. This analysis highlights the specific identity of the grids and the differences/similarities between them. The dendrogram (Figure 2) shows how the six classes are organized and linked.

Figure 2.

Dendrogram: Distribution of the corpus into six classes with significant terms (χ²).

The six classes are split into two main branches including on one hand Classes 1, 4, and 5 (Quadrants C + D) and on the other Classes 2, 6, and 3 (Quadrants A + B).

In the first branch (Quadrants C + D), two classes containing psychology/psychiatry grids stand out: Class 1 with its constructivist/interpretivist approach (c1) and its criteria focused on experience, the meaning of things and comprehension; and Class 5, which is more focused on validation criteria (triangulation, member-checking) and a constructivist approach mixed with post-positivist criteria (c1, pp2). Some medicine, nursing, and public health grids (Class 4) can also be found in the constructivist/interpretivist group (c1), using criteria that are more based on practices, care, research use value, and clinical contributions.

The second branch (Quadrants A + B) includes grids dealing with a post-positivist approach: On one hand, public health grids (Class 6) with a post-positivist approach, using certain criteria specific to the interpretivist field (pp2) and describing mostly sample selection methods and inclusion/exclusion criteria, and on the other nursing grids (Class 2) focused on literature, work framework, research process, and being a little more related to the constructivist field (pp2, c2). As for the research methodology grids in Class 3, they present very diverse approaches, ranging from post-positivism to post-modern (pp1, c3, c2), with their content focused on practices, tools and research techniques. In that respect, these grids, compared with the other groups, seem to be more linked through criteria based on methodological rigor alone, than through the discipline or paradigm.

It is worth noting that Quadrant D (Classes 1 and 4) mainly corresponds to the constructivist/interpretivist position (c1) while Quadrant B (Classes 2 and 6) mainly corresponds to the post-positivist/interpretivist position (pp2). These two quadrants include criteria from grids from four of the five disciplinary fields; however, Quadrant C corresponds to certain criteria from the psychology grids only (Class 5) and Quadrant A to certain criteria from the research methodology grids (Class 3).

These initial results show that the characteristics of the grids do not necessarily cover a classification based on the disciplines and that their differences or similarities are more linked to the paradigmatic and epistemological positions that underlie them. Only two classes (5 and 3) seem to be more determined by disciplinary proximity (Class 5 to a lesser extent though). Consequently, these differences must be examined to determine their meaning. The following analysis provides further detail on the six classes based on their elementary context units (e.c.u., semantic content) and contextual variables (disciplines and paradigms) to better understand their weight in the differentiation of the grids.

Analysis of the Classes According to the Significant Terms and Contextual Variables

Quadrants C and D (Classes 1, 5, and 4): “Mainly” constructivist approaches

Class 1 (psy, c1)

This class includes a group of grids that stems mainly from psychology/psychiatry (psy, χ² = 60), with a constructivist/interpretivist approach (c1, χ² = 8). This group of grids is characterized by terms such as experience, meaning, effect, self, understand, change, live, preconception, reflexivity, affect, instrument, and so on (i.e., in the grids of Curtin & Fossey, 2007; Elliott, Fischer, & Rennie, 1999; or Fossey, Harvey, McDermott, & Davidson, 2002). A joint review of the class’s elementary context units shows that the epistemological approaches of this class’s criteria are geared toward the notions of the experience lived by a subject in its context of meaning, culture, interpretation, and empowerment as an impact on the research participants or reader. In general, “authenticity” and “trust” are based on the principle that the research accurately reflects the meanings and experiences lived and perceived by participants. On the other hand, a self-reflective attitude is recommended. Indeed, comprehension and interpretation are linked to the assumptions of the researcher who is deemed to be integrated to the research both socially and culturally, and interacting with the subject in its context to co-construct the interpretations and help to build new theories.

This class proposes specific evaluation criteria for qualitative research. They are linked to the meaning produced, in a context explained by researchers who clarified the implications, and demonstrated their training in terms of interpretative theories. In this group of grids, and in keeping with the constructivist paradigm as defined, the goal of the research is to understand and “reconstruct” its initial subject, as a consensus between the researchers and participants, and be open to new interpretations. As such, the validation of the research places a high value on the researchers’ training and experience, as well as on the consideration of their own “biases” and the characteristics that are likely to influence the research. The criteria proposed in these grids are usually adapted to qualitative or mixed methodologies.

Class 5 (psy, c1, pp2)

A second group of psychology/psychiatry grids (χ² = 3), different from the first, holds a distinctive place in this class. This group is also dominated by a constructivist/interpretivist approach (c1, χ² = 9), but with a post-positivist-interpretivist component (pp2, χ² = 4) also present, which provides this group with an approach that is increasingly marked by research bias validation and reduction notions (e.g., Meyrick, 2006 or Stiles, 1999). This class presents significant terms such as member, establish, valid, different, confirm, external, corroborate, truth, triangulation, and audit. With the examination of the class’s elementary context units, they help to specify the preferred objectives in these grids that promote the intersection of numerous perspectives (triangulation) to reach a sufficient level of trust (credibility) to validate the research. In each case, the strategy involves cross-checking the results with other sources and perspectives. Criteria that are more specific to qualitative research and the constructivist approach are also proposed: member-checking to see whether the interpretations are consistent, conducting an iterative analysis, or looking for the saturation/recurrence of the themes presented in the results.

In this group, as per the constructivist paradigm, research is considered to be a collaborative and collective co-construction endeavor whose researcher and participants are tools in the search for consistency and the integrity of the reports and inferences. The content and vocabulary used by the quality criteria of the grids, however, also indicate a need to objectify the research through validation techniques stemming from more positivist approaches. The criteria proposed in these grids are adapted to qualitative or mixed methodologies.

Class 4 (med, nurs, pubh, c1)

A group of grids in medicine (χ² = 10), public health (χ² = 5) and nursing journals (χ² = 2) holds a distinctive place in Quadrant D, while still being close to Classes 1 (psy, c1) and 2 (nurs, pp2, c2). For example, it includes the Cohen and Crabtree (2008) grid published in medicine, the Walsh and Downe (2006) grid for nursing and the Dixon-Woods et al. (2004) grid published in public health. Various significant terms given by the software, such as practice, knowledge, discipline, contribute, policy, social, practitioner, standard, academic, and develop, direct the grids in this class toward notions dealing with the applicability of the research and its relevance to the disciplines concerned. The epistemological and paradigmatic position of these grids is constructivist/interpretivist (c1), and their elementary context units generally show an interest in development, interdisciplinarity, originality, the meaning given to events in a specific content, and a foundation built on existing knowledge. Two major axes have been identified. The first deals with the evaluation of qualitative research (reaching “trustworthiness,” a sufficient level of reliability and contextualization). The second deals with its practical and/or scientific use in a broad way (its purpose).

Most grids in the class highlight the importance of the connection between the study conducted and the existing knowledge, the fact that any new knowledge can contribute to new conceptualizations or questions, lead to the development of basic theories or hypotheses, question existing theories or provide methodological information leading to improvements in the practice. Certain criteria, especially in public health, focus on the social mandate, moral obligation as a rationale for health sciences. Like for the grids in Class 1, Class 6 displays a paradigmatic constructivist approach in which the goal of the research is the “reconstruction” of its subject and the development of new interpretations. The criteria proposed in these grids are adapted to qualitative or mixed methodologies.

Quadrants A and B (Classes 2, 6, and 3): “Mainly” post-positivist approaches

Class 2 (nurs, pp2, c2)

This class includes a group of grids stemming from the nursing field (χ² = 11), mainly displaying a post-positivist/interpretivist approach (pp2, χ² = 4), with criteria stemming from a critical constructivist approach (c2, χ² = 3). For example, there are Cobb and Hagemaster (1987), Russell and Gregory (2003) and Beck’s (2009) grids. This class’s significant lexical content includes terms such as framework, question, paper, study, review, reference, discuss, purpose, rationale, and explain. A complete examination of the significant terms in Class 2 shows that the approach is mostly marked by notions dealing with the research anchor in a clear theoretical framework, and scientific validation through a comprehensive literature review that must be used to place the new study in a broader context and show its specificity. The objectives and steps must be explained and the connections between the research and the clinic highlighted, as well as the impact on health practices and policies (a common aspect with Class 4). In this group, the researcher explains the research’s logical steps to demonstrate the validity and reliability of the reasoning. Another major objective of the grids in this group is to bring about new models and theories and contribute to international debates. Comparing the results obtained with those of other studies must ensure their transferability.

In a mainly post-positivist/interpretivist perspective, the criteria in these grids focus on the reasoning of the researcher and methodological rigor, with, as its validation terms and conditions, the faithful reporting of the logic of the steps, sampling and data collection. The criteria proposed in these grids are generally adapted to quantitative or mixed methodologies.

Class 6 (pubh, pp2)

This class is closely intertwined with the one formed by the nursing grids described previously (Class 2). It also includes grids with a post-positivist/interpretivist approach (pp2, χ² = 14), but mostly stemming from public health journals (pubh, χ² = 13), such as the grids by Tong, Sainsbury, and Craig (2007), Daly et al. (2007), or the Critical Appraisal Skills Programme (CASP; 2006). This group of grids is composed of criteria that differ from those in the public health grids of Class 4 (med, nurs, pubh, c1), and are more closely linked to methodology and sampling issues. The main significant terms of this class highlight the technical considerations surrounding sampling and its justification: sample, select, sampling, size, inclusion, purposive, case, exclusion, convenience, age, extreme, or recruit. In this group, it is important to explain the logic governing the selection of the target sample, what the inclusion and exclusion choices are, and why and how the populations, events, people, and so on were selected.

Often closely linked to positivist criteria, the criteria in Class 6 marks the need to represent the population (sometimes related to statistical analysis) to ensure the transferability of the results. Like for the grids in Class 2, step description and data collection are also preferred in guaranteeing the rigor of the research as a whole. The criteria in these grids are usually generally adapted to quantitative or mixed methodologies both because of the paradigmatic approach (pp2) and the discipline concerned (pubh).

Class 3 (meth, pp1, c3, c2)

This class is well specified and does not overlap onto any other class. On the disciplinary front, there are many grids stemming from research methodology articles and/or literature (meth, χ² = 94). From a paradigmatic and epistemological point of view, however, it remains very diversified, with an essentially post-positivist predominance (pp1, χ² = 29), but also including the presence of criteria linked to a post-modern (c3, χ² = 9) or critical constructivist approach (c2, χ² = 6). For example, there are grids by Flick (2006) or Sandelowski and Barroso (2007). The most significant terms in this class are linked to the quality of the data (from their collection to their retention) and the training of the researchers in terms of the methods used: record, data, note, document, collect, consent, transcript, procedure, audio, observe, confidential, or protect.

The grids in this class provide a thorough description of the research process, emphasizing the data collection tools and data retention and protection methods. Indeed, data storage must allow for independent control, and the specific description of the process must show how the data are processed and how they correspond to the goal of the study. In keeping with the post-positivist paradigm as described, concern for validation is reflected through issues related to the proof and quality control (of the data, processing, and training of the researcher). There are also ethical criteria among the quality criteria. In this class, this criterion constantly refers back to the consent of participants as well as to the information they receive on the objectives and process of the research, more than to the researcher’s position. Generally speaking, the content of the grids refers back to general criteria for qualitative research, and to criteria specifically adapted to quantitative or mixed methodologies. Finally, this class stands out from the rest because of its roots in issues related to methods, dominant both among the authors of the grids stemming from positivist approaches and constructivist approaches. This may explain its diverse character, given that it includes grids from post-positivist (pp1), post-modern (c3), and critical constructivist (c2) paradigms.

Our results show a certain order among the six classes. In Quadrants C and D, there is a first group that includes grids whose dominating theoretical model is identified as constructivist, that is, they base their relationship with reality on the idea that the human being is constructed collectively, and that the best we can do is interpret in a more or less shifting social, cultural, or temporal context. The criteria of this group of grids present the researcher as an integral part of his or her study subject, and in constant interaction with it. As a guarantee of the “authenticity” of the research data and results, these grids emphasize the faithful reporting of the experiences lived by participants and the clarification of the researcher’s personal implications and biases, as well as his or her experience and training in terms of interpretative theories. The second group (Quadrants A and B) includes grids whose dominating theoretical model was identified as post-positivist, with the addition of criteria stemming from post-positivist and interpretivist approaches for some. The criteria in this group of grids suggest looking at reality as a series of existing facts and/or events irrespectively of the researcher. The goal is to discover or at least approximate. Therefore, methodological rigor is a guarantee of validity, with emphasis put on the logic of thought and the transparent description of the research process.

Table 3 summarizes the results and highlights how the grids are split according to a paradigmatic and epistemological continuum, all the while showing that the boundaries between post-positivism and constructivism are not clearly specified. This result is even more important in that the sum of grids pp2 (post-positivism/interpretivism) and c1 (constructivism/interpretivism) is 88/133. As such, the “interpretive” position seems to represent the quality criteria grids for qualitative research the most. Moreover, given that the software Alceste® sorted the data “Interpretation” into two separate categories, we acknowledged that there are at least two distinct meanings for this concept. The first is more or less solipsistic, that is, the researcher knows and can interpret the data using the right theory. The second is more specific in that the interpretation is actually a co-constructed language between the researcher/subject context. In the first case, the interpretation is linked to the examination of a preexisting reality. Linked with the literature and theory, it must explain a situation using an appropriate sample selection method. In the second case, the interpretation is linked to meaning, in connection with cultural comprehension; it calls on the researcher’s reflexivity and commitment, helps to clarify the practice, and allows for useful applicability for patients. More generally, we also notice that few grids were classified in/under the “post-positivist” and “post-modern” paradigms. By contrast, a greater number were included in the columns that bring together mixed approaches. An important finding stemming from the classification of grids based on their underlying research conceptions was their distribution along a continuum from post-positivism to post-modernism, including interpretivist and constructivist approaches, and representing a Bell curve (Table 3).

Table 3.

Distribution of the Paradigmatic and Epistemological Positions According to a Continuum: Summary of the Results.

Paradigmatic position	Post-positivism (with different dimensions)		Constructivism, critical constructivism, and post-modernism
Epistemological positions	Post-positivism	Post-positivism/Interpretivism	Constructivism/Interpretivism	Critical constructivism	Post-modernism
Alceste® code	pp1	pp2	c1	c2	c3
Number of grids	13 grids	57 grids	31 grids	27 grids	Five grids

Characteristics	Two methodologies can be linked in one research, or adapted criteria stemming from a positivist field: Mixed and explicit methods	Mixed criteria, some from the interpretivist field and others from the constructivist field. Mixed epistemologies (not very explicit)	Specific qualitative criteria applied to qualitative research only. Used in a general interpretative-type approach	Specific qualitative criteria applied to qualitative research only. Constructivist and/or critical approaches	Explicit demand for a post-modern approach (depends on the authors’ declaration)
Quadrants	Quadrant A	Quadrant B	Quadrants C + D	Quadrants A + B	Quadrant A
Class detail	Class 3 (partial)	Classes 2, 6 (5 partial)	Classes 1, 4, (5 partial)	Classes 2, 3 (partial)	Class 3 (partial)
	Procedures/methods	Procedures/methods	Meaning/context/use	Procedures/methods	Procedures/methods
Disciplinary field	Research methodology	Nursing, public health, psychiatry/psychology	Psychiatry/psychology, medicine, public health, nursing	Nursing, research methodology	Research methodology
Significant terms	Procedure, data, record, interview, document, protect, ethics	Literature, framework, clinical, rationale, sample, select, recruit, random	Experience, meanings, cultural, reflexivity, practice, contribute, patients, useful	Phenomenon, explicit, philosophy, observation, procedure, confidential	Procedure, data, record, interview, document, protect, ethics

Discussion

Our content analysis and Alceste® lexicometric analysis confirm the presence of major discrepancies between the quality criteria grids examined and their relationship with the various paradigmatic and epistemological approaches identified in the grids. References to the major research paradigms or epistemological positions of the researchers are generally implicit, and are not easily identified because of the complexity of the content and structure of the grids. Indeed, on one hand, the same approach can be identified in grids belonging to different classes, and on the other hand, different approaches can be merged into the same grid. Although the six classes stemming from the lexicometric analysis can be placed in a consistent order based on the vertical partition of the previous factorial design (Figure 1) and the significant terms related to them (Figure 2), this distribution remains global. Indeed, the dispersion of items from the same grid to different classes reveals a lack of uniformity in the structure of the grids and the underlying theoretical approaches in the content of the criteria. This diversity shows how unclear the underlying paradigmatic and epistemological approaches in the evaluation grids are and how complex their links are. Consequently, it is hard to draw a clear line between them and to place them in only one category. We can hypothesize that the authors of the grids themselves seem to have overlooked this construction diversity. Yet, it may “objectively” explain the experts’ “subjective” difficulty when they say that they do not have the tools needed to evaluate qualitative research. The latter notices the lack of consistency between the different grids and even sometimes within the same criteria grid. Still, beyond this lack of consistency, our results highlight various key points.

The 133 grids can be categorized on a continuum. First, the grids underlie theoretical approaches close to quantitative work, with the criteria “translated” into qualitative, while being rather focused on the description of the research steps, with an emphasis on methodology and virtually no criteria referring back to the theories, epistemology, and values. Second, grids underlie theoretical approaches and criteria that are more specific to qualitative work, with reference to the researchers’ training and their research position, little or no description of the steps, and an emphasis on the values, epistemology, and theoretical analysis.

The notion of interpretation is a pivotal point that differentiates the research positions, while contributing to their semantic confusion. It seems that interpretivism can be given at least two meanings. It can be an interpretation by the researcher, according to his or her theories regarding the words spoken and the observations made of a preexisting reality, and can also be a co-constructed search for meaning between the researcher (reflexive) and actors, according to a context and in a practical use objective. This last definition could also correspond, in part, to the “participative” position proposed by Lincoln, Lynham, and Guba (2011).

We had considered the call for “ethical” criteria in the grids as an indicator of the researcher’s position with regards to the research, its absence indicating an objectivist position and its presence a position that takes into account the values of the research and the impacts. Our results question this premise. It also seems here that the presence of criteria linked to ethics in the grids indicates at least two different meanings, which would also make this axis a pivotal point among the grids. Indeed, the term ethics only rarely seems significant in the corpus and when it does appear, it is directly related to the methodological precaution issues in a post-positivist context. Hence, the realization that ethics is more about the researcher making sure that he or she completed all the administrative work falling under it, than a question of position with regards to the research. Therefore, it seems that the term reflexivity more often refers back to a notion close to ethics, which is then taking the research into account, its impacts, use, and acceptability among the actors and determining the co-constructed meaning.

Finally, our results show the need to differentiate “mixed methodologies” and “mixed epistemologies” as they do not have the same consequences. The call for mixed methodologies conducted rigorously is now more established (Creswell, 2013; Creswell & Plano Clark, 2010; Paterson & Pentland, 2008; Tashakkori & Teddlie, 2010). On the contrary, the reference to mixed epistemologies is more complex and only slightly identified. Yet, the compatibility of the paradigms is not only a question of philosophy but also refers back to the practical use of the grids as certain criteria are contradictory. Among these grids, some intersect with various paradigmatic and epistemological positions, and their authors propose a combination of the quality criteria that correspond both to positivist and constructivist approaches, with, for each grid, a conception that slightly differs from the combination to adopt. The fact that criteria linked to different epistemologies can coexist in the same grid, visibly unbeknownst to their author, reveals the existence of confusion between the terms used and questions the validity of their use. This confusion can stem from (a) the different meanings of a term (as seen for the terms interpretive and ethics); or (b) the authors’ action of successively borrowing from different grids to create a new one, without thinking of the consequences of this action; or (c) a lack of knowledge of the philosophical notions and frameworks of the terms used for the paradigms, methods, and theories; or finally (d) diversified knowledge of the notions based on the disciplinary origin of each group. All of these parameters contribute to creating a diverse and ultimately very nonconsensual image of the evaluation grids for qualitative research in the health sciences.

Conclusion

The results of our research explain in large part the difficulties in coming to a consensus on the quality criteria for qualitative research. The conception diversity, both inter- and intra-grid, prevents the coherence of the grids. Our results show that the grids examined are not representative of a discipline but rather of a position with regards to qualitative research and objectives inherent to each one. They thus translate the vision that the authors of the grids have of qualitative research, one that is partial at times, and almost always explicit, as per the diverse results obtained. Yet, the goal is now to provide researchers with explicit and adequate tools to assess their work, and to provide experts (and readers) with grids that allow them to evaluate the quality of qualitative work in various health research fields according to comparable criteria. Although standardizing the grids is not the objective, their acknowledgement requires clarifying the terms and comparing them to show the validity of their use and their scientific reliability. The identification of the paradigms underlying the quality criteria conducted in this document is a first step toward understanding the diversity of the qualitative research evaluation grids in the health sciences. As this article is only one part of a more extended research project conducted from 2011 to 2014, these findings therefore cover one aspect of our study. Another part of the research project included the participation of international experts in qualitative research in the health sciences. Each expert and their peers (56 persons) assessed the grids in their own fields. As a result, these experts considered 12 quality criteria consensual. The main outcomes have been published on a website aimed at the dissemination of this research: https://wp.unil.ch/qualityofqualitativeresearch. However, despite its insightful results, our research has two main limitations. First, this study was carried out on the basis of 133 quality grids published in the health sciences field, but we did not conduct a systematic meta-analysis on all the grids published in the field. This may have influenced the results with regard to the distribution of grids by disciplines and the number and diversity of the criteria. Second, we chose Denzin and Lincoln’s (1994/2011) classification, close to our own approach, and this choice resulted in our classification of the epistemological approaches underlying the grids that would have perhaps differed if based on other fundamental works on paradigms in qualitative health.

In conclusion, more work remains to be done, however, for as long as the concepts considered to be major by the authors of the grids are not clearly explained, the grids will only be of limited use in improving and establishing qualitative research in the health sciences.

Footnotes

The authors would like to thank Dr Valérie Capdevielle-Mougnibas from the University of Toulouse (France),for her valuable help in processing the data for Alceste®,and Dr Maria del Rio Carral from the University of Lausanne for her careful reading of this manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: The author(s) express their gratitude to the Swiss National Science Foundation for the financial support (Grant N° CR13Ι1_126983) provided to this study.

Author Biographies

Marie Santiago-Delefosse has been a full professor in health psychology at the University of Lausanne (Switzerland) since 2003. Her empirical research (fertility,HIV,chronic pain,etc.) has led her to propose an alternative theorization of the biopsychosocial model. Her work pays particular attention to the history of ideas and to epistemology in psychology;it is mostly qualitative and based on a historico-cultural and phenomenological approach. She is the author of more than 100 articles and several books.

Christine Bruchez has been a teaching assistant,then research coordinator,in health psychology at the University of Lausanne since January 2004. She has completed a DESS (advanced graduate diploma) in clinical psychology,and is currently a PhD candidate under the supervision of Prof. Marie Santiago-Delefosse. Her research interests in epistemology and methodology have led her toward the comparison of qualitative methods and the experimenting with new research tools in psychology.

Amaelle Gavin has been research coordinator in health psychology at the University of Lausanne since October 2012. She is currently training in clinical sexology and since February 2015,she occupies a position in the university as a PhD candidate under the supervision of Prof. Marie Santiago-Delefosse. Her different trainings allowed her to develop skills and interests in various domains,such as sexology,couple counseling,ethics,critical health psychology,and qualitative research among others.

Sarah Lilian Stephen worked as Junior SNSF (Swiss National Science Foundation) researcher in an SNSF-funded project directed by Prof. Santiago-Delefosse at the Faculty of Social and Political Sciences at the University of Lausanne. Since 2011,she is also a PhD student based at the Faculty of Business and Economics at the University of Lausanne. Her interests include quality of quantitative and qualitative research and interdisciplinary research methods,as well as paradigms underlying research.

Pauline Roux has been a Junior Lecturer at the University of Lausanne (Switzerland) since March 2015. She presented her thesis in social psychology in 2013 at the University of Lyon (France). Her research interests include patient-doctor relationship and communication,medically unexplained illnesses,and qualitative methodology and triangulation.

References

Aubert-Lotarski

Capdevielle-Mougnibas

(2002, March). Dialogue méthodologique autour de l’utilisation du logiciel Alceste en sciences humaines et sociales: “lisibilité” du corpus et interprétation des résultats [Methodological dialogue on the use of Alceste software in social and human sciences: “Readability” of the corpus and interpretation of results]. 6èmes Journées Internationales d’Analyse Textuelle [6th International Conference on Textual Analysis], Saint-Malo, France.

Beck

C. T.

(2009). Critiquing qualitative research. AORN Journal, 90, 543-554.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101.

Chamberlain

Stephens

Lyons

A. C.

(1997). Encompassing experience: Meanings and methods in health psychology. Psychology and Health, 12, 691-709.

Cicourel

(1964). Method and measurement in sociology. New York, NY: Free Press.

Cobb

A. K.

Hagemaster

J. N.

(1987). Ten criteria for evaluating qualitative research proposals. Journal of Nursing Education, 26, 138-143.

Cohen

D. J.

Crabtree

B. F.

(2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. The Annals of Family Medicine, 6, 331-339.

Creswell

J. W.

(2013). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE.

Creswell

J. W.

Plano Clark

V. L.

(2010). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE.

10.

Critical Appraisal Skills Programme. (2006). Qualitative Research Checklist. Oxford, UK: Public Health Resource Unit. Retrieved from http://media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf

11.

Curtin

Fossey

(2007). Appraising the trustworthiness of qualitative studies: Guidelines for occupational therapists. Australian Occupational Therapy Journal, 54, 88-94.

12.

Daly

Willis

Small

Green

Welch

Kealy

Hughes

(2007). A hierarchy of evidence for assessing qualitative health research. Journal of Clinical Epidemiology, 60, 43-49.

13.

Denzin

N. K.

Lincoln

Y. S.

(Eds.). (2011). The SAGE handbook of qualitative research. London, England: SAGE. (Original work published 1994)

14.

Devers

K. J.

(1999). How will we know “Good” qualitative research when we see it? Beginning the dialogue in health services research. Health Services Research, 34, 1153-1188.

15.

Dixon-Woods

Shaw

R. L.

Agarwal

Smith

J. A.

(2004). The problem of appraising qualitative research. Quality Safety Health Care, 13, 223-225.

16.

Elliott

Fischer

C. T.

Rennie

D. L.

(1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38, 215-229.

17.

Flick

(2006). Quality criteria in qualitative research. In Flick

(Ed.), An introduction to qualitative research (pp. 367-389). London, England: SAGE.

18.

Flick

(2007). Managing quality in qualitative research. Thousand Oaks, CA: SAGE.

19.

Fossey

Harvey

McDermott

Davidson

(2002). Understanding and evaluating qualitative research. Australian & New Zealand Journal of Psychiatry, 36, 717-732.

20.

Garfinkel

(1967). Studies in Ethnomethodology. Cambridge, MA: Polity Press.

21.

Garric

Capdevielle-Mougnibas

(2009). La variation comme principe d’exploration de corpus: Intérêts et limites de l’analyse lexicométrique interdisciplinaire pour l’étude du discours [Variation as a principle to explore the research corpus: Interests and limitations of the interdisciplinary lexicometric analysis to study discursive material]. Corpus, 8, 105-128.

22.

Green

Britten

(1998). Qualitative research and evidence based medicine. British Medical Journal, 316, 1230-1232.

23.

Greenhalgh

Taylor

(1997). How to read a paper: Papers that go beyond numbers (qualitative research). British Medical Journal, 315, 740-743.

24.

Hamberg

Johansson

Lindgren

Westman

(1994). Scientific rigour in qualitative research: Examples from a study of women’s health in family practice. Family Practice, 11, 176-181.

25.

Kuhn

(1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.

26.

Lincoln

Y. S.

Lynham

S. A.

Guba

E. G.

(2011). Paradigmatic controversies, contradictions, and emerging confluences, revisited. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The SAGE handbook of qualitative research (pp. 97-128). London, England: SAGE.

27.

Mays

Pope

(2000). Qualitative research in health care: Assessing quality in qualitative research. British Medical Journal, 320, 50-52.

28.

Meyrick

(2006). What is good qualitative research? A first step towards a comprehensive approach to judging rigour/quality. Journal of Health Psychology, 11, 799-808.

29.

Miles

M. B.

Huberman

A. M.

(1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: SAGE.

30.

Morse

J. M.

Barrett

Mayan

Olson

Spiers

(2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 1-19.

31.

Murray

(2000). Level of narrative analysis in health psychology. Journal of Health Psychology, 5, 331-342.

32.

Paterson

Pentland

(2008). The use of hermeneutics in a mixed methods design. The Qualitative Report, 13, 116-134. Retrieved from http://www.nova.edu/ssss/QR/QR13-1/vonzweck.pdf

33.

Popay

Rogers

Williams

(1998). Rationale and standards for the systematic review of qualitative literature in health services research. Qualitative Health Research, 8, 341-351.

34.

Pope

Mays

(2006). Qualitative research in health care. Oxford, UK: Blackwell.

35.

Porter

(2002). Critical realist ethnography. In May

(Ed.), Qualitative research in action (pp. 53-72). London, England: SAGE.

36.

Reinert

(1990). Alceste. Une méthodologie d’analyse des données textuelles et une application: Aurélia de Gérard de Nerval [Alceste. A methodology for analyzing textual data and its related application: Aurélia by Gérard de Nerval]. Bulletin de méthodologie sociologique/Bulletin of Sociological Methodology, 26, 24-54.

37.

Reinert

(1993). Les “mondes lexicaux” et leur “logique” à travers l’analyse statistique d’un corpus de récits de cauchemars [The “lexical worlds” and their “logics” through the statistical analysis of a corpus of stories of nightmares]. Langage et société/Language & Society, 60, 5-39.

38.

Robert

Bouillaguet

(2002). L’analyse de contenu [Content analysis]. Paris: Presses Universitaires de France.

39.

Russell

C. K.

Gregory

D. M.

(2003). Evaluation of qualitative research studies. Evidence-Based Nursing, 6, 36-40.

40.

Sandelowski

Barroso

(2007). Appraising reports of qualitative studies. In Sandelowski

Barroso

(Eds.), Handbook for synthesizing qualitative research (pp. 75-132). New York, NY: Springer.

41.

Santiago-Delefosse

(2007). Recherche en psychologie et turbulences paradigmatiques [Research in psychology and Paradigmatic turbulences]. Bulletin de Psychologie/Bulletin of Psychology, Hors série/Special issue, 97-102.

42.

Santiago-Delefosse

Bruchez

Gavin

Stephen

S. L.

(2015). Diversity of the Quality Criteria in Qualitative Research in the Health Sciences: Lessons from a Lexicometric Analysis Composed of 133 Guidelines. Forum: Qualitative Social Research, 16(2), Art.11.

43.

Santiago-Delefosse

Rouan

(Director). (2001). Les méthodes qualitatives en psychologie [Qualitative methods in psychology]. Paris: Dunod.

44.

Schneider

K. J.

(1998). Toward a science of the heart. Romanticism and the revival of psychology. American Psychologist, 53, 277-289.

45.

Schutz

(1962). Collected papers (2 Vols.). The Hague, The Netherlands: Martinus Nijhoff.

46.

Silverman

(1993). Interpreting qualitative data: Methods for analyzing talk, text and interaction. London, England: SAGE.

47.

Smith

Harré

Van Langenhove

(1995). Rethinking methods in psychology. London, England: SAGE.

48.

Stiles

W. B.

(1999). Evaluating qualitative research. Evidence-Based Mental Health, 2, 99-101.

49.

Tashakkori

Teddlie

(Eds.). (2010). SAGE handbook of mixed methods in social & behavioural research. Thousand Oaks, CA: SAGE.

50.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19, 349-357.

51.

Van den Hoonaard

Given

Lévy

McGinn

O’Neill

Palys

. (2004). Giving voice to the spectrum: Report of the social sciences and humanities Research Ethics Special Working Committee to the Interagency Advisory Panel on Research Ethics. Ottawa, Ontario, Canada: Interagency Advisory Panel and Secretariat on Research Ethics. Retrieved from http://www.sfu.ca/~palys/SSHWC-GivingVoice-2004.pdf

52.

Walsh

Downe

(2006). Appraising the quality of qualitative research. Midwifery, 22, 108-119.

53.

Yardley

(2008). Demonstrating validity in qualitative psychology. In Smith

J. A.

(Ed.), Qualitative psychology: A practical guide to research methods (pp. 235-251). London, England: SAGE.

Complexity of the Paradigms Present in Quality Criteria of Qualitative Research Grids

Abstract

Keywords

Introduction

Problem: A Shift in the Quarrels From the Last Century?

Method

Make-Up of the Corpus

Thematic Analysis of the Content and Theoretical and Methodological Foundations of the Analysis Categories

Presentation of Alceste® and Relevance of the Lexicometric Analysis Based on Our Hypothesis

Results

Lexicometric Analysis and Distribution of the Corpus Among Six Classes

Dendrogram of the Six Classes and Significant Terms

Analysis of the Classes According to the Significant Terms and Contextual Variables

Quadrants C and D (Classes 1, 5, and 4): “Mainly” constructivist approaches

Class 1 (psy, c1)

Class 5 (psy, c1, pp2)

Class 4 (med, nurs, pubh, c1)

Quadrants A and B (Classes 2, 6, and 3): “Mainly” post-positivist approaches

Class 2 (nurs, pp2, c2)

Class 6 (pubh, pp2)

Class 3 (meth, pp1, c3, c2)

Discussion

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

Author Biographies

References