Abstract
Introduction
“How to measure the validity of a new research model?” It may be the first question of new researchers when they have finished their modeling process. This question is also reasonable in regard to the research feasibility in its implementation. In the information system (IS) research area, many literatures represent statistical examinations based on surveys in a pilot study to validate a research model. Gable (1994) who cited Lee (1991) described that this may refer to the phenomenon that the methods “are the only truly scientific one” (p. 2). Despite the fact that the researchers may have performed the qualitative assessment in part of their research, a few of them elucidated the assessment. For example, it is a common procedure in postgraduate studies that all scholars should perform a continuous academic activity (e.g., interview, consultation, discussion, or seminar) to develop their research models, but for validating the proposed model, the fact remains that the activities are rarely expressed rather than the statistical examination. Clearly, the performance of both quantitative and qualitative methods is not necessary in the validation, but its representation will provide complementary completeness, development, expansion, confirmation, compensation, and diversity (Venkatesh, Brown, & Bala, 2013) of the model validation. Moreover, Gable described that the use of both methods will give potential synergies through integrating the strengths and weaknesses of these two methods.
This article shows the performance of a focus group study (FGS) to understand the validation of the IS project success model (Subiyakto & Ahlan, 2014) and to explore its feasibility for the subsequent research implementation based on the perspective of the participants who had interests, skills and knowledge, and experiences in the IS research fields. In addition, to attain both above-mentioned objectives, the following research questions were proposed in this study:
The findings represent the validity of the proposed model in the context of its methodological aspects and its feasibility recommendations for the next stage of the research. Besides, this article contributes to the debates on the use of a variety of methods in a research model validation; it also demonstrates a practical implementation of the qualitative validation of an IS research model in terms of the use of the FGS techniques. In the following sections of this article, the authors elucidate the performance of the FGS from the selected previous studies and give an overview of the proposed research model; a methodological section includes the implementation of the four FGS types, the research process, and the used data analysis techniques. Furthermore, the article describes the eight formulated themes in the “Results and Analysis” section. In the “Discussion” section, these formulated themes are then discussed into four validation points to answer the research questions. Last, the authors draw the conclusion considering the learned lessons for further studies.
Literature Review
The use of the qualitative aspects has been the special interest in the IS and information technology (IT) studies for many years, especially to explore human, process, procedure, and their relationships with technology and system (Arshad, Ahlan, Ibrahim, & Norhafiza, 2013; O’Neill, 2012), but its methodological presentations still need to be reputed substantially for the use. One of the forms that deals with the above-mentioned issues is FGS. This mode has been popular in various disciplines within the social sciences for many years (Ho, 2006; Wilson, 2012). Wilson stated that FGS “allows researchers to collect a large amount of data from a substantial group of people in a relatively short amount of time,” particularly for exploring how people perceive, feel about, or view a certain domain (e.g., service, product, or topic) through interviews, consultations, discussions, or seminars (Finch & Lewis, 2003, p. 1).
Boateng (2012) who has cited Krueger (1988) described that the strengths of this technique related to its convenience, economic advantage, high face validity, and speedy results. Like another type of the qualitative techniques, FGS data are also understood as social enactments (Halkier, 2010). However, Wilson also suggested that researchers need to select the best people as their participants to ensure the data quality (Finch & Lewis, 2003; Frenk, Anderson, Chaves, & Martin, 2011; Homburg, Klarmann, Reimann, & Schilke, 2012; O’Neill, 2012; Yin, Figueiredo, & Mira da Silva, 2011). Moreover, O’Neill also warned about its problematic issues within sharing sensitive information, educating the participants, measuring the strength of the participant responds, generalizing data, and covering statistical projections.
In addition, unlike in the other disciplines of science, such as marketing and health, the use of a FGS in IT/IS studies has not been applauded and widely used yet, particularly, for validating a research model. Several researchers (Grösser & Schwaninger, 2012; Sargent, 2013) described that the validity aspect is an important issue in every discipline of science. It reflects how the system is being modeled quantitatively and qualitatively to establish trust and confidence of the model in the impossibility of an absolute acceptance. Specifically, in the qualitative inquiry, Guion, Diehl, and McDonald (2011) described that the validity refers to whether the research findings accurately reflect the situation and are supported by the evidence. This description was also indicated by many scholars (DeLone & McLean, 2003; Petter, DeLone, & McLean, 2008; Urbach & Müller, 2012; Urbach, Smolnik, & Riempp, 2009) who indirectly concluded the validity of a popular success model in their studies using a number of the previous literatures on the similar topic.
In short, as indicated by many scholars (Beringer, Jonas, & Kock, 2013; Creswell, 2013; Frenk et al., 2011; Homburg et al., 2012; Marshall & Rossman, 2010; O’Neill, 2012; Wilson, 2012) about the significant role of the research participants who had similar interests, skills, knowledge, and experiences, these key informants may also be reasonable to be involved in a FGS to ensure its findings’ validity.
The Proposed Model
The proposed model in this validation work was a research model (Figure 1) made to measure the success of an IS project (Subiyakto & Ahlan, 2014). The authors explained that the modeling was done based on comparison, adoption, adaptation, and combination of the five previous theories and models referred to the Belout and Gauvreau’s (2004) revelation that “most models explaining project success are based on theory rather than on empirical proof” (p. 2). The theories and models were Davis’s (1998) hierarchy plus input–process–output (HIPO) model, the project success theories (de Wit, 1988; Jugdev & Müller, 2005; Wateridge, 1998), DeLone and McLean’s (2003; D&M) IS success model, McLeod and MacDonell’s (2011) project classificatory framework, and the project environment theories (Howsawi, Eager, & Bagia, 2011; Lim & Mohamed, 1999).
The IS project success measurement model based on the IPO model.
The rationale of the modeling was the realization of the developed conceptual framework (Subiyakto & Ahlan, 2013). Subiyakto and Ahlan (2013) described that the framework was developed for understanding comprehensively what factors that affect the success of IT/IS projects. Figure 2 demonstrates the integration of the model within the framework. The proposed model consisted of nine variables, with 54 indicators and 36 relationships among the variables. Table 1 represents the references of the relationships. In brief, the authors concluded that “the model development of this study presented that the model was developed using the previous theories rather than on empirical proofs. Therefore, further researches can be conducted quantitatively and qualitatively to test validity of the variables (Subiyakto & Ahlan, 2014, p. 5610).”
The developed theoretical framework.
List of the Relationship References.
Method
FGS
In this study, the FGS was defined as a study to explore experiential data through capitalizing interactions (Asbury, 1995) of the participants using individual interviews, consultations, discussions, and seminars (Finch & Lewis, 2003) within the Information Systems, Internet and Organizations (ISIO) research group, Department of IS, International Islamic University Malaysia (IIUM). The participants were 16 of 20 enrolled members of the research group who had interests, skills, knowledge, and experiences in the IS research fields, including 9 doctoral students, 3 doctoral candidates, and 4 academicians (see Figure 3). They were selected because of their credible characteristics as the key informants (Asbury, 1995; Beringer et al., 2013; Finch & Lewis, 2003; Frenk et al., 2011; Homburg et al., 2012; O’Neill, 2012; Wilson, 2012). Accordingly, the authors believed that the involvement of these key informants would provide valid assessments in the context of the study. As regards the Boateng’s (2012) description, the responses of the participants were collected using the writing notes or tape recording as the material of the analysis stage.
Distribution of the participants.
Interview
This technique was performed in the preliminary stage of this study involving four participants. They were two doctoral students and two doctoral candidates. Following Lewis’s (2003) explanations, the researchers used informally in-depth interviews to investigate individual or organizational contexts of the participants to develop the research program. The interviews were conducted 2 to 3 times each lasting about an hour using the question list (see Table 2) in the preparatory stage.
List of the Interview Questions.
Consultation
This technique was carried out to get the personal guidance from an expert. Although the mode was focused on the preliminary study, it was performed regularly at least once in every 2 weeks for 30 min to 1 hr throughout the research period. The aims were to clarify and understand the motivations and decisions, and to explore the impacts and outcomes of the study (Lewis, 2003) through the face-to-face meeting, email, or telephone procedures in regard to the state of study as part of a postgraduate research.
Discussion
This technique was implemented 3 times before performing the seminar sessions for 1 to 2 hr for each meeting. The implementation was conducted by involving three to five doctoral students, including the moderator in the research group. In addition, the researchers also participated in the four discussions on similar topics by the other group members, who had discussed the other research models.
Seminar
The three seminars were conducted through collaboration between the researchers, the group administrator, the department, and the postgraduate office of the faculty. Therefore, the performance and procedure of these seminars followed formally the institutional guideline (see Table 3). The first seminar was performed openly for approximately 2 hr and involved 13 participants (8 doctoral students, a doctoral candidate, and 4 senior academicians) from the research group. The second one was also conducted openly for 2.5 hr and involved 10 participants (7 doctoral students, 2 doctoral candidates, and a senior academician) from the department. The last seminar was done based on an open-close-ended procedure throughout 2.5 hr with 11 participants (6 doctoral students, a doctoral candidate, and 4 academicians) from the faculty.
Guideline for Defense Proposal Master’s/PhD.
Research Process
The five stages of this empirical study were preliminary study, data collection, data analysis, interpretation, and report writing (see Figure 4).
The research process.
Stage 1: Preliminary FGS
This stage was conducted during January 2014 to provide the researchers with a well-rounded collection of appropriate information to develop the research program (Lewis, 2003; Rabiee, 2004; Turner, 2010). This developed research program guided the performances of the subsequent stages in this study. The interview and consultation techniques were useful and helpful in regard to reinforcing the practical understandings and motivations of the researchers.
Stage 2: Data collection
Based on the developed research program in the first stage, the researchers collected data using three FGS techniques, that is, consultation, discussion, and seminar. The focus of these techniques concerned how to explore the participant’s understanding toward the proposed method and its research implementation. Besides, the authors also studied a number of IT/IS project management literatures to support modeling. This stage was done serially and repeatedly with the data analysis and interpretation stages from February to April 2014. The reason of this cycle-repeated procedure was to explore maximally the knowledge, understandings, and experiences of the participants about the proposed model. The aim was to get the comprehensive data by covering overall ideas, suggestions, and critiques of the participants (Lewis, 2003). At the end of this stage, the faculty seminar session was performed formally in front of the postgraduate office committee. The results were the collections of the participant’s responses and the theoretical references of the research proposal and its proposed model, including the writing notes and transcription of the tape recordings.
Stage 3: Data analysis
In this stage, the researchers followed three iterative stages through data management, descriptive accounts, and explanatory accounts (Spencer, Ritchie, & O’Connor, 2003). In the data collection stage, this iterative process was also performed throughout February to April 2014, and the result was eight formulated themes (see Table 4) as the basis of the interpretation stage.
List of the Formulated Themes.
Stage 4: Interpretation
In this stage, the researchers used the interpretation approach in the context of self-understanding where they tried to formulate in a condensed form what the participants themselves meant and understood (Spencer et al., 2003). The researchers interpreted theoretically and generalized coherently the eight themes to answer the research questions (Corbin & Strauss, 2008; Creswell, 2013) in four validation points. The results were the coherent generalizations of the themes regarding the research questions. The iterative performance on this stage was done during February to April 2014.
Stage 5: Writing report
Although the writing process was done within the study period, the researchers could only be able to focus on final report writing after the proposed model was approved in the early May 2014.
Data Analysis Techniques
As suggested by Spencer et al. (2003), the researchers conducted data analysis using three techniques that were performed within an iterative process during the study. The tools used were Microsoft Office 2007, particularly Microsoft Office Word and Microsoft Office Excel. The three techniques were data management, descriptive accounts, and explanatory accounts. First, the researchers conducted the data management activities through reviewing, labeling, sorting, and summarizing the data to reduce the raw data. Second, the researchers identified, mapped, and classified the key dimensions, and developed the typologies to conceptualize the themes in line with the research questions. The eight themes were then determined at the end of this substage. Third, in the explanatory accounts, the researchers explained why the data took the forms that were found and presented. In short, the focus of this stage was to analyze the contents, contexts, and evaluations of the data concerning the answers to research questions.
Results and Analysis
The qualitative data analysis and its interpretation results revealed eight overarching themes (see Table 4) across the four FGS types and the literature study. These themes address both validity (the first five themes) and feasibility (the last three themes) issues in respect of the two research questions. A number of subthemes were then identified and discussed within the descriptions of the eight main themes.
The five of the eight formulated themes represented this research question, including the development, distinction, contribution, focus, and context of the proposed model. The descriptions below explain these five themes.
Theme 1: Development of the proposed model
Across the FGSs, this theme was one of the major themes that had been of interest to the participants throughout the study. The authors noted that most of the participants understood and accepted the modeling process. In the last consultation session before the faculty seminar, one of the senior academicians revealed, I see. The process is reasonable. Despite the use of previous models, that is, its adoption, combination, and adaptation; are the common procedures in every modeling work, the implementations in your modeling were clear, especially the developed theoretical framework (see Figure 2) which it was supported by four previous meta-analysis studies. It was the significant point. Unavoidably, it represents the theoretical development and I suggest you to use the drafted theoretical framework as the empirical validation.
Similar to the senior academician mentioned above, all of the three doctoral candidates who participated in all discussions indicated their understandings toward the model proposition as it was stated by a candidate in a discussion: Although, I was unclear regarding the explanation of the proposed model at the first time, but I think, the development process answered my six modeling guideline questions. As the guideline questions when I had been developing my model, I used these what, where, who, why, which, and how questions. The assessment points of the examiners in my proposal defense reflected these questions. In order to make your examiner deeply understand, please improve your paper explanations!
In addition, most of the students understood and some were even enlightened about the modeling process, as it was noted by the researcher in the second seminar at the department level: I am a bit confused with the explanation of the model development process, but this logical process is also interesting for me particularly for the consideration part in my model development. You formulated clearly the four stages, including the comparison, adoption, adaptation and combination stages in your modeling process.
Moreover, most of the academicians approved the modeling process in the third seminar, but one of the involved academicians criticized the process as it was represented in her seminar report: A lot of work has been done on the literature review. The student needs to organize better to reflect support for dimensions, relationships between dimensions and theories. A summary table of the theories and the literatures must be provided as the prior suggestion. The scope is too big and the research model is complex and confusing. Therefore, clarification is needed for the model components and dimensions used. Combining all theories may not be the right way. Streamline the research model and check the relationships of all the dimensions! Pilot Study will likely clarify some of them.
Theme 2: Distinctions of the model among the previous models
In all FGS techniques, the participants raised concerns regarding this theme, particularly the relationship of the proposed model with the prior ones. This focus may be seeing the novelty of the model, as it was stated by a doctoral student in one of the discussions: Based on my short review, adoption of the McLeod and McDonell’s model and its combination with the DeLone and McLean’s model are the distinctions of your model, especially for describing the overall process of an IS project as you described (see Figure 5).
The distinction of the processional and causal models between the proposed model and the previous five models.
The same responses were also represented by the doctoral candidates, as it was noted, by the authors, in the discussion before the third seminar: In my opinion, with the published model in a journal, it is one of the distinctive evidence of the model. I believe that the reviewers may have considered the novelties, but the important point is how to describe the distinction of your model in your seminar.
In addition, most of the academicians in the three seminars tried to explore the otherness of the model. The authors recorded from the question–answer session in the first seminar that was done by the research group: An academician: What is the relevance of your work to other researchers? The authors: Commonly, it is related to the five identified gaps of the information and communication technology (ICT) project management discipline which underlined the proposed study. Specifically, the proposed model was developed to expand the DeLone and McLean’s model in terms of the comprehensiveness and validity of this prior model as suggested by four meta-analysis studies proposed by Peter et al., Urbach et al. McLeod & MacDonell, and Urbach & Muller. An academician: What is the difference of this study with the Mr. Anonymous’s study? Is it a similar topic? The authors: The difference is in the topic area. Although, the context of our studies is similar, the Mr. Anonymous’s research is about IT implementation and this study is in the IS project measurement. We believe that both these studies are in the different area.
Theme 3: Contributions of the proposed model
Across the FGSs, the third focus of the participants was this theme related to the theoretical and practical aspects of the model proposition. As it was stated by a doctoral candidate in the first discussion group, Sir, I read your proposal. I know that the explanations was described indirectly in your proposal, but some points were not mentioned clearly at the significance section of your proposal manuscript. According to my experiences, of course, the description of the modeling must be explained clearly in the model development section, but it is my suggestion that you can also consider to include the point of your modeling process as one of the theoretical contributions. And, do not forget to justify the point based on your literature review done before.
Discussing the similar section, another doctoral candidate expressed a response based on his study experiences. As it was recorded by the authors, It is true; you also need to reformulate the point as the practical contribution by describing an inversion interrelationship from the model proposition into the problem statement of your study.
Furthermore, all academicians indicated their understanding toward this theme in the first seminar that was done approximately 1 week after the first discussion. This was represented by the interactions between the authors and an academician in the question–answer session of the first seminar: The academician: Where is the model improvements if they are compared with the prior models? Please, describe the theoretical contributions of your study! Authors: The two main points of the theoretical contribution are related to the revelation of the DeLone and McLean’s IS success model and the developmental concept of an IS project success measurement. First, the proposed model reveals the processional and causal model of the DeLone and McLean’s model in order to capture comprehensively its processional and causal model and to improve its findings validity. Second, the model was developed to cover IS project success model based on comparison, adoption, adaptation, and combinations of the five previous theories and models, including the information processing theory, the project success theories, the DeLone and McLean’s IS success model, the McLeod and MacDonell’s project classificatory framework, and the project environment model.
Theme 4: Focus of the model
Across the FGSs, most participants tried to present this theme in detailed revelation to assess the model validation. It was noted by the authors during one of the dialogue sessions between them and a doctoral candidate in the second discussion: The doctoral candidate: OK. The development and theoretical bases are clear, but my next question is “What is the model focus?” I questioned this point because I am still confused about the “thing” that will be measured by your model. Is it about IS measurement or its project? The authors: Thank you for your question, Sir. The answer of your question is, it is represented by the title of this study. The focus of the study is about the measurement of an IS project success, especially in its processional and causal aspects. The method to measure the project includes the input–process–output (IPO) dimensions based on the HIPO logic model by Davis. The doctoral candidate: I see. The authors: Specifically, we adopted project classificatory framework from MacLeod and McDonell’s project classificatory framework in the input dimension and the DeLone and McLean’s IS success model in the process and output dimensions to represent the processional aspect. In addition, we also adapted placement of the constructs based on the project success theories from de Wit, Wateridge, Jugdev and Müller, and the project environment models by Lim and Mohamed, and Howsawi et al. As it was displayed in the theoretical references (see Table 1).
In addition, most involved academicians were also concerned with this theme in the third seminar. As it was reported by an academician in her seminar report, A lot of work has been done in the literature, but they need to be organized to reflect the support of the model dimensions, relationships between the dimensions and theories.
Theme 5: Context of the model
Several participants, particularly the academicians, raised this theme across the consultation and seminar. As it was noted by the authors in a consultation session, The academician: According to the question about detachment of the contextual issue in a model, I believe that the use of the previous valid models in the modeling is one of the assessment criteria, especially its utilization which based on the suggestions of the previous meta-analysis studies. The authors: Yes, Sir. This is the table of the theoretical bases (see Table 1). For example, to adopt the DeLone and McLean’s IS success model; we conducted the modeling based on suggestions and indications from Peter et al., Urbach et al., and Urbach and Muller. Specifically, the use of the project classificatory framework proposed by McLeod and MacDonell as the input dimension art is based on indications of the McLeod and MacDonell’s article, and the Jugdev and Müller’s article for placing the process constructs. The academician: OK, your modeling represents this aspect.
Similar to the consultation mentioned above, several academicians also tried to explore indirectly this theme as it was presented in the question–answer session of the second seminar: The academician: Why do you choose an implementation of your research performance and use higher education sector in Indonesia as your focus? The authors: Thank you, Prof. Although, we believe that my proposed model will be able to be implemented in another sector or country, I chose to implement the model in Indonesia because the availability of the study resources which it will support the success of the study. It was emphasized by Blaxter et al., that the readiness of the researcher’s resources is one of the main CSFs of a research.
The three of the eight themes were interpreted and classified within this second research question, including the complexity and scope of the model, the implementation of the used research method, and the time consumption of the research performance. The following sections describe interpretations of these themes.
Theme 6: Complexity of the proposed research model
This theme was one of the highlighted themes for all of the participants in all FGSs. Most participants expressed, questioned, and discussed a range of experiences that they felt about this aspect in respect of the amount of the model dimensions, the variables, its indicators, and the relationships among the constructs that will be examined within the research implementation. For instance, many doctoral students expressed their feelings and opinions across FGSs as it was noted by the authors in a discussion session: The doctoral student: I do not know. It is just my feeling that you will find difficulties in the operationalization stage in formulating the model into the questionnaires in respect of the number of the variables and indicators. Have you prepared for this? Perhaps, the validation of the respondent’s answers will be questioned because the bored condition among participants in answering 54 questions provided in your questionnaire. The authors: Thank you for your opinion, Sir. I will consider this aspect. Another doctoral student: Yes, I agree with that. And also, do not forget that it also relates to the analysis and interpretation stages appropriate to 36 examinations of the relationships.
However, many doctoral candidates also reviewed this implementation aspect of the model across the FGSs, but several of them proposed suggestions based on their research experiences, as it was noted by the authors in the discussion: Of course, the complexity of a research model is one of the significant barriers for the researchers, but the simplicity also means unnecessary to assess validity of a model. In this context, I think, the problem is how to ensure the research implementation. My suggestions are, first, the researchers can adopt the questionnaires from the selected models and adapt the questionnaires appropriate to the current research. Second, the researchers must use the appropriate technique and strategy in the sampling stage, and the last, using the online method in the data collection stage will help the researchers solve the problem. I have applied these three points in my research, of course, the creativity is required for it.
Similarly, most academicians were also concerned with this theme across the FGSs, especially in the seminars, as it was written by a senior academician in his seminar report of the third seminar: Complexity is the main problem of this model. Too many components and dimensions are used. The theoretical framework has been described in the proposal, but clarifications are also needed to describe the framework clearly. My recommendation is conducting the pilot study to evaluate the model. Perhaps, the quantitative method will support the validation work.
Theme 7: Implementation of the research method
Across the FGSs, this theme has been of interest to the participants, especially the academicians in regard to the selections of the population and sampling method. Nevertheless, most of them questioned this issue from the beginning, but the issue then was understood by the academicians. As it was recorded by the authors in the question–answer session of the last seminar, The authors: The population of this research is the IS project stakeholders in the higher education institutions (HEIs) in Indonesia. Two purposive sampling levels will be performed to ensure the representativeness of the data. First, the authors will select a number of stakeholders who are in the 18 HEIs accredited institutions in “A” range of the national accreditation. Second, purposive sampling then will carry out based on the key informant techniques suggested by the previous literatures, for example, Marshall and Rossman. The second technique relies on the identification of a selected set of individuals, who are best arranged to provide information about a specific social setting based on their special qualifications. An academician: OK. According to your data collection method, is it possible with the four respondent types selected? Please, describe it more detail! The authors: Yes, Sir. Based on the guidelines of Marshall and Rossman, Suhaimi, and the Project Management Body of Knowledge or PMBOK, I planned to conduct a survey towards the business managers, the project managers, and the project team members, and also interview the business managers. Moreover, the quantitative method is the major which it will support by the qualitative one in this study.
In addition, another senior academician questioned this issue, particularly the feasibility of the sampling size in line with the data collection techniques planned. As it was noted by the authors in the question–answer session of the seminar, The academician: Design and method are specified. However, the amount of the targeted respondents among the stakeholders specified need more review, is that right? The authors: Thank you, Madam. This amount is the total respondents targeted. I believe that it will be reasonable with the use of the combination techniques in the data collection stage. As presented in the proposal, the data collection will use a combination of the two techniques, including the direct visiting into the institutions and some of the national association meetings, and utilization of the electronic questionnaires using email and web. The academician: I see. It is also reasonable.
Theme 8: Time consumption of the research performance
In all FGSs, most of the participants raised concerns with this theme. Consecutively, this theme was the end of the previous themes, especially related to the focus and complexity of the model, as it was stated by a doctoral student in an informal interview: Are you crazy, man? Your model is very complex. How much time would you like to finish your research? Although, we have similarities, particularly in the research methodology, you will need extra time to finish the study because of the number of the examination and its interpretation. But, I believe you for time management. Every time is based on schedule, right?
Similar responses were also expressed by the doctoral candidates, but most of these participants also proposed several suggestions according to this issue. A doctoral candidate’s expression was noted by the authors together with his motivations for the research implementation at the end of a discussion before the third seminar: Admittedly, the complexity will be the main concern at the first glance. It is according to the number of your constructs and its relationships. But, if you are ready for it, please defend your model! I believe that this modeling is a comprehensive work. Of course, being open-minded will be useful for its refinements. Yeah, it is a postgraduate study. So, welcome to the community!
Moreover, the third seminar report summary also mentioned this issue as one of the faculty seminar recommendations. It was reported by the head committee in the seminar report: The model looks too big; therefore, the student needs to focus more on the proposed model. He must target the system or project and reconsider the model in the height of standard definition. Therefore (the faculty seminar committee) allow the student to continue the research, but he must limit the model first.
Discussion
This article illustrates how influential arguments were used to validate the research model through an inductive-qualitative approach rather than continual hypothesis testing. Following the previous researchers (Morgan, 2010; Wilson, 2012), this study pointed out that the research questions were answered using the exploration of the developed themes. The performance of this qualitative validation was proven and especially valuable to describe and illuminate the context and condition under which the research implementation would have been conducted (Kaplan & Duchon, 1988), considering the modeling process, the theoretical bases, the methodological aspects, and the research resource availability. After the initial-experiential analysis of data, the following paragraphs represent the four above-mentioned validation points.
First, the results indicate that the modeling process was one of the contextual validation points in this study. Most of the participants in all FGSs were concerned with this modeling issue, and it was even illuminated by several participants, particularly the doctoral students. As it was described by Guion et al. (2011), the validity of the proposed model referred to whether the research findings reflect accurately the situation with the provided evidences. Following Guion et al. and Halkier (2010), the validity was assessed from the understanding level of the participants toward the first five themes. Specifically, why and how the previous models or theories were compared, adopted, adapted, and combined in the model development. In addition, the specification of the group and the participants represented accuracy and quality of the findings (Finch & Lewis, 2003; Frenk et al., 2011; Homburg et al., 2012; O’Neill, 2012; Yin et al., 2011).
Second, the study also represents the fact that the theoretical bases were the important validation point for the proposed research model as described by Belout and Gauvreau (2004) that “most of models were developed using the previous theories rather than on empirical proofs” (p. 2). This was presented with the result descriptions of the first five themes, that is, the development, contribution, focus, context, and complexity themes. Although this issue was questioned by some academicians in the consultation and seminar sessions, but as it was referred by Halkier (2010) about the social enactment issues, most of both doctoral students and doctoral candidates understood the bases of modeling.
Third, the results further illustrate that across the FGSs, most participants understood the methodological aspects of the proposed study (e.g., the research sampling and the data collection techniques) regarding its feasibility in the implementation. These are reasonable responses in line with Blaxter, Hughes, and Tight’s (2001) descriptions about contextual aspects of the research implementation feasibility. The model will be feasible to be implemented in the context of the use of the appropriate methods, techniques, and procedures.
Fourth, the results also explain that resource availabilities of the researchers were the focus of all participants in the overall FGS, especially to ensure the success of the subsequent research performance in the term of the postgraduate study. It was related to the researcher’s readiness (Blaxter et al., 2001). The formulated themes that covered this point were the context of the model, the implementation of the research method, and the time consumption of the research performed.
In short, the exploration of the previous theories or models in the modeling process, the involvement of the participants across the study, and the implementation of the methodological aspect were highlighted points of this study (Belout & Gauvreau, 2004; Blaxter et al., 2001; Guion et al., 2011; Halkier, 2010). Moreover, this study indicates that participants and their interactions in the nature had dynamic features. These features may change over time and interact differently according to how individual participants view the study contexts. The contexts may change for another case study.
Conclusion
This article describes how a qualitative validation method was performed in a case study of an IS project success model using FGS through interviews, consultation, discussion, and seminar. A total of 16 of 20 enrolled participants participated in at least one of the four sessions. In the data analysis stage, the authors formulated eight themes, that is, the development, distinction, contribution, focus, context, complexity of the model, the implementation of the research method, and the time consumption of the research performed. Furthermore, these eight themes were interpreted theoretically and generalized coherently into four validation points to answer the research questions. There was a modeling process, the theoretical basis, the methodological aspect, and the asset availability of the research.
The performance of this validation was proven especially valuable to describe and illuminate the context and condition under which the research performance will be conducted considering the four validation points. Thus, although this qualitative validation provided less explanation of variance in statistical terms than quantitative one, it yielded data of which process theories and richer explanations of how and why modeling processes occur can be developed.
We are aware that like the other validation method, this qualitative validation method applying FGS techniques also had limitations related to the context and subjectivity of the validation perspectives. The highlighted point of this study is the need for the concretion and objectivity of the validation, the similar reason that the combination of the perspectives can be more valuable in a particular study to provide completeness of the validation. Therefore, subsequent studies with different instances are called for. This validation study is in progress and further analyses and interpretations of the results are still possible to be carried out, particularly for the triangulation validation in a pilot study.