Abstract
Keywords
Introduction
In recent years, with the rapid development of information technology and enhancement of people's health awareness, people's demand for online health information continues to expand. Online health issues have received much attention. Online health community (OHC), which is an open network platform for users which can provide Q&A consultation, information exchange, experience sharing and social support on health care issues 1 are prospering in many countries such as Doctor on Demand in America, Covidom in France and Patient Voice in Spain.2,3 China has been actively promoting the implementation of the Health China strategy. In April 2018, the General Office of the State Council of China issued the opinions on promoting the development of “Internet plus Medical Health,” 4 stressing the need to accelerate the development of “Internet plus Medical Health” and proposed various development measures. In November 2020, the Ministry of Industry and Information Technology and the National Health Commission of China jointly issued the Notice on further strengthening telemedicine network capacity building, 5 which proposed to improve the network standard system, increase network capacity, promote network applications and accelerate the application and development of “Internet plus” in the field of medical and health care. At the same time, with the impact of the coronavirus pandemic, residents’ demand for medical e-commerce and OHCs also shows explosive growth. By June 2022, the number of online medical users in China has reached 300 million, accounting for 28.5% of the total Internet users. Online health business revenue continues to grow and various OHCs are in fullswing. 6
Online Health Community plays a very important role in disseminating health information, popularizing medical knowledge, seeking medical treatment and improving users’ health literacy.7,8 However, the current service quality of OHCs is uneven. 9 Problems such as information service homogenization, charging and pricing confusion, lack of emotional care, inaccurate retrieval results, serious information pollution, difficult to distinguish true and false information, user’s privacy leakage have come one after another, which not only has a great impact on the user's search, identification and use of health information, but also seriously restricts the sustainable development of OHCs. Therefore, there is an urgent need to build a scientific and reasonable evaluation mechanism for the service quality of OHC. How to construct a reasonable evaluation indicator system? How to choose an appropriate evaluation method and comprehensively evaluate the service quality of OHC? These problems have become urgent practical problems for managers to solve.
The grounded theory emphasizes that theory comes from practice, which can abandon preconceived thinking and improve the scientific and credibility of research results.
10
It has good applicability in the identification of cause and effect, process interpretation and exploration of new things. Therefore, this study collected data by semistructured interviews and coded the obtained original materials based on the grounded theory. An indicator system for evaluating the service quality of OHCs was established. Then an empirical test was conducted through questionnaires. Six representative OHCs were selected and their service quality was evaluated using the entropy weight TOPSIS method. Specific suggestions are put forward from the aspects of content quality, function quality, interaction quality and emotional experience quality to improve the development of OHC. The study will help relevant operators improve their service quality on the one hand. On the other hand, it can help industry associations and government departments to regulate OHC service quality and guide the healthy and benign development of the industry. Our study plans to achieve the following objectives:
Construction of evaluation system of OHC service quality. Evaluating of OHC service quality based on entropy weight TOPSIS method. Targeted suggestions for improving the service quality of OHCs according to the research results.
Literature review
The rapid development of OHC has attracted extensive attention from scholars at home and abroad. Relevant researches such as community value, operation mechanism, service optimization and user behavior are constantly emerging. Among them, the service level of OHC fundamentally determines whether the goal of alleviating doctor–patient conflicts and reducing information asymmetry can be achieved. Therefore, the research on OHC service has become an important topic for scholars. Through literature review, the existing research relevant to OHC service can be divided into the following four aspects:
(
In summary, scholars at home and abroad have carried out many researches on OHC services and actively explored the evaluation of OHC service quality. However, there are still several deficiencies. First, there are relatively few studies on the comprehensive evaluation of OHC service quality. A systematic and convincing evaluation indicator system and evaluation model is still lacking. Second, the determination of service quality evaluation indicators focuses on the micro perspective, which is more subjective and lack of empirical testing. Third, the evaluation method relies too much on qualitative analysis. Analytic hierarchy process and other subjective methods are often used to calculate the indicator weight, which is difficult to effectively evaluate the quality of OHC service. To solve the above problems, this paper firstly constructs an evaluation indicator system of service quality of OHC based on the grounded theory. Through empirical research the indicator system is tested. Then the entropy weight TOPSIS method which combines qualitative and quantitative methods and objectively reflects the gap between evaluation schemes is used to evaluate the service quality of OHC. Finally, relevant suggestions are put forward in order to provide reference for the development of OHC.
Methods and results
Initial evaluation indicator system of OHC service quality based on grounded theory
The grounded theory, proposed by Galsser and Strauss in 1967, is a set of methods to generalize and construct theories from original data. It includes three steps: open coding, spindle coding and selective coding. It has been widely used in conceptual development and theoretical construction. The grounded theory emphasizes that theory comes from practice, which can abandon preconceived thinking, improve the scientific and credibility of research results, 10 overcome the problems of insufficient depth of traditional qualitative and quantitative research to a certain extent, and has good applicability in the identification of cause and effect, process interpretation, complex logical relationship chain and exploration of new things. This paper collected data through semi-structured interviews. Then grounded theory is adopted to carry out open coding of data and the relationship between categories is established through spindle coding. Finally, an evaluation indicator system through selective coding is constructed.
Sample selection and data collection
The survey objects selected in this study were mainly Internet users aged 18–40 years who used OHC regularly (at least 6 months to 1 year) and had sought health information in the OHC and communicated with physicians. These users have experience in using OHCs and good expression ability.
33
According to Holstein, the sample size of in-depth interviews should be controlled at 28–40 people to ensure the validity and focus of the study.
34
The interviewees were selected in the following ways: (1) Survey the understanding and usage of OHC among friends, relatives and classmates and select eligible users to participate in the interview. (2) Recruit eligible volunteers on social platforms such as Weibo, Zhihu and WeChat groups, and attract users to participate in interviews through material rewards (50 RMB). Then, through semistructured interview, focusing on the theme of “service quality of OHC,” the interviewees were interviewed in depth according to the outline as follows.
What OHC do you use most? How often is it used? What is your expectation of using OHC to enjoy medical services? Does the actual use of OHC reach the expected effect? What factors do you feel affect the use of OHC? When enjoying the medical services of OHC, what aspects do you attach more importance to? What do you think should be done to improve the quality of OHC service?
From June 2022 to July of 2022, we conducted in-depth interviews with 36 interviewees through a combination of offline and online (mainly one-to-one WeChat or telephone communication), of which 61.1% were online and 38.9% were offline. Each interview lasted between 15 and 30 min. The interview sample consisted of 44.4% males and 55.6% females. 33.3% of the interviewees had completed a master's degree or higher. 41.7% had completed a bachelor's degree. And the remaining 25% had completed junior college or less. In terms of monthly income, 11.1% earn less than 3000 RMB per month, 19.4% earn 3001–5000 RMB per month, 41.7% earn 5001–8000 RMB per month and 27.8% earn 8000 RMB or above. People who use OHC more than three years accounted for 36.1%; 1–3 years accounted for 41.7%; half a year to a year accounted for 22.2%. According to the 2021 China Internet Medical Content Industry Research Report,
33
as well as the research of Lu X, Yan Z, A.F. Audrain-Pontevia25,35–37 etc., OHCs’ users are characterized as young, female and highly educated. Therefore, this sample is representative to a certain extent and can be used to analyze the service quality of OHCs.
After the interview, the materials were transcribed and 36 original data with about 98,000 words were obtained after excluding the content significantly unrelated to this study. According to Glaser and Strauss, the theoretical model based on grounded theory should be tested for theoretical saturation, which means that if additional sampling continues, no new genera or related topics will emerge. 38 Quality (sufficient) is the key to saturation and the sample size of saturation test is generally selected according to actual research. 39 In this research, we select 31 samples randomly from the obtained data for analysis, and five samples (more than 10%) were reserved for saturation testing. The standardized text was imported into NVivo Plus12—an analysis software commonly used in qualitative research to prepare for further text analysis.
Data analysis and construction of indicator system
(1) Open coding
In the process of open coding of original text materials, the requirements of Glaser's rooted paradigm 38 were strictly followed. It was carefully checked whether each line contains events which may come from a word, a sentence or multiple lines of text. By coding and refining the 31 text materials sentence by sentence and line by line, more than 450 original statements and their corresponding concepts such as rich experience, convenient navigation and comprehensive content were finally obtained. Examples of the original statements and the first-order codes are shown in Table 1.
Examples of original materials and open coding.
The text materials were further analyzed by referring to a large amount of literature and communicating with users of OHCs. By comparing the original texts, concepts and initial categories, 39 first-order codes and 16 second-order codes were finally extracted as shown in Table 2.
Open coding results.
(2) Spindle coding
Spindle coding is to correlate the second-order codes formed in the previous step and summarize the concepts that express the same subject to form a theme through comparative analysis. By analyzing the connotation and internal relations of each second-order code, four themes of content quality, function quality, interaction quality and emotional experience quality are finally refined, as shown in Table 3.
Spindle coding results.
(3) Selective coding
Selective coding is the process of systematically analyzing concepts and codes, identifying core categories and exploring the connections between core categories and other categories to form theories.
In this study, on the basis of comparing the second-order codes and the themes with each other, we compared the summarized first-order codes and second-order codes, extracted four themes of content quality, functional quality, interaction quality and emotional experience quality, and built a theoretical model accordingly (see Figure 1). In this model, the influencing factors of OHC service quality have four first-level indicators of C1, C2, C3 and C4 and 16 second-level indicators from B1 to B16.

Construction of theoretical model.
(4) Theoretical saturation test
The theoretical saturation test is to check whether the model built by the researcher can reflect all the original materials and whether the recoding process will produce new concepts. In this process, the five reserved text materials were refined and coded sentence by sentence and line by line. Finally, no new concepts and categories were refined during the coding process, indicating that the model established in this study has a good theoretical saturation.
Empirical test of evaluation indicator system of OHC service quality
Descriptive statistical analysis and reliability test
To further verify the validity of the indicator system proposed above, we designed questionnaires. The five-point Likert-type response format that ranges from “strongly disagree” to “strongly agree” was used to measure items. The questionnaires were distributed nationwide in China to users who had experience of using OHC by on-site and Wenjuanxing platform, WeChat, QQ, email, etc. We collected questionnaires in the following ways: (1) Target subjects were recruited to fill out questionnaires on the Wenjuanxing platform. As a powerful Chinese online survey platform in China, 40 Wenjuanxing has a large number of users. (2) Solicit eligible OHC users on social platforms such as Weibo, Zhihu and WeChat groups, and then send questionnaires to them through WeChat, QQ, email and other means. (3) We also look for OHC users in hospitals and nearby communities and fill in the paper questionnaire on site. After all questionnaires were completed, the online subjects received reward through the Wenjuanxing platform and the on-site users were presented with small gifts. Finally, a total of 667 questionnaires were distributed and 641 questionnaires were returned, of which 628 questionnaires are valid. The effective rate of recovery is 94.2%. The sample information is shown in Table 4. Over half were 19–35 years old (69.4%), female (54.5%), and had at least a bachelor's degree (64%). OHCs’ users are characterized as young, female and highly educated.25,33,35–37 Therefore, this sample is representative to a certain extent and can be used to verify the validity of the indicator system.
Sample distribution.
Then SPSS 26.0 is used to test the reliability of the data. The results showed that the overall Cronbach's α coefficient of the scale was 0.882. The Cronbach's α coefficient of each dimension was greater than 0.8, which indicated that the questionnaire had good reliability.
Exploratory factor analysis
Exploratory factor analysis (EFA) and Confirmatory factor analysis (CFA) require to use different data for testing. 41 The 628 questionnaires collected were randomly divided into two parts. The former 314 questionnaires are for EFA and the latter 314 questionnaires are for validation factor analysis. The KMO value of the former data was 0.875. The approximate chi-square significance level of Bartlett's spherical test was 0.000 which is less than 0.01, indicating that this part of data is suitable for factor analysis.
Principal component analysis was used for EFA. Orthogonal rotation was performed based on the maximum variance method and a total of four principal components were extracted from the 16 independent variables, which explained 76.81% of the variance. The rotating component matrix is shown in Table 5, indicating that the indicators of this study have good construction validity. Therefore, it can be concluded that the four extracted factors are ideal in adequately extracting and explaining the information of the original variables.
Rotating component matrixα.
Confirmatory factor analysis
Confirmatory factor analysis of the latter data was performed by AMOS25.0. The results showed that CMIN/DF = 1.049 < 3, GFI = 0.963 > 0.9, AGFI = 0.949 > 0.9, NFI = 0.968 > 0.9, TLI = 0.998 > 0.9, IFI = 0.998 > 0.9, CFI = 0.998 > 0.9, RMR = 0.043 < 0.08 and RMSEA = 0.013 < 0.08. Therefore, the model exhibited a good fit.
From the results of CFA in Table 6, it can be seen that the standardized factor load of the 16 variables is greater than 0.6. CR of the four factors is greater than 0.7 and AVE is greater than 0.5, indicating that the variables have good convergent validity. Meanwhile, the square root of AVE of the 16 variables is greater than the normalized correlation coefficient. Therefore, the model had good discriminant validity. In summary, according to the results of CFA, the indicator system proposed above is reasonable and valid.
Results of CFA.
Evaluation of OHC service quality based on entropy weight TOPSIS method
To further verify the applicability and operability of the evaluation model, 6 relatively representative OHCs in China named Haodf (P1), WeDoctor (P2), Chunyu Doctor (P3), Clove Doctor (P4), Ping An Good Doctor (P5) and AliHealth (P6) were selected for service quality evaluation. Questionnaires were distributed to 24 senior experts and scholars in this field, of which 20 experts completed the scoring for the six communities. The return rate of the questionnaire was 83.3%. Then the entropy weight TOPSIS method was used for evaluation, and the steps were as follows.
Suppose there are m OHCs and n evaluation indicators.
Entropy values and weights of each indicator.
On the basis of the original matrix A, perform vector normalization for attribute and multiply it with the weight of each indicator
The weighted normalized matrix.
The positive ideal solution
The closeness rank of OHC.
Discussion
First-level indicator analysis
Based on the research results above, it can be seen that the indicator system is reasonable and effective and the evaluation method has strong applicability and operability.
Analyze the first-level indicators according to Table 7. It can be seen that the weights of indicators are C1 (0.3169), C4 (0.2893), C3 (0.2234) and C2 (0.1704) from large to small. That is content quality, emotional experience quality, interaction quality and functional quality, of which content quality has the highest weight.
Users often seek health information to prevent diseases and maintain health. High-quality health information can answer users’ health doubts and improve health literacy, while vague or incorrect information may lead to serious adverse consequences. Therefore, content quality is the most important factor of OHC service quality, which is consistent with the findings of Qian Minghui (2019). 18
When users are emotionally supported by companions, receive follow-up services after consultation and their privacy is protected, they can gain emotional comfort and satisfaction, which in turn motivates them to continue using OHC and are likely to recommend it to others.36,42,43 Thus, the quality of emotional experience is the second critical factor of the service quality of OHCs.
Frequent and effective communication and interaction between users and doctors, especially the timely and friendly responses from doctors to users’ questions and the variety of interaction modes are the basic and key services of the community. It is also the key aspect to increase users’ stickiness. Therefore, interaction quality has an important impact on the service quality of OHC.
Users’ behaviors of usage and participation in various services of OHCs need to be supported by community functions. The quality of functions has a significant impact on user satisfaction. 44 Easy-to-use, practical and humanized functions, as well as search intelligence are important factors that influence user experience. Although the functions of each community vary and have their own characteristics, due to the rapid development of present technology, most communities have functions that can meet the basic needs of users. So the weight of function quality is low.
Second-level indicator analysis
Analyze the indicators of the second level. It can be seen that B14 (Perceived cost reasonableness) is ranked first with a weight of 11.27%, followed by B3 (Content professionalism), B12 (Effectiveness of interactive content) and B1 (Content richness). The weight of B9 (Friendliness of interactive service), B5 (Ease of use) and B11 (Diversity of interactive mode) are the lowest.
Users traditionally pay for consultation and medical treatment offline according to hospital grade, doctor's qualification and title etc. While currently more and more people use online consultation due to time, distance, environmental safety and other factors. There are many online hospital and doctor resources of various grades and qualifications. The fees vary widely from a few yuan to hundreds of yuan. Due to the inability to conduct face-to-face communication and diagnosis, many users concern about the quality of online consultation because of reasons such as the trust of doctor's professional skills and ability etc. They are generally unwilling to pay the same cost for online consultation compared to offline consultation. So they attach great importance to the reasonableness of perceived cost, which affects their choice of hospitals and doctors and even whether they will adopt online consultation. Thus, the reasonableness of perceived cost has the greatest weight.
Indicators such as the content professionalism, effectiveness of interactive content and content richness are crucial to answering users’ health questions, solving health problems and improving health literacy. Therefore, they are the second more important factors affecting the service quality of OHC.
For the indicators of friendliness of interactive service, ease of use and diversity of interactive mode, the resources and services vary across communities, but the differences are not very pronounced. Physicians in OHCs have generally experienced the training of communities and the interactive services are generally friendly due to assessment of their service performance and personal characteristics. In terms of community function design, due to the current technological development and the community's concern for user experience, the ease and convenience of use are often considered. In addition, most communities have text, graphic and voice interaction together. Some communities also have video interaction. And most users have strong ability to understand and operate software, so the impact of ease of use and the diversity of interactive mode on service quality of OHCs are not very different. Therefore, these three indicators have the lowest weights.
Rank analysis of service quality of OHC
According to the closeness of the service quality in Table 9, the OHC P5 and P6 are in the first and second place with the closeness of 62.6 and 53.44, respectively, followed by P4, P1 and P3. P2 is in the last place. So from a practical perspective, this paper has the following practical implications:
1. In terms of content quality, P4 leads and performs well in terms of content richness, content professionalism, update timeliness and expression friendliness. This is followed by P5, P6 and P2, but P1 and P3 have lower closeness. As the saying goes, content is king. Content quality is an important means for OHCs to promote user growth, enhance user stickiness and reduce user churn, so P1 and P3 community should strengthen content quality management and improve the richness and professionalism of content. Specifically, improvements can be made in the following areas.
First of all, improve the comprehensiveness and professionalism of information. Medical resources should be included as much as possible. Disease introduction, prevention, discovery, clinical manifestation, treatment and related cases should be integrated and shown to users in the form of scientific articles or reports to meet their different health information needs. For the authority and professionalism of information, it can be ensured and improved by strengthening the control of information source and intensifying the supervision and audit of information.
Secondly, timely update the website information and pay attention to the reliability and accuracy of the information. Clarify the rules of information release and audit. If there is any problem, both the publisher and the auditor should take the responsibility to avoid false and wrong information to mislead the consumers, so that users can enjoy comprehensive and high-quality services.
Thirdly, plan the boards of OHC occupied by advertisements properly and control the quantity and quality of advertisements. Advertisements are helpful to improve users’ health level and literacy and expand their horizons. But too many advertisements in the community may cause users to resist and think the site is not reliable enough. So OHC should pay attention to the quality and quantity of advertisements placed.
2. In terms of emotional experience quality, the OHC P6 leads with the closeness of 76.57. While P3, P5, P1 and P2 are in the second to fifth positions, and P4 is at the bottom. Protection of user's privacy security, perceived cost reasonableness and perceived social support are helpful for improvement of user's emotional experience and have a significant impact on users’ continued use of OHCs. The OHC P4 needs to pay particular attention to the improvement of emotional experience quality. Specifically, this can be done in the following ways.
First, design a reasonable pricing mechanism to enhance user's perceived cost reasonableness. Some famous doctors and experts have higher pricing for their services. The community can take measures such as propagating the doctor's title, reputation, successful cases and patient comments to improve user's perceived cost reasonableness.
Second, enhance users’ sense of community belonging. By providing positive psychological relief, giving incentives and following up with patients after diagnosis, users will deeply appreciate the community's concern for their illness and health, which will make them generate the sense of belonging and enhance their willingness to continue using OHC.
Third, users’ privacy information should be fully protected. There is a risk of information leakage when users consulate and comment in OHC, which may pose a threat or harm to them. For this reason, the construction of information security should be strengthened to prevent the intrusion of lawless elements. Technical support should be increased. The privacy terms of OHC should be open and transparent to enhance users’ trust. The rules and regulations of OHC should be improved and the sense of social responsibility should be enhanced. A complaint function could be added in the community so that when users’ privacy is violated, they can timely defend rights through complaints.
3. In terms of interaction quality, the closeness of P1 ranks first followed by P2 and P5. P3, P6 and P4 rank lower. High quality interactive services in the OHC can enhance user stickiness. The diversity of interactive mode and the warmth and friendliness of doctors all will have a positive impact on users’ community engagement. P3, P6 and P4 should pay attention to improving the interaction quality of services. Specifically, it can be carried out in the following aspects.
Firstly, the online consultation channels for users should be expanded. On the basis of user's single text and picture inquiry, voice or video interaction, personal doctors and other channels can be explored to form a multi-level online communication channel and interaction system.
Secondly, the training of OHC doctors should be strengthened to improve the service quality. For example, a reasonable interval could be set for the timeliness of reply, and the reply should be targeted and friendly.
Thirdly, intelligent robots could be added to answer questions for some patients. Through the combination of intelligent robots and doctors’ services, the problem of unequal number of patients and doctors and the difficulty of timely answer by doctors can be solved. But the accuracy and effectiveness should be noticed.
Fourthly, the interaction function of OHCs should be improved. User recommendation could be well developed so that users in OHC can easily find others with similar health conditions or experiences. The health group can be established so that users can join the corresponding group according to their health conditions or diseases. Users will not only get more health information but also share their treatment experience to get comfort and support.
4. In terms of functional quality, the OHC P5 is in the first place with the closeness of 77.52 which is much higher than P4 and P1, while the closeness of the rest OHCs are all below 40.00. Especially P2 is in the last place. Since functional quality can directly affect users’ smooth usage and continuous engagement with OHC, 45 P2, P3 and P6 need to furtherly improve functional quality. Specifically, it can be carried out in the following ways.
Firstly, the technology advanced level of OHC should be improved. The quality of information systems should be improved according to user needs to ensure their experience. For example, the web server can be regularly optimized and communication technology can be timely updated.
Secondly, optimize the level of intelligent retrieval. On the one hand, the accuracy of information retrieval should be improved. On the other hand, duplicate and useless information should be filtered to reduce the impact of information overload on users.
Thirdly, the design and perception of software interface should be optimized. A simple and beautiful interface with reasonable perceptual design can improve users’ experience. Hence, it is important to improve the empathy perception of users and strengthen design innovation to enhance the micro-sensory experience of users. For example, personalized menu could be developed so that users can add frequently used functions to the main page according to their needs, which will provide convenience for subsequent use. A map of a website can be set up to improve the efficiency of community use, according to which users can directly and conveniently access health information without searching through layers of navigation.
Conclusions
This study provides a unique approach to solve the problem of service quality evaluation of OHCs. The evaluation system of OHC service quality was firstly constructed based on the grounded theory which includes 4 first-level indicators and 16 second-level indicators. Then the rationality and validity of the evaluation system were verified through empirical analysis. The results show that the weights of the first-level indicators from large to small are content quality, emotional experience quality, interaction quality and function quality. Among the second-level indicator weights, the top three are perceived cost reasonableness, content professionalism and effectiveness of interactive content. Then six OHCs were selected and evaluated by applying the entropy weight TOPSIS method and relevant conclusions and recommendations were given from quality of content, function, interaction and emotional experience etc. This study will provide theoretical guidance for community platform operators and relevant departments to design effective evaluation mechanism of OHC service quality, offering a reference for decisions and policymakers.
OHCs are in a phase of rapid growth with constant changes. With the development of information technology and related industries, the research horizon should also keep pace with the times and the following aspects could be further explored in future research.
Objective data such as Internet comments could be obtained. Through text mining and other methods, the evaluation indicator system could be further improved and the scientific rationality of the indicator system could be enhanced. Explore other more effective evaluation methods. The entropy weighted TOPSIS method used in this paper can reflect the gap between the evaluated OHC service quality and reveal the characteristics and problems of OHC services. But it is slightly complicated in calculating the distance from the standardized vector of each indicator to positive ideal solution and negative ideal solution. In the future, other evaluation methods such as artificial neural networks could be explored. The evaluation results of each method could be compared to improve the accuracy of evaluation. Explore the relationship between the OHC service quality and factors such as user loyalty and user churn. Moreover, the service quality evaluation system could also be combined with the annual report of enterprises to explore the relationship between OHC service quality and profitability.
