Abstract
Keywords
Introduction
Complaints submitted by patients and their families are increasingly recognized not merely as expressions of dissatisfaction, but as pivotal insights into care quality, communication breakdowns, and unmet needs within healthcare delivery systems. As unstructured textual data, these narratives can serve as vital inputs for service redesign and quality improvement strategies. However, their full potential remains untapped due to challenges in manual analysis. This study addresses that gap by exploring the application of Valence Aware Dictionary and Sentiment Reasoner (VADER) sentiment analysis to swiftly and reliably interpret patient complaint data, thereby advancing digital capabilities that support responsive,
As healthcare shifts toward patient-centered models, free-text data offers richer insights than traditional closed-ended surveys, helping capture nuanced patient experiences.1–3 To support this intent and develop patient-oriented healthcare services, systems and processes need to be improved to collect, organize, analyze, and quickly address patient and family complaints, and establish an accurate
To extract value from rich free-text data regarding patient experience, establishing trust by devising quicker case resolution loops is considered to be of key relevance.4–9 Regarding this intent, it is appropriate to say that the scale and diversity of free-text feedback make it difficult for human agents to swiftly extract and digest actionable information. Sentiment analysis, a pivotal technique in natural language processing (NLP) that involves identifying and extracting subjective information from textual data, can help automate this process by classifying textual feedback into emotional categories such as positive, neutral, or negative. 2 Beyond basic polarity detection, advanced sentiment models can also detect intensity and emotional cues, offering a nuanced understanding of customer grievances.5–9
VADER, a rule-based sentiment analysis tool, is considered to be a reliable solution in this domain. 2 It was initially designed to perform on social media text, but is increasingly being tested and valued as effective in other domains.10–13 VADER combines a robust lexicon with heuristic rules to capture contextual nuances in sentiment, making it easy to use and highly accurate.13–16
In a context (healthcare) where the use of sentiment analysis, namely in terms of patient experience management support, is considered to be still significantly underexplored, 13 this paper aims to explore the extent to which rule-based sentiment analysis tools such as VADER can contribute to more efficient, swift, and fair patient complaints resolution by quickly establishing an accurate patient's voice,1–3 in complex and critical sectors such as healthcare and large-sized organizations aiming to develop their technological capabilities such as public hospitals.
The study's dataset consists of the total number and content of patient complaints (
State of the Art
Over the last decade, sentiment analysis, conversational artificial intelligence (AI), and opinion mining have been emerging as critical areas of research, with significant applications across industries such as e-commerce, healthcare, social media, and customer service.2,13–16 During this timeframe, sentiment analysis methods, in concrete, have developed significantly, driven by advances in NLP and machine learning techniques. Current primary methodologies in the sentiment analysis field include10–16:
Closely intertwined with the aforementioned developments, sentiment analysis methods have become increasingly explored and applied in industries and domains where understanding customer and user sentiments can drive more effective managerial decision-making and improve overall experiences.4–9 For instance, businesses have been leveraging sentiment analysis to gain insights into customer feedback, enhancing product offerings and service quality. 13 In healthcare, sentiment analysis has been explored to aid in monitoring patient opinions and emotions, contributing to better patient care and satisfaction.9,16
As healthcare shifts from a
Despite the recent observed advancements in the healthcare space, sentiment analysis, particularly in patient experience management through free-text data mining, remains significantly underexplored. 13 However, it continues to gain traction as healthcare organizations strive to enhance and optimize the services provided to patients and their families, in particular through process digitalization and automation. The present study aims to contribute to this progress.
Methodology
Materials
The dataset that was used in the study consisted of the total number and content of patient complaints (
Methods
Data Preprocessing
Data preprocessing included the following procedures: (i) Translation: complaints data was translated from Portuguese to English using DeepL Pro; (ii) stopword removal was ensured using Natural Language Toolkit (NLTK)'s English stopword list; (iii) lemmatization was performed with spaCy's English model; and (iv) noise reduction was ensured by manual review which removed nontextual artifacts and informal language. Figure S1 in the Supplemental materials presents the dataset labeling per feedback type.
Data Analysis
Analysis was made by intertwining the usage of Orange text mining capabilities with its VADER sentiment analysis widget. In Orange, VADER is used to analyze English-language text, providing positive, negative, neutral, and compound sentiment scores. These scores help users understand the overall sentiment of the established

Text mining model (using Orange).
Composite scores were obtained by summing the valence scores of each word in the lexicon, including adjustments based on predefined rules, and normalizing the score to a range between −1 (extremely negative) and +1 (extremely positive). In the classification process, the following criteria was used to determine the sentiment (positive, negative, or neutral) of each comment
14
:
In terms of analytical results presentation, given the research goal of exploring the potential usefulness or appropriateness of using a sentiment analysis technique and tool as means to digest patient complaints in healthcare settings and support quick resolution actions and/or decisions by key stakeholders, it was decided to split analytical outputs per emotional tone—
Results
Key Findings
Overall, 3 main research results should be highlighted: (1) the existence of a significant number of neutral and positive scores in complaints free-text suggesting that, although labeled as complaints, the tone is not necessarily (strongly) emotional; (2) where an overall positive emotional tone was observed (compound score), text labeling was mixed (complaint, praise, and/or disputes/contestations); and (3) to be able to identify and differentiate these nuances in a timely manner, can be particularly useful for users and decision-makers to quickly trigger (or not) follow-up case resolution actions.
In a more qualitative perspective, complementing VADER's sentiment analysis output, it was possible to summarize the most frequent content feedback and link it with negative toned feedback (eg, misdiagnosis and incorrect surgery scheduling, postsurgery pain and neglect, poor communication/updates), neutral feedback (eg, constructive suggestions of service improvements), and positive feedback (eg, gratitude for the professionalism and care).
Figure 2 presents a sample of the output that was obtained with VADER that had a negative tone; Figure 3 presents an output sample where a neutral or a positive tone was observed. In Figure 3, that aggregates negative tone cases, most texts were labeled as complaint, with 1 case of dispute/contestation. Compound scores range from −0.9984 (extremely negative) to −0.0323 (slightly negative), indicating VADER's ability to differentiate the intensity of the sentiment expressed in a complaint. Most complaints have negative compound scores (see Figure S2 in the Supplemental materials), reflecting predominantly negative sentiments. The coexistence of neutral and positive scores in complaint texts suggests that, although they are classified as complaints, the tone is not necessarily (strongly) emotional.

VADER output data table with negative emotional tone (compound score).

VADER output data table with a neutral or positive emotional tone (compound score).
In Figure 3, despite overall positively toned answers, text labeling is mixed—complaint, praise, or disputes/contestations were observed. Compound scores range from 0 (neutral) to 0.9994 (extremely positive), indicating the intensity of the sentiment. The negative scores are generally low, with most values close to 0, indicating minimal negative sentiment. Neutral scores are high (ranging from 0.577 to 1), suggesting that many texts have a neutral tone. Some of these cases likely do not require the immediate direct intervention of a healthcare service member.
Comparative Analysis With BERT
For analytical control and output comparison purposes, BERT was used to double-check the accuracy of the results achieved with VADER. BERT grouped results are presented in Figure 4.

BERT output data table—grouped sentiment analysis.
Overall, as illustrated in the literature,
2
the overall orientation of VADER and BERT results can be considered to be similar, namely the existence of a significant number of neutral and positive scores in complaints free-text. VADER analysis, given its rule-based nature, is more straightforward in terms of legibility and call-to-action, but the chance of missing subtle sentiments or contextual-dependent details exists (eg, a sarcastic comment can classified as positive; overestimation of neutral or positive sentiments in ambiguous cases can be observed). BERT results are likely to be more accurate, especially for nuanced or sarcastic text. A sarcastic comment classified as neutral by VADER is likely to be classified as negative by BERT, thus providing approach to analytical accuracy.
18
A subset of 20 complaints was manually annotated by 2 independent reviewers as means to support model evaluation. Interrater agreement was high (Cohen's

VADER versus BERT model evaluation.
Benchmarks for VADER in general-purpose sentiment tasks often hover around 0.70 to 0.75 accuracy, depending on the dataset.
15
Research results (accuracy: 0.76,
Discussion
Practical Contributions of VADER
The research demonstrates VADER's potential to support patient experience management in digitally evolving healthcare settings, by transforming informal patient feedback into actionable insights, especially in healthcare settings with emerging digital capabilities. Its efficiency in processing free-text enables quick, meaningful overviews that support responsive decision-making.
In terms of research results, the research provides an illustration of how VADER sentiment analysis can support the extraction of meaningful insights from rich free-text data on patient experiences, offering a view of customer grievances. As illustrated by past studies,4–9 VADER human-centric design enhances sentiment understanding, making it a potentially valuable tool in healthcare-related analytics, going beyond basic polarity detection, by detecting the intensity of emotional cues. This makes it particularly useful for applications and domains where interpretability and speed are crucial, as is the case of patience experience management and the growing need to craft an accurate
Beyond reinforcing previous studies findings,2,3,13,15,16 it is worthwhile highlighting, in particular, three (3) concrete aspects provided by the present study that suggest the appropriateness of deepening and expanding the usage of VADER into the patience experience management space:
Evaluation and Benchmarking
To further validate these findings, a subset of complaints was manually annotated by 2 independent reviewers. Interrater agreement was high (Cohen's
Sentiment Complexity and Limitations
While VADER-based sentiment analysis may offer valuable, actionable insights to patient feedback handling, three (3) challenges and limitations need to be considered:
Additionally, the scope of this study is constrained by its dataset: 63 complaints from a single-surgery service within one (1) public hospital. While this focused sample allowed for detailed analysis and methodological control, it limits the generalizability of the findings. Patient sentiment may vary significantly across departments, institutions, and cultural contexts. Future research should expand to multisite, cross-departmental datasets to validate these insights and explore how sentiment patterns differ across healthcare environments. This broader approach would help ensure that sentiment-informed triage and analytics tools are scalable and adaptable to diverse healthcare systems.
Future Directions
Although direct metrics on downstream impact were not captured in this study, we propose a framework for sentiment-informed triaging. By flagging high-intensity negative feedback, healthcare teams could prioritize cases more effectively and allocate resources with greater precision. Future pilot testing will evaluate whether this approach improves resolution times, satisfaction scores, and overall responsiveness. As healthcare systems evolve toward digital-first models, sentiment analysis offers scalable ways to amplify the patient's voice. Future work will explore hybrid models that combine rule-based methods for initial sentiment scoring with deep learning techniques for refinement. Integrating syntactic and semantic features with learned embeddings may further improve the detection of subtle, context-dependent sentiment cues in patient narratives.
For instance, sentiment-informed triage can be operationalized through automated dashboards that flag complaints with compound scores below −0.8 for immediate review. A complaint expressing severe dissatisfaction with postsurgical care, for example, could be routed directly to a patient experience officer or clinical lead. This prioritization mechanism can be embedded into digital workflows, enabling real-time alerts and escalation protocols. Over time, such systems can be calibrated to incorporate additional metadata (such as department, complaint type, or historical resolution time) to refine triage accuracy and improve service responsiveness. This approach aligns with broader efforts to integrate AI-driven insights into healthcare operations and enhance the timeliness of patient-centered interventions.
Conclusion
The present study illustrates the effectiveness and potential usefulness of using VADER sentiment analysis to analyze patient complaints in healthcare settings. Three key study findings suggest the advantages of using VADER to support healthcare patient experience management: (a) the efficiency–effectiveness (accuracy) balance offered by VADER, by delivering quick and computationally efficient sentiment analysis without sacrificing accuracy; (b) support given to patient-centered care; and (c) support given to process optimization intents (via, eg, sentiment data-infused case prioritization). Future work will focus on integrating advanced machine learning models to enhance the accuracy and depth of sentiment analysis, as demonstrated by the comparison with the BERT sentiment analysis model. Additionally, expanding the dataset to include cross-sectional feedback from multiple departments and hospitals will provide a broader and more comprehensive perspective on patient sentiments across healthcare systems, helping to identify recurring issues and prioritize areas for improvement.
Supplemental Material
sj-jpg-1-jpx-10.1177_23743735251413861 - Supplemental material for Enhancing Public Healthcare Through VADER Sentiment Analysis: A Case Study on Patient Complaints
Supplemental material, sj-jpg-1-jpx-10.1177_23743735251413861 for Enhancing Public Healthcare Through VADER Sentiment Analysis: A Case Study on Patient Complaints by João Vasco Coelho and Liliana da Costa Barbosa in Journal of Patient Experience
Supplemental Material
sj-jpg-2-jpx-10.1177_23743735251413861 - Supplemental material for Enhancing Public Healthcare Through VADER Sentiment Analysis: A Case Study on Patient Complaints
Supplemental material, sj-jpg-2-jpx-10.1177_23743735251413861 for Enhancing Public Healthcare Through VADER Sentiment Analysis: A Case Study on Patient Complaints by João Vasco Coelho and Liliana da Costa Barbosa in Journal of Patient Experience
Footnotes
Acknowledgments
The authors would like to acknowledge the healthcare organization included in the study for providing access to the data used in this research. No funding is associated with the research.
Author Contributions
João Vasco Coelho: conceptualization and writing—review and editing. Liliana da Costa Barbosa: writing—original draft, project management, research operationalization, and data curation.
Informed Consent
Patient consent was obtained but is not needed for the study accomplishment.
Data Availability
The data that has been used is confidential.
Declaration of Competing Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval
This study was approved by the Ethics Committee of healthcare organization included in the study on October, 2024. Email written consent by the Ethics Committee was obtained prior to the development of the study. This research was conducted ethically in accordance with the World Medical Association Declaration of Helsinki.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
