Word embeddings have been successfully used in diverse tasks of Natural Language Processing, including sentiment analysis and emotion classification, even though these embeddings do not contain any emotional or sentimental information. This article proposes a method to refine pre-trained embeddings with emotional and sentimental content. To this end, a Multi-output Neural Network is proposed to learn emotions and sentiments simultaneously. The resulting embeddings are tested in emotion classification and sentiment analysis tasks, showing an improvement compared with the pre-trained vectors and other proposes in the state-of-the-art for fine-grained emotion classification.
AgrawalA.AnA.PapagelisM., Learning emotion-enriched word representations. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 950–961). Santa Fe, New Mexico, USA: Association for Computational Linguistics. (2018) https://aclanthology.org/C18-1081.
2.
BengioY.DucharmeR.VincentP.JanvinC., A neuralprobabilistic language model, J. Mach. Learn. Res. 3, (2003), 1137–1155.
FaruquiM.DodgeJ.JauharS.K.DyerC.HovyE.SmithRetrofittingN.A., Retrofitting word vectors to semantic lexicons. In Proceedings of 13 the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1606–1615). Denver, Colorado: Association for Computational Linguistics (2015). https://doi.org/10.3115/v1/N15-1184.
5.
HoY.WookeyS., The real-world-weight crossentropy loss function: Modeling the costs of mislabeling, IEEE Access8 (2020), 4806–4813. https://doi.org/10.1109/ACCESS.2019.2962617.
6.
KulkarniM.BhattacharyyaP., Retrofitting of pretrained emotion words with VAD-dimensions and the Plutchik emotions. In Proceedings of the 18th International Conference on Natural Language Processing (ICON) (pp. 529–536). National Institute of Technology Silchar, Silchar, India: NLP Association of India (NLPAI) (2021).
7.
LouviereJ.J.FlynnT.N.MarleyA.A.J., Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press. (2015), https://doi.org/10.1017/CBO9781107337855.
8.
MaasA.L.DalyR.E.PhamP.T.HuangD.NgA.Y.PottsC., Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 142–150). Portland, Oregon, USA: Association for Computational Linguistics. (2011) https://aclanthology.org/P11-1015.
9.
MikolovT.ChenK.CorradoG.S.DeanJ., Efficient estimation of word representations in vector space. In International Conference on Learning Representations (2013).
10.
MillerF.P.VandomeA.F.McBrewsterJ., net Movie Database. Alpha Press (2009).
11.
MohammadS., #emotional tweets. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics –Volume 1: Proceedings 14 of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012) (pp. 246–255). Montréal, Canada: Association for Computational Linguistics (2012). https://aclanthology.org/S12-1033.
12.
MohammadS., Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 174–184). Melbourne, Australia: Association for Computational Linguistics. (2018). https://doi.org/10.18653/v1/P18-1017
MrkšićN.Ó SéaghdhaD.ThomsonB.GašićM.Rojas-BarahonaL.M.SuP.-H.VandykeD.WenT.-H.YoungS. ,Counter-fitting word vectors to linguistic constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 142–148). San Diego, California: Association for Computational Linguistics (2016). https://doi.org/10.18653/v1/N16-1018.
15.
NaderalvojoudB.SezerE.A., Sentiment aware word embeddingsusing refinement and senti-contextualized learning approach, Neurocomputing405 (2020), 149–160. https://doi.org/10.1016/j.neucom.2020.03.094.
16.
NakovP.RosenthalS.KozarevaZ.StoyanovV.RitterA.,WilsonT., SemEval-2013 task 2: Sentiment analysis in Twitter. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013) (pp. 312–320). Atlanta, Georgia, USA: Association for Computational Linguistics (2013). https://aclanthology.org/S13-2052.15.
17.
PangB.LeeL., A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04) (pp. 271–278). Barcelona, Spain (2004). https://doi.org/10.3115/1218955.1218990.
PenningtonJ.SocherR.ManningC., GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1532–1543). Doha, Qatar: Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/D14-1162.
20.
PlutchikR., Chapter 1 –a general psychoevolutionary theoryof emotion. In R. Plutchik, and H. Kellerman (Eds.), Theories of Emotion (pp. 3–33). Academic Press (1980). https://doi.org/10.1016/B978-0-12-558701-3.50007-7.
RayA.MishraS.NunnaA.BhattacharyyaP., A multimodal corpus for emotion recognition in sarcasm. In Proceedings of the Thirteenth Language Resources and Evaluation Conference (pp. 6992–7003). Marseille, France: European Language Resources Association (2022). https://aclanthology.org/2022.lrec-1.756.
23.
RosenthalS.FarraN.NakovP., SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceedings of the 11th International Workshop on 16 Semantic Evaluation SemEval ’17. Vancouver, Canada: Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/S17-2088.
24.
SeyeditabariA.TabariN.GholizadeA.ZadroznyW., Emotional embeddings: Refining word embeddings to capture emotional content of words (2019). https://doi.org/10.48550/ARXIV.1906.00112.
25.
ShahS.ReddyS.BhattacharyyaP., Affective retrofitted word embeddings. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 550–561). Online only: Association for Computational Linguistics (2022). https://aclanthology.org/2022.aacl-main.42.
26.
ShaverP.R.SchwartzJ.C.KirsonD.O’ConnorC., Emotionknowledge: further exploration of a prototype approach, Journalof Personality and Social Psychology52(6) (1987), 1061–86
27.
SocherR.PerelyginA.WuJ.ChuangJ.ManningC.D.NgA.Y.PottsC., Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL (pp. 1631–1642). ACL (2013). https://aclanthology.org/D13-1170.
28.
StaianoJ.GueriniM., Depeche mood: a lexicon for emotion analysis from crowd annotated news. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp.427–433). Baltimore, Maryland: Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/P14-2070.
TangD.WeiF.YangN.ZhouM.LiuT.QinB., Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1555–1565). Baltimore, Maryland: Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/P14-1146.
31.
WallbottH.G.SchererK.R., How universal and specific isemotional experience? evidence from 27 countries on five continents, Social Science Information25 (1986), 763–795. https://doi.org/10.1177/053901886025004001.
32.
WarrinerA.KupermanV.BrysbaertM., Norms of valence,arousal, and dominance for 13,915 english lemmas, BehaviorResearch Methods45 (2013), https://doi.org/10.3758/s13428-012-0314-x.
33.
WilsonT.WiebeJ.HoffmannP., Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (pp. 347–354). Vancouver, British Columbia, Canada: Association for Computational Linguistics (2005). https://aclanthology.org/H05-1044.
34.
XuD.ShiY.TsangI.W.OngY.-S.GongC.ShenX., Survey onmulti-output learning, IEEE Transactions on Neural Networks and Learning Systems31 (2020), 2409–2429. https://doi.org/10.1109/TNNLS.2019.2945133.18.
