Sage Journals: Discover world-class research

Abstract

The details of facial expressions can effectively improve the realism of virtual animated characters. However, many expression generation algorithms suffer from problems such as inaccurate facial feature recognition. Therefore, based on the virtual reality technology, this study proposes a real-time simulation model for animated character facial expressions that integrates improved multi-task cascaded convolutional networks, and conducts experiments on the effectiveness and superiority of the model. The experiment outcomes showed that the lowest normalized average error for feature point recognition in the dataset was only 2.48, and the accuracy of facial action unit extraction was basically over 95%. The highest average in different datasets reached 97.68%, showing high intra-group correlation coefficient and low average absolute error. In addition, the rigid pose estimation algorithm that introduced head eye coordination had the smallest absolute deviation of the mean line of sight estimation, only 4.98°. Finally, the model simulated normal and exaggerated expressions in real-time, with an average frame rate of 54 frames and 38 frames, and a matching degree of 96.48% and 92.74%. Overall, this research can promote the advancement of virtual reality technology, which has significant importance for the metaverse, enhancing virtual interaction experience, and enhancing the realism of animated characters.

Keywords

metaverse animated characters facial expressions virtual reality real-time simulation

Get full access to this article

View all access options for this article.

References

Zhang

Chen

Zheng

. Facial expression retargeting from human to avatar made easy. IEEE Trans Vis Comput Graph 2020; 28(2): 1274–1287.

Zhu

. From games to education: research on immersive experience-based interactive design of children’s educational games. Adv Ind Eng Manag. 2023; 12(1).

Wang

Tao

. Self-supervised pose adaptation for cross-domain image animation. IEEE Trans Artif Intell 2020; 1(1): 34–46.

Gonzalez-Franco

Steed

Hoogendyk

, et al. Using facial animation to increase the enfacement illusion and avatar self-identification. IEEE Trans Vis Comput Graph 2020; 26(5): 2023–2029.

Pal

Roy

Shivakumara

, et al. Adapting a swin transformer for license plate number and text detection in drone images. Artificial Intelligence and Applications 2023; 1(3): 145–154.

Silvani

. Learning through watching: using animation movie to improve students’ writing ability. Journal of English Language Teaching and Linguistics 2020; 5(2): 233–247.

Cao

Zhang

Zhu

. Deep learning-based classification of the polar emotions of “moe”-style cartoon pictures. Tsinghua Sci Technol 2020; 26(3): 275–286.

Paier

Hilsmann

Eisert

. Interactive facial animation with deep neural networks. IET Comput Vis 2020; 14(6): 359–369.

Cao

Agrawal

De La Torre

, et al. Real-time 3D neural facial animation from binocular video. ACM Trans Graph 2021; 40(4): 1–17.

10.

Paier

Hilsmann

Eisert

. Example-based facial animation of virtual reality avatars using auto-regressive neural networks. IEEE Comput Graph Appl 2021; 41(4): 52–63.

11.

Berson

Soladié

Stoiber

. Intuitive facial animation editing based on a generative RNN framework. Comput Graph Forum 2020; 39(8): 241–251.

12.

Xia

Zheng

Wang

, et al. Local and global perception generative adversarial network for facial expression synthesis. IEEE Trans Circuits Syst Video Technol 2021; 32(3): 1443–1452.

13.

Zhou

Huang

. Flnet: landmark driven fetching and learning network for faithful talking facial animation synthesis. Proc AAAI Conf Artif Intell 2020; 34(7): 10861–10868.

14.

Zhao

Liu

Zhou

. Robust lightweight facial expression recognition network with label distribution training. Proc AAAI Conf Artif Intell 2021; 35(4): 3510–3519.

15.

Pise

Vadapalli

Sanders

. Facial emotion recognition using temporal relational network: an application to E-learning. Multimed Tools Appl 2022; 81(19): 26633–26653.

16.

Mohd

Javeed

Nowsheena

WMA

, et al. Sentiment analysis using lexico-semantic features. J Inf Sci 2024; 50(6): 1449–1470.

17.

Mohd

Wani

Khanday

, et al. Semantic-summarizer: semantics-based text summarizer for English language text. Software Impacts 2023; 18: 100582.

18.

Osler

Krueger

. Taking Watsuji online: betweenness and expression in online spaces. Cont Philos Rev 2022; 55(1): 77–99.

19.

Wang

Mayer

, et al. Animated pedagogical agents enhance learning outcomes and brain activity during learning. J Comput Assist Learn 2022; 38(3): 621–637.

20.

Agrawal

Mittal

. Using CNN for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. Vis Comput 2020; 36(2): 405–412.