Human motion prediction is a classic problem in computer vision and graphics, and the prediction of human motion diversity has a wide range of practical applications. To tackle this problem, this study proposes predicting the future motion diversity of the human body based on conditional denoising diffusion probabilistic models combined with the kinematics of human joints. First, the observed and predicted sequences were integrated into the same sample space using the mask mechanism, and Gaussian noise was gradually injected into the predicted sequence leveraging the cosine noise scheduler to destroy the sequence structure. Subsequently, the spatial-temporal feature extractor and channel enhancement module were used to form a denoiser to learn the temporal dynamic evolution of the sample and the potential correlation between the nodes in the diffusion process to complete the noise prediction and restore the sample information. The proposed method was verified on the Human3.6M and HumanEva-I datasets, and the experimental results show that the proposed method is competitive with previous methods in diversity prediction.
AhnH.MascaroE. V.LeeD. (2023). Can we use diffusion probabilistic models for 3D motion prediction? arXiv preprint arXiv:2302.14503. https://doi.org/10.48550/arXiv.2302.14503
2.
AksanE.KaufmannM.CaoP.HilligesO. (2021). A spatio-temporal transformer for 3D human motion prediction. In 2021 International conference on 3D vision (3DV) (pp. 565–574). IEEE. https://doi.org/10.1109/3DV53792.2021.00066
3.
BarqueroG.EscaleraS.PalmeroC. (2023). Belfusion: Latent diffusion for behavior-driven human motion prediction. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 2317–2327). https://doi.org/10.1109/ICCV51070.2023.00220
4.
BouaziziA.HolzbockA.KresselU.DietmayerK.BelagiannisV. (2022). Motionmixer: Mlp-based 3D human body pose forecasting. arXiv preprint arXiv:2207.00499. https://doi.org/10.48550/arXiv.2207.00499
5.
ButepageJ.BlackM. J.KragicD.KjellstromH. (2017). Deep representation learning for human motion prediction and classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6158–6166). https://doi.org/10.1109/CVPR.2017.173
6.
BütepageJ.KjellströmH.KragicD. (2018). Anticipating many futures: Online human motion prediction and generation for human-robot interaction. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 4563–4570). IEEE. https://doi.org/10.1109/ICRA.2018.8460651
7.
ChenL. H.ZhangJ.LiY.PangY.XiaX.LiuT. (2023). Humanmac: Masked motion completion for human motion prediction. arXiv preprint arXiv:2302.03665. https://doi.org/10.1109/ICCV51070.2023.00875
8.
ChiuH. K.AdeliE.WangB.HuangD. A.NieblesJ. C. (2019). Action-agnostic human pose forecasting. In 2019 IEEE winter conference on applications of computer vision (WACV) (pp. 1423–1432). IEEE. https://doi.org/10.48550/arxiv.1810.09676
9.
CreswellA.WhiteT.DumoulinV.ArulkumaranK.SenguptaB.BharathA. A. (2018). Generative adversarial networks: An overview. IEEE Signal Processing Magazine, 35(1), 53–65. https://doi.org/10.1109/MSP.2017.2765202
10.
CuiQ.SunH.YangF. (2020). Learning dynamic relationships for 3D human motion prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6519–6527).https://doi.org/10.1109/CVPR42600.2020.00655
11.
CurreliC.MuhleD.SarohaA.YeZ.MarinR.CremersD. (2025). Nonisotropic Gaussian diffusion for realistic 3D human motion prediction. arXiv preprint arXiv:2501.06035.https://doi.org/10.1109/CVPR52734.2025.00181
12.
DangL.NieY.LongC.ZhangQ.LiG. (2021). Msr-gcn: Multi-scale residual graph convolution networks for human motion prediction. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 11467–11476). https://doi.org/10.48550/arXiv.2108.07152
13.
DangL.NieY.LongC.ZhangQ.LiG. (2022). Diverse human motion prediction via gumbel-softmax sampling from an auxiliary space. In: Proceedings of the 30th ACM international conference on multimedia (pp. 5162–5171). https://doi.org/10.1145/3503161.3547956
14.
DhariwalP.NicholA. (2021). Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34, 8780–8794. https://doi.org/10.48550/arXiv.2105.05233
15.
GaoX.YangY.WuY.DuS.QiG. J. (2024). Multi-condition latent diffusion network for scene-aware neural human motion prediction. IEEE Transactions on Image Processing, 33(2024), 3907–3920. https://doi.org/10.1109/TIP.2024.3414935
16.
HernandezA.GallJ.Moreno-NoguerF. (2019). Human motion prediction via spatio-temporal inpainting. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 7134–7143). https://doi.org/10.1109/ICCV.2019.00723
17.
HoJ.JainA.AbbeelP. (2020). Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33, 6840–6851. https://doi.org/10.48550/arXiv.2006.11239
18.
IonescuC.PapavaD.OlaruV.SminchisescuC. (2013). Human3. 6m: Large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7), 1325–1339. https://doi.org/10.1109/TPAMI.2013.248
19.
KimS.KimH.YoonS. (2022). Guided-tts 2: A diffusion model for high-quality adaptive text-to-speech with untranscribed data. arXiv preprint arXiv:2205.15370. https://doi.org/10.48550/arXiv.2205.15370
20.
KunduJ. N.GorM.BabuR. V. (2019). Bihmp-gan: Bidirectional 3D human motion prediction gan. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, pp. 8553–8560). https://doi.org/10.1609/aaai.v33i01.33018553
LiuY.ZhangH.XuD.HeK. (2022a). Graph transformer network with temporal kernel attention for skeleton-based action recognition. Knowledge-Based Systems, 240, 108146. https://doi.org/10.1016/j.knosys.2022.108146
23.
LiuZ.WuS.JinS.JiS.LiuQ.LuS.ChengL. (2022b). Investigating pose representations and motion contexts modeling for 3D motion prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 681–697. https://doi.org/10.1109/TPAMI.2021.3139918
24.
LugmayrA.DanelljanM.RomeroA.YuF.TimofteR.Van GoolL. (2022). Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11461–11471).https://doi.org/10.1109/CVPR52688.2022.01117
25.
MaT.NieY.LongC.ZhangQ.LiG. (2022). Progressively generating better initial guesses towards next stages for high-quality human motion prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6437–6446). https://doi.org/10.1109/CVPR52688.2022.00633
26.
MangalamK.AdeliE.LeeK. H.GaidonA.NieblesJ. C. (2020). Disentangling human dynamics for pedestrian locomotion forecasting with noisy supervision. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (pp. 2784–2793). https://doi.org/10.1109/WACV45572.2020.9093350
27.
MaoW.LiuM.SalzmannM. (2021). Generating smooth pose sequences for diverse human motion prediction. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 13309–13318).https://doi.org/10.1109/ICCV48922.2021.01306
28.
MaoW.LiuM.SalzmannM.LiH. (2019). Learning trajectory dependencies for human motion prediction. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 9489–9497).https://doi.org/10.1109/ICCV.2019.00958
29.
NicholA. Q.DhariwalP. (2021). Improved denoising diffusion probabilistic models. In International conference on machine learning (pp. 8162–8171). PMLR. https://doi.org/10.48550/arXiv.2102.09672
30.
PadenB.čápM.YongS. Z.YershovD.FrazzoliE. (2016). A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Transactions on Intelligent Vehicles, 1(1), 33–55. https://doi.org/10.1109/TIV.2016.2578706
31.
SahariaC.ChanW.SaxenaS.LiL.WhangJ.DentonE. L.GhasemipourK.Gontijo LopesR.Karagol AyanB.SalimansT.HoJ. (2022). Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35, 36479–36494. https://doi.org/10.48550/arXiv.2205.11487
32.
SigalL.BalanA. O.BlackM. J. (2010). Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision, 87(1-2), 4–27. https://doi.org/10.1007/s11263-009-0273-6
33.
SunJ.ChowdharyG. (2024). Comusion: Towards consistent stochastic human motion prediction via motion diffusion. In European conference on computer vision (pp. 18–36). Springer. https://doi.org/10.1007/978-3-031-73036-8_2
34.
TashiroY.SongJ.SongY.ErmonS. (2021). Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34, 24804–24816. https://doi.org/10.48550/arXiv.2107.03502
35.
TianS.ZhengM.LiangX. (2024). Transfusion: A practical and effective transformer-based diffusion model for 3D human motion prediction. IEEE Robotics and Automation Letters, 9(7), 6232–6239. https://doi.org/10.1109/LRA.2024.3401116
36.
VaswaniA.ShazeerN.ParmarN.UszkoreitJ.JonesL.GomezA. N.KaiserŁ.PolosukhinI. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 6000–6010. https://doi.org/10.5555/3295222.3295349
37.
WeiD.SunH.LiB.LuJ.LiW.SunX.HuS. (2023). Human joint kinematics diffusion-refinement for stochastic motion prediction. In Proceedings of the AAAI conference on artificial intelligence (Vol. 37, pp. 6110–6118). https://doi.org/10.1609/aaai.v37i5.25754
XuS.WangY. X.GuiL. Y. (2022). Diverse human motion prediction guided by multi-level spatial-temporal anchors. In European conference on computer vision (pp. 251–269). Springer.https://doi.org/10.1007/978-3-031-20047-2_15
40.
YanX.RastogiA.VillegasR.SunkavalliK.ShechtmanE.HadapS.YumerE.LeeH. (2018). Mt-vae: Learning motion transformations to generate multimodal human dynamics. In Proceedings of the European conference on computer vision (ECCV) (pp. 276–293). https://doi.org/10.1007/978-3-030-01228-1_17
41.
YuH.HouY.PeiW.OngY. S.ZhangQ. (2024). Divdiff: A conditional diffusion model for diverse human motion prediction. IEEE Transactions on Multimedia, vol. 27, 1848–1859. https://doi.org/10.1109/TMM.2024.3521821
42.
YuanY.KitaniK. (2020). Dlow: Diversifying latent flows for diverse human motion prediction. In Computer Vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16 (pp. 346–364). Springer. https://doi.org/10.1007/978-3-030-58545-7_20
43.
ZhongC.HuL.ZhangZ.YeY.XiaS. (2022). Spatio-temporal gating-adjacency gcn for human motion prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6447–6456). https://doi.org/ 10.1109/CVPR52688.2022.00634
