Wearable sensors, such as smartwatches, often face challenges in providing accurate readings due to variations in sensor quality and placement irregularities. Furthermore, deep convolutional neural networks (CNNs) can be resource-intensive, making them less suitable for real-time applications in wearable technology. This study introduces a novel approach utilizing Edge Tensor Processing Units (EdgeTPUs) combined with Field-Programmable Gate Arrays (FPGAs) for activity classification in athletes, specifically designed for resource-constrained environments. EdgeTPU models are optimized for low memory usage and high computational efficiency, making them ideal for microcontrollers used in wearable devices. To address model size and complexity, we employ quantization techniques and a lightweight neural network design. Experiments using the UCI-HAR, WISDM, and WEAR datasets show that our approach achieves a classification accuracy of 98.97%. The FPGA implementation offers high throughput and low latency, critical for real-time processing in wearable sensors. Additionally, we present three innovative approaches for complex loop optimization to enhance convolution efficiency. With a usage rate of 24.6%, the EdgeTPU model implemented on a Virtex-7 FPGA demonstrates effective hardware utilization. Our results indicate that the integration of FPGA and EdgeTPU technologies enables efficient real-time monitoring of athletic activities in wearable devices with limited resources.
BhavyaPVermaSMahajanP, et al. An insight into the wearable technologies based on novel hybrid piezoelectric-triboelectric nanogenerators. Energy Technol2023; 11: 2300224.
2.
GhanbariKRoshaniMGoicoecheaHC, et al. Developing an elegant and integrated electrochemical-theoretical approach for detection of DNA damage induced by 4-nonylphenol. Heliyon2019; 5(10): e02755.
3.
FaruquiNYousufMAWhaiduzzamanM, et al. LungNet: a hybrid deep-CNN model for lung cancer diagnosis using CT and wearable sensor-based medical IoT data. Comput Biol Med2021; 139: 104961.
4.
HaotongQRuihaoGXianglongL, et al. Binary neural networks: a survey. Pattern Recognit2020; 105: 107281.
5.
Pham-QuocCNguyenXQThinhTN.Towards an FPGA-targeted hardware/software co-design framework for CNN-based edge computing. Mobile Netw Appl2022; 27: 2024–2035.
6.
ZhengweiF.Computer aided fencing sports wearable equipment based on FPGA microprocessor and sensors. Microprocess and Microsyst2021; 81: 103693.
7.
LiangSYinSLiuL, et al. FP-BNN: binarized neural network on FPGA. Neurocomputing2018; 275: 1072–1086.
8.
HuangWWuHChenQ, et al. FPGA-based high-throughput CNN hardware accelerator with high computing resource utilization ratio. IEEE Trans Neural Netw Learn Syst2022; 33: 4069–4083.
9.
HuAYuGWangQ, et al. Efficient hardware accelerator design of non-linear optimization correlative scan matching algorithm in 2D LiDAR SLAM for mobile robots. Sensors2022; 22: 1–18.
10.
LiGZhangMZhangQ, et al. Efficient binary 3D convolutional neural network and hardware accelerator. J Real-Time Image Proc2022; 19: 61–71.
11.
DiouriO.Comparison study of hardware architectures performance between FPGA and DSP processors for implementing digital signal processing algorithms: application of FIR digital filter. Results Eng2022; 16: 1–18.
12.
ZhaoRSongWZhangW, et al. Accelerating binarized convolutional neural networks with software-programmable FPGAs. In: Proceedings of the 2017 ACM/SIGDA international symposium on field-programmable gate arrays, Monterey CA, USA, 22–24 February 2017, pp.15–24. New York, NY: ACM.
13.
YonekawaHNakaharaH. On-chip memory based binarized convolutional deep neural network applying batch normalization free technique on an FPGA. In: Proceedings of the 2017 IEEE international parallel and distributed processing symposium workshops (IPDPSW), Lake Buena Vista, FL, USA, 29 May–2 June 2017, pp.98–105. New York, NY: IEEE.
14.
NakaharaHQueZLukW. High-throughput convolutional neural network on an FPGA by customized JPEG compression. In: Proceedings of the 2020 IEEE 28th annual international symposium on field-programmable custom computing machines (FCCM), Fayetteville, AR, USA, 3–6 May 2020, pp.1–9. New York, NY: IEEE.
15.
WangPLiFLiG, et al. Extremely sparse networks via binary augmented pruning for fast image classification. IEEE Trans Neural Netw Learn Syst2023; 34: 4167–4180.
16.
NuruzzamanFMohammadAYWhaiduzzamanMD, et al. LungNet: a hybrid deep-CNN model for lung cancer diagnosis using CT and wearable sensor-based medical IoT data. Comput Biol Med2021; 139: 104961.
17.
JaegyunPWon-SeonLDae-WonK, et al. GTSNet: flexible architecture under budget constraint for real-time human activity recognition from wearable sensor. Eng Appl Artif Intell2023; 124: 1–15.
18.
Ganesh BabuRChellaswamyCGeethaTS, et al. Fibre optic sensor based multi-gas detection using optimized convolutional neural network. J Mod Opt2022; 69: 403–417.
19.
IandolaFNHanSMoskewiczMW, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv preprint arXiv:1602.07360. 2016.
20.
ZhangXZhouXLinM, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018, pp.6848–6856. New York, NY: IEEE.
21.
SandlerMHowardAZhuM, et al. Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018, pp.4510–4520. New York, NY: IEEE.
22.
YaonanCMengdaLXiaoyuG, et al. Research on multi-signal milling tool wear prediction method based on GAF-ResNext. Robot Comput Integr Manuf2024; 85: 1–17.
23.
WuBWanAYueX, et al. Shift: a zero flop, zero parameter alternative to spatial convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018, pp.9127–9135. New York, NY: IEEE.
24.
Jin-YuZAn-TaiYWeiJ, et al. FPGA-based acceleration for binary neural networks in edge computing. J Electr Sci Technol2023; 21(2): 1–16.
25.
FuCZhuS-LSuH, et al. Towards fast and energy-efficient binarized neural network inference on FPGA. In: Proceedings of ACM/SIGDA international symposium on field-programmable gate arrays, Seaside, USA, 24–26 February 2019, pp.306–317. New York, NY: ACM.
26.
XianZ-KLiH-GLiY-L.Weight isolation-based binarized neural networks accelerator. In: Proceedings of IEEE 2020 international symposium on circuits and systems (ISCAS), Seville, Spain, 12–14 October 2020, pp.1–4. New York, NY: IEEE.
27.
WangP-SHeX-YLiG, et al. Sparsity-inducing binarized neural networks. In: Proceedings of 34th AAAI conference on artificial intelligence, New York, 7–12 February 2020, pp.12192–12199. New York: Association for the Advancement of Artificial Intelligence.
28.
VillalbaJHormigoJ.High-radix formats for enhancing floating-point FPGA implementations. Circ Syst Signal Process2022; 41: 1683–1703.
29.
SánchezCSchneiderGAhrendtW, et al. A survey of challenges for runtime verification from advanced application domains (beyond software). Form Methods Syst Des2019; 54: 279–335.
30.
JiaKRinardM. Efficient exact verification of binarized neural networks. In: Proceedings of 34th international conference on neural information processing systems, Vancouver, Canada, 6–12 December 2020, vol.151, pp.1–14. New York, NY: ACM.
31.
Rodríguez CorralJMCivit-MasotJLuna-PerejónF, et al. Energy efficiency in edge TPU vs. Embedded GPU for computer-aided medicalimaging segmentation and classification. Eng Appl Artif Intell2024; 127: 107298.
32.
Civit-MasotJLuna-PerejónFMaría Rodríguez CorralJ, et al. A study on the use of Edge TPUs for eye fundus image segmentation. Eng Appl Artif Intell2021; 104: 104384.
33.
CaiHGanCWangT, et al. Once-for-all: train one network and specialize it for efficient deployment. arXiv preprint arXiv:1908.09791. 2019.
34.
ChellaswamyCMuthu ManjulaMRamasubramanianB, et al. FPGA-based remote target classification in hyperspectral imaging using multi-graph neural network. Microprocess Microsyst2024; 105: 105008.
35.
ZhangCLiPSunG, et al. Optimizing FPGA-based accelerator design for deep convolutional neural networks. In: Proceedings ACM/SIGDA international symposium on field-programmable gate arrays (FPGA), Monterey, USA, 22–24 February 2015, pp.161–170. New York, NY: ACM.
KwapiszJRWeissGMMooreS. A.Activity Recognition using Cell Phone Accelerometers. Proceedings of the Fourth International Workshop on Knowledge Discovery from Sensor Data (at KDD-10), Washington DC. 2010, https://www.cis.fordham.edu/wisdm/dataset.php (accessed 15 June 2024).
39.
BockMKuehneHVan LaerhovenK, et al. WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2024; 8(4): Article 175. https://mariusbock.github.io/wear/?utm_source=chatgpt.com (accessed 21 December 2024).
CoutinhoFDLSilvaHSGeorgievaP, et al. A flexible CNN architecture for real-time FPGA implementation. In: 2022 IEEE international conference on machine learning and applied network technologies (ICMLANT), Soyapango, El Salvador, 15–16 December 2022, pp.1–6. New York, NY: IEEE.
42.
GuoKSuiLQiuJ, et al., Angel-eye: a complete design flow for mapping CNN onto customized hardware. In: 2016 IEEE computer society annual symposium on VLSI (ISVLSI), Pittsburgh, PA, USA, 2016, pp.24–29. New York, NY: IEEE.
43.
YuYWuCZhaoT, et al. OPU: an FPGA-based overlay processor for convolutional neural networks. IEEE Trans Very Large Scale Integr Syst2020; 28: 35–47.
44.
LuLXieJHuangR, et al. An efficient hardware accelerator for sparse convolutional neural networks on FPGAs. In: 2019 IEEE 27th annual international symposium on field-programmable custom computing machines (FCCM), San Diego, CA, USA, 28 April–1 May 20192019, pp.17–25. New York, NY: IEEE.
45.
YuWYibingLJiameiY, et al. An FPGA-based online reconfigurable CNN edge computing device for object detection. Microelectr J2023; 137: 1–15.
46.
FanHFeriancMQueZ, et al. FPGA-based acceleration for Bayesian convolutional neural networks. IEEE Trans Comput Aided Des Integr Circ Syst2022; 41(12): 5343–5356.
