Abstract
Introduction
A recent report expected that the Internet of things (IoT)-based healthcare market be anticipated to grow from $41.22 billion in 2017 to $158.07 billion by 2022, at a compound annual growth rate (CAGR) of 30.8% from 2017 to 2022. It indicates how medical care and healthcare services are important in the IoT field. 1 In IoT, the high degree of autonomy is considered one of the important and desired goals. Thus, an efficient, reliable, and effective approach needs to be developed.2,3
It is well known that the attributes of physical body conditions are inherently diverse in nature and dynamically varied over time in terms of age, sex, and hereditary factors. However, these attributes in practice are often invisible to physical examination especially for persons. 4 For processing diverse properties of body-related sensor data, recent advances in computer systems (e.g. multi-core CPUs and graphics processing units), have catalyzed high-performance computing including scientific, engineering, big data, and financial applications.5,6
At the same time, with the rapid deployment of cloud computing, these applications and workloads are migrating to the cloud computing environments.7,8 One of the advantages of facilitating cloud computing is that application developers do not have to maintain a large number of clusters in the in-house data center.9,10
Unfortunately, developing an efficient sensing method of physical body data on the cloud computing system is challenging. As various applications and workloads move to the cloud computing system, traditional approaches of processing sensor data cannot be applied, and current data sensing architectures are not ready to support efficient processing, real time, and mobility. 11
When an existing sensing architecture is used in the cloud computing system, tenants may experience incompatibility and unpredictable performance variation due to inefficient implementations. Such limitations prevent application developers from moving their applications to the cloud computing system.
In this article, we study a cloud-based physical body data sensing architecture for artificial intelligence computation. To this end, we design a framework that securely aggregates physical sensor data to the cloud computing system and efficiently processes them. The essence of our design is to fully utilize the features of the cloud computing system by integrating resource management services.
In addition, the architectural design enables application developers to analyze the aggregated sensor data in order to identify the relationship between body activities and health for persons. Another benefit of our prototype architecture is that a user is able to identify similarities and differences in body activities with respect to other users and to detect potential health problems through big data analysis. In section “Performance evaluation”, we will show that the proposed sensing architecture provides a flexible and efficient framework based on cloud computing capabilities and retains all the benefits of the traditional approaches.
The remainder of the article is organized as follows. The following section gives the related work. Section “The proposed architecture” describes the proposed architecture based on edge cloud computing. In section “Optimization techniques,” we present the optimization techniques of our architecture for efficiency and compatibility. Then, we evaluate the performance of the proposed solution through extensive experimentations in section “Performance evaluation.” Finally, section “Conclusion” covers the conclusions and potential future directions.
Related work
The recent development of IoT has revitalized the feature scale of the smart city, wearable, and mobile applications. The landscape of mobile applications based on the wireless technology encountering big data processing needs to be reframed on cloud computing instead of limited computational resources. Various techniques for healthcare services based on cloud computing systems have been proposed.12,13 Hassanalieragh et al. 12 proposed a mobile-based application for Android to monitor electrocardiogram signals. Fortino et al. 13 proposed a multi-tier application-level architecture that integrates a cloud computing platform and body sensor networks data streams middleware that allows collecting, processing, storing, and analyzing them. However, the solution does not support real-time and edge computing. In other words, a decision can be made on the receipt of the result from the cloud computing system after transferring the monitoring data from sensors to the cloud.
Task offloading14,15 is an attractive technique for IoT and fog computing.16,17 It can occur between the sensor (IoT) devices, edge nodes, and cloud nodes. Criteria of task offloading are based on various factors (requirements of applications for storage and computation, load balancing, energy consumption, network latency, etc.), and it can be controlled by a central cloud server. In this direction, Zhao et al. 18 proposed a task offloading method for mobile devices. Despite the demand for mobile devices to run computational-intensive applications is increasing, mobile devices still have limited computation capabilities due to small physical size. In this regard, the mobile device can offload tasks to the cloud. However, task offloading is infeasible when the network delay is intolerable. Based on this observation, the authors proposed an offloading algorithm to reduce the energy consumption of mobile devices considering the response time requirements and the maximum transmission power.
Pu et al. 19 developed another task offloading algorithm by taking the incentive constraints into account to prevent the over-exploiting and free-riding behaviors based on the device to device communication. A device contacts nearby neighbor devices that requires task offloading. However, the decision-making process for the task offloading is made by a central server, which introduces more messages to be exchanged in the communication system. Considering this, the authors designed an online task offloading algorithm with Lyapunov optimization utilizing the status of the computing system.
Fortino et al. 20 proposed a cloud-based wireless body area networks and leveraged a flexible storage and processing platform of the cloud to perform online/offline analysis. Along the same line, Quwaider and Jararweh 21 proposed a cloudlet-based22,23 wireless body area networks for sensor data aggregation of the network. In this work, the authors considered a large-scale wireless body area network to be available to the cloud user and to the service provider in a reliable fashion. For big data processing of real-time data, Zhang et al. 24 proposed an adaptive MapReduce framework based on cloud resources to scale the computation for real-time applications.
While significant efforts have been investigated into cloud-based body sensing architectures, the performance cannot meet the requirements of real-time mobile services, and there is a room for improvement in terms of the mobility of mobile users, dominance of wireless network, geographical distribution of nodes, and real-time data. Therefore, in this article, we dive into the details of the edge cloud computing technique and investigate leveraging the three-tier architecture and storage optimization in the cloud server.
Furthermore, we avoid the failures due to network delay and resource limitation by providing edge devices in the middle of mobile users and the central cloud servers. In addition, our cloud architecture for physical body data reduces overall server utilization and, therefore, the energy consumption can also be reduced. This enables us to build a scalable system architecture in edge computing environments.
As for artificial intelligence in cloud computing environments, there are a few studies. Grzonka et al. 25 proposed a multi-agent-based cloud monitoring model to detect false tasks by incorporating artificial intelligence algorithms. However, it uses a master–slave architecture of agents, and therefore, it may suffer from scalability issues in large-scale infrastructures. However, our work does not entail a bottleneck on the coat tails of a layered architecture. The details of the proposed architecture are characterized in the next section.
The proposed architecture
Figure 1 shows an overview of physical body data flows based on the cloud computing system. The first layer is similar to the wireless body area network that provides a continuous health monitoring of persons without any constraint on their normal daily life activities. 26 The real-time health monitoring of persons significantly improves the rate of successful diagnosis in case of life-threatening diseases. 27 In addition, it reduces the cost incurred in diagnosis because of the early detection of various diseases. 28 In wireless body area networks, biomedical sensors can be broadly divided into two categories: on-body sensors and off-body sensors. 29 On-body sensors are sewed to the cloth or attached on the skin of a person’s body (e.g. smart watch) and off-body sensors are kept away few centimeters from a person’s body (e.g. Zigbee). 30

Overview of physical body data flow.
In our architectural design, the sensing data are forwarded to the edge cloud servers rather than the central cloud servers directly. The rationale of having the three-tier architecture is to optimize the cloud computing systems by taking the control of computing applications, data, and services away from the core to the edge. 31 Note that we add and modify the existing edge-based cloud computing system for real-time and mobility supports. More specifically, cloud computing can store, manage, and process large amounts of data on remote data centers via the Internet. However, cloud computing technologies have potential drawbacks, in particular, increased latency of information processing. For time-critical applications, there is a need for a faster and more flexible solution. Edge cloud is a response to this challenge. Hence, cloud capabilities should be distributed across the network to form an edge cloud, which places computing resources where the traffic is—at the edge of the network. Edge cloud servers are suitable for resources that may not be continuously connected to a network such as implanted medical devices, fields of highly distributed sensors, and mobile devices.
Edge computing covers a wide range of technologies including wireless sensor networks, mobile data acquisition, mobile signature analysis, peer-to-peer networks, and ad hoc networking. The edge cloud servers then forward sensing data to the central cloud servers. In this way, a user’s vital signs are examined by big data analytics on the cloud computing system. The monitoring data include heartbeat rate, respiratory rate, electro encephalogram (EEG), electro cardiogram (ECG), blood pressure, temperature, glucose level, and so on. For analytical evaluation, we use accelerometer and gyroscope sensor data from smartphones and smartwatches.
In our edge cloud–based architecture, where mobile services and applications are managed at the edge devices, while rest are controlled in the central cloud servers, the body data generated from sensors generate massive amounts of data on a real-time basis that can be analyzed and processed to produce the optimal insights and decisions.
In the central cloud servers, the aggregated physical data will be stored in the stable storage. To process and analyze the data, the cloud coordinator in the central cloud servers schedules virtual machines. Then, the virtual machine process the data for health analytics. 32 Based on the service-level agreement and quality of services, the cloud coordinator may schedule multiple virtual machines simultaneously. The output of the analytics can also be forwarded to the edge cloud servers, and then the results are notified to the mobile user.
Figure 2 shows the proposed mobile service architecture to support real-time and mobility. In the previous mobile service architecture, a user contacts a central data center directly with a sensor device. In this case, failures due to mobility and resource limit may occur. For instance, when a user moves to another location, where wireless signals are broken and the user cannot proceed with the mobile application. In addition, when the mobile application user requests computation resources suddenly, the central data center may require more time to prepare the resource.

Proposed mobile service architecture to support real-time and mobility.
To support real-time mobile applications, the proposed architecture employs the edge servers between the central data center and users. On the receipt of the service request of a user, the slave daemon on the edge server sends user-related data to the master daemon on the central server along with the service request. Then, the master daemon performs the service modeling and profile management. Next, it forwards service-related data for the user to the slave daemon.
Since the edge servers are distributed around the users, it supports mobility and, therefore, there is a low probability that a failure occurs. When a user requests a lot of computational resources, the edge servers can provide reserved resources for the user at once. In this way, our proposed mobile service architecture supports both real-time and mobility. In the performance evaluation section, we verify the efficiency of the proposed architecture with respect to the previous architecture.
Figure 3 depicts the cloud-based data sensing architecture. At the bottom layer, there are services for the sensing devices. The sensing devices capture the user’s movement and status, and therefore, the real-time data are stored in local devices or forwarded to the edge cloud. At this stage, the level of the service-level agreement can be checked. According to the applications, one or multiple service requests are forwarded to the edge cloud.

Edge cloud–based data sensing architecture.
The second layer is related to the edge cloud servers and services. Since the edge cloud servers are not reliable as the central cloud servers, it is necessary to implement fault tolerance techniques. The basic component of the fault tolerance techniques is the fault prediction. Note that the results of the fault prediction component can be normal, warning, or alert. Based on history and status information, we can further perform the fault avoidance or fault recovery process. If the result of the fault prediction indicates a warning or alert, the fault avoidance process will be performed. When a fault occurs, the fault recovery process will be performed by recovering data with intermediate results.
The top layer is for the central cloud servers. The role of the central cloud servers is to collect data and process them efficiently. Since the basic processing unit is a virtual machine, the central cloud servers manage the virtual machines in the system (i.e. create, destroy, and migrate). Even in the presence of virtual machine failures, the data are stored in the stable storage in the cloud computing system. Therefore, when a virtual machine fails, the cloud coordinator can schedule another virtual machine to process services or workloads with the data in the stable storage.
To process big data analytics efficiently, the data can be replicated or chunked. By splitting one big data into a fixed size, multiple instances of the virtual machines can perform the big data applications concurrently without transferring the data to the node. Furthermore, when a data block is lost, the replicated data block in another node can be used. If multiple virtual machines are running on a host machine, the service-level agreement module is triggered. Among the virtual machines, the slowest virtual machine can be selected for live migration to another host machine that can afford the virtual machine. In this regards, our architecture achieves the efficiency and fault tolerance at the same time.
Optimization techniques
Storage optimization
To improve the reliability of the data storage, we design the storage architecture to support multiple types of applications (both stateful and stateless). If an application’s type is stateful, it is necessary to implement the persistent storage for them. While an application’s type is stateless, the lifetime of data is dependent on that of a virtual machine.
Figure 4 shows the proposed storage architecture supporting multiple types of applications. There are applications

Application-aware storage architecture.
For each type of applications, an application is logically connected to a stable volume via the network and the volume size is automatically allocated based on the application’s profile. For the best matching between applications and storage volumes, the cloud storage controller keeps track of each available volume in the database. As for stateless applications, our storage architecture attaches a temporary volume to a virtual machine when provisioning and securely destroys when the virtual machine is turned off.
Based on the monitored resources, we can optimize the aggregate throughput for multiple virtual machines by solving the following problem with linear programming
where
Because of the first constraint of the problem, the performance degradation due to the unpleasant mapping between virtual machines and storage volumes is minimized. The throughput of the problem can be modeled as input/output operations per second (IOPS) and the data transfer rate. Note that IOPS represents how quickly a given storage device or medium can read and write commands in every second, and the throughput can be affected by IOPS. Therefore, an edge cloud–based architecture can help reduce the bottleneck for the throughput by offloading the input/output operations to the edge.
CPU-aware scheduling
The basic computing unit on the cloud computing system is a virtual machine. 33 Therefore, it is essential to employ an efficient virtual machine scheduling algorithm for server consolidation. 34 One observation for consolidating the cloud is that the relationship between the CPU utilization and energy consumption is not linear. 35 In fact, the energy consumption of CPU grows more than linearly as the CPU utilization increases. In particular, when the CPU utilization exceeds 90%, the energy consumption quickly increases. Thus, stacking many virtual machines on a physical machine is not a suitable solution for both energy consumption and performance.
To exploit this property, one can use a round-robin-based virtual machine scheduling policy. In this approach, virtual machines are assigned to each physical machine in equal portions and in a circular order. Another benefit of using the round-robin policy is load balancing, since the healthcare service’s task length is around the same. When a virtual machine finishes the job for a user, the virtual machine is scheduled to shut down. If a physical machine has no more virtual machine, the physical machine can be turned off. Therefore, additional energy saving can be achieved.
To simplify the load balance with the CPU-aware scheduling, we exploit the probability of the balls-and-bins problem. Suppose a scheduler has to choose a suitable physical machine for provisioning a virtual machine. To find the least-loaded resource, the scheduler may monitor the load on all the physical machines before provisioning the virtual machine. This procedure is very expensive in the cloud computing system, since it requires sending and receiving a message for every physical resource. Instead, in our architectural design, if the scheduler samples the monitoring information of two physical machines and sends the provisioning request to the least loaded one, then the total utilization for virtual machines remains small. After all the balls (virtual machines) are placed, the maximum load of any bin (physical machines) is at most ln
Performance evaluation
In this section, we evaluate the performance of our cloud architecture with a healthcare application compared with a previous cloud architecture. The reason why we evaluate healthcare application is that cloud computing has the capacity to revolutionize healthcare, rendering it more efficient through a decentralized approach and improving the patient experience by providing services comparable to those offered by internal IT organizations—yet, at significantly lower costs.37,38
By using cloud services, healthcare organizations only need to pay for what they use, such as storage, applications, and infrastructure service. Another benefit of the cloud for healthcare is flexibility, as providers can scale resources up or down as needed. The cloud provides real-time and remote access to applications and resources with ease. Specifically, we evaluate the failed tasks, service time, processing time, and server utilization of the cloud system by increasing the number of mobile devices. Then, we evaluate the human activity recognition analysis on the cloud based on the heterogeneity human activity recognition data set from smartphones and smartwatches. 39
Cloud performance
Our cloud testbed is built on servers with a discrete event executor in Java, which is equipped with Intel Core i7-6700K (4.00 GHz, four cores, eight threads) and 32 GB memory. We consider failures due to various reasons, that is, virtual machine capacity, bandwidth (reject, unfinished), and mobility. The hypervisor of data centers is Xen, 40 and therefore, the host operating system is Linux. The number of host machines is 15, and there are 2 virtual machines on a host machine with 2 threads and 2 GB memory. To implement the edge cloud experimental environment, we set up an edge server nearby the user, and the edge server is configured to interact with both the user and the central cloud server.
The properties of the health application are as follows: low task length, low virtual machine utilization, high data transfer ratio (upload/download), and a low active period. We evaluate the percentage of failed tasks, the number of failures due to virtual machine capacity, service time, processing time, server utilization, and network delay by increasing the number of mobile devices from 100 to 400. Note that for this experiment, we generate the artifact mobile traffic based on the number of mobile devices. For the cloud performance evaluation, we use a virtual environment, which is similar to health checks where users go around and request a task with the help of a cloud computing service located at the department. We assume that individual departments equip with cloud computing capabilities from a few numbers of virtual machines.
Figure 5 shows the percentage of failed tasks and the number of failures due to the virtual machine capacity. The previous indicates the cloud system without the edge devices, and the proposed indicates the cloud system with the edge devices. As the number of mobile devices increases, the performance gap between the previous and the proposed increases. When the number of mobile devices is 100, the percentages of failed tasks for previous and proposed are 12% and 1.1%, respectively.

The percentage of failed tasks and the number of failures due to virtual machine capacity: (a) the percentage of failed tasks and (b) the number of failures due to virtual machine capacity.
However, when the number of mobile devices is 400, the percentages of failed tasks for previous and proposed are 18.2% and 7.7%, respectively. This indicates that the proposed cloud architecture reduces the percentage of failures. The primary reason for the failures is due to the virtual machine capacity. Figure 5(b) depicts the number of failures due to the virtual machine capacity. The proposed cloud architecture based on edge cloud always outperforms in terms of the number of failures due to the virtual machine capacity. The ratio of the previous and proposed (previous/proposed) for the number of failures due to the virtual machine capacity is about 2.6.
Figure 6 shows the performance results of the service time and processing time. When measuring the service time and processing time, the lower is better, since it will have more idle time when the service time and processing time is short. Therefore, the lower service time and processing time result in low energy consumption than those of higher values. For service time (cf. Figure 6(a)), the proposed cloud architecture has less than 0.29, 0.42, 0.35, and 0.77 s on average when the numbers of mobile devices are 100, 200, 300, and 400, respectively, compared to the previous architecture.

Performance results of the service time and processing time: (a) service time and (b) processing time.
The phenomenon is similar to the results from the previous experiment for processing time (cf. Figure 6(b)). For processing time, the proposed cloud architecture has less than 0.38, 0.52, 0.45, and 0.88 s on average when the numbers of mobile devices are 100, 200, 300, and 400, respectively, with respect to the previous architecture. The normalized processing times of the proposed cloud architecture in terms of the previous are about 0.79, 0.77, 0.80, and 0.69 when the numbers of mobile devices are 100, 200, 300, and 400, respectively. This means our cloud architecture is suitable for scalable systems.
Figure 7 depicts the percentage of server utilization for cloud host machines. As the number of mobile devices increases, server utilization of both approaches increases. However, server utilization of the previous approach fluctuates as the number of mobile devices increases, while that of the proposed approach stably increases with lower server utilization than the previous architecture. This shows the robustness of our cloud architecture based on edge devices.

Performance results of server utilization.
Table 1 shows the network delay from mobile devices to the central cloud servers. It is interesting that the proposed architecture has more network delays than the previous architecture. This is because the proposed approach is a three-tier architecture, while the previous approach is a two-tier architecture. Although the proposed architecture has more network delays, the difference between the two approaches is less than 100 ms. This is negligible and does not affect the overall performance of the healthcare application, since most of operations of the application interact with the edge cloud rather than the central cloud.
Network delay from sensors to central clouds (milliseconds).
As shown in Figure 8, however, the local area network (LAN) delays for the two approaches are about the same. Note that in the previous approach, the LAN delay is measured from the sensor device to the central cloud server, while that is measured from the sensor device to the edge cloud server in the proposed approach. The total network delay for the proposed approach is the sum of the LAN delay and wide area network (WAN) delay (Table 1). With this experiment, we confirm that our edge cloud–based body data sensing architecture achieves real-time performance with less failure and utilization in than the previous approach. With our proposed architecture that supports real-time and mobility, other applications that require functions within a time frame and offloading capabilities for data processing such as video conference, voice over Internet protocol, online gaming, e-commerce transactions, and instant messaging.

Performance results of LAN delay.
Analytics performance
To show the effectiveness of the proposed cloud architecture, we evaluate one of the artificial intelligence applications, that is, activity recognition based on sensor data. The number of instances is more than 43 million based on accelerometer and gyroscope sensor data, whose size is about 1.5 GB. The activities are biking, sitting, standing, walking, stair up, and stair down. We implement the classifier of the physical activities collected by smartphones and smartwatches in Python. The libraries used to implement the classifier are Numpy, Matplotlib, Pandas, Keras, and Scikit-Learn. There is a three-layer neural network based on a long short-term memory model. The first layer has 24 outputs with 4128 parameters, the second layer has 12 outputs with 1776 parameters, and the third layer has 6 outputs with 78 parameters. Note that for the proof-of-concept, we do not implement other models such as Gaussian mixture model (GMM) or support vector machine (SVM), which sometimes require a priori knowledge (e.g. probability distributions and kernel-specific parameters).
Figure 9 shows the performance results for the classifier. Note that each result for an iteration is an averaged value over three epochs. For processing time, it decreases as the iteration continues. The processing time for iterations 1, 2, and 3 are about 85, 43, and 23 s, respectively. The times taken from each step for the iterations are 87, 44, and 23 μs, respectively. For loss (error), it decreases from 25.9% to 23.8% as the iteration step increases. However, the accuracy increases from 89.3% to 90.1% as the iteration step increases. The accuracy is dependent on the model used and the percentage of the training data set. We use about 80% of the data set for training and 20% for testing. With our testbed, we validate the analytics based on sensor data that can be performed well with high accuracy.

Performance results for analytics: (a) processing time for each iteration, (b) processing time for each step, (c) loss, and (d) accuracy.
Conclusion
The real-time service for mobile users has two inherent properties: real-time and mobility. To support real-time and mobility, we proposed an edge cloud–based body data sensing architecture for artificial intelligence computation. In the proposed architecture, the volume of data that is required to be transferred from the sensor devices to the central cloud server is reduced by deploying the edge devices between the sensor devices and the central cloud server. Performance results show that our edge cloud–based architecture outperforms the previous architecture in terms of failures, processing time, and scalability. It also indicates that the energy consumption of the cloud server can be reduced, since the server utilization of the proposed architecture is lower than that of the previous one. Future work includes the integration of artificial intelligence techniques to predict failures of edge cloud servers and development of applications based on our database.
