Abstract
Introduction
Recently, cloud computing 1 has been positioned as a large-scale distributed model for providing on-demand information technology (IT) resources such as storage, servers, networks, and application as a service through data centers or high-speed networks. Cloud computing centers are increasingly facilitated with virtualization technologies to make better use of provider’s resources and to guarantee higher flexibility and reliability. The concept of network virtualization (NV) has provided a promise for the current and future infrastructure as a service (IaaS) provision in cloud networks. NV2,3 has become a popular concept in a wide range of technologies. NV enables network operators to share their physical infrastructure with different tenants by forming multiple virtual networks (VNs). In the cloud environment of NV, the Internet service provider (ISP) is classified into the infrastructure providers (InPs) and service providers (SPs). InPs build, maintain, and manage the physical infrastructure, while service provides are owners of VNs. SPs deploy different protocols as well as offer multiple end-to-end heterogenenous services to the end user. 4
NV brings many potential benefits. First, it offers a virtual environment to design and evaluate new network architectures or protocols. Second, it enables resource sharing among VNs and optimizes resource allocation of physical infrastructure. In NV, a VN is a combination of virtual nodes and virtual links on top of a substrate network (SN). A virtual node can be hosted on any available substrate node and interconnected by a set of virtual links. Virtual links can be spanned over several substrate links, forming a virtual topology. SPs send virtual network requests (VNRs) with the information of virtual nodes and virtual links capacity requirements to InPs which map the VNRs onto SN in turn, as shown in Figure 1. In cloud infrastructures, VNs are offered as a service to interconnect distributed cloud components and resources to support a plethera of cloud-based services for customers. As a result, the cloud service request can be abstracted as a VNR where virtual nodes stand for virtual machines and virtual links stand for cloud infrastructure links. VN provision can be realized via cloud systems such as cloud stack or open stack using software-defined network controllers and underlying data plan system OpenvSwitch.

Virtual network embedding problem.
A major challenge of the NV technology is known as a virtual network embedding (VNE) problem that deals with the efficient allocation of substrate resources to virtual requests. An efficient utilization of SN resources depends on the VNE algorithms that coordinates node and link resource constraints. The node constraints consist of several factors such as CPU performance, memory, and storage capacity or even the location of nodes in the substrate topology. The link constraints can be bandwidth, bit error rate, delay, throughput, jitter, and so on. Consequently, the VNE can be divided into two stages: virtual node mapping which assigns the virtual nodes to the substrate nodes and virtual link mapping where the virtual links are mapped onto physical links connecting these virtual nodes after the virtual node mapping. It is well-known that the VNE problem is non-deterministic polynomial-time (NP)-hard5–7 and the problem can be reduced to multi-way separator problem. 8
Recently, the VNE problem has received significant research attentions that aim to design diversified Internet architectures. The primary goal of VNE study is to maximize the economic benefit (called long-term average revenue) by accommodating as many VNRs as possible on a given physical network.9–12 However, the revenue of InPs depends on the VNRs, and in real-world situations, VNRs are dynamic and unpredictable. It is known as the problem with online VNRs. However, most prior works have just focused on static VNRs. Also, most of previous researches separated the VNE problem into two independent phases. That is, the virtual nodes are mapped onto substrate nodes at the first phase according to the local resource of substrate nodes without caring the other attributes such as location of substrate nodes or available bandwidth between nodes while these parameters have significant influence on the VNE algorithms’ performance. Then, the virtual links are mapped simply using the shortest path algorithm. 9 However, the separation of virtual node mapping and virtual link mapping might lead neighboring virtual nodes to be separated widely in the physical topology. Besides, this separation increases the cost of virtual link mapping phase. So, the acceptance ratio, as well as long-term revenue, might decreases.
With the aforementioned motivations, in this article, we investigate a new strategy to coordinate virtual link mapping and virtual node mapping in terms of handling online VNRs. The overview of our strategy is shown in Figure 2, and the arriving VNRs are handled within scan-time intervals; if there is any VNR which cannot be processed within a scan-time window, it is deferred to try in the next scan-time windows. In order to maximize the long-term revenue of InPs, we propose a new strategy to handle VNRs in each scan-time window. VNRs are grouped, analyzed, and ordered following CPU capacity and bandwidth capacity demand. We also consider the resources of the SN simultaneously to optimize resource allocation. The benefit of proposed algorithm lies in the fact that it can further leverage the dynamic VNRs and thus increase the revenue of InPs. The key idea of our algorithm is that it first chooses X embedding candidates where X is the search number, and in each candidate embedding, we choose a substrate node as the parent node (root node) which is mapped to the first VN node which is chosen by a selection rule. Then, VN nodes and links are mapped simultaneously based on joint node and link constraint. Finally, we choose the one with the best quality and which is feasible for embedding. The details of proposed algorithm are described in the later section. Through extensive simulations, we demonstrate that our proposed algorithm outperforms the prior algorithms.6,9,11

The overview of RT-VNE algorithm.
The main contributions of this article can be summarized as follows:
We find out the limitation of existing schemes. Without the ability to adapt dynamic VNRs and the separation in virtual node and link mapping phases, the existing schemes may perform inefficiently in case of dynamic and large-scale networks.
A new VNE algorithm is proposed. We introduce a new real-time VNE strategy which coordinates virtual node and virtual link mapping while handling dynamic VNRs as in the real-world cloud environment. The key point is that the algorithm handles a group of VNRs in each scan-time window.
Through extensive experiments, we show that the proposed scheme achieves a significant improvement compared to prior algorithms in terms of acceptance rate, resource utilization, and long-term average revenue.
The rest of this article is organized as follows: related works and issues are discussed in section “Related works.” In section “System modeling,” we formulate the VNE problem. In section “VNE algorithm,” we present our real-time VNE algorithm. Then, we evaluate our proposed VNE algorithm in section “Performance evaluation.” Finally, section “Conclusion” concludes of this article.
Related works
The VNE problem has received significant research attentions in both academic and industrial areas and presented the main resource allocation challenge in virtual cloud infrastructure. There are many studies to find out the efficient strategy to maximize the benefit for InPs including subgraph isomorphism, 12 path splitting and migration, 9 topology-aware node ranking 13 or other heuristic algorithms, and meta-heuristic algorithms.14–16 Generally, the existing studies can be divided into two primary categories: online approaches and offline approaches.
The offline algorithm processes are based on the assumption that all VNRs are known and defined in advance. Beck et al.17,18 proposed a distributed and parallel VNE (DPVNE) framework, which can be used in combination with various cost-reducing embedding algorithms. Through evaluation results, they proved that DPVNE achieves smaller message overhead and suggests a trade-off between message overhead and parallelism level. Fajjari et al. 19 introduced a Max-Min Ant Colony Meta-heuristic algorithm which uses parallel artificial ants to explore the search space iteratively for feasible VNR candidates and select the candidates based on several specific resource constraints. The work of J Lu and Turner 20 considered the single VNE in specific backbone-star topologies and assumed that the substrate resources are unlimited with only bandwidth constraints. In these offline VNE approaches, the VNR can be considered to gather for embedding, which may achieve a better performance compared to online approaches. However, offline VNE approaches do not contemplate the possibility of re-mapping existing VNRs to enhance the performance of the embedding SN. The arrival of VNR is unpredictable in a real environment, where dynamic algorithms are more suitable and preferred.
The dynamic VNE approaches are about the on-demand VNR assignment problem where algorithms dynamically compute a feasible set of substrate nodes and links to embed the virtual nodes and links upon the arrival of VNRs. Chowdhury et al. 11 proposed a coordinate node and link mapping strategy by formulating the VNE as a mixed integer programming (MIP), in which the node mapping is achieved by facilitating the link mapping in the subsequent phase. Randomized and deterministic rounding techniques are used to guarantee better correlation between node and link mapping phases. Yu M et al. 9 proposed the rethinking design of the SN to enable simpler mapping algorithms and more efficient use of physical resources, without restricting the problem space. They first designed the SN to split a virtual link to span over multiple substrate paths. They employed the path migration to periodically re-optimize the utilization of the SN to support the accommodating of new requests. In this research, a virtual link can be mapped to multiple substrate paths for splittable flows, and path migration is used to optimize the utilization of physical resources periodically. X Cheng et al. 21 handled VNE problem using the PageRank algorithm, where both substrate nodes and virtual nodes are ranked based on metrics, and then, the VN is embedded based on these ranks. The goal is to increase the acceptance ratio of online VNRs as well as overall revenue.
However, Ricci et al. 5 provided a heuristic algorithm based on a centralized algorithm which tends to maintain low and balanced substrate nodes and substrate links. A distributed VN mapping algorithm was proposed by Houdi et al. 22 In this research, the VNRs can be decomposed into hub-and-spoke clusters, and these clusters can be embedded independently, so it achieves to reduce the complexity of VNE. However, this approach has lower performance and it was efficiently compared to centralized approaches.
Besides, in cloud environments, the VNE problems also subject to dynamic changes and have to deal with node and link failures.23–26 Houidi et al. 23 and Gou et al. 24 proposed adaptive and survivable embedding algorithms to deal with link failures where the objective is to minimize the embedding cost. There are several types of failures such as physical node failure, physical link failure, and virtual link failure.
System modeling
In this section, we present the general embedding problem. Notations used in this context are summarized in Table 1.
Terminology used in this article.
VN: virtual network; SN: substrate network; BW: bandwidth.
Network models
SN
A physical can be described as a weighted undirected graph
VN
Similar to the SN, we denote a VNR by
VNE problems
The VNE problem is defined by an embedding
where
We have
where
where
where
Performance metrics
In this article, the following metrics are used to evaluate the performance of our proposed algorithm. First, we define the
where
Second, in each time window, the ratio of the number of successful embedded virtual network to all VNRs is defined as
Third, higher substrate resource utilization has always been an important objective. Resource utilization includes three different types: node utilization, link utilization, and network resource utilization
Besides, there are also unpopular metrics such as total allocated bandwidth, delay and throughput, and the number of active physical nodes or links.
VNE algorithm
In this section, we introduce our real-time VNE strategy. The overview of our proposed algorithm is shown in Figure 2, and the conjunction of virtual node-link mapping is depicted in detail.
Online VNR handling
In reality, the VNRs dynamically arrive and stay in SN for an interval time before leaving. In order to handle online VNRs, we divide time into scan-time windows (intervals) which consider how long the VNRs are executed frequently (i.e. an hour or several days). In detail, VNRs in each scan-time window are processed sequentially. The details are depicted in Figure 2.
Virtual node mapping
In this step, we perform the node mapping process which considers how to map virtual nodes on substrate nodes. Because node mapping affects link mapping directly, we want to perform node mapping first. Then, link mapping will be successful and be optimized.
Followings are the key idea of our node mapping algorithm to achieve this goal. First, we measure the CPU capacity and bandwidth resource of virtual nodes and substrate nodes. Then, we collect a group of
In terms of virtual nodes, when a VNR arrives, we need to determine the mapping sequence of virtual nodes to the available substrate nodes. We define the
First of all, over each time interval, the topology of each VNR is detected using Breath-First Search (BFS) algorithm, and then, virtual nodes are sorted in ascending order of hop distance from the parent virtual node which has highest required
where

RT-VNE algorithm example: (a) the VNR requests to map into substrate network, (b) node
Virtual node-link mapping conjunction
Next, we explain how to embed virtual nodes and links simultaneously onto SN. First of all, we define a metric called
Figure 3 shows an example of
Equation (12) also shows the bandwidth capacity requirement ratio to map virtual link between a pair of virtual node
Performance evaluation
In this section, we evaluate the performance of our proposed VNE algorithm in detail and compare to the prior algorithms under various scenarios. Our evaluation concentrates on quantifying the advantage of coordinate node and link mapping phases as well as proposed dynamic VNRs handling in terms of long-term average revenue and acceptance ratio. All simulations are performed on the server Intel® Core™ 2 Quad C Q9550 at 2.83 GHz (4Cs), 8GB memory, 1 TB disk and OS Linux 2.6.
Simulation environment
SN
Similar to most of the prior studies, we used the Georgia Tech Internet Topology Model (GT-ITM) tool 27 to generate SN and VN topology. The SN is composed of 100 nodes, close to mid-size of an ISP scale. These substrate nodes are distributed geographically in a (10 × 10) grid square. Each pair of substrate nodes is connected with a probability of 0.5. The CPU capacity of nodes and bandwidth capacity of links in SN follow a uniform distribution from 0 to 300 unit.
VNR
We evaluate the performance of VNE algorithms in various cases of VNRs. In each VN, the number of virtual nodes is distributed uniformly from 2 to 10 nodes for a small size, 10 to 50 nodes for mid-size, and 50 to 80 nodes for a large size. The average of VN connectivity is set to be a probability of 0.5. Similar to SN configuration, the CPU capacity and bandwidth capacity of each virtual node and link are uniformly distributed between 0 and 30 units. The arrival of VNRs is followed by a Poisson process with an average which ranges from 2 to 14 VNRs per scan-time window. The lifetime of each VNR at the SN follows an exponential distribution with an average of five scan-time windows. In each simulation, there are approximately 500 scan-time windows, which correspond to around 2500 VNRs on average. The value of candidate VNRs

Impact of the number of candidates.

Average run time per VN request.
Comparison setup
Our proposed algorithm is implemented in C/C++ and we compared our algorithm to previous state-of-art algorithms. To evaluate the performance of our proposed algorithm RT-VNE, we compare our algorithm to existing algorithms (Table 2) including greedy mapping algorithm,
9
a cluster-center VNE algorithm,
6
and three ViNE algorithms.
11
In greedy mapping algorithm (G-SP), virtual nodes with higher priority are mapped to substrate nodes with higher resources, and virtual links are embedded using
Compared algorithm.
VNE: virtual network embedding; MCF: multicommodity flow; MSF: multicommodity k-splittable flow.
RT-VNE algorithm performance
Long-term average revenue
We compare the proposed VNE algorithm RT-VNE to five existing algorithms including G-SP, CC-VNE, R-ViNE, D-ViNE, andD-ViNE-SP through 500 scan-time windows. In our scenarios, VNRs are divided into three groups: small-sized [2,10] nodes, medium-sized [10,50] nodes, and large-sized [50,80] nodes. From Figures 6–8, we find out several interesting observations. First, from the result in Figure 7, RT-VNE leads to better long-term average revenue than the existing algorithms in three cases which are small, medium, and large in size in the range of 500 scan-time windows. Along with these results, it implies that our proposed algorithm embeds VNRs that make more benefit revenue without taking care the size of VNRs (small, medium, or large) and increases the acceptance ratio of embedding. We believe that the reason is the higher acceptance ratio of RT-VNE (Figure 6) compared to existing algorithms as well as mapping virtual nodes to the substrate nodes with adequate resources increasing the probability of satisfying the resource requirements of the VNRs and consequently helping to accelerate convergence. Moreover, in addition to node mapping constraints, RT-VNE also considers the link bandwidth constraint which obviously makes link mapping phase easier. Thus, RT-VNE algorithm achieves a higher probability of accepting VNRs.

Acceptance ratio of algorithms.

Average revenue of algorithms.

Average revenue between algorithms on different VN scales.
Acceptance ratio
As shown in Figure 6, the proposed algorithm, RT-VNE, leads to higher acceptance ratio compared to existing algorithms through a joint node and link mapping. The acceptance ratio of RT-VNE (over 80%) is stable and higher than others (G-SP is around 52%, CC-VNE is about 72%, and R-ViNE, D-ViNE, and D-ViNE-SP are around 76%, 78%, and 63%, respectively). Since the linear programming (LP) relaxation and rounding techniques used by {R|D}-ViNE algorithms may incur infeasible VNE solution, they suffer low performance in terms of VNR acceptance ratio and consequently result in lower long-term average revenue (Figures 7 and 8). G-SP and CC-VNE always select maximum substrate objects for embedding. Thanks to
Resource utilization
The average resource utilization of substrate nodes and substrate links from different VNE algorithms are depicted in Figures 9 and 10. In terms of small, medium and large scales, the values of node utilization are around 20%, 38%, and 58%, respectively. The values of link utilization are approximately 20%, 40%, and 79%, respectively. Thanks to textitLink-traffic-ratio, even the acceptance ratio of RT-VNE is higher than others, and the average node and link utilization of RT-VNE are quite the same with the others. Consequently, network utilization is enhanced. As the VN scale increases, all compared algorithms have the higher node and link utilization. This is because there are more substrate nodes which have to be active to handle incoming VN nodes as well as the number of SN links increases when VN scale increases.

Average node utilization on different VN scales.

Average link utilization on different VN scales.
Summary
We can see that our proposed RT-VNE achieves higher benefit revenue and acceptance ratio than state-of-the-art existing algorithms (Figures 6–8). This can be explained by the fact that RT-VNE accepts many more VNRs than related approaches. In addition, coordination of nodes and links mapping also leads to efficient mapping and consequently generates a higher income. We conclude that the effect of joint nodes and link mapping is significant. The proposed algorithm considers the neighborhood influence of substrate nodes, so with the limited embedding candidate number, it is highly chosen as efficient embedding candidates. Besides, the proposed algorithm checks the feasibility of embedding candidates before embedding, and it is necessary for efficient embedding.
Conclusion
VN embedding is a critical feature to NV technology. This article studied the VN embedding problem which is NP-hard and computationally intractable. We propose a new resource efficient algorithm for VNE which coordinates node and link mapping phases and handles the online dynamic VNR in the real world, resulting in the improvement of embedding efficiency. The incoming VNRs are grouped, analyzed, and ordered following selection rules. In each VNR, a VN node is selected as a root node and then mapped to root SN node into SN network. In comparison with some previous algorithms, the new strategies can markedly improve both resource utilization and long-term average revenue while the complexity is kept at an acceptable level.
