A Framework of Data Express Service

As rapid development of the AI technologies, we need more powerful computing capability. In addition, sharing of the data and computing resource would also be necessary. Under this situation, the network needs to provide Data Express Service (DES) between different entities. The target of the service is normally to transfer a large file or a bundle of files, and the size may be at the GB, TB level, or even bigger. Although we can connect to a cloud service anywhere now, sometimes the file upload experience is not satisfying. Alternatively, we can post a hard disk to others for this big data transferring. However, if the job is repeated, it is surely boring and inconvenient. As the requirements of Data Express Service are growing fast, we need to provide a more convenient approach. The mechanisms in Data Express Service should be more automatic and convenient than the current approaches. Comparing to the approach of transferring by using the Internet directly, the DES should be quicker and automatic. Comparing to the manual hard disk express, the DES should be more extensible and convenient. In details, the DES scenarios may include inter-cloud backup and disaster recovery, film and television editing, scientific computing, cooperation of intelligent computing centers. This document aims to analyse the requirements of DES, and proposes a potential framework for it.

The demands of the DES traffic would be slightly different from the current Internet services. The first one is that it may be not that real-time. It is only required to be completed within a time limitation. Some may need to be completed within hours, and some may need to be completed within one day or two days. The traffic amount would be huge normally, and the flow would last for a while in the network. It would appear as an elephant flow in the network. Hence, the traffic characteristic of the DES would be more predictable than the current Internet service, and we can provide specific treatment for these traffic. Even, we can schedule the transport time for them, only if we can finish it on-time. In other words, we can store the file waiting to be transported.

A framework for DES is proposed in this section as shown in Figure 1. We recognize two network environments in the framework, i.e., the access network and the core network. Traditionally, the client can connecte to the cloud directly. However, due to the complexed E2E network environment, the E2E transport speed sometimes is not satisfying. For the DES, we can split the E2E connection into three parts: from the client to a Data Harbor, among the Data Harbors, and from the Data Harbor to the cloud. The Data Harbors can communicate with each other at a high speed; meanwhile, it can store the files temporarily.

As an example, the procedures of the DES service are as follows: Step1: a client connects to an available Data Harbor and uploads its files, if its speed to the target cloud is not satisfying. Meanwhile, the client should provide the time limitation for the files. Step2: the Data Harbors should schedule the transport time for the files according to its time limitation and the link load between target Data Harbors. Step3: after receiving the files, the target Data Harbors should upload the files to the cloud according to its time limitation and the link load between the target Data Harbors and the cloud. If we have many clients that need the DES services, we can built a high speed link between specific Data Harbors. This is the key feature that the DES can be more quick than a normal Internet connection. Under this situation, we also need a high speed transport protocol to transport files efficiently, or more specifically, a specific designed congestion control method for the Data Harbors. The methods mentioned in can be considered here. Optionally, to improve the speed between the Data Harbors, we can also deploy RDMA mechanisms.

TBD.