Flow Aggregation for Enhanced DetNet

Introduction The clearly states that Deterministic Networking (DetNet) operates at the IP layer and delivers service which provides extremely low data loss rates and bounded latency. The DetNet Quality of Service (QoS) includes the bounded latency indicating the minimum and maximum end-to-end latency from source to destination and bounded jitter (packet delay variation). As per , the DetNet data plane must support the aggregation of DetNet flows in order to support larger numbers of DetNet flows and improve scalability by reducing the per-hop states. Without aggregation, the considerable state information and the large number of signalling exchanges among individual flows to provision deterministic services in scaling networks are unbearable, and the per-relay system may limit the scale of DetNet networks. The introduces that the flow aggregation is the ability to aggregate individual flows along with their associated resource control into a large aggregate. It is recommended that the DetNet flow aggregation be enabled for compatible flows with the same or very similar QoS and CoS characteristics via the use of wildcards, masks, prefixes, and ranges. It also suggests the reduction of per-hop states help avoid the per DetNet-flow specific state maintenance in a transit node. It further provides arguments on how DetNet services might be realized in term of delay bound, delay jitter and bandwidth provisioning. Further, the , has proposed and expanded two methods of flow aggregation, one being the aggregation via LSP hierarchy and the other to aggregate DetNet flows as a new combined DetNet flow. In the maturing IETF draft for scaling networks, i.e., , an enhanced DetNet should support different levels of co-existed applications with different SLAs requirements. From the use cases in and , DetNet applications differ in their network topologies and specific desired behaviors. Different DetNet flows transmit and forward with different QoS behaviors. It should provide fine-grained service provisioning to achieve differentiated DetNet QoS. The DetNet flows with the same level of service requirements can be aggregated to receive collective treatments and forwarding behaviors. The DetNet flows can be classified and aggregated based on flow-specific characteristics. Moreover, the existing aggregation of individual flows may be still challenging for network operations. The aggregated flows still requires a large amount of control signaling to establish and maintain the states of DetNet flows when there will be large-scale deterministic flows over the dynamic network topology in enhanced DetNet. It is required to improve the scalability and forward packets at the class-aggregate level, instead of the per-flow or flow-aggregate level, for which the flow identification of aggregated-class might be adopted to indicate the per-hop behavior without the maintenance of excessive states in scaling networks. This document, in the , describes the objectives & requirements of flow aggregation and proposes an aggregation scheme for DetNet flows based on DetNet flow-specific classification in enhanced DetNet. The proposal argues that the flow identification of an aggregated-class can be used to indicate the required treatment and forwarding behaviors in scaling networks. The discusses the realization of DetNet flow aggregation for 5GS.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Terminology The terminology is defined as .

Objectives & Requirements: Flow Aggregation in Enhanced DetNet

DetNet Services to Aggregated Flows across Domains Flow aggregation is recommended in the multi-domain scenario to achieve the end-to-end QoS guarantees for aggregated flow(s) that span across multiple domains. As per , different network implementations may be intended for different application domains, where there is no additional requirements for the coordination. As defined in [ITU-T Y.2122], the network operating parameters of a flow aggregate should be exchanged among different network domains. As shown in Figure 1, the DetNet domain B receiving an aggregated flow should identify the flow and get the service requirements such as the QoS parameters of the flow aggregate.

Individual Flows +-----------------+ +-----------------+ -------> | | | | ...... | DetNet Domain A | Aggregated Flow | DetNet Domain B | -------> | | --------------> | | +-----------------+ +-----------------+

Aggregated vs. Fine-grained QoS Provisioning The draft specifies that different levels of applications differ in the SLAs requirements such as tight jitter, strict latency, loose latency and so on. While these types of aggregated requirements might bear the coarse-grained nature, individual flows demand differentiated DetNet treatments and more granular QoS forwarding behaviors. A DetNet node or domain adopting multiple forwarding technologies needs to transmit individual flows by aggregating them into a selected treatment solution that corresponds to one of some pre-defined per-hop QoS behaviors, as shown in Figure 2. For example, as per , a 5GS as a logical DetNet node requires to achieve the service requirements and service levels of the aggregated flows, along with the provisioning of fine-grained per-hop behavior (PHB) to each individual flow.

Scale Down States Maintenance at Transit Nodes As per , the treatment solutions in data plane can be categorized based on performance and functional characteristics. For example, the category of a solution can be classified based on the traffic granularity, e.g., flow aggregate vs. class level. The class level is provided to simplify the control and accommodate traffic fluctuations by combining flows requiring the same or similar levels of service requirements. The flow aggregation based on the class level could further improve the scalability. As per , there may be a large number of traffic flows in a scaling network, which makes it nearly impossible to achieve the flow-specific state identification. As shown in the Figure 3, the flow identification of aggregated-class can be used to indicate the required treatment and forwarding behaviors without the need to maintain excessive states at transit nodes.

Individual Aggregated Flows +-------------+ Flow(s) +-------------+ +-------------+ -------> | | | | | | ... |DetNet Node A|---------->|DetNet Node B|----->...|DetNet Node N| -------> | | | | | | +-------------+ +-------------+ +-------------+ 'Bucketed' into Large number of Fewer number of classes Individual Flows -----------------> consisting of aggregated flows When DetNet flows are aggregated based on service-class level, transit nodes provide deterministic services to a flow aggregate and go through the per-class scheduling without the burden to maintain excessive per-flow states. Still, a transit node performing aggregation should ensure all per-flow service requirements within an aggregated class are achieved. For example, the latency or jitter bounds of an aggregated class shall not exceed the corresponding metrics of any individual flow that has been bucketed into the class. The aggregation based on the class level has data plane and controller plane aspects.

Enhancement Consideration for Flow Aggregation This document proposes an enhancement for DetNet flow aggregation including the functions such as flow classification, flow identification and flow coordination.

Flow Classification The DetNet QoS can be achieved by aggregating flows based on DetNet flow-specific traffic classification and providing the traffic-forwarding treatment. The flow classification should consider the flow-specific characteristics such as traffic specification and service requirements. For example, the DetNet flows MAY be classified based on the service SLAs requirements of applications in scaling networks as per . And the services can also be classified into tight/loose latency, large/small burst, periodic/non-periodic and large/small scale services as per . Several classes can be predefined to indicate the different levels of applications with SLAs requirements and each class demands differentiated QoS behaviors and treatment as well as different DetNet capabilities in scaling networks. For the controller plane, the service should be provisioned on an aggregated-class level. The resources should be controlled and scheduled on a per-class basis and the routes should be established to meet the service requirements with the allocated resources. The edge nodes must be able to handle admission control for DetNet flows to an aggregated class based on the available resources. For the data plane, when DetNet flows are aggregated to a class, transit nodes provide service to the aggregate and not on a per-DetNet-flow basis. And the transit nodes supporting this type of aggregation should identify the class of the aggregated flows and ensure that individual flows receive the corresponding traffic treatment and forwarding behaviour.

Flow Identification The flow identification is used to identify the aggregated flow and to indicate the required treatment and forwarding behaviors. As described in , it is required to be dynamic and simplified to ensure the aggregated flows have compatible DetNet flow-specific QoS characteristics. For the data plane, individual flows may be aggregated for treatment based on shared service specification on aggregated-class level which is identified by an aggregation class (A-Class). A transit node should provide the class level traffic treatment based on A-Class. The aggregation class information may be used alone or together with other metadata to indicate the required queuing and forwarding behaviors. The encoding of the A-Class may reuse the DSCP/TC or existing field such as the TC field in A-Label as per . And it also can be encapsulated with the deterministic latency information as per .

Flow Coordination In scaling networks, flow aggregations become more prevalent, with flows frequently joining and leaving, which may potentially lead to accumulated bursts of flows across multiple hops. Such challenges can be mitigated by coordinating packets within aggregated flows such as proportional scheduling and interleaving. Proportional scheduling could allocate transmission opportunities based on flow weights, ensuring that each flow receives a fair share of network resources. Interleaving could achieve micro burst smoothing by rotating the transmission of packets across different flows through timed gates as described in .

Realization of Flow Aggregation for 5GS DetNet The 3GPP in its document has defined and standardized how a 5G system (5GS) may behave as a logical DetNet node, as well as how a 5GS DetNet node may integrate into the IP-domain DetNet as described in . 3GPP has realized the functionalities of the DetNet forwarding sub-layer. As a logical DetNet transit node, a 5GS behaves as a transparent box to external DetNet entities. It can connect to either DetNet end systems or full-fledged IP DetNet domain(s) or both. The 3GPP has demonstrated a ‘composite’ architecture in that a 5GS could act as one or more DetNet nodes upon the integration into the IP DetNet domain. The granularity of determining a 5GS DetNet node is per UPF for each network instance, with the corresponding UPF-id identified as the 5GS DetNet node-id. The 3GPP has implicitly specified two types of DetNet related traffic parameters. One type is the higher-level per-(logical)-node QoS requirements like the node max-latency, max-loss, etc., while the other is more granular settings with which DetNet flow attributes and specifications are mapped to the Flow Description information. The DetNet flow specifications could be based on IP-tuple information, e.g., including IP address, protocol type, ToS, TCP/DUP ports, etc. The document has provided more details.

Realization of 5GS DetNet Service across Domains 3GPP has so far standardized the forward sub-layer functionality for 5GS. It indicates a 5GS (logical) DetNet node could connect to other end systems and/or IP DetNet domains, together to form a holistic end-to-end DetNet. Thanks to the 'composite' architecture of a 5GS node, along with the interaction between an CPF:DetNet controller in IETF domain and the NF TSCTSF in 3GPP domain , a 5GS node might realize much more advanced DetNet services than a traditional IP DetNet transit node. In scenarios where the (IETF-domain) CPF:Detnet Controller could well divide the DetNet QoS service requirements that are in reality associated with an integrated DetNet domain into multiple QoS sub-requirements that together form the original end-to-end QoS equivalence, a 5GS might be considered as a standalone DetNet (sub-)domain with its own DetNet QoS (sub-)requirements that would be provisioned from the CPF:DetNet controller. The 5GS DetNet QoS (sub-)requirements serve a portion of the original requirements of the integrated DetNet domain. These together form a scaling network to realize the 5GS DetNet service across domains.

5GS QoS Provisioning: Aggregated vs. Fine-grained We have explained previously that the 3GPP has implicitly specified two categories of DetNet related traffic parameters. One type bears the aggregated nature for (5GS DetNet) node-level requirements, while the other addresses the more granular DetNet flow-level attributes and specifications. Evidently, with this kind of two-hierarchy architecture, a 5GS DetNet node could achieve not only the node-level aggregated QoS requirements, but also the more fine-grained flow-level QoS provisioning. This reflects the true value of applying our flow aggregation model in scaling networks to realizing advanced DetNet services for 5GS. Here, we want to point out that the feasibility of applying our flow aggregation scheme indeed depends on the hierarchical nature of a 5GS DetNet node. Had the same aggregation scheme been applied to DetNet nodes that do not have the similar intrinsic hierarchy, the effectiveness could be certainly impaired.

State Maintenance at a 5GS Transit node The 5GS QoS architecture is roughly comprised of three levels, i.e., the UE, the PDU-session, and the QoS-flow levels. Technically, a 5GS DetNet node is of 'composite' nature with a large number of configuration, provisioning, operation and runtime states to maintain. At first glance, this might undermine the state-reduction objective via the flow aggregation for a 5GS DetNet transit node. Fortunately, the 5GS DetNet work owns intrinsically a couple of aspects to handle the challenges: First, also as we have mentioned before, a 5GS node behaves as a transparent entity participating in the DetNet domain. Thus, even having a significant number of states, this can naturally have the 5GS DetNet related states remain hidden from the external entities(and domains). Second, the 3GPP NF TSCTSF exchanges only traffic parameters with the IETF CPF:Detnet Controller, but not the states that are maintained inside a 5GS DetNet node. The external DetNet domain does not care the inside status of a 5GS, nor can it.

Flow Classification & Identification at 5GS node As we have explained so far, the IETF domain CPF:DetNet controller provides traffic parameters & specifications to 3GPP NF TSCTSF. Thus, the SLA requirements of applications in scaling networks could be readily pre-specified in the IETF DetNet CPF, which would then apply the flow classification mapping (to aggregated service classes) and send them over to a 5GS DetNet node to enforce. This model can also relieve the classification burden of a 5GS node in reality. The 5GS has excellent control logics to address flow identification. For example, PDRs (Packet Detection Rules), SDF (Service Data Flow) filters (e.g., IP-filter, MAC-filter, etc.), etc., are all good tools to differentiate flows . Further, the 5GS has standardized powerful procedures for the establishment & update of PDU sessions/QoS flows, which accordingly achieves the flow dynamics (e.g., flow joining & leaving a flow-aggregate as manifested potentially by a PDU session) . Moreover, some QoS parameters, e.g., Aggregated Bit Rate (ABR), may stand at different levels, including UE-ABR, Session-ABR, flow-ABR, etc., that would make the service differentiation & sharing on the aggregated-class (A-Class) level feasible.

Security Considerations Security considerations for DetNet are covered in the DetNet Architecture and DetNet security considerations .

IANA Considerations This document makes no requests for IANA action.

Acknowledgements TBA