Flow Aggregation for Enhanced DetNet

Introduction The clearly states that Deterministic Networking (DetNet) operates at the IP layer and delivers service which provides extremely low data loss rates and bounded latency. The DetNet data plane must provide the aggregation of DetNet flows in order to support larger numbers of DetNet flows and improve scalability by reducing the per-hop states. The introduces that the flow aggregation is the ability to aggregate individual flows along with their associated resource control into a large aggregate. It is recommended that the DetNet flow aggregation be enabled for compatible flows with the same or very similar QoS and CoS characteristics via the use of wildcards, masks, prefixes, and ranges. It also suggests the reduction of per-hop states help avoid the per DetNet-flow specific state maintenance in a transit node. It further provides arguments on how DetNet services might be realized in term of delay bound, delay jitter and bandwidth provisioning. Furthermore, the has proposed and expanded two methods of flow aggregation, one being the aggregation via LSP hierarchy and the other to aggregate DetNet flows as a new combined DetNet flow. For enhanced DetNet, has described the data plane enhancement requirements such as the aggregated flow identification in section 4.1. For example, explicit aggregated flow identification in IPv6 networks and the flow identification with service-level aggregation should be supported. In scaling networks, it also should consider the aggregated flows over multi-domains and achieve different levels of co-existed applications with different SLAs requirements which requiring the fine-grained QoS provisioning through flow aggregation. Moreover, the aggregated flows still requires to improve the scalability to avoid the large amount of control signaling and the states maintaining of DetNet flows in enhanced DetNet. This document describes the specific requirements of flow aggregation in enhanced DetNet and provides the enhancement considerations. It also discusses the realization of DetNet flow aggregation for 5GS as well.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Terminology The terminology is defined as .

Objectives & Requirements: Flow Aggregation in Enhanced DetNet

Aggregated Flows across Multi-domains Flow aggregation is recommended in the multi-domain scenario to achieve the end-to-end QoS guarantees for aggregated flow(s) that span across multiple domains. As per , different network implementations may be intended for different application domains, where there is no additional requirements for the coordination. As defined in [ITU-T Y.2122], the network operating parameters of a flow aggregate should be exchanged among different network domains. As shown in Figure 1, the DetNet domain B receiving an aggregated flow should identify the flow and get the service requirements such as the QoS parameters of the flow aggregate.

Individual Flows +-----------------+ +-----------------+ -------> | | | | ...... | DetNet Domain A | Aggregated Flow | DetNet Domain B | -------> | | --------------> | | +-----------------+ +-----------------+

Aggregated Flows with Fine-grained QoS Provisioning The draft specifies that different levels of applications differ in the SLAs requirements such as tight jitter, strict latency, loose latency and so on. While these types of aggregated requirements might bear the coarse-grained nature, individual flows demand differentiated DetNet treatments and more granular QoS forwarding behaviors. A DetNet node or domain adopting multiple forwarding technologies needs to transmit individual flows by aggregating them into a selected treatment solution that corresponds to one of some pre-defined per-hop QoS behaviors, as shown in Figure 2. The DetNet flows with the same level of service requirements can be aggregated to receive collective treatments and forwarding behaviors. The DetNet flows can be aggregated to several pre-defined classes. For example, as per , a 5GS as a logical DetNet node requires to achieve the service requirements and service levels of the aggregated flows, along with the provisioning of fine-grained per-hop behavior (PHB) to each individual flow.

Improve Scalability of Aggregated Flows at Class-aggregate As per , the treatment solutions in data plane can be categorized based on performance and functional characteristics. For example, the category of a solution can be classified based on the traffic granularity, e.g., flow aggregate vs. class aggregate. The class aggregate is provided to simplify the control and accommodate traffic fluctuations by combining flows requiring the same or similar levels of service requirements. The flow aggregation based on the class aggregate could further improve the scalability. As per , there may be a large number of traffic flows in a scaling network, which makes it nearly impossible to achieve the flow-specific state identification. As shown in the Figure 3, the flow identification of aggregated-class can be used to indicate the required treatment and forwarding behaviors without the need to maintain excessive states at transit nodes.

Individual Aggregated Flows +-------------+ Flow(s) +-------------+ +-------------+ -------> | | | | | | .... |DetNet Node A|---------->|DetNet Node B|----->...|DetNet Node N| -------> | | | | | | +-------------+ +-------------+ +-------------+ 'Bucketed' into Large number of Fewer number of classes Individual Flows -----------------> consisting of aggregated flows When DetNet flows are aggregated based on service-class, transit nodes provide deterministic services to a flow aggregate and go through the per-class scheduling without the burden to maintain excessive per-flow states. Still, a transit node performing aggregation should ensure all per-flow service requirements within an aggregated class are achieved. For example, the latency or jitter bounds of an aggregated class shall not exceed the corresponding metrics of any individual flow that has been bucketed into the class. The aggregation based on the class aggregate has data plane and controller plane aspects.

Enhancement Considerations for Flow Aggregation This document describes the enhancement considerations for DetNet flow aggregation including the functions such as flow classification, flow identification and flow coordination.

Flow Classification The DetNet QoS can be achieved by aggregating flows based on DetNet flow-specific traffic classification and providing the traffic-forwarding treatment. The flow classification should consider the flow-specific characteristics such as traffic specification and service requirements. For example, the DetNet flows MAY be classified based on the service SLAs requirements of applications in scaling networks as per . And the services can also be classified into tight/loose latency, large/small burst, periodic/non-periodic and large/small scale services as per . Several classes can be predefined to indicate the different levels of applications with SLAs requirements and each class demands differentiated QoS behaviors and treatment as well as different DetNet capabilities in scaling networks. For the controller plane, the service should be provisioned on an aggregated-class. The resources should be controlled and scheduled on a per-class basis and the routes should be established to meet the service requirements with the allocated resources. The edge nodes must be able to handle admission control for DetNet flows to an aggregated class based on the available resources. For the data plane, when DetNet flows are aggregated to a class, transit nodes provide service to the aggregate and not on a per-DetNet-flow basis. And the transit nodes supporting this type of aggregation should identify the class of the aggregated flows and ensure that individual flows receive the corresponding traffic treatment and forwarding behaviour.

Flow Identification It is required to be dynamic and simplified to ensure the aggregated flows have compatible DetNet flow-specific QoS characteristics. As per , the aggregated flow identification is used to explicitly identify the aggregated flow such as an Flow ID or an Aggregation ID. The deterministic services may also demand different deterministic QoS requirements according to different levels of application and service requirements. The individual flows may be aggregated based on a sharing aggregated level of service specification which is identified by an aggregation level or class. A transit node should provide different levels of treatments and forwarding behaviors based on the aggregation level or class. The aggregation information may be used alone or together with other metadata to indicate the required queuing and forwarding behaviors. The encoding of the aggregation information may reuse the DSCP/TC or existing field such as the TC field in A-Label as per . And it also can be encapsulated with the aggregation-based metadata as per .

Flow Coordination In scaling networks, flow aggregations become more prevalent, with flows frequently joining and leaving, which may potentially lead to accumulated bursts of flows across multiple hops. Such challenges can be mitigated by coordinating packets within aggregated flows such as proportional scheduling and interleaving. Proportional scheduling could allocate transmission opportunities based on flow weights, ensuring that each flow receives a fair share of network resources. Interleaving could achieve micro burst smoothing by rotating the transmission of packets across different flows through timed gates as described in .

Realization of Flow Aggregation for 5GS DetNet The 3GPP in its document has defined and standardized how a 5G system (5GS) may behave as a logical DetNet node, as well as how a 5GS DetNet node may integrate into the IP-domain DetNet as described in . 3GPP has realized the functionalities of the DetNet forwarding sub-layer. As a logical DetNet transit node, a 5GS behaves as a transparent box to external DetNet entities. It can connect to either DetNet end systems or full-fledged IP DetNet domain(s) or both. The 3GPP has demonstrated a ‘composite’ architecture in that a 5GS could act as one or more DetNet nodes upon the integration into the IP DetNet domain. The granularity of determining a 5GS DetNet node is per UPF for each network instance, with the corresponding UPF-id identified as the 5GS DetNet node-id. The 3GPP has implicitly specified two types of DetNet related traffic parameters. One type is the higher-level per-(logical)-node QoS requirements like the node max-latency, max-loss, etc., while the other is more granular settings with which DetNet flow attributes and specifications are mapped to the Flow Description information. The DetNet flow specifications could be based on IP-tuple information, e.g., including IP address, protocol type, ToS, TCP/DUP ports, etc. The document has provided more details. Please note that this draft revolves around the general discussions of the flow aggregations in enhanced DetNet across multiple domains. It emphasizes the objectives & requirements, along with insightful considerations for the possible enhancement to the matter. This indicates the generic principles that are related to the cross-domain flow aggregation as raised in the draft. While the 3GPP defines a 5GS may behave as a logical DetNet (transit) node and the 5GS does own certain advantageous features for a 'composite' DetNet instantiation, the (DetNet) flow aggregation is not an intrinsic characteristics that has been fulfilled in the 5GS. As we explain in the following subsection , the realization of flow aggregation for a 5GS DetNet 'composite' node participating in an enhanced DetNet domains requires the seamless interactions between the IETF domain (DetNet) controller and the 5GS domain counterpart.

Realization of 5GS DetNet Service across Domains 3GPP has so far standardized the forward sub-layer functionality for 5GS. It indicates a 5GS (logical) DetNet node could connect to other end systems and/or IP DetNet domains, together to form a holistic end-to-end DetNet. Thanks to the 'composite' architecture of a 5GS node, along with the interaction between an CPF:DetNet controller in IETF domain and the NF TSCTSF in 3GPP domain , a 5GS node might realize much more advanced DetNet services than a traditional IP DetNet transit node. In scenarios where the (IETF-domain) CPF:Detnet Controller could well divide the DetNet QoS service requirements that are in reality associated with an integrated DetNet domain into multiple QoS sub-requirements that together form the original end-to-end QoS equivalence, a 5GS might be considered as a standalone DetNet (sub-)domain with its own DetNet QoS (sub-)requirements that would be provisioned from the CPF:DetNet controller. The 5GS DetNet QoS (sub-)requirements serve a portion of the original requirements of the integrated DetNet domain. These together form a scaling network to realize the 5GS DetNet service across domains.

5GS QoS Provisioning: Aggregated vs. Fine-grained We have explained previously that the 3GPP has implicitly specified two categories of DetNet related traffic parameters. One type bears the aggregated nature for (5GS DetNet) node-level requirements, while the other addresses the more granular DetNet flow-level attributes and specifications. Evidently, with this kind of two-hierarchy architecture, a 5GS DetNet node could achieve not only the node-level aggregated QoS requirements, but also the more fine-grained flow-level QoS provisioning. This reflects the true value of applying our flow aggregation model in scaling networks to realizing advanced DetNet services for 5GS. Here, we want to point out that the feasibility of applying our flow aggregation scheme indeed depends on the hierarchical nature of a 5GS DetNet node. Had the same aggregation scheme been applied to DetNet nodes that do not have the similar intrinsic hierarchy, the effectiveness could be certainly impaired.

State Maintenance at a 5GS Transit node The 5GS QoS architecture is roughly comprised of three levels, i.e., the UE, the PDU-session, and the QoS-flow levels. Technically, a 5GS DetNet node is of 'composite' nature with a large number of configuration, provisioning, operation and runtime states to maintain. At first glance, this might undermine the state-reduction objective via the flow aggregation for a 5GS DetNet transit node. Fortunately, the 5GS DetNet work owns intrinsically a couple of aspects to handle the challenges: First, also as we have mentioned before, a 5GS node behaves as a transparent entity participating in the DetNet domain. Thus, even having a significant number of states, this can naturally have the 5GS DetNet related states remain hidden from the external entities(and domains). Second, the 3GPP NF TSCTSF exchanges only traffic parameters with the IETF CPF:Detnet Controller, but not the states that are maintained inside a 5GS DetNet node. The external DetNet domain does not care the inside status of a 5GS, nor can it.

Flow Classification & Identification at 5GS node As we have explained so far, the IETF domain CPF:DetNet controller provides traffic parameters & specifications to 3GPP NF TSCTSF. Thus, the SLA requirements of applications in scaling networks could be readily pre-specified in the IETF DetNet CPF, which would then apply the flow classification mapping (to aggregated service classes) and send them over to a 5GS DetNet node to enforce. This model can also relieve the classification burden of a 5GS node in reality. The 5GS has excellent control logics to address flow identification. For example, PDRs (Packet Detection Rules), SDF (Service Data Flow) filters (e.g., IP-filter, MAC-filter, etc.), etc., are all good tools to differentiate flows . Further, the 5GS has standardized powerful procedures for the establishment & update of PDU sessions/QoS flows, which accordingly achieves the flow dynamics (e.g., flow joining & leaving a flow-aggregate as manifested potentially by a PDU session) . Moreover, some QoS parameters, e.g., Aggregated Bit Rate (ABR), may stand at different levels, including UE-ABR, Session-ABR, flow-ABR, etc., that would make the service differentiation & sharing on the aggregated-class (A-Class) level feasible.

Security Considerations Security considerations for DetNet are covered in the DetNet Architecture and DetNet security considerations .

IANA Considerations This document makes no requests for IANA action.

Acknowledgements The authors would like to thank Lou Berger, Janos Farkas for their review, suggestions and comments to this document.