Step 3: Associate Ethernet0 to VLAN 10 After configuring VLAN 10 and bringing up interface Ethernet0, the next step is to make Ethernet0 an untagged member of VLAN 10. GONFIG_DB and APPL_DB Figure 5-11 gives a conceptual view of the Redis database updates resulting from the SONiC command: sudo config vlan member add 10 Ethernet0 --untagged The command defines our intent: Ethernet0 should…
Enable Interface Ethernet0 After creating VLAN 10, the next step is to enable interface Ethernet0. The baseline configuration for Ethernet0 already exists in the PORT table of CONFIG_DB. This configuration is populated during the SONiC switch startup process using the platform configuration and configuration files, including config_db.json. When the switch boots, the configuration from…
Introduction In this chapter, we examine what happens under the hood when we configure a Layer-2 VLAN on a SONiC switch, enable two host-facing interfaces, and associate them with the previously created VLAN as untagged access ports. Figure 5-1 illustrates the conceptual configuration in which Host-1 and Host-2 are connected to interfaces Eth1/1 and Eth1/2, respectively. Both interfaces are…
The previous section described at a high level how the config_db.json file is loaded into the Redis CONFIG_DB database in the Database container. Next we look at the topic in slightly more detail. First, we examine how physical port information, such as the port's default speed and lane configuration, is processed during the initial boot before it is programmed into the switch ASIC. Step 1:…
Introduction Before a switch can forward data traffic, its ASIC must first be programmed with the device's port configuration. This includes information such as the number of front-panel ports, their supported speeds, the number of lanes assigned to each port, the speed of each lane, and the administrative and operational state of every port. SONiC relies heavily on a Redis-based database model.…
Figure 3-1 illustrates a simplified SONiC container startup sequence. First, systemd reads the service unit files and evaluates their dependency and ordering directives. It then starts docker.service, making the Docker daemon available. After that, SONiC service containers are started according to the dependencies and ordering rules defined in their service unit files. In this simplified example,…
ONIE-Based SONiC Installation Many switch vendors have added SONiC NOS support to at least part of their switch portfolio. Depending on the vendor and switch model, customers may be able to order a switch with a vendor-customized SONiC version that is supported at the same level as the vendor's own network operating system. Some vendors also allow customers to run the community-based SONiC…
Introduction SONiC (Software for Open Networking in the Cloud) is a Linux-based open-source network operating system that was originally developed at Microsoft and is now maintained by a broader open-source community. Its core idea is that the same network operating system can run on switch platforms from multiple hardware vendors. This reduces vendor lock-in and provides a more consistent…
SONiC is a vendor-neutral, Linux-based network operating system (NOS) that uses a database-driven architecture. Its software components run in multiple containers and exchange information through Redis. In SONiC, several named databases are defined for different functions, and these databases are mapped to Redis logical database IDs. Through this design, configuration data, application state,…
Introduction This chapter explains how to create and deploy a simple SONiC-based Clos topology in WSL using Containerlab. First, we open VS Code from WSL to create and edit a topology definition file. Next, we build the topology by defining nodes (SONiC switches and Linux hosts) and the links between them. Before deploying the lab, we verify the wiring with Containerlab’s built-in topology graph.…
Introduction This chapter explains how to build a SONiC virtual test environment on a Windows computer. First, we enable the required Windows features for WSL 2 and update and verify the WSL installation. Next, we install an Ubuntu distribution and validate that the Linux environment is working correctly, including basic resource checks (CPU, memory, and disk). After the Linux environment is…
Receive Network Processing Unit (Rx NPU) Figure 9-4 illustrates a simplified receive-side processing pipeline, starting from the moment a Packet Header Vector (PHV), constructed by the Rx IFG, is delivered to the Receive Network Processing Unit (Rx NPU). When the PHV arrives at the Rx NPU, it is dispatched to one of the Run-to-Completion (RTC) cores in the Packet Processing Array (PPA). Each RTC…
Virtual Output Queue (VOQ) The Silicon One VOQ Architecture Instead of using dedicated deep interface buffers for packet queuing, Cisco Silicon One utilizes a Centralized Shared Memory architecture paired with a logical Virtual Output Queue (VOQ) mechanism. Because the VOQ concept is implemented within the Ingress (Rx) NPU entity, this queuing stage occurs after the initial ingress lookups but…
Receive Interface Group (Rx IFG) Ingress Pre-Processing and Integrity The Receive Interface Group (Rx IFG) is the ingress pre-processing stage that handles the incoming Ethernet bitstream before the packet enters the Packet Processing Array (PPA) of the Receive Network Processing Unit (Rx NPU) in the Cisco Silicon One architecture. Processing begins at the Rx MAC. The Rx MAC reconstructs…
Introduction The previous chapter introduced the Ultra Ethernet (UE) Transport Layer and its endpoint-centric congestion control mechanisms: Network Signaled Congestion Control (NSCC) and Receiver Credit-based Congestion Control (RCCC). This chapter moves down to the UE Network Layer and introduces Packet Trimming (PT). While node-based approaches rely on NIC-to-NIC feedback loops, Packet Trimming…
Introduction Receiver Credit-Based Congestion Control (RCCC) is a cornerstone of the Ultra Ethernet transport architecture, specifically designed to eliminate incast congestion. Incast occurs at the last-hop switch when the aggregate data rate from multiple senders exceeds the egress interface capacity of the target’s link. This mismatch leads to rapid buffer exhaustion on the outgoing interface,…
Figure 6-14 depicts a demonstrative event where Rank 4 receives seven simultaneous flows (1). As these flows are processed by their respective PDCs and handed over to the Semantic Sublayer (2), the High-Bandwidth Memory (HBM) Controller becomes congested. Because HBM must arbitrate multiple fi_write RMA operations requiring concurrent memory bank access and state updates, the incoming packet rate…
Inflight Packet Adjustment Figure 6-12 depicts the ACK_CC header structure and fields. When NSCC is enabled in the UET node, the PDS must use the pds.type ACK_CC in the prologue header, which serves as the common header structure for all PDS messages. Within the actual PDS ACK_CC header, the pds.cc_type must be set to CC_NSCC. The pds.ack_cc_state field describes the values and states for…
Network-Signaled Congestion Control (NSCC) The Network-Signaled Congestion Control (NSCC) algorithm operates on the principle that the network fabric itself is the best source of truth regarding congestion. Rather than waiting for packet loss to occur, NSCC relies on proactive feedback from switches to adjust transmission rates in real time. The primary mechanism for this feedback is Explicit…
Ultra Ethernet Transport (UET) uses a vendor-neutral, sender-specific congestion window–based congestion control mechanism together with flow-based, adjustable entropy-value (EV) load balancing to manage incast, outcast, local, link, and network congestion events. Congestion control in UET is implemented through coordinated sender-side and receiver-side functions to enforce end-to-end congestion…
Calculating Base RTT [Edit: January 7 2026, RTT role in CWND adjustment process] As described in the previous section, the Bandwidth-Delay Product (BDP) is a baseline value used when setting the maximum size (MaxWnd) of the Congestion Window (CWND). The BDP is calculated by multiplying the lowest link speed among the source and destination nodes by the Base Round-Trip Time (Base_RTT). In addition…
Congestion Control Context Updated 5.1.2026: Added CWND computation example into figure. Added CWND cmputaiton into text. Updated 13.1.2026: Deprectade by: Ultra Ethernet: Congestion Control Context Ultra Ethernet Transport (UET) uses a vendor-neutral, sender-specific congestion window–based congestion control mechanism together with flow-based, adjustable entropy-value (EV) load balancing to…
Introduction Figure 6-1 depicts a simple scale-out backend network for an AI data center. The topology follows a modular design, allowing the network to scale out or scale in as needed. The smallest building block in this example is a segment, which consists of two nodes, two rail switches, and one spine switch. Each node in the segment is equipped with a dual-port UET NIC and two GPUs. Within a…
This section brings together the processes described earlier and explains the packet flow from the node perspective. A detailed network-level packet walk is presented in the following sections.. Initiator – SES Request Packet Transmission After the Work Request Entity (WRE) and the corresponding SES and PDS headers are constructed, they are submitted to the NIC as a Work Element (WE). As part of…
Semantic Sublayer (SES) Operation [Rewritte 12. Dec-2025] After a Work Request Entity (WRE) is created, the UET provider generates the parameters needed by the Semantic Sublayer (SES) headers. At this stage, the SES does not construct the actual wire header. Instead, it provides the header parameters, which are later used by the Packet Delivery Context (PDC) state machine to construct the final…