Worth Reading: LLM Prompts for Network Engineers
Tony Mattke put together a long list of recommendations that might help you get more out of your LLM tokens. Definitely worth reading instead of yelling at the stupid AI .
ipSpace.net blog
Latest posts
Compress netlab Lab Topologies with Dot Notation
netlab is using the Python Box library to make the code easier to read 1 . When I started the project, I hated the way you fetch values from Python dictionaries with stuff like node['ospf']['area'] ; Python Box lets you write node.ospf.area . Even better 2 , you can tell Python Box to create intermediate dictionaries as needed. node.ospf.area = 1 will automatically create the node.ospf dictionary.…
On ACLs and AI-Generated Device Configurations
Last Friday, I had a lovely chat with Steinn Bjarnarson and Urs Baumann , resulting in the NetworkAutoMagic episode 11 . We couldn’t avoid mentioning netlab , the seven layers of ACL hell (which is still balmy compared to the QoS hell), and the gotchas of AI-generated device configurations. Fortunately, I don’t have to go into more details; Steinn published extensive notes , and if you…
netlab 26.08: ArcOS, VPP (FD.io), ACLs, and DNS
netlab release 26.08 brings a few humongous additions: ArcOS support by @roc-ops VPP (FD.io) with FRR or BIRD control plane by @jbemmel SONiC containers (also by @roc-ops ) IPv4/IPv6 access control lists in the routing module by @DanPartelly But wait, there’s more: Read more …
On the Futility of Opening Ansible Issues
Remember the anonymous troll who chided me for writing a rant instead of opening a GitHub issue in the relevant Ansible repository? Well, I decided to be an open-source poster boy when I stumbled upon the next Ansible bug, and ended up feeling like a sweet summer child :( Read more …
Content: New Parameter in Multiple something_config Ansible Modules
Last December, I wrote a pretty ranty post explaining how Ansible release 12 broke (some?) network device configuration playbooks. The inevitable anonymous troll (why are they always anonymous?) couldn’t resist asking whether I opened an issue on GitHub . I didn’t (more about that later), but when the solution to that rant was “ we’re deprecating using templates in src…
Worth Reading: Git Oh-Shit Toolkit
Tony Mattke published a blog post I wish I’d read 10 years ago. His Oh-Shit Toolkit includes several tools one can use after messing up Git branches or commit history. Definitely worth reading, even if you’ve been working with Git for ages.
netlab 26.07: GRE, Wireguard, Graceful Restart, and Scale-Out Labs
The highlights of netlab release 26.07 include: The multiserver plugin by @muddyblack distributes containerlab devices across multiple servers. The GRE tunnel plugin supports GRE tunnels on Cisco IOS, FRR, VyOS, and Junos (vSRX and vJunos-router) (most device implementations done by @ssasso and @jbemmel ). The WireGuard tunnel plugin by @jbemmel supports WireGuard tunnels on FRR. The bgp.session…
netlab: 2000 Pull Requests Later ;))
Jeroen van Bemmel noticed an interesting fact yesterday evening: the netlab GitHub repository has 2000 merged/closed pull requests . I never expected the tiny “ let’s learn Python while working around the complexities of Vagrantfile ” project to get this far ;)) Thanks a million to everyone who contributed , reported a bug , asked a question , or helped us spread the word . You…
Worth Reading: Scripting Good Practices in Python
Just in case you decide to build a simple Python project over the summer: read scripting good practices in Python by Brett Cannon and tell your AI friend to adhere to them ;)
Worth Reading: More VXLAN and EVPN Labs
Ali Bahadır Coşkun continued his EVPN/VXLAN journey. Using the free netlab-powered EVPN/VXLAN labs , he finished the basic EVPN/VXLAN lab (adding EVPN control plane to VXLAN underlay), and then completed the whole VXLAN/IRB/anycast saga . Want to do something similar? The free EVPN/VXLAN lab exercises include six VXLAN labs , almost a dozen EVPN labs , and a few EVPN designs . I might add a lab or…
Worth Checking: EVPN Centralized Anycast Gateway
Daniel Blažek couldn’t resist testing Arista EOS centralized anycast gateway functionality (on top of IPv6 underlay to make it even more fun) and published working device configurations in a GitHub repo . His repository includes a containerlab topology definition, so you can start the lab directly from the repository.
Getting Started with netlab @ Cisco DevNet
Cisco DevNet channel has published an hour-long Getting Started with netlab interview with Suresh Vina , resulting in netlab.tools documentation having more weekly visits than ipspace.net blog for the first time. Thanks a million ;)) If you’re new to netlab , I hope you’ll enjoy the video. If you have any follow-up questions, don’t hesitate to start a discussion .
Time for Another Summer Break?
I can confirm that an old saying is true: the older you are, the faster years pass. Can’t believe it’s time for another summer break. I hope you’ll manage to get away from work, turn off the Internet, and enjoy a few days in your favorite spot with your loved ones! I also promise I won’t be annoying you with boring stuff like EVPN next hops or pointers to AI myth-busters (I…
Worth Reading: Appearing Productive in The Workplace
The Appearing Productive in The Workplace article I stumbled upon is yet another masterful description of how AI slop, used by Expert Beginners, wastes everyone’s time and energy. Try to have fun reading it, even though it may be way too close to the mark.
Anycast-Only Gateways in EVPN Asymmetric IRB
In the previous blog post , I described how ARP works in an EVPN asymmetric IRB environment where the PE devices share an anycast MAC/IP address in addition to a unicast MAC/IP address. Today, let’s see how well things work if the PE devices have only the anycast MAC/IP address: Packet forwarding in an EVPN asymmetric IRB design using only anycast gateways Read more …
Worth Reading: My Network is Talking Back Thanks to SuzieQ MCP
Claudia de Luna published a step-by-step description of how you can use SuzieQ data with an AI agent . That’s definitely interesting, but I found the list of MCP resources at the end of her blog post even more valuable; that’s a keeper even if you never looked at SuzieQ (in which case you REALLY SHOULD ).
Worth Reading: AI Enthusiasts Against AI Skeptics
Charity Majors wrote an excellent article describing AI enthusiasts in a race against time and AI skeptics in a race against entropy . Fair warning: its very first sentence triggered an acute case of PTSD: I recently attended a talk where one of the presenters made some pretty…astonishing claims about what they had achieved by the pure, uncut power of vibe coding. I’ve seen way too many…
Worth Reading: Extending a VLAN with Static VXLAN
Ali Bahadır Coşkun wrote a nice article describing how he mastered extending a VLAN with static VXLAN with the help of free netlab-powered VXLAN labs . The same set of lab exercises includes six VXLAN labs , almost a dozen EVPN labs , and a few EVPN designs . I might add a lab or two during the summer break.
ARP with Anycast Gateways in EVPN Asymmetric IRB
In previous blog posts, I described the ARP issues in EVPN environments, starting with centralized routing , and then asymmetric IRB with unicast (per-leaf-switch) first-hop gateways . Of course, no self-respecting vendor would tell you to do that; anycast gateways are all the rage these days. As always, anycast gateways could mean different things, depending on which vendor documentation you read…
Worth Reading: Leading Intelligent Networks
Chris Grundemann wrote an interesting article arguing that you should structure your network operations around teams, not heroes . Even if you feel you’re perfectly OK with your network being held together by exhausted heroes (and duct tape), it could be a bit harder to deploy network automation in an always-busy hero culture. However, the choice, as they say, is yours.
SR-MPLS with OSPFv2
I started my part of the Segment Routing workshop @ ITNOG10 exploring SR-MPLS with IS-IS ( simple SR-MPLS , dual-stack SR-MPLS , SR-MPLS over unnumbered IPv4 interfaces ). Next step: let’s change the routing protocol to OSPF while using the same network topology: Read more …
AI in Networking with Andrew Yourtchenko
I always wanted to find someone who is more positive about AI than I am, while having solid “can deliver working stuff at scale” credentials. Andrew Yourtchenko definitely fits the bill. I first met him (online) when he was still an engineer in Cisco TAC, and when we finally met in person, he was busy automating the deployment of Cisco Live networking infrastructure . He was also…
Using netlab to Configure Live Devices
Leo Fleskes sent me an interesting question after reading my Generate Partial Device Configurations with netlab blog post: What is stopping us from eventually, given enough usage and coverage, using netlab to configure devices in the live network? In theory, nothing. In practice, you might hit a few hurdles: Read more …
Goodbye, Leaf-and-Spine Networks?
A friend of mine sent me links to a new paper published by AWS engineers , and an associated LinkedIn post which claims: We got lean, resilient, massive aggregation fabrics that provide 33% better throughput with 69% fewer routers, savings 27% of costs, cutting power usage by 40%, and reducing CO2 emissions. The obvious question one should ask after reading the hyperventilated Radical Network…
Worth Reading: Genie Tarpit
Following a link in Martin Fowler ’s Fragments , I stumbled upon Genie Tarpit by Kent Beck – a perfect summary of my experiences with AI coding (code reviews are OK, new code less so). He also provided a good reason for that behavior: The “plausible deniability” task orientation of the genie leaves it claiming success even though the code doesn’t work at all. And the proposed solution?…
netlab 26.06: OSPFv3 on FortiOS, MPLS/VPN on SR Linux
netlab release 26.06 adds OSPFv3 support on FortiOS (by @a-v-popov ) and MPLS/VPN support on SR Linux. We also ensured the installation scripts work on Ubuntu 26.04 (everything else was OK) and updated the installed Vagrant version to 2.4.9 (we’re not using new Vagrant features; you don’t have to upgrade it in an existing installation). Other than that, we added a few improvements and…
Lab: Implementing VRF-Lite with VXLAN
Did you know that you can implement a VRF-Lite design with VXLAN? All you need are devices that can run VRF routing protocols over VXLAN-backed VLAN segments. Compared to the “traditional” VRF-Lite design, in which you need a set of VLANs on every link and every device running the routing protocol for every VRF, the VXLAN-based design needs just IP routing on the core switches,…
Using netlab to Argue with Vendor TAC
A happy netlab user sent me an unexpected use case: they successfully used its multi-vendor capabilities to argue with a vendor TAC. Here’s the gist of the story (edited/anonymized for obvious reasons): They deployed a configuration change that resulted in an unexpected outage. The outage partially disrupted the data center network, so they didn’t have the luxury of collecting data and…
EVPN Centralized Routing with Arista EOS
A month ago, I described ARP issues in EVPN centralized routing design , and Naveen Kumar Devaraj was kind enough to add some Arista EOS implementation details . Today, let’s explore what EVPN routes Arista EOS generates in that scenario. We’ll use a very simple lab topology with a spine switch acting as a router. The leaf switches are layer-2 switches. Packet forwarding in centralized…
Worth Reading: Building the Tools I Wished I Had
Tony Mattke built several networking-focused CLI tools and released them on GitHub . You might find them useful.
SR-MPLS over Unnumbered Interfaces
After the simple SR-MPLS demo and the dual-stack SR-MPLS setup , it was time for the next obvious question: Does SR-MPLS work over unnumbered IPv4 interfaces 1 , assuming the implementation of the underlying routing protocol supports them? Of course it does; let’s go through the details, using the same topology I used throughout the Segment Routing workshop @ ITNOG10. Read more …
130 Years of Wireless Communications
Here’s a short glimpse into the history of telecommunications: in a building at the top of this mountain (barely noticeable blip across the saddle from the radio tower; search for Capo Figari for more details), Guglielmo Marconi conducted experiments in the ~1930s (after inventing the wireless telegraph system in the late 1890s). The original radio could “transmit” at most 40-60…
Worth Reading: Ephemeral BGP Leaks
Doug Madory wrote an interesting article (published on APNIC blog) arguing that we shouldn’t worry about ephemeral BGP leaks that can be observed only during the BGP path hunting process that follows a route withdrawal. I have to disagree with that. It’s never a good idea to ignore a dead canary in the coal mine . While the ephemeral leaks do not impact the end result (after all, the…
SR Linux MPLS/VPN Configuration Notes
After fixing the SR Linux LDP configuration template, I decided to add MPLS/VPN to the netlab SR Linux MPLS features. After all, the one-page recipe seemed simple enough, more so as we already have tested VRF and EVPN templates. Alas, nothing is as simple as it looks. There were two details that tripped me pretty badly. Read more …
Worth Reading: Your Code Is Worthless
Did you manage not to stumble on a dramatic post explaining how someone generated 10,000 lines of code with AI while wasting time on your LinkedIn feed? Congratulations, you’re lucky. However, as Nathaniel Fishel explained in his Your Code Is Worthless article, the “lines of code” is a useless vanity metric that sounds great in a LinkedIn self-promotion, but doesn’t matter…
Public Videos: OpenFlow Deep Dive
Remember OpenFlow , the One Protocol to Bind Them All 1 ? I haven’t heard anyone even mention it in ages, and I never bothered to ask whether anyone is still using it after the dismal results of the 2022 poll . Anyway, if you still have to deal with that ancient blunder, six hours of deep dive videos I recorded a decade ago might still be useful. You can watch them without an ipSpace.net…
Dual-Stack SR-MPLS
After the introduction to SR-MPLS demo I did during the Segment Routing workshop @ ITNOG10, we moved to dual-stack SR-MPLS – can we assign node segment identifiers (SIDs) to IPv4 and IPv6 prefixes? The demo used the same three-router network as the previous one, with IPv4 SIDs starting at one and IPv6 SIDs starting at 101: Read more …
Worth Reading: Agentic AI Setup: Sandboxes and Worktrees
Most of the hyperventilated AI “success stories” are as useful as the “ ANSIBLE!!! ” movement was a few years ago. It’s thus always a pleasure to find someone with well-established software development chops who took the time to describe what works for them. One cannot argue with Mike McQuaid’s credentials (at least if you happen to be using homebrew on MacOS,…
netlab 26.05: BGP-free SRv6 Core, Junos Features
netlab release 26.05 is out. Here are the highlights: Support for global BGP routes with SRv6 next hops (inspired by proof-of-concept by @jvbemmel ) on FRR and IOS XR Junos OSPF/IS-IS route redistribution , VRF IS-IS instances , and OSPF interface parameters Streamline and speed up the FortiOS initial device configuration by @a-v-popov Support for Juniper cSRX container by @leec-666 Read more…
Goodbye, Ubuntu 20.04 (netlab 26.05)
netlab release 26.05 is out. I’ll write about its highlights tomorrow; today, I want to focus on one of its breaking changes: netlab no longer works with Python 3.8 (which reached end-of-life in October 2024), so you can no longer install it on a vanilla Ubuntu 20.04 (which reached end of standard support a year ago). We wanted to get rid of old Python versions for ages, but never did…
Lab: EVPN Asymmetric IRB with Anycast Gateways
I postponed the discussion of ARP issues with EVPN anycast gateways to keep yesterday’s blog post reasonably short. If you’re impatient and want to try that out, I have just the right lab exercise for you; you’ll have to extend VLANs into end-to-end MAC-VRF instances and add IRB and anycast gateways: Explore the lab exercise You can run the lab on your own netlab -enabled…
ARP with EVPN Asymmetric IRB
In a previous blog post , I described the ARP issues you’ll encounter when using centralized routing (on a spine switch) between two EVPN MAC-VRF instances (a fancy name for a VLAN encapsulated in VXLAN or MPLS). That blog post established a baseline that will help us unravel the ARP behavior in a more realistic scenario: asymmetric Integrated Routing and Bridging (IRB). That’s a…
Hands-On Introduction to SR-MPLS
The second demo 1 I did during the Segment Routing workshop @ ITNOG10 illustrated how easy it is to set up and explore a small SR-MPLS network with netlab . The lab topology described a small three-router network (you need three routers to see “true” labels besides the penultimate-hop popping ones): Read more …
MUST READ: The Future of Everything is Lies, I Guess
Kyle Kingsbury published a long (10-part) article about his frustrations with AI, aptly named The Future of Everything is Lies, I Guess . Regardless of where you are on the skeptic-to-fanboy spectrum, I would highly recommend you read it, even if you believe you’ll disagree with everything he wrote.
Reorganized ipSpace.net Segment Routing Resources
I created nine sample SR-MPLS topologies for the ITNOG 10 SR-MPLS workshop , and of course, we ran out of time. I plan to cover those topologies and resulting printouts in a series of blog posts; to prepare for those, I cleaned up and reorganized the Segment Routing blog category, which is now split into two: Segment Routing with MPLS Labels (SR-MPLS) Segment Routing over IPv6 (SRv6) Hope…
On ARP and MAC Aging Timers
Naveen Kumar Devaraj mentioned an interesting fact in his EVPN-related comment : The EOS default ARP timeout is 4 hours, and MAC aging is 5 minutes. Arista is not the only platform using these default values; did you ever wonder where they came from? Read more …
Hmmm: Cloudflare's Automatic Return Routing
A while ago, I found the How Automatic Return Routing solves IP overlap article on Cloudflare’s blog. They evidently have a technology that addresses a pain point well worth solving (access to shared resources from clients using overlapping address ranges). I just hate how they’re selling it. Go read the article first; I’ll wait. OK, here’s what bothers me: the “VRFs…
ARP Issues in EVPN Centralized Routing Design
Adding IRB to a EVPN MAC-VRFs (the fancy way of saying stretched VLANs ) seems like a no-brainer: Add IP addresses to VLAN interfaces Optionally add a shared anycast gateway Declare “Mission Accomplished” (and try to ignore the inevitable phone call at 2 AM on a Sunday night) Making that work in a multi-vendor environment is even more fun 1 , as I sadly discovered when creating the…
SwiNOG 41: It Was Nice to Be Back
Last week’s SwiNOG was (as expected) great fun at a phenomenal location , starting with the first slide of the first presentation: “6 Stages of Network De-sh*tification”. I particularly loved the “talk less, chat more” schedule. The longer breaks gave us plenty of time to catch up with old friends and discuss interesting, sometimes completely unexpected, topics. For…