Recent content on BGP.guru
Important things to know when trying to test or use MPLS Traffic Engineering in a Mikrotik RouterOS 7.x network. Config Required! You need both an MPLE TE area, and an MPLE TE address defined! If either is not defined, you will have problems establishing an traffic engineering tunnel. UI Issues with Hops Winbox (3.x and 4.x both) AND the web UI are currently broken and can’t build a list of…
I ran into a small but important detail while working with Mikrotik DHCP server lease logging: scripts only fire on new lease, or lease expiry/termination. Similarly, RADIUS accounting events will only fire then. Any configuration for this type of setup needs to exist in advance of leases being handed out, or existing leases will not be seen at all. If long term stable leases are likely in your…
I did a talk about my experience going down the Anycast rabbit hole side quest at The Long Con this weekend. In preparation for this talk I built a fleet of new Debian 13 nodes, using the Debian provided KVM image which supports cloud-init. Getting More Loopbacks Sooooooo… I’ve been using Debian for a fairly long time, since Potato or Woody. I’ve been using…
I’ve been testing out using RouterOS User Manager as a RADIUS server for a small network. It has been working fairly well, but I did run into one issue (twice) so far… If you make a RADIUS request to a RADIUS server on its loopback address, the reply will come back from a completely different (interface) IP address, which no OS or client is expecting. The solution is to use the…
Introduction A simple test of the container functionality in RouterOS 7. We build a custom go binary, build a container for it, export that container, import that container on the router, and run it! I’m not a Docker guy at all. This is my first steps into actually using it. Lab Setup Mikrotik RB5009UPr+S+ USB SanDisk 3.2Gen1 32GB Firmware versions tested: 7.19.2, 7.19.3, 7.20beta5 and…
Introduction MikroTik has been known in the WISP wireless world for a long time for their wireless PTP and PTMP products. More recently they have also started making WiFi products aimed at home, office, and enterprise networks. They also have several series of their “Cloud Router Switch” (CRS) switching line. This is about my experiences with the CRS3xx series, specifically the…
Introduction This is a story about when I had duplicate MAC addresses on a network, and the particular way the chaos ensued. The Topology The topology is a common enterprise scenario. Site Alpha Site Beta Edge FWs 1 Edge FWs 2 Core FWs 1 Core FWs 2 Core SWs 1 Core SWs 2 Two sites, with identical topology of paired firewalls in HA, connected together at layer-2 via the Core switches.
Introduction This is an example policy/config I’ve been working on for several years. It supports Transit (DFZ), IXP Peering, and giving transit to customers. BGP Policy This policy tries to follow all of the best practices. Roles customer 65551:0:2 transit 65551:0:6 peering 65551:0:8 + 65551:0:IXP_ID Transit Policy Inbound Transit_Name_In Protect_In Size validation Reject Customer Policy…
Introduction One of the complexities that comes up when you manage a large number of routers is keeping their configs all in sync. Specifically any commonly access-lists that are used. I wanted to solve this using the Mikrotik API, but that has changed formats between v6 and v7 so its a bit more complex to cover off both versions with one codebase at this time. However by pulling instead of…
Introduction I recently acquired a Mikrotik CCR2004-1G-12S+2XS, which is a much faster, much more modern router than the CCR1036-8G-2S+ routers I was using previously. This gives me enough router CPU power to run BGP with full routes, and RPKI validate the routing table. I looked at the software options available to me, and I picked routinator as my first RTR software to test. Router Config It is…
This is a fifth in a blog series about DNS, specifically to address changes in exabgp 4.x, and how its built in healthcheck works. Getting Started I guess the easiest way to begin is to lay out a sample config. This is going to have 100.64.100.64 advertised from 192.0.2.135/2001:db8::135. Extrapolate this out sideways to meet your needs. /etc/dnsdist/dns-lb-1 This config sets up a healthcheck on…
Sometimes the easiest way to geo fence an application is to just whitelist certain ISP’s networks only. Nginx has a handy way to map IPs into subnets using the geo module. Requirements job/aggregate6 to aggregate/unique-ify the prefix lists that are generated OpenBSD Looking Glass to get a list of prefixes by source-as vhost.conf geo $geolimit { default 1; include…
I’ve written about OpenBSD and BGP Looking Glasses previously and before that . OpenBSD has since removed nginx from base , and replaced it with their own httpd. This is OK with me since I prefer having my OpenBSD systems fully self-contained and running from base. Install your system as you choose, I did a fairly default install as per the FAQ . My hardware in this case is virtual VMware…
I had a (perhaps common) misunderstanding about how MPLS applied labels. I thought that MPLS would stack up labels for the entire path (so like 6 hops, 5 labels). This of course would affect MTU of MPLS packets, especially on the originating edge. While reading the excellent presentation Demystifying IPv6 Over MPLS by Kam Agahian, I discovered that my understanding of how MPLS worked was actually…
Introduction I ran into an strange issue today. The problem was reported as “can’t upload on a speed test”. I started looking into the reported speed test issue by doing my own speed test between routers. I got expected speeds. Then did a reboot of everything in between. Same result reported, a speed test was failing to complete the upload part. Tested without the router in place…
Introduction MikroTik has been known in the WISP wireless world for a long time for their wireless PTP and PTMP products. However more recently they have also started making WiFi products aimed at home, office, and enterprise networks. These products can be centrally managed with Controlled Access Point system MANager, or CAPsMAN. CAPsMAN Forwarding Options CAPsMAN offers both local forwarding at…
I was recently read into a Problem for which the particular vendor was downplaying the severity of the issue and the usual games of Enterprising ‘saving face.’ They were haranged into providing a means for detecting the failure state and gave ‘us’ a script to run. This is where I get involved as the interested parties had no infrastructure do run said tool and since no real…
This blog post is about an actual real life, local network situation that happened several days ago. This is an excellent learning example about a perfect storm of small things causing a network outage. It is also an excellent learning example of how logs of what happened in public BGP via MRT files can be extremely useful. The Summary AS53443 was turning up a second BGP transit provider, and…
I’ve spent some time trying to manually install Fastnetmon community edition on Ubuntu 18.04 Bionic Beaver. I was unsuccessful. However, I didn’t do an apt-cache search fastnetmon . There is a package in universe . Bionic Beaver also includes exabgp 4.0.2 by default. I installed both today as a test, copied the config from my existing install, and started the service. Sending some…
I’ve been logging BGP route updates into MySQL for some time but this doesn’t scale well, and on my hobby VM system, requires that I dump the DB and start over once a day. One of the things I’ve wanted to do for a long time is to be able to work with MRT files – to injest route updates from MRT files, and once I had that capability, to produce my own MRT files.
This year I’ve build two totally different MPLS Service Provider networks. One network has limited MPLS with VPLS on MikroTik, the other is a full MPLS L3VPN on HPE Comware gear. Articles in this Series Normally MPLS is abstracted out on network diagrams as one or more clouds with MPLS written on them. In those networks, it is someone else’s job to run that. This blog series is about…
Genes Telecom is an established residential and business Service Provider in Northern Manitoba. Specifically The Pas and surrounding area (Cranberry Portage, Wanless, Bakers Narrows, Rocky Lake, Clearwater Lake, & Flin Flon), Thompson, and Snow Lake. Before A previous network redesign project had taken the network from a flat layer 2 network to a fully routed OSPF network, using BGP externally.…
AS112 provides an anycasted reverse DNS sink hole for the private addresses set aside in RFC1918 and RFC6890. Using BIRD to inject an AS112 instance into your own system is quick, easy, and painless! This is nearly identical to using exabgp, except that in this case we’ll be installing routes to the kernel. Requirements BGP: your own autonomous system, plus we’ll be operating AS112…
AS112 provides an anycasted reverse DNS sink hole for the private addresses set aside in RFC1918 and RFC6890. Using exabgp to inject an AS112 instance into your own system is quick, easy, and painless! Requirements BGP: your own autonomous system, plus we’ll be operating AS112 which will announce/service the following prefixes: 192.175.48.0/24 (RFC6304) 192.31.196.0/24 (RFC7534)…
I wrote a small web application which I’m calling “PSaas” – Port Scan as a Service. This webapp was an opportunity for me to do some coding and interaction with RabbitMQ. RabbitMQ has come up a few times for me lately. First, RabbitMQ was mentioned by Woulter Milterburg in a RIPE Labs article describing the next generation of Remote Route Collectors , then Robert Keizer…
OSX has a built-in TFTP server, but starting it the OSX way may not be 100% obvious to you. sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist sudo launchctl start com.apple.tftpd The default TFTP path is /private/tftpboot . Stopping it is the reverse of the first step. sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist
Over the past four or so years I’ve been researching BGP routing in Canada. My first interest in studying Canadian BGP came as a result of studying historical BGP hijacks. I talk about this during my BSidesWpg 2013 talk: The practical applications of my study have taken several forms as I learn what works and what doesn’t work (at all).
This is a fourth in blog series about DNS, specifically awesome things that can be done with dnsdist. This one was inspired by this blog post about exabgp and healthchecks , as well as this custom lua dnsdist load balancing policy . Routing, Front and Back Ends BGP: AS395089 on router, AS65101 w/ exabgp/dnsdist load balancers 4x dnsdist load balancers as client facing DNS resolvers 4x PowerDNS…
I’ve often wished for a way of logging pings when troubleshooting connectivity problems. Today I revisited the problem using fping. fping and fping6 both have options to run a continuous ping ( -l ), and when combined with -D to insert the unix timestamp, can log the latency and loss to a destination in 1s increments. When working with unix timestamps, the following one-liner is handy: perl…
As part of the ISP services I operate for myself, Recursive DNS server IP addresses are part of this. It is annoying when a DNS server fails for some reason and lookups have a delay to fail over to the other IP. Operating systems aren’t always intelligent about it either. Some try each and every time, others remember state from previous queries. Before 2x DNS Server IPs were handled, 1 each,…
I recently setup IPv6 first-hop redundancy in my colocated environment using MikroTik and VRRP. It didn’t work the same way I’d come to expect from using it in IPv4 environments and I set out to figure out why. Expectations My expecations for VRRP and IPv6 were the same as IPv4, that a backup router would take over the default gateway IP address on a subnet and the mac address…
DDoS attacks continue to be a wide-spread problem on the internet. Their size has grown over the past few years to where BGP Blackholing to reduce collateral damage has become widespread. As more and more networks built support for BGP Blackholing – each with their own BGP community – it became clear that there was a need for a standardized “well known” community for BGP…
General AS Information Hextet Systems (as395089) is a hobby/research ASN that I operate. It operates out of 1 datacenter in downtown Winnipeg (currently). One of the users of the IP space that Hextet Systems advertises via BGP is Coldhak who operates several Tor exit relays. These Tor exit relays attract the occasional retaliatory DDoS attack. I also operate “IPv6 Tunnels for Nerds”…
One of the things I noticed when I received a 32-bit ASN was that I was unable to use my ASN in my BGP communities. I’m not the only one who has noticed this, and work has begun on a Draft Standard, Large BGP Community to address this shortcoming. RFC Draft The draft has gone thru several revisions, 00 thru 03 have been released, and 04 is being worked on at the moment on Github .
I ran into an obscure limitation in PowerDNS 4.0 authoritative server recently. I have one nameserver which also acts as a slave to two other sets of zones with are transfered using AXFR. Some of those zones are DNSSEC enabled, and PowerDNS is only able to handle DNSSEC on the first backend loaded. This was causing several forward and reverse zones to fail to serve the DNSSEC records along with…
This is a second in blog series about DNS, specifically awesome things that can be done with dnsdist. Front and Back Ends 2x dnsdist load balancers as client facing DNS resolvers 2x PowerDNS recursor backends dnsdist Configuration The config for dnsdist is very simple, skipping over the binds and ACLs as they’re not relevant here, below is the parts of the config that matter:…
If you’re using Slack then you should already know how easy it is to integrate almost anything into slack using its web APIs. If you’re not already using Slack, what are you waiting for? As a Slack user and a network administrator I often find myself leaving Slack to check a looking glass, or to whois something. A while back I made an API that supplies various BGP related information…
I recently installed VMware ESXi 6.0 Update 02 aka VMware vSphere Hypervisor on a server, and using the new VMware Host Client at https://<hostip>/ui/ was able to fully set up a Debian 8/Jessie guest using only the trial license. The HTML console was quite functional, and the OSX Remote Console application also launched and connected correctly and quickly. Mac Admin Friendly I now no longer have a…
I first came across dnsdist in a NANOG post in the discussion of exploitation of a BIND DOS bug last summer. Jared Mauch had recommended dnsdist to easily implement DNS backend diversity. I was interested at the time. but wasn’t doing enough DNS at the time to find time to play around with it. I recommend using DNSDIST to balance traffic at a protocol level as you can have implementation…
Yesterday morning the company I own, Hextet Systems, was assigned AS 395089 from ARIN’s pool of ASNs. By mid-afternoon I had turned up BGP with my first uptream for IPv6, and was advertising a /44. By 11PM I had finished setting up IPv4 BGP as well and was advertising my /24. I had succeeded in turning up an ASN fully on the very first day. I’ve had the fun of setting up several…
I operate “IPv6 Tunnels for Nerds” from my routers and route various friends some IPv6 space over MikroTik EoIP Tunnels. Before IP space was staticly routed by my upstream, and routed to a router I controlled for v4/v6 tunneling. After IPv4 & IPv6 space was announced via public BGP to the global BGP routing table.
Earlier this month the local Internet Exchange I’m involved with received some reports of spam coming from the MBIX Peering LAN IP space. Normally an Internet Exchange’s Peering LAN IP space is not announced to the world via BGP. Only participants who have a next-hop in that particular network need to know about its existence. Upon investigation it turns out that someone spun up this…
Genes Telecom is an established residential and business Service Provider in Northern Manitoba. Specifically The Pas and surrounding area (Cranberry Portage, Wanless, Bakers Narrows, Rocky Lake, Clearwater Lake, & Flin Flon), Thompson, and Snow Lake. Before Genes Telecom had IP space routed to them via their upstream network(s). After Genes Telecom operates AS394989 as part of the global BGP route…
The makers of League of Legends, Riot Games, have posted a very interesting engineering blog about how they’ve engineered their AS for optimal game performance. They mention important things like, buying from a Tier 1 doesn’t solve all your problems, and that peering at IX’s is super critical to their success.
Otherwise known as Filtering BGP Advertisements Using BGP Communities For Fun and Profit but that made the layout terrible. Filtering BGP advertisements using communities is a simple and effective way to control your advertisements. This can prevent common BGP route leaks with AS-PATHs that look like SOMEAS LEAKER-TRANSIT1 LEAKER LEAKER-TRANSIT2 PEER LEAKER-CUSTOMER when filtering transit sessions…
“But can you make the alerts look nicer?” I’ve always liked the simple, plain text, easily customized commands used for email and SMS alerting in Nagios. I recently set up SMS alerting using nagios-twilio and some custom alert commands. Knowing this, I knew it would be possible to add HTML formatted alerts, and easily too. I did a quick google and found the following blog post…
I have been using Nagios extensively to monitor infrastructure that I am interested in for about 10 years now, and each time I build a net-new Nagios monitoring system I version the configuration. I am up to Generation 4 now which includes: Approx 75 hosts at the moment Approx 130 service checks Service checks for: Generic TCP services (Tor ORPorts for example) Ping checks (v4 and v6) HTTP/HTTPS…
I recently renamed my internal LAN domain name. For some crazy reason I’d thought .int was not a public TLD and didn’t check at all before using that before the last time I renamed my internal LAN. I had no issues for several years, but I felt with the holidays it was time to move away from this invalid domain internally to something valid. I chose ciscodude.co this time and registered…
I’ve been waiting to add some features to ipquail.com for a while, but the way in which I was handling API endpoints at the moment needed to change before I could accommodate anything fancier – at the moment I was using simple Server Side Includes (“SSI” in 90’s Apache web server terminology) and some mime-type modifications to fake API endpoints. This needed to…
This post is a guest blog by Manitoba Internet Exchange Network Administrator, Jonathan Stewart . This post was brought to you by a real situation at MBIX. (This procedure is on its way to becoming a BCP RFC as well. As of January 2018, draft-ietf-grow-bgp-session-culling-05 is a draft with lots of support on the IETF GROW mailing list.) How to gracefully shutdown BGP on Layer 2 When you want to…