539 lines
30 KiB
HTML
539 lines
30 KiB
HTML
<!DOCTYPE html>
|
|
|
|
|
|
<html class="no-js" lang="en">
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<title>Tools to explore BGP</title>
|
|
<meta name="author" content="Julia Evans">
|
|
<meta name="HandheldFriendly" content="True">
|
|
<meta name="MobileOptimized" content="320">
|
|
<meta name="description" content="Tools to explore BGP">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
|
|
<meta property="og:title" content='Tools to explore BGP'>
|
|
<meta property="og:type" content="website" />
|
|
<meta property="og:url" content="https://jvns.ca/blog/2021/10/05/tools-to-look-at-bgp-routes/" />
|
|
<meta property="og:site_name" content="Julia Evans" />
|
|
|
|
<link rel="canonical" href="https://jvns.ca/blog/2021/10/05/tools-to-look-at-bgp-routes/">
|
|
<link href="/favicon.ico" rel="icon">
|
|
|
|
<link href="/stylesheets/screen.css" rel="preload" type="text/css" as="style">
|
|
|
|
<link href="/stylesheets/screen.css" media="screen, projection" rel="stylesheet" type="text/css">
|
|
<link href="/stylesheets/print.css" media="print" rel="stylesheet" type="text/css">
|
|
|
|
|
|
<link href="/atom.xml" rel="alternate" title="Julia Evans" type="application/atom+xml">
|
|
|
|
|
|
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.css" integrity="sha384-vKruj+a13U8yHIkAyGgK1J3ArTLzrFGBbBc0tDp4ad/EyewESeXE/Iv67Aj8gKZ0" crossorigin="anonymous">
|
|
<script defer data-domain="jvns.ca" src="https://plausible.io/js/script.js"></script>
|
|
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.js" integrity="sha384-PwRUT/YqbnEjkZO0zZxNqcxACrXe+j766U2amXcgMg5457rve2Y7I6ZJSm2A0mS4" crossorigin="anonymous"></script>
|
|
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/contrib/auto-render.min.js" integrity="sha384-+VBxd3r6XgURycqtZ117nYw44OOcIax56Z4dCRWbxyPt0Koah1uHoK0o4+/RRE05" crossorigin="anonymous" onload="renderMathInElement(document.body);"></script>
|
|
|
|
<script defer type="text/javascript">
|
|
window.heap=window.heap||[],heap.load=function(e,t){window.heap.appid=e,window.heap.config=t=t||{};var r=document.createElement("script");r.type="text/javascript",r.async=!0,r.src="https://cdn.heapanalytics.com/js/heap-"+e+".js";var a=document.getElementsByTagName("script")[0];a.parentNode.insertBefore(r,a);for(var n=function(e){return function(){heap.push([e].concat(Array.prototype.slice.call(arguments,0)))}},p=["addEventProperties","addUserProperties","clearEventProperties","identify","resetIdentity","removeEventProperty","setEventProperties","track","unsetEventProperty"],o=0;o<p.length;o++)heap[p[o]]=n(p[o])};
|
|
heap.load("2242143965");
|
|
</script>
|
|
</head>
|
|
<body>
|
|
<div id="skiptocontent">
|
|
<a href="#main">Skip to main content</a>
|
|
</div>
|
|
<div id="wrap">
|
|
<header role="banner">
|
|
<hgroup>
|
|
<h1><a href="/">Julia Evans</a></h1>
|
|
</hgroup>
|
|
<ul class="header-links">
|
|
<li><a href="/about">About</a></li>
|
|
<li><a href="/talks">Talks</a></li>
|
|
<li><a href="/projects/">Projects</a></li>
|
|
<li><a rel="me" href="https://social.jvns.ca/@b0rk">Mastodon</a></li>
|
|
<li><a href="https://bsky.app/profile/b0rk.jvns.ca">Bluesky</a></li>
|
|
<li><a href="https://github.com/jvns">Github</a></li>
|
|
</ul>
|
|
</header>
|
|
<nav role="navigation" class="header-nav"><ul class="main-navigation">
|
|
<li><a href="/categories/favorite/">Favorites</a></li>
|
|
<li><a href="/til/">TIL</a></li>
|
|
<li><a href="https://wizardzines.com">Zines</a></li>
|
|
<li class="subscription" data-subscription="rss"><a href="/atom.xml" rel="subscribe-rss" title="subscribe via RSS">RSS</a></li>
|
|
</ul>
|
|
</nav>
|
|
<div id="main">
|
|
<div id="content">
|
|
|
|
|
|
<div>
|
|
<article class="hentry" role="article">
|
|
<header>
|
|
<h1 class="entry-title">Tools to explore BGP</h1>
|
|
|
|
<div class="post-tags">
|
|
|
|
</div>
|
|
<p class="meta sans">
|
|
<time class="date" datetime="2021-10-05T08:53:24" pubdate data-updated="true">
|
|
|
|
October 5, 2021
|
|
|
|
</time>
|
|
</p>
|
|
</header>
|
|
<main>
|
|
<p>Yesterday there was a big Facebook outage caused by BGP. I’ve been vaguely
|
|
interested in learning more about BGP for a long time, so I was reading a
|
|
couple of articles.</p>
|
|
<p>I got frustrated because none of the articles showed me how I could actually
|
|
look up information related to BGP on my computer, so I <a href="https://twitter.com/b0rk/status/1445199475195236356">wrote a tweet asking for tools</a>.</p>
|
|
<p>I got a bunch of useful replies as always, so this blog post shows some tools
|
|
you can use to look up BGP information. There might be an above average number
|
|
of things wrong in this post because I don’t understand BGP that well.</p>
|
|
<h3 id="i-can-t-publish-bgp-routes" class="post-heading">
|
|
<a href="#i-can-t-publish-bgp-routes">
|
|
I can’t publish BGP routes
|
|
</a>
|
|
</h3>
|
|
<p>One of the reasons I’ve never learned much about BGP is – as far as I know, I don’t have access to publish BGP routes on the internet.</p>
|
|
<p>With most networking protocols, you can pretty trivially get access to
|
|
implement the protocol yourself if you want. For example you can:</p>
|
|
<ul>
|
|
<li>issue your own TLS certificates</li>
|
|
<li>write your own HTTP server</li>
|
|
<li>write your own TCP implementation</li>
|
|
<li>write your own authoritative DNS server for your domain (I’m trying to do that right now for a small project)</li>
|
|
<li>set up your own certificate authority</li>
|
|
</ul>
|
|
<p>But with BGP, I think that unless you own your own ASN, you can’t publish routes yourself!
|
|
(you <em>could</em> implement BGP on your home network, but that feels a bit boring to
|
|
me, when I experiment with things I like them to actually be on the real internet).</p>
|
|
<p>Anyway, even though I can’t experiment with it, I still think it’s super
|
|
interesting because I love networking, so I’m going to show you some tools I
|
|
found to learn about BGP :)</p>
|
|
<p>First let’s talk through some BGP terminology though. I’m going to go pretty
|
|
fast because I’m more interested in the tools and there are a lot of high level
|
|
explanations of BGP out there (like this <a href="https://blog.cloudflare.com/october-2021-facebook-outage/">cloudflare post</a>).</p>
|
|
<h3 id="what-s-an-as-autonomous-system" class="post-heading">
|
|
<a href="#what-s-an-as-autonomous-system">
|
|
What’s an AS (“autonomous system”)
|
|
</a>
|
|
</h3>
|
|
<p>The first thing we need to understand is an AS. Every AS:</p>
|
|
<ol>
|
|
<li>is owned by an organization (usually a large organization like your ISP, a government, a university, Facebook, etc)</li>
|
|
<li>controls a specific set of IP addresses (for example my ISP’s AS includes 247,808 IP addresses)</li>
|
|
<li>has a number (like 1403)</li>
|
|
</ol>
|
|
<p>Here are some observations I made about ASes just by doing some experimentation:</p>
|
|
<ul>
|
|
<li>Some fairly big tech companies don’t have their own AS. For example, I
|
|
looked up Patreon on BGPView, and as far as I can tell they don’t own as AS
|
|
– their main site (patreon.com,
|
|
<a href="https://bgpview.io/ip/104.16.6.49">104.16.6.49</a>) is in Cloudflare’s AS.</li>
|
|
<li>An AS can include IPs in many countries. Facebook’s AS (<a href="https://bgpview.io/asn/32934#prefixes-v4">AS32934</a>) definitely has IP addresses in Singapore, Canada, Nigeria, Kenya, the US, and more countries.</li>
|
|
<li>It seems like IP address can be in more than one AS. For example, if I look up <a href="https://bgpview.io/ip/209.216.230.240">209.216.230.240</a>, it has 2 ASNs associated with it – AS6130 and AS21581. Apparently when this happens the more specific route takes priority – so packets to that IP would get routed to AS21581.</li>
|
|
</ul>
|
|
<h3 id="what-s-a-bgp-route" class="post-heading">
|
|
<a href="#what-s-a-bgp-route">
|
|
what’s a BGP route?
|
|
</a>
|
|
</h3>
|
|
<p>There are a lot of routers on the internet. For example, my ISP has routers.</p>
|
|
<p>When I send my ISP a packet (for example by running <code>ping 129.134.30.0</code>), my ISP’s routers
|
|
needs to figure out how to actually get my packet to the IP address
|
|
<code>129.134.30.0</code>.</p>
|
|
<p>The way the router figures this out is that it has a <strong>route table</strong> – it has a list
|
|
of a bunch of IP ranges (like <code>129.134.30.0/23</code>), and routes it knows about to
|
|
get to that subnet.</p>
|
|
<p>Here’s an example of a real route for <code>129.134.30.0/23</code>: (one of Facebook’s subnets). This one isn’t from my ISP.</p>
|
|
<pre><code> 11670 32934
|
|
206.108.35.2 from 206.108.35.254 (206.108.35.254)
|
|
Origin IGP, metric 0, valid, external
|
|
Community: 3856:55000
|
|
Last update: Mon Oct 4 21:17:33 2021
|
|
</code></pre>
|
|
<p>I think that this is saying that one path to <code>129.134.30.0</code> is through the
|
|
machine <code>206.108.35.2</code>, which is on its local network. So the router might send
|
|
my ping packet to <code>206.108.35.2</code> next, and then <code>206.108.35.2</code> will know how to
|
|
get it to Facebook. The two numbers at the beginning (<code>11670 32934</code>) are ASNs.</p>
|
|
<h3 id="what-s-bgp" class="post-heading">
|
|
<a href="#what-s-bgp">
|
|
what’s BGP?
|
|
</a>
|
|
</h3>
|
|
<p>My understanding of BGP is very shaky, but it’s a protocol that companies use to
|
|
advertise BGP routes.</p>
|
|
<p>What happened yesterday with Facebook is that they basically made BGP
|
|
announcements withdrawing all their BGP routes, so every router in the world
|
|
deleted all of its routes related to Facebook, so no traffic could get there.</p>
|
|
<p>Okay, now that we’ve covered some basic terminology, let’s talk about tools you
|
|
can use to look at autonomous systems and BGP!</p>
|
|
<h3 id="tool-1-look-at-your-isp-s-as-with-bgpview" class="post-heading">
|
|
<a href="#tool-1-look-at-your-isp-s-as-with-bgpview">
|
|
tool 1: look at your ISP’s AS with BGPView
|
|
</a>
|
|
</h3>
|
|
<p>To make this AS thing less abstract, let’s use a tool called <a href="https://bgpview.io">BGPView</a> to look at a real AS.</p>
|
|
<p>My ISP (EBOX) owns <a href="https://bgpview.io/asn/1403">AS 1403</a>. Here are the <a href="https://bgpview.io/asn/1403#prefixes-v4">IP addresses my ISP owns</a>. If I look up my
|
|
computer’s public IPv4 address, I can check that it’s one of the IP addresses
|
|
my ISP owns – it’s in the <code>104.163.128.0/17</code> block.</p>
|
|
<p>BGPView also has this graph of how my ISP is connected to other ASes</p>
|
|
<img src="https://jvns.ca/images/ebox-graph.png">
|
|
<h3 id="tool-2-traceroute-a-and-mtr-z" class="post-heading">
|
|
<a href="#tool-2-traceroute-a-and-mtr-z">
|
|
tool 2: <code>traceroute -A</code> and <code>mtr -z</code>
|
|
</a>
|
|
</h3>
|
|
<p>Okay, so we’re interested in autonomous systems. Let’s see which ASes I go through from</p>
|
|
<p><code>traceroute</code> and <code>mtr</code> both have options to tell you the ASN for every IP you go through. The flags are <code>traceroute -A</code> and <code>mtr -z</code>, respectively.</p>
|
|
<p>Let’s see which autonomous systems I go through on my way to facebook.com with <code>mtr</code>!</p>
|
|
<pre><code>$ mtr -z facebook.com
|
|
1. AS??? LEDE.lan
|
|
2. AS1403 104-163-190-1.qc.cable.ebox.net
|
|
3. AS??? 10.170.192.58
|
|
4. AS1403 0.et-5-2-0.er1.mtl7.yul.ebox.ca
|
|
5. AS1403 0.ae17.er2.mtl3.yul.ebox.ca
|
|
6. AS1403 0.ae0.er1.151fw.yyz.ebox.ca
|
|
7. AS??? facebook-a.ip4.torontointernetxchange.net
|
|
8. AS32934 po103.psw01.yyz1.tfbnw.net
|
|
9. AS32934 157.240.38.75
|
|
10. AS32934 edge-star-mini-shv-01-yyz1.facebook.com
|
|
</code></pre>
|
|
<p>This is interesting – it looks like we go directly from my ISP’s AS (1403) to
|
|
Facebook’s AS (32934), with an “internet exchange” in between.</p>
|
|
<small>
|
|
<p>I’m not sure what an <a href="https://en.wikipedia.org/wiki/Internet_exchange_point">internet
|
|
exchange</a> is but I know
|
|
that it’s an extremely important part of the internet. That’s going to be for
|
|
another day though. My best guess is that it’s the part of the internet that
|
|
enables “peering” – like an IX is a server room with a gigantic switch with
|
|
infinite bandwith in it where a bunch of different companies put their
|
|
computers so they can send each other packets.</p>
|
|
</small>
|
|
<h3 id="mtr-looks-up-asns-with-dns" class="post-heading">
|
|
<a href="#mtr-looks-up-asns-with-dns">
|
|
mtr looks up ASNs with DNS
|
|
</a>
|
|
</h3>
|
|
<p>I got curious about how mtr looks up ASNs, so I used strace. I saw that it looked like it was using DNS, so I ran <a href="https://github.com/jvns/dnspeep/">dnspeep</a>, and voila!</p>
|
|
<pre><code>$ sudo dnspeep
|
|
...
|
|
TXT 1.190.163.104.origin.asn.cymru.com 192.168.1.1 TXT: 1403 | 104.163.176.0/20 | CA | arin | 2014-08-14, TXT: 1403 | 104.163.160.0/19 | CA | arin | 2014-08-14, TXT: 1403 | 104.163.128.0/17 | CA | arin | 2014-08-14
|
|
...
|
|
</code></pre>
|
|
<p>So it looks like we can find the ASN for <code>104.163.190.1</code> by looking up the <code>txt</code> record on <code>1.190.163.104.origin.asn.cymru.com</code>, like this:</p>
|
|
<pre><code>$ dig txt 1.190.163.104.origin.asn.cymru.com
|
|
1.190.163.104.origin.asn.cymru.com. 13911 IN TXT "1403 | 104.163.160.0/19 | CA | arin | 2014-08-14"
|
|
1.190.163.104.origin.asn.cymru.com. 13911 IN TXT "1403 | 104.163.128.0/17 | CA | arin | 2014-08-14"
|
|
1.190.163.104.origin.asn.cymru.com. 13911 IN TXT "1403 | 104.163.176.0/20 | CA | arin | 2014-08-14"
|
|
</code></pre>
|
|
<p>That’s cool! Let’s keep moving though.</p>
|
|
<h3 id="tool-3-the-packet-clearing-house-looking-glass" class="post-heading">
|
|
<a href="#tool-3-the-packet-clearing-house-looking-glass">
|
|
tool 3: the packet clearing house looking glass
|
|
</a>
|
|
</h3>
|
|
<p>PCH (“packet clearing house”) is the organization that runs a lot of internet
|
|
exchange points. A “looking glass” seems to be a generic term for a web form
|
|
that lets you run network commands from another person’s computer. There are
|
|
looking glasses that don’t support BGP, but I’m just interested in ones that
|
|
show you information about BGP routes.</p>
|
|
<p>Here’s the PCH looking glass: <a href="https://www.pch.net/tools/looking_glass/">https://www.pch.net/tools/looking_glass/</a>.</p>
|
|
<p>In the web form on that site, I picked the Toronto IX (“TORIX”), since that’s what <code>mtr</code> said I was using to go to facebook.com.</p>
|
|
<p><strong>thing 1: “show ip bgp summary”</strong></p>
|
|
<p>Here’s the output. I’ve redacted some of it:</p>
|
|
<pre><code>IPv4 Unicast Summary:
|
|
BGP router identifier 74.80.118.4, local AS number 3856 vrf-id 0
|
|
BGP table version 33061919
|
|
RIB entries 513241, using 90 MiB of memory
|
|
Peers 147, using 3003 KiB of memory
|
|
Peer groups 8, using 512 bytes of memory
|
|
|
|
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
|
|
...
|
|
206.108.34.248 4 1403 484672 466938 0 0 0 05w3d03h 50
|
|
...
|
|
206.108.35.2 4 32934 482088 466714 0 0 0 01w6d07h 38
|
|
206.108.35.3 4 32934 482019 466475 0 0 0 01w0d06h 38
|
|
...
|
|
|
|
Total number of neighbors 147
|
|
</code></pre>
|
|
<p>My understanding of what this is saying is that the Toronto Internet Exchange
|
|
(“TORIX”) is directly connected to both my ISP (EBOX, AS 1403) and Facebook (AS
|
|
32934).</p>
|
|
<p><strong>thing 2: “show ip bgp 129.134.30.0”</strong></p>
|
|
<p>Here’s the output of picking “show ip bgp” for <code>129.134.30.0</code> (one of Facebook’s IP addresses):</p>
|
|
<pre><code>BGP routing table entry for 129.134.30.0/23
|
|
Paths: (4 available, best #4, table default)
|
|
Advertised to non peer-group peers:
|
|
206.220.231.55
|
|
11670 32934
|
|
206.108.35.2 from 206.108.35.254 (206.108.35.254)
|
|
Origin IGP, metric 0, valid, external
|
|
Community: 3856:55000
|
|
Last update: Mon Oct 4 21:17:33 2021
|
|
|
|
11670 32934
|
|
206.108.35.2 from 206.108.35.253 (206.108.35.253)
|
|
Origin IGP, metric 0, valid, external
|
|
Community: 3856:55000
|
|
Last update: Mon Oct 4 21:17:31 2021
|
|
|
|
32934
|
|
206.108.35.3 from 206.108.35.3 (157.240.58.225)
|
|
Origin IGP, metric 0, valid, external, multipath
|
|
Community: 3856:55000
|
|
Last update: Mon Oct 4 21:17:27 2021
|
|
|
|
32934
|
|
206.108.35.2 from 206.108.35.2 (157.240.58.182)
|
|
Origin IGP, metric 0, valid, external, multipath, best (Older Path)
|
|
Community: 3856:55000
|
|
Last update: Mon Oct 4 21:17:27 2021
|
|
</code></pre>
|
|
<p>This seems to be saying that there are 4 routes to Facebook from that internet exchange.</p>
|
|
<p><strong>the quebec internet exchange doesn’t seem to know anything about Facebook</strong></p>
|
|
<p>I also tried the same thing from the Quebec internet exchange QIX (which is
|
|
presumably closer to me, since I live in Montreal and not Toronto). But the QIX
|
|
doesn’t seem to know anything about Facebook – when I put in <code>129.134.30.0</code> it
|
|
just says “% Network not in table”.</p>
|
|
<p>So I guess that’s why I was sent through the Toronto IX and not the Quebec one.</p>
|
|
<h3 id="more-bgp-looking-glasses" class="post-heading">
|
|
<a href="#more-bgp-looking-glasses">
|
|
more BGP looking glasses
|
|
</a>
|
|
</h3>
|
|
<p>Here are some more websites with looking glasses that will give you similar
|
|
information from other points of view. They all seem to support the same <code>show ip bgp</code> syntax, maybe because they’re running the same software? I’m not sure.</p>
|
|
<ul>
|
|
<li><a href="http://www.routeviews.org/routeviews/index.php/collectors/">http://www.routeviews.org/routeviews/index.php/collectors/</a></li>
|
|
<li><a href="http://www.routeservers.org/">http://www.routeservers.org/</a></li>
|
|
<li><a href="https://lg.he.net/">https://lg.he.net/</a></li>
|
|
</ul>
|
|
<p>There seem to be a LOT of these looking glass services out there, way more than just those 3 lists.</p>
|
|
<p>Here’s an example session with one of the servers on this list:
|
|
route-views.routeviews.org. This time I connected via telnet and not through a
|
|
web form, but the output looks like it’s in the same format.</p>
|
|
<pre><code>$ telnet route-views.routeviews.org
|
|
|
|
route-views>show ip bgp 31.13.80.36
|
|
|
|
BGP routing table entry for 31.13.80.0/24, version 1053404087
|
|
Paths: (23 available, best #2, table default)
|
|
Not advertised to any peer
|
|
Refresh Epoch 1
|
|
3267 1299 32934
|
|
194.85.40.15 from 194.85.40.15 (185.141.126.1)
|
|
Origin IGP, metric 0, localpref 100, valid, external
|
|
path 7FE0C3340190 RPKI State valid
|
|
rx pathid: 0, tx pathid: 0
|
|
Refresh Epoch 1
|
|
6939 32934
|
|
64.71.137.241 from 64.71.137.241 (216.218.252.164)
|
|
Origin IGP, localpref 100, valid, external, best
|
|
path 7FE135DB6500 RPKI State valid
|
|
rx pathid: 0, tx pathid: 0x0
|
|
Refresh Epoch 1
|
|
701 174 32934
|
|
137.39.3.55 from 137.39.3.55 (137.39.3.55)
|
|
Origin IGP, localpref 100, valid, external
|
|
path 7FE1604D3AF0 RPKI State valid
|
|
rx pathid: 0, tx pathid: 0
|
|
Refresh Epoch 1
|
|
20912 3257 1299 32934
|
|
212.66.96.126 from 212.66.96.126 (212.66.96.126)
|
|
Origin IGP, localpref 100, valid, external
|
|
Community: 3257:8095 3257:30622 3257:50001 3257:53900 3257:53904 20912:65004
|
|
path 7FE1195AF140 RPKI State valid
|
|
rx pathid: 0, tx pathid: 0
|
|
Refresh Epoch 1
|
|
7660 2516 1299 32934
|
|
203.181.248.168 from 203.181.248.168 (203.181.248.168)
|
|
Origin IGP, localpref 100, valid, external
|
|
Community: 2516:1030 7660:9001
|
|
path 7FE0D195E7D0 RPKI State valid
|
|
rx pathid: 0, tx pathid: 0
|
|
|
|
</code></pre>
|
|
<p>Here there are a few options for routes:</p>
|
|
<ul>
|
|
<li><code>3267 1299 32934</code></li>
|
|
<li><code>6939 32934</code></li>
|
|
<li><code>701 174 32934</code></li>
|
|
<li><code>20912 3257 1299 32934</code></li>
|
|
<li><code>7660 2516 1299 32934</code></li>
|
|
</ul>
|
|
<p>I think the reason there’s more than one AS in all of these is that <code>31.13.80.36</code> is a Facebook IP address in
|
|
Toronto, so this server (which is maybe on the US west coast, I’m not sure) is
|
|
not able to connect to it directly, it needs to go to another AS first. So all of the routes have one or more ASNs</p>
|
|
<p>The shortest one is 6939 (“Hurricane Electric”), which is a “global internet backbone”. They also have their own <a href="https://lg.he.net/">hurricane electric looking glass</a> page.</p>
|
|
<h3 id="tool-4-bgplay" class="post-heading">
|
|
<a href="#tool-4-bgplay">
|
|
tool 4: BGPlay
|
|
</a>
|
|
</h3>
|
|
<p>All the other tools so far have just shown us the current state of Facebook
|
|
routing where everything is fine, but this 4th tool lets us see the history of
|
|
this Facebook BGP internet disaster! It’s a GUI tool so I’m going to include a bunch of screenshots.</p>
|
|
<p>The tool is at <a href="https://stat.ripe.net/special/bgplay">https://stat.ripe.net/special/bgplay</a>. I typed in the IP address 129.134.30.12 (one of Facebook’s IPs), if you want to play along.</p>
|
|
<p>First, let’s look at the state of things before everything went wrong. I clicked in the timeline at 13:11:28 on Oct. 4, and got this:</p>
|
|
<img src="https://jvns.ca/images/bgplay-before.png">
|
|
<p>I originally found this very overwhelming. What’s happening? But then someone
|
|
on Twitter pointed out that the next place to look is to click on the timeline
|
|
right <em>after</em> the Facebook disaster happened (at 18:38 on Oct. 4).</p>
|
|
<img src="https://jvns.ca/images/bgplay-after.png">
|
|
<p>It’s pretty clear that something is wrong in this picture – all the BGP routes are gone! oh no!</p>
|
|
<p>The text at the top shows the last Facebook BGP route disappearing:</p>
|
|
<pre><code>Type: W > withdrawal Involving: 129.134.30.0/24
|
|
Short description: The route 50869, 25091, 32934 has been withdrawn.
|
|
Date and time: 2021-10-04 16:02:33 Collected by: 20-91.206.53.12
|
|
</code></pre>
|
|
<p>If I then click the “fast forward” button, we see the BGP routes start to come back:</p>
|
|
<img src="https://jvns.ca/images/bgplay-return.png">
|
|
<p>The first one announced is <code>137409 32934</code>. I don’t think this is <em>actually</em> the
|
|
first one announced though – there are a lot of route announcements inside the
|
|
same second (at 2021-10-04 21:00:40), and I think the ordering inside BGPlay is arbitrary.</p>
|
|
<p>If I click the “fast forward” button again, more and more routes start to come back and routing starts to go back to normal</p>
|
|
<p>I found looking at this outage in BGPlay really fun, even though the interface is pretty confusing at first.</p>
|
|
<h3 id="maybe-it-is-important-to-understand-a-little-about-bgp" class="post-heading">
|
|
<a href="#maybe-it-is-important-to-understand-a-little-about-bgp">
|
|
maybe it is important to understand a little about BGP?
|
|
</a>
|
|
</h3>
|
|
<p>I started out this post by saying you can’t change BGP routes
|
|
BGP, but then I remembered that in 2016 or 2017 there was a <a href="https://news.ycombinator.com/item?id=14246888">Telia routing issue</a> that caused us some minor
|
|
network at work. And when that happens, it is actually useful to understand why
|
|
your customers can’t reach your site, even if it’s totally out of your control.
|
|
I didn’t know about any of these tools at that time but I would have liked to!</p>
|
|
<p>I think for most companies all you can do to respond to outages caused by
|
|
someone else’s bad BGP routes is “do nothing and wait for it to get fixed”, but
|
|
it’s nice to be able to <em>confidently</em> do nothing.</p>
|
|
<h3 id="some-ways-to-publish-bgp-routes" class="post-heading">
|
|
<a href="#some-ways-to-publish-bgp-routes">
|
|
some ways to publish BGP routes
|
|
</a>
|
|
</h3>
|
|
<p>If you want to (as a hobbyist) actually publish BGP routes, here are some links from the comments:</p>
|
|
<ul>
|
|
<li><a href="https://labs.ripe.net/author/samir_jafferali/build-your-own-anycast-network-in-nine-steps/">a guide to getting your own ASN</a></li>
|
|
<li><a href="https://dn42.eu/Home">dn42</a> seems to have a playground for BGP (it’s not on the public internet, but it does have other people on it which seems more fun than just doing BGP by yourself at home)</li>
|
|
</ul>
|
|
<h3 id="that-s-all-for-now" class="post-heading">
|
|
<a href="#that-s-all-for-now">
|
|
that’s all for now
|
|
</a>
|
|
</h3>
|
|
<p>I think there are a lot more BGP tools (like PCH has a bunch of <a href="https://www.pch.net/resources/Routing_Data/IPv4_daily_snapshots/">daily snapshots of routing data</a> which look like fun), but
|
|
this post is already pretty long and there are other things I need to do today.</p>
|
|
<p>I was surprised by how much information I could get about BGP just as a regular
|
|
person, I always think of it as a “secret network wizard” thing but apparently there
|
|
are all kind of public machines anybody can just telnet to and use to look at the
|
|
route tables! Who knew!</p>
|
|
|
|
</main>
|
|
|
|
<footer>
|
|
|
|
<style type="text/css">
|
|
#mc_embed_signup{background:#fff; clear:left; font:14px Helvetica,Arial,sans-serif; display: inline;}
|
|
#mc_embed_signup {
|
|
display: inline;
|
|
}
|
|
#mc_embed_signup input.button {
|
|
background: #ff5e00;
|
|
display: inline;
|
|
color: white;
|
|
padding: 6px 12px;
|
|
}
|
|
</style>
|
|
<div class="sharing">
|
|
|
|
<style>
|
|
.form-inline {
|
|
display:flex; flex-flow: row wrap; justify-content: center;
|
|
}
|
|
.form-inline input, .form-inline span {
|
|
padding: 10px;
|
|
}
|
|
.form-inline input {
|
|
display:inline;
|
|
max-width:30%;
|
|
margin: 0 10px 0 0;
|
|
background-color: #fff;
|
|
border: 1px solid #ddd;
|
|
border-radius: 5px;
|
|
padding: 10px;
|
|
}
|
|
button {
|
|
background-color: #f50;
|
|
box-shadow: none;
|
|
border: 0;
|
|
border-radius: 5px;
|
|
color: white;
|
|
padding: 5px 10px;
|
|
}
|
|
@media (max-width: 800px) {
|
|
.form-inline input {
|
|
margin: 10px 0;
|
|
max-width:100% !important;
|
|
}
|
|
.form-inline {
|
|
flex-direction: column;
|
|
align-items: stretch;
|
|
}
|
|
}
|
|
</style>
|
|
|
|
<div align="center">
|
|
<form class="form-inline" action="https://app.convertkit.com/forms/1052396/subscriptions" method="post" data-uid="8884355abb" data-format="inline" data-version="5">
|
|
<span> Want a weekly digest of this blog?</span>
|
|
<input name="email_address" type="text" placeholder="Email address" />
|
|
<button type="submit" data-element="submit">Subscribe</button>
|
|
</form>
|
|
</div>
|
|
|
|
|
|
</div>
|
|
|
|
<p class="meta">
|
|
|
|
<a class="basic-alignment left" href="https://jvns.ca/blog/2021/10/02/all-my-zines-are-now-available-in-print/" title="Previous Post: All my zines are now available in print!">All my zines are now available in print!</a>
|
|
|
|
|
|
<a class="basic-alignment right" href="https://jvns.ca/blog/2021/10/21/how-to-get-useful-answers-to-your-questions/" title="Next Post: How to get useful answers to your questions">How to get useful answers to your questions</a>
|
|
|
|
</p>
|
|
</footer>
|
|
|
|
</article>
|
|
</div>
|
|
|
|
</div>
|
|
</div>
|
|
<nav role="navigation" class="footer-nav"> <a href="/">Archives</a>
|
|
</nav>
|
|
<footer role="contentinfo"><span class="credit">© Julia Evans. </span>
|
|
<span>If you like this, you may like <a href="https://web.archive.org/web/20181228051203/http://www.uliaea.ca/">Ulia Ea</a> or, more seriously, this list of <a href="https://jvns.ca/blogroll">blogs I love</a> or some <a href="https://jvns.ca/bookshelf">books I've read</a>. <br>
|
|
<p class="rc-scout__text"><i class="rc-scout__logo"></i>
|
|
You might also like the <a class="rc-scout__link" href="https://www.recurse.com/scout/click?t=546ea46360584b522270b8c3e5d830f8">Recurse Center</a>, my very favorite programming community <a href="/categories/hackerschool/">(my posts about it)</a></p>
|
|
</span>
|
|
<style class="rc-scout__style" type="text/css">.rc-scout{display:block;padding:0;border:0;margin:0;}.rc-scout__text{display:block;padding:0;border:0;margin:0;height:100%;font-size:100%;}.rc-scout__logo{display:inline-block;padding:0;border:0;margin:0;width:0.85em;height:0.85em;background:no-repeat center url('data:image/svg+xml;utf8,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2012%2015%22%3E%3Crect%20x%3D%220%22%20y%3D%220%22%20width%3D%2212%22%20height%3D%2210%22%20fill%3D%22%23000%22%3E%3C%2Frect%3E%3Crect%20x%3D%221%22%20y%3D%221%22%20width%3D%2210%22%20height%3D%228%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%222%22%20y%3D%222%22%20width%3D%228%22%20height%3D%226%22%20fill%3D%22%23000%22%3E%3C%2Frect%3E%3Crect%20x%3D%222%22%20y%3D%223%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%2361ae24%22%3E%3C%2Frect%3E%3Crect%20x%3D%224%22%20y%3D%223%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%2361ae24%22%3E%3C%2Frect%3E%3Crect%20x%3D%226%22%20y%3D%223%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%2361ae24%22%3E%3C%2Frect%3E%3Crect%20x%3D%223%22%20y%3D%225%22%20width%3D%222%22%20height%3D%221%22%20fill%3D%22%2361ae24%22%3E%3C%2Frect%3E%3Crect%20x%3D%226%22%20y%3D%225%22%20width%3D%222%22%20height%3D%221%22%20fill%3D%22%2361ae24%22%3E%3C%2Frect%3E%3Crect%20x%3D%224%22%20y%3D%229%22%20width%3D%224%22%20height%3D%223%22%20fill%3D%22%23000%22%3E%3C%2Frect%3E%3Crect%20x%3D%221%22%20y%3D%2211%22%20width%3D%2210%22%20height%3D%224%22%20fill%3D%22%23000%22%3E%3C%2Frect%3E%3Crect%20x%3D%220%22%20y%3D%2212%22%20width%3D%2212%22%20height%3D%223%22%20fill%3D%22%23000%22%3E%3C%2Frect%3E%3Crect%20x%3D%222%22%20y%3D%2213%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%223%22%20y%3D%2212%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%224%22%20y%3D%2213%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%225%22%20y%3D%2212%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%226%22%20y%3D%2213%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%227%22%20y%3D%2212%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%228%22%20y%3D%2213%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3Crect%20x%3D%229%22%20y%3D%2212%22%20width%3D%221%22%20height%3D%221%22%20fill%3D%22%23fff%22%3E%3C%2Frect%3E%3C%2Fsvg%3E');}.rc-scout__link:link,.rc-scout__link:visited{color:#61ae24;text-decoration:underline;}.rc-scout__link:hover,.rc-scout__link:active{color:#4e8b1d;}</style>
|
|
</footer>
|
|
<script type="text/rocketscript">
|
|
(function(){
|
|
var twitterWidgets = document.createElement('script');
|
|
twitterWidgets.type = 'text/javascript';
|
|
twitterWidgets.async = true;
|
|
twitterWidgets.src = 'http://platform.twitter.com/widgets.js';
|
|
document.getElementsByTagName('head')[0].appendChild(twitterWidgets);
|
|
})();
|
|
</script>
|
|
</div>
|
|
</body>
|
|
</html>
|
|
|