408 lines
15 KiB
HTML
408 lines
15 KiB
HTML
<!doctype html>
|
||
<!--[if lt IE 7]><html class="no-js lt-ie9 lt-ie8 lt-ie7" lang="en"> <![endif]-->
|
||
<!--[if (IE 7)&!(IEMobile)]><html class="no-js lt-ie9 lt-ie8" lang="en"><![endif]-->
|
||
<!--[if (IE 8)&!(IEMobile)]><html class="no-js lt-ie9" lang="en"><![endif]-->
|
||
<!--[if gt IE 8]><!--> <html class="no-js" lang="en"><!--<![endif]-->
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<title>Oh, Molly! – /techblog</title>
|
||
<meta name="description" content="Redpill Linpro Tech Blog">
|
||
|
||
|
||
|
||
<!-- Open Graph -->
|
||
<meta property="og:locale" content="en_US">
|
||
<meta property="og:type" content="article">
|
||
<meta property="og:title" content="Oh, Molly!">
|
||
<meta property="og:description" content="Redpill Linpro Tech Blog">
|
||
<meta property="og:url" content="/techblog/2015/12/19/molly-guard.html">
|
||
<meta property="og:site_name" content="/techblog">
|
||
|
||
|
||
|
||
|
||
|
||
<link rel="canonical" href="/techblog/2015/12/19/molly-guard.html">
|
||
<link href="/techblog/feed.xml" type="application/atom+xml" rel="alternate" title="/techblog Feed">
|
||
|
||
<!-- http://t.co/dKP3o1e -->
|
||
<meta name="HandheldFriendly" content="True">
|
||
<meta name="MobileOptimized" content="320">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||
|
||
<!-- For all browsers -->
|
||
<link rel="stylesheet" href="/techblog/assets/css/main.css">
|
||
<!-- Webfonts -->
|
||
<link href="//fonts.googleapis.com/css?family=Lato:300,400,700,300italic,400italic" rel="stylesheet" type="text/css">
|
||
|
||
|
||
|
||
<meta http-equiv="cleartype" content="on">
|
||
|
||
<!-- Load Modernizr -->
|
||
<script src="/techblog/assets/js/vendor/modernizr-2.6.2.custom.min.js"></script>
|
||
|
||
<!-- Icons -->
|
||
<link rel="shortcut icon" href="/techblog/favicon.ico">
|
||
|
||
<!-- Matomo tracking -->
|
||
<script>
|
||
var _paq = window._paq = window._paq || [];
|
||
_paq.push(['trackPageView']);
|
||
_paq.push(['enableLinkTracking']);
|
||
(function() {
|
||
var u="https://matomo.redpill-linpro.com/";
|
||
_paq.push(['setTrackerUrl', u+'matomo.php']);
|
||
_paq.push(['setSiteId', '1']);
|
||
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
|
||
g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
|
||
})();
|
||
</script>
|
||
|
||
|
||
|
||
</head>
|
||
|
||
<body id="post" class="feature">
|
||
|
||
<!--[if lt IE 9]><div class="upgrade"><strong><a href="http://whatbrowser.org/">Your browser is quite old!</strong> Why not upgrade to a different browser to better enjoy this site?</a></div><![endif]-->
|
||
|
||
|
||
<div class="nav-header">
|
||
<nav id="dl-menu" class="dl-menuwrapper" role="navigation">
|
||
<button class="dl-trigger">Open Menu</button>
|
||
<ul class="dl-menu">
|
||
<li><a href="/techblog/">Home</a></li>
|
||
<li><a href="/techblog/about/">About</a></li>
|
||
<li><a href="/techblog/feed.xml">RSS Feed</a></li>
|
||
<li><a href="/techblog/posts/">Show all Posts</a></li>
|
||
<li><a href="/techblog/tags/">Show all Tags</a></li>
|
||
</ul>
|
||
</nav>
|
||
|
||
<div class="logo">
|
||
<a href="https://redpill-linpro.com/"><img src="/techblog/images/Redpill-Linpro-logo-red-200px.png" alt="Logo"></a>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
|
||
<div class="entry-header">
|
||
|
||
|
||
|
||
|
||
|
||
<div class="image-credit">
|
||
<a target="_blank" href="https://www.redpill-linpro.com/">
|
||
Martin Skjerven/Redpill Linpro
|
||
</a>
|
||
| <a href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0
|
||
</a>
|
||
|
||
</div>
|
||
|
||
|
||
<div class="entry-image">
|
||
<img src="/techblog/images/features/sysadvent/rack_lab.jpg" alt="Oh, Molly!">
|
||
</div><!-- /.entry-image -->
|
||
</div><!-- /.entry-header -->
|
||
|
||
|
||
<div id="main" role="main">
|
||
<article class="hentry">
|
||
<header class="header-title">
|
||
<div class="header-title-wrap">
|
||
|
||
<h1 class="entry-title"><a href="/techblog/2015/12/19/molly-guard.html" rel="bookmark" title="Oh, Molly!">Oh, Molly!</a></h1>
|
||
|
||
<h2><span class="entry-date date published"><time datetime="2015-12-19T00:00:00+00:00">December 19, 2015</time></span></h2>
|
||
|
||
</div><!-- /.header-title-wrap -->
|
||
</header>
|
||
<div class="entry-content">
|
||
<p><em>This post appeared originally in our sysadvent series and has been
|
||
moved here following the discontinuation of the sysadvent microsite</em></p>
|
||
<div class="sysadvent"><img src="/techblog/images/features/sysadvent/RPLP_Logo.png"></div>
|
||
</div>
|
||
<div class="entry-content">
|
||
<p>I’m sure we all have had “that feeling once”. You patch your desktop
|
||
or laptop, then type in reboot in a shell in order to boot your
|
||
computer. And that crucial server you were working on starts shutting down.</p>
|
||
|
||
<p>But fear not - a solution exists for this and similar problems.</p>
|
||
|
||
<h3 id="history">History</h3>
|
||
|
||
<p>Molly-guard was (according to Internet) originally a improvised
|
||
plexiglass cover shielding the kill switch on an IBM 4341. It was
|
||
named after a programmers daughter - Molly - who tripped this kill
|
||
switch repeatedly. The name has obviously stuck around.</p>
|
||
|
||
<h3 id="your-own-virtual-molly-guard">Your own virtual Molly guard</h3>
|
||
|
||
<p>molly-guard is a small program that tries to prevent you from shutting
|
||
down or rebooting servers. On Debian and derivatives, it can be usually
|
||
be installed by running</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apt <span class="nb">install </span>molly-guard
|
||
</code></pre></div></div>
|
||
|
||
<p>while repackaged RPM packages can be found for RedHat and derivatives.</p>
|
||
|
||
<h3 id="how-it-works">How it works</h3>
|
||
|
||
<p>In a nutshell, this program works by forcing one or more checks before
|
||
the commands halt, shutdown, poweroff or reboot are run. In order to
|
||
achieve this, these commands are replaced with scripts invoking the
|
||
molly-guard functionality (ie. the check scripts).</p>
|
||
|
||
<p>The checks resides in the <code class="language-plaintext highlighter-rouge">/etc/molly-guard/run.d directory</code> - all
|
||
scripts in this directory are run and all needs to exit successfully,
|
||
that is returns a exit value of 0. After all scripts have exited
|
||
successfully, the original command is executed.</p>
|
||
|
||
<p>Typically, these checks includes you having to type in the name of the
|
||
host you want to halt or boot when you have logged in via
|
||
SSH. Entering the wrong name will abort your command.</p>
|
||
|
||
<p>If you are in a re-attached <code class="language-plaintext highlighter-rouge">screen</code> session, molly-guard will not
|
||
find your SSH process and think you are on a local console where the
|
||
chance of a screw up is smaller, so it won’t ask.</p>
|
||
|
||
<p>If you set the <code class="language-plaintext highlighter-rouge">ALWAYS_QUERY_HOSTNAME</code> variable in the
|
||
<code class="language-plaintext highlighter-rouge">/etc/molly-guard/rc</code> configuration file, this script will also force
|
||
a check when in screen/imux, logged in via console etc.</p>
|
||
|
||
<p>Other hypothetical checks can force the user to give a reason or
|
||
work-order for rebooting a server in order to comply with more strict
|
||
operations regimes.</p>
|
||
|
||
<h3 id="on-redhat">On RedHat</h3>
|
||
|
||
<p>Since molly-guard isn’t packaged for EL systems, we use a simpler
|
||
approach, aliasing the commands <code class="language-plaintext highlighter-rouge">reboot</code>, <code class="language-plaintext highlighter-rouge">shutdown</code> etc.</p>
|
||
|
||
<p>It is as simple as this:</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>clumsy_protect<span class="o">()</span> <span class="o">{</span>
|
||
<span class="nb">local </span><span class="nv">cmd</span><span class="o">=</span><span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span>
|
||
<span class="nb">shift
|
||
echo</span> <span class="s2">"Running </span><span class="nv">$cmd</span><span class="s2"> on </span><span class="nv">$HOSTNAME</span><span class="s2"> in 2 seconds!"</span>
|
||
<span class="nb">sleep </span>2 <span class="o">||</span> <span class="k">return
|
||
</span><span class="nb">command</span> <span class="s2">"</span><span class="nv">$cmd</span><span class="s2">"</span> <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span>
|
||
<span class="o">}</span>
|
||
<span class="nb">alias </span><span class="nv">reboot</span><span class="o">=</span><span class="s2">"clumsy_protect reboot"</span>
|
||
<span class="nb">alias </span><span class="nv">shutdown</span><span class="o">=</span><span class="s2">"clumsy_protect shutdown"</span>
|
||
<span class="nb">alias </span><span class="nv">poweroff</span><span class="o">=</span><span class="s2">"clumsy_protect poweroff"</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>Put the code in <code class="language-plaintext highlighter-rouge">/etc/profile.d/clumsy-protect.sh</code>. This gives you 2
|
||
seconds to realise that you typed reboot on the wrong machine, and if
|
||
you hit ^C in time, you can heave a great sigh of relief.</p>
|
||
|
||
<p>(Aside: the fact that an interrupted <code class="language-plaintext highlighter-rouge">sleep</code> returns failure can be
|
||
used in idioms like <code class="language-plaintext highlighter-rouge">while sleep 1; do something; done</code> instead of
|
||
<code class="language-plaintext highlighter-rouge">while true; do something; done</code> where you may have to mash ^C like a
|
||
madman to make it stop.)</p>
|
||
|
||
<p>The downside to the alias approach is that <code class="language-plaintext highlighter-rouge">sudo</code> does not look for
|
||
aliases, so only sysadmins who like to do everything in a root shell
|
||
get this protection.</p>
|
||
|
||
<h3 id="more-on-sudo">More on sudo</h3>
|
||
|
||
<p>In bash, you can work around that alias problem. If a trailing space
|
||
is added to the expanded alias value, bash will perform alias
|
||
expansion on the rest of the command as well!</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">alias sudo</span><span class="o">=</span><span class="s2">"sudo "</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>Now, the problem becomes that the function clumsy_protect is not
|
||
available in root’s environment. The easy fix is to put the function
|
||
in a script file in <code class="language-plaintext highlighter-rouge">$PATH</code> instead.</p>
|
||
|
||
<h3 id="going-further">Going further</h3>
|
||
|
||
<p>So you rebooted the correct server - but that quick reboot didn’t turn
|
||
out to be so quick! That terabyte file-system needs fsck, or worse,
|
||
your initrd was corrupt!</p>
|
||
|
||
<p>Introducing the all-singing, all-dancing clumsy_protect. It includes
|
||
a utility which checks your <code class="language-plaintext highlighter-rouge">/etc/fstab</code> for typos like:</p>
|
||
|
||
<ul>
|
||
<li>does that LABEL exist?</li>
|
||
<li>did you forget to remove mounting of that logical volume you deleted?</li>
|
||
<li>did you update file-system type when you upgraded from ext3 to ext4?</li>
|
||
</ul>
|
||
|
||
<p>clumsy_protect also check that your initrd has the correct format. On
|
||
RedHat, it even reruns prelink (if needed) before the reboot so that
|
||
you don’t get that annoying alert that your server is running outdated
|
||
libraries.</p>
|
||
|
||
<h3 id="pulling-it-all-together">Pulling it all together</h3>
|
||
|
||
<p>So where can you get this wonder? Look no further - <a href="http://github.com/kjetilho/clumsy_protect">its on
|
||
github</a>!</p>
|
||
|
||
<footer class="entry-meta">
|
||
<span class="entry-tags"><a href="/techblog/tags/#molly-guard" title="Pages tagged molly-guard" class="tag"><span class="term">molly-guard</span></a><a href="/techblog/tags/#bash" title="Pages tagged bash" class="tag"><span class="term">bash</span></a></span>
|
||
|
||
|
||
<div class="social-share">
|
||
|
||
<ul class="socialcount socialcount-small inline-list">
|
||
|
||
</ul>
|
||
<ul class="socialcount socialcount-small inline-list">
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<li class="twitter">
|
||
<a href="//twitter.com/share?text=Oh%2C+Molly%21&url=https://redpill-linpro.com/techblog/2015/12/19/molly-guard.html&via=redpilllinpro" onclick="window.open(this.href, 'twitter-share', 'width=550,height=255');return false;" title="Share on Twitter">
|
||
<span class="count"><i class="fa fa-twitter-square"></i> Tweet</span></a></li>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<li class="facebook">
|
||
<a href="//www.facebook.com/sharer.php?t=Oh%2C+Molly%21&u=https://redpill-linpro.com/techblog/2015/12/19/molly-guard.html" onclick="window.open(this.href, 'facebook-share', 'width=550,height=255');return false;" title="Share on Facebook">
|
||
<span class="count"><i class="fa fa-facebook-square"></i> Like</span></a></li>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<li class="linkedin">
|
||
<a href="//www.linkedin.com/shareArticle?mini=true&url=https://redpill-linpro.com/techblog/2015/12/19/molly-guard.html" onclick="window.open(this.href, '-share', 'width=550,height=255');return false;" title="Share on LinkedIn">
|
||
<span class="count"><i class="fa fa-linkedin-square"></i> Share</span></a></li>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</ul>
|
||
</div><!-- /.social-share -->
|
||
|
||
</footer>
|
||
</div><!-- /.entry-content -->
|
||
|
||
|
||
|
||
|
||
|
||
<div class="author-box">
|
||
|
||
<div>
|
||
<h2 class="author-name">Kjetil Homme</h2>
|
||
<p><span class="author-title"><a href="https://redpill-linpro.com/">Senior Systems Consultant at Redpill Linpro</a></span></p>
|
||
<p><span class="author-desc">Kjetil works with infrastructure at Redpill Linpro. He's been with Redpill Linpro for 8 years, and is currently working on automating our IT operations through Puppet, our storage solutions and backup rig. Kjetil been working with Linux since the early 90's and has made several contributions to the kernel and other associated projects.
|
||
</span></p>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
|
||
|
||
<div class="read-more">
|
||
|
||
|
||
|
||
|
||
|
||
<div class="read-more-header">
|
||
<a href="/techblog/2015/12/18/stateless-osd-servers.html" class="read-more-btn">More Posts</a>
|
||
</div><!-- /.read-more-header -->
|
||
<div class="read-more-content">
|
||
<h3><a href="/techblog/2026/06/25/el9s-grubby-sorting.html" title="EL9's grubby sorting">EL9's grubby sorting</a></h3>
|
||
<p><p>So I had this el9 machine that had been sitting quietly waiting for updates for a bit too long. Blinded by courage, I went in and did a</p> <p><code class="language-plaintext highlighter-rouge">dnf update</code>, rebooted, and lo and behold, it worked without a hitch afterwards. Except just one small one: It did not automatically boot the newest kernel. I tried the standard things. Look at <code class="language-plaintext highlighter-rouge">/etc/default/grub</code>, look at the output of various grubby commands. To no avail. It simply would <a href="/techblog/2026/06/25/el9s-grubby-sorting.html">... [continue reading]</a></p>
|
||
</div><!-- /.read-more-content -->
|
||
|
||
<div class="read-more-list">
|
||
|
||
<div class="list-item">
|
||
<h4><a href="/techblog/2026/03/20/from-luddite-to-vibe-coder.html" title="From a Luddite to a Vibe-Coder">From a Luddite to a Vibe-Coder</a></h4>
|
||
<span>Published on March 20, 2026</span>
|
||
</div><!-- /.list-item -->
|
||
|
||
<div class="list-item">
|
||
<h4><a href="/techblog/2026/02/27/Ontology-Guide.html" title="Ontology: A Guide to Understanding and Structuring Data">Ontology: A Guide to Understanding and Structuring Data</a></h4>
|
||
<span>Published on February 27, 2026</span>
|
||
</div><!-- /.list-item -->
|
||
|
||
</div><!-- /.read-more-list -->
|
||
</div><!-- /.read-more -->
|
||
|
||
</article>
|
||
</div><!-- /#main -->
|
||
|
||
<div class="footer-wrapper">
|
||
<footer role="contentinfo">
|
||
<span><a href="https://www.redpill-linpro.com">© 2026 Redpill Linpro</a></span>
|
||
|
||
</footer>
|
||
</div><!-- /.footer-wrapper -->
|
||
|
||
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
|
||
<script>window.jQuery || document.write('<script src="/techblog/assets/js/vendor/jquery-1.9.1.min.js"><\/script>')</script>
|
||
<script src="/techblog/assets/js/scripts.min.js"></script>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html>
|