Files
nexus/sreweekly/articles/533/05-what-sres-should-automate-and-never-automate-with-ai.html

157 lines
116 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8" data-next-head=""/><meta name="viewport" content="width=device-width" data-next-head=""/><script async="" src="https://www.googletagmanager.com/gtag/js?id=G-ECJJ2Q2SJQ"></script><title data-next-head=""></title><link rel="preconnect" href="https://bridge.hackernoon.com" data-next-head=""/><link rel="preconnect" href="https://cdn.hackernoon.com" data-next-head=""/><link rel="preconnect" href="https://hackernoon.imgix.net" data-next-head=""/><link rel="dns-prefetch" href="https://cdn.hackernoon.com" data-next-head=""/><meta name="description" content="Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task." data-next-head=""/><meta property="og:title" content="What SREs Should Automate — and Never Automate — with AI | HackerNoon" data-next-head=""/><meta property="og:description" content="Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task." data-next-head=""/><meta name="image" property="og:image" content="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg" data-next-head=""/><meta property="twitter:title" content="What SREs Should Automate — and Never Automate — with AI | HackerNoon" data-next-head=""/><meta property="twitter:description" content="Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task." data-next-head=""/><meta property="twitter:image" content="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg" data-next-head=""/><meta name="twitter:card" content="summary_large_image" data-next-head=""/><meta name="twitter:site" content="@hackernoon" data-next-head=""/><link rel="canonical" href="https://hackernoon.com/what-sres-should-automate-and-never-automate-with-ai" data-next-head=""/><link rel="preload" as="font" href="/fonts/HackerNoonFont/hackernoonv1-regular-webfont.woff2" type="font/woff2" crossorigin="anonymous"/><link rel="preconnect" href="https://fonts.googleapis.com"/><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin="anonymous"/><link data-next-font="" rel="preconnect" href="/" crossorigin="anonymous"/><link rel="preload" href="/_next/static/css/893998b3a2d586fd.css" as="style"/><link rel="preload" href="/_next/static/css/6d530d6069fd563f.css" as="style"/><link rel="preload" as="image" imageSrcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=640 640w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=750 750w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=828 828w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1080 1080w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1200 1200w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1920 1920w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=2048 2048w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=3840 3840w" imageSizes="(max-width: 768px) 100vw, 900px" data-next-head=""/><script type="application/ld+json" data-next-head="">{"@context":"http://schema.org","@type":"Article","name":"What SREs Should Automate — and Never Automate — with AI","headline":"What SREs Should Automate — and Never Automate — with AI","author":{"@type":"Person","name":"Sai Joshitha Kathari"},"datePublished":"2026-08-27","image":"https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg","articleSection":"site-reliability-engineering","articleBody":"Five key takeaways: Five key takeaways: Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task.\nAlert triage, anomaly detection, incident summaries, capacity forecasting — these are the easy wins. Low risk, high value.\nProduction changes, incident command, security response, severity calls — keep a human&apos;s name on these. Always.\nReversibility and blast radius are better questions than &quot;can the AI do this.&quot;\nThe goal isn&apos;t AI replacing engineers. It&apos;s AI clearing enough noise that engineers can actually think. Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task. Alert triage, anomaly detection, incident summaries, capacity forecasting — these are the easy wins. Low risk, high value. Production changes, incident command, security response, severity calls — keep a human&apos;s name on these. Always. Reversibility and blast radius are better questions than &quot;can the AI do this.&quot; The goal isn&apos;t AI replacing engineers. It&apos;s AI clearing enough noise that engineers can actually think. I&apos;ve sat through the version of this conversation that sounds like a vendor pitch — AI triages everything, drafts your runbooks, predicts outages before they happen, and nobody gets paged at 2 a.m. anymore. I&apos;ve also watched the other version happen in real time: an automated remediation script restarts the wrong service, confidently, at 11 p.m., and a 20-minute blip turns into a four-hour outage while everyone tries to figure out why the &quot;fix&quot; made things worse. Both of those are real. AI is already inside SRE workflows whether or not anyone signed off on it — the question that actually matters is where it belongs, and where a human still needs to be the one holding the decision. None of what follows comes from a whitepaper. It&apos;s from watching what breaks when teams move too fast with this stuff, and what quietly gets better when they don&apos;t. Reversibility and blast radius Here&apos;s the mental model I keep coming back to before automating anything: can you undo it, and how bad is it if you&apos;re wrong? Restarting a pod — reversible, low stakes. Deleting a database backup — not reversible, at all. Scaling a service up is easy to walk back. Silencing an alert for six hours is technically reversible too, except the six hours where something real happened and nobody saw it isn&apos;t something you get back. Blast radius is the other half of it, and it&apos;s not the same thing as severity. A misclassified low-priority alert costs a few wasted minutes. A misrouted sev-1 costs an hour of response time during an active outage, while the right team sits there not knowing they should be paged. And blast radius scales with what the action touches — one service versus a shared piece of infrastructure everything depends on, even when both look equally &quot;minor&quot; on paper. Anything with low reversibility and a wide blast radius shouldn&apos;t be running on autopilot. Anything reversible and contained is fair game. The stuff in between is where you actually need judgment — specifically, judgment from the people who&apos;ll be the ones on call when it goes sideways. Notice this framing never asks whether the AI can do something. It asks what happens if it&apos;s wrong. That&apos;s the more useful question, and it&apos;s the one most teams skip. can Where this actually works well Alert noise. This is the least controversial win there is. Somewhere between 30 and 60% of production alerts are noise by the time a human sees them — duplicates, transients, things that resolved themselves three minutes ago. AI grouping related alerts, suppressing known-flapping signals, correlating spikes with recent deploys — worst case, something gets mislabeled and a human still catches it. Low blast radius, fully reversible. This is exactly the profile you want. Alert noise. One catch: it only works well tuned to your environment, not a generic model. An alert that always fires right before a nightly batch job and clears itself a minute later is trivial to suppress — but only if the model actually knows about your batch schedule. Skip that step and you&apos;ve just added a second layer of noise on top of the first. First drafts of runbooks and postmortems. Runbook rot is one of the oldest problems in this field. The doc that was accurate in 2022 is a landmine now — nobody updates it, an incident hits, someone follows it anyway, and step four references a service that got decommissioned eight months ago. AI is genuinely good at pulling together a first draft from past incidents, change logs, whatever documentation exists. Same for postmortems — a draft that someone who actually lived through the incident reviews before it goes out saves real hours. First drafts of runbooks and postmortems. Forecasting and anomaly detection. This is pattern matching, and models are good at pattern matching. A holiday traffic spike that happens once a year gives engineers almost no reps to build intuition about — but a model trained across several years of that same spike has plenty. The important part: keep this as a recommendation a human acts on, not something that auto-provisions infrastructure on its own. The moment it stops informing a decision and starts making one, the blast radius changes. Forecasting and anomaly detection. Narrow, well-understood auto-remediation. This one comes with real caveats, but it earns its place. A specific service that needs a restart when it hits a known stuck state, a queue that needs draining past a defined threshold — fine, if the failure class is precisely defined, tested, and low-blast-radius by design. And there has to be a circuit breaker. If the fix doesn&apos;t work within a set window, it stops and escalates instead of retrying forever on a wrong diagnosis. Automation that keeps trying the same broken fix is worse than doing nothing. Narrow, well-understood auto-remediation. Where it doesn&apos;t belong Severity calls. Get this wrong either direction and it costs you. A real sev-1 marked as low pulls in the wrong people at the wrong urgency while an SLA clock runs. A minor issue marked critical drags a response team into something that didn&apos;t need them at 3 a.m. AI can surface context and flag patterns worth escalating — but the actual call needs a name attached, someone accountable for it. &quot;The model said it was low severity&quot; doesn&apos;t hold up in a postmortem. Severity calls. Production changes without sign-off. Config changes, scaling decisions, anything touching a database directly, restarts outside that narrow bounded case above — a human authorizes these. AI can prep the change, check it against known-good patterns, even simulate the blast radius. What it shouldn&apos;t do is decide the moment is right and pull the trigger itself. Production changes without sign-off. Security incidents. Different risk shape entirely. Miss something real and an active compromise sits there while the system waits for more confirmation. False-positive and you&apos;ve locked out legitimate engineers mid-response. AI correlating logs to surface signal fast — genuinely useful. Containment and escalation decisions — that needs someone who can weigh legal and business context a model was never trained on. Security incidents. Root cause, as a stated fact. AI narrowing the search space by correlating deploy timing with metric shifts is useful groundwork. But writing &quot;root cause: X&quot; in a postmortem is a claim that shapes what the org fixes next and what it decides to ignore. Get that wrong because a correlation looked convincing, and the actual bug ships again next quarter. Root cause, as a stated fact. Who to escalate to. This is context a model just doesn&apos;t have — who&apos;s already underwater tonight, what else is on fire across the org, whether the responding engineer&apos;s confidence is real or performed. Escalation is a trust call as much as a technical one. Who to escalate to. The thing nobody&apos;s measuring There&apos;s a slower cost that never shows up in a single incident review: engineers stop building intuition when AI absorbs all the routine reps. The edge cases are exactly where judgment matters most — and they&apos;re exactly the cases you need practice on the boring stuff to be ready for. A team leaning hard on automation can look great for a long stretch, right up until something shows up that doesn&apos;t match anything the model — or the team — has seen before. This isn&apos;t an argument against automating things. It&apos;s an argument for being honest about which reps you&apos;re willing to give away. A few practices worth adopting Decide, as a team, which categories of action AI can take alone versus which need a sign-off — decide this before an incident forces the question at 2 a.m.\nKeep an actual human accountable for anything irreversible. Not nominally &quot;in the loop&quot; — actually reviewing before it executes.\nBuild in a circuit breaker for anything automated. If it doesn&apos;t work within a defined window, it escalates instead of retrying.\nRotate people through the routine cases sometimes, even when AI could handle it, so the skill doesn&apos;t quietly disappear.\nRevisit the boundary as systems change. A failure class that was well-understood six months ago might not be anymore after an architecture shift. Decide, as a team, which categories of action AI can take alone versus which need a sign-off — decide this before an incident forces the question at 2 a.m. Keep an actual human accountable for anything irreversible. Not nominally &quot;in the loop&quot; — actually reviewing before it executes. Build in a circuit breaker for anything automated. If it doesn&apos;t work within a defined window, it escalates instead of retrying. Rotate people through the routine cases sometimes, even when AI could handle it, so the skill doesn&apos;t quietly disappear. Revisit the boundary as systems change. A failure class that was well-understood six months ago might not be anymore after an architecture shift. Skip this and you end up with automation debt, eroded skills, and a production system nobody fully understands anymore — which is a worse place to be than where you started. Where this leaves things It&apos;s not really a question of whether to use AI. It&apos;s whether you&apos;re using it somewhere judgment genuinely isn&apos;t needed, or somewhere it is and you&apos;ve just decided waiting for a human is too slow. One of those is a real force multiplier. The other is a liability with a delay timer on it."}</script><link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;700&amp;family=IBM+Plex+Sans:wght@400;700&amp;family=Inter:wght@400;600;900&amp;family=Source+Code+Pro:wght@400;500;600;700&amp;display=swap" rel="stylesheet" media="print"/><noscript><link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;700&amp;family=IBM+Plex+Sans:wght@400;700&amp;family=Inter:wght@400;600;900&amp;family=Source+Code+Pro:wght@400;500;600;700&amp;display=swap" rel="stylesheet"/></noscript> <!-- --><script id="ga4-init">
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
// Consent Mode: default to denied
gtag('consent', 'default', {
'ad_storage': 'denied',
'analytics_storage': 'denied',
'ad_user_data': 'denied',
'ad_personalization': 'denied'
});
gtag('js', new Date());
gtag('config', 'G-ECJJ2Q2SJQ');
</script><script id="iubenda-init">
function initIubenda() {
(async function () {
try {
const res = await fetch("https://geolocation-db.com/json/");
const data = await res.json();
const country = data && data.country_code;
const GDPR_COUNTRIES = [
"AT","BE","BG","HR","CY","CZ","DK","EE","FI","FR","DE","GR","HU",
"IE","IT","LV","LT","LU","MT","NL","PL","PT","RO","SK","SI","ES",
"SE","IS","LI","NO","UK","GB"
];
var isGdpr = GDPR_COUNTRIES.indexOf(country) > -1;
window._iub = window._iub || [];
window._iub.csConfiguration = {
siteId: 1848357,
cookiePolicyId: 18778700,
lang: "en",
enableTcf: false,
googleAdditionalConsentMode: true,
banner: {
position: "bottom",
rejectButtonDisplay: true,
explicitWithdrawal: true,
customizeButtonDisplay: true,
acceptButtonDisplay: true,
showTotalNumberOfProviders: false,
display: isGdpr
}
};
var iubScript = document.createElement("script");
iubScript.src = "https://cdn.iubenda.com/cs/iubenda_cs.js";
iubScript.async = true;
document.head.appendChild(iubScript);
if (!isGdpr) {
gtag('consent', 'update', {
'ad_storage': 'granted',
'analytics_storage': 'granted',
'ad_user_data': 'granted',
'ad_personalization': 'granted'
});
}
} catch (e) {
console.error("Iubenda geolocation failed", e);
}
})();
}
// Defer until browser is idle — never blocks initial render
if (typeof requestIdleCallback !== 'undefined') {
requestIdleCallback(initIubenda, { timeout: 3000 });
} else {
setTimeout(initIubenda, 1000);
}
</script><script id="iubenda-consent-bridge">
window.addEventListener("iubenda_consent_given", function () {
gtag('consent', 'update', {
'ad_storage': 'granted',
'analytics_storage': 'granted',
'ad_user_data': 'granted',
'ad_personalization': 'granted'
});
gtag('event', 'page_view', {
page_title: document.title,
page_location: location.href,
page_path: location.pathname + location.search
});
});
</script><link rel="stylesheet" href="/_next/static/css/893998b3a2d586fd.css" data-n-g=""/><link rel="stylesheet" href="/_next/static/css/6d530d6069fd563f.css" data-n-p=""/><noscript data-n-css=""></noscript><script defer="" noModule="" src="/_next/static/chunks/polyfills-42372ed130431b0a.js"></script><script src="https://accounts.google.com/gsi/client" defer="" data-nscript="beforeInteractive"></script><script defer="" src="/_next/static/chunks/7618.8cb06698e4978306.js"></script><script defer="" src="/_next/static/chunks/3213.7a85381e883f5859.js"></script><script defer="" src="/_next/static/chunks/7127.9c425c4d3409a6ac.js"></script><script defer="" src="/_next/static/chunks/3304.7c3523eee5ba4042.js"></script><script defer="" src="/_next/static/chunks/1826.1ab3736f712279dc.js"></script><script defer="" src="/_next/static/chunks/b6790ad6-21a72b711b29c9a6.js"></script><script defer="" src="/_next/static/chunks/4829-32ed3fa27f9fba8a.js"></script><script defer="" src="/_next/static/chunks/8145-56bb137bc815feca.js"></script><script defer="" src="/_next/static/chunks/7878-038a9b85114cf28d.js"></script><script defer="" src="/_next/static/chunks/9407-02afcd7299ecf2e9.js"></script><script defer="" src="/_next/static/chunks/1866-36cbb79df614e742.js"></script><script defer="" src="/_next/static/chunks/997.043403d1cfb4d583.js"></script><script defer="" src="/_next/static/chunks/9997.0bcf115a883cf0c1.js"></script><script defer="" src="/_next/static/chunks/9752.dad8cad18a61c6fa.js"></script><script defer="" src="/_next/static/chunks/1486.6875746f7e7b3bd6.js"></script><script defer="" src="/_next/static/chunks/2348.6ff1c8ab2b7f714e.js"></script><script src="/_next/static/chunks/webpack-2907a44b50458fad.js" defer=""></script><script src="/_next/static/chunks/framework-594babcea68f40f6.js" defer=""></script><script src="/_next/static/chunks/main-f4c6b80eccf8d9c3.js" defer=""></script><script src="/_next/static/chunks/pages/_app-e9f748c6202c8d20.js" defer=""></script><script src="/_next/static/chunks/4004-46cbf9060446c734.js" defer=""></script><script src="/_next/static/chunks/8230-f743aded49f5ec53.js" defer=""></script><script src="/_next/static/chunks/3363-3997af8403818196.js" defer=""></script><script src="/_next/static/chunks/5857-ec7c040b0c106d7c.js" defer=""></script><script src="/_next/static/chunks/7871-49db796a808d2c70.js" defer=""></script><script src="/_next/static/chunks/3261-28f7d7d5ddf137c8.js" defer=""></script><script src="/_next/static/chunks/1902-922299f711a16f76.js" defer=""></script><script src="/_next/static/chunks/8581-1c63d69fe79360dc.js" defer=""></script><script src="/_next/static/chunks/4581-8f149836b5130c5d.js" defer=""></script><script src="/_next/static/chunks/4680-486b209e44768b17.js" defer=""></script><script src="/_next/static/chunks/2562-5aa3cd1aa6e464a5.js" defer=""></script><script src="/_next/static/chunks/8373-1c61bad6a8e1fdcb.js" defer=""></script><script src="/_next/static/chunks/7225-8c8308d5122595b5.js" defer=""></script><script src="/_next/static/chunks/pages/%5Bslug%5D-e18135e2a995d794.js" defer=""></script><script src="/_next/static/E2JcG3BvGILbUHTlEzcxe/_buildManifest.js" defer=""></script><script src="/_next/static/E2JcG3BvGILbUHTlEzcxe/_ssgManifest.js" defer=""></script></head><body><link rel="preload" as="image" imageSrcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=640 640w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=750 750w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=828 828w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1080 1080w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1200 1200w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1920 1920w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=2048 2048w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=3840 3840w" imageSizes="(max-width: 768px) 100vw, 1200px" fetchPriority="high"/><link rel="preload" as="image" href="https://hackernoon.imgix.net/avatars/robot-b5.png"/><link rel="preload" as="image" href="https://hackernoon.imgix.net/avatars/robot-b6.png"/><div id="__next"><div class="bg-light text-lightText font-[ibm-plex-mono]"><main><header class="font-[ibm-plex-sans] fixed top-0 left-0 w-full z-50 transition-all duration-500 ease-in-out translate-y-0"><div class="flex items-center justify-between bg-primary lg:navbar h-[50px] sm:min-h-[75px] transition-all duration-100 shadow-md w-full"><div class="hidden lg:flex navbar-start h-full items-center ml-1"><button class="flex items-center hover:scale-[1.01] justify-center rounded-lg text-base px-4 font-bold py-2 border-none bg-primary-content text-primaryContentText">Discover Anything<i class="hn hn-search text-lg ml-4 text-primaryContentText "></i></button></div><div class="nav-start lg:navbar-center ml-2 lg:ml-0 min-w-0 flex-shrink"><a class="relative z-10 flex items-center space-x-2 hover:scale-[1.02]" aria-label="HackerNoon Homepage" href="/"><svg class="w-[180px] xs:w-[200px] sm:w-[240px] lg:w-[260px] h-auto" viewBox="0 0 2150 260" fill="none" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMidYMid meet" style="transition:fill 150ms ease"><g style="transition:fill 150ms ease"><path d="M269.997 20.0005V0H130V20.0005V40.0011V60.0016H150H169.995V40.0011H189.995H229.996V60.0016H249.997H269.997V40.0011V20.0005Z" fill="transparent"></path><path d="M130.006 80.003V60.0024H110.006V80.003V100.004H130.006V80.003Z" fill="transparent"></path><path d="M110 119.998V100.003H90V119.998V139.998V159.999H110V139.998V119.998Z" fill="transparent"></path><path d="M270 100.004H290V80.003V60.0024H270V80.003V100.004Z" fill="transparent"></path><path d="M310 119.997V100.002H290V119.997V139.998V159.998H310V139.998H330.001V119.997H310Z" fill="transparent"></path><path d="M130 159.998H110V179.998V199.999H130V179.998V159.998Z" fill="transparent"></path><path d="M270 179.998V199.999H290V179.998V159.998H270V179.998Z" fill="transparent"></path><path d="M130 260V240V219.999V199.999H150H169.995V219.999H189.995H209.996H229.996V199.999H249.997H269.997V219.999V240V260H130Z" fill="transparent"></path><path d="M210.415 39.74V59.7405V79.7411V99.7416V119.736V139.737H190.415V119.736V99.7416V79.7411V59.7405V39.74H210.415Z" fill="transparent"></path><path d="M390 200V60H417.801V116.676H501.206V60H530V200H501.206V144.517H417.801V200H390Z" fill="transparent"></path><path d="M672.199 116.676V88.8352H588.794V116.676H672.199ZM560 200V60H700V200H672.199V144.517H588.794V200H560Z" fill="transparent"></path><path d="M730 200V60H870V88.8352H758.794V172.159H870V200H730Z" fill="transparent"></path><path d="M900 200V60H928.794V116.276H984.397V143.724H928.794V200H900ZM1012.2 171.368H984.397V143.724H1012.2V171.368ZM1012.2 171.368H1040V199.013H1012.2V171.368ZM1012.2 88.6319V116.276H984.397V88.6319H1012.2ZM1012.2 88.6319V60H1040V88.6319H1012.2Z" fill="transparent"></path><path d="M1070 200V60H1210V88.8352H1098.79V116.676H1154.4V144.517H1098.79V172.159H1210V200H1070Z" fill="transparent"></path><path d="M1351.24 116.519V88.7589H1267.76V116.519H1351.24ZM1240 200V60H1380V144.479H1351.24V172.24H1380V200H1351.24V172.24H1323.48V144.479H1267.76V200H1240Z" fill="transparent"></path><path d="M1410 200V60H1550V200H1522.24V88.7589H1438.76V200H1410Z" fill="transparent"></path><path d="M1692.24 172.24V88.7589H1608.76V172.24H1692.24ZM1580 200V60H1720V200H1580Z" fill="transparent"></path><path d="M1862.04 172.24V88.7589H1778.97V172.24H1862.04ZM1750 200V60H1890V200H1750Z" fill="transparent"></path><path d="M1920 200V60H2060V200H2032.24V88.7589H1948.76V200H1920Z" fill="transparent"></path></g></svg></a></div><div class="navbar-end h-full flex items-center min-w-[100px] lg:min-w-[200px] space-x-4 mr-2"><div class=" h-[40px] flex items-center justify-center"></div><div class="hidden sm:flex space-x-4 "><button class="px-4 font-bold text-base py-1 sm:py-2 bg-primary-content text-primaryContentText rounded-md transition-all duration-300">Signup</button><a class="px-4 hover:scale-105 font-bold text-base py-2 bg-primary-content text-primaryContentText rounded-md " href="/new">Write</a></div><button class="btn border-none p-0 m-0 lg:hidden bg-transparent hover:bg-transparent text-primary-content hover:scale-110"><i class="hn hn-search text-xl mr-2 "></i></button><button class="relative lg:flex hidden items-center hover:scale-110 text-primary-content" aria-label="Notifications"><i width="20" class="hn hn-bell text-2xl w-4 h-4 sm:w-6 sm:h-6"></i></button><button class="flex items-center hover:scale-110 text-primary-content" aria-label="Menu"><i class="hn hn-bars text-2xl text-primary-content"></i></button></div></div><div class="z-20 hidden lg:block h-[52px] transition-all duration-500 ease-in-out"><nav class="h-[52px] bg-secondary animate-pulse"></nav></div><div class="flex items-center "><div class="bg-accent text-accent-content flex items-center h-[62px] sm:min-h-[80px] font-[ibm-plex-sans] w-full relative z-10 opacity-100"><div class="h-[62px] sm:h-[80px] bg-[transparent] animate-pulse"></div></div></div></header><div class="transition-all duration-200 pt-[112px] sm:pt-[155px] lg:pt-[207px]"><div data-rht-toaster="" style="position:fixed;z-index:9999;top:16px;left:16px;right:16px;bottom:16px;pointer-events:none"></div><div class=""><div class="bg-light text-lightText h-auto xl:mx-2 "><div class="col-span-12"><div class="max-w-[1200px] mx-auto px-2 xs:px-4 xl: xl:px-0 mt-10"><div class=""><div><div class="text-xs"><div class="mb-4 flex gap-2"><span class="bg-lightAlt p-2 rounded-lg inline-flex gap-2 items-center justify-start"><i class="hn hn-star-solid"></i> <!-- -->569<!-- --> <!-- -->reads</span></div><h1 class="font-bold line-clamp-4 leading-snug text-lightTextStrong
text-xl sm:text-2xl xl:text-3xl 3xl:text-4xl tracking-wide
">What SREs Should Automate — and Never Automate — with AI</h1><div class="flex flex-wrap border-y border-lightBorder my-4 sm:my-2 sm:border-t-0 py-2 items-center justify-between text-lightTextLight text-sm sm:text-base xl:text-xl "><div class="flex flex-wrap justify-between w-full xs:w-auto items-center gap-2"><span class="flex items-center flex-wrap gap-2 mr-10 sm:mr-0 ">by<div class="dropdown dropdown-hover "><label tabindex="0"><a aria-label="View profile of Sai Joshitha Kathari" href="/u/sai-joshitha-kathari"><strong class="...">Sai Joshitha Kathari</strong></a></label><div class="dropdown-content z-[1] pt-2 sm:pt-1 left-[-40px] w-[280px] xs:w-[320px] sm:w-[400px] bg-transparent menu rounded "><div class="w-full "><div class=" p-4 border border-lightBorder bg-light rounded-lg"><a href="/u/sai-joshitha-kathari" target="_blank" rel="noopener noreferrer" class="flex items-start text-sm rounded-lg group gap-2"><div class=""><img alt="Sai Joshitha Kathari" loading="lazy" width="40" height="40" decoding="async" data-nimg="1" class="w-10 h-9 border-solid border border-lightBorder rounded-full object-contain" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=48 1x, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96 2x" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96"/></div><span class="flex flex-col min-w-0 w-full justify-center"><span class="flex items-center gap-1 text-ellipsis overflow-hidden whitespace-nowrap"><span class="font-bold group-hover:underline text-xs truncate"><span class="text-xs font-light mr-1">by</span>Sai Joshitha Kathari</span><span class="text-xs font-light opacity-50">|</span><span class="text-sm false text-ellipsis overflow-hidden whitespace-nowrap" title="@sai-joshitha-kathari">@<!-- -->sai-joshitha-kathari</span></span><span class="text-xs text-ellipsis overflow-hidden whitespace-nowrap mt-0.5" title="Senior Site Reliability Engineer at ">Senior Site Reliability Engineer<!-- --> at </span></span></a><p class="text-sm overflow-x-auto mt-2 text-bodyTxtLight">Senior Site Reliability Engineer based in Austin, Texas, specializing in distributed systems, Kubernetes, and production reliability. </p><div class="mt-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div></div></div></div></div></span><div class="flex gap-2 items-center cursor-pointer"><span class="hidden sm:block w-1 h-1 bg-lightTextLight mx-4 rounded-full"></span><a href="/archives/2026/08/27"><span class="text-xs xs:text-sm lg:text-base">August 27th, 2026</span></a></div></div><div class="hidden xl:block"><div class=" flex items-center gap-4 "><span class="tooltip tooltip-left tooltip-left w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Terminal Reader"><img alt="Read on Terminal Reader" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&amp;w=32 1x, https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&amp;w=48 2x" src="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&amp;w=48"/></span><span class="tooltip tooltip-left tooltip-left w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Print this story"><img alt="Print this story" data-tip="true" data-for="print-page" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=32 1x, https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48 2x" src="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48"/></span><span class="tooltip tooltip-left tooltip-left w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Read this story w/o Javascript"><img alt="Read this story w/o Javascript" data-tip="true" data-for="arweave-backup" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=32 1x, https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48 2x" src="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48"/></span></div></div></div></div><div class="mb-2 "><div class="flex justify-between "><button class="flex m-1 px-2 xl:px-4 h-[40px] items-center font-[hackernoon2] bg-dark text-darkText text-xs sm:text-sm rounded-lg border border-lightBorder">TLDR <i class="hn hn-angle-right text-base ml-1 "></i></button><div class="flex items-center gap-2"><div class="hidden sm:block xl:hidden"><div class=" flex items-center gap-4 "><span class="tooltip tooltip-left undefined w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Terminal Reader"><img alt="Read on Terminal Reader" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&amp;w=32 1x, https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&amp;w=48 2x" src="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&amp;w=48"/></span><span class="tooltip tooltip-left undefined w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Print this story"><img alt="Print this story" data-tip="true" data-for="print-page" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=32 1x, https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48 2x" src="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48"/></span><span class="tooltip tooltip-left undefined w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Read this story w/o Javascript"><img alt="Read this story w/o Javascript" data-tip="true" data-for="arweave-backup" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=32 1x, https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48 2x" src="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&amp;w=48"/></span></div></div><div class="xl:hidden"></div></div><div class="hidden xl:flex items-center flex-wrap gap-2"></div></div></div></div></div></div><div class="max-w-[1200px] mx-auto"><div class="flex items-center justify-center w-full h-full"><div class="relative group cursor-zoom-in transition-transform hover:scale-[1.01] max-w-full mx-auto px-[14px] md:px-0 w-full"><button class="absolute top-2 right-5 z-10 w-6 h-6 rounded flex items-center justify-center opacity-0 group-hover:opacity-100 transition-opacity"><i class="hn hn-download text-darkText bg-dark p-2 rounded-xl text-base"></i></button><img alt="featured image - What SREs Should Automate — and Never Automate — with AI" fetchPriority="high" loading="eager" width="512" height="512" decoding="async" data-nimg="1" class="w-full h-auto object-contain rounded-lg shadow-lg my-0" style="color:transparent;background-size:cover;background-position:50% 50%;background-repeat:no-repeat;background-image:url(&quot;data:image/svg+xml;charset=utf-8,%3Csvg xmlns=&#x27;http://www.w3.org/2000/svg&#x27; viewBox=&#x27;0 0 512 512&#x27;%3E%3Cfilter id=&#x27;b&#x27; color-interpolation-filters=&#x27;sRGB&#x27;%3E%3CfeGaussianBlur stdDeviation=&#x27;20&#x27;/%3E%3CfeColorMatrix values=&#x27;1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 100 -1&#x27; result=&#x27;s&#x27;/%3E%3CfeFlood x=&#x27;0&#x27; y=&#x27;0&#x27; width=&#x27;100%25&#x27; height=&#x27;100%25&#x27;/%3E%3CfeComposite operator=&#x27;out&#x27; in=&#x27;s&#x27;/%3E%3CfeComposite in2=&#x27;SourceGraphic&#x27;/%3E%3CfeGaussianBlur stdDeviation=&#x27;20&#x27;/%3E%3C/filter%3E%3Cimage width=&#x27;100%25&#x27; height=&#x27;100%25&#x27; x=&#x27;0&#x27; y=&#x27;0&#x27; preserveAspectRatio=&#x27;none&#x27; style=&#x27;filter: url(%23b);&#x27; href=&#x27;data:image/svg+xml;base64,CiAgICA8c3ZnIHdpZHRoPSI1MTIiIGhlaWdodD0iNTEyIiB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiPgogICAgICA8ZGVmcz4KICAgICAgICA8bGluZWFyR3JhZGllbnQgaWQ9ImciPgogICAgICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzIyMiIgb2Zmc2V0PSIyMCUiIC8+CiAgICAgICAgICA8c3RvcCBzdG9wLWNvbG9yPSIjMGYwIiBvZmZzZXQ9IjUwJSIgLz4KICAgICAgICAgIDxzdG9wIHN0b3AtY29sb3I9IiMyMjIiIG9mZnNldD0iODAlIiAvPgogICAgICAgIDwvbGluZWFyR3JhZGllbnQ+CiAgICAgIDwvZGVmcz4KICAgICAgPHJlY3Qgd2lkdGg9IjUxMiIgaGVpZ2h0PSI1MTIiIGZpbGw9IiMyMjIiIC8+CiAgICAgIDxyZWN0IGlkPSJyIiB3aWR0aD0iNTEyIiBoZWlnaHQ9IjUxMiIgZmlsbD0idXJsKCNnKSIgLz4KICAgICAgPGFuaW1hdGUgeGxpbms6aHJlZj0iI3IiIGF0dHJpYnV0ZU5hbWU9IngiIGZyb209Ii01MTIiIHRvPSI1MTIiIGR1cj0iMXMiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiAgLz4KICAgIDwvc3ZnPg==&#x27;/%3E%3C/svg%3E&quot;)" sizes="(max-width: 768px) 100vw, 1200px" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=640 640w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=750 750w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=828 828w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1080 1080w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1200 1200w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=1920 1920w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=2048 2048w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=3840 3840w" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg?auto=format%2Ccompress&amp;w=3840"/></div></div></div><div class="px-2 xs:px-4 3xl:px-0 my-4 sm:mt-4 sm:mb-6 max-w-[1200px] 6xl:max-w-[1200px] mx-auto "><div class="w-full flex flex-col justify-center rounded-lg"><audio src="https://storage.googleapis.com/hackernoon/audios/6a7a7a897378522c3c58fcbe-en-US-Wavenet-I-MALE--8147770a3114d.mp3" preload="metadata">Your browser does not support the <code>audio</code> element.</audio><div class="hidden sm:flex justify-between items-center "><span></span></div><div class="flex gap-2 items-center"><div class="flex items-center justify-center mx-auto gap-2 xs:gap-4 lg:gap-6 flex-1"><button aria-label="play/pause" class="text-darkAccent max-w-[40px] max-h-[40px] sm:min-w-[48px] sm:min-h-[48px] border order-1 border-darkBorder bg-dark p-2 rounded-full flex items-center justify-center" title="Play/Pause"><i class="hn hn-play-solid text-base xs:text-lg sm:text-2xl "></i></button><div class="dropdown order-2 dropdown-hover"><label tabindex="0" class="flex items-center hn hn-playlist-solid text-base xs:text-lg sm:text-xl lg:text-2xl rounded-lg " title="Speed &amp; Voice"></label><ul tabindex="0" class="dropdown-content border z-40 menu p-4 shadow bg-light rounded-box w-60"><div class="text-lightText flex bg-light p-2 rounded w-full mb-2 items-center justify-between"><span class="text-xs font-bold">Speed</span><button class="bg-lightAlt ml-2 px-4 py-2 rounded-full text-sm font-bold min-w-[100px]">1x</button></div><div class="text-lightText flex flex-col bg-light p-2 rounded w-full"><span class="text-xs font-bold mb-2">Voice</span><div class="flex flex-col gap-2 max-h-60 overflow-auto pr-1"><button class="bg-lightAlt px-3 py-2 rounded-lg text-sm font-bold text-left flex items-center justify-between ring-2 ring-green-600"><span class="truncate mr-2">Dr. One </span><img src="https://hackernoon.imgix.net/avatars/robot-b5.png" alt="Dr. One (en-US)" class="w-6 h-6 rounded-full"/></button><button class="bg-lightAlt px-3 py-2 rounded-lg text-sm font-bold text-left flex items-center justify-between "><span class="truncate mr-2">Ms. Hacker </span><img src="https://hackernoon.imgix.net/avatars/robot-b6.png" alt="Ms. Hacker (en-US)" class="w-6 h-6 rounded-full"/></button></div></div></ul></div><div class="flex gap-2 order-3 sm:items-center sm:flex-row w-full"><div class="rounded-lg flex-1 bg-lightAccentTextAlt relative"><div class="hidden lg:block"><div class="relative max-w-[1000px] h-full flex items-center cursor-pointer rounded-lg "><canvas class="bg-transparent absolute top-0 left-0 w-full h-full rounded-lg "></canvas><div></div><div class=" top-0 left-0 h-full overflow-hidden bg-lightAccentAlt border rounded-l-lg" style="width:0px"><canvas class="bg-transparent w-full h-full text-green-500"></canvas></div></div></div><div class="hidden sm:block lg:hidden"><div class="relative max-w-[1000px] h-full flex items-center cursor-pointer rounded-lg "><canvas class="bg-transparent absolute top-0 left-0 w-full h-full rounded-lg "></canvas><div></div><div class=" top-0 left-0 h-full overflow-hidden bg-lightAccentAlt border rounded-l-lg" style="width:0px"><canvas class="bg-transparent w-full h-full text-green-500"></canvas></div></div></div><div class="w-full sm:hidden"><div class="relative max-w-[1000px] h-full flex items-center cursor-pointer rounded-lg "><canvas class="bg-transparent absolute top-0 left-0 w-full h-full rounded-lg "></canvas><div></div><div class=" top-0 left-0 h-full overflow-hidden bg-lightAccentAlt border rounded-l-lg" style="width:0px"><canvas class="bg-transparent w-full h-full text-green-500"></canvas></div></div></div></div><div class="hidden lg:ml-2 sm:block"></div><div class=" flex items-center sm:hidden"></div></div></div></div></div></div><div class=" flex xl:hidden bg-light z-10 mx-auto gap-4 items-center border-y sticky top-[62px] sm:top-[80px] py-2 sm:py-0 "><div class="w-full max-w-[1200px] mx-auto px-4 flex justify-between items-center"><div class="6xl:hidden dropdown dropdown-bottom dropdown-hover"><div tabindex="0" role="button" class="flex text-sm rounded-lg py-2"><div class="mr-2 flex -space-x-2 items-center "><div class=""><img alt="Sai Joshitha Kathari" loading="lazy" width="48" height="48" decoding="async" data-nimg="1" class="w-10 h-10 sm:w-12 sm:h-12 bg-light relative border border-lightBorder rounded-full object-contain" style="color:transparent;z-index:1" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=48 1x, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96 2x" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96"/></div></div><div class="flex-col hidden sm:flex flex-wrap"><span class="font-bold mx-2 text-xs"><span class="text-xs font-light mr-1">by</span>Sai Joshitha Kathari</span><span class="text-sm ml-2">@<!-- -->sai-joshitha-kathari</span></div></div><ul tabindex="0" class="dropdown-content menu w-[300px] py-1 left-[-20px] bg-light px-1 ml-4 rounded-b-lg 3xl:border-none rounded-boxabsolute z-50"><div class="flex w-full flex-col gap-4"><div><div class="w-full "><div class=" p-4 border border-lightBorder bg-light rounded-lg"><a href="/u/sai-joshitha-kathari" target="_blank" rel="noopener noreferrer" class="flex items-start text-sm rounded-lg group gap-2"><div class=""><img alt="Sai Joshitha Kathari" loading="lazy" width="40" height="40" decoding="async" data-nimg="1" class="w-10 h-9 border-solid border border-lightBorder rounded-full object-contain" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=48 1x, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96 2x" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96"/></div><span class="flex flex-col min-w-0 w-full justify-center"><span class="flex items-center gap-1 text-ellipsis overflow-hidden whitespace-nowrap"><span class="font-bold group-hover:underline text-xs truncate"><span class="text-xs font-light mr-1">by</span>Sai Joshitha Kathari</span><span class="text-xs font-light opacity-50">|</span><span class="text-sm false text-ellipsis overflow-hidden whitespace-nowrap" title="@sai-joshitha-kathari">@<!-- -->sai-joshitha-kathari</span></span><span class="text-xs text-ellipsis overflow-hidden whitespace-nowrap mt-0.5" title="Senior Site Reliability Engineer at ">Senior Site Reliability Engineer<!-- --> at </span></span></a><p class="text-sm overflow-x-auto mt-2 text-bodyTxtLight">Senior Site Reliability Engineer based in Austin, Texas, specializing in distributed systems, Kubernetes, and production reliability. </p><div class="mt-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div></div></div></div><div class="xl:hidden"><div class="group transition-all duration-300"><div class="p-2 border border-lightBorder bg-light rounded-lg transition-all duration-300 pb-0"><span class="font-bold mx-2 text-sm text-lightTextLight">Story&#x27;s Credibility</span><div class="
mt-2 flex gap-2 flex-wrap m-2 transition-all duration-300 ease-in-out
flex-row
"><div class="flex items-start gap-2 text-sm rounded-lg max-w-[250px]
transition-all duration-300 ease-in-out p-1
cursor-default"><img alt="Original Reporting" loading="lazy" width="16" height="16" decoding="async" data-nimg="1" class="w-4 h-4 rounded-full transition-transform duration-300 group-hover:scale-105 cursor-pointer" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/img-oi03r0q.png?auto=format%2Ccompress&amp;w=32 1x" src="https://hackernoon.imgix.net/images/img-oi03r0q.png?auto=format%2Ccompress&amp;w=32"/></div></div></div></div></div></div></ul></div><div class="w-[200px] 6xl:hidden"><div class=" flex flex-row flex-row-reverse items-start gap-4 "><span class="tooltip tooltip-left cursor-pointer" data-tip="Bookmark"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-bookmark text-lightText text-2xl"></i></button></span><span class="tooltip tooltip-left cursor-pointer" data-tip="Comment"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-comment text-lightText text-2xl"></i></button></span><div class="dropdown dropdown-bottom dropdown-hover group "><label tabindex="0" class="flex items-center cursor-pointer justify-center border border-lightBorder 3xl:group-hover:bg-lightAlt group-hover:bg-light h-[40px] w-[40px] p-2 rounded "><i class="hn hn-share text-2xl"></i></label><ul tabindex="0" class="dropdown-content bg-light z-[1] py-4 px-4 3xl:px-0 3xl:py-2 border 3xl:border-none flex flex-col items-center justify-center gap-2 "><button class="border p-2 rounded hover:bg-lightAlt"><i class=" hn hn-copy text-lightText text-2xl "></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-facebook-round text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-x text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-linkedin text-lightText text-2xl"></i></button><a href="mailto:?subject=I&#x27;d like to share a link with you &amp;body=" class="border p-2 rounded inline-block hover:bg-lightAlt"><i class="hn hn-envelope text-lightText text-2xl"></i></a></ul></div></div></div></div></div><div class="flex max-w-[1100px] 2xl:max-w-[1200px] mx-auto justify-center flex-row gap-4 items-start mt-10 px-4 xl:px-0"><div class="hidden xl:flex xl:flex-col self-stretch"><div class="sticky top-[99px] z-40"><div class="dropdown z-25 dropdown-right dropdown-hover"><div class="mr-2 flex gap-2 flex-col justify-center flex-wrap items-center "><div class="relative h-12 w-12 bg-black rounded-full overflow-hidden flex-shrink-0"><img alt="Sai Joshitha Kathari" loading="lazy" decoding="async" data-nimg="fill" class="rounded-full object-contain " style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;color:transparent;z-index:1" sizes="100vw" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=640 640w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=750 750w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=828 828w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=1080 1080w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=1200 1200w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=1920 1920w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=2048 2048w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=3840 3840w" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=3840"/></div></div><ul tabindex="0" class="dropdown-content w-[280px] xs:w-[320px] sm:w-[400px] rounded-lg z-[100] bg-light flex flex-col items-center justify-center gap-2"><div class="flex w-full flex-col gap-4"><div><div class="w-full "><div class=" p-4 border border-lightBorder bg-light rounded-lg"><a href="/u/sai-joshitha-kathari" target="_blank" rel="noopener noreferrer" class="flex items-start text-sm rounded-lg group gap-2"><div class=""><img alt="Sai Joshitha Kathari" loading="lazy" width="40" height="40" decoding="async" data-nimg="1" class="w-10 h-9 border-solid border border-lightBorder rounded-full object-contain" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=48 1x, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96 2x" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=96"/></div><span class="flex flex-col min-w-0 w-full justify-center"><span class="flex items-center gap-1 text-ellipsis overflow-hidden whitespace-nowrap"><span class="font-bold group-hover:underline text-xs truncate"><span class="text-xs font-light mr-1">by</span>Sai Joshitha Kathari</span><span class="text-xs font-light opacity-50">|</span><span class="text-sm false text-ellipsis overflow-hidden whitespace-nowrap" title="@sai-joshitha-kathari">@<!-- -->sai-joshitha-kathari</span></span><span class="text-xs text-ellipsis overflow-hidden whitespace-nowrap mt-0.5" title="Senior Site Reliability Engineer at ">Senior Site Reliability Engineer<!-- --> at </span></span></a><p class="text-sm overflow-x-auto mt-2 text-bodyTxtLight">Senior Site Reliability Engineer based in Austin, Texas, specializing in distributed systems, Kubernetes, and production reliability. </p><div class="mt-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div></div></div></div><div class="xl:hidden"><div class="group transition-all duration-300"><div class="p-2 border border-lightBorder bg-light rounded-lg transition-all duration-300 pb-0"><span class="font-bold mx-2 text-sm text-lightTextLight">Story&#x27;s Credibility</span><div class="
mt-2 flex gap-2 flex-wrap m-2 transition-all duration-300 ease-in-out
flex-row
"><div class="flex items-start gap-2 text-sm rounded-lg max-w-[250px]
transition-all duration-300 ease-in-out p-1
cursor-default"><img alt="Original Reporting" loading="lazy" width="16" height="16" decoding="async" data-nimg="1" class="w-4 h-4 rounded-full transition-transform duration-300 group-hover:scale-105 cursor-pointer" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/img-oi03r0q.png?auto=format%2Ccompress&amp;w=32 1x" src="https://hackernoon.imgix.net/images/img-oi03r0q.png?auto=format%2Ccompress&amp;w=32"/></div></div></div></div></div></div></ul></div></div></div><div class="flex-1 flex flex-col justify-center "><div class=""><div class=" story-body font-sans flex items-center justify-center "><div class="prose max-w-[1020px] min-w-[150px] xs:p-0 lg:px-0 prose-a:break-words prose-table:table prose-div:bg-transparent prose-table:!table prose-table:w-max prose-table:min-w-full prose-td_a:whitespace-nowrap prose-td_a:break-keep prose-td_a:overflow-wrap-normal prose-td_a:word-break-normal [&amp;_th]:!hyphens-none [&amp;_td]:!hyphens-none [&amp;_th]:!break-normal [&amp;_td]:!break-normal [&amp;_th]:![overflow-wrap:normal] [&amp;_td]:![overflow-wrap:break-word] [&amp;_th]:whitespace-nowrap xs:prose-table:mx-auto prose-table:overflow-x-auto leading-relaxed tracking-wide prose-p:text-lightTextLight prose-strong:text-lightTextStrong prose-strong:font-bold prose-small:text-lightTextLight prose-small:font-light prose-a:text-lightTextLight prose-p:mx-0 prose-p:my-2 prose [&amp;_.line-space]:my-0 prose-p:my-2 prose-p:text-base sm:prose-p:text-lg prose-blockquote:my-0 prose-blockquote:border-l-[5px] prose-blockquote:border-lightTextAccent prose-blockquote:pl-4 prose-blockquote:leading-relaxed prose-blockquote:text-lightText prose-h2:text-lightTextStrong prose-li:marker:text-lightText prose-h2:text-xl sm:prose-h2:text-3xl prose-h2:font-bold prose-h2:my-6 prose-h3:text-lightTextStrong prose-hr:m-2 prose-h3:text-xl sm:prose-h3:text-2xl prose-h3:font-bold prose-h3:my-5 prose-h4:text-xl prose-h4:font-bold prose-h4:my-4 prose-td:text-lightTextLight prose-td:border prose-td:border-lightBorder prose-td:px-2 prose-td:[&amp;_p]:my-0 prose-th:[&amp;_p]:my-0 prose-th:text-lightTextLight prose-th:border prose-th:border-lightBorder prose-th:px-2 prose-li:text-lg prose-li:text-lightTextLight prose-li:px-0 prose-li:mb-3 prose-li:ml-3 prose-li:leading-relaxed prose-ul:pl-2 prose-ul:sm:pl-8 prose-ol:pl-2 prose-ol:sm:pl-8 hover:prose-a:text-lightTextAccent prose-a:rounded prose-code:text-lightTextLight prose-code:break-all prose-pre:rounded-lg prose-pre:text-sm prose-pre:my-4 prose-pre:p-3 prose-pre:overflow-x-scroll prose-pre:whitespace-pre-wrap prose-pre:break-words "><div class="w-full flex items-center justify-center "><p><strong>Five key takeaways:</strong></p>
<ol>
<li>Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task.</li>
<li>Alert triage, anomaly detection, incident summaries, capacity forecasting — these are the easy wins. Low risk, high value.</li>
<li>Production changes, incident command, security response, severity calls — keep a human&#x27;s name on these. Always.</li>
<li>Reversibility and blast radius are better questions than &quot;can the AI do this.&quot;</li>
<li>The goal isn&#x27;t AI replacing engineers. It&#x27;s AI clearing enough noise that engineers can actually think.</li>
</ol>
<hr/>
<p>I&#x27;ve sat through the version of this conversation that sounds like a vendor pitch — AI triages everything, drafts your runbooks, predicts outages before they happen, and nobody gets paged at 2 a.m. anymore. I&#x27;ve also watched the other version happen in real time: an automated remediation script restarts the wrong service, confidently, at 11 p.m., and a 20-minute blip turns into a four-hour outage while everyone tries to figure out why the &quot;fix&quot; made things worse.</p>
<p>Both of those are real. AI is already inside SRE workflows whether or not anyone signed off on it — the question that actually matters is where it belongs, and where a human still needs to be the one holding the decision.</p>
<p>None of what follows comes from a whitepaper. It&#x27;s from watching what breaks when teams move too fast with this stuff, and what quietly gets better when they don&#x27;t.</p>
<h2 id="h-reversibility-and-blast-radius">Reversibility and blast radius</h2>
<p>Here&#x27;s the mental model I keep coming back to before automating anything: can you undo it, and how bad is it if you&#x27;re wrong?</p>
<p>Restarting a pod — reversible, low stakes. Deleting a database backup — not reversible, at all. Scaling a service up is easy to walk back. Silencing an alert for six hours is technically reversible too, except the six hours where something real happened and nobody saw it isn&#x27;t something you get back.</p>
<p>Blast radius is the other half of it, and it&#x27;s not the same thing as severity. A misclassified low-priority alert costs a few wasted minutes. A misrouted sev-1 costs an hour of response time during an active outage, while the right team sits there not knowing they should be paged. And blast radius scales with what the action touches — one service versus a shared piece of infrastructure everything depends on, even when both look equally &quot;minor&quot; on paper.</p>
<p>Anything with low reversibility and a wide blast radius shouldn&#x27;t be running on autopilot. Anything reversible and contained is fair game. The stuff in between is where you actually need judgment — specifically, judgment from the people who&#x27;ll be the ones on call when it goes sideways.</p>
<p>Notice this framing never asks whether the AI <em>can</em> do something. It asks what happens if it&#x27;s wrong. That&#x27;s the more useful question, and it&#x27;s the one most teams skip.</p>
<h2 id="h-where-this-actually-works-well">Where this actually works well</h2>
<p><strong>Alert noise.</strong> This is the least controversial win there is. Somewhere between 30 and 60% of production alerts are noise by the time a human sees them — duplicates, transients, things that resolved themselves three minutes ago. AI grouping related alerts, suppressing known-flapping signals, correlating spikes with recent deploys — worst case, something gets mislabeled and a human still catches it. Low blast radius, fully reversible. This is exactly the profile you want.</p>
<p>One catch: it only works well tuned to your environment, not a generic model. An alert that always fires right before a nightly batch job and clears itself a minute later is trivial to suppress — but only if the model actually knows about your batch schedule. Skip that step and you&#x27;ve just added a second layer of noise on top of the first.</p>
<p><strong>First drafts of runbooks and postmortems.</strong> Runbook rot is one of the oldest problems in this field. The doc that was accurate in 2022 is a landmine now — nobody updates it, an incident hits, someone follows it anyway, and step four references a service that got decommissioned eight months ago. AI is genuinely good at pulling together a first draft from past incidents, change logs, whatever documentation exists. Same for postmortems — a draft that someone who actually lived through the incident reviews before it goes out saves real hours.</p>
<p><strong>Forecasting and anomaly detection.</strong> This is pattern matching, and models are good at pattern matching. A holiday traffic spike that happens once a year gives engineers almost no reps to build intuition about — but a model trained across several years of that same spike has plenty. The important part: keep this as a recommendation a human acts on, not something that auto-provisions infrastructure on its own. The moment it stops informing a decision and starts making one, the blast radius changes.</p>
<p><strong>Narrow, well-understood auto-remediation.</strong> This one comes with real caveats, but it earns its place. A specific service that needs a restart when it hits a known stuck state, a queue that needs draining past a defined threshold — fine, if the failure class is precisely defined, tested, and low-blast-radius by design. And there has to be a circuit breaker. If the fix doesn&#x27;t work within a set window, it stops and escalates instead of retrying forever on a wrong diagnosis. Automation that keeps trying the same broken fix is worse than doing nothing.</p>
<h2 id="h-where-it-doesnt-belong">Where it doesn&#x27;t belong</h2>
<p><strong>Severity calls.</strong> Get this wrong either direction and it costs you. A real sev-1 marked as low pulls in the wrong people at the wrong urgency while an SLA clock runs. A minor issue marked critical drags a response team into something that didn&#x27;t need them at 3 a.m. AI can surface context and flag patterns worth escalating — but the actual call needs a name attached, someone accountable for it. &quot;The model said it was low severity&quot; doesn&#x27;t hold up in a postmortem.</p>
<p><strong>Production changes without sign-off.</strong> Config changes, scaling decisions, anything touching a database directly, restarts outside that narrow bounded case above — a human authorizes these. AI can prep the change, check it against known-good patterns, even simulate the blast radius. What it shouldn&#x27;t do is decide the moment is right and pull the trigger itself.</p>
<p><strong>Security incidents.</strong> Different risk shape entirely. Miss something real and an active compromise sits there while the system waits for more confirmation. False-positive and you&#x27;ve locked out legitimate engineers mid-response. AI correlating logs to surface signal fast — genuinely useful. Containment and escalation decisions — that needs someone who can weigh legal and business context a model was never trained on.</p>
<p><strong>Root cause, as a stated fact.</strong> AI narrowing the search space by correlating deploy timing with metric shifts is useful groundwork. But writing &quot;root cause: X&quot; in a postmortem is a claim that shapes what the org fixes next and what it decides to ignore. Get that wrong because a correlation looked convincing, and the actual bug ships again next quarter.</p>
<p><strong>Who to escalate to.</strong> This is context a model just doesn&#x27;t have — who&#x27;s already underwater tonight, what else is on fire across the org, whether the responding engineer&#x27;s confidence is real or performed. Escalation is a trust call as much as a technical one.</p>
<h2 id="h-the-thing-nobodys-measuring">The thing nobody&#x27;s measuring</h2>
<p>There&#x27;s a slower cost that never shows up in a single incident review: engineers stop building intuition when AI absorbs all the routine reps. The edge cases are exactly where judgment matters most — and they&#x27;re exactly the cases you need practice on the boring stuff to be ready for. A team leaning hard on automation can look great for a long stretch, right up until something shows up that doesn&#x27;t match anything the model — or the team — has seen before.</p>
<p>This isn&#x27;t an argument against automating things. It&#x27;s an argument for being honest about which reps you&#x27;re willing to give away.</p>
<h2 id="h-a-few-practices-worth-adopting">A few practices worth adopting</h2>
<ul>
<li>Decide, as a team, which categories of action AI can take alone versus which need a sign-off — decide this before an incident forces the question at 2 a.m.</li>
<li>Keep an actual human accountable for anything irreversible. Not nominally &quot;in the loop&quot; — actually reviewing before it executes.</li>
<li>Build in a circuit breaker for anything automated. If it doesn&#x27;t work within a defined window, it escalates instead of retrying.</li>
<li>Rotate people through the routine cases sometimes, even when AI could handle it, so the skill doesn&#x27;t quietly disappear.</li>
<li>Revisit the boundary as systems change. A failure class that was well-understood six months ago might not be anymore after an architecture shift.</li>
</ul>
<p>Skip this and you end up with automation debt, eroded skills, and a production system nobody fully understands anymore — which is a worse place to be than where you started.</p>
<h2 id="h-where-this-leaves-things">Where this leaves things</h2>
<p>It&#x27;s not really a question of whether to use AI. It&#x27;s whether you&#x27;re using it somewhere judgment genuinely isn&#x27;t needed, or somewhere it is and you&#x27;ve just decided waiting for a human is too slow.</p>
<p>One of those is a real force multiplier. The other is a liability with a delay timer on it.</p></div></div></div></div></div><div class="hidden xl:flex xl:flex-col self-stretch"><div class="sticky top-[99px] px-3"><div class=" flex flex-col flex-row-reverse items-start gap-4 "><span class="tooltip tooltip-left cursor-pointer" data-tip="Bookmark"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-bookmark text-lightText text-2xl"></i></button></span><span class="tooltip tooltip-left cursor-pointer" data-tip="Comment"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-comment text-lightText text-2xl"></i></button></span><div class="dropdown dropdown-bottom dropdown-hover group "><label tabindex="0" class="flex items-center cursor-pointer justify-center border border-lightBorder 3xl:group-hover:bg-lightAlt group-hover:bg-light h-[40px] w-[40px] p-2 rounded "><i class="hn hn-share text-2xl"></i></label><ul tabindex="0" class="dropdown-content bg-light z-[1] py-4 px-4 3xl:px-0 3xl:py-2 border 3xl:border-none flex flex-col items-center justify-center gap-2 "><button class="border p-2 rounded hover:bg-lightAlt"><i class=" hn hn-copy text-lightText text-2xl "></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-facebook-round text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-x text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-linkedin text-lightText text-2xl"></i></button><a href="mailto:?subject=I&#x27;d like to share a link with you &amp;body=" class="border p-2 rounded inline-block hover:bg-lightAlt"><i class="hn hn-envelope text-lightText text-2xl"></i></a></ul></div></div></div></div></div><div class="px-4 lg:px-0 mx-auto w-full lg:max-w-[1000px] flex-col flex items-center justify-center "></div><div id="aboutCard" class=" max-w-[1000px] mx-auto flex flex-col items-center gap-6 "><div class="w-full lg:border border-lightBorder rounded-2xl"><div class=" w-full px-4 py-3 sm:px-8 sm:py-6 "><h3 class="text-xl xs:text-2xl sm:text-3xl font-bold mb-6">About Author</h3><div class="flex flex-col items-start"><div class="flex gap-4 flex-row items-start w-full"><div class="relative shadow-md rounded-full flex-shrink-0 min-w-[50px] w-[50px] h-[50px] sm:min-w-[75px] sm:h-[75px] ring-4 ring-gray-300"><a href="/u/sai-joshitha-kathari"><img alt="Sai Joshitha Kathari HackerNoon profile picture" loading="lazy" decoding="async" data-nimg="fill" class="rounded-full" style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;object-fit:cover;color:transparent" sizes="100vw" srcSet="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=640 640w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=750 750w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=828 828w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=1080 1080w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=1200 1200w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=1920 1920w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=2048 2048w, https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=3840 3840w" src="https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg?auto=format%2Ccompress&amp;w=3840"/></a></div><div class="flex-1 min-w-0 flex flex-col justify-center"><div class="flex flex-col"><div class="flex flex-wrap items-center gap-1 sm:gap-2 text-base text-bodyTxtLight"><span class="text-xs font-light mr-1">by</span><a class="hover:underline" href="/u/sai-joshitha-kathari"><strong class="font-bold text-lightTextStrong">Sai Joshitha Kathari</strong></a><span class="text-xs font-light opacity-50">|</span><a class="hover:underline text-sm sm:text-base text-bodyTxtLight" href="/u/sai-joshitha-kathari">@<!-- -->sai-joshitha-kathari</a></div><div class="text-sm text-bodyTxtLight mt-1">Senior Site Reliability Engineer<!-- --> at <!-- --> <span class="font-medium"></span></div></div></div></div><p class="text-sm text-bodyTxtLight break-words overflow-wrap mt-4 mb-4 w-full">Senior Site Reliability Engineer based in Austin, Texas, specializing in distributed systems, Kubernetes, and production reliability. </p><div class="w-full mb-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div><div class="flex-1 w-full"><div class="flex flex-col flex-wrap gap-2 items-start justify-start mt-2 mb-2"></div><div class="flex flex-col sm:flex-row gap-4 w-full"><a class="text-base flex-1 px-4 py-2 font-bold rounded-lg border-2 border-lightBorder transition text-center w-full sm:w-auto bg-light hover:bg-lightAlt text-lightText hover:bg-bodyAccent hover:text-bodyAccentTxt " href="/u/sai-joshitha-kathari">Read my stories</a><a class="text-base break-words flex-1 break-all px-4 py-2 font-bold rounded-lg border-2 border-lightBorder transition text-center w-full sm:w-auto bg-light hover:bg-lightAlt text-lightText hover:bg-bodyAccent hover:text-bodyAccentTxt " href="/about/sai-joshitha-kathari">Learn More</a></div></div></div></div></div><span id="aboutCard" class="hidden"></span><section class="w-full py-3 px-4 sm:px-0 sm:py-6 "><h4 class="text-xl xs:text-2xl sm:text-3xl font-bold mb-4 sm:mb-6">TOPICS</h4><div class="flex flex-wrap gap-2"><div class=" flex flex-wrap items-center gap-2 border-lightBorder"><a href="/c/engineering" target="_blank" rel="noopener noreferrer" class="text-lg border-lightBorder hover:bg-lightAccent hover:text-lightAccentText hover:border-lightAccentText bg-lightAlt text-lightText flex items-center px-2 py-1 border rounded"><span class="mr-2"><i class="hn hn-programming !leading-[inherit]"></i></span><span>Software Engineering</span></a></div><a href="/tagged/site-reliability-engineering" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->site-reliability-engineering</a><a href="/tagged/site-reliability-engineer" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->site-reliability-engineer</a><a href="/tagged/artificial-intelligence" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->artificial-intelligence</a><a href="/tagged/devops" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->devops</a><a href="/tagged/kubernetes" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->kubernetes</a><a href="/tagged/incident-management" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->incident-management</a><a href="/tagged/automation" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->automation</a><a href="/tagged/ai-in-production" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->ai-in-production</a></div></section></div></div></div></div></div><div class="min-h-[200px]"></div></main><div class="flex flex-col gap-4 hidden"><button class="mr-auto"><i class="hn-sun hn text-2xl"></i></button><h2 class="text-sm font-semibold text-darkText">Light-Mode</h2><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Classic</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#0F0"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#F5EC43"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#212428"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Newspaper</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#FFFFFF"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#F5F5F5"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#454545"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Proof of Usefulness</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#FFFFFF"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#26AB5C"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#D2FBE2"></span></div></div><h2 class="text-sm font-semibold text-darkText">Dark-Mode</h2><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Neon Noir</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#1E1E1E"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#0F0"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#F5EC43"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Minty</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#061F19"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#2AAA74"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#63FF86"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Startups of the Year</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#08085E"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#A2EF44"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#1B1B95"></span></div></div></div></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"data":{"pageLang":"en","datePublished":"2026-08-27","slug":"what-sres-should-automate-and-never-automate-with-ai","articleBody":"Five key takeaways: Five key takeaways: Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task.\nAlert triage, anomaly detection, incident summaries, capacity forecasting — these are the easy wins. Low risk, high value.\nProduction changes, incident command, security response, severity calls — keep a human's name on these. Always.\nReversibility and blast radius are better questions than \"can the AI do this.\"\nThe goal isn't AI replacing engineers. It's AI clearing enough noise that engineers can actually think. Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task. Alert triage, anomaly detection, incident summaries, capacity forecasting — these are the easy wins. Low risk, high value. Production changes, incident command, security response, severity calls — keep a human's name on these. Always. Reversibility and blast radius are better questions than \"can the AI do this.\" The goal isn't AI replacing engineers. It's AI clearing enough noise that engineers can actually think. I've sat through the version of this conversation that sounds like a vendor pitch — AI triages everything, drafts your runbooks, predicts outages before they happen, and nobody gets paged at 2 a.m. anymore. I've also watched the other version happen in real time: an automated remediation script restarts the wrong service, confidently, at 11 p.m., and a 20-minute blip turns into a four-hour outage while everyone tries to figure out why the \"fix\" made things worse. Both of those are real. AI is already inside SRE workflows whether or not anyone signed off on it — the question that actually matters is where it belongs, and where a human still needs to be the one holding the decision. None of what follows comes from a whitepaper. It's from watching what breaks when teams move too fast with this stuff, and what quietly gets better when they don't. Reversibility and blast radius Here's the mental model I keep coming back to before automating anything: can you undo it, and how bad is it if you're wrong? Restarting a pod — reversible, low stakes. Deleting a database backup — not reversible, at all. Scaling a service up is easy to walk back. Silencing an alert for six hours is technically reversible too, except the six hours where something real happened and nobody saw it isn't something you get back. Blast radius is the other half of it, and it's not the same thing as severity. A misclassified low-priority alert costs a few wasted minutes. A misrouted sev-1 costs an hour of response time during an active outage, while the right team sits there not knowing they should be paged. And blast radius scales with what the action touches — one service versus a shared piece of infrastructure everything depends on, even when both look equally \"minor\" on paper. Anything with low reversibility and a wide blast radius shouldn't be running on autopilot. Anything reversible and contained is fair game. The stuff in between is where you actually need judgment — specifically, judgment from the people who'll be the ones on call when it goes sideways. Notice this framing never asks whether the AI can do something. It asks what happens if it's wrong. That's the more useful question, and it's the one most teams skip. can Where this actually works well Alert noise. This is the least controversial win there is. Somewhere between 30 and 60% of production alerts are noise by the time a human sees them — duplicates, transients, things that resolved themselves three minutes ago. AI grouping related alerts, suppressing known-flapping signals, correlating spikes with recent deploys — worst case, something gets mislabeled and a human still catches it. Low blast radius, fully reversible. This is exactly the profile you want. Alert noise. One catch: it only works well tuned to your environment, not a generic model. An alert that always fires right before a nightly batch job and clears itself a minute later is trivial to suppress — but only if the model actually knows about your batch schedule. Skip that step and you've just added a second layer of noise on top of the first. First drafts of runbooks and postmortems. Runbook rot is one of the oldest problems in this field. The doc that was accurate in 2022 is a landmine now — nobody updates it, an incident hits, someone follows it anyway, and step four references a service that got decommissioned eight months ago. AI is genuinely good at pulling together a first draft from past incidents, change logs, whatever documentation exists. Same for postmortems — a draft that someone who actually lived through the incident reviews before it goes out saves real hours. First drafts of runbooks and postmortems. Forecasting and anomaly detection. This is pattern matching, and models are good at pattern matching. A holiday traffic spike that happens once a year gives engineers almost no reps to build intuition about — but a model trained across several years of that same spike has plenty. The important part: keep this as a recommendation a human acts on, not something that auto-provisions infrastructure on its own. The moment it stops informing a decision and starts making one, the blast radius changes. Forecasting and anomaly detection. Narrow, well-understood auto-remediation. This one comes with real caveats, but it earns its place. A specific service that needs a restart when it hits a known stuck state, a queue that needs draining past a defined threshold — fine, if the failure class is precisely defined, tested, and low-blast-radius by design. And there has to be a circuit breaker. If the fix doesn't work within a set window, it stops and escalates instead of retrying forever on a wrong diagnosis. Automation that keeps trying the same broken fix is worse than doing nothing. Narrow, well-understood auto-remediation. Where it doesn't belong Severity calls. Get this wrong either direction and it costs you. A real sev-1 marked as low pulls in the wrong people at the wrong urgency while an SLA clock runs. A minor issue marked critical drags a response team into something that didn't need them at 3 a.m. AI can surface context and flag patterns worth escalating — but the actual call needs a name attached, someone accountable for it. \"The model said it was low severity\" doesn't hold up in a postmortem. Severity calls. Production changes without sign-off. Config changes, scaling decisions, anything touching a database directly, restarts outside that narrow bounded case above — a human authorizes these. AI can prep the change, check it against known-good patterns, even simulate the blast radius. What it shouldn't do is decide the moment is right and pull the trigger itself. Production changes without sign-off. Security incidents. Different risk shape entirely. Miss something real and an active compromise sits there while the system waits for more confirmation. False-positive and you've locked out legitimate engineers mid-response. AI correlating logs to surface signal fast — genuinely useful. Containment and escalation decisions — that needs someone who can weigh legal and business context a model was never trained on. Security incidents. Root cause, as a stated fact. AI narrowing the search space by correlating deploy timing with metric shifts is useful groundwork. But writing \"root cause: X\" in a postmortem is a claim that shapes what the org fixes next and what it decides to ignore. Get that wrong because a correlation looked convincing, and the actual bug ships again next quarter. Root cause, as a stated fact. Who to escalate to. This is context a model just doesn't have — who's already underwater tonight, what else is on fire across the org, whether the responding engineer's confidence is real or performed. Escalation is a trust call as much as a technical one. Who to escalate to. The thing nobody's measuring There's a slower cost that never shows up in a single incident review: engineers stop building intuition when AI absorbs all the routine reps. The edge cases are exactly where judgment matters most — and they're exactly the cases you need practice on the boring stuff to be ready for. A team leaning hard on automation can look great for a long stretch, right up until something shows up that doesn't match anything the model — or the team — has seen before. This isn't an argument against automating things. It's an argument for being honest about which reps you're willing to give away. A few practices worth adopting Decide, as a team, which categories of action AI can take alone versus which need a sign-off — decide this before an incident forces the question at 2 a.m.\nKeep an actual human accountable for anything irreversible. Not nominally \"in the loop\" — actually reviewing before it executes.\nBuild in a circuit breaker for anything automated. If it doesn't work within a defined window, it escalates instead of retrying.\nRotate people through the routine cases sometimes, even when AI could handle it, so the skill doesn't quietly disappear.\nRevisit the boundary as systems change. A failure class that was well-understood six months ago might not be anymore after an architecture shift. Decide, as a team, which categories of action AI can take alone versus which need a sign-off — decide this before an incident forces the question at 2 a.m. Keep an actual human accountable for anything irreversible. Not nominally \"in the loop\" — actually reviewing before it executes. Build in a circuit breaker for anything automated. If it doesn't work within a defined window, it escalates instead of retrying. Rotate people through the routine cases sometimes, even when AI could handle it, so the skill doesn't quietly disappear. Revisit the boundary as systems change. A failure class that was well-understood six months ago might not be anymore after an architecture shift. Skip this and you end up with automation debt, eroded skills, and a production system nobody fully understands anymore — which is a worse place to be than where you started. Where this leaves things It's not really a question of whether to use AI. It's whether you're using it somewhere judgment genuinely isn't needed, or somewhere it is and you've just decided waiting for a human is too slow. One of those is a real force multiplier. The other is a liability with a delay timer on it.","arweave":"QHzOdJ8kR4xcrgdZBSNtCgUrTzL-70dw_Qo2rrDsXGI","createdAt":"2026-08-27T05:19:05.978Z","draftId":"6a7a7a897378522c3c58fcbe","emoji":[{"description":"This story contains new, firsthand information uncovered by the writer.","label":"Original Reporting","image":"https://cdn.hackernoon.com/images/img-oi03r0q.png","prompt":"","value":0}],"excerpt":"Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task.","firstSeenAt":false,"fromSlack":false,"id":"6a7a7a897378522c3c58fcbe","imageSizes":{},"linkAccreditation":{"goals":"","isBlogging":null,"isBusiness":null,"debut":true,"isPersonal":null},"mainImage":"https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg","mainImageHeight":512,"mainImageWidth":512,"markup":null,"owner":"s8yzU7tVitTuz0oU5YBte4GJad72","parsed":"\u003cp\u003e\u003cstrong\u003eFive key takeaways:\u003c/strong\u003e\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eAutomate based on impact and recoverability, not on whether the AI is technically capable of doing the task.\u003c/li\u003e\n\u003cli\u003eAlert triage, anomaly detection, incident summaries, capacity forecasting — these are the easy wins. Low risk, high value.\u003c/li\u003e\n\u003cli\u003eProduction changes, incident command, security response, severity calls — keep a human's name on these. Always.\u003c/li\u003e\n\u003cli\u003eReversibility and blast radius are better questions than \"can the AI do this.\"\u003c/li\u003e\n\u003cli\u003eThe goal isn't AI replacing engineers. It's AI clearing enough noise that engineers can actually think.\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003cp\u003eI've sat through the version of this conversation that sounds like a vendor pitch — AI triages everything, drafts your runbooks, predicts outages before they happen, and nobody gets paged at 2 a.m. anymore. I've also watched the other version happen in real time: an automated remediation script restarts the wrong service, confidently, at 11 p.m., and a 20-minute blip turns into a four-hour outage while everyone tries to figure out why the \"fix\" made things worse.\u003c/p\u003e\n\u003cp\u003eBoth of those are real. AI is already inside SRE workflows whether or not anyone signed off on it — the question that actually matters is where it belongs, and where a human still needs to be the one holding the decision.\u003c/p\u003e\n\u003cp\u003eNone of what follows comes from a whitepaper. It's from watching what breaks when teams move too fast with this stuff, and what quietly gets better when they don't.\u003c/p\u003e\n\u003ch2 id=\"h-reversibility-and-blast-radius\"\u003eReversibility and blast radius\u003c/h2\u003e\n\u003cp\u003eHere's the mental model I keep coming back to before automating anything: can you undo it, and how bad is it if you're wrong?\u003c/p\u003e\n\u003cp\u003eRestarting a pod — reversible, low stakes. Deleting a database backup — not reversible, at all. Scaling a service up is easy to walk back. Silencing an alert for six hours is technically reversible too, except the six hours where something real happened and nobody saw it isn't something you get back.\u003c/p\u003e\n\u003cp\u003eBlast radius is the other half of it, and it's not the same thing as severity. A misclassified low-priority alert costs a few wasted minutes. A misrouted sev-1 costs an hour of response time during an active outage, while the right team sits there not knowing they should be paged. And blast radius scales with what the action touches — one service versus a shared piece of infrastructure everything depends on, even when both look equally \"minor\" on paper.\u003c/p\u003e\n\u003cp\u003eAnything with low reversibility and a wide blast radius shouldn't be running on autopilot. Anything reversible and contained is fair game. The stuff in between is where you actually need judgment — specifically, judgment from the people who'll be the ones on call when it goes sideways.\u003c/p\u003e\n\u003cp\u003eNotice this framing never asks whether the AI \u003cem\u003ecan\u003c/em\u003e do something. It asks what happens if it's wrong. That's the more useful question, and it's the one most teams skip.\u003c/p\u003e\n\u003ch2 id=\"h-where-this-actually-works-well\"\u003eWhere this actually works well\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eAlert noise.\u003c/strong\u003e This is the least controversial win there is. Somewhere between 30 and 60% of production alerts are noise by the time a human sees them — duplicates, transients, things that resolved themselves three minutes ago. AI grouping related alerts, suppressing known-flapping signals, correlating spikes with recent deploys — worst case, something gets mislabeled and a human still catches it. Low blast radius, fully reversible. This is exactly the profile you want.\u003c/p\u003e\n\u003cp\u003eOne catch: it only works well tuned to your environment, not a generic model. An alert that always fires right before a nightly batch job and clears itself a minute later is trivial to suppress — but only if the model actually knows about your batch schedule. Skip that step and you've just added a second layer of noise on top of the first.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFirst drafts of runbooks and postmortems.\u003c/strong\u003e Runbook rot is one of the oldest problems in this field. The doc that was accurate in 2022 is a landmine now — nobody updates it, an incident hits, someone follows it anyway, and step four references a service that got decommissioned eight months ago. AI is genuinely good at pulling together a first draft from past incidents, change logs, whatever documentation exists. Same for postmortems — a draft that someone who actually lived through the incident reviews before it goes out saves real hours.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eForecasting and anomaly detection.\u003c/strong\u003e This is pattern matching, and models are good at pattern matching. A holiday traffic spike that happens once a year gives engineers almost no reps to build intuition about — but a model trained across several years of that same spike has plenty. The important part: keep this as a recommendation a human acts on, not something that auto-provisions infrastructure on its own. The moment it stops informing a decision and starts making one, the blast radius changes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNarrow, well-understood auto-remediation.\u003c/strong\u003e This one comes with real caveats, but it earns its place. A specific service that needs a restart when it hits a known stuck state, a queue that needs draining past a defined threshold — fine, if the failure class is precisely defined, tested, and low-blast-radius by design. And there has to be a circuit breaker. If the fix doesn't work within a set window, it stops and escalates instead of retrying forever on a wrong diagnosis. Automation that keeps trying the same broken fix is worse than doing nothing.\u003c/p\u003e\n\u003ch2 id=\"h-where-it-doesnt-belong\"\u003eWhere it doesn't belong\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eSeverity calls.\u003c/strong\u003e Get this wrong either direction and it costs you. A real sev-1 marked as low pulls in the wrong people at the wrong urgency while an SLA clock runs. A minor issue marked critical drags a response team into something that didn't need them at 3 a.m. AI can surface context and flag patterns worth escalating — but the actual call needs a name attached, someone accountable for it. \"The model said it was low severity\" doesn't hold up in a postmortem.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProduction changes without sign-off.\u003c/strong\u003e Config changes, scaling decisions, anything touching a database directly, restarts outside that narrow bounded case above — a human authorizes these. AI can prep the change, check it against known-good patterns, even simulate the blast radius. What it shouldn't do is decide the moment is right and pull the trigger itself.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSecurity incidents.\u003c/strong\u003e Different risk shape entirely. Miss something real and an active compromise sits there while the system waits for more confirmation. False-positive and you've locked out legitimate engineers mid-response. AI correlating logs to surface signal fast — genuinely useful. Containment and escalation decisions — that needs someone who can weigh legal and business context a model was never trained on.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRoot cause, as a stated fact.\u003c/strong\u003e AI narrowing the search space by correlating deploy timing with metric shifts is useful groundwork. But writing \"root cause: X\" in a postmortem is a claim that shapes what the org fixes next and what it decides to ignore. Get that wrong because a correlation looked convincing, and the actual bug ships again next quarter.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWho to escalate to.\u003c/strong\u003e This is context a model just doesn't have — who's already underwater tonight, what else is on fire across the org, whether the responding engineer's confidence is real or performed. Escalation is a trust call as much as a technical one.\u003c/p\u003e\n\u003ch2 id=\"h-the-thing-nobodys-measuring\"\u003eThe thing nobody's measuring\u003c/h2\u003e\n\u003cp\u003eThere's a slower cost that never shows up in a single incident review: engineers stop building intuition when AI absorbs all the routine reps. The edge cases are exactly where judgment matters most — and they're exactly the cases you need practice on the boring stuff to be ready for. A team leaning hard on automation can look great for a long stretch, right up until something shows up that doesn't match anything the model — or the team — has seen before.\u003c/p\u003e\n\u003cp\u003eThis isn't an argument against automating things. It's an argument for being honest about which reps you're willing to give away.\u003c/p\u003e\n\u003ch2 id=\"h-a-few-practices-worth-adopting\"\u003eA few practices worth adopting\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eDecide, as a team, which categories of action AI can take alone versus which need a sign-off — decide this before an incident forces the question at 2 a.m.\u003c/li\u003e\n\u003cli\u003eKeep an actual human accountable for anything irreversible. Not nominally \"in the loop\" — actually reviewing before it executes.\u003c/li\u003e\n\u003cli\u003eBuild in a circuit breaker for anything automated. If it doesn't work within a defined window, it escalates instead of retrying.\u003c/li\u003e\n\u003cli\u003eRotate people through the routine cases sometimes, even when AI could handle it, so the skill doesn't quietly disappear.\u003c/li\u003e\n\u003cli\u003eRevisit the boundary as systems change. A failure class that was well-understood six months ago might not be anymore after an architecture shift.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eSkip this and you end up with automation debt, eroded skills, and a production system nobody fully understands anymore — which is a worse place to be than where you started.\u003c/p\u003e\n\u003ch2 id=\"h-where-this-leaves-things\"\u003eWhere this leaves things\u003c/h2\u003e\n\u003cp\u003eIt's not really a question of whether to use AI. It's whether you're using it somewhere judgment genuinely isn't needed, or somewhere it is and you've just decided waiting for a human is too slow.\u003c/p\u003e\n\u003cp\u003eOne of those is a real force multiplier. The other is a liability with a delay timer on it.\u003c/p\u003e","profile":{"handle":"sai-joshitha-kathari","displayName":"Sai Joshitha Kathari","bio":"Senior Site Reliability Engineer based in Austin, Texas, specializing in distributed systems, Kubernetes, and production reliability. ","avatar":"https://cdn.hackernoon.com/images/s8yzU7tVitTuz0oU5YBte4GJad72-ok035xu.jpg","isBrand":false,"currentJob":{"title":"Senior Site Reliability Engineer","company":"","startDate":""},"jobHistory":[{"title":"Senior Devops Engineer","company":"ATOS","startDate":"","endDate":""}],"about_page_settings":{"blocked":false,"createdAt":"2026-08-27T05:19:28.852Z","updatedAt":"2026-08-27T05:19:28.852Z","style":{"headline_pos":"center","layout":0,"skin":0},"published":true,"owner":"s8yzU7tVitTuz0oU5YBte4GJad72"},"callToActions":[{"active":true,"icon":"fa fa-book","name":"Read My Stories","url":"https://hackernoon.com/u/sai-joshitha-kathari","id":"127c21b8a1f"}],"isTrusted":false,"allowSubscribers":true},"publishedAt":1787807948.183,"super_category":"engineering","tags":["site-reliability-engineering","site-reliability-engineer","artificial-intelligence","devops","kubernetes","incident-management","automation","ai-in-production"],"title":"What SREs Should Automate — and Never Automate — with AI","tldr":"Automate based on impact and recoverability, not on whether the AI is technically capable of doing the task.","youtubeTranscriptData":null,"backlinks":{"fetched":"2026-08-27T05:19:09.698Z","urls":["https://x.com/hackernoon/status/2092844342687773142","https://bsky.app/profile/hackernoon.com/post/3mu23wvhguc2c"]},"parentCategory":"programming","annotations":[],"coAuthorProfiles":[],"commentsCount":0,"fromMongo":true,"relatedStories":[{"title":"Automate AEM Offline Revision Cleanup with This Powerful Bash Script","mainImage":"https://cdn.hackernoon.com/images/0xdv1H3SiLXiSmbJdxrqlY0JO5I2-8s13203.png","slug":"automate-aem-offline-revision-cleanup-with-this-powerful-bash-script","tags":["devops-tools","aem-oak-cleanup","scripting","bash-scripting","software-maintenance","oak-compaction-bash-script","aem-devops-automation","aem-offline-revision-cleanup"],"excerpt":"Automate AEM Offline Revision Cleanup with a script that transforms manual maintenance into a one-command operation with built-in safety checks and monitoring.","publishedAt":1740404007995,"profile":{"handle":"realgpp","avatar":"https://cdn.hackernoon.com/avatars/robot-b2.png","displayName":"Giuseppe Baglio","isBrand":false},"recommended":true},{"title":"Automate All Production: Leisure, Not Work, Is Our Goal","mainImage":"https://cdn.hackernoon.com/images/BidO7U8T9IQmETD142QgQ3cMVSH3-e6038x3.jpeg","slug":"automate-all-production-leisure-not-work-is-our-goal","tags":["future-of-work","automation","technology","business","which-jobs-are-futureproof","career-advice","types-of-work-for-the-future","bertrand-russell"],"excerpt":"Work should be a vocation.","publishedAt":1636377172376,"profile":{"handle":"djcampbell","avatar":"https://cdn.hackernoon.com/images/undefined-3583ohu.jpeg","displayName":"DJCampbell","isBrand":false},"recommended":true},{"title":"Automate and Deploy a Docker Container to Google Cloud Run from Scratch Using Pulumi and Go","mainImage":"https://cdn.hackernoon.com/images/2CnaTwkypCg3LK5uANIw5XkUw453-io92oit.jpeg","slug":"automate-and-deploy-a-docker-container-to-google-cloud-run-from-scratch-using-pulumi-and-go","tags":["devops","docker","golang","pulumi","google-cloud-run","container-deployment","google-cloud-platform","deploy-a-docker-container"],"excerpt":"Automate and Deploy a Docker Container to Google Cloud Run from Scratch using Pulumi and Go with minimal permissions.\n\nStep by Step","publishedAt":1710166330988,"profile":{"handle":"josejaviasilis","avatar":"https://cdn.hackernoon.com/images/2CnaTwkypCg3LK5uANIw5XkUw453-61037c6.jpeg","displayName":"Jose Javi Asilis","isBrand":false},"recommended":true},{"title":"Automate API monitoring with this open source software","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"automate-api-monitoring-with-this-open-source-software-707cdb187eca","tags":["javascript","api","api-management","mongodb","open-source"],"excerpt":"Nowadays, so many software applications rely on 3rd party services to power their features. For example, one application may use the \u003ca href=\"https://hackernoon.com/tagged/google\" target=\"_blank\"\u003eGoogle\u003c/a\u003e Maps API to place markers on different locations on a map or posting to social media through the Instagram or Twitter API. It’s not always easy and some what of a waste of time to track the different API’s to ensure that they are functioning as they should.","publishedAt":1533264136439,"profile":{"handle":"simranjitkamboj","avatar":"https://hackernoon.com/fallback-profile.png","displayName":"Simranjit Kamboj"},"recommended":true},{"title":"Automate Customer Creation in Oracle Apps R12 with This Simple API Trick","mainImage":"https://cdn.hackernoon.com/images/arbitrary-graph-on-a-large-computer-screen-knj1kwpwvnnszfcr70npo6b0.png","slug":"automate-customer-creation-in-oracle-apps-r12-with-this-simple-api-trick","tags":["oracle","oracle-r12","oracle-customer-creation","oracle-apps","oracle-erp-automation","trading-community-architecture","oracle-ar-customer-setup","database-workflow-automation"],"excerpt":"\u003cp\u003eCustomers are the core of the Order-to-Cash (O2C) cycle in Oracle Applications R12. Managing customer data efficiently is critical for processing sales orders, invoices, payments, and overall financial reporting. While customer records can be created manually through the Oracle Receivables (AR) forms, many organizations prefer using APIs for automation, bulk uploads, and integration with external [\u0026#8230;]\u003c/p\u003e\n\u003cp\u003eThe post \u003ca href=\"https://vinish.dev/create-customer-using-api-in-oracle-apps-r12\"\u003eHow to Create Customer Using API in Oracle Apps R12\u003c/a\u003e appeared first on \u003ca href=\"https://vinish.dev\"\u003eVinish.Dev\u003c/a\u003e.\u003c/p\u003e","publishedAt":1757066404187,"profile":{"handle":"vinish","avatar":"https://cdn.hackernoon.com/avatars/robot-a1.png","displayName":"Vinish","isBrand":false},"recommended":true},{"title":"Automate Designs with Bannerbear and n8n","mainImage":"https://cdn.hackernoon.com/drafts/323y2b1m.png","slug":"automate-designs-with-bannerbear-and-n8n-r69a3vwr","tags":["tutorial","graphic-design","n8n","bannerbear","api","marketing-automation","no-code","graphics"],"excerpt":"As a designer and self-proclaimed data nerd, I’ve been involved in proceduralizing creative deliverables for some time. Up until now however, my concepts always had to reconcile with the limitations of InDesign macros or my coding skills. n8n empowers me to apply my basic understanding of data objects and the interwebs to create some pretty slick automations with minimal effort.","publishedAt":1590102917835,"profile":{"handle":"max-tkacz","avatar":"https://hackernoon.com/images/avatars/9FthIwwEe9VtHQt23QQXCUhFlTF2.jpg","displayName":"Max Tkacz"},"recommended":true},{"title":"Automate EC2 Deployments on AWS with Terraform Modules","mainImage":"https://cdn.hackernoon.com/images/l82ukgORhpX0BcfZBklvH6MHceU2-7l23tey.jpeg","slug":"automate-ec2-deployments-on-aws-with-terraform-modules","tags":["aws","terraform","devops-tools","devops","cloud-computing","cloud-infrastructure","infrastructure-as-code","terraform-guide"],"excerpt":"In cloud computing, managing infrastructure efficiently has now become an important part of modern infrastructure operations. ","publishedAt":1734972325982,"profile":{"handle":"omah","avatar":"https://cdn.hackernoon.com/images/l82ukgORhpX0BcfZBklvH6MHceU2-e183uhb.jpeg","displayName":"Ijay","isBrand":false},"recommended":true},{"title":"Automate GraphQL Backed Applications' Security Testing ","mainImage":"https://firebasestorage.googleapis.com/v0/b/hackernoon-app.appspot.com/o/images%2FadgItGyPsTYzWPaC2OwEOChrM5m2-k2o28dj.png?alt=media\u0026token=7ba1efa5-67fc-47eb-98a6-3084f3b2b4fa","slug":"automate-graphql-backed-applications-security-testing-n1t3xrz","tags":["graphql","cyber-security","web-security","software-engineering","devops-security","graphql-api","website-security","api-security"],"excerpt":"Working with the latest tech is fun. It’s fresh and exciting. As developers we \nfeel invigorated by being on the bleeding edge. Consider us thrill \nseekers.","publishedAt":1596997809215,"profile":{"handle":"adam-baldwin","avatar":"https://cdn.hackernoon.com/images%2FadgItGyPsTYzWPaC2OwEOChrM5m2-y90282v.png?alt=media\u0026token=42528e53-6724-44e5-9129-8eae13778650","displayName":"Adam Baldwin"},"recommended":true},{"title":"Automate implementing the DISA STIG for PostgreSQL","mainImage":"https://hackernoon.com/hn-images/1*S2jsrM3a0QT7jJ5Smm25gA.png","slug":"automate-implementing-the-disa-stig-for-postgresql-a7e3267c83d0","tags":["postgres","docker","devops","database","security"],"excerpt":"In this article I’ll show how to do a security audit of the configuration of a PostgreSQL server using \u003ca href=\"https://github.com/OSSIndex/DevAudit\" target=\"_blank\"\u003eDevAudit\u003c/a\u003e, an open-source cross-platform multi-purpose security auditing program, with an audit rule-set that automates checking a server’s compliance with the \u003ca href=\"https://www.crunchydata.com/postgres-stig/PGSQL-STIG-9.5+.pdf\" target=\"_blank\"\u003ePostgreSQL 9.x Security Technical Implementation Guide\u003c/a\u003e published by the United States Defense Information Systems Agency (DISA).","publishedAt":1495171252294,"profile":{"handle":"allisterb","avatar":"https://hackernoon.com/images/avatars/byyBIeQCMgdx887WlsJiWCUFDr22.jpg","displayName":"Allister Beharry"},"recommended":true},{"title":"Automate Infrastructure and Secure Your App with Terraform and Cognito","mainImage":"https://cdn.hackernoon.com/images/hZKeZMVXJERZHZxcljac4wmlXYC3-qh03z23.jpeg","slug":"automate-infrastructure-and-secure-your-app-with-terraform-and-cognito","tags":["terraform","aws-services","cloud-infrastructure","aws-cognito","web-development","infrastructure-as-code","aws-cloudformation","ansible"],"excerpt":"Terraform is an open-source tool that allows you to define and provision infrastructure using code.","publishedAt":1737804891855,"profile":{"handle":"bukolasobowale","avatar":"https://cdn.hackernoon.com/images/hZKeZMVXJERZHZxcljac4wmlXYC3-uj83y1t.jpeg","displayName":"Bukola Sobowale","isBrand":false},"recommended":true},{"title":"Automate JavaScript deployment of npm packages with Github actions","mainImage":"https://cdn.hackernoon.com/images/yvou3yaj.jpg","slug":"automate-javascript-deployment-of-npm-packages-with-github-actions-3e4o3y35","tags":["javascript","cicd","github-actions","npm","nodejs","continuous-integration","development","coding"],"excerpt":"What does Factorio and CI/CD pipelines have in common?","publishedAt":1587825903428,"profile":{"handle":"niclas","avatar":"https://hackernoon.com/images/avatars/yGoSiDlhAYUC9yOQxV7noULaNhn2.jpg","displayName":"niclas@bitfront.se"},"recommended":true},{"title":"Automate npm releases with semantic-release and human-written change logs","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"automate-npm-releases-with-semantic-release-and-human-written-change-logs-2adb1dce487","tags":["javascript","git","npm","open-source","nodejs"],"excerpt":"Making new releases is one of the most boring and tedious tasks in open source.","publishedAt":1478769897250,"profile":{"handle":"sapegin","avatar":"https://hackernoon.com/images/avatars/DyOr1KLFwfbejmL6GKoPp7fB8we2.jpg","displayName":"Artem Sapegin"},"recommended":true},{"title":"Automate or Perish: Why You Should Be Focused On Business Automation and DevOps","mainImage":"https://hackernoon.com/hn-images/1*SveFQMf2SnJSKE8ht6-Spg.png","slug":"automate-or-perish-why-you-should-be-focused-on-business-automation-and-devops-e4dcb6640913","tags":["cloud-computing","docker","artificial-intelligence","machine-learning","technology"],"excerpt":"Recently a lot of the people I come across in the tech industry are either working on some kind of Artificial Intelligence, machine learning, IoT (Internet of Things), or big data aggregation. Being a technologist, I’m well aware that A.I. and machine learning are the future and the implications for humanity will be amazing. But as a human being I am also aware that A.I. can and will put a lot of people out of work, but it doesn’t have to be that way.","publishedAt":1498170872362,"profile":{"handle":"trentlapinski","avatar":"https://cdn.hackernoon.com/avatars/robot-a6.png","displayName":"Trent Lapinski"},"recommended":true}],"staticData":{"frLangTooltip":"Lisez cette histoire en Français!","about":"About","enLangTooltip":"Read this story in the original language, English!","loggedOutBookmark":"Create an account to store your bookmarks","learnMore":"Learn More","stats":"Stats","editStory":"Edit Story","audioPresented":"Audio Presented by","by":"by","audioTranslationText":null,"newStory":"New Story","loggedInBookmark":"Bookmark story","esLangTooltip":"Lee esta historia en Español!","relatedStories":"RELATED STORIES","addComment":"Add Comment","ptLangTooltip":"Leia esta história em português!","hiLangTooltip":"इस कहानी को हिंदी में पढ़ें!","comments":"Comments","removeBookmark":"Remove bookmark","commentReply":"Reply","minutes":"min","reads":"reads","trLangTooltip":"Bu hikayeyi Türkçe okuyun!","tags":"TOPICS","jaLangTooltip":"この物語を日本語で読んでください!","bnLangTooltip":"এই গল্পটি বাংলায় পড়ুন!","storyMentions":"MENTIONED IN THIS STORY","ruLangTooltip":"Прочтите эту историю на русском языке!","deLangTooltip":"Lesen Sie diese Geschichte auf Deutsch!","featuredIn":"THIS ARTICLE WAS FEATURED IN","tldrTitle":"Too Long; Didn't Read","koLangTooltip":"이 이야기를 한국어로 읽어보세요!","zhLangTooltip":"用繁體中文閱讀這個故事!","viLangTooltip":"Đọc bài viết này bằng tiếng Việt!"},"searchTopics":["automate"],"stats":{"pageviews":569},"socialPreviewImage":"https://hackernoon.imgix.net/images/s8yzU7tVitTuz0oU5YBte4GJad72-5683bex.jpeg","gptZeroMsg":"This story is AI-assisted.","audioData":[{"url":"https://storage.googleapis.com/hackernoon/audios/6a7a7a897378522c3c58fcbe-en-US-Wavenet-I-MALE--8147770a3114d.mp3","nickname":"Dr. One (en-US)","avatar":"https://cdn.hackernoon.com/avatars/robot-b5.png","audioPath":"audios/6a7a7a897378522c3c58fcbe-en-US-Wavenet-I-MALE--8147770a3114d.mp3"},{"url":"https://storage.googleapis.com/hackernoon/audios/6a7a7a897378522c3c58fcbe-en-US-Wavenet-H-FEMALE--2771f75ff7d8.mp3","nickname":"Ms. Hacker (en-US)","avatar":"https://cdn.hackernoon.com/avatars/robot-b6.png","audioPath":"audios/6a7a7a897378522c3c58fcbe-en-US-Wavenet-H-FEMALE--2771f75ff7d8.mp3"}]},"slug":"what-sres-should-automate-and-never-automate-with-ai"},"__N_SSG":true},"page":"/[slug]","query":{"slug":"what-sres-should-automate-and-never-automate-with-ai"},"buildId":"E2JcG3BvGILbUHTlEzcxe","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[77618,63213,87127,71206,89752,41116,31486,42348],"gsp":true,"scriptLoader":[]}</script><script>(function(){function c(){var b=a.contentDocument||(a.contentWindow&&a.contentWindow.document);if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a38e710429222ea8',t:'MTc4OTA0MzQwMA=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body></html>