Files
nexus/sreweekly/markdown/106/06-what-makes-a-failure-a-disaster.md
2026-09-12 17:23:01 +08:00

14 KiB
Raw Blame History

What Makes a Failure a Disaster?

简介

This article from Heidi Waterhouse at Launch Darkly starts off with a really interesting take on the Y2K bug and continues on to discuss risk management in operations.

正文

Blog Landing Background Image

LaunchDarkly Blog

Search

Runtime Control

You can't control what you can't see

What LaunchDarkly showed live on the Control Panel: how to see what's happening in production, act on it in real time, and test on data you already trust.

Kellye King

AI Agents

A human look at the AI future

Honest reflections on the uncertainty, excitement, and opportunities of the agentic era.

AI Agents

Introducing the LaunchDarkly AI SDK

The LaunchDarkly AI SDK is available for Python and JavaScript and is the path we recommend for every new AgentControl integration.

Product Updates

See what's launched lately.

Engineering

Stories from the Factory Floor: Building a self-driving ops triage loop

How the Foundation team at LaunchDarkly automated ops triage with three Cursor agents that take an alert all the way to an open PR.

Ari Salem

AI Agents

Introducing the LaunchDarkly AI SDK

The LaunchDarkly AI SDK is available for Python and JavaScript and is the path we recommend for every new AgentControl integration.

Kelvin Yap

AI Agents

A human look at the AI future

Honest reflections on the uncertainty, excitement, and opportunities of the agentic era.

Sarah Day

Sarah Day

Engineering

Stories from the Factory Floor: Running my baseball side project on an AI software factory

I turned my personal side project into a real-world testbed for our internal software factory implementation. Over a few weeks, the factory created and wired 21 flags for me—and changed how I ship.

Seth Payne

Runtime Control

You can't control what you can't see

What LaunchDarkly showed live on the Control Panel: how to see what's happening in production, act on it in real time, and test on data you already trust.

Kellye King

Feature Flags

LaunchDarkly is now native on the Vercel Marketplace

You can now install LaunchDarkly from the Vercel Marketplace and evaluate your first flag in minutes. Start on our free Developer plan and upgrade when you need to, all managed from Vercel.

Bhargav Brahmbhatt

Bhargav

Runtime Control

Best CI/CD Pipelines for Containerized AI Development

Containerized AI applications require sophisticated deployment infrastructure to manage Docker images.

Scarlett Attensil

Scarlett Attensil

Experimentation

ML Experiment Tracking: What to Track Across Models, Data, and Production

The vast majority of teams working on large language models (LLMs) and machine learning (ML) systems diligently track hyperparameters.

Scarlett Attensil

Scarlett Attensil

Experimentation

Best Practices for Experiment Tracking in MLOps

Machine learning experimentation scales quickly.

Scarlett Attensil

Scarlett Attensil

Engineering

Stories from the Factory Floor: Our AI software factory saved me from an incident and I lived to tell the tale

Last summer, I shipped what I thought was a routine cleanup to production. It turned out to be a bug. But before the vast majority of users ever saw it, our AI software factory caught it and rolled back my change automatically.

Alex Engelberg

AI Agents

Podcast recap: Observability won’t save your agents

On a recent episode of the MonkCast, Marek Poliks spoke with James Governor about why governing agents from the outside leaves teams perpetually one step behind.

LaunchDarkly

AI Agents

Agent Optimization: Define what better means, and let AgentControl find it

Agent Optimization, now in beta in AgentControl, automatically searches for a better agent configuration against criteria you define.

Kelvin Yap

Engineering

Stories from the Factory Floor: Building a software factory on our scariest code

We pointed coding agents at our oldest, most business-critical frontend. Here’s what it taught me about what a healthy AI software factory actually looks like.

Alexis Georges

Engineering

Stories from the Factory Floor: Empowering agents with LaunchDarkly MCP tools

A new capability on the LaunchDarkly MCP server offers a practical look at what an automated software factory could look like in practice.

Ramon Niebla

Runtime Control

Why AI deployment breaks standard CI/CD

Learn why AI deployment can break standard CI/CD and how runtime controls, shadow testing, rollouts, and rollback reduce risk.

Scarlett Attensil

Scarlett Attensil

AI Agents

Entering the AI software factory era

What automating the SDLC at LaunchDarkly taught me about speed, control, and the job of an engineer.

Jonathan Nolen

Jonathan Nolen

Runtime Control

Observability is not enough

With runtime control, teams can extend observability by moving beyond reactive monitoring and toward proactive remediation.

Betsy Sallee

Experimentation

Warehouse-native experimentation comes to BigQuery, Databricks, and Redshift

Analyze your experiments on the same trusted data your business already runs on, so results never come with an asterisk.

Lavanya Sureka

Developer productivity

Feature flags were always important. SRE agents make them essential.

AI-powered SRE agents are getting very good at identifying when something is wrong in production. What they haven't solved, however, and what most teams have dramatically underinvested in, is what happens after the agent knows.

Cameron Etezadi

Developer productivity

Why LaunchDarkly is standardizing on New Relic

Today, we are announcing that LaunchDarkly is officially moving its primary observability and telemetry workloads to New Relic.

Cameron Etezadi

Runtime Control

Speed isn't the risk. Lack of control is.

Why controlling code and agents in the AI era matters—and why we built AgentControl.

Kellye King

Experimentation

The Complete AI Experimentation Guide: Test, compare, validate, and ship safely

Artificial intelligence tools aren’t like traditional software.

Scarlett Attensil

Scarlett Attensil

Feature Flags

Release management tools: What they are and how they work

Understanding the control layer between your CI/CD pipeline and your users.

Jesse Sumrak

Jesse Sumrak headshot

Feature Flags

Feature flags vs. feature branching: Why you need both for faster, safer releases

Learn where each fits into your delivery workflow.

Jesse Sumrak

Jesse Sumrak headshot

Resources to get started.

Check out the blog.

Blog Read about LaunchDarkly news, product updates, and more.

Explore our docs.

Docs Learn best practices for getting started with LaunchDarkly.

Watch on demand.

Videos Check out demos and tutorials to see LaunchDarkly in action.

Connect at events.

Events Explore ways to connect with us in person and virtually.