99 lines
156 KiB
HTML
99 lines
156 KiB
HTML
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8" data-next-head=""/><meta name="viewport" content="width=device-width" data-next-head=""/><script async="" src="https://www.googletagmanager.com/gtag/js?id=G-ECJJ2Q2SJQ"></script><title data-next-head=""></title><link rel="preconnect" href="https://bridge.hackernoon.com" data-next-head=""/><link rel="preconnect" href="https://cdn.hackernoon.com" data-next-head=""/><link rel="preconnect" href="https://hackernoon.imgix.net" data-next-head=""/><link rel="dns-prefetch" href="https://cdn.hackernoon.com" data-next-head=""/><meta name="description" content="There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I feel about deploying on Friday, but also based on the science of it, as well as my learned experience." data-next-head=""/><meta property="og:title" content="Deploy on Fridays, or Don't. | HackerNoon" data-next-head=""/><meta property="og:description" content="There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I feel about deploying on Friday, but also based on the science of it, as well as my learned experience." data-next-head=""/><meta name="image" property="og:image" content="https://hackernoon.imgix.net/drafts/9gou3zam.png" data-next-head=""/><meta property="twitter:title" content="Deploy on Fridays, or Don't. | HackerNoon" data-next-head=""/><meta property="twitter:description" content="There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I feel about deploying on Friday, but also based on the science of it, as well as my learned experience." data-next-head=""/><meta property="twitter:image" content="https://hackernoon.imgix.net/drafts/9gou3zam.png" data-next-head=""/><meta name="twitter:card" content="summary_large_image" data-next-head=""/><meta name="twitter:site" content="@hackernoon" data-next-head=""/><link rel="canonical" href="https://hackernoon.com/deploy-on-fridays-or-dont-qg2y32jk" data-next-head=""/><link rel="preload" as="font" href="/fonts/HackerNoonFont/hackernoonv1-regular-webfont.woff2" type="font/woff2" crossorigin="anonymous"/><link rel="preconnect" href="https://fonts.googleapis.com"/><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin="anonymous"/><link data-next-font="" rel="preconnect" href="/" crossorigin="anonymous"/><link rel="preload" href="/_next/static/css/121be0391d0b0ef0.css" as="style"/><link rel="preload" href="/_next/static/css/6d530d6069fd563f.css" as="style"/><link rel="preload" as="image" imageSrcSet="https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=640 640w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=750 750w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=828 828w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1080 1080w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1200 1200w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1920 1920w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=2048 2048w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=3840 3840w" imageSizes="(max-width: 768px) 100vw, 900px" data-next-head=""/><script type="application/ld+json" data-next-head="">{"@context":"http://schema.org","@type":"Article","name":"Deploy on Fridays, or Don't.","headline":"Deploy on Fridays, or Don't.","author":{"@type":"Person","name":"Dave Mangot"},"datePublished":"2019-10-25","image":"https://hackernoon.imgix.net/drafts/9gou3zam.png","articleSection":"devops-principles","articleBody":"There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I about deploying on Friday, but also based on the science of it, as well as my learned experience. feel With a title like “Deploy on Fridays, or Don’t”, I realize the expected continuation of that statement would be “...I don’t care”. However, nothing could be further from the truth. Let me explain. My advice to anyone who will listen is, if you’re cautious about your Friday deploys, don’t feel bad, and don’t let anyone make you feel bad. Shaming It is pretty disconcerting to see a like Kelly’s where the vast majority of the comment thread are attempts to shame anyone for having that opinion. The arguments basically boil down to some variant on tweet Deploying shouldn’t be scary You should be confident in your deploys You lose 20% of your productivity without deploying on Friday You just need more tests Those are all interesting ideas, and they reflect a very interesting type of smug optimism. They are often “backed up” by quoting or the . They eventually arrive at a compromise that you should do the best you can, and keep maturing your deployments until you can deploy anytime you like 365 days with “confidence”. There is also acknowledgement that this can be hard, and having worked with a number of clients and companies over the years, with this I agree. Accelerate State of DevOps report However, here are most of the problems with the logic: Quality Engineering Even Elite performers have change failure % > 0 Mores are not Moratoriums All days are not the same Lastly there is some strange argument that choosing not to deploy on Friday "Shouldn't be a source of glee and pride". That one I haven’t figured out yet, because I have always had a lot of glee and pride in being extremely (overly?) of the engineers who either work for me, or with me. I don’t expect that to change. protective of the work/life balance Quality Engineering While working at Salesforce, I had the opportunity to learn a lot about . This was also the time when I read This was a book that changed my life, and I say that confidently. One of the things I loved about that book was the idea that the more testing that was done, the more confident you could be in the artifact you were deploying. When pitching CI/CD pipeline proposals to executives, they would ask how confident we could be in our artifacts, and I would respond with “How much do you want to spend?”. The more money they were willing to spend, the better testing we could do, and therefore, the more confidence we would have. One of the other things Continuous Delivery taught me was how important it is to have fast feedback. Ultimately, your confidence when deploying to production is going to be some compromise between those two. If you do automated testing for 15 hours, you should obviously have more confidence than if you do so for 30 seconds. quality Continuous Delivery by Jez Humble and Dave Farley. One thing that was present in the book however, was any notion that you could be 100% confident in what you tested. That is, you could not the quality of that artifact tested. Now, Salesforce has a very mature testing pipeline. There were literally hundreds of thousands of tests that were being run more than 5 years ago and yet, they had a quality engineering discipline, but not a quality assurance discipline. Why? not assure Because one cannot assure quality in software. In manufacturing, if I am making shampoo, I can have quality assurance test for quality. QA takes a statistically representative sample of each bottle of shampoo coming down the line, and test to make sure that the chemical composition of what is being produced is within the tolerances as described by the quality specifications. They statistically assure the company that the quality is consistent. In software, you cannot do this. You can not take a random sample of code coming through your continuous delivery pipeline, test those lines of code, and then assure that when that code is deployed to production, it will perform at a level consistent with what has been defined. Therefore it is foolish to lecture people that they should be deploying on Fridays, because they just need to “be confident in their code”, or “write more tests”. How many people dispensing this advice have hundreds of thousands of tests being run on their code? How many people have 100% code coverage in their tests (if this sounds appealing: please don’t do this, the last percentage points will suffer from the laws of diminishing returns)? What I did not take from Jez and Dave’s book, is that you should fool yourself into thinking that you should be 100% confident in everything you push, just because you have tests. Thankfully, Jez continues to talk (along with Nicole, Gene, and others) about tests in the State of DevOps report. DORA Report The DORA Report is often referenced as proof that you should deploy on Fridays, just like any other day. Because it provides data to help classify organizations, including defining how higher vs. lower performing organizations deploy, it’s useful to look at in cases like this. For instance: Elite performers - mean change failure rate 7.5%, recovery < 1hr, deploy multiple times a day Low performers - mean change failure rate 53%, recovery > 1w, deploy > 1 month So, assuming that the advice being dispensed is saying “just become an elite performer”, which is in no way a trivial exercise, they still have a mean change failure rate of 7.5%! Does that sound like they will not have any failures during deploys? That’s one way to make Friday afternoon more exciting! I realize the mean recovery time is < 1 hr, but that is also the . What does the distribution look like? Is there a cluster at 10 hours? Is there a cluster at 2 minutes? I don’t know. But regardless, there is no guarantee of any deploy being failure free, because even the elite performers have failures. mean We also know that change is the leading cause of outages. I’ve seen estimates as high as 75+% of all incidents are at the change boundary. As a friend has said “introducing a change boundary in the 4-6 remaining hours before the whole team is off for 50+ hours seems … like not a high probability play”. But, let’s use the 7.5% change failure rate for elite performers. Do you wear a seatbelt in the car? Yes? Why? What if you had a 7.5% chance of a minor accident? What % chance of a major accident would make you wear a seatbelt? If your argument is that accidents are out of your control, I’d like to introduce you to complex distributed systems… To put it another way, at Google when you violate your SLO, what is slowed down? Releases. Not more tests, not more monitoring, releases. The other problem is that it’s not even for you to cause an outage for your weekend to be interrupted. I learned long ago to be very careful about when I did firewall upgrades. Why? Because every firewall upgrade was generally accompanied by days of spurious correlations about whether something was affected by the upgrade. “Dave, I can’t print, didn’t you recently upgrade the firewall?” “Neither the network traffic for your laptop, nor the printer goes through the firewall.” “But couldn’t…” “No”. necessary Having the capability is necessary Now choosing not to deploy on Fridays is very different than having the capability to deploy on Fridays. You should have the capability to deploy at any time. Things break, code needs to be shipped. You should absolutely be developing this capability if you do not have it. I have worked with elite performers. We still chose to be very careful about our Friday deploys. We also chose to make sure our feature flagging, blue/green, and dark launching capabilities were robust. We had developers deploying their own code whenever they wanted. We deployed multiple services multiple times a day. Every day. But when Friday afternoon came, if someone was going to push a deploy, they would consider if that was necessary, or if it could wait until Monday. After all, there were other things to do. Cultural Norms are not Moratoriums Being choosy about deploying on Friday is not the same as a moratorium. Moratoriums generally require some kind of change advisory board to approve special cases for releasing during the moratorium. Instead, minimizing risk during times where the results of a failure can have outsized impacts is part of the communication and respect we see in DevOps. If the core of DevOps is , then considering the impact our actions can have on others is exactly that - empathetic. Besides, change advisory boards are useless! Moratoriums empathy Netflix I was happy to learn that I’m not the only one who has worked at places that were cautious about their Friday deploys when Aaron Blohowiak tweeted about Netflix: Even the tech giant Netflix has a cultural norm to avoid Friday afternoons. Not a moratorium, but part of the . culture of the company Another great example from Netflix is the Chaos Monkey. The Chaos Monkey runs so that people will be more available to respond should something untoward occur. Is that because Netflix doesn’t do enough testing? Or maybe their monitoring is not good enough to run on the weekend? If every hour of every day is exactly the same as every other day, this would make no sense. Instead, they run the Monkey when people are around to address problems, not when they have other commitments. during business hours Complex Distributed Systems The fact remains that generally we are working on complex distributed systems, and the causes of outages are often elusive. Often when we discover the nature of a problem, it is only obvious in hindsight. A number of years ago, I was rebuilding a SQL proxy tier at a company. We were automating our proxy builds, and deploying new versions of the software. We used these proxies to keep the short lived nature of our PHP application requests from opening thousands of requests per second to the database. The cost of connection initiation and teardown was not free, so we had an intermediate tier that was designed to take this kind of load much more effectively than the database itself. I was building the new tier but was not sending any traffic to it because the weekend was coming. After everything was built, things seemed calm. Until about an hour later, when the database started having problems. Connections were randomly timing out. We looked and saw that the database was often hitting max connections and as a result, many requests were not making it through. Ultimately we determined this was because my new tier had opened connections to the DB, even though they were not being used and that had pushed us over the limit when a certain class of traffic appeared. Seems pretty obvious in hindsight and we ultimately determined what happened through monitoring. But the facts were: This was a new tier that had taken production traffic never It was in a brand new VLAN that had never seen production This was a new version of the software The databases had been running fine the entire time this tier was being created The new tier had all the latest monitoring on it and showed no signs of problems The tier it was replacing also showed no signs of problems And yet the database was dropping connections, and it was an all hands on deck situation on a Friday afternoon when most people were thinking about their weekend. Thankfully, it was relatively easy to resolve. Are these types of things common with releasing new software? No, but they happen. Four Day Work Week So, if we’re not going to deploy on Friday afternoons, what do we do with that time? Do we just give everyone Friday off? Less shipping means no work? One thing we can do is be protective of our employees through work life balance and reduction of stress. I have read with great interest about . Among the validated results of moving to a four day work week were: Four day work week experiments Boosted productivity 24% improvement in work life balance Being protective of employees is something of which I’ve always been very supportive. Whether it’s booting the person off Hangouts who had a blanket over their head and a hot bowl of soup in their hands, or insisting people take the day off when they’ve been up all night troubleshooting an especially difficult issue after a bad deploy. I realize most companies are not going to investigate a four day work week, but Friday afternoons can be used for: Demos and retrospectives Writing documentation Mentoring 20% time or exploration time Working in the staging environment Backlog grooming and prioritization ! Team lunches At one of the companies I worked for, most of the Ops team would go out for lunch every Friday. That meant lots of Friday morning deploys, and then lots of great collaboration in the afternoon. If you’re working with a globally distributed team, do you do a Friday afternoon deploy on the west coast of the United States? That’s almost the next day in most parts of Europe. Most Europeans are not excited to be called back to work late in their evening to help figure out why 8% of traffic is getting 500 errors after a Friday afternoon deploy. Weekends (all days are not the same) One of my favorite things about working at Salesforce was the number of people who chose to wear Hawaiian shirts on Friday. This is something I’d done on and off over the years ever since the release of as a way of recognizing the specialness of Friday. Office Space If the argument is that releasing new software is the same regardless of the day, that ignores what people do on weekends. People make plans to go away, they go camping, they go to the opera, they read books in a hammock by the shore. They go to their kids soccer games, they work in the community garden, etc. Do some of those things happen occasionally on a Tuesday night? Sure they do. But the vast majority of weekend travel happens on the and doing activities that can jeopardize that doesn’t show a lot of respect for your coworkers, or your employees’ work/life balance. They need to take this time to rest and recharge. weekend This is one of the reasons I always liked on-call rotations that rolled over on a Thursday. I always wanted my teams to be able to take a Friday off to get away for a three day weekend. The more downtime in a block, the better. If the argument is that not shipping on Friday afternoon is going to hurt productivity, remember the old Agile adage “you have to go slow to go fast”. Driving a system at 100% capacity is actually a way to reduce your throughput, not maintain it. Does this mean that you should never deploy on Friday? Of course not. That also does not mean you shouldn’t consider what you’re deploying. It may seem to make sense when someone says “We scheduled the move from Oracle to Postgres for 4 hours, so if we start at 1 p.m., we should be done in plenty of time.” My answer to that logic is: NOOOOOOOO. You should be able to, but you don’t have to deploy on Friday afternoon. You should not be shamed for having a culture that respects Friday as being different. It different. is Deploy on Friday’s or Don’t. The choice is up to you."}</script><link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;700&family=IBM+Plex+Sans:wght@400;700&family=Inter:wght@400;600;900&family=Source+Code+Pro:wght@400;500;600;700&display=swap" rel="stylesheet" media="print"/><noscript><link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;700&family=IBM+Plex+Sans:wght@400;700&family=Inter:wght@400;600;900&family=Source+Code+Pro:wght@400;500;600;700&display=swap" rel="stylesheet"/></noscript> <!-- --><script id="ga4-init">
|
||
window.dataLayer = window.dataLayer || [];
|
||
function gtag(){dataLayer.push(arguments);}
|
||
|
||
// Consent Mode: default to denied
|
||
gtag('consent', 'default', {
|
||
'ad_storage': 'denied',
|
||
'analytics_storage': 'denied',
|
||
'ad_user_data': 'denied',
|
||
'ad_personalization': 'denied'
|
||
});
|
||
|
||
gtag('js', new Date());
|
||
gtag('config', 'G-ECJJ2Q2SJQ');
|
||
</script><script id="iubenda-init">
|
||
function initIubenda() {
|
||
(async function () {
|
||
try {
|
||
const res = await fetch("https://geolocation-db.com/json/");
|
||
const data = await res.json();
|
||
const country = data && data.country_code;
|
||
|
||
const GDPR_COUNTRIES = [
|
||
"AT","BE","BG","HR","CY","CZ","DK","EE","FI","FR","DE","GR","HU",
|
||
"IE","IT","LV","LT","LU","MT","NL","PL","PT","RO","SK","SI","ES",
|
||
"SE","IS","LI","NO","UK","GB"
|
||
];
|
||
var isGdpr = GDPR_COUNTRIES.indexOf(country) > -1;
|
||
|
||
window._iub = window._iub || [];
|
||
window._iub.csConfiguration = {
|
||
siteId: 1848357,
|
||
cookiePolicyId: 18778700,
|
||
lang: "en",
|
||
enableTcf: false,
|
||
googleAdditionalConsentMode: true,
|
||
banner: {
|
||
position: "bottom",
|
||
rejectButtonDisplay: true,
|
||
explicitWithdrawal: true,
|
||
customizeButtonDisplay: true,
|
||
acceptButtonDisplay: true,
|
||
showTotalNumberOfProviders: false,
|
||
display: isGdpr
|
||
}
|
||
};
|
||
|
||
var iubScript = document.createElement("script");
|
||
iubScript.src = "https://cdn.iubenda.com/cs/iubenda_cs.js";
|
||
iubScript.async = true;
|
||
document.head.appendChild(iubScript);
|
||
|
||
if (!isGdpr) {
|
||
gtag('consent', 'update', {
|
||
'ad_storage': 'granted',
|
||
'analytics_storage': 'granted',
|
||
'ad_user_data': 'granted',
|
||
'ad_personalization': 'granted'
|
||
});
|
||
}
|
||
} catch (e) {
|
||
console.error("Iubenda geolocation failed", e);
|
||
}
|
||
})();
|
||
}
|
||
|
||
// Defer until browser is idle — never blocks initial render
|
||
if (typeof requestIdleCallback !== 'undefined') {
|
||
requestIdleCallback(initIubenda, { timeout: 3000 });
|
||
} else {
|
||
setTimeout(initIubenda, 1000);
|
||
}
|
||
</script><script id="iubenda-consent-bridge">
|
||
window.addEventListener("iubenda_consent_given", function () {
|
||
gtag('consent', 'update', {
|
||
'ad_storage': 'granted',
|
||
'analytics_storage': 'granted',
|
||
'ad_user_data': 'granted',
|
||
'ad_personalization': 'granted'
|
||
});
|
||
|
||
gtag('event', 'page_view', {
|
||
page_title: document.title,
|
||
page_location: location.href,
|
||
page_path: location.pathname + location.search
|
||
});
|
||
});
|
||
</script><link rel="stylesheet" href="/_next/static/css/121be0391d0b0ef0.css" data-n-g=""/><link rel="stylesheet" href="/_next/static/css/6d530d6069fd563f.css" data-n-p=""/><noscript data-n-css=""></noscript><script defer="" noModule="" src="/_next/static/chunks/polyfills-42372ed130431b0a.js"></script><script src="https://accounts.google.com/gsi/client" defer="" data-nscript="beforeInteractive"></script><script defer="" src="/_next/static/chunks/7618.8cb06698e4978306.js"></script><script defer="" src="/_next/static/chunks/3213.7a85381e883f5859.js"></script><script defer="" src="/_next/static/chunks/7127.9c425c4d3409a6ac.js"></script><script defer="" src="/_next/static/chunks/3304.7c3523eee5ba4042.js"></script><script defer="" src="/_next/static/chunks/1826.1ab3736f712279dc.js"></script><script defer="" src="/_next/static/chunks/b6790ad6-21a72b711b29c9a6.js"></script><script defer="" src="/_next/static/chunks/4829-32ed3fa27f9fba8a.js"></script><script defer="" src="/_next/static/chunks/8145-56bb137bc815feca.js"></script><script defer="" src="/_next/static/chunks/7878-038a9b85114cf28d.js"></script><script defer="" src="/_next/static/chunks/9407-02afcd7299ecf2e9.js"></script><script defer="" src="/_next/static/chunks/1866-36cbb79df614e742.js"></script><script defer="" src="/_next/static/chunks/997.043403d1cfb4d583.js"></script><script defer="" src="/_next/static/chunks/9997.0bcf115a883cf0c1.js"></script><script defer="" src="/_next/static/chunks/9752.5dc7ee3796d8a882.js"></script><script defer="" src="/_next/static/chunks/1486.6875746f7e7b3bd6.js"></script><script defer="" src="/_next/static/chunks/2348.6ff1c8ab2b7f714e.js"></script><script src="/_next/static/chunks/webpack-b0b0e650ef898a91.js" defer=""></script><script src="/_next/static/chunks/framework-594babcea68f40f6.js" defer=""></script><script src="/_next/static/chunks/main-f4c6b80eccf8d9c3.js" defer=""></script><script src="/_next/static/chunks/pages/_app-e9f748c6202c8d20.js" defer=""></script><script src="/_next/static/chunks/4004-46cbf9060446c734.js" defer=""></script><script src="/_next/static/chunks/8230-f743aded49f5ec53.js" defer=""></script><script src="/_next/static/chunks/3363-3997af8403818196.js" defer=""></script><script src="/_next/static/chunks/5857-ec7c040b0c106d7c.js" defer=""></script><script src="/_next/static/chunks/7871-49db796a808d2c70.js" defer=""></script><script src="/_next/static/chunks/3261-28f7d7d5ddf137c8.js" defer=""></script><script src="/_next/static/chunks/1902-922299f711a16f76.js" defer=""></script><script src="/_next/static/chunks/8581-1c63d69fe79360dc.js" defer=""></script><script src="/_next/static/chunks/4581-8f149836b5130c5d.js" defer=""></script><script src="/_next/static/chunks/4680-486b209e44768b17.js" defer=""></script><script src="/_next/static/chunks/2562-5aa3cd1aa6e464a5.js" defer=""></script><script src="/_next/static/chunks/8373-1c61bad6a8e1fdcb.js" defer=""></script><script src="/_next/static/chunks/7225-8cb630c00db3e752.js" defer=""></script><script src="/_next/static/chunks/pages/%5Bslug%5D-e18135e2a995d794.js" defer=""></script><script src="/_next/static/qqTckmfliewRaBLUp_8jP/_buildManifest.js" defer=""></script><script src="/_next/static/qqTckmfliewRaBLUp_8jP/_ssgManifest.js" defer=""></script></head><body><link rel="preload" as="image" imageSrcSet="https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=640 640w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=750 750w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=828 828w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1080 1080w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1200 1200w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1920 1920w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=2048 2048w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=3840 3840w" imageSizes="(max-width: 768px) 100vw, 1200px" fetchPriority="high"/><link rel="preload" as="image" href="https://hackernoon.imgix.net/avatars/robot-b5.png"/><div id="__next"><div class="bg-light text-lightText font-[ibm-plex-mono]"><main><header class="font-[ibm-plex-sans] fixed top-0 left-0 w-full z-50 transition-all duration-500 ease-in-out translate-y-0"><div class="flex items-center justify-between bg-primary lg:navbar h-[50px] sm:min-h-[75px] transition-all duration-100 shadow-md w-full"><div class="hidden lg:flex navbar-start h-full items-center ml-1"><button class="flex items-center hover:scale-[1.01] justify-center rounded-lg text-base px-4 font-bold py-2 border-none bg-primary-content text-primaryContentText">Discover Anything<i class="hn hn-search text-lg ml-4 text-primaryContentText "></i></button></div><div class="nav-start lg:navbar-center ml-2 lg:ml-0 min-w-0 flex-shrink"><a class="relative z-10 flex items-center space-x-2 hover:scale-[1.02]" aria-label="HackerNoon Homepage" href="/"><svg class="w-[180px] xs:w-[200px] sm:w-[240px] lg:w-[260px] h-auto" viewBox="0 0 2150 260" fill="none" xmlns="http://www.w3.org/2000/svg" preserveAspectRatio="xMidYMid meet" style="transition:fill 150ms ease"><g style="transition:fill 150ms ease"><path d="M269.997 20.0005V0H130V20.0005V40.0011V60.0016H150H169.995V40.0011H189.995H229.996V60.0016H249.997H269.997V40.0011V20.0005Z" fill="transparent"></path><path d="M130.006 80.003V60.0024H110.006V80.003V100.004H130.006V80.003Z" fill="transparent"></path><path d="M110 119.998V100.003H90V119.998V139.998V159.999H110V139.998V119.998Z" fill="transparent"></path><path d="M270 100.004H290V80.003V60.0024H270V80.003V100.004Z" fill="transparent"></path><path d="M310 119.997V100.002H290V119.997V139.998V159.998H310V139.998H330.001V119.997H310Z" fill="transparent"></path><path d="M130 159.998H110V179.998V199.999H130V179.998V159.998Z" fill="transparent"></path><path d="M270 179.998V199.999H290V179.998V159.998H270V179.998Z" fill="transparent"></path><path d="M130 260V240V219.999V199.999H150H169.995V219.999H189.995H209.996H229.996V199.999H249.997H269.997V219.999V240V260H130Z" fill="transparent"></path><path d="M210.415 39.74V59.7405V79.7411V99.7416V119.736V139.737H190.415V119.736V99.7416V79.7411V59.7405V39.74H210.415Z" fill="transparent"></path><path d="M390 200V60H417.801V116.676H501.206V60H530V200H501.206V144.517H417.801V200H390Z" fill="transparent"></path><path d="M672.199 116.676V88.8352H588.794V116.676H672.199ZM560 200V60H700V200H672.199V144.517H588.794V200H560Z" fill="transparent"></path><path d="M730 200V60H870V88.8352H758.794V172.159H870V200H730Z" fill="transparent"></path><path d="M900 200V60H928.794V116.276H984.397V143.724H928.794V200H900ZM1012.2 171.368H984.397V143.724H1012.2V171.368ZM1012.2 171.368H1040V199.013H1012.2V171.368ZM1012.2 88.6319V116.276H984.397V88.6319H1012.2ZM1012.2 88.6319V60H1040V88.6319H1012.2Z" fill="transparent"></path><path d="M1070 200V60H1210V88.8352H1098.79V116.676H1154.4V144.517H1098.79V172.159H1210V200H1070Z" fill="transparent"></path><path d="M1351.24 116.519V88.7589H1267.76V116.519H1351.24ZM1240 200V60H1380V144.479H1351.24V172.24H1380V200H1351.24V172.24H1323.48V144.479H1267.76V200H1240Z" fill="transparent"></path><path d="M1410 200V60H1550V200H1522.24V88.7589H1438.76V200H1410Z" fill="transparent"></path><path d="M1692.24 172.24V88.7589H1608.76V172.24H1692.24ZM1580 200V60H1720V200H1580Z" fill="transparent"></path><path d="M1862.04 172.24V88.7589H1778.97V172.24H1862.04ZM1750 200V60H1890V200H1750Z" fill="transparent"></path><path d="M1920 200V60H2060V200H2032.24V88.7589H1948.76V200H1920Z" fill="transparent"></path></g></svg></a></div><div class="navbar-end h-full flex items-center min-w-[100px] lg:min-w-[200px] space-x-4 mr-2"><div class=" h-[40px] flex items-center justify-center"></div><div class="hidden sm:flex space-x-4 "><button class="px-4 font-bold text-base py-1 sm:py-2 bg-primary-content text-primaryContentText rounded-md transition-all duration-300">Signup</button><a class="px-4 hover:scale-105 font-bold text-base py-2 bg-primary-content text-primaryContentText rounded-md " href="/new">Write</a></div><button class="btn border-none p-0 m-0 lg:hidden bg-transparent hover:bg-transparent text-primary-content hover:scale-110"><i class="hn hn-search text-xl mr-2 "></i></button><button class="relative lg:flex hidden items-center hover:scale-110 text-primary-content" aria-label="Notifications"><i width="20" class="hn hn-bell text-2xl w-4 h-4 sm:w-6 sm:h-6"></i></button><button class="flex items-center hover:scale-110 text-primary-content" aria-label="Menu"><i class="hn hn-bars text-2xl text-primary-content"></i></button></div></div><div class="z-20 hidden lg:block h-[52px] transition-all duration-500 ease-in-out"><nav class="h-[52px] bg-secondary animate-pulse"></nav></div><div class="flex items-center "><div class="bg-accent text-accent-content flex items-center h-[62px] sm:min-h-[80px] font-[ibm-plex-sans] w-full relative z-10 opacity-100"><div class="h-[62px] sm:h-[80px] bg-[transparent] animate-pulse"></div></div></div></header><div class="transition-all duration-200 pt-[112px] sm:pt-[155px] lg:pt-[207px]"><div data-rht-toaster="" style="position:fixed;z-index:9999;top:16px;left:16px;right:16px;bottom:16px;pointer-events:none"></div><div class=""><div class="bg-light text-lightText h-auto xl:mx-2 "><div class="col-span-12"><div class="max-w-[1200px] mx-auto px-2 xs:px-4 xl: xl:px-0 mt-10"><div class=""><div><div class="text-xs"><div class="mb-4 flex gap-2"><span class="bg-lightAlt p-2 rounded-lg inline-flex gap-2 items-center justify-start"><i class="hn hn-star-solid"></i> <!-- -->13,920<!-- --> <!-- -->reads</span></div><h1 class="font-bold line-clamp-4 leading-snug text-lightTextStrong
|
||
text-xl sm:text-2xl xl:text-3xl 3xl:text-4xl tracking-wide
|
||
|
||
">Deploy on Fridays, or Don't.</h1><div class="flex flex-wrap border-y border-lightBorder my-4 sm:my-2 sm:border-t-0 py-2 items-center justify-between text-lightTextLight text-sm sm:text-base xl:text-xl "><div class="flex flex-wrap justify-between w-full xs:w-auto items-center gap-2"><span class="flex items-center flex-wrap gap-2 mr-10 sm:mr-0 ">by<div class="dropdown dropdown-hover "><label tabindex="0"><a aria-label="View profile of Dave Mangot" href="/u/dmangot"><strong class="...">Dave Mangot</strong></a></label><div class="dropdown-content z-[1] pt-2 sm:pt-1 left-[-40px] w-[280px] xs:w-[320px] sm:w-[400px] bg-transparent menu rounded "><div class="w-full "><div class=" p-4 border border-lightBorder bg-light rounded-lg"><a href="/u/dmangot" target="_blank" rel="noopener noreferrer" class="flex items-start text-sm rounded-lg group gap-2"><div class=""><img alt="Dave Mangot" loading="lazy" width="40" height="40" decoding="async" data-nimg="1" class="w-10 h-9 border-solid border border-lightBorder rounded-full object-contain" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=48 1x, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96 2x" src="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96"/></div><span class="flex flex-col min-w-0 w-full justify-center"><span class="flex items-center gap-1 text-ellipsis overflow-hidden whitespace-nowrap"><span class="font-bold group-hover:underline text-sm truncate"><span class="text-xs font-light mr-1">by</span>Dave Mangot</span><span class="text-xs font-light opacity-50">|</span><span class="text-sm false text-ellipsis overflow-hidden whitespace-nowrap" title="@dmangot">@<!-- -->dmangot</span></span></span></a><p class="text-sm overflow-x-auto mt-2 text-bodyTxtLight">Dave Mangot is the principal at Mangoteque, & has successfully led DevOps & Ops transformations.</p><div class="mt-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
|
||
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
|
||
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div></div></div></div></div></span><div class="flex gap-2 items-center cursor-pointer"><span class="hidden sm:block w-1 h-1 bg-lightTextLight mx-4 rounded-full"></span><a href="/archives/2019/10/25"><span class="text-xs xs:text-sm lg:text-base">October 25th, 2019</span></a></div></div><div class="hidden xl:block"><div class=" flex items-center gap-4 "><span class="tooltip tooltip-left tooltip-left w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Terminal Reader"><img alt="Read on Terminal Reader" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&w=32 1x, https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&w=48 2x" src="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&w=48"/></span><span class="tooltip tooltip-left tooltip-left w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Print this story"><img alt="Print this story" data-tip="true" data-for="print-page" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&w=32 1x, https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&w=48 2x" src="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&w=48"/></span><span class="tooltip tooltip-left tooltip-left w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Read this story w/o Javascript"><img alt="Read this story w/o Javascript" data-tip="true" data-for="arweave-backup" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&w=32 1x, https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&w=48 2x" src="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&w=48"/></span></div></div></div></div><div class="mb-2 "><div class="flex justify-between "><button class="flex m-1 px-2 xl:px-4 h-[40px] items-center font-[hackernoon2] bg-dark text-darkText text-xs sm:text-sm rounded-lg border border-lightBorder">TLDR <i class="hn hn-angle-right text-base ml-1 "></i></button><div class="flex items-center gap-2"><div class="hidden sm:block xl:hidden"><div class=" flex items-center gap-4 "><span class="tooltip tooltip-left undefined w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Terminal Reader"><img alt="Read on Terminal Reader" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&w=32 1x, https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&w=48 2x" src="https://hackernoon.imgix.net/computer.png?auto=format%2Ccompress&w=48"/></span><span class="tooltip tooltip-left undefined w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Print this story"><img alt="Print this story" data-tip="true" data-for="print-page" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&w=32 1x, https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&w=48 2x" src="https://hackernoon.imgix.net/images/Print%20Icon%20%4025px.png?auto=format%2Ccompress&w=48"/></span><span class="tooltip tooltip-left undefined w-7 h-7 flex items-center justify-center cursor-pointer" data-tip="Read this story w/o Javascript"><img alt="Read this story w/o Javascript" data-tip="true" data-for="arweave-backup" loading="lazy" width="20" height="20" decoding="async" data-nimg="1" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&w=32 1x, https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&w=48 2x" src="https://hackernoon.imgix.net/images/Lite%20Icon%20%4025px.png?auto=format%2Ccompress&w=48"/></span></div></div><div class="xl:hidden"></div></div><div class="hidden xl:flex items-center flex-wrap gap-2"></div></div></div></div></div></div><div class="max-w-[1200px] mx-auto"><div class="flex items-center justify-center w-full h-full"><div class="relative group cursor-zoom-in transition-transform hover:scale-[1.01] max-w-full mx-auto px-[14px] md:px-0 w-full"><button class="absolute top-2 right-5 z-10 w-6 h-6 rounded flex items-center justify-center opacity-0 group-hover:opacity-100 transition-opacity"><i class="hn hn-download text-darkText bg-dark p-2 rounded-xl text-base"></i></button><img alt="featured image - Deploy on Fridays, or Don't." fetchPriority="high" loading="eager" width="2552" height="1697" decoding="async" data-nimg="1" class="w-full h-auto object-contain rounded-lg shadow-lg my-0" style="color:transparent;background-size:cover;background-position:50% 50%;background-repeat:no-repeat;background-image:url("data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 2552 1697'%3E%3Cfilter id='b' color-interpolation-filters='sRGB'%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3CfeColorMatrix values='1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 100 -1' result='s'/%3E%3CfeFlood x='0' y='0' width='100%25' height='100%25'/%3E%3CfeComposite operator='out' in='s'/%3E%3CfeComposite in2='SourceGraphic'/%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3C/filter%3E%3Cimage width='100%25' height='100%25' x='0' y='0' preserveAspectRatio='none' style='filter: url(%23b);' href='data:image/svg+xml;base64,CiAgICA8c3ZnIHdpZHRoPSIyNTUyIiBoZWlnaHQ9IjE2OTciIHZlcnNpb249IjEuMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayI+CiAgICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZyI+CiAgICAgICAgICA8c3RvcCBzdG9wLWNvbG9yPSIjMjIyIiBvZmZzZXQ9IjIwJSIgLz4KICAgICAgICAgIDxzdG9wIHN0b3AtY29sb3I9IiMwZjAiIG9mZnNldD0iNTAlIiAvPgogICAgICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzIyMiIgb2Zmc2V0PSI4MCUiIC8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgPC9kZWZzPgogICAgICA8cmVjdCB3aWR0aD0iMjU1MiIgaGVpZ2h0PSIxNjk3IiBmaWxsPSIjMjIyIiAvPgogICAgICA8cmVjdCBpZD0iciIgd2lkdGg9IjI1NTIiIGhlaWdodD0iMTY5NyIgZmlsbD0idXJsKCNnKSIgLz4KICAgICAgPGFuaW1hdGUgeGxpbms6aHJlZj0iI3IiIGF0dHJpYnV0ZU5hbWU9IngiIGZyb209Ii0yNTUyIiB0bz0iMjU1MiIgZHVyPSIxcyIgcmVwZWF0Q291bnQ9ImluZGVmaW5pdGUiICAvPgogICAgPC9zdmc+'/%3E%3C/svg%3E")" sizes="(max-width: 768px) 100vw, 1200px" srcSet="https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=640 640w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=750 750w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=828 828w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1080 1080w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1200 1200w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=1920 1920w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=2048 2048w, https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=3840 3840w" src="https://hackernoon.imgix.net/drafts/9gou3zam.png?auto=format%2Ccompress&w=3840"/></div></div></div><div class="px-2 xs:px-4 3xl:px-0 my-4 sm:mt-4 sm:mb-6 max-w-[1200px] 6xl:max-w-[1200px] mx-auto "><div class="w-full flex flex-col justify-center rounded-lg"><audio src="https://storage.googleapis.com/hackernoon/audios/TjnmEeP0JCGMYYQHtk4D-en-US-Wavenet-I-MALE--c8a1b4cada081.mp3" preload="metadata">Your browser does not support the <code>audio</code> element.</audio><div class="hidden sm:flex justify-between items-center "><span></span></div><div class="flex gap-2 items-center"><div class="flex items-center justify-center mx-auto gap-2 xs:gap-4 lg:gap-6 flex-1"><button aria-label="play/pause" class="text-darkAccent max-w-[40px] max-h-[40px] sm:min-w-[48px] sm:min-h-[48px] border order-1 border-darkBorder bg-dark p-2 rounded-full flex items-center justify-center" title="Play/Pause"><i class="hn hn-play-solid text-base xs:text-lg sm:text-2xl "></i></button><div class="dropdown order-2 dropdown-hover"><label tabindex="0" class="flex items-center hn hn-playlist-solid text-base xs:text-lg sm:text-xl lg:text-2xl rounded-lg " title="Speed & Voice"></label><ul tabindex="0" class="dropdown-content border z-40 menu p-4 shadow bg-light rounded-box w-60"><div class="text-lightText flex bg-light p-2 rounded w-full mb-2 items-center justify-between"><span class="text-xs font-bold">Speed</span><button class="bg-lightAlt ml-2 px-4 py-2 rounded-full text-sm font-bold min-w-[100px]">1x</button></div><div class="text-lightText flex flex-col bg-light p-2 rounded w-full"><span class="text-xs font-bold mb-2">Voice</span><div class="flex flex-col gap-2 max-h-60 overflow-auto pr-1"><button class="bg-lightAlt px-3 py-2 rounded-lg text-sm font-bold text-left flex items-center justify-between ring-2 ring-green-600"><span class="truncate mr-2">Dr. One </span><img src="https://hackernoon.imgix.net/avatars/robot-b5.png" alt="Dr. One (en-US)" class="w-6 h-6 rounded-full"/></button></div></div></ul></div><div class="flex gap-2 order-3 sm:items-center sm:flex-row w-full"><div class="rounded-lg flex-1 bg-lightAccentTextAlt relative"><div class="hidden lg:block"><div class="relative max-w-[1000px] h-full flex items-center cursor-pointer rounded-lg "><canvas class="bg-transparent absolute top-0 left-0 w-full h-full rounded-lg "></canvas><div></div><div class=" top-0 left-0 h-full overflow-hidden bg-lightAccentAlt border rounded-l-lg" style="width:0px"><canvas class="bg-transparent w-full h-full text-green-500"></canvas></div></div></div><div class="hidden sm:block lg:hidden"><div class="relative max-w-[1000px] h-full flex items-center cursor-pointer rounded-lg "><canvas class="bg-transparent absolute top-0 left-0 w-full h-full rounded-lg "></canvas><div></div><div class=" top-0 left-0 h-full overflow-hidden bg-lightAccentAlt border rounded-l-lg" style="width:0px"><canvas class="bg-transparent w-full h-full text-green-500"></canvas></div></div></div><div class="w-full sm:hidden"><div class="relative max-w-[1000px] h-full flex items-center cursor-pointer rounded-lg "><canvas class="bg-transparent absolute top-0 left-0 w-full h-full rounded-lg "></canvas><div></div><div class=" top-0 left-0 h-full overflow-hidden bg-lightAccentAlt border rounded-l-lg" style="width:0px"><canvas class="bg-transparent w-full h-full text-green-500"></canvas></div></div></div></div><div class="hidden lg:ml-2 sm:block"></div><div class=" flex items-center sm:hidden"></div></div></div></div></div></div><div class=" flex xl:hidden bg-light z-10 mx-auto gap-4 items-center border-y sticky top-[62px] sm:top-[80px] py-2 sm:py-0 "><div class="w-full max-w-[1200px] mx-auto px-4 flex justify-between items-center"><div class="6xl:hidden dropdown dropdown-bottom dropdown-hover"><div tabindex="0" role="button" class="flex text-sm rounded-lg py-2"><div class="mr-2 flex -space-x-2 items-center "><div class=""><img alt="Dave Mangot" loading="lazy" width="48" height="48" decoding="async" data-nimg="1" class="w-10 h-10 sm:w-12 sm:h-12 bg-light relative border border-lightBorder rounded-full object-contain" style="color:transparent;z-index:1" srcSet="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=48 1x, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96 2x" src="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96"/></div></div><div class="flex-col hidden sm:flex flex-wrap"><span class="font-bold mx-2 text-sm"><span class="text-xs font-light mr-1">by</span>Dave Mangot</span><span class="text-sm ml-2">@<!-- -->dmangot</span></div></div><ul tabindex="0" class="dropdown-content menu w-[300px] py-1 left-[-20px] bg-light px-1 ml-4 rounded-b-lg 3xl:border-none rounded-boxabsolute z-50"><div class="flex w-full flex-col gap-4"><div><div class="w-full "><div class=" p-4 border border-lightBorder bg-light rounded-lg"><a href="/u/dmangot" target="_blank" rel="noopener noreferrer" class="flex items-start text-sm rounded-lg group gap-2"><div class=""><img alt="Dave Mangot" loading="lazy" width="40" height="40" decoding="async" data-nimg="1" class="w-10 h-9 border-solid border border-lightBorder rounded-full object-contain" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=48 1x, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96 2x" src="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96"/></div><span class="flex flex-col min-w-0 w-full justify-center"><span class="flex items-center gap-1 text-ellipsis overflow-hidden whitespace-nowrap"><span class="font-bold group-hover:underline text-sm truncate"><span class="text-xs font-light mr-1">by</span>Dave Mangot</span><span class="text-xs font-light opacity-50">|</span><span class="text-sm false text-ellipsis overflow-hidden whitespace-nowrap" title="@dmangot">@<!-- -->dmangot</span></span></span></a><p class="text-sm overflow-x-auto mt-2 text-bodyTxtLight">Dave Mangot is the principal at Mangoteque, & has successfully led DevOps & Ops transformations.</p><div class="mt-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
|
||
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
|
||
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div></div></div></div></div></ul></div><div class="w-[200px] 6xl:hidden"><div class=" flex flex-row flex-row-reverse items-start gap-4 "><span class="tooltip tooltip-left cursor-pointer" data-tip="Bookmark"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-bookmark text-lightText text-2xl"></i></button></span><span class="tooltip tooltip-left cursor-pointer" data-tip="Comment"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-comment text-lightText text-2xl"></i></button></span><div class="dropdown dropdown-bottom dropdown-hover group "><label tabindex="0" class="flex items-center cursor-pointer justify-center border border-lightBorder 3xl:group-hover:bg-lightAlt group-hover:bg-light h-[40px] w-[40px] p-2 rounded "><i class="hn hn-share text-2xl"></i></label><ul tabindex="0" class="dropdown-content bg-light z-[1] py-4 px-4 3xl:px-0 3xl:py-2 border 3xl:border-none flex flex-col items-center justify-center gap-2 "><button class="border p-2 rounded hover:bg-lightAlt"><i class=" hn hn-copy text-lightText text-2xl "></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-facebook-round text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-x text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-linkedin text-lightText text-2xl"></i></button><a href="mailto:?subject=I'd like to share a link with you &body=" class="border p-2 rounded inline-block hover:bg-lightAlt"><i class="hn hn-envelope text-lightText text-2xl"></i></a></ul></div></div></div></div></div><div class="flex w-full min-w-0 max-w-[1100px] 2xl:max-w-[1200px] mx-auto justify-center flex-row gap-4 items-start mt-10 px-4 xl:px-0"><div class="hidden xl:flex xl:flex-col self-stretch"><div class="sticky top-[99px] z-40"><div class="dropdown z-25 dropdown-right dropdown-hover"><div class="mr-2 flex gap-2 flex-col justify-center flex-wrap items-center "><div class="relative h-12 w-12 bg-black rounded-full overflow-hidden flex-shrink-0"><img alt="Dave Mangot" loading="lazy" decoding="async" data-nimg="fill" class="rounded-full object-contain " style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;color:transparent;z-index:1" sizes="100vw" srcSet="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=640 640w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=750 750w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=828 828w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=1080 1080w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=1200 1200w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=1920 1920w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=2048 2048w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=3840 3840w" src="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=3840"/></div></div><ul tabindex="0" class="dropdown-content w-[280px] xs:w-[320px] sm:w-[400px] rounded-lg z-[100] bg-light flex flex-col items-center justify-center gap-2"><div class="flex w-full flex-col gap-4"><div><div class="w-full "><div class=" p-4 border border-lightBorder bg-light rounded-lg"><a href="/u/dmangot" target="_blank" rel="noopener noreferrer" class="flex items-start text-sm rounded-lg group gap-2"><div class=""><img alt="Dave Mangot" loading="lazy" width="40" height="40" decoding="async" data-nimg="1" class="w-10 h-9 border-solid border border-lightBorder rounded-full object-contain" style="color:transparent" srcSet="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=48 1x, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96 2x" src="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=96"/></div><span class="flex flex-col min-w-0 w-full justify-center"><span class="flex items-center gap-1 text-ellipsis overflow-hidden whitespace-nowrap"><span class="font-bold group-hover:underline text-sm truncate"><span class="text-xs font-light mr-1">by</span>Dave Mangot</span><span class="text-xs font-light opacity-50">|</span><span class="text-sm false text-ellipsis overflow-hidden whitespace-nowrap" title="@dmangot">@<!-- -->dmangot</span></span></span></a><p class="text-sm overflow-x-auto mt-2 text-bodyTxtLight">Dave Mangot is the principal at Mangoteque, & has successfully led DevOps & Ops transformations.</p><div class="mt-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
|
||
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
|
||
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div></div></div></div></div></ul></div></div></div><div class="flex-1 flex flex-col justify-center min-w-0 w-full"><div class=""><div class="story-body font-sans w-full min-w-0"><div class="prose max-w-[1020px] w-full min-w-0 xs:p-0 lg:px-0 prose-a:break-words prose-table:table prose-div:bg-transparent prose-table:!table prose-table:max-w-full prose-table:w-full prose-td_a:whitespace-nowrap prose-td_a:break-keep prose-td_a:overflow-wrap-normal prose-td_a:word-break-normal [&_th]:!hyphens-none [&_td]:!hyphens-none [&_th]:!break-normal [&_td]:!break-normal [&_th]:![overflow-wrap:normal] [&_td]:![overflow-wrap:break-word] [&_th]:whitespace-nowrap xs:prose-table:mx-auto prose-table:overflow-x-auto leading-relaxed prose-p:text-lightTextLight prose-strong:text-lightTextStrong prose-strong:font-bold prose-small:text-lightTextLight prose-small:font-light prose-a:text-lightTextLight prose-p:mx-0 prose-p:my-2 prose [&_.line-space]:my-0 prose-p:my-2 prose-p:text-base sm:prose-p:text-lg prose-blockquote:my-0 prose-blockquote:border-l-[5px] prose-blockquote:border-lightTextAccent prose-blockquote:pl-4 prose-blockquote:leading-relaxed prose-blockquote:text-lightText prose-h2:text-lightTextStrong prose-li:marker:text-lightText prose-h2:text-xl sm:prose-h2:text-3xl prose-h2:font-bold prose-h2:my-6 prose-h3:text-lightTextStrong prose-hr:m-2 prose-h3:text-xl sm:prose-h3:text-2xl prose-h3:font-bold prose-h3:my-5 prose-h4:text-xl prose-h4:font-bold prose-h4:my-4 prose-td:text-lightTextLight prose-td:border prose-td:border-lightBorder prose-td:px-2 prose-td:[&_p]:my-0 prose-th:[&_p]:my-0 prose-th:text-lightTextLight prose-th:border prose-th:border-lightBorder prose-th:px-2 prose-li:text-lg prose-li:text-lightTextLight prose-li:px-0 prose-li:mb-3 prose-li:ml-3 prose-li:leading-relaxed prose-ul:pl-2 prose-ul:sm:pl-8 prose-ol:pl-2 prose-ol:sm:pl-8 hover:prose-a:text-lightTextAccent prose-a:rounded prose-code:text-lightTextLight prose-code:break-all prose-pre:rounded-lg prose-pre:text-sm prose-pre:my-4 prose-pre:p-3 prose-pre:overflow-x-scroll prose-pre:whitespace-pre-wrap prose-pre:break-words "><div class="w-full flex items-center justify-center "><p class="paragraph">There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I <strong>feel</strong> about deploying on Friday, but also based on the science of it, as well as my learned experience.</p><p class="paragraph">With a title like “Deploy on Fridays, or Don’t”, I realize the expected continuation of that statement would be “...I don’t care”. However, nothing could be further from the truth. Let me explain.</p><p class="paragraph">My advice to anyone who will listen is, if you’re cautious about your Friday deploys, don’t feel bad, and don’t let anyone make you feel bad.</p><div class="image-container"><div class="w-full my-4"><div class="flex items-center justify-center w-full h-full"><div class="relative group cursor-zoom-in transition-transform hover:scale-[1.01] max-w-full mx-auto flex flex-col items-center"><button class="absolute top-2 right-5 z-10 w-6 h-6 rounded flex items-center justify-center opacity-0 group-hover:opacity-100 transition-opacity"><i class="hn hn-download text-darkText bg-dark p-2 rounded-xl text-base"></i></button><img alt="Dave Mangot's image-870838" fetchPriority="low" loading="lazy" width="930" height="638" decoding="async" data-nimg="1" class="w-full h-auto object-contain rounded-lg shadow-lg my-0" style="color:transparent;background-size:cover;background-position:50% 50%;background-repeat:no-repeat;background-image:url("data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 930 638'%3E%3Cfilter id='b' color-interpolation-filters='sRGB'%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3CfeColorMatrix values='1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 100 -1' result='s'/%3E%3CfeFlood x='0' y='0' width='100%25' height='100%25'/%3E%3CfeComposite operator='out' in='s'/%3E%3CfeComposite in2='SourceGraphic'/%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3C/filter%3E%3Cimage width='100%25' height='100%25' x='0' y='0' preserveAspectRatio='none' style='filter: url(%23b);' href='data:image/svg+xml;base64,CiAgICA8c3ZnIHdpZHRoPSI5MzAiIGhlaWdodD0iNjM4IiB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiPgogICAgICA8ZGVmcz4KICAgICAgICA8bGluZWFyR3JhZGllbnQgaWQ9ImciPgogICAgICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzIyMiIgb2Zmc2V0PSIyMCUiIC8+CiAgICAgICAgICA8c3RvcCBzdG9wLWNvbG9yPSIjMGYwIiBvZmZzZXQ9IjUwJSIgLz4KICAgICAgICAgIDxzdG9wIHN0b3AtY29sb3I9IiMyMjIiIG9mZnNldD0iODAlIiAvPgogICAgICAgIDwvbGluZWFyR3JhZGllbnQ+CiAgICAgIDwvZGVmcz4KICAgICAgPHJlY3Qgd2lkdGg9IjkzMCIgaGVpZ2h0PSI2MzgiIGZpbGw9IiMyMjIiIC8+CiAgICAgIDxyZWN0IGlkPSJyIiB3aWR0aD0iOTMwIiBoZWlnaHQ9IjYzOCIgZmlsbD0idXJsKCNnKSIgLz4KICAgICAgPGFuaW1hdGUgeGxpbms6aHJlZj0iI3IiIGF0dHJpYnV0ZU5hbWU9IngiIGZyb209Ii05MzAiIHRvPSI5MzAiIGR1cj0iMXMiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiAgLz4KICAgIDwvc3ZnPg=='/%3E%3C/svg%3E")" srcSet="https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-qn163zae.jpg?auto=format%2Ccompress&w=1080 1x, https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-qn163zae.jpg?auto=format%2Ccompress&w=1920 2x" src="https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-qn163zae.jpg?auto=format%2Ccompress&w=1920"/></div></div></div></div><h2><strong>Shaming</strong></h2><p class="paragraph">It is pretty disconcerting to see a <a href="https://twitter.com/kvlly/status/1180075727267467264?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">tweet</a> like Kelly’s where the vast majority of the comment thread are attempts to shame anyone for having that opinion. The arguments basically boil down to some variant on </p><ul><li>Deploying shouldn’t be scary</li><li>You should be confident in your deploys</li><li>You lose 20% of your productivity without deploying on Friday</li><li>You just need more tests</li></ul><p class="paragraph">Those are all interesting ideas, and they reflect a very interesting type of smug optimism. They are often “backed up” by quoting <a href="https://itrevolution.com/book/accelerate/?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">Accelerate</a> or the <a href="https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">State of DevOps report</a>. They eventually arrive at a compromise that you should do the best you can, and keep maturing your deployments until you can deploy anytime you like 365 days with “confidence”. There is also acknowledgement that this can be hard, and having worked with a number of clients and companies over the years, with this I agree.</p><p class="paragraph">However, here are most of the problems with the logic:</p><ul><li>Quality Engineering</li><li>Even Elite performers have change failure % > 0</li><li>Mores are not Moratoriums</li><li>All days are not the same</li></ul><p class="paragraph">Lastly there is some strange argument that choosing not to deploy on Friday "Shouldn't be a source of glee and pride". That one I haven’t figured out yet, because I have always had a lot of glee and pride in being extremely (overly?) <a href="https://www.youtube.com/watch?v=lzl4nu0ZHQo&ref=hackernoon.com#action=share" target="_blank" rel="noopener noreferrer ugc">protective of the work/life balance</a> of the engineers who either work for me, or with me. I don’t expect that to change.</p><h2>Quality Engineering</h2><p class="paragraph">While working at Salesforce, I had the opportunity to learn a lot about <a href="https://www.youtube.com/watch?v=H4ZwBO0P7-E&ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">quality</a>. This was also the time when I read <a href="https://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley-dp-0321601912/dp/0321601912/ref=mt_hardcover?_encoding=UTF8&me=&qid=&ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">Continuous Delivery by Jez Humble and Dave Farley.</a> This was a book that changed my life, and I say that confidently. One of the things I loved about that book was the idea that the more testing that was done, the more confident you could be in the artifact you were deploying. When pitching CI/CD pipeline proposals to executives, they would ask how confident we could be in our artifacts, and I would respond with “How much do you want to spend?”. The more money they were willing to spend, the better testing we could do, and therefore, the more confidence we would have. One of the other things Continuous Delivery taught me was how important it is to have fast feedback. Ultimately, your confidence when deploying to production is going to be some compromise between those two. If you do automated testing for 15 hours, you should obviously have more confidence than if you do so for 30 seconds.</p><p class="paragraph">One thing that was <strong>not</strong> present in the book however, was any notion that you could be 100% confident in what you tested. That is, you could not <strong>assure</strong> the quality of that artifact tested. Now, Salesforce has a very mature testing pipeline. There were literally hundreds of thousands of tests that were being run more than 5 years ago and yet, they had a quality engineering discipline, but not a quality assurance discipline. Why?</p><p class="paragraph">Because one cannot assure quality in software. In manufacturing, if I am making shampoo, I can have quality assurance test for quality. QA takes a statistically representative sample of each bottle of shampoo coming down the line, and test to make sure that the chemical composition of what is being produced is within the tolerances as described by the quality specifications. They statistically assure the company that the quality is consistent.</p><p class="paragraph">In software, you cannot do this. You can not take a random sample of code coming through your continuous delivery pipeline, test those lines of code, and then assure that when that code is deployed to production, it will perform at a level consistent with what has been defined.</p><p class="paragraph">Therefore it is foolish to lecture people that they should be deploying on Fridays, because they just need to “be confident in their code”, or “write more tests”. How many people dispensing this advice have hundreds of thousands of tests being run on their code? How many people have 100% code coverage in their tests (if this sounds appealing: please don’t do this, the last percentage points will suffer from the laws of diminishing returns)?</p><p class="paragraph">What I did not take from Jez and Dave’s book, is that you should fool yourself into thinking that you should be 100% confident in everything you push, just because you have tests. Thankfully, Jez continues to talk (along with Nicole, Gene, and others) about tests in the State of DevOps report.</p><h2>DORA Report</h2><p class="paragraph">The DORA Report is often referenced as proof that you should deploy on Fridays, just like any other day. Because it provides data to help classify organizations, including defining how higher vs. lower performing organizations deploy, it’s useful to look at in cases like this. For instance: </p><ul><li>Elite performers - mean change failure rate 7.5%, recovery < 1hr, deploy multiple times a day </li><li>Low performers - mean change failure rate 53%, recovery > 1w, deploy > 1 month</li></ul><p class="paragraph">So, assuming that the advice being dispensed is saying “just become an elite performer”, which is in no way a trivial exercise, they still have a mean change failure rate of 7.5%! Does that sound like they will not have any failures during deploys? That’s one way to make Friday afternoon more exciting! I realize the mean recovery time is < 1 hr, but that is also the <strong>mean</strong>. What does the distribution look like? Is there a cluster at 10 hours? Is there a cluster at 2 minutes? I don’t know. But regardless, there is no guarantee of any deploy being failure free, because even the elite performers have failures.</p><p class="paragraph">We also know that change is the leading cause of outages. I’ve seen estimates as high as 75+% of all incidents are at the change boundary. As a friend has said “introducing a change boundary in the 4-6 remaining hours before the whole team is off for 50+ hours seems … like not a high probability play”. But, let’s use the 7.5% change failure rate for elite performers. Do you wear a seatbelt in the car? Yes? Why? What if you had a 7.5% chance of a minor accident? What % chance of a major accident would make you wear a seatbelt? If your argument is that accidents are out of your control, I’d like to introduce you to complex distributed systems…</p><p class="paragraph">To put it another way, at Google when you violate your SLO, what is slowed down? Releases. Not more tests, not more monitoring, releases.</p><p class="paragraph">The other problem is that it’s not even <em>necessary</em> for you to cause an outage for your weekend to be interrupted. I learned long ago to be very careful about when I did firewall upgrades. Why? Because every firewall upgrade was generally accompanied by days of spurious correlations about whether something was affected by the upgrade. “Dave, I can’t print, didn’t you recently upgrade the firewall?” “Neither the network traffic for your laptop, nor the printer goes through the firewall.” “But couldn’t…” “No”.</p><h2>Having the capability is necessary </h2><p class="paragraph">Now choosing not to deploy on Fridays is very different than having the capability to deploy on Fridays. You should have the capability to deploy at any time. Things break, code needs to be shipped. You should absolutely be developing this capability if you do not have it.</p><p class="paragraph">I have worked with elite performers. We still chose to be very careful about our Friday deploys.</p><p class="paragraph">We also chose to make sure our feature flagging, blue/green, and dark launching capabilities were robust. We had developers deploying their own code whenever they wanted. We deployed multiple services multiple times a day. Every day.</p><p class="paragraph">But when Friday afternoon came, if someone was going to push a deploy, they would consider if that was necessary, or if it could wait until Monday. After all, there were other things to do.</p><h2>Cultural Norms are not Moratoriums</h2><p class="paragraph"><strong>Moratoriums</strong><br/>Being choosy about deploying on Friday is not the same as a moratorium. Moratoriums generally require some kind of change advisory board to approve special cases for releasing during the moratorium. Instead, minimizing risk during times where the results of a failure can have outsized impacts is part of the communication and respect we see in DevOps. If the core of DevOps is <a href="https://youtu.be/M1mbI4kUskQ?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">empathy</a>, then considering the impact our actions can have on others is exactly that - empathetic. Besides, change advisory boards are useless!</p><div class="image-container"><div class="w-full my-4"><div class="flex items-center justify-center w-full h-full"><div class="relative group cursor-zoom-in transition-transform hover:scale-[1.01] max-w-full mx-auto flex flex-col items-center"><button class="absolute top-2 right-5 z-10 w-6 h-6 rounded flex items-center justify-center opacity-0 group-hover:opacity-100 transition-opacity"><i class="hn hn-download text-darkText bg-dark p-2 rounded-xl text-base"></i></button><img alt="Dave Mangot's image-5b7788" fetchPriority="low" loading="lazy" width="948" height="1018" decoding="async" data-nimg="1" class="w-full h-auto object-contain rounded-lg shadow-lg my-0" style="color:transparent;background-size:cover;background-position:50% 50%;background-repeat:no-repeat;background-image:url("data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 948 1018'%3E%3Cfilter id='b' color-interpolation-filters='sRGB'%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3CfeColorMatrix values='1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 100 -1' result='s'/%3E%3CfeFlood x='0' y='0' width='100%25' height='100%25'/%3E%3CfeComposite operator='out' in='s'/%3E%3CfeComposite in2='SourceGraphic'/%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3C/filter%3E%3Cimage width='100%25' height='100%25' x='0' y='0' preserveAspectRatio='none' style='filter: url(%23b);' href='data:image/svg+xml;base64,CiAgICA8c3ZnIHdpZHRoPSI5NDgiIGhlaWdodD0iMTAxOCIgdmVyc2lvbj0iMS4xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHhtbG5zOnhsaW5rPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIj4KICAgICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJnIj4KICAgICAgICAgIDxzdG9wIHN0b3AtY29sb3I9IiMyMjIiIG9mZnNldD0iMjAlIiAvPgogICAgICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzBmMCIgb2Zmc2V0PSI1MCUiIC8+CiAgICAgICAgICA8c3RvcCBzdG9wLWNvbG9yPSIjMjIyIiBvZmZzZXQ9IjgwJSIgLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICA8L2RlZnM+CiAgICAgIDxyZWN0IHdpZHRoPSI5NDgiIGhlaWdodD0iMTAxOCIgZmlsbD0iIzIyMiIgLz4KICAgICAgPHJlY3QgaWQ9InIiIHdpZHRoPSI5NDgiIGhlaWdodD0iMTAxOCIgZmlsbD0idXJsKCNnKSIgLz4KICAgICAgPGFuaW1hdGUgeGxpbms6aHJlZj0iI3IiIGF0dHJpYnV0ZU5hbWU9IngiIGZyb209Ii05NDgiIHRvPSI5NDgiIGR1cj0iMXMiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiAgLz4KICAgIDwvc3ZnPg=='/%3E%3C/svg%3E")" srcSet="https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-fe2m3zs2.jpg?auto=format%2Ccompress&w=1080 1x, https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-fe2m3zs2.jpg?auto=format%2Ccompress&w=1920 2x" src="https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-fe2m3zs2.jpg?auto=format%2Ccompress&w=1920"/></div></div></div></div><p class="line-space"> <br/> </p><p class="paragraph"><strong>Netflix</strong> </p><p class="paragraph">I was happy to learn that I’m not the only one who has worked at places that were cautious about their Friday deploys when Aaron Blohowiak tweeted about Netflix:</p><div class="image-container"><div class="w-full my-4"><div class="flex items-center justify-center w-full h-full"><div class="relative group cursor-zoom-in transition-transform hover:scale-[1.01] max-w-full mx-auto flex flex-col items-center"><button class="absolute top-2 right-5 z-10 w-6 h-6 rounded flex items-center justify-center opacity-0 group-hover:opacity-100 transition-opacity"><i class="hn hn-download text-darkText bg-dark p-2 rounded-xl text-base"></i></button><img alt="Dave Mangot's image-b46498" fetchPriority="low" loading="lazy" width="1080" height="474" decoding="async" data-nimg="1" class="w-full h-auto object-contain rounded-lg shadow-lg my-0" style="color:transparent;background-size:cover;background-position:50% 50%;background-repeat:no-repeat;background-image:url("data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 1080 474'%3E%3Cfilter id='b' color-interpolation-filters='sRGB'%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3CfeColorMatrix values='1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 100 -1' result='s'/%3E%3CfeFlood x='0' y='0' width='100%25' height='100%25'/%3E%3CfeComposite operator='out' in='s'/%3E%3CfeComposite in2='SourceGraphic'/%3E%3CfeGaussianBlur stdDeviation='20'/%3E%3C/filter%3E%3Cimage width='100%25' height='100%25' x='0' y='0' preserveAspectRatio='none' style='filter: url(%23b);' href='data:image/svg+xml;base64,CiAgICA8c3ZnIHdpZHRoPSIxMDgwIiBoZWlnaHQ9IjQ3NCIgdmVyc2lvbj0iMS4xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHhtbG5zOnhsaW5rPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIj4KICAgICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJnIj4KICAgICAgICAgIDxzdG9wIHN0b3AtY29sb3I9IiMyMjIiIG9mZnNldD0iMjAlIiAvPgogICAgICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzBmMCIgb2Zmc2V0PSI1MCUiIC8+CiAgICAgICAgICA8c3RvcCBzdG9wLWNvbG9yPSIjMjIyIiBvZmZzZXQ9IjgwJSIgLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICA8L2RlZnM+CiAgICAgIDxyZWN0IHdpZHRoPSIxMDgwIiBoZWlnaHQ9IjQ3NCIgZmlsbD0iIzIyMiIgLz4KICAgICAgPHJlY3QgaWQ9InIiIHdpZHRoPSIxMDgwIiBoZWlnaHQ9IjQ3NCIgZmlsbD0idXJsKCNnKSIgLz4KICAgICAgPGFuaW1hdGUgeGxpbms6aHJlZj0iI3IiIGF0dHJpYnV0ZU5hbWU9IngiIGZyb209Ii0xMDgwIiB0bz0iMTA4MCIgZHVyPSIxcyIgcmVwZWF0Q291bnQ9ImluZGVmaW5pdGUiICAvPgogICAgPC9zdmc+'/%3E%3C/svg%3E")" srcSet="https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-1fa23zxz.jpg?auto=format%2Ccompress&w=1080 1x, https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-1fa23zxz.jpg?auto=format%2Ccompress&w=3840 2x" src="https://hackernoon.imgix.net/images/FspsB7JTwpPQEzoWQv4iuBE61X83-1fa23zxz.jpg?auto=format%2Ccompress&w=3840"/></div></div></div></div><p class="paragraph">Even the tech giant Netflix has a cultural norm to avoid Friday afternoons. Not a moratorium, but part of the <a href="https://web.archive.org/web/20180630193826/http://blog.librato.com/posts/black-friday?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">culture of the company</a>.</p><p class="paragraph">Another great example from Netflix is the Chaos Monkey. The Chaos Monkey runs <em>during business hours</em> so that people will be more available to respond should something untoward occur. Is that because Netflix doesn’t do enough testing? Or maybe their monitoring is not good enough to run on the weekend? If every hour of every day is exactly the same as every other day, this would make no sense. Instead, they run the Monkey when people are around to address problems, not when they have other commitments.</p><p class="paragraph"><strong>Complex Distributed Systems</strong></p><p class="paragraph">The fact remains that generally we are working on complex distributed systems, and the causes of outages are often elusive. Often when we discover the nature of a problem, it is only obvious in hindsight.</p><p class="paragraph">A number of years ago, I was rebuilding a SQL proxy tier at a company. We were automating our proxy builds, and deploying new versions of the software. We used these proxies to keep the short lived nature of our PHP application requests from opening thousands of requests per second to the database. The cost of connection initiation and teardown was not free, so we had an intermediate tier that was designed to take this kind of load much more effectively than the database itself.</p><p class="paragraph">I was building the new tier but was not sending any traffic to it because the weekend was coming. After everything was built, things seemed calm. Until about an hour later, when the database started having problems. Connections were randomly timing out. We looked and saw that the database was often hitting max connections and as a result, many requests were not making it through. Ultimately we determined this was because my new tier had opened connections to the DB, even though they were not being used and that had pushed us over the limit when a certain class of traffic appeared. Seems pretty obvious in hindsight and we ultimately determined what happened through monitoring.</p><p class="paragraph">But the facts were:</p><ul><li>This was a new tier that had <strong>never</strong> taken production traffic</li><li>It was in a brand new VLAN that had never seen production</li><li>This was a new version of the software</li><li>The databases had been running fine the entire time this tier was being created</li><li>The new tier had all the latest monitoring on it and showed no signs of problems</li><li>The tier it was replacing also showed no signs of problems</li></ul><p class="paragraph">And yet the database was dropping connections, and it was an all hands on deck situation on a Friday afternoon when most people were thinking about their weekend. Thankfully, it was relatively easy to resolve.</p><p class="paragraph">Are these types of things common with releasing new software? No, but they happen.</p><h2>Four Day Work Week</h2><p class="paragraph">So, if we’re not going to deploy on Friday afternoons, what do we do with that time? Do we just give everyone Friday off? Less shipping means no work? One thing we can do is be protective of our employees through work life balance and reduction of stress. </p><p class="paragraph">I have read with great interest about <a href="https://www.nytimes.com/2018/07/19/world/asia/four-day-workweek-new-zealand.html?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc"> Four day work week experiments</a>. Among the validated results of moving to a four day work week were:</p><ul><li>Boosted productivity</li><li>24% improvement in work life balance</li></ul><p class="paragraph">Being protective of employees is something of which I’ve always been very supportive. Whether it’s booting the person off Hangouts who had a blanket over their head and a hot bowl of soup in their hands, or insisting people take the day off when they’ve been up all night troubleshooting an especially difficult issue after a bad deploy.</p><p class="paragraph">I realize most companies are not going to investigate a four day work week, but Friday afternoons can be used for:</p><ul><li><a href="http://tech.mangot.com/blog/2019/01/09/an-agile-sre-meeting-plan/?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">Demos and retrospectives </a></li><li>Writing documentation</li><li>Mentoring</li><li>20% time or exploration time</li><li>Working in the staging environment</li><li>Backlog grooming and prioritization</li><li><a href="https://twitter.com/davemangot/status/1184874200609083392?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">Team lunches</a>! </li></ul><p class="paragraph">At one of the companies I worked for, most of the Ops team would go out for lunch every Friday. That meant lots of Friday morning deploys, and then lots of great collaboration in the afternoon.</p><p class="paragraph">If you’re working with a globally distributed team, do you do a Friday afternoon deploy on the west coast of the United States? That’s almost the next day in most parts of Europe. Most Europeans are not excited to be called back to work late in their evening to help figure out why 8% of traffic is getting 500 errors after a Friday afternoon deploy.</p><h2>Weekends (all days are not the same)</h2><p class="paragraph">One of my favorite things about working at Salesforce was the number of people who chose to wear Hawaiian shirts on Friday. This is something I’d done on and off over the years ever since the release of <a href="https://www.imdb.com/title/tt0151804/?ref=hackernoon.com" target="_blank" rel="noopener noreferrer ugc">Office Space</a> as a way of recognizing the specialness of Friday.</p><p class="paragraph">If the argument is that releasing new software is the same regardless of the day, that ignores what people do on weekends. People make plans to go away, they go camping, they go to the opera, they read books in a hammock by the shore. They go to their kids soccer games, they work in the community garden, etc. Do some of those things happen occasionally on a Tuesday night? Sure they do. But the vast majority of weekend travel happens on the <strong>weekend</strong> and doing activities that can jeopardize that doesn’t show a lot of respect for your coworkers, or your employees’ work/life balance. They need to take this time to rest and recharge.</p><p class="paragraph">This is one of the reasons I always liked on-call rotations that rolled over on a Thursday. I always wanted my teams to be able to take a Friday off to get away for a three day weekend. The more downtime in a block, the better.</p><p class="paragraph">If the argument is that not shipping on Friday afternoon is going to hurt productivity, remember the old Agile adage “you have to go slow to go fast”. Driving a system at 100% capacity is actually a way to reduce your throughput, not maintain it.</p><p class="paragraph">Does this mean that you should never deploy on Friday? Of course not. That also does not mean you shouldn’t consider what you’re deploying. It may seem to make sense when someone says “We scheduled the move from Oracle to Postgres for 4 hours, so if we start at 1 p.m., we should be done in plenty of time.” My answer to that logic is: NOOOOOOOO.</p><p class="paragraph">You should be able to, but you don’t have to deploy on Friday afternoon. You should not be shamed for having a culture that respects Friday as being different. It <em>is</em> different.</p><p class="paragraph">Deploy on Friday’s or Don’t. The choice is up to you.</p><p class="line-space"> <br/> </p><p class="line-space"> <br/> </p></div></div></div></div></div><div class="hidden xl:flex xl:flex-col self-stretch"><div class="sticky top-[99px] px-3"><div class=" flex flex-col flex-row-reverse items-start gap-4 "><span class="tooltip tooltip-left cursor-pointer" data-tip="Bookmark"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-bookmark text-lightText text-2xl"></i></button></span><span class="tooltip tooltip-left cursor-pointer" data-tip="Comment"><button class="3xl:hover:bg-lightAlt hover:bg-light p-1 md:p-2 rounded h-[40px] w-[40px] flex items-center justify-center border border-lightBorder"><i class="hn hn-comment text-lightText text-2xl"></i></button></span><div class="dropdown dropdown-bottom dropdown-hover group "><label tabindex="0" class="flex items-center cursor-pointer justify-center border border-lightBorder 3xl:group-hover:bg-lightAlt group-hover:bg-light h-[40px] w-[40px] p-2 rounded "><i class="hn hn-share text-2xl"></i></label><ul tabindex="0" class="dropdown-content bg-light z-[1] py-4 px-4 3xl:px-0 3xl:py-2 border 3xl:border-none flex flex-col items-center justify-center gap-2 "><button class="border p-2 rounded hover:bg-lightAlt"><i class=" hn hn-copy text-lightText text-2xl "></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-facebook-round text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-x text-lightText text-2xl"></i></button><button class="border p-2 rounded hover:bg-lightAlt"><i class="hn hn-linkedin text-lightText text-2xl"></i></button><a href="mailto:?subject=I'd like to share a link with you &body=" class="border p-2 rounded inline-block hover:bg-lightAlt"><i class="hn hn-envelope text-lightText text-2xl"></i></a></ul></div></div></div></div></div><div class="px-4 lg:px-0 mx-auto w-full lg:max-w-[1000px] flex-col flex items-center justify-center "></div><div id="aboutCard" class=" max-w-[1000px] mx-auto flex flex-col items-center gap-6 "><div class="w-full lg:border border-lightBorder rounded-2xl"><div class=" w-full px-4 py-3 sm:px-8 sm:py-6 "><h3 class="text-xl xs:text-2xl sm:text-3xl font-bold mb-6">About Author</h3><div class="flex flex-col items-start"><div class="flex gap-4 flex-row items-start w-full"><div class="relative shadow-md rounded-full flex-shrink-0 min-w-[50px] w-[50px] h-[50px] sm:min-w-[75px] sm:h-[75px] ring-4 ring-gray-300"><a href="/u/dmangot"><img alt="Dave Mangot HackerNoon profile picture" loading="lazy" decoding="async" data-nimg="fill" class="rounded-full" style="position:absolute;height:100%;width:100%;left:0;top:0;right:0;bottom:0;object-fit:cover;color:transparent" sizes="100vw" srcSet="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=640 640w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=750 750w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=828 828w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=1080 1080w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=1200 1200w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=1920 1920w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=2048 2048w, https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=3840 3840w" src="https://hackernoon.imgix.net/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg?auto=format%2Ccompress&w=3840"/></a></div><div class="flex-1 min-w-0 flex flex-col justify-center"><div class="flex flex-col"><div class="flex flex-wrap items-center gap-1 sm:gap-2 text-base text-bodyTxtLight"><span class="text-xs font-light mr-1">by</span><a class="hover:underline" href="/u/dmangot"><strong class="font-bold text-lightTextStrong">Dave Mangot</strong></a><span class="text-xs font-light opacity-50">|</span><a class="hover:underline text-sm sm:text-base text-bodyTxtLight" href="/u/dmangot">@<!-- -->dmangot</a></div></div></div></div><p class="text-sm text-bodyTxtLight break-words overflow-wrap mt-4 mb-4 w-full">Dave Mangot is the principal at Mangoteque, & has successfully led DevOps & Ops transformations.</p><div class="w-full mb-4"><div class="w-full flex justify-start"><div class="w-full"><form class="w-full flex flex-col items-start gap-2 "><div class="flex w-full"><input class="p-2 flex-grow border rounded-l-md text-lightText bg-light focus:outline-none focus:ring-0 focus:ring-transparent border-lightBorder w-full text-base px-2}
|
||
}" placeholder="name@company.com" type="email" required="" name="email" value=""/><button type="submit" class="text-base}
|
||
bg-lightAlt border border-l-0 border-lightBorder hover:bg-green-700 text-lightText hover:bg-dark hover:text-darkText px-2 py-1 rounded-r-md font-bold">Subscribe</button></div></form></div></div></div><div class="flex-1 w-full"><div class="flex flex-col flex-wrap gap-2 items-start justify-start mt-2 mb-2"></div><div class="flex flex-col sm:flex-row gap-4 w-full"><a class="text-base flex-1 px-4 py-2 font-bold rounded-lg border-2 border-lightBorder transition text-center w-full sm:w-auto bg-light hover:bg-lightAlt text-lightText hover:bg-bodyAccent hover:text-bodyAccentTxt " href="/u/dmangot">Read my stories</a><a class="text-base break-words flex-1 break-all px-4 py-2 font-bold rounded-lg border-2 border-lightBorder transition text-center w-full sm:w-auto bg-light hover:bg-lightAlt text-lightText hover:bg-bodyAccent hover:text-bodyAccentTxt " href="/about/dmangot">About @dmangot</a></div></div></div></div></div><span id="aboutCard" class="hidden"></span><section class="w-full py-3 px-4 sm:px-0 sm:py-6 "><h4 class="text-xl xs:text-2xl sm:text-3xl font-bold mb-4 sm:mb-6">TOPICS</h4><div class="flex flex-wrap gap-2"><div class=" flex flex-wrap items-center gap-2 border-lightBorder"><a href="/c/web3" target="_blank" rel="noopener noreferrer" class="text-lg border-lightBorder hover:bg-lightAccent hover:text-lightAccentText hover:border-lightAccentText bg-lightAlt text-lightText flex items-center px-2 py-1 border rounded"><span class="mr-2"><i class="hn hn-web3 !leading-[inherit]"></i></span><span>web3</span></a></div><a href="/tagged/devops-principles" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->devops-principles</a><a href="/tagged/deployment-challenges" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->deployment-challenges</a><a href="/tagged/continuous-deployment" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->continuous-deployment</a><a href="/tagged/developer-operations" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->developer-operations</a><a href="/tagged/continuous-delivery" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->continuous-delivery</a><a href="/tagged/developer-productivity" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->developer-productivity</a><a href="/tagged/release-management" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->release-management</a><a href="/tagged/hackernoon-top-story" target="_blank" rel="noopener noreferrer" class="text-sm xs:text-base sm:text-lg flex items-center px-2 py-1 border hover:bg-lightAlt border-lightBorder rounded">#<!-- -->hackernoon-top-story</a></div></section></div></div></div></div></div><div class="min-h-[200px]"></div></main><div class="flex flex-col gap-4 hidden"><button class="mr-auto"><i class="hn-sun hn text-2xl"></i></button><h2 class="text-sm font-semibold text-darkText">Light-Mode</h2><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Classic</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#0F0"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#F5EC43"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#212428"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Newspaper</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#FFFFFF"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#F5F5F5"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#454545"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Proof of Usefulness</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#FFFFFF"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#26AB5C"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#D2FBE2"></span></div></div><h2 class="text-sm font-semibold text-darkText">Dark-Mode</h2><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Neon Noir</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#1E1E1E"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#0F0"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#F5EC43"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Minty</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#061F19"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#2AAA74"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#63FF86"></span></div></div><div class="cursor-pointer p-3 rounded-lg hover:scale-105 transition-transform "><h3 class="text-sm uppercase mb-2 ">Startups of the Year</h3><div class="flex space-x-1"><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#08085E"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#A2EF44"></span><span class="w-6 h-6 rounded border border-darkBorder" style="background-color:#1B1B95"></span></div></div></div></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"data":{"pageLang":"en","datePublished":"2019-10-25","slug":"deploy-on-fridays-or-dont-qg2y32jk","articleBody":"There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I about deploying on Friday, but also based on the science of it, as well as my learned experience. feel With a title like “Deploy on Fridays, or Don’t”, I realize the expected continuation of that statement would be “...I don’t care”. However, nothing could be further from the truth. Let me explain. My advice to anyone who will listen is, if you’re cautious about your Friday deploys, don’t feel bad, and don’t let anyone make you feel bad. Shaming It is pretty disconcerting to see a like Kelly’s where the vast majority of the comment thread are attempts to shame anyone for having that opinion. The arguments basically boil down to some variant on tweet Deploying shouldn’t be scary You should be confident in your deploys You lose 20% of your productivity without deploying on Friday You just need more tests Those are all interesting ideas, and they reflect a very interesting type of smug optimism. They are often “backed up” by quoting or the . They eventually arrive at a compromise that you should do the best you can, and keep maturing your deployments until you can deploy anytime you like 365 days with “confidence”. There is also acknowledgement that this can be hard, and having worked with a number of clients and companies over the years, with this I agree. Accelerate State of DevOps report However, here are most of the problems with the logic: Quality Engineering Even Elite performers have change failure % \u003e 0 Mores are not Moratoriums All days are not the same Lastly there is some strange argument that choosing not to deploy on Friday \"Shouldn't be a source of glee and pride\". That one I haven’t figured out yet, because I have always had a lot of glee and pride in being extremely (overly?) of the engineers who either work for me, or with me. I don’t expect that to change. protective of the work/life balance Quality Engineering While working at Salesforce, I had the opportunity to learn a lot about . This was also the time when I read This was a book that changed my life, and I say that confidently. One of the things I loved about that book was the idea that the more testing that was done, the more confident you could be in the artifact you were deploying. When pitching CI/CD pipeline proposals to executives, they would ask how confident we could be in our artifacts, and I would respond with “How much do you want to spend?”. The more money they were willing to spend, the better testing we could do, and therefore, the more confidence we would have. One of the other things Continuous Delivery taught me was how important it is to have fast feedback. Ultimately, your confidence when deploying to production is going to be some compromise between those two. If you do automated testing for 15 hours, you should obviously have more confidence than if you do so for 30 seconds. quality Continuous Delivery by Jez Humble and Dave Farley. One thing that was present in the book however, was any notion that you could be 100% confident in what you tested. That is, you could not the quality of that artifact tested. Now, Salesforce has a very mature testing pipeline. There were literally hundreds of thousands of tests that were being run more than 5 years ago and yet, they had a quality engineering discipline, but not a quality assurance discipline. Why? not assure Because one cannot assure quality in software. In manufacturing, if I am making shampoo, I can have quality assurance test for quality. QA takes a statistically representative sample of each bottle of shampoo coming down the line, and test to make sure that the chemical composition of what is being produced is within the tolerances as described by the quality specifications. They statistically assure the company that the quality is consistent. In software, you cannot do this. You can not take a random sample of code coming through your continuous delivery pipeline, test those lines of code, and then assure that when that code is deployed to production, it will perform at a level consistent with what has been defined. Therefore it is foolish to lecture people that they should be deploying on Fridays, because they just need to “be confident in their code”, or “write more tests”. How many people dispensing this advice have hundreds of thousands of tests being run on their code? How many people have 100% code coverage in their tests (if this sounds appealing: please don’t do this, the last percentage points will suffer from the laws of diminishing returns)? What I did not take from Jez and Dave’s book, is that you should fool yourself into thinking that you should be 100% confident in everything you push, just because you have tests. Thankfully, Jez continues to talk (along with Nicole, Gene, and others) about tests in the State of DevOps report. DORA Report The DORA Report is often referenced as proof that you should deploy on Fridays, just like any other day. Because it provides data to help classify organizations, including defining how higher vs. lower performing organizations deploy, it’s useful to look at in cases like this. For instance: Elite performers - mean change failure rate 7.5%, recovery \u003c 1hr, deploy multiple times a day Low performers - mean change failure rate 53%, recovery \u003e 1w, deploy \u003e 1 month So, assuming that the advice being dispensed is saying “just become an elite performer”, which is in no way a trivial exercise, they still have a mean change failure rate of 7.5%! Does that sound like they will not have any failures during deploys? That’s one way to make Friday afternoon more exciting! I realize the mean recovery time is \u003c 1 hr, but that is also the . What does the distribution look like? Is there a cluster at 10 hours? Is there a cluster at 2 minutes? I don’t know. But regardless, there is no guarantee of any deploy being failure free, because even the elite performers have failures. mean We also know that change is the leading cause of outages. I’ve seen estimates as high as 75+% of all incidents are at the change boundary. As a friend has said “introducing a change boundary in the 4-6 remaining hours before the whole team is off for 50+ hours seems … like not a high probability play”. But, let’s use the 7.5% change failure rate for elite performers. Do you wear a seatbelt in the car? Yes? Why? What if you had a 7.5% chance of a minor accident? What % chance of a major accident would make you wear a seatbelt? If your argument is that accidents are out of your control, I’d like to introduce you to complex distributed systems… To put it another way, at Google when you violate your SLO, what is slowed down? Releases. Not more tests, not more monitoring, releases. The other problem is that it’s not even for you to cause an outage for your weekend to be interrupted. I learned long ago to be very careful about when I did firewall upgrades. Why? Because every firewall upgrade was generally accompanied by days of spurious correlations about whether something was affected by the upgrade. “Dave, I can’t print, didn’t you recently upgrade the firewall?” “Neither the network traffic for your laptop, nor the printer goes through the firewall.” “But couldn’t…” “No”. necessary Having the capability is necessary Now choosing not to deploy on Fridays is very different than having the capability to deploy on Fridays. You should have the capability to deploy at any time. Things break, code needs to be shipped. You should absolutely be developing this capability if you do not have it. I have worked with elite performers. We still chose to be very careful about our Friday deploys. We also chose to make sure our feature flagging, blue/green, and dark launching capabilities were robust. We had developers deploying their own code whenever they wanted. We deployed multiple services multiple times a day. Every day. But when Friday afternoon came, if someone was going to push a deploy, they would consider if that was necessary, or if it could wait until Monday. After all, there were other things to do. Cultural Norms are not Moratoriums Being choosy about deploying on Friday is not the same as a moratorium. Moratoriums generally require some kind of change advisory board to approve special cases for releasing during the moratorium. Instead, minimizing risk during times where the results of a failure can have outsized impacts is part of the communication and respect we see in DevOps. If the core of DevOps is , then considering the impact our actions can have on others is exactly that - empathetic. Besides, change advisory boards are useless! Moratoriums empathy Netflix I was happy to learn that I’m not the only one who has worked at places that were cautious about their Friday deploys when Aaron Blohowiak tweeted about Netflix: Even the tech giant Netflix has a cultural norm to avoid Friday afternoons. Not a moratorium, but part of the . culture of the company Another great example from Netflix is the Chaos Monkey. The Chaos Monkey runs so that people will be more available to respond should something untoward occur. Is that because Netflix doesn’t do enough testing? Or maybe their monitoring is not good enough to run on the weekend? If every hour of every day is exactly the same as every other day, this would make no sense. Instead, they run the Monkey when people are around to address problems, not when they have other commitments. during business hours Complex Distributed Systems The fact remains that generally we are working on complex distributed systems, and the causes of outages are often elusive. Often when we discover the nature of a problem, it is only obvious in hindsight. A number of years ago, I was rebuilding a SQL proxy tier at a company. We were automating our proxy builds, and deploying new versions of the software. We used these proxies to keep the short lived nature of our PHP application requests from opening thousands of requests per second to the database. The cost of connection initiation and teardown was not free, so we had an intermediate tier that was designed to take this kind of load much more effectively than the database itself. I was building the new tier but was not sending any traffic to it because the weekend was coming. After everything was built, things seemed calm. Until about an hour later, when the database started having problems. Connections were randomly timing out. We looked and saw that the database was often hitting max connections and as a result, many requests were not making it through. Ultimately we determined this was because my new tier had opened connections to the DB, even though they were not being used and that had pushed us over the limit when a certain class of traffic appeared. Seems pretty obvious in hindsight and we ultimately determined what happened through monitoring. But the facts were: This was a new tier that had taken production traffic never It was in a brand new VLAN that had never seen production This was a new version of the software The databases had been running fine the entire time this tier was being created The new tier had all the latest monitoring on it and showed no signs of problems The tier it was replacing also showed no signs of problems And yet the database was dropping connections, and it was an all hands on deck situation on a Friday afternoon when most people were thinking about their weekend. Thankfully, it was relatively easy to resolve. Are these types of things common with releasing new software? No, but they happen. Four Day Work Week So, if we’re not going to deploy on Friday afternoons, what do we do with that time? Do we just give everyone Friday off? Less shipping means no work? One thing we can do is be protective of our employees through work life balance and reduction of stress. I have read with great interest about . Among the validated results of moving to a four day work week were: Four day work week experiments Boosted productivity 24% improvement in work life balance Being protective of employees is something of which I’ve always been very supportive. Whether it’s booting the person off Hangouts who had a blanket over their head and a hot bowl of soup in their hands, or insisting people take the day off when they’ve been up all night troubleshooting an especially difficult issue after a bad deploy. I realize most companies are not going to investigate a four day work week, but Friday afternoons can be used for: Demos and retrospectives Writing documentation Mentoring 20% time or exploration time Working in the staging environment Backlog grooming and prioritization ! Team lunches At one of the companies I worked for, most of the Ops team would go out for lunch every Friday. That meant lots of Friday morning deploys, and then lots of great collaboration in the afternoon. If you’re working with a globally distributed team, do you do a Friday afternoon deploy on the west coast of the United States? That’s almost the next day in most parts of Europe. Most Europeans are not excited to be called back to work late in their evening to help figure out why 8% of traffic is getting 500 errors after a Friday afternoon deploy. Weekends (all days are not the same) One of my favorite things about working at Salesforce was the number of people who chose to wear Hawaiian shirts on Friday. This is something I’d done on and off over the years ever since the release of as a way of recognizing the specialness of Friday. Office Space If the argument is that releasing new software is the same regardless of the day, that ignores what people do on weekends. People make plans to go away, they go camping, they go to the opera, they read books in a hammock by the shore. They go to their kids soccer games, they work in the community garden, etc. Do some of those things happen occasionally on a Tuesday night? Sure they do. But the vast majority of weekend travel happens on the and doing activities that can jeopardize that doesn’t show a lot of respect for your coworkers, or your employees’ work/life balance. They need to take this time to rest and recharge. weekend This is one of the reasons I always liked on-call rotations that rolled over on a Thursday. I always wanted my teams to be able to take a Friday off to get away for a three day weekend. The more downtime in a block, the better. If the argument is that not shipping on Friday afternoon is going to hurt productivity, remember the old Agile adage “you have to go slow to go fast”. Driving a system at 100% capacity is actually a way to reduce your throughput, not maintain it. Does this mean that you should never deploy on Friday? Of course not. That also does not mean you shouldn’t consider what you’re deploying. It may seem to make sense when someone says “We scheduled the move from Oracle to Postgres for 4 hours, so if we start at 1 p.m., we should be done in plenty of time.” My answer to that logic is: NOOOOOOOO. You should be able to, but you don’t have to deploy on Friday afternoon. You should not be shamed for having a culture that respects Friday as being different. It different. is Deploy on Friday’s or Don’t. The choice is up to you.","arweave":"NOw9vUJs_F3RYHAVMh9_Ytm6Qyywgnt0VGHoHtruJNY","createdAt":"2019-11-07T06:56:07.860Z","draftId":"grYOhuLvfXnQXRszyXZA","excerpt":"There seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc. There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I feel about deploying on Friday, but also based on the science of it, as well as my learned experience.","featureImgColor":"rgb(211 179 132 / 70%)","firstSeenAt":false,"id":"TjnmEeP0JCGMYYQHtk4D","imageSizes":{"https://hackernoon.com/images/FspsB7JTwpPQEzoWQv4iuBE61X83-fe2m3zs2.jpg":{"width":948,"height":1018},"https://hackernoon.com/images/FspsB7JTwpPQEzoWQv4iuBE61X83-qn163zae.jpg":{"width":930,"height":638},"https://hackernoon.com/images/FspsB7JTwpPQEzoWQv4iuBE61X83-1fa23zxz.jpg":{"width":1080,"height":474}},"mainImage":"https://hackernoon.imgix.net/drafts/9gou3zam.png","mainImageHeight":1697,"mainImageWidth":2552,"markup":"\u003cdiv class=\"paragraph\"\u003eThere seems to be a debate that has gone on for quite some time now on the Twitters about whether or not you should do Friday deploys, and whether there should be Friday moratoriums, etc.\u0026#xA0; There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. To be very clear, I am not a big fan of Friday deploys. That opinion is not based on merely how I \u003cstrong\u003efeel\u003c/strong\u003e about deploying on Friday, but also based on the science of it, as well as my learned experience.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eWith a title like \u0026#x201C;Deploy on Fridays, or Don\u0026#x2019;t\u0026#x201D;, I realize the expected continuation of that statement would be \u0026#x201C;...I don\u0026#x2019;t care\u0026#x201D;.\u0026#xA0; However, nothing could be further from the truth. Let me explain.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eMy advice to anyone who will listen is, if you\u0026#x2019;re cautious about your Friday deploys, don\u0026#x2019;t feel bad, and don\u0026#x2019;t let anyone make you feel bad.\u003c/div\u003e\u003cdiv class=\"image-container\"\u003e\u003cimg src=\"https://hackernoon.com/images/FspsB7JTwpPQEzoWQv4iuBE61X83-qn163zae.jpg\" alt\u003e\u003c/div\u003e\u003ch2\u003e\u003cstrong\u003eShaming\u003c/strong\u003e\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003eIt is pretty disconcerting to see a \u003ca href=\"https://twitter.com/kvlly/status/1180075727267467264\"\u003etweet\u003c/a\u003e like Kelly\u0026#x2019;s where the vast majority of the comment thread are attempts to shame anyone for having that opinion.\u0026#xA0; The arguments basically boil down to some variant on\u0026#xA0;\u003c/div\u003e\u003cul\u003e\u003cli\u003eDeploying shouldn\u0026#x2019;t be scary\u003c/li\u003e\u003cli\u003eYou should be confident in your deploys\u003c/li\u003e\u003cli\u003eYou lose 20% of your productivity without deploying on Friday\u003c/li\u003e\u003cli\u003eYou just need more tests\u003c/li\u003e\u003c/ul\u003e\u003cdiv class=\"paragraph\"\u003eThose are all interesting ideas, and they reflect a very interesting type of smug optimism.\u0026#xA0; They are often \u0026#x201C;backed up\u0026#x201D; by quoting \u003ca href=\"https://itrevolution.com/book/accelerate/\"\u003eAccelerate\u003c/a\u003e or the \u003ca href=\"https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling\"\u003eState of DevOps report\u003c/a\u003e.\u0026#xA0; They eventually arrive at a compromise that you should do the best you can, and keep maturing your deployments until you can deploy anytime you like 365 days with \u0026#x201C;confidence\u0026#x201D;.\u0026#xA0; There is also acknowledgement that this can be hard, and having worked with a number of clients and companies over the years, with this I agree.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eHowever, here are most of the problems with the logic:\u003c/div\u003e\u003cul\u003e\u003cli\u003eQuality Engineering\u003c/li\u003e\u003cli\u003eEven Elite performers have change failure % \u0026gt; 0\u003c/li\u003e\u003cli\u003eMores are not Moratoriums\u003c/li\u003e\u003cli\u003eAll days are not the same\u003c/li\u003e\u003c/ul\u003e\u003cdiv class=\"paragraph\"\u003eLastly there is some strange argument that choosing not to deploy on Friday \u0026quot;Shouldn\u0026apos;t be a source of glee and pride\u0026quot;. That one I haven\u0026#x2019;t figured out yet, because I have always had a lot of glee and pride in being extremely (overly?) \u003ca href=\"https://www.youtube.com/watch?v=lzl4nu0ZHQo#action=share\"\u003eprotective of the work/life balance\u003c/a\u003e of the engineers who either work for me, or with me.\u0026#xA0; I don\u0026#x2019;t expect that to change.\u003c/div\u003e\u003ch2\u003eQuality Engineering\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003eWhile working at Salesforce, I had the opportunity to learn a lot about \u003ca href=\"https://www.youtube.com/watch?v=H4ZwBO0P7-E\"\u003equality\u003c/a\u003e.\u0026#xA0; This was also the time when I read \u003ca href=\"https://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley-dp-0321601912/dp/0321601912/ref=mt_hardcover?_encoding=UTF8\u0026amp;me=\u0026amp;qid=\"\u003eContinuous Delivery by Jez Humble and Dave Farley.\u003c/a\u003e\u0026#xA0; This was a book that changed my life, and I say that confidently.\u0026#xA0; One of the things I loved about that book was the idea that the more testing that was done, the more confident you could be in the artifact you were deploying.\u0026#xA0; When pitching CI/CD pipeline proposals to executives, they would ask how confident we could be in our artifacts, and I would respond with \u0026#x201C;How much do you want to spend?\u0026#x201D;. \u0026#xA0; The more money they were willing to spend, the better testing we could do, and therefore, the more confidence we would have. One of the other things Continuous Delivery taught me was how important it is to have fast feedback.\u0026#xA0; Ultimately, your confidence when deploying to production is going to be some compromise between those two.\u0026#xA0; If you do automated testing for 15 hours, you should obviously have more confidence than if you do so for 30 seconds.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eOne thing that was \u003cstrong\u003enot\u003c/strong\u003e present in the book however, was any notion that you could be 100% confident in what you tested.\u0026#xA0; That is, you could not \u003cstrong\u003eassure\u003c/strong\u003e the quality of that artifact tested. \u0026#xA0; Now, Salesforce has a very mature testing pipeline.\u0026#xA0; There were literally hundreds of thousands of tests that were being run more than 5 years ago and yet, they had a quality engineering discipline, but not a quality assurance discipline.\u0026#xA0; Why?\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eBecause one cannot assure quality in software.\u0026#xA0; In manufacturing, if I am making shampoo, I can have quality assurance test for quality.\u0026#xA0; QA takes a statistically representative sample of each bottle of shampoo coming down the line, and test to make sure that the chemical composition of what is being produced is within the tolerances as described by the quality specifications.\u0026#xA0; They statistically assure the company that the quality is consistent.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eIn software, you cannot do this.\u0026#xA0; You can not take a random sample of code coming through your continuous delivery pipeline, test those lines of code, and then assure that when that code is deployed to production, it will perform at a level consistent with what has been defined.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eTherefore it is foolish to lecture people that they should be deploying on Fridays, because they just need to \u0026#x201C;be confident in their code\u0026#x201D;, or \u0026#x201C;write more tests\u0026#x201D;.\u0026#xA0; How many people dispensing this advice have hundreds of thousands of tests being run on their code? How many people have 100% code coverage in their tests (if this sounds appealing: please don\u0026#x2019;t do this, the last percentage points will suffer from the laws of diminishing returns)?\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eWhat I did not take from Jez and Dave\u0026#x2019;s book, is that you should fool yourself into thinking that you should be 100% confident in everything you push, just because you have tests. \u0026#xA0; Thankfully, Jez continues to talk (along with Nicole, Gene, and others) about tests in the State of DevOps report.\u003c/div\u003e\u003ch2\u003eDORA Report\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003eThe DORA Report is often referenced as proof that you should deploy on Fridays, just like any other day.\u0026#xA0; Because it provides data to help classify organizations, including defining how higher vs. lower performing organizations deploy, it\u0026#x2019;s useful to look at in cases like this. For instance:\u0026#xA0;\u003c/div\u003e\u003cul\u003e\u003cli\u003eElite performers - mean change failure rate 7.5%, recovery \u0026lt; 1hr, deploy multiple times a day\u0026#xA0;\u003c/li\u003e\u003cli\u003eLow performers -\u0026#xA0; mean change failure rate 53%, recovery \u0026gt; 1w, deploy \u0026gt; 1 month\u003c/li\u003e\u003c/ul\u003e\u003cdiv class=\"paragraph\"\u003eSo, assuming that the advice being dispensed is saying \u0026#x201C;just become an elite performer\u0026#x201D;, which is in no way a trivial exercise, they still have a mean change failure rate of 7.5%!\u0026#xA0; Does that sound like they will not have any failures during deploys? That\u0026#x2019;s one way to make Friday afternoon more exciting! I realize the mean recovery time is \u0026lt; 1 hr, but that is also the \u003cstrong\u003emean\u003c/strong\u003e. \u0026#xA0; What does the distribution look like? \u0026#xA0; Is there a cluster at 10 hours? Is there a cluster at 2 minutes?\u0026#xA0; I don\u0026#x2019;t know. But regardless, there is no guarantee of any deploy being failure free, because even the elite performers have failures.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eWe also know that change is the leading cause of outages.\u0026#xA0; I\u0026#x2019;ve seen estimates as high as 75+% of all incidents are at the change boundary.\u0026#xA0; As a friend has said \u0026#x201C;introducing a change boundary in the 4-6 remaining hours before the whole team is off for 50+ hours seems \u0026#x2026; like not a high probability play\u0026#x201D;.\u0026#xA0; But, let\u0026#x2019;s use the 7.5% change failure rate for elite performers. Do you wear a seatbelt in the car? Yes? Why? What if you had a 7.5% chance of a minor accident?\u0026#xA0; What % chance of a major accident would make you wear a seatbelt? If your argument is that accidents are out of your control, I\u0026#x2019;d like to introduce you to complex distributed systems\u0026#x2026;\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eTo put it another way, at Google when you violate your SLO, what is slowed down?\u0026#xA0; Releases. Not more tests, not more monitoring, releases.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eThe other problem is that it\u0026#x2019;s not even \u003cem\u003enecessary\u003c/em\u003e for you to cause an outage for your weekend to be interrupted.\u0026#xA0; I learned long ago to be very careful about when I did firewall upgrades.\u0026#xA0; Why? Because every firewall upgrade was generally accompanied by days of spurious correlations about whether something was affected by the upgrade.\u0026#xA0; \u0026#x201C;Dave, I can\u0026#x2019;t print, didn\u0026#x2019;t you recently upgrade the firewall?\u0026#x201D; \u0026#x201C;Neither the network traffic for your laptop, nor the printer goes through the firewall.\u0026#x201D;\u0026#xA0; \u0026#x201C;But couldn\u0026#x2019;t\u0026#x2026;\u0026#x201D; \u0026#x201C;No\u0026#x201D;.\u003c/div\u003e\u003ch2\u003eHaving the capability is necessary\u0026#xA0;\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003eNow choosing not to deploy on Fridays is very different than having the capability to deploy on Fridays.\u0026#xA0; You should have the capability to deploy at any time. Things break, code needs to be shipped. You should absolutely be developing this capability if you do not have it.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eI have worked with elite performers.\u0026#xA0; We still chose to be very careful about our Friday deploys.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eWe also chose to make sure our feature flagging, blue/green, and dark launching capabilities were robust.\u0026#xA0; We had developers deploying their own code whenever they wanted. We deployed multiple services multiple times a day. \u0026#xA0; Every day.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eBut when Friday afternoon came, if someone was going to push a deploy, they would consider if that was necessary, or if it could wait until Monday. \u0026#xA0; After all, there were other things to do.\u003c/div\u003e\u003ch2\u003eCultural Norms are not Moratoriums\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003e\u003cstrong\u003eMoratoriums\u003c/strong\u003e\u003cbr\u003eBeing choosy about deploying on Friday is not the same as a moratorium.\u0026#xA0; Moratoriums generally require some kind of change advisory board to approve special cases for releasing during the moratorium.\u0026#xA0; Instead, minimizing risk during times where the results of a failure can have outsized impacts is part of the communication and respect we see in DevOps. \u0026#xA0; If the core of DevOps is \u003ca href=\"https://youtu.be/M1mbI4kUskQ\"\u003eempathy\u003c/a\u003e, then considering the impact our actions can have on others is exactly that - empathetic.\u0026#xA0; Besides, change advisory boards are useless!\u003c/div\u003e\u003cdiv class=\"image-container\"\u003e\u003cimg src=\"https://hackernoon.com/images/FspsB7JTwpPQEzoWQv4iuBE61X83-fe2m3zs2.jpg\" alt\u003e\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003e\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003e\u003cstrong\u003eNetflix\u003c/strong\u003e\u0026#xA0;\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eI was happy to learn that I\u0026#x2019;m not the only one who has worked at places that were cautious about their Friday deploys when Aaron Blohowiak tweeted about Netflix:\u003c/div\u003e\u003cdiv class=\"image-container\"\u003e\u003cimg src=\"https://hackernoon.com/images/FspsB7JTwpPQEzoWQv4iuBE61X83-1fa23zxz.jpg\" alt\u003e\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eEven the tech giant Netflix has a cultural norm to avoid Friday afternoons.\u0026#xA0; Not a moratorium, but part of the \u003ca href=\"https://web.archive.org/web/20180630193826/http://blog.librato.com/posts/black-friday\"\u003eculture of the company\u003c/a\u003e.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eAnother great example from Netflix is the Chaos Monkey.\u0026#xA0; The Chaos Monkey runs \u003cem\u003eduring business hours\u003c/em\u003e so that people will be more available to respond should something untoward occur.\u0026#xA0; Is that because Netflix doesn\u0026#x2019;t do enough testing? Or maybe their monitoring is not good enough to run on the weekend?\u0026#xA0; If every hour of every day is exactly the same as every other day, this would make no sense. Instead, they run the Monkey when people are around to address problems, not when they have other commitments.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003e\u003cstrong\u003eComplex Distributed Systems\u003c/strong\u003e\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eThe fact remains that generally we are working on complex distributed systems, and the causes of outages are often elusive.\u0026#xA0; Often when we discover the nature of a problem, it is only obvious in hindsight.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eA number of years ago, I was rebuilding a SQL proxy tier at a company.\u0026#xA0; We were automating our proxy builds, and deploying new versions of the software.\u0026#xA0; We used these proxies to keep the short lived nature of our PHP application requests from opening thousands of requests per second to the database.\u0026#xA0; The cost of connection initiation and teardown was not free, so we had an intermediate tier that was designed to take this kind of load much more effectively than the database itself.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eI was building the new tier but was not sending any traffic to it because the weekend was coming.\u0026#xA0; After everything was built, things seemed calm. Until about an hour later, when the database started having problems.\u0026#xA0; Connections were randomly timing out. We looked and saw that the database was often hitting max connections and as a result, many requests were not making it through.\u0026#xA0; Ultimately we determined this was because my new tier had opened connections to the DB, even though they were not being used and that had pushed us over the limit when a certain class of traffic appeared.\u0026#xA0; Seems pretty obvious in hindsight and we ultimately determined what happened through monitoring.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eBut the facts were:\u003c/div\u003e\u003cul\u003e\u003cli\u003eThis was a new tier that had \u003cstrong\u003enever\u003c/strong\u003e taken production traffic\u003c/li\u003e\u003cli\u003eIt was in a brand new VLAN that had never seen production\u003c/li\u003e\u003cli\u003eThis was a new version of the software\u003c/li\u003e\u003cli\u003eThe databases had been running fine the entire time this tier was being created\u003c/li\u003e\u003cli\u003eThe new tier had all the latest monitoring on it and showed no signs of problems\u003c/li\u003e\u003cli\u003eThe tier it was replacing also showed no signs of problems\u003c/li\u003e\u003c/ul\u003e\u003cdiv class=\"paragraph\"\u003eAnd yet the database was dropping connections, and it was an all hands on deck situation on a Friday afternoon when most people were thinking about their weekend.\u0026#xA0; Thankfully, it was relatively easy to resolve.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eAre these types of things common with releasing new software?\u0026#xA0; No, but they happen.\u003c/div\u003e\u003ch2\u003eFour Day Work Week\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003eSo, if we\u0026#x2019;re not going to deploy on Friday afternoons, what do we do with that time?\u0026#xA0; Do we just give everyone Friday off? Less shipping means no work? One thing we can do is be protective of our employees through work life balance and reduction of stress.\u0026#xA0;\u0026#xA0;\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eI have read with great interest about \u003ca href=\"https://www.nytimes.com/2018/07/19/world/asia/four-day-workweek-new-zealand.html\"\u003e\u0026#xA0;Four day work week experiments\u003c/a\u003e.\u0026#xA0; Among the validated results of moving to a four day work week were:\u003c/div\u003e\u003cul\u003e\u003cli\u003eBoosted productivity\u003c/li\u003e\u003cli\u003e24% improvement in work life balance\u003c/li\u003e\u003c/ul\u003e\u003cdiv class=\"paragraph\"\u003eBeing protective of employees is something of which I\u0026#x2019;ve always been very supportive.\u0026#xA0; Whether it\u0026#x2019;s booting the person off Hangouts who had a blanket over their head and a hot bowl of soup in their hands, or insisting people take the day off when they\u0026#x2019;ve been up all night troubleshooting an especially difficult issue after a bad deploy.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eI realize most companies are not going to investigate a four day work week, but Friday afternoons can be used for:\u003c/div\u003e\u003cul\u003e\u003cli\u003e\u003ca href=\"http://tech.mangot.com/blog/2019/01/09/an-agile-sre-meeting-plan/\"\u003eDemos and retrospectives\u0026#xA0;\u003c/a\u003e\u003c/li\u003e\u003cli\u003eWriting documentation\u003c/li\u003e\u003cli\u003eMentoring\u003c/li\u003e\u003cli\u003e20% time or exploration time\u003c/li\u003e\u003cli\u003eWorking in the staging environment\u003c/li\u003e\u003cli\u003eBacklog grooming and prioritization\u003c/li\u003e\u003cli\u003e\u003ca href=\"https://twitter.com/davemangot/status/1184874200609083392\"\u003eTeam lunches\u003c/a\u003e!\u0026#xA0;\u0026#xA0;\u003c/li\u003e\u003c/ul\u003e\u003cdiv class=\"paragraph\"\u003eAt one of the companies I worked for, most of the Ops team would go out for lunch every Friday.\u0026#xA0; That meant lots of Friday morning deploys, and then lots of great collaboration in the afternoon.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eIf you\u0026#x2019;re working with a globally distributed team, do you do a Friday afternoon deploy on the west coast of the United States?\u0026#xA0; That\u0026#x2019;s almost the next day in most parts of Europe. Most Europeans are not excited to be called back to work late in their evening to help figure out why 8% of traffic is getting 500 errors after a Friday afternoon deploy.\u003c/div\u003e\u003ch2\u003eWeekends (all days are not the same)\u003c/h2\u003e\u003cdiv class=\"paragraph\"\u003eOne of my favorite things about working at Salesforce was the number of people who chose to wear Hawaiian shirts on Friday.\u0026#xA0; This is something I\u0026#x2019;d done on and off over the years ever since the release of \u003ca href=\"https://www.imdb.com/title/tt0151804/\"\u003eOffice Space\u003c/a\u003e as a way of recognizing the specialness of Friday.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eIf the argument is that releasing new software is the same regardless of the day, that ignores what people do on weekends.\u0026#xA0; People make plans to go away, they go camping, they go to the opera, they read books in a hammock by the shore. They go to their kids soccer games, they work in the community garden, etc.\u0026#xA0; Do some of those things happen occasionally on a Tuesday night? Sure they do. But the vast majority of weekend travel happens on the \u003cstrong\u003eweekend\u003c/strong\u003e and doing activities that can jeopardize that doesn\u0026#x2019;t show a lot of respect for your coworkers, or your employees\u0026#x2019; work/life balance.\u0026#xA0; They need to take this time to rest and recharge.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eThis is one of the reasons I always liked on-call rotations that rolled over on a Thursday.\u0026#xA0; I always wanted my teams to be able to take a Friday off to get away for a three day weekend.\u0026#xA0; The more downtime in a block, the better.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eIf the argument is that not shipping on Friday afternoon is going to hurt productivity, remember the old Agile adage \u0026#x201C;you have to go slow to go fast\u0026#x201D;.\u0026#xA0; Driving a system at 100% capacity is actually a way to reduce your throughput, not maintain it.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eDoes this mean that you should never deploy on Friday?\u0026#xA0; Of course not. That also does not mean you shouldn\u0026#x2019;t consider what you\u0026#x2019;re deploying. It may seem to make sense when someone says \u0026#x201C;We scheduled the move from Oracle to Postgres for 4 hours, so if we start at 1 p.m., we should be done in plenty of time.\u0026#x201D; \u0026#xA0; My answer to that logic is: NOOOOOOOO.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eYou should be able to, but you don\u0026#x2019;t have to deploy on Friday afternoon.\u0026#xA0; You should not be shamed for having a culture that respects Friday as being different. It \u003cem\u003eis\u003c/em\u003e different.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003eDeploy on Friday\u0026#x2019;s or Don\u0026#x2019;t.\u0026#xA0; The choice is up to you.\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003e\u003c/div\u003e\u003cdiv class=\"paragraph\"\u003e\u003c/div\u003e","mentions":[{"image":"http://logo.bigpicture.io/logo/google.com","name":"Google","collection":"companies","id":"google","manual":false},{"image":"http://logo.bigpicture.io/logo/netflix.com","name":"Netflix","collection":"companies","id":"netflix","manual":false},{"image":"http://logo.bigpicture.io/logo/oracle.com","name":"Oracle","collection":"companies","id":"oracle","manual":false},{"image":"http://logo.bigpicture.io/logo/salesforce.com","name":"Salesforce","id":"salesforce","collection":"companies","manual":false}],"owner":"FspsB7JTwpPQEzoWQv4iuBE61X83","parentCategory":"programming","profile":{"about_page_settings":{"owner":"FspsB7JTwpPQEzoWQv4iuBE61X83","createdAt":"2021-12-16T19:29:28.162Z","blocked":false,"style":{"layout":0,"skin":0,"headline_pos":"center"},"published":true,"updatedAt":"2021-12-16T19:29:28.162Z"},"adLink":"https://mangoteque.com/","adText":"Helping Ops Help Companies","avatar":"https://hackernoon.com/images/avatars/FspsB7JTwpPQEzoWQv4iuBE61X83.jpg","bio":"Dave Mangot is the principal at Mangoteque, \u0026 has successfully led DevOps \u0026 Ops transformations.","callToActions":[{"name":"Helping Ops Help Companies","icon":"far fa-analytics","active":true,"id":"ed50952f1a89c-brand","url":"https://mangoteque.com/"},{"name":"Book a call","icon":"fad fa-phone-volume","active":true,"id":"7c59d6a29efc6","url":"https://superpeer.com/hackernoon/davemangot"},{"icon":"fa fa-book","name":"Read My Stories","active":true,"id":"d9104e90328b","url":"https://hackernoon.com/u/dmangot"},{"icon":"far fa-analytics","name":"Helping Ops Help Companies","active":true,"id":"bdbcb2cfece23-brand","url":"https://mangoteque.com/"},{"icon":"fad fa-phone-volume","name":"Book a call","active":true,"id":"b31612e8be645","url":"https://superpeer.com/hackernoon/davemangot"}],"displayName":"Dave Mangot","handle":"dmangot","isTrusted":false,"allowSubscribers":true},"publishedAt":1572012370.449,"tags":["devops-principles","deployment-challenges","continuous-deployment","developer-operations","continuous-delivery","developer-productivity","release-management","hackernoon-top-story"],"title":"Deploy on Fridays, or Don't.","tldr":" There are a lot of accusations being thrown around about fear, testing, time to recover, and the like. The arguments basically boil down to some variant on \"Deploying shouldn’t be scary,\" \"You lose 20% of your productivity without deploying on Friday,\" and \"You just need more tests\" Don't let anyone make you feel bad about your Friday deploys, and don't feel bad, says Kelly Kelly. Kelly: Deploying on Friday \"Shouldn't be a source of glee and pride\"","backlinks":{"fetched":"2025-01-26T01:05:11.476Z","urls":["https://techcrunch.com/sponsor/movavi/7-social-media-strategies-that-it-companies-can-use-to-attract-new-users-in-2022/","https://builtin.com/software-engineering-perspectives/friday-deploys","https://www.podcastworld.io/topics/incident-response-lx9yb8q8","https://charity.wtf/tag/operations/","https://www.zhouexin.com/issues/27/","https://youngk.im/2019/10/28/young-reacts-46/","https://udbjorg.net/2019/11/Links","https://devopsweeklyarchive.com/461/","https://beaulebens.com/2019/11/25/deploys-to-production-its-not-actually-about-fridays/","https://sreweekly.com/sre-weekly-issue-192/","https://alper.nl/dingen/category/business/agile/page/4/","https://www.jvt.me/mf2/2020/06/2mavp/","https://learnrepo.hashnode.dev/65-stories-to-learn-about-continuous-delivery","https://learnrepo.com/67-stories-to-learn-about-devops-principles","http://gs-studio.com/news-about-it/33998-devops--27-docker-enterprise-helm-30-clusterapi----email","https://charity.wtf/category/observability/page/2/","https://beaulebens.com/2020/12/13/are-friday-code-deploys-bad-for-work-life-balance/","https://alper.nl/dingen/author/alper/page/8/","https://hnwaybackmachine.aryan.app/2019/10/27/"]},"super_category":"web3","annotations":[],"coAuthorProfiles":[],"commentsCount":0,"fromMongo":true,"relatedStories":[{"title":"FRIDAY’s TECH STORIES THAT MATTER","mainImage":"https://hackernoon.com/hn-images/1*o3qQcd0QBdIs2gRELUxamw.png","slug":"fridays-tech-stories-that-matter-bea6ece46fd1","tags":["tech-stories-that-matter","hackernoon","hackernoon-letter","equity-crowdfunding","tech-newsletters"],"excerpt":"\u003ca href=\"http://bit.ly/HNshares\" target=\"_blank\"\u003eHacker Noon\u003c/a\u003e jut passed the halfway point of \u003ca href=\"http://bit.ly/HNshares\" target=\"_blank\"\u003eour equity crowdfunding campaign\u003c/a\u003e and we’re up to \u003ca href=\"http://bit.ly/HNshares\" target=\"_blank\"\u003e$704,000+ from 597 investors\u003c/a\u003e. We’ve been hard at work. We are making great strides on our new tech stack, have been \u003ca href=\"https://hackernoonpartner.paperform.co/\" target=\"_blank\"\u003epre-selling sponsorships for Hacker Noon 2.0\u003c/a\u003e (contact partners@hackernoon.com for more info), and have some great brands to announce soon.","publishedAt":1544171890339,"profile":{"handle":"David","avatar":"https://cdn.hackernoon.com/images/N0ENUd29UdNJCFcl7GnmZHdk2fA2-9093uxj.jpeg","displayName":"David Smooke","isBrand":false},"recommended":true},{"title":"Technology Fridays: Enigma Solves The Two Biggest Issues for the Mainstream Adoption of Blockchains","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"technology-fridays-enigma-solves-the-two-biggest-issues-for-the-mainstream-adoption-of-blockchains-2655eeac8f47","tags":["blockchain","cryptocurrency","ethereum","blockchain-mainstream","blockchain-adoption"],"excerpt":"Blockchains might be the most ground-breaking technologies in several generations but its mainstream adoption remains relatively limited compared to other technology trends. The blockchain community is very aware that there are several challenges that need to be solved in order to streamline the adoption of blockchain technologies. Among those challenges, none two are more important than privacy and scalability.","publishedAt":1529065422675,"profile":{"handle":"jrodthoughts","avatar":"https://hackernoon.com/images/avatars/40ymA95p0lcRGbZcoWg7Dwyzpw62.jpg","displayName":"Jesus Rodriguez"},"recommended":true},{"title":"Technology Fridays: MLDB is the Database Every Data Scientist Dreams Of","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"technology-fridays-mldb-is-the-database-every-data-scientist-dreams-of-368b50b5a434","tags":["machine-learning","deep-learning","artificial-intelligence"],"excerpt":"Machine learning solutions in the real world are rarely just a matter of building and testing models. Managing and automating the lifecycle of machine learning models from training to optimization is, by far, the hardest problem to solve in machine learning solutions. To control the lifecycle of a model, data scientists need to be able to persist and query its state at scale. This problem might seem trivial until you consider that any average deep learning model can include hundreds of hidden layers and millions of interconnected nodes\u0026nbsp;;) Storing and accessing large computation graphs is far from trivial. Most of the times, data science teams spend a lot of time trying to adapt commodity NOSQL databases to machine learning models before arriving to the not-so-obvious conclusion: \u003cstrong\u003eMachine learning solutions need a new type of database\u003c/strong\u003e.","publishedAt":1528463766944,"profile":{"handle":"jrodthoughts","avatar":"https://hackernoon.com/images/avatars/40ymA95p0lcRGbZcoWg7Dwyzpw62.jpg","displayName":"Jesus Rodriguez"},"recommended":true},{"title":"Technology Fridays: Why Bonsai is a Great Addition to Microsoft’s AI Stack","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"technology-fridays-why-bonsai-brings-is-a-great-addition-to-microsofts-ai-stack-bf8ad832e0c8","tags":["machine-learning","artificial-intelligence","bonsai","bonsai-microsoft","microsoft"],"excerpt":"Earlier this week, Microsoft announced that is continuing its acquisition spree by \u003ca href=\"https://www.zdnet.com/article/microsoft-buys-machine-learning-startup-bonsai/\" target=\"_blank\"\u003epurchasing machine learning startup Bonsai\u003c/a\u003e. The should not be completely surprising as Microsoft had already indicated its interest in Bonsai by \u003ca href=\"http://www.businessinsider.com/microsoft-and-nea-invest-76-million-in-bonsai-ai-2017-5\" target=\"_blank\"\u003eleading a $7.6M investment round in the startup last year\u003c/a\u003e. I’ve written about Bonsai extensively before but today I would like to dig deeper into the elements that make this platform unique in the deep learning space.","publishedAt":1529669820683,"profile":{"handle":"jrodthoughts","avatar":"https://hackernoon.com/images/avatars/40ymA95p0lcRGbZcoWg7Dwyzpw62.jpg","displayName":"Jesus Rodriguez"},"recommended":true},{"title":"Technology Fridays: ZILLIQA Can Become the NodeJS of the Blockchain","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"technology-fridays-zilliqa-can-become-the-nodejs-of-the-blockchain-20235c927058","tags":["blockchain","cryptocurrency","ethereum"],"excerpt":"Not a day goes by in the blockchain space in which we don’t hear complains about the scalability and performance issues with first generation blockchain platforms such as Ethereum. Providing alternatives to Ethereum has become like its own market within the blockchain ecosystem with solutions such as EOS, Stellar or Cardano trying to become the next generation blockchain platforms. Recently, I started deep diving into the \u003ca href=\"https://www.zilliqa.com/\" target=\"_blank\"\u003eZILLIQA \u003c/a\u003eproject and have been blown away by their technological depth and mathematical rigor. Without getting into the predictions game, it is safe to assume that \u003ca href=\"https://www.zilliqa.com/\" target=\"_blank\"\u003eZILLIQA \u003c/a\u003eis one of the projects that can play a prominent role in the next generation of blockchain platforms.","publishedAt":1527857414771,"profile":{"handle":"jrodthoughts","avatar":"https://hackernoon.com/images/avatars/40ymA95p0lcRGbZcoWg7Dwyzpw62.jpg","displayName":"Jesus Rodriguez"},"recommended":true},{"title":"Can ChatGPT Outperform the Market? Week 15","mainImage":"https://substackcdn.com/image/fetch/$s_!L4Xj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3b065ff8-e538-4eb4-9bce-970d002614f6_2964x1762.png","slug":"can-chatgpt-outperform-the-market-week-15","tags":["ai","ai-controls-stock-account","can-chatgpt-outperform-market","ai-stock-portfolio","ai-outperform-the-market","ai-outperforms-the-market","chatgpt-outperform-the-market","hackernoon-top-story"],"excerpt":"Survives Friday's selloff...","publishedAt":1762786804296,"profile":{"handle":"nathanbsmith729","avatar":"https://cdn.hackernoon.com/images/e9Wv1HlyiwYBay5WMi0sPkKUt6p1-q803vmu.png","displayName":"A.I. Controls Stock Account","isBrand":false},"recommended":true},{"title":"Don’t Push to Production on Friday. Yet.\t\t","mainImage":"https://cdn.hackernoon.com/images/4heWOm7oGwNwUUrXGb8YLf1kc4P2-en031t8.jpeg","slug":"dont-push-to-production-on-friday-yet","tags":["product-development","software-development","coding-life","release-management-process","testing-in-production","production-management","bugs","software-bugs"],"excerpt":"Releasing code on Fridays can be risky. It's important to have a solid release process and quality control measures in place to minimise potential issues.","publishedAt":1727138830672,"profile":{"handle":"viktordidenchuk","avatar":"https://cdn.hackernoon.com/images/4heWOm7oGwNwUUrXGb8YLf1kc4P2-ok83aly.jpeg","displayName":"Viktor Didenchuk","isBrand":false},"recommended":true},{"title":"What I learned from Index Conference","mainImage":"https://hackernoon.com/hn-images/1*BlNn9bP0onBzx0syXJ9o6A.jpeg","slug":"what-i-learned-from-index-conference-9a34d80c8177","tags":["blockchain","tech","public-speaking","business","thoughts"],"excerpt":"Well, it’s Friday, and I consider Fridays holidays. Only 52 of them a year! I feel great. Got a good workout in the morning, and now I’m sharing some of the amazing things I learned at INDEX CONF. So, let’s get to the point. My favorite parts of \u003ca href=\"https://developer.ibm.com/indexconf/\" target=\"_blank\"\u003eIndex Conference\u003c/a\u003e. Disclaimer: this will cover public speaking, \u003ca href=\"https://hackernoon.com/tagged/blockchain\" target=\"_blank\"\u003eblockchain\u003c/a\u003e, and other random thoughts, in that order.","publishedAt":1519415980240,"profile":{"handle":"horeaporutiu","avatar":"https://cdn.hackernoon.com/avatars/robot-a3.png","displayName":"Horea Blockchain"},"recommended":true},{"title":"Why We Love Concurrency (And You Should, Too!)","mainImage":"https://hackernoon.com/fallback-feat.png","slug":"why-we-love-concurrency-and-you-should-too-c64c2d08a059","tags":["payments","golang","concurrency","mobile-money","api-gateway"],"excerpt":"Normally Fridays evenings are my off days, time to unwind and drain a few MLs of Tennessee gold. But on this night we started talking about how to handle many transactions in our system without affecting the performance. It was an interesting brain draining discussion. Our non-technical friends ended up sleeping before we reached a consensus with my friend \u003ca href=\"https://medium.com/u/65866b62f457\" target=\"_blank\"\u003eChebon Dennis\u003c/a\u003e.","publishedAt":1497779251615,"profile":{"handle":"drizzentic","avatar":"https://cdn.hackernoon.com/avatars/robot-b3.png","displayName":"Rono Derrick"},"recommended":true},{"title":"Friday Night Manifesto for Better Web Development","mainImage":"","slug":"friday-night-manifesto-for-better-web-development-11183cdd8b63","tags":["web-development","js","javascript","web","web-development-manifesto"],"excerpt":"\u003cstrong\u003eFriday Night Manifesto for Better Web Development\u003c/strong\u003e","publishedAt":1512179269949,"profile":{"handle":"jsborked","avatar":"https://cdn.hackernoon.com/avatars/robot-a2.png","displayName":"Just Chris"},"recommended":true},{"title":"Friday Still In The Reds, Crypto Markets Back To Waiting Game","mainImage":"https://hackernoon.com/hn-images/0*iFn796iCn3UNLciW","slug":"friday-still-in-the-reds-crypto-markets-back-to-waiting-game-faedab702ee8","tags":["bitcoin","market-recap","cryptocurrency","ethereum","blockchain"],"excerpt":"After dropping hard yesterday, the crypto markets have not seen any bounce yet. The Top 100 crypto board is actually a mix of red and with some greens but there are no big changes to the coins and tokens.","publishedAt":1539351397533,"profile":{"handle":"coindna_wmv6f","avatar":"https://hackernoon.com/fallback-profile.png","displayName":"CoinDNA"},"recommended":true},{"title":"Friday Vibes - Vibe Coding a Winamp Visualizer","mainImage":"https://cdn.hackernoon.com/images/l61lXLP6ciWhnSiMVtVqNgQvDJ62_am70t1hg.jpeg","slug":"friday-vibes-vibe-coding-a-winamp-visualizer","tags":["vibe-coding","claude","nodejs","visamp","ai-coding","ai-assisted-coding","claude-code","hackernoon-top-story"],"excerpt":"I vibe-coded a Winamp visualizer and ... well ... Claude has a lot to learn about human emotion.","publishedAt":1780664399926,"profile":{"handle":"jacoblandry","avatar":"https://cdn.hackernoon.com/images/l61lXLP6ciWhnSiMVtVqNgQvDJ62-jp03op5.png","displayName":"Jacob Landry"},"recommended":true},{"title":"Friday Vibes: Vibe Coding a Hacker Game","mainImage":"https://cdn.hackernoon.com/images/l61lXLP6ciWhnSiMVtVqNgQvDJ62-6w83s7s.jpeg","slug":"friday-vibes-vibe-coding-a-hacker-game","tags":["claude-code","ai","game-design","game-development","chatgpt","ai-game-development","vibe-coding","chatgpt-coding"],"excerpt":"I vibe coded a hacker game with Claude and learned some valuable lessons in both game dev and using AI effectively. ","publishedAt":1784180289761,"profile":{"handle":"jacoblandry","avatar":"https://cdn.hackernoon.com/images/l61lXLP6ciWhnSiMVtVqNgQvDJ62-jp03op5.png","displayName":"Jacob Landry"},"recommended":true}],"staticData":{"frLangTooltip":"Lisez cette histoire en Français!","about":"About","enLangTooltip":"Read this story in the original language, English!","loggedOutBookmark":"Create an account to store your bookmarks","learnMore":"Learn More","stats":"Stats","editStory":"Edit Story","audioPresented":"Audio Presented by","by":"by","audioTranslationText":null,"newStory":"New Story","loggedInBookmark":"Bookmark story","esLangTooltip":"Lee esta historia en Español!","relatedStories":"RELATED STORIES","addComment":"Add Comment","ptLangTooltip":"Leia esta história em português!","hiLangTooltip":"इस कहानी को हिंदी में पढ़ें!","comments":"Comments","removeBookmark":"Remove bookmark","commentReply":"Reply","minutes":"min","reads":"reads","trLangTooltip":"Bu hikayeyi Türkçe okuyun!","tags":"TOPICS","jaLangTooltip":"この物語を日本語で読んでください!","bnLangTooltip":"এই গল্পটি বাংলায় পড়ুন!","storyMentions":"MENTIONED IN THIS STORY","ruLangTooltip":"Прочтите эту историю на русском языке!","deLangTooltip":"Lesen Sie diese Geschichte auf Deutsch!","featuredIn":"THIS ARTICLE WAS FEATURED IN","tldrTitle":"Too Long; Didn't Read","koLangTooltip":"이 이야기를 한국어로 읽어보세요!","zhLangTooltip":"用繁體中文閱讀這個故事!","viLangTooltip":"Đọc bài viết này bằng tiếng Việt!"},"searchTopics":["fridays"],"stats":{"pageviews":13920},"socialPreviewImage":"https://hackernoon.imgix.net/drafts/9gou3zam.png","gptZeroMsg":"This story is mostly human.","audioData":[{"avatar":"https://cdn.hackernoon.com/avatars/robot-b5.png","audioPath":"audios/TjnmEeP0JCGMYYQHtk4D-en-US-Wavenet-I-MALE--c8a1b4cada081.mp3","url":"https://storage.googleapis.com/hackernoon/audios/TjnmEeP0JCGMYYQHtk4D-en-US-Wavenet-I-MALE--c8a1b4cada081.mp3","nickname":"Dr. One (en-US)"}]},"slug":"deploy-on-fridays-or-dont-qg2y32jk"},"__N_SSG":true},"page":"/[slug]","query":{"slug":"deploy-on-fridays-or-dont-qg2y32jk"},"buildId":"qqTckmfliewRaBLUp_8jP","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[77618,63213,87127,71206,89752,41116,31486,42348],"gsp":true,"scriptLoader":[]}</script><script>(function(){function c(){var b=a.contentDocument||(a.contentWindow&&a.contentWindow.document);if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a39ab83ac9305e27',t:'MTc4OTE3MjE0Ng=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script></body></html> |