# Increased errors for ChatGPT

- **期号**: SRE Weekly Issue #462（2025-02-02）
- **作者**: OpenAI
- **链接**: https://status.openai.com/incidents/k934n9lt39h8

## 简介

OpenAI posted this mini report on their outage on January 30.

## 正文

Resolved·Full outage

Affected components

Updates

Write-up published

Resolved

Impact

Between 3:30am PT and 7:08am PT [chatgpt.com](http://chatgpt.com) and a small number of APIs experienced elevated error rates.


Root Cause & Remediation

The root cause of the initial issue was a failure with Cosmos DB in ChatGPT. Our service provider recovered the affected DBs at 4:25am PT.

The second issue was caused by web service pods crash-looping due to their health checks failing, marking pods as unhealthy and subsequently not having enough resources available to service all the requests to the web layer. At this time we believed this was due to users retrying requests. These issues recovered after we implemented a fix to the web server’s health checking logic.

Prevention

- We will make a series of changes to the web systems to more gracefully degrade when they fail including concurrency control and better load shedding.
- A larger investment in making our DB layer more resilient to service provider outages is already underway.

We know that extended outages can impact your products, businesses, and lives. We will prioritize the preventative measures outlined above to continue improving our reliability.

Thu, Jan 30, 2025, 03:57 PM

This issue has now been resolved. Between 4:23am and 7:10am PST, customers experienced elevated errors on ChatGPT

Thu, Jan 23, 2025, 04:05 PM(`6` days earlier)

Monitoring

We are continuing to monitor for any further issues.

Thu, Jan 23, 2025, 03:11 PM(`54` minutes earlier)

A fix has been implemented and we are monitoring the results.

Thu, Jan 23, 2025, 03:09 PM

Identified

We are continuing to work on a fix for this issue.

Thu, Jan 23, 2025, 02:34 PM(`34` minutes earlier)

We have identified the root cause of this issue, and are currently working to implement a fix.

Thu, Jan 23, 2025, 01:43 PM(`51` minutes earlier)

Investigating

We are currently experiencing elevated error rates for ChatGPT. We are currently investigating.

Thu, Jan 23, 2025, 01:12 PM(`30` minutes earlier)