Resolved
The issue has been resolved. All impacted systems are now operating normally. Over the course of the incident a few thousand HTTP 500s occurred.
Investigating
What’s happening: We're still seeing some elevated error rates in the Notification#Create endpoint
Who’s affected:
Customers using the public Notifications api endpoint may experience failures when trying to create a new notification
Current Status: We are investigating the issue and will provide an update in 1 hour
Investigating
Notification#Create is experiencing elevated rates of HTTP 500 status codes.
The team is investigating a root cause and is rolling over the faulty pods.
We'll give another status update in the next 30 minutes
