At 13:45 UTC the AWS host running the database and cache for our plugin environment failed. Auto-recovery restarted it by 14:00 UTC, but the cache service and several microservices required manual restarts. Full recovery: ~15:00 UTC.

Impact (UTC). Autosave was unavailable 13:48–13:57, then severely slow (~86 s per request vs. under 7 s normally) until ~14:40. Some plugins using external AutoSave callbacks saw saves fail. Co-editing was degraded 14:16–15:00. Editor initialization also failed for some users early on.

Data. No evidence of loss: crash recovery completed correctly, all patches that reached the database were persisted, and duplicate resubmissions were safely rejected. Contact us with a timestamp if you suspect otherwise.

Cause. Hardware failure on the underlying AWS host. It escalated because autosave retries external callbacks synchronously inside the request, exhausting the database connection pool; the cache had no auto-start on reboot; and connection pools did not self-heal. No alarm covered host status or cache availability, so detection came from user reports.

Fixes. Alerting on host status checks and cache availability; cache auto-start with a health check; connection pool recovery; moving external callbacks out of the request path; alerts on pool saturation, error rate and callback latency; a redundancy review for this environment.

We apologize for the disruption. If the impact on your users differed from the above, please contact our support team.