Root cause
The incident was caused by an issue during a planned rotation of the access key used for the primary image storage service.
The new key was not correctly propagated to one of the services responsible for handling storage operations. As a result, requests from that service could not authenticate with the storage, causing image and document upload operations to fail.
We also identified a gap in our internal documentation related to the key rotation procedure, which contributed to the incident.
Resolution
The storage credentials were corrected and successfully propagated to all affected services. Once the configuration was updated, upload functionality returned to normal.
We have also updated the relevant internal documentation to include the previously missing configuration step.
Preventive actions
We are reviewing the credential rotation procedure to ensure that all dependent services are covered and that configuration changes are validated before and after rotation. We are also improving our internal documentation and verification procedures to reduce the risk of similar incidents in the future.
We apologize for the disruption and appreciate your patience while our team worked to resolve the issue.