Summary of Impact: Between 05:30 UTC and 07:20 UTC on 3rd of march 2021, a subset of connection had problems establishing outgoing sockets.
Root Cause: New Api logging.
The new API-logging caused an increase of outgoing connections which at times caused issues allocating new outgoing sockets for new requests.
Mitigation: We have reverted this logging to the old logging-system and are looking at increasing the outgoing sockets.
Once the Logging was paused at 07:20 UTC, dependent services started recovering. We declared full mitigation at 08:30 UTC.
Next steps: We sincerely apologize for the impact this event had on our customers. Next steps include but are not limited to: • Increase amount of outgoing sockets.
Posted Mar 03, 2021 - 09:31 CET
Update
We are continuing to monitor for any further issues.
Posted Mar 03, 2021 - 08:41 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 03, 2021 - 08:30 CET
Investigating
We are currently investigating this issue.
Posted Mar 03, 2021 - 08:16 CET
This incident affected: Bex (Api & Integrations, Bex Backoffice).