SafetyCulture iOS and Android application global connectivity issues on 14 March 2024 (UTC)

We use third-party providers to improve the security and availability of the SafetyCulture platform. In this instance, a feature designed to protect against Distributed Denial of Service (DDoS) attacks malfunctioned and began blocking legitimate traffic from our mobile applications to the SafetyCulture platform.

iOS or Android application users would have experienced:

• Errors when attempting to sync changes.
• Errors on the Home screen or other parts of the mobile application.

Upon resolution of the issue, iOS or Android devices were able to successfully reconnect and sync pending changes.

The following is a recount of the events from the incident with timestamps in Coordinated Universal Time (UTC).

2024-03-14 00:58 - After experiencing a small increase in traffic, our third-party provider began blocking traffic from mobile users. This was an automated action based on rules within their platform. Users would have been unable to use the SafetyCulture mobile apps at this stage.

2024-03-14 02:13 - We identified the issue and evaluated methods to mitigate the impact to users.

2024-03-14 02:19 - We introduced a fix to prevent blocking of requests from our mobile users. Devices were able to connect to the SafetyCulture platform and functionality was restored.

2024-03-14 02:45 - We resolved the incident after a monitoring period.

The issue was resolved by modifying the rules within our third-party provider’s DDoS prevention feature to ensure requests from our mobile users were not incorrectly being treated as malicious.

To prevent a reoccurrence of this issue, we are working with our third-party provider to identify why legitimate traffic patterns were flagged as malicious.
We are also investigating other methods to reduce the impact to users in the event of reoccurrence, whilst maintaining the security of our platform.

In light of this incident, we will continue our ongoing efforts to understand and safeguard SafetyCulture from the ever-changing security threat landscape.

We apologize for the disruption this outage caused and thank our users for their patience while we worked to solve it. The security and reliability of our platform is our utmost priority, and we continue to make investments to provide a great experience to our users.