Eptura Asset Detailed Root Cause Analysis (RCA) – Severity 2 Event Jan 5, 2024

We are profoundly grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident.

Description:

On January 5th , 2024 8:20 AM MST we received reports from several customers that Asset and WO modules were not loading the list view. This issue affected our Classic only customers, who represent less than 3% of our customer base.

Type of Event:

S2 event - Service disruption. Module views not loading for the Asset and WO modules, respectively.

Services\Modules Impacted:

Asset and Work Order Modules

Remediation:

Once the issue was identified, immediate steps were taken to fix the issue. In this case a script wasn’t deployed properly during the standard release the day prior, so the changes were reverted to resolve the issue.

Timeline:

8:20 AM – Support team reports that multiple cases related to this same issue were being reported

8:29 AM - QA confirms issue

9:10 AM – Status.io initiated

10:12 AM - Revert initiated

11:22 AM - Affected customers confirmed issue as resolved

11:35 AM - Fire Alarm closed and All Clear

Total Duration of Event:

3 Hours 15 Minutes

Root Cause Analysis:

Improper deployment during standard release the previous day of said script.

Preventative Action:

Process documentation was updated, and team members notified to prevent this in the future.