Incident - Unresponsive API Servers; High CPU Utilization
07/23/2020 -- 10:29 AM PT: Authentic8’s Site Reliability team received internal communication of high system load with the API servers.
07/23/2020 -- 10:48 AM PT: Site Reliability team began investigating the anomaly.
07/23/2020 -- 11:15 AM PT: Site Reliability team performed system restarts for API servers showing high connection counts. CPU utilization returned to normal once servers were restarted on a rolling order.
To help alleviate the server bottleneck condition, Authentic8 has reviewed capacity limits of the API and associated services and has taken appropriate measures to add capacity and improve data flow to prevent similar bottlenecks from occurring in the future. In addition, significant improvements to our monitoring have been implemented to provide an earlier warning signal to our technical teams.
Please contact Support if you have any additional questions and/or require further information.