Review of Optus emergency calls outage finds process, accountability gaps in Singtel unit
[SINGAPORE] An independent review into Optus’ major September outage has uncovered gaps in process, accountability and escalation and information protocols.
Two people died as a result of the Sep 18 incident where Triple Zero calls – emergency numbers used for life-threatening situations or emergencies – were blocked due to an outage following a network upgrade.
The report, which Optus’ board released on Thursday (Dec 18), found that “challenges in Optus’ culture” had impacted decision-making and response times.
It also reinforced the need for “further work to address the issues facing emergency call services and device behaviour, and the importance of industry-wide collaboration,” said Optus, a Singtel subsidiary and Australia’s second-largest telecommunications provider.
Optus said on Tuesday that its board had accepted all 21 recommendations put forth by the report and agreed to “move swiftly with their implementation”.
The recommendations build on the multi-year transformation plans that are underway and changes introduced to address shortcomings identified during the initial response to the incident, Optus added.
The independent review conducted a comprehensive examination of the causes of the incident that led to a failure of Triple Zero services in South Australia, Western Australia, the Northern Territory and part of New South Wales.
It was led by Dr Kerry Schott, a former chair for rail-related panels and reviews. Schott previously led a short review into incident management at Sydney Trains for the New South Wales Government Cabinet Office.
What caused the outage?
The review looked into three significant questions regarding the incident, which occurred during a firewall upgrade that led to emergency Triple Zero calls failing shortly after midnight, with the services being restored only more than 14 hours later.
First, the review found that mistakes made during the planning and implementing of the firewall upgrade led to the disconnection of Triple Zero calls from the network.
At least 10 actions were not properly executed, the report said. Incorrect instructions about the upgrade’s implementation were provided, which appear to be due to a lack of attention by firewall network engineers, it added.
Second, the response was severely delayed as Optus failed to recognise the problem for around 13 hours despite receiving two earlier alerts, which did not lead to any corrective action.
Had the alerts received “more than cursory attention”, this would have allowed a correction about 30 minutes into the outage, the report said.
“The fact that alerts can be overlooked because they are related to ongoing equipment upgrade work is astounding, when the reason for those alerts may be unanticipated problems caused by that work,” the report said.
Third, only 150 of the 605 call attempts to reach emergency call services were successfully connected, with the remaining 75 per cent unconnected. This was despite how emergency calls are supposed to automatically divert to other available networks in the event of a network failure, the report said.
Shares of Singtel ended Wednesday 0.2 per cent or S$0.01 lower at S$4.55.
Decoding Asia newsletter: your guide to navigating Asia in a new global order. Sign up here to get Decoding Asia newsletter. Delivered to your inbox. Free.
