Heathrow Closure

I have seen reports of the cost to Heathrow airport for the incident being in the region of £20m - with costs to the wider industry and economy boutne by others.  Given that Heathrow has grown organically with demand being added over decades, i wonder how much of the situation arose because the problem became too big to solve. 

All supplies to the site were connected to a single feeder - my initial thought was to wonder why the load was not distributed across the avialble feeders, but if it would have taken half a day to reconfigure the supplies, with multiple system shut downs required then perhaps a reconfiguration was deeemed not feasible or not palatable.

Looking at the cost - If there are wide spread loads (critical and non-critcal) that are not generator backed across the site, then a project to cover these with generators would cost tens of millions - probably in excess of the quoted £20m loss to 'Hetahrow Airport Limited'.  Perhaps there was always an awareness of the risk but a conscious decision not to mitigate 

I wasn't suggesting that everything needed to be done in one night. There is such a thing as staging over weeks or months. 

What happens when the airport has to close because of bad weather (example Fog).

My wife experienced a situation many years ago when her fight into London was delayed because of bad weather and her plane was diverted into a airport in Scotland for a few hours.

Just think what happens when the local clocks are changed in spring and fall and the airlines are forced to adjust their schedules.

Peter Brooks

Palm Bay FL

Not sure what happened at Heathrow but for sure the emergency lights worked and presumably the control tower standby generator but security devises in passport control and computers were probably run off the airports mains power circuits which were not backed up with emergency generation.

Same with the baggage handling equipment.  Hopefully, Heathrow have by now installed these critical areas with standby plant as it certainly should not cost as much as £20 million.

Have a look in this thread for one of my previous post with attachment where I simply put

Worth a read

The cost to the Heathrow reputation is 10 times more than the stated financial cost of £20m.  It is also worth looking at who were the decision makers and who has just sold off a large share of Heathrow airport from their portfolio. 

Final report published today https://www.neso.energy/document/363891/download 

Thanks Chris.

An interesting read (with some lessons in the detail) although sadly at the highest level does sound like the common "opportunities were missed" maintenance failure.

"Using forensic analysis from both National Grid Electricity Transmission and London Fire
Brigade, this review has seen evidence that a catastrophic failure on one of SGT3’s high
voltage bushings caused a fire to ignite on the supergrid transformer. This was most likely
caused by moisture entering the bushing causing a short circuit. The electricity likely then
‘arced’ (causing sparks) which combined with air and heat to ignite the oil, resulting in a fire.
2.4 An elevated moisture reading in one of SGT3’s bushings had been detected in oil samples
taken in July 2018. According to National Grid Electricity Transmission’s relevant guidance, such
readings indicate ‘an imminent fault and that the bushing should be replaced’. While the
reading was recorded in National Grid Electricity Transmission’s online system, the mitigations
appropriate to its severity were not actioned."

So that is the nature of the network fault, but, and it is quite a big but, why, given Heathrow has 3 in-feeds, could they not switch over to the other 2 and keep going - clearly that is what National Grid expected would happen as they did not consider the transformer bushing a priority - and presumably would have had to power some kit off to change it anyway.
Mike

If you mean the airport, with three infeeds, I'd say by the read of it they could (and did within 6-7 hours from memory) but it required manual intervention. Probably this was in part due to SSEN enforcing open points to ensure that their feeders aren't paralleled through the customer's network (usually this means transferrable key interlocks etc).

If you mean the Hyde 66kV SS, the first SGT tripped and the substation did indeed recover using the third SGT which was provided as a running reserve. But the fire caused the second SGT to trip a little bit later either due to fire damage to its control kiosk or exploding bushings, and the third SGT was taken out automatically by the control scheme as it wasn't designed to operate solo.

Still does not explain why Heathrow internal network relied on manual switching to re-configure their network.Most

large hospitals have  automatic reconfiguration of their own network in event of a fault.

And sufficient on site generation to run the whole hospital.

Surely Heathrow should have the same on their site??

There's also a LinkedIn comment at https://www.linkedin.com/posts/simoncgallagher_heathrow-neso-powercut-activity-7346083051480707073-KKrU (I've stripped the tracking codes) highlighting that Heathrow should also be taking much of the blame.

[It was their battle to lose, as much by their inability to utilise the multiplicity of supply points]

There's also a LinkedIn comment at https://www.linkedin.com/posts/simoncgallagher_heathrow-neso-powercut-activity-7346083051480707073-KKrU (I've stripped the tracking codes) highlighting that Heathrow should also be taking much of the blame.

[It was their battle to lose, as much by their inability to utilise the multiplicity of supply points]

You can see if from both sides.

The network companies knowing the intricacies of their network design, point to Heathrow having supplies from several different substations, placing the responsibility on Heathrow as a very large and well resourced customer to be able to operate from any of the incoming supplies.

Heathrow's affected substation A had two separate incoming supplies from the DNO and it is the end customer, it cannot be assumed to know the intricacies of the network company's network configuration which are beyond its control. Heathrow may have assumed that two separate supplies to substation A was sufficient redundancy, that those two incoming supplies should have sufficient independence such that they should not both fail at once. Further, the restrictions on parallel interconnection between different substations imposed by the network companies meant the use of any within-site interconnectors would be highly restricted anyway.

The topic of independence is key to the whole incident, people focus on the bushing failure which was the trigger, but it was the lack of a functioning fire deluge system (due to the failure to repair the long standing fault on the fire pumps) which allowed the incident to escalate. The deluge system was there to prevent a fire on one transformer (SGT3), compromising the independence of the adjacent transformer (SGT1) and even after the initial failure of the bushing on SGT3, SGT1 continued to function for 20 minutes as the fire raged on SGT3. SGT1 finally tripping off as the fire incinerated its marshalling kiosk. At which point, SGT2 which was physically independent of SGT1 and SGT3, at the other side of the sub, but electrically dependent on SGT1 sharing the same feed also tripped off which resulted in the loss of supply.

Looking at the incident as an outsider, clearly the bushing failure was the cause of the SGT3 failure, but it was the lack of the functioning fire deluge system combined with the lack of independent 275kV supplies to SGT2 which escalated the incident to cause the loss of supply.

TurboGen said:

Heathrow may have assumed that two separate supplies to substation A was sufficient redundancy, that those two incoming supplies should have sufficient independence such that they should not both fail at once.

Maybe, but it's simply unconscionable that Heathrow Airport Holdings Ltd hadn't carried out and acted upon a basic Failure Modes and Effect Analysis for such a critical piece of national infrastructure.  Of course, it's quite possible that they did understand the risks entirely, but decided that it wasn't worth the extra cost of implementing any mitigation as they calculated that they wouldn't bear the brunt of the costs of any outage.

"Of course, it's quite possible that they did understand the risks entirely, but decided that it wasn't worth the extra cost of implementing any mitigation as they calculated that they wouldn't bear the brunt of the costs of any outage."

This was my thoughts - Further up the chain i had refernced that an article suggested the outage cost Heathrow Airport £20m.  The cost for the private network reconfiguration, additional generators, UPSs etc could easiliy run to multiples of this given the airport virtually never shuts.  I could see that someone may have thought why bother with all the disruption and compensation to airlines/retailers etc when if the 'black swan' event happened they could opportunistically and necessarily reconfigure the network and it would all be someone else's fault.

Heathrow Airport's arguement here would be that the 'black swan' event was not actually a black swan event and was readily known about by National Grid. 

I'm not sure anyone in a room full of engineers would have predcted one of the transformer would fail in such spectacular fashion and take out two other supplies at the same time.

Heathrow may have assumed that two separate supplies to substation A was sufficient redundancy, that those two incoming supplies should have sufficient independence such that they should not both fail at once.

I'm not sure I could be convinced of that logic - even if the supply to the substation was multiply redundant, the cable from the substation to the airport would still provide a single point of failure. I can't see how anyone can reasonably treat any single grid supply as 100% reliable - if it hadn't happened for this reason, it could well have happened for a dozen other perfectly justifiable reasons. They had multiple supplies, but didn't seem capable of utilizing them within a reasonable timescale (i.e. co-ordinated with UPS run times for critical equipment).

   -  Andy. 

There are two cables from the North Hyde substation to the airport - the incident report shows that Heathrow Substation A was supplied by two separate circuits, fed separately at 66kV from the North Hyde substation with each circuit having its own 66kV/33kV step-down transformer, providing two separate 33kV supplies into Substation A - see Single Line Diagram on page 13, Figure 3.

Therefore a single cable fault, or other fault, on one circuit would not be expected to interrupt all supplies to substation A.

Further, the North Hyde 66kV substation had 5 separate 66kV incoming supplies - 3 x 275kV/66kV Supergrid transformers and 2 x 66kV interconnectors to/from Iver.

Given the 5 separate supplies into North Hyde, on the face of it, it doesn't seem unreasonable to assume at least one of the circuits supplying Substation A would remain energised, or if it did fail would be restored within a short time frame. As it was, the report timeline shows it took ~10 hours to re-energise the 66kV substation.

At least one engineer knew - the one who assessed the fire risk, identifying that with the lack of physical separation, a fire on one transformer was at risk of causing damage or the fire to spread to the adjacent one, taking them both out of service, and so went to the efforts of getting a deluge fire protection system added to protect them both.

There's plenty of hindsight bias to go around;-)