Crossrail: Breaking Down the Crisis

148 comments

On Thursday 26th July 2018 a meeting took place at City Hall. The subject was Crossrail – the Elizabeth line – and whether it would open on time. Up front were Simon Wright, Crossrail’s then-CEO, and Sir Terry Morgan, its Chairman. To the assembled senior officials from TfL and City Hall, their presentation would have left little room for doubt: Crossrail would not be ready to open in December.

For some in the room, this may well have come as something of a surprise. Present were Mike Brown, Commissioner of Transport for London and head of TfL (in effect, Crossrail’s lead sponsor), Mark Wild (then of TfL, now Crossrail CEO) and a selection of senior officials from City Hall – David Bellamy (Chief of Staff), Heidi Alexander (the Deputy Mayor for Transport) and, most importantly, Sadiq Khan, Mayor of London.

The presentation by Wright and Morgan laid the facts bare. Siemens (the signalling contractor on the central section and subcontractor on the train) and Bombardier (TfL’s direct contractor for the train), should have been able to begin testing on the central section by now – that is, test the trains and signalling together in increasingly complex ways to prove it would all work in passenger service. This required the ‘routeway’ – in essence the bits that Crossrail had to deliver – to be finished by now. Despite previous assurances from Crossrail, however, it was not.

The Crossrail presentation was bleak. It offered three different scenarios for when testing was likely to start. There was a 10% chance that it would begin in November, pushing trial running and actual Elizabeth line operations back to somewhere between February and May. This assumed that early testing (and software iteration) by Bombardier and Siemens was possible and went to plan.

If this didn’t happen, there was a still a 50% chance that with some parallel working the Elizabeth line could open between April and June. What was most likely though, if work on the routeway continued to be delayed,was that the line could open between May and August 2019, although that would depend on how reliable the trains and signalling proved to be.

Slide 6, presentation to Mayor by Crossrail, 26th July 2018

None of this was likely to be news to Heidi Alexander and David Bellamy. A week earlier they had been briefed by TfL on the state of the project as known – and this briefing had included almost exactly the same slides.

Slide 2, presentation to Alexander and Bellamy, 20th July 2018

Similarly, this also wasn’t news to Mike Brown. Regardless of whether TfL had been made aware of Crossrail’s build issues by the company themselves before that point, the Transport Commissioner had been tipped off to the fact that there were launch problems several months before. According to LR sources, Siemens had written to him directly in early 2018. In that letter they had warned that, from their perspective, the train programme (which they were heavily involved in signals integration for) was seriously behind schedule. That a delay of 18 months to the trains programme existed was also alluded to by Terry Morgan at the beginning of December on Radio 4’s PM programme. Siemens had warned Mike Brown that even at that stage, as things stood, it was unlikely that they would be able to commission the signalling and pass the trains through testing in time for a December opening.

Even this shouldn’t have been news to him. Crossrail themselves had issued a notice to TfL and the DfT (Crossrail’s other sponsor) as early as July 2017 warning about the challenge of software development for the train and meeting key dates.

Indeed the only person in the room for whom all of this may well have been unknown territory was the Mayor himself.

Up to this point, the Mayor had rarely interacted directly with Crossrail themselves as, rightly, it was through TfL (as a project sponsor) that he received the majority of project updates. It is therefore possible that the status of the routeway and build problems had to this point been entirely unknown to him (although the question remains as to whether it was unknown to Crossrail’s senior management themselves). It is possible, however, that he may have been aware that there were looming issues with the trains and signalling, as this was directly within the remit of TfL.

Either way, this was likely the first point at which the Mayor became fully aware that the Elizabeth line could not open in its planned configuration in December 2018.

This left one question to be answered – if it couldn’t open as planned, then could at least part of it open instead?

Unfortunately, the next part of the Crossrail presentation answered this question pretty definitively:

“2 options for partial or sectional opening in December have been analysed.” The presentation stated.“ They are judged as not feasible.”

Slide 7, presentation to Mayor by Crossrail, 26th July 2018

On the 26th July 2018, to those who were collectively charged with its construction, operation and opening, one thing was now unavoidably clear: The Elizabeth line would not open on time.

A month later in August, after the necessary mechanisms of accountability and market notification had ground through, the Mayor was finally in a position to announce the delay to both the public at large and the London Assembly. Indeed in front of the Assembly his anger was palpable as he insisted he had been unaware as to the true state of the engineering crisis that had been developing ostensibly on his watch.

Questions about just what he had known, when continued – particularly from the London Assembly who were (and clearly still feel) aggrieved that they were not informed, even unofficially, before the news was publicly announced.

This ultimately culminated in a heated confrontation at Mayor’s Question Time in October 2018 between AM Keith Prince and the Mayor, when Prince asked the Mayor if ‘the buck stops with you?’

“I’m the Chair of the TfL Board, yeah.” Khan responded.

Prince questioned whether that mean Khan had known about the delayed opening earlier than he had previously let on – suggesting the Crossrail Board meeting on 19th July as a possible reason for this.

“Towards the end of July there was concern expressed by the TfL Board – and others – about the pressures and challenge that Crossrail were talking about. Crossrail still thought they could meet the opening of the central section by December 2018.” The Mayor replied, tersely. Before explaining that towards the end of July some concern had been expressed, triggering TfL to commission a major review. It was that review, he insisted, that represented the point at which Heidi Alexander and the Mayor himself learned that the opening would definitely be delayed until August 2019 at the earliest.

“So are you saying then Mr Mayor that you were aware that there could be a delay earlier than the date you initially claimed you were aware?” Prince pushed.

“I met with – put aside the TfL Board for a second – I met with Crossrail Ltd. people.” The Mayor continued. “That includes Terry Morgan, and Heidi may have been there, at the end of July, where they talked about challenges and pressures.”

“As is often the case, there are scenarios.” He said. “What was not told to us then was that there would be a delay until August 2019”

Again, the Mayor insisted it was only after the TfL review that he became aware that a delay would be likely. Prince pushed again.

“The question is, where you aware that there was the potential for any kind of delay prior to that 29th [August] date?”

“I think it would be fair to say one of the scenarios – and there are often different scenarios – probably was delay. But nobody talked about… Crossrail Ltd were not saying that scenario was going to happen, and Crossrail Ltd were not saying that that scenario would lead to a delay from December until August.”

Prince pushed again.

“So you were not aware of any potential delay, at all?”

“One of the scenarios given was probably some delay.” Khan answered. “But that wasn’t the scenario that Crossrail envisioned would happen… Crossrail Ltd. were confident, and have been confident throughout, that the central section of Crossrail will open in December 2018. And the only time that has moved back to August 2019 is in the late August meeting I referred to.”

Prince reiterated that he wasn’t challenging the Mayor’s word that this was the first time he knew that there would be a nine month delay, but again he asked:

“[Were] you aware that there was the potential for a delay, but didn’t know how long it would be?”

“It’s even further than that.” Khan answered tersely. “Crossrail Ltd. weren’t saying there was going to be a delay, even in July. What I’m saying to you is that one of the scenarios is always – I’ve not got the document here – but would have been that there’s the possibility of there being a delay.”

“That’s not what Crossrail Ltd. were saying.” He stressed.

Even using the most charitable of interpretations, given the slides we have available for this meeting – which include the 10% chance, in bold red text, for an opening in February and the clear statement that no December opening option is available, it is almost impossible to reconcile the Mayor’s account with the factual documents that exist for that meeting.

In the course of five minutes of very specific questioning the Mayor had, at the very least ‘in spirit’, lied multiple times to the London Assembly.

In doing so he created a political crisis, which continues to unfold, of the same scale as the engineering one that City Hall, TfL and Crossrail now collectively face.

Breaking down the blame game

At this stage, it is important not to speculate too heavily about where the problems with Crossrail’s delivery lie. TfL’s own final review into the project, undertaken by KPMG (the one referenced by the Mayor) has yet to be publicly released, although they have promised they will do so.

We already know, however, that its conclusions have lead to an emergency funding package and agreement between the DfT and City Hall that will cover an estimated £1.6bn – £2bn project overrun. Most of this London will need to pay back, drawing on the existing Business Subsidy (money that it was hoped would, within that timescale, be available to Crossrail 2) as well as other financial commitments from City Hall.

We also know that this that has questioned whether even an autumn opening is optimistic, although critically we do not know how much of this is down to issues with building works or problems with the train. There is also a National Audit Office review of the project underway.

What we can say for certain, based off information already released and other sources, is that three interrelated crises have resulted in the situation London finds itself in today:

  1. Crossrail Ltd. did not complete their element of tunnel fit-out or station construction on time.
  2. The trains, contracted for by TfL from Bombardier, still require extensive testing and software updates to work with the signalling in places and may not have been capable of running an effective service even had Crossrail Ltd. completed work on time.
  3. The Mayor – and senior officials (current and former) at both TfL and Crossrail – have created a political blame crisis that, unchecked, threatens to impact on ongoing efforts to solve the first two problems.

The issues at Crossrail

It is worth reiterating that it is only with the release of reviews to come that the full scale of the issues at Crossrail – along with the causes – become clear.

Already, however, there is sufficient evidence to be able to build up a broad picture of what went wrong. On a fundamental level, this is that multiple strands of construction and fitout work, across the project became individually delayed. Each of these delays in themselves may have been recoverable, but cumulatively they eventually formed a package of recovery work that simply wasn’t practical – or logistically achievable – within either the existing project deadline or the risk period.

The key ‘tipping point’ in the chain of events leading to this situation appears to have been the November 2017 ‘explosion’ of a voltage transformer at Pudding Mill Lane substation, caused by substandard electrical work by a contractor. We covered this incident in detail in December 2017 (you can read the full piece here ), but the headline effect was that it seems to have caused a cascade of work delays and issues – including, critically, to the energisation of traction power in the central section and the start of testing.

Identifying the issues (and again, there are likely to be more that emerge) is only half the question, however. Just as importantly, what needs to be identified is why the sheer scale of the issues remained unidentified (or undisclosed) for so long.

The first clue to that can be found in another document released as part of a Mayoral Crossrail document dump on 10th December 2018. This is the Jacobs Report, an assessment of the state of Crossrail as a project after Crossrail Ltd. first revealed they were likely to run over budget in May 2018. Jacobs had also been appointed by the DfT and TfL as the Project Representative at the start of the programme in 2009 to undertake independent assurance (a role they continue to undertake).

The existence of the Jacobs report is something that provokes its own, related, immediate question – why did an independent review in May not reveal problems on such a grand scale that became so obvious to all just three months later?

The answer to that question gives us our first real clue as to what went wrong. In its terms and assumptions, the Jacobs Report makes it clear that it is a very surface-level review, due to the limited time available. As a result, much of its conclusions – broadly, that Crossrail is fine if a little over budget – have been made on a combination of substantive and subjective evidence.

That subjective evidence is based on the level of project completion that Jacobs were able to determine from questioning the senior Crossrail project executives in detail.

Handily, the report even includes a pie chart, indicating how much of the report is based on subjective opinions Crossrail’s senior leadership held about the projects progress, rather than physical evidence.

Subjective vs Supported evidence balance, Jacobs Report p26

With the benefit of hindsight, the above pie chart makes for interesting reading. It shows that almost half of the evidence Jacobs had to rely on was the informed opinion of those running the Crossrail Project. This isn’t entirely unusual for a review of this nature, but it also suggests that Crossrail fell foul of a problem that seems to repeatedly plague the railways – most recently with regards to the May Timetable rollout – the ‘Thermocline of Truth’

Back in September, after the initial delay was announced, we questioned whether this might prove to have been an issue. You can read our full piece here, but a basic description of the Thermocline of Truth from that piece is included below:

It is the principle that bad news tends to accrue at a lower management level, because no one wants to be the person who moves a project risk marker from ‘yellow’ to ‘red’ on a RAG chart.

As a result, pessimism and a belief that the project will overrun ‘bubbles up’ to a certain decision-making level but never beyond, as if hitting the thermal layer that exists in the ocean.

Eventually, the issues reach critical mass and force their way through, leaving senior management wonder why everything ‘suddenly’ went wrong, when in fact the signs that the project was troubled existed at a lower level for some time.

In the last few weeks, additional information about the current problems has emerged which suggests that not only was the above a problem, but that it was aggravated by a further issue as well – many of Crossrail’s key work packages were functioning as silos. So it wasn’t just that information wasn’t filtering up, it wasn’t filtering sideways either. This meant that where contractors were slipping, they looked to available contingency in the future as a way to get back on track so reported that they could still deliver. What they were failing to consider – and what no one in the centre of the project seems to have been set up to spot until it was too late – was that contractors were essentially ‘double-booking’ the same blocks of contingency.

Some support for this state of affairs can be seen in comments by Mike Brown to TfL’s Programme and Investment Committee this week, where it was admitted that only now was Crossrail really adopting an integrated project approach.

Where all this leaves Crossrail now – and in terms of going forward – is something we will tackle in detail in a future piece. Broadly though, the challenge for Mark Wild (as new CEO) and Crossrail Ltd. themselves will first be establishing what state of play the ‘critical path’ really is in. That will then need unpicking into something that is genuinely deliverable, with a degree of confidence – something that the KPMG summaries suggest is still unknown.

One positive, however, did emerge at the same meeting. Wild asserted that he was confident enough to announce that testing would begin on 14th January 2019. Testing will run five days in every seven, although just how long that will take to reach a point where a deliverable service exists remains to be seen.

The issues at TfL

Whereas the issues at Crossrail are becoming increasingly clear, perhaps the biggest issue with the project from a TfL perspective right now is that the exact opposite situation seems to exist: little information seems to be emerging as to what the scale of problems are with the train.
In this regard, the absence of information is as concerning as its presence. Some facts, however, are clear.

The independent warning of problems with the train programme, by Siemens, stands as a serious marker that things haven’t progressed as planned. The issuance of a sponsor’s notice by Crossrail on the state of the Bombardier train software does the same.

This is important – and largely separate to any issues that Crossrail themselves are having. Traditionally, people tend to think of signalling as something that happens at trackside and interacts with the train, but this isn’t how the most modern signalling – such as that which will operate the Elizabeth line – works. Modern signalling functionality is as much on the train as off, heavily interwoven into the train’s software. The rollout of new software on a line such as this is as big a systems integration project on the train as it is at trackside.

In this instance, the warnings received don’t necessarily suggest that something has gone wrong on a management information level to the same degree as on the ‘build side’ of the project. But they are – and should have been – a cause for concern in their own right.

There is physical evidence as well, however, that the train and signalling integration is still a work in progress, even in places where Crossrail Ltd’s own failings aren’t a factor. Most notably, on the service out to Heathrow via the Heathrow tunnels.

Back in February we looked in detail at the (then) current issues trying to get the Bombardier rolling stock working in the Heathrow tunnels (you can read the full article here). This was meant to have been completed by May 2018 when TfL took over the operation of various services to Heathrow.

Indeed the new TfL Business Plan makes a big thing of pointing out that this is one of the few ‘green ticks’ on the current Crossrail schedule. However, in doing so they are committing a sin of omission: because, at time of writing, it is still our understanding that not a single Bombardier 345 has been successfully tested in the Heathrow Tunnel.

Elizabeth line opening plan diagram, TfL Business Plan

TfL’s green tick is not incorrect, but it is profoundly misleading. It exists because of the ingenuity and forward planning of the Elizabeth line’s Operations Director Howard Smith, who foresaw this issue and created a backstop solution involving leased rolling stock. Not because the underlying issue with the track-to-train setup, and onboard software on the Bombardier 345s, is delivering to spec.

All of this paints a picture which suggests that significant issues remain to be solved with the Bombardier rolling stock and how it interfaces with signalling, both at track and train level. It is perfectly understandable and reasonable for TfL to blame some of this on Crossrail Ltd. And to a certain extent, whether the Bombardier rolling stock would have been ready to run in service in December had Crossrail Ltd. delivered (thus allowing TfL the testing time required) is entirely academic as they were not given the opportunity to prove that either way.

What isn’t academic, however, is that there have been – and still are – fundamental issues to solve with the trains before they can enter service. And, if Wild’s assertion that testing can start in January is true, then at that point any such issues which remain will be as large a problem for the delivery of the project as anything Crossrail Ltd. have done so far.

Bluntly, TfL’s current attitude that it is ‘all about the build’ is something that they themselves should be wary of, as should the Assembly, the Mayor and the wider London public. It risks missing problems – or the very least the opportunity to accept and own them – with the train that could have been diagnosed much earlier. In effect, they may be replicating the mistake that Crossrail themselves have clearly made with the build.

From a TfL organisational perspective, it should also raise a further red flag. For train issues are the one thing that TfL cannot claim to have lacked prior knowledge of. Given the warnings issued to Mike Brown directly and TfL much earlier in the project, it seems likely that the Mayor would regard any lack of communication over issues here with the same level of anger aimed at Crossrail Ltd. before. What has already cost Sir Terry Morgan a job could easily do the same for the Transport Commissioner, should the Mayor feel – correctly or incorrectly – that he has been misled.

The issues at City Hall

Whilst the clear issues at Crossrail and the (not-so-clear) issues at TfL relate to the quality of work being delivered by their respective contractors, the issues at City Hall are firmly of the Mayor’s own making.

Their origins are laid out clearly at the head of this article, and the consequences are increasingly obvious. The Mayor, and his team, seem to have reacted to the unfolding engineering crisis at Crossrail not by asserting ownership of it – contrary to the Mayor’s own words, at times, about his role as ostensible head of TfL – but by attempting to divest himself of any blame.

It’s a very ‘Westminster’ reaction, and one that harks back to Khan’s arrival in office and his reaction to criticism for the ill-advised Fare Freeze promise. This was, at least in part, caused by a mix-up between TfL and Khan, as candidate, over what was genuinely practical when it came to costs. Khan’s reaction to that crisis was to lean further into his argument that it was possible and distribute the blame for any mix up elsewhere.

Here too, a similar state of affairs seems to be unfolding. In August, the Mayor insisted that he had been as in the dark as everyone else. Despite increasing evidence to the contrary, he continued to lean into this strategy.

The net result has been the creation of an entirely unnecessary political crisis. City Hall’s efforts to defend and deflect from that crisis increasingly risk causing real damage to the very thing the Mayor needs to happen the most: the completion and opening of the Elizabeth line.

This is evidenced by the concerted effort in the media this week to confuse and obfuscate over what the Mayor knew, when. The Mayoral document dump on the 10th of December ‘helpfully’ summarises all the ways the dates and documents it includes confirm the Mayor’s interpretation of events. Read the actual documents included for the events last July, however, and they clearly contradict that narrative.

For the Mayor not to have known that Crossrail was delayed after his meeting with TfL on the 26th of July the following premises must be accepted:

  1. The combined heads of Crossrail Ltd. (its CEO and Chairman) must have failed to point out that the project was now delayed in any way – something the released slide deck makes clear they did not.
  2. The Mayor must have failed to understand the clearly marked diagrams that deck contained, which show both dates of testing, percentage-based chances of meeting those targets and text clearly stating that there were no options for delivery in December.
  3. Everyone else in the room who didn’t work for Crossrail – including the Transport Commissioner, his Deputy Mayor for Transport and the man he has now appointed to lead Crossrail – must have failed to have comment on, or explain to him, what this all meant.

It’s a combination of events that is laughably unlikely. The idea that the Mayor left that meeting without understanding that Crossrail would not open in December fails the most basic principle of the ‘duck test’ – if it looks like a duck, swims like a duck and quacks like a duck then it is probably a duck.

To use a cinematic metaphor, it would be as if Jack, whilst clinging to Rose’s flotsam raft, kept insisting to her that there was still technically a chance the Titanic could make it to New York on time.

The denial of this knowledge is remarkable precisely because it was so unnecessary. It was entirely appropriate for the Mayor to know in July that a delay was coming, and it was entirely appropriate for him not to reveal that same knowledge to others until August. By denying his own role in that process, however, the Mayor has created an environment in which the politics – and the organisational efforts necessary within TfL and a much-leaner Crossrail Ltd. necessary to deal with them – are quickly spiralling out of control.

The chances of any crisis – political or engineering – damaging Khan’s chances of re-election were always small. At worst, they may provide a brief opportunity for Shaun Bailey to remind the world that politically he still exists. Instead the Mayor’s actions threaten to create an environment in which new Crossrail Ltd. CEO Mark Wild will spend as much time answering questions about who knew what, when as he does trying to oversee completion, testing and opening of the railway. They also leave TfL in a tricky position – to take a bunker mentality themselves over any issues with the train and remain silent, or risk being on the receiving end of the Mayor’s wrath themselves.

Finding a way forward

Emerging from the triple crisis that is affecting the Crossrail Project is going to require significant effort on the part of all those involved. There is still no question that Crossrail will be delivered. There is also still no question that, in time, the project will be regarded as a significant success and a vital addition to London.

What is critical now, however, is that Crossrail, TfL and City Hall all accept their own element of blame and move on and deliver.

All three would do well to note the behaviour – at least in public – of the project’s other sponsor, the DfT. Rather than get involved in the blame game, the Department have focused on putting in place the mechanisms – including the financial deals – necessary to put the project back on track.

In addition, for Crossrail this means focusing on pulling apart the tangle of clashing deliverables and getting them out the door. For TfL, it means focusing on making sure that when that work is complete, the train is ready to go and completes testing as quickly as possible. Already, information has begun to emerge as to how both organisations are trying to do that and we will investigate those next steps in a future article.

For the Mayor though, it means accepting that he has embroiled himself in a political crisis of his own making, and that his current strategy for handling both Crossrail and TfL risks hindering the work of both. He has made few genuine, major missteps so far in his time in office, but this is one.

How he handles this crisis now and how he owns his own highly misleading, if not outright untruthful, comments may not have a material impact on his chances of getting re-elected, but they will have an enormous impact on just how quickly the Elizabeth line opens.

And in London, a Mayor’s legacy isn’t formed from what they knew, when. It is drawn in the colours that they spread across Harry Beck’s map.

Like transport and a good natter? Join us for drinks tonight near London Bridge! LR is also a community-funded website. You can also help us continue to write by backing us on Patreon or visiting our shop.

Written by John Bull
John Bull is the Editor of London Reconnections. A transport journalist and historian, his writing often focuses on the political or strategic challenges facing London's transport network and beyond.