The TTC has been keeping two sets of statistics about the reliability of its new Bombardier-made streetcars, the Star has learned: a public version the agency says is proof the cars are “performing exceptionally well,” and a version that hasn’t been published — which indicates the vehicles experienced 72 per cent more instances of delays than publicly reported.
According to the TTC, the public numbers reflect only the number of mechanical delays for which Bombardier has been deemed responsible under the terms of its $1-billion vehicle contract. They show reliability of the cars has substantially improved in recent months, and in July hit a key target for the first time.
The unpublished figures, calculated using what the TTC calls the “legacy method,” factor in a broader range of problems affecting the operation of the new vehicles, not just those for which Bombardier is technically responsible. They indicate the cars experience far more failures than the public numbers show, and the vehicles’ reliability has remained flat in recent months.
An internal TTC document obtained by the Star shows that according to the unpublished “legacy” figures, in August the cars travelled an average of 16,400 km without a significant delay. The distance reported in the public numbers was 51,500 km.
In an emailed statement, TTC spokesperson Stuart Green said the agency keeps two sets of statistics because the numbers are used for different purposes.
The ones the TTC releases publicly and to its board in CEO Rick Leary’s monthly report are used “for holding Bombardier to account against the contract,” Green said.
The unpublished “legacy” numbers take into account all vehicle-related failures regardless of cause, and “this method is most representative of customer experience,” Green said.
He didn’t explain why the agency hasn’t published the second, worse set of reliability figures.
The “legacy” methodology is what the TTC uses in the CEO’s report to describe the reliability of its older model streetcars. The reports don’t make clear the numbers for the new cars that appear in the same document are calculated a different way.
Green acknowledged this was misleading and said future versions of the report will spell out that the reliability of the new vehicles have been calculated differently than the older ones.
“If you’re looking at it, it does look like we’re measuring them using the same standard, and we’re not. So that needs to be clarified,’ Green said.
Kaven Delarosbil, a spokesperson for Bombardier, said the reliability numbers the TTC publishes are verified according to the contract, and are an accurate record of the cars’ performance as defined by those terms.
He said the reliability of the cars is good and the company is committed to making them better.
“We received positive feedback from riders and the TTC about our cars … we are making steady progress and will continue to work with the TTC on improving the reliability,” he said.
When shown the two sets of numbers, Coun. Jim Karygiannis (Ward 22, Scarborough-Agincourt), who sits on the TTC board, said he was “very concerned” the TTC hadn’t reported the lower reliability figures.
He said he’s worried the new cars, which are intended to serve as the workhorses of Toronto’s surface rail fleet for the next 30 years, “are not up to standard.”
Karygiannis said he’s spoken to front-line TTC workers who told him the new streetcars are “breaking down” frequently, and who also raised concerns performance measures for the cars were being “fudged.”
At a public meeting of the TTC board last Tuesday, Karygiannis asked agency staff about complaints the cars weren’t as reliable as they should be.
Rich Wong, the TTC’s chief vehicle officer, answered by referring to the publicly reported reliability statistics, and made no mention of the less rosy “legacy” numbers.
Karygiannis now says he feels misled.
“I am concerned that we’re using different methods of reporting. I am concerned that the public does not know the straight facts,” he said.
Wong didn’t return a request for comment Friday.
In an emailed statement, TTC chair Coun. Jaye Robinson (Ward 15, Don Valley West) said the agency should report both sets of numbers to the board “to provide a full picture of vehicle and system performance.”
Get more of today’s top stories in your inbox
Sign up for the Star’s Morning Headlines newsletter for a briefing of the day’s big news.
“It is important for the board to be aware of any issues that may affect service and customer experience on the streetcar network,” she said.
The key statistic the TTC uses to gauge streetcar reliability is called “mean distance between failure” (MDBF), which is the average distance the cars go before experiencing an equipment failure that causes a service delay of five minutes or more.
The vehicle contract the TTC and Bombardier signed in 2009 set a MDBF target of 35,000 km. The cars were supposed to reach that figure by the time the 60th vehicle was delivered. That car arrived in January 2018, but the new fleet failed to hit the target then or in subsequent months.
That changed this summer. As Bombardier edged closer to completing its delivery of the 204-car fleet, and the TTC weighed the option of placing an order for additional streetcars with the company, the publicly reported reliability figures shot up.
They showed the cars had an MDBF of 36,500 km in July, and 51,500 km in August, the best the fleet has recorded since the early days of the order. CEO Leary cited that most recent figure at last Tuesday’s meeting as evidence the cars are “performing exceptionally well.”
However, over the same period the unpublished reliability figures didn’t improve. The “legacy” numbers showed an MDBF of just 16,400 km in August, which while much better than the early months of the year, was virtually unchanged from the mark set in May.
The unpublished “legacy” figures are consistently significantly worse than those the contractual numbers.
During the 20-month period between January 2018 and August 2019 covered by the document the Star obtained, the unpublished “legacy” figures showed the cars experienced an average of more than 51 significant delays per month, 72 per cent more than the 30 delays in the public figure.
Green said the “legacy” numbers are worse because they include delays caused by failures of parts of the cars for which Bombardier is not responsible. For instance, the equipment streetcar operators use to speak to transit control is supplied by a different company, and if it malfunctions the delay wouldn’t be included in the publicly reported contractual numbers.
According to Green, even delays that originate from a problem with equipment for which Bombardier is responsible may not be reflected in the public figures. As an example, he cited a hypothetical scenario in which a car experiences a brake failure that should take the TTC two minutes to resolve, but due to actions by transit agency employees, takes more than the five-minute contractual measure.
“There was a seven-minute delay to service because of a brake issue, however it’s a brake issue that should have been resolved in that under five-minute threshold. So we would not hold (Bombardier) responsible for that service delay” and it wouldn’t be reflected in the publicly reported reliability statistics, he said.
Green said that helps explain delay figures like those recorded in August. Internal TTC documents reviewed by the Star show that in that month the new streetcars experienced dozens of delays related to faulty brakes, malfunctioning doors, broken HVAC units, and short circuit warnings. The agency tabulated 43 significant delays during that period, but only 15 were deemed Bombardier’s responsibility and included in the version of the stats that are made public.
That lower delay figure led to the contractual number of the cars running more than 51,500 km without a failure.
Last week, the TTC reported Bombardier had delivered 175 of the new vehicles. The agency is optimistic that after years of missed deadlines, the company will deliver the rest of the 204-car order by the end of the year as promised.