In the US, the DoT has legislation backing its acquisition of airline data. We have noted several times that the US DoT data has granularity and is the most detailed source on airline operations worldwide. It is, generally speaking, useful for analysis both within and outside the US as an industry benchmark. AirInsight readers are familiar with our extensive use of DoT data in stories and analyses we publish. However, there are, on occasion, problems with the data airlines submit to the DoT.
This story is part one of what will become at least two parts regarding some failings, issues, or obfuscations of data that we have identified. As AirInsight continues to dig into the data sets, more issues may emerge – though we hope not. As we will explain, there appears to be a peculiar behavior in certain filings by airlines, and worse, an odd response from DoT. However, the basis of the post requires a brief update on an economic theory that helps explain why any such failing/error/obfuscation is bad for all concerned.
The Tragedy of the Commons
In economics, there is a fascinating theory named The Tragedy of the Commons. The key item for this story is encompassed in this statement: The tragedy of the commons is a problem in economics that occurs when individuals neglect the well-being of society in the pursuit of personal gain. Feel free to click on the link to delve further into the details.
For the purposes of this post, the key issues are:
- The US airline industry is required to file information with the DoT
- The data filed with DoT should be accurate
- Inaccurate data hurts everyone (the Commons Tragedy theory)
Inaccurate data filed with the DoT hurts the entire industry because compromised data encourages every airline to stoop to the same level of inaccuracy. Those who analyze the data, as we do, can see the fallacious numbers that appear out of line with reality. No doubt competing airlines see the same errors that we see. That begs the question of why airlines should be honest in sharing their data accurately if others are not. It also begs a crucial question of why the DoT apparently does not audit the data nor fine or sanction those who file inaccurate data.
DoT has fined airlines before. Here are a few examples:
- Southwest for weight and balance
- American for mail delivery times
- Spirit for bumping
- United for criminal fraud
- Delta Air Lines for false reporting
The Form 41 Example
There is a DoT data source named Form 41. The DoT defines it this way: “Form 41 Financial Schedule consists of financial information on large U.S. certified air carriers–includes balance sheet, cash flow, employment, income statement, fuel cost and consumption, aircraft operating expenses, and operating expenses.” Those familiar with it will testify to its wide range and depth. It is or should be a great resource for industry analysis. We have models using this dataset dating back to 2000. It is the history that allows odd data to be identified.
Within Form 41 are several tables. The one we want to focus on is named Schedule P5.2. This provides: “The table contains detailed quarterly aircraft operating expenses for large certificated U.S. air carriers. It includes information such as flying expenses (including payroll expenses and fuel costs), direct expenses for maintenance of flight equipment, equipment depreciation costs, and total operating expenses.”
We have heard chatter from several industry veterans about suspicious data in this table. Our experience with the P5.2 was satisfactory until recently when we made a discovery that was clearly inaccurate or at least highly suspect.
Case #1 – Delta Air Lines
As noted in the P5.2 language direct expenses for maintenance of flight equipment are included. We model the DoT data to watch trends and see where changes are occurring. The P5.2 maintenance cost data reported for Delta Air Lines was, to say the least, inconsistent. Below is a model for you to see this for yourself. The 2022 data is through June.
There are three pages and for the quickest grasp of what we are claiming, stay on page 2. Click the double-headed arrow on the bottom right for optimal viewing.
- At the bottom left of the chart is a “Play” button.
- Select American Airlines and then click play. Perhaps watch the model a few times to get a feel for how the data history evolves.
- Then select United Airlines and do the same – play it a few times, too.
- Finally, select Delta and do the same.
How is it that Delta’s fleet maintenance costs are in a straight line? Play it again to see what we mean. Its peer group, American and United don’t show that. The Delta numbers are, in our view, implausible.
We asked the airline and DoT about this. The airline was asked about this on August 18, August 22, and August 29. We were advised on August 23 “Still working on it” and have heard nothing since. The DoT response was: “I agree with you, but not sure anything can be done about it“.
Could we be mistaken?
Yes, of course, we can. We checked the results with DoT and they concur with what we found. The airline could have challenged our numbers but ghosted us instead.
Is this issue a concern? We think so.
Analyses are compromised by inaccurate data. If the DoT is not able to do anything about it, then frankly, the entire DoT data library is potentially compromisable. Delta might not be asked to refile accurate data and, perhaps not even be sanctioned for the data error. Why is there no fine for inaccurate data filing?
Inaccurate data serves nobody. Likes the Commons Tragedy, in the end, there are only losers. Focusing on self-interest is a dead end. While Delta can ignore this issue, the bigger problem is how other airlines respond. If DoT really cannot audit the filed data and sanction those who misfile (let’s be polite about this), then US taxpayers are funding diminishing value tools. And any public policy guided by DoT data is pretty useless.
Co-Founder AirInsight. My previous life includes stints at Shell South Africa, CIC Research, and PA Consulting. Got bitten by the aviation bug and ended up an Avgeek. Then the data bug got me, making me a curious Avgeek seeking data-driven logic. Also, I appreciate conversations with smart people from whom I learn so much. Summary: I am very fortunate to work with and converse with great people.