This is how you lie with statistics. The FAA claims that 96% of medical certification applicants are ultimately certified. Another FAA source claims 0.1% get denied. And some say hell runs a little hot.
If you answer “yes” and disclose mental health issues to your AME, you are really looking at a roughly 50-50 coin-flip that takes 1+ years to determine whether you get your license.
Let’s take a close look at the July 12, 2023 Inspector General’s (IG) report on the FAA and see how I get there.
In 2023, the IG audited the FAA and reported the findings to Congress. For their audit, the IG queried the FAA’s database and identified 23,526 applications for licenses for pilots. They selected for pilots under 65 years of age who were applying for medical certificates. The pilots had to be 1st or 2nd class (meaning commercial carriers, not hobbyists). The screening encompassed the years 2019 and 2020. Most importantly, it was further limited to those that answered “yes” to Form 8500-8, Question 18. This is the question where pilots self-identify having mental health or substance abuse issues.
Of the 23,526 applications, duplicates were identified. Because the database included two years of files, this was to be expected. Eliminating the duplicates brought the number down to 10,843 unique people.
Of these, here is how the IG broke down the mental health issue:
Note that there were 8 pilots that admitted to suicide attempts in the cohort. We will come back to these.
The IG then sampled a subset of this group. Initially they took 99 individuals and, then, for unclear reasons, reduced the sampling to 69 people. Here, they actually looked at redacted case files and did some further analysis. The 69 “charts” they looked at included all 8 of the suicide attempts. The other 61 individuals were selected to be proportionally identical to the sample above – i.e., 63% of them had just mental illness, 21% had alcohol problems, etc.
Here is what the IG found:
Only 42% of the charts examined were deferred. That means that around 58% already had a medical condition that, in some prior year, the FAA had already reviewed. As a result, the local AME was able to approve the license and the paperwork was just pushed up and into the FAA’s database.
So, what happened with the remaining 29 applicants that were deferred and evaluated by the FAA, proper? Of these, the FAA ended up denying licenses for 17 of them – roughly 60%.
Now, this is not a fair representation as it specifically includes all 8 of the pilots that made suicide attempts. These charts were disproportionately examined so that the IG could look at how the high-risk cases were handled. So, for the sake of this data analysis, let’s assume all 8 of the suicide attempts were summarily denied licenses. It is a fair assumption and the most conservative one to make.
If we exclude them, we get 9 denials among the remaining 21 applications. That is, 43% of the applicants. 43%!! Not the 0.1% the FAA advertises. 43% of the pilots that endorsed some mental health issue were initially deferred, waited some 12 to 36 months, and then were denied their licenses.
Why were the pilots denied licenses? The IG summarizes the reasons. In their analysis, they included the 8 people that made suicide attempts. We are not able to tease them out. But nonetheless, the data remains useful.
The IG then sampled a subset of this group. Initially they took 99 individuals and, then, for unclear reasons, reduced the sampling to 69 people. Here, they actually looked at redacted case files and did some further analysis. The 69 “charts” they looked at included all 8 of the suicide attempts. The other 61 individuals were selected to be proportionally identical to the sample above – i.e., 63% of them had just mental illness, 21% had alcohol problems, etc.
Here is what the IG found:
Only 42% of the charts examined were deferred. That means that around 58% already had a medical condition that, in some prior year, the FAA had already reviewed. As a result, the local AME was able to approve the license and the paperwork was just pushed up and into the FAA’s database.
So, what happened with the remaining 29 applicants that were deferred and evaluated by the FAA, proper? Of these, the FAA ended up denying licenses for 17 of them – roughly 60%.
Now, this is not a fair representation as it specifically includes all 8 of the pilots that made suicide attempts. These charts were disproportionately examined so that the IG could look at how the high-risk cases were handled. So, for the sake of this data analysis, let’s assume all 8 of the suicide attempts were summarily denied licenses. It is a fair assumption and the most conservative one to make.
If we exclude them, we get 9 denials among the remaining 21 applications. That is, 43% of the applicants. 43%!! Not the 0.1% the FAA advertises. 43% of the pilots that endorsed some mental health issue were initially deferred, waited some 12 to 36 months, and then were denied their licenses.
Why were the pilots denied licenses? The IG summarizes the reasons. In their analysis, they included the 8 people that made suicide attempts. We are not able to tease them out. But nonetheless, the data remains useful.
This is interesting. Almost half the pilots were denied licenses because they would or could not satisfy the FAA’s demands for more information. For those of you reading my blog, this should come as no surprise. We have reviewed in depth the overwhelming obstacles that the FAA requires to change a deferral to a “pass.” Roughly half the pilots failed due to these paperwork demands.
I believe this is underappreciated. It amounts to a passive and unobjectionable way for the FAA to fail out pilots. The FAA can argue the pilot did not get the medical exemption due to their own lack of effort, something that leaves the Agency clean. Indeed, I would not be surprised if they are pulled from the FAA's own stats as “withdrawn” due to lack of pilot compliance, rather than a FAA “Fail.”
Another surprising fact the IG uncovered is that roughly 20% of pilots were rejected for being on the “wrong” medication. I suppose we should have anticipated this but not at this frequency. Psychiatrists are not taught which medications the FAA approves or disapproves of. Nor are they told that polypharmacy is not allowed. So, it appears a lot of pilots are started on the “wrong” medication or are “supplemented” with add-on meds. And, if someone is responding well to a treatment, the last thing anyone wants to do is modify the prescription.
Finally, It is worth noting one other IG finding. “Of the 69 applicants, it took the Agency an average of about 138 days…to issue a medical certificate or deny the application.“ The range of days was zero to 459 days. The IG number is actually a significant underestimate of the wait time. It includes the charts with no delay – “zero days” – in processing.
“Zero days” is addressed in a footnote to the report – these are the pilots that already had a medical exemption from a prior year. They saw the AME for their routine follow-up, were found to be in compliance with the prior exemption, and were immediately given a “pass.” In the IG sample, 58% of the applications were approved by the AME and, therefore, presumably had a no delay in licensing.
Correcting for this, the algebra is:
[(0 days x 40 pilots) + (X days x 29 pilots)] / 69 pilots = 138 days.
X= 328 days.
So the average time from the AME visit to the denial/acceptance for a deferred pilot in the IG sample was around 328 days and at least one pilot in the sample was up to 459 days. But we know even this is a low estimate. This data is from 2019-20 and delays are longer now. The FAA has less staff staff and an increased number of applications.
The IG report should raise other questions. Why is this basic data so hard to find? Why are the fundamental metrics that the FAA runs on so hidden away that one needs to dive deep into an IG report to calculate them? Why is the data so divergent from the FAA's own publicized numbers?
I believe this is underappreciated. It amounts to a passive and unobjectionable way for the FAA to fail out pilots. The FAA can argue the pilot did not get the medical exemption due to their own lack of effort, something that leaves the Agency clean. Indeed, I would not be surprised if they are pulled from the FAA's own stats as “withdrawn” due to lack of pilot compliance, rather than a FAA “Fail.”
Another surprising fact the IG uncovered is that roughly 20% of pilots were rejected for being on the “wrong” medication. I suppose we should have anticipated this but not at this frequency. Psychiatrists are not taught which medications the FAA approves or disapproves of. Nor are they told that polypharmacy is not allowed. So, it appears a lot of pilots are started on the “wrong” medication or are “supplemented” with add-on meds. And, if someone is responding well to a treatment, the last thing anyone wants to do is modify the prescription.
Finally, It is worth noting one other IG finding. “Of the 69 applicants, it took the Agency an average of about 138 days…to issue a medical certificate or deny the application.“ The range of days was zero to 459 days. The IG number is actually a significant underestimate of the wait time. It includes the charts with no delay – “zero days” – in processing.
“Zero days” is addressed in a footnote to the report – these are the pilots that already had a medical exemption from a prior year. They saw the AME for their routine follow-up, were found to be in compliance with the prior exemption, and were immediately given a “pass.” In the IG sample, 58% of the applications were approved by the AME and, therefore, presumably had a no delay in licensing.
Correcting for this, the algebra is:
[(0 days x 40 pilots) + (X days x 29 pilots)] / 69 pilots = 138 days.
X= 328 days.
So the average time from the AME visit to the denial/acceptance for a deferred pilot in the IG sample was around 328 days and at least one pilot in the sample was up to 459 days. But we know even this is a low estimate. This data is from 2019-20 and delays are longer now. The FAA has less staff staff and an increased number of applications.
The IG report should raise other questions. Why is this basic data so hard to find? Why are the fundamental metrics that the FAA runs on so hidden away that one needs to dive deep into an IG report to calculate them? Why is the data so divergent from the FAA's own publicized numbers?
The FAA is working hard to get pilots to trust them and honestly divulge their health information. Frankly, the FAA's lack of transparency and the actual data suggest such trust is misplaced. The mismatch between what the FAA claims and what is reality is simply too great.
Next time, we will review recent developments, such as the FAA data mining medical records and proposed automated “advancements” to get more information, one way or another.
Next time, we will review recent developments, such as the FAA data mining medical records and proposed automated “advancements” to get more information, one way or another.
No comments:
Post a Comment