Airmen Written Testing: an Investigation in Test Legitimacy

Abstract

Becoming an airman in America requires one to pass one or more Federal Aviation Administration (FAA) knowledge tests. For some airmen tests, commercially available study guides exist which contain a question and answer database nearly identical to the database used to generate official FAA knowledge tests. This research determined whether deliberate memorization of questions and answers from a commercially available source is a common study method. This research also examined whether airmen knowledge test applicants performed more poorly on an unpublished assessment of aeronautical knowledge than they performed on their official knowledge test. It was determined that 88% of research participants agreed or strongly agreed that they attempted to memorize questions and answers from a commercially available source while studying for their FAA knowledge test. Furthermore, on average applicants performed more poorly on an unpublished assessment of aeronautical knowledge than on their official FAA knowledge test. The difference was sufficient to be considered statistically significant when all research questions were retained. When the most ambiguous research question was omitted from the calculation of results, the statistical significance was lost.