Measurement Decision Theory -
Advocated by Wald (1947), first applied to measurement by Cronbach and Gleser (1957), and now widely used in engineering, agriculture, and computing, decision theory provides a simple model for the analysis of categorical data. It is most applicable in measurement when the goal is to classify examinees into one of two categories, e.g. pass/fail or master/non-master. From pilot testing, one estimates:
1) The proportion of master and non masters in the population, and 2) The conditional probabilities of examinees in each mastery state responding correctly to each item. After the test is administered, one can compute (based on the examinee’s responses and the pilot data): 1) The likelihood of an examinee’s response pattern for masters and for non-masters, and 2) The probability that the examinee is a master and the probability that the examinee is a non-master. This tutorial provides an overview of measurement decision theory. Key concepts are presented and illustrated using a binary classification (pass/fail) test and a sample three-item test. This has been added to Education and Distance Learning Resources 2005 Internet MiniGuide.