It is shown that the Angoff and bookmarking cut scores are examples of true score equating that in the real world must be applied to observed scores. In the context of defining minimal competency, the percentage ``failed'' by such methods is a function of the length of the measuring instrument. It is argued that this length is largely arbitrary, being heavily influenced by practical educational constraints. Hence, there is an ambiguity or nonuniqueness about the percentage failed. An argument is advanced that the failure rate should reflect the percentage of true scores below the cut score. A modification to the cut score is derived that achieves this outcome and simultaneously removes the nonuniqueness in the percentage failed.
Angoff, W.H. (1971). Scales, norms and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.
2.
Beretvas, S.N. (2004). Comparison of bookmark difficulty locations under different item response models. Applied Psychological Measurement , 28, 25-47.
3.
Berk, R.A. (1996). Standard setting: The next generations (where few psychometricians have gone before!). Applied Measurement in Education, 9, 215-235.
4.
Biddle, R. (1993). How to set cutoff scores for knowledge tests used in promotion, training, certification and licensing. Public Personnel Management, 22, 63-70.
5.
Brennan, R.L., &Lockwood, R.E. (1980). A comparison of the Nedelsky and Angoff cutting score procedures using generalizability theory. Applied Psychological Measurement, 4, 219-240.
6.
Busch, J.C., &Jaeger, R.M. (1990). Influence of type of judge, normative information, and discussion on standards recommended for the National Teacher Examinations . Journal of Educational Measurement, 27, 145-163.
7.
Chang, L. (1999). Judgmental item analysis of the Nedelsky and Angoff standard-setting methods. Applied Measurement in Education , 12, 151-165.
8.
Cizek, G.J. (1996). Standard setting guidelines. Educational Measurement: Issues and Practice, 15, 13-21.
9.
Cross, L.H., Impara, J.C., Frary, R.B., &Jaeger, R.M. (1984). A comparison of three methods for establishing minimum standards on the National Teacher Examinations. Journal of Educational Measurement, 21, 113-130.
10.
Feldt, L.S., &Brennan, R.L. (1993). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). Phoenix, AZ: Oryx Press.
11.
Giraud, G., Impara, J.C., &Buckendahl, C. (2000). Making the cut in public schools: Alternative methods for standard setting. Educational Assessment, 6, 291-304.
12.
Goodwin, L.D. (1999). Relations between observed item difficulty levels and Angoff minimum passing levels for a group of borderline examinees. Applied Measurement in Education, 12, 13-28.
13.
Hambleton, R.K. (2001). Setting performance standards on educational assessments and criteria for evaluating the process. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 89-116). Mahwah, NJ: Lawrence Erlbaum.
14.
Harasym, P.H. (1981). A comparison of the Nedelsky and modified Angoff standard setting procedure on evaluation outcomes. Educational and Psychological Measurement, 41, 725-734.
15.
Jaeger, R.M. (1993). Certification of student competence. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 485-514). Phoenix, AZ : Oryx Press.
16.
Komaroff, E. (1997). Effect of simultaneous violations of essential tau-equivalence and uncorrelated error on coefficient alpha. Applied Psychological Measurement, 21, 337-348.
17.
Levine, R.S. (1955). Equating the score scales of alternate forms administered to samples of different ability (ETS Research Bulletin No. 23). Princeton, NJ: Educational Testing Service.
18.
Livingston, S.A., &Zieky, M.J. (1989). A comparative study of standard-setting methods . Applied Measurement in Education, 2, 121-141.
19.
Lord, F.M., &Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
20.
MacCann, R.G. (2004). Reliability as a function of the number of item options derived from the knowledge or random guessing model. Psychometrika, 69, 147-157.
21.
MacCann, R.G., &Stanley, G. (2004). Estimating the standard error of the judging in a modified-Angoff standards setting procedure. Practical Assessment, Research & Evaluation, 9(5). Retrieved September 5, 2006, from http://www.pareonline.net/getvn.asp?v=9&n=5
22.
Mitzel, H.C., Lewis, D.M., Patz, R.J., &Green, D.R. (2001). The bookmark procedure: Psychological perspectives . In G. J. Cizek (Ed.), Setting performance standards (pp. 249-281). Mahwah, NJ: Lawrence Erlbaum.
23.
Novick, M.R., &Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1-13.
24.
Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. British Journal of Mathematical and Statistical Psychology, 54, 315-323.
25.
Rudner, L.M. (2001). Computing the expected proportions of misclassified examinees. Practical Assessment, Research and Evaluation, 7(14). Retrieved July 25, 2006, from http://pareonline.net/getvn.asp?v=7&n=14
26.
Wang, N. (2003). Use of the Rasch IRT model in standard setting: An item-mapping method. Journal of Educational Measurement , 40, 231-253.
