Reliability score evaluation of continuous assessment tests: A longitudinal study

Abstract

In this paper, a longitudinal study is presented about the score reliability in continuous assessment tests of the subject Network Architecture II, which is taught in the Telecommunication Technologies and Services Engineering degree, offered at Universidad Autónoma de Madrid, Spain. It is important to evaluate if scores in tests are reliable, because it shows if the phenomenon under study is measured with precision and low error, which are necessary conditions for a fair assessment. Thus, an analysis is provided about the scores obtained in the 28 continuous assessment tests taken across seven years. Correlation, corrected correlation and Cronbach’s α are used as psychometric indicators. Results show that, in general, there is a high correlation in the students’ scores among the different assessment tests every year, which let us confirm that they have been correctly set out. Additionally, the results are compared among different years, showing that they are similar in the different cohorts. However, variations have also been observed over the years, with tests with poor correlation. Finally, based on the analysed data, a novel method is proposed to early detect problems in the tests’ evaluation before the course ends, by correlating their scores in pairs. A low correlation between two tests in the same year, empirically below 0.2, would imply issues in the evaluation.