Integrating Cloze tests with Item Response Theory to validate a readability formulae

Abstract

The present study validates a non-linear readability formula (Sung et al., 2012) with readers’performance in cloze tests, which integrates the item response theory to improve the deficits of traditional cloze tests. The first study adopts a two-factor mixed design to examine the performances of readers at different levels. The second study transforms the original cloze test scores through the Rasch Model with the purpose of evaluating text difficulty and the readers’reading ability, which improves not only the scoring of cloze tests but also the validity of cloze test measuring text difficulty. The results show that as compared with traditional cloze tests, the IRT-based cloze tests are superior in discriminant validity. Furthermore, the cloze tests scores show that the Sung et al. (2012) readability formulae can appropriately reflect the difficulty levels of texts.