Investigation of Two Facets Design With Generalizability In Item Response Modeling

Abstract

An approach called generalizability in item response modeling (GIRM) is investigated with two facets sx(i: t) design and results are compared with results of generalizability theory in this study. In this study simulated data is used. In Generalizability Theory linear model random facets balanced bx(m: h) design are used for generating data. Generated data are differed by factors. These factors are testlet effect, testlet length and number of testlet. All generated data consist of two different universes and all universes have four different conditions. According to the results of this study the estimates of variance components obtained using GIRM approach are generally quite similar to those obtained using GT approach. Briggs and Wilson's study is supported this result. There is no difference between results of GIRM and GT but error variance could be separated from residual variance with GIRM. This study also examines the reliability of testlets under different conditions. Testlets are more reliable when person-item variance is smaller. Furthermore, when testlet effect is increased, reliability is decreased. When conditions of all universes are investigated it is concluded that it is effective to have more items to increase reliability.