Investigation of Concept Map Scores' Reliability: Example of Crossed Mixed Design in Generalizability Theory

Abstract

This study investigates the parameters obtained through the generalizability theory which measures the topic "basic concepts related with measurement and evaluation" in the measurement and evaluation course. Data for the research were collected from 91 (49 girls 42 boys) students who are studied in the Sakarya University science education department in 2011-2012 semester. The o (student), X m (item) X h (map) were crossed in the study, and due to the fact that the item facet was random but the map facet was fix, the mixed design was used. The variances of the results obtained with filling concept maps and the percentages of explaining the total variances were researched with the G study and with the C and S techniques; and the changes in the G and Phi coefficients were researched by increasing and decreasing the item numbers with the D study. With this study, we aimed to exemplify the preferability compared to the classical test theory in having reliability of the concept map by leaning on the power, suitability and flexibility of the G theory. On the other hand, this current research is different from the generalizability studies available in literature in that it includes the mixed design sample, and therefore, it is believed to be important.