JEWLScholar@MTSU Repository

APPLICATION OF THE IRT AND TRT MODELS TO A READING COMPREHENSION TEST

Show simple item record

dc.contributor.advisor Elleman, Amy
dc.contributor.author Kim, Weon Hee
dc.date.accessioned 2017-10-04T19:43:16Z
dc.date.available 2017-10-04T19:43:16Z
dc.date.issued 2017-06-19
dc.identifier.uri http://jewlscholar.mtsu.edu/xmlui/handle/mtsu/5369
dc.description.abstract The purpose of the present study is to apply the item response theory (IRT) and testlet response theory (TRT) models to a reading comprehension test. This study applied the TRT models and the traditional IRT model to a seventh-grade reading comprehension test (n = 8,815) with eight testlets. These three models were compared to determine the best model for a testlet-based reading comprehension assessment. The goodness-of-fit indices such as -2 log likelihood, Akaike information criterion, and Bayesian information criterion were utilized as model comparison indices. The standardized local dependence X2 statistic was computed for a comparison of local dependence among the three different models. Scatter plots were obtained to evaluate parameter-estimation consistency among models. Correlations and mean differences between the estimated parameters were also examined to detect and quantify the magnitude of inaccuracy due to the use of a worse-fitting model. Finally, items were evaluated based on the item parameters from the TRT models and compared to the results from the Coh Metrix.
dc.description.abstract According to the three goodness-of-fit indices, the bi-factor model was the best-fitting model and the testlet-effects model was the second best-fitting model among the three models, showing a superiority of the TRT models for the testlet-based assessments compared to the more traditional IRT model. The standardized local dependence X2 statistic revealed significant local dependencies when the traditional IRT model was applied to the data, while the application of the TRT models resolved the problematic local dependencies. Items requiring the comprehension ability of context-related vocabulary showed high local dependencies when the 3-plm was applied. Comparisons of the item and person parameters and their standard error estimates showed a slight underestimation and less stable standard errors of the item discrimination parameters under the locally dependent condition. The results of the item parameters from the TRT models supported the advantages of the TRT models in selecting good items for accurate assessments. This study provided replicable evidence that the TRT models are crucial for testlet-based reading comprehension assessments. For future study, various comprehension tests containing more questions for each testlet as well as smaller sample sizes might be useful.
dc.publisher Middle Tennessee State University
dc.subject Item Analysis
dc.subject Item Response Theory (IRT)
dc.subject Local Item Dependence
dc.subject Reading Comprehension
dc.subject Testlet-based Assessments
dc.subject Testlet Response Theory (TRT)
dc.title APPLICATION OF THE IRT AND TRT MODELS TO A READING COMPREHENSION TEST
dc.type Dissertation
dc.contributor.committeemember Jin, Ying
dc.contributor.committeemember Albakry, Mohammed
dc.thesis.degreelevel Doctoral
dc.thesis.degreegrantor Middle Tennessee State University
dc.subject.umi Educational tests & measurements
dc.subject.umi American studies
dc.subject.umi Reading instruction
dc.description.degree Ph.D.
dc.contributor.department Education en_US


Files in this item

This item appears in the following Collection(s)

Show simple item record

Search JEWLScholar@MTSU


Browse

My Account

Statistics