Predictive framework for estimating the 3PL model parameters of mathematics test items - Montemayor, Janet Lynn S. (College of Education, University of the Philippines Diliman, 2022)
Accurately estimating the statistical parameters and the amount of information an item can provide is more essential during test construction to effectively fit the test with the examinees, thus ensuring that the instrument is more valid and reliable. There is no known established system that estimates the probable statistical attributes of test items. The primary objective of the study was to generate a predictive framework that provides statistical parameter estimates of test questions in Mathematics. The effect of item format, cognitive level, and context on the difficulty, discrimination, pseudo-chance level, and amount of information provided were analyzed using Item Response Theory. Other physical attributes that influence the statistical parameters of test questions were determined using content analysis. Mixed methods research design, particularly the embedded experimental model, was employed. A 48-item Mathematics test was administered to 2 231 Grade 7 students from seven public high schools. The three- parameter logistic (3PL) model was utilized to determine the statistical parameters of each item. Six teachers and 30 students were interviewed to validate the physical attributes identified. Mathematics questions in multiple-choice format were less difficult, less discriminating, and more prone to guessing than short-answer items. Cognitive level and context did not significantly affect the statistical parameters of items. The amount of information obtained about examinees was greater for short-answer items, items requiring higher order thinking skills, and those written in the local context. These classifications of test items provided more information about and are therefore most suitable for examinees with ability levels of average or above. Physical attributes that influence statistical parameters of test items include the form of stimuli (i.e., mathematical expression, illustration, and solution) and type of options. From the results of quantitative and qualitative data analyses, decision trees that provide estimates of difficulty, discrimination, and pseudo-chance level indices are generated. These decision trees are imperative in the field of Assessment and Evaluation as it is a potential material in appraising the statistical parameters of items during test development.