Item Response Theory
Item Response Theory (IRT) brings a revolution in education and psychological measurement, replacing the traditional classical measurement theory. After years of sound theory work, today, IRT already be widely used in various large-scale test, including known GRE, TOEFL and so on. So, everyone engaging in measurement will have to learn IRT, even it is their focus or not. Normally, even for new idea, the performance need be challenged by IRT, the dominating theory. For IRT itself, improvement and adjustment are welcome too since we know that no best but better.
posted by Shunkai Fu @ 5:30 PM
4 Comments:
" IRT rests on two basic postulates: (a) The performance of an examinee on a test item can be predicted (or explained) by a set of factors called traits, latent traits, or abilities; and (b) the relationship between examinees' item performance and the set of traits underlying item performance can be described by a monotonically increasing function called an item characteristic function or item characteristic curve (ICC)." -- Fundamentals of Item Response Theory
The most obvious feature of IRT is that it is based on ITEM, which is very different from traditonal measurement theory that is test but not item based. This characteristic will let a test built on IRT be more measureable (grid is item) and more accurate further.
"Many possible item response model exist, differentiating in the mathematical form of the item characteristic function and/or the number of parameters specified in the model. All IRT models contain one or more parameters describing the item and one or more parameters describing the examinee. The first step in any IRT application is to estimate these parameters."
It is a continuous case - continuous trait range, and continuous response probability. For measurement decision theory, both the trait category and conditional probability are discrete, so, in fact, it is a special, but easier, case of IRT.
"In IRT, the relationships among an examinee's responses to several test items are due to the traits (abilities) influencing performance on the items. After "partialling out " the abilities (i.e., conditioning on ability), the examinee's responses to the items are likely to be independent. For this reason, the assumption of local independence can also be referred to as the assumption of conditional indepedence."
In factor analysis, items are regarded as indicators, and we measure the latent abilities through examinees' observable performance on indicators. Local indepedence comes from our assumption that latent trait is the only factor that determine one examinee's response on item. In fact, there are possible other minor factors, but they are ignored if their influcence can be ignored or the whole influence (or mean) is zero (negative and positive).
"Local independence can be obtained, however, even when the data set is not unidimensional. Local independence will be obtained whenever the complete latent space has been specified; that is, when all the ability dimensions influence performance have been taken into account."
"Conversely, local independence does not hold when the complete latent space has not been specified. For example, on a mathematics test item that requires a high level of reading skill, examinees with poor reading skills will not answer the item correctly regardless of their mathematical proficiency. Hence, a dimension other than mathematical proficiency will influence performance on the item; if a unidimensional IRT model is fitted to the data, local independence will not hold. On the other hand, if all the examinees have the requisite reading skills, only mathematical proficiency will influence performance on the item and local independence will be obtained when a unidimensional model is fitted. Local independence also may not hold when a test item contains a clue to the correct answer, or provides information that is helpful in answering another item. In this case, some examinees will detect the clue and some examinees will not. The ability to detect the clue is a dimension other than the ability being tested. If a unidimensional model is fitted, local independence will not hold."
Post a Comment
<< Home