Biomedical subjects

David Andrich

Publications and source records attributed to David Andrich.

5 recordsLinked to original sources

PubMed2005

Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education.

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards.

Adult↗

PubMed2005

Estimating parameters in the Rasch model in the presence of null categories.

A category with a frequency of zero is called a null category. When null categories are present in polytomous responses, then in the Rasch model for such responses, the thresholds that define the categories are inestimable with the commonly used joint maximum likelihood, marginal maximum likelihood, or standard conditional maximum likelihood estimation algorithms. The reason for this situation is that in principle, these estimation algorithms involve frequencies of each category. Andrich and Luo (2003) describe an algorithm in which the thresholds are reparameterized into their principal components and in which the estimate of any threshold is based on a function of the frequencies of all categories of the item rather than the frequency of a particular category. This algorithm works in the presence of null categories. However, in situations where the null categories are at the extremes of a set of categories, the estimates themselves can become too extreme. This paper describes a procedure in which the solution algorithm described by Andrich and Luo is further adapted in the presence of null categories by using their expected frequencies. The procedure is demonstrated with simulated and real data.

Algorithms↗

PubMed2004

Controversy and the Rasch model: a characteristic of incompatible paradigms?

The development of Rasch models in educational and psychologic measurement in the 1960s coincided with the introduction of other similar models, now described as models of item response theory (IRT). The application of IRT models has now extended to other social sciences, including health. Originally, there was substantial controversy between those who saw Rasch models as simply special cases of IRT models and those who saw them as essentially different. Because these different perspectives continue to manifest themselves in various ways, it seems relevant to understand the source of the original controversy. This paper attempts to do so by invoking Kuhn's studies in the history and philosophy of science at 3 levels. First, it suggests that the 2 perspectives reflect Kuhn's concept of legitimate, incompatible paradigms in which controversy is a typical manifestation. Second, because Kuhn recognizes individual histories in the development of paradigms, Rasch's own shift in perspective is summarized. Third, because proponents of the Rasch models emphasize the models' compatibility with fundamental measurement found in physical science, an analogy is made between how Kuhn explains the role of measurement in the physical sciences and how proponents of Rasch models explain the role of these models in the social sciences. In particular, these roles cannot be gleaned from textbooks in science and statistics, respectively.

Education↗

PubMed2003

Conditional pairwise estimation in the Rasch model for ordered response categories using principal components.

In the Rasch model for items with more than two ordered response categories, the thresholds that define the successive categories are an integral part of the structure of each item in that the probability of the response in any category is a function of all thresholds, not just the thresholds between any two categories. This paper describes a method of estimation for the Rasch model that takes advantage of this structure. In particular, instead of estimating the thresholds directly, it estimates the principal components of the thresholds, from which threshold estimates are then recovered. The principal components are estimated using a pairwise maximum likelihood algorithm which specialises to the well known algorithm for dichotomous items. The method of estimation has three advantageous properties. First, by considering items in all possible pairs, sufficiency in the Rasch model is exploited with the person parameter conditioned out in estimating the item parameters, and by analogy to the pairwise algorithm for dichotomous items, the estimates appear to be consistent, though unlike for the dichotomous case, no formal proof has yet been provided. Second, the estimates of each item parameter is a function of frequencies in all categories of the item rather than just a function of frequencies of two adjacent categories. This stabilizes estimates in the presence of low frequency data. Third, the procedure accounts readily for missing data. All of these properties are important when the model is used for constructing variables from large scale data sets which must account for structurally missing data. A simulation study shows that the quality of the estimates is excellent.

Algorithms↗

PubMed2002

Understanding resistance to the data-model relationship in Rasch's paradigm: a reflection for the next generation.

The case for the Rasch models, that The comparison between two stimuli should be independent of which particular individuals were instrumental for the comparison; and vice versa (Rasch, 1961), does not depend on the models accounting for any data set. This has two distinctive consequences on the data-model relationship for the Rasch models. First, and this was recognized by Rasch, when there are deviations of one sort or another, it turns upside down the question of whether it is the model or the test that has gone wrong (Rasch, 1960). Second, because the invariance of comparisons among stimuli, and vice versa, is built into the model rather than being merely a requirement of data, further implications of this requirement can be derived mathematically. These implications, too, inevitably turn some questions, and their solutions, upside down. It is argued that having to look at these implications upside down produces substantial psychological and intellectual resistance amongst those schooled in looking at them in the traditional way. It is also argued that in turning the question upside down, Rasch had an insight that goes beyond the mathematical derivations, and that to sustain this insight requires a paradigm shift (Kuhn, 1970) in the data-model relationship. Using an illustrative example, it is suggested that to maintain this paradigm shift, even by those who research the Rasch models, requires the same uncompromising consistency and passion that Rasch displayed in maintaining faith in his insight.

Humans↗