The Effect of Missing Data Treatment on Mantel-Haenszel DIF Detection

Authors

  • Barnabas C. Emenogu Ontario Institute for Studies in Education, University of Toronto
  • Olesya Falenchuk Ontario Institute for Studies in Education, University of Toronto
  • Ruth A. Childs Ontario Institute for Studies in Education, University of Toronto

DOI:

https://doi.org/10.11575/ajer.v56i4.55429

Keywords:

differential item functioning, missing data, large-scale assessment, bias

Abstract

Most implementations of the Mantel-Haenszel differential item functioning procedure delete records with missing responses or replace missing responses with scores of 0. These treatments of missing data make strong assumptions about the causes of the missing data. Such assumptions may be particularly problematic when groups differ in their patterns of nonresponse. Using both real data from Ontario students taking the English- and French-language versions of two large-scale assessments and simulated data, this study compared the Mantel-Haenszel DIF indices produced using a total score or a proportion score as matching variables and treating missing data by listwise deletion, analysiswise deletion, or scoring missing data as incorrect. The results suggest that when many responses are missing, differential nonresponse rates may be a source of DIF.

La plupart des mises en oeuvre de la méthode du fonctionnement différentiel des items de MantelHaenszel suppriment les observations ayant des réponses qui manquent ou bien elles remplacent les réponses manquantes par un 0. Ces façons de traiter l'information manquante repose sur des hypothèses quant aux causes des lacunes, hypothèses qui pourraient s'avérer particulièrement inquiétantes quand l'absence de réponse chez les différents groupes suit un schéma distinct. En nous appuyant tant sur des données réelles d'élèves de l'Ontario ayant passé les versions en anglais et en français de deux évaluations à grande échelle, que sur des données simulées, nous avons comparé les indices de la méthode du fonctionnement différentiel des items de MantelHaenszel produits par l'emploi d'un score total ou d'un score de proportion comme variables appariées et en traitant les données manquantes de trois façons : suppression de toutes les observations comportant au moins une valeur manquante; calcul de la corrélation pour chaque couple de variables à partir des données disponibles; ou considération des données manquantes comme des erreurs. Les résultats portent à croire que lorsqu'il manque plusieurs réponses, les taux différentiels d'absence de réponse peuvent constituer une source de fonctionnement différentiel des items.

Author Biographies

Barnabas C. Emenogu, Ontario Institute for Studies in Education, University of Toronto

Barnabas Emenogu is a senior research coordinator in the Literacy and Numeracy Secretariat, Ministry of Education, Toronto. His work involves the application of multivariate and multilevel statistics in the study of large-scale assessments, large-scale educational reforms, policy analyses, and program evaluation.

Olesya Falenchuk, Ontario Institute for Studies in Education, University of Toronto

Olesya Falenchuk works as a research systems analyst in the Education Commons, a role in which she provides research design and analysis consulting to students and faculty and external clients. Olesya holds a doctorate in measurement and evaluation.

Ruth A. Childs, Ontario Institute for Studies in Education, University of Toronto

Ruth Childs is an associate professor in the Department of Human Development and Applied Psychology.

Downloads

How to Cite

Emenogu, B. C., Falenchuk, O., & Childs, R. A. (2011). The Effect of Missing Data Treatment on Mantel-Haenszel DIF Detection. Alberta Journal of Educational Research, 56(4), 459–469. https://doi.org/10.11575/ajer.v56i4.55429

Issue

Vol. 56 No. 4 (2010): Winter

Section

ARTICLES

License

UNIVERSITY OF ALBERTA COPYRIGHT LICENSE AND PUBLICATION AGREEMENT

If accepted, authors will be asked to sign a copyright agreement with the following points:

A. Where there is any inconsistency between this Copyright License and Publication Agreement and any other document or agreement in relation to the same subject matter, the terms of this Agreement shall govern.

B. This document sets out the rights you are granting in relation to publication of your article, book review, or research note entitled (the “Article”) through inclusion in the academic journal titled Alberta Journal of Educational Research (the “Journal”) published through the Faculty of Education, representing the Governors of the University of Alberta (the “Journal Editor”).

C. There will be no payment to you for this publication and grant of rights. In consideration of the agreement to publish the Article in the Journal:

1. You are warranting that:

  1. the content of the Article is your original work, and its content does not contain any material infringing the copyright of others; or, where the Article is not entirely your original work, you have obtained all necessary permissions in writing to grant the rights you are giving in this agreement;
  2. the content of the Article does not contain any material that is defamatory of, or violates the privacy rights of, or discloses the confidential information of, any other person;
  3. the Article has not been published elsewhere in whole or in part, and you will not allow publication of the Article elsewhere without the consent of the Journal Editor;
  4. the names of all co-authors and contributors to the Article are:

2. You agree to license the copyright in the Article to the Journal Editor, on a worldwide, perpetual, royalty free basis; and to the extent required by the terms of this agreement. You shall retain the right at all times to be acknowledged as the/an author of the Article.

3. You further agree that the Journal Editor has the entitlement to deal with the Article as the Journal Editor sees fit, and including in the following manner;

  1. The right to print, publish, market, communicate and distribute the Article and the Journal, in this and any subsequent editions, in all media (including electronic media), in all languages, and in all territories, ing the full term of copyright, and including any form of the Article separated from the Journal, such as in a database, abstract, offprint, translation or otherwise, and to authorize third parties to do so;
  2. The right to register copyright of the Journal;
  3. The right to edit the Article, to conform to editorial policy as the Journal Editor sees fit.

4. If any co-author or contributor to the Article does not sign this agreement, the Journal Editor reserves the right to refuse to publish the Article.