Core Competencies and Credit Requirements for the Research Methods, Measurement, & Evaluation Doctoral Program
Competency 1: Research Methodology and Quantitative Expertise
(24 credits or 8 courses)
| Course Number | Description |
|---|---|
| Courses for which students may test out | |
| EPSY 5601 | Introduction to Educational Research Methods* |
| EPSY 5605 | Introduction to Quantitative Methods I* |
| EPSY 5607 | Introduction to Quantitative Methods II* |
| Required | |
| EPSY 5610 | Applied Regression Analysis |
| EPSY 5613 | Multivariate Analysis in Educational Research |
| EPSY 6601 | Methods and Techniques of Educational Research |
| EPSY 6611 | Hierarchical Linear Models |
| EPSY 6615 | Structural Equation Modeling |
| EPSY 6619 | Advanced Modeling Using Latent Variable Techniques |
| EPSY 6651 | Methods for Causal Inference from Educational Data |
| EPSY 6655 | Advanced Methods for Causal Inference from Data |
*Expected (equivalent or competency exam required to be waived) but do not count towards 24 credits in this area
Competency 2: RMME: Theories, Methods and Models
(21 credits or 7 courses)
| Course Number | Description |
|---|---|
| Required | |
| EPSY 5602 | Educational Tests and Measurements |
| EPSY 5621 | Construction of Evaluation Instruments |
| EPSY 6621 | Program Evaluation |
| EPSY 6623 | Advanced Program Evaluation |
| EPSY 6636 | Measurement Theory and Application |
| EPSY 6637 | Item Response Theory |
| EPSY 6638 | Advanced Item Response Theory |
Competency 3: Theories of Educational Psychology
(3 credits)
| Course Number | Description |
|---|---|
| Required | |
| EPSY 5510 | Learning: Its Implications for Education** |
** Students may request to have this requirement waived if they have taken a graduate-level Educational Psychology or Learning course from another university and earned a B or better.
Additional Coursework (12+ credits)
Students must take a combination of elective courses and independent study/practica which total at least 12 credits. At least 3 of these credits must be from coursework and at least 3 of these credits must be from independent study/practica.
Dissertation Research (15 credits)
Students must also register for 15 credits of dissertation research.
The anticipated total number of credits for the PhD is 75. This includes 54-57 credits of coursework, 3-6 credits of independent study and 15 credits of dissertation preparation (required by the Graduate School). For most students, the PhD degree will require four or five years of full-time study, although for students with Masters’ Degrees in highly relevant areas, such as statistics, it may be possible to complete the PhD in as little as three years of full time study. For more information about the course sequences, including sample plans of study, please consult the RMME Handbook.
Research Methods, Measurement, & Evaluation (RMME) Doctoral Degree Course Descriptions
EPSY 5195: Evaluation Practicum
EPSY 5195 is the third (and final) course in the RMME program evaluation course sequence. In this course, students transition from planning evaluations (as in EPSY 6623) to conducting evaluations. Typically, Evaluation Practicum students carry out the evaluation they developed and proposed after participating in EPSY 6623. Students successfully completing this course will be able to: (1) Apply program evaluation foundations (e.g., standards, guidelines, principles, approaches, and theories); (2) Implement technical aspects of an evaluation (e.g., framing questions, designing studies, collecting and analyzing data, reporting findings); (3) Describe the unique circumstances and settings of evaluations, including key stakeholders; (4) Develop and execute logistical components of an evaluation (e.g., developing and monitoring work plans and timelines); and (5) Exhibit interpersonal evaluation competencies.
EPSY 5601: Principles and Methods in Educational Research
EPSY 5601 provides introductory-level coverage of the theory and practice of research with primary application to K-12 settings. The goal of the course is to help students understand, evaluate, and make use of educational research and literature. Therefore, students will learn the basic concepts, procedures, and habits of mind for conducting and evaluating educational research and will become better producers and consumers of research. Students will learn to distinguish between spot good and bad science, helpful and unhelpful theory, strong and weak instruments, etc. Students successfully completing this course will be able to: (1) Describe and recognize the major types of quantitative and qualitative research; (2) Recognize the connection between research questions, research design and analysis (3) Explain measurement concepts in quantitative and qualitative research; (4) Understand descriptive and inferential statistical concepts and techniques used with quantitative data, and analysis concepts and techniques used with qualitative data; and (5) Locate, classify, synthesize, and evaluate published research. Topics for this course may include: basics of the educational research process, identification of research problems, formulation of research questions, research ethics, literature reviews, qualitative research methods, quantitative research methods, mixed methods, qualitative data collection and analysis, quantitative data collection and analysis, special topics in research methods, etc.
EPSY 5602: Educational Tests and Measurements
EPSY 5602 provides graduate students in education and other related fields with an overview of the concepts, procedures, and issues involved in testing, measurement, and assessment. Emphasis is on current developments in the field of measurement. Students successfully completing this course will be able to: (1) Apply the professional jargon of measurement, assessment, and testing; (2) Critically evaluate the quality of educational and psychological instruments; (3) Analyze reliability and validity evidence from test manuals, test reviews, and academic research; (4) Explain different measurement theories, including their underlying assumptions; (5) Explain which types of validity evidence are required for different test score interpretations; (6) Describe the process of test design and validation; and (7) Accurately interpret test scores and other psychometric indicators and estimates. Topics for this course may include: assessment purposes & measure types & formats; the process of test construction; cognitive items & performance assessments; non-cognitive/affective items, including item scaling and item response sets; Classical Test Theory; standard error of measurement; forms & types of reliability; generalizability theory; traditional forms of validity; current conceptualization of validity; types of validity evidence & applications; classification accuracy, sensitivity, & specificity; standard setting; standardized scores & score interpretations; norming tests; linking & equating; item difficulty & discrimination; bias & fairness in testing; ethical guidelines for test users; questionable measurement practices, etc.
EPSY 5605: Quantitative Methods in Research I
EPSY 5605 is the first course in the RMME quantitative methods course sequence. EPSY 5605 introduces foundational concepts/skills in quantitative methods, with illustrations and examples from educational research. Students are routinely encouraged to share applications of course content from their “home” disciplines or personal research experiences to illustrate the cross-disciplinary nature of quantitative methods. Students successfully completing this course will be able to: (1) Use statistical terminology appropriately to describe general principles of statistical analysis and inference; (2) Construct tabular and graphical displays to summarize a given set of data; (3) Identify and calculate appropriate descriptive statistics for variables in a given data set; (4) Construct scatter plots, calculate measures of bivariate relationship, and perform simple linear regressions for variables in a given data set; (5) Identify the parameters of interest in a given research context and select an appropriate statistical procedure for answering the research questions; (6) Construct and correctly interpret confidence intervals for the mean, proportion, correlation, difference between means, and difference between proportions; (7) State and test hypotheses about the mean, proportion, correlation, difference between means, and difference between proportions; (8) State and test hypotheses about bivariate relationships among variables using simple linear regression procedures and chi-square tests of association; and (9) Draw clear and correctly stated conclusions with respect to research questions of interest based on statistical analyses of a given set of data. Topics for this course may include: methods for displaying and summarizing data (frequency distributions, graphical displays, measures of central tendency and variability, measures of relative standing / percentiles, correlation, linear regression); probability and statistical inference (probability distributions, the normal distribution, sampling distributions for the mean and proportion); inference about a single population parameter (confidence interval for the mean and proportion, testing hypotheses about the mean and proportion); inference about the difference between two population parameters (inference about the difference between means / t-test for dependent and independent samples, inference about the difference between proportions / z-test for dependent and independent samples); inference about relationships (inference about correlations and regression slopes, chi-squared test of association); one-way analysis of variance, etc.
EPSY 5607: Quantitative Methods in Research II
EPSY 5607 is the second course in the RMME quantitative methods course sequence. EPSY5607 provides students with an understanding of the models and analysis procedures necessary for carrying out quantitative research projects, with restriction to univariate procedures. Students successfully completing this course will be able to: (1) Select appropriate analysis of variance (ANOVA) and linear regression models and data analysis procedures for data from a given research design; (2) Perform univariate one-way, factorial, randomized-blocks, and repeated-measures ANOVAs; (3) Interpret the results of univariate one-way, factorial, randomized-blocks, and repeated-measures ANOVAs; (4) Test specified contrasts using Scheffé, Bonferroni, and Tukey multiple-comparison procedures; (5) Fit linear regression models with continuous predictors; (6) Perform a regression analysis with appropriately constructed dummy variables for categorical predictors; (7) Interpret the results of linear regression analyses with continuous and/or categorical predictors, including subset tests, regression coefficient values, t-statistics, tolerance values, and part and partial correlations for predictors in the model; (8) Perform and interpret the results of an ANCOVA analysis including checks of test assumptions, coefficient interpretations, and post-hoc tests; and (9) Understand how to run regression diagnostics to test model assumptions and identify outliers and other unusual observations. Topics for this course may include: one-way ANOVA; multiple comparisons; two-way ANOVA; randomized-blocks and repeated-measures designs; linear regression with continuous predictors; linear regression with categorical predictors; regression diagnostics, etc.
EPSY 5610: Applied Regression
EPSY 5610 presents an in-depth study of linear regression with one or more predictors. Students successfully completing this course will be able to: (1) Estimate linear regression models and interpret model parameters; (2) Make appropriate inferences about population parameters based on linear models; (3) Evaluate model fit and the validity of inferences using diagnostic criteria; (4) Conduct analyses and graphically represent results; and (5) Communicate results of regression analyses for informed audiences. Topics for this course may include: purposes of regression, variance/covariance, plotting data, one-predictor regression models, least squares estimators, residuals, centering of predictors, ANOVA decomposition, categorical predictors, model diagnostics, residual plots, inference in regression, sampling distributions, inference for slopes / points on line / new observations, statistical power, overestimating effects, reproducible science, logistic regression, categorical outcomes, matrix representation of regression, multiple predictors, Venn diagrams for variance, general linear F-test, partial correlation, adjusted R-squared, hierarchical regression, multicollinearity, partial plots, hidden outliers, leverage, influential points and residuals, DFBETAS, interactions, polynomial regression, model building, stepwise regression, missing data, causal inference, mediation, suppression effects, etc.
EPSY5613: Multivariate Analysis in Educational Research
EPSY 5613 extends and expands the content of EPSY 5607 by familiarizing students in the field of education and the social sciences with techniques for analyzing multivariate data.Topics for this course may include: a review of Matrix Algebra (matrix operations, scalar functions of matrices like the trace and determinant, inverse, eigenvalues and eigenvectors); Distribution Theory (joint distributions, conditional distributions, marginal distributions, algebra of expectations, mean vectors, variance-covariance matrices, correlation matrices, variance of linear combinations); Univariate General Linear Model (matrix formulation, the general linear hypothesis, experimental design models); Multivariate General Linear Models (multivariate regression, experimental design models like multivariate analysis of variance [MANOVA] and multivariate analysis of covariance [MANCOVA], repeated-measures designs including multivariate and mixed models analyses, discriminant analysis, canonical correlations); and the Structure of Multivariate Data (principal components, factor analysis, path analysis); etc.
EPSY5621: Construction of Evaluation Instruments
EPSY 5621 teaches students how to develop and examine validation evidence for attitude, evaluation, and other affective instruments. Students successfully completing this course will be able to: (1) Critically read research literature on test construction and instrument validation; (2) Develop and pilot test an affective instrument, and prepare an instrument validation report; (3) Independently conduct factor analyses, reliability analyses, and simple item analyses; and (4) Construct a validity argument for a self-report instrument. Topics for this course may include: instrument and item development, research ethics surrounding instrument design, unidimensionality, scaling procedures, reliability, content validity, unidimensionality, construct validity, reliability/validity tradeoffs, “factorial” validity, issues surrounding reliability, exploratory factor analyses, confirmatory factor analyses, and internal consistency reliability analyses, current issues in the field of instrument development, etc.
EPSY5641: Research Design and Measurement for Data Science
EPSY 5641 presents research design and measurement issues relevant to data science. Students successfully completing this course will be able to: (1) Apply basic measurement principles (i.e. reliability and validity) to problems of practice within data science, including the evaluation and/or development of surveys; (2) Design experiments to test specific hypotheses within a data science framework; (3) Determine whether causal inferences are warranted under a variety of design and analytic conditions common in data science; (4) Critique experiments by identifying threats to validity; (5) Design studies to minimize threats to internal, external, and construct validity; (6) Explain the importance of open and reproducible science to the practice of research design and measurement. Topics for this course may include: statistical validity; open science; strategies for increasing scientific reproducibility; measurement in data science (related to survey design, measurement error, measurement validity as an evidentiary argument, and human-machine reliability); causal inference (including theories of causal inference, threats to internal and external validity, and experimental designs for causal inference); investigation of the challenge and danger of differential threats to validity (e.g., gender and racial bias), etc.
EPSY5643: Text Analytics
EPSY 5643 explores methods for analyzing text data, which can generate valuable insights into psychological, cognitive, and social processes in fields like education, public policy, and psychology. This course serves as an applied introduction to common methods and toolkits for text analysis in the Python ecosystem. It covers three text analytics methods in depth: lexical-based methods (a.k.a. dictionaries), text classification, and topic modeling. Students successfully completing this course will be able to: (1) Explain the text analytic approach and discuss the validity of findings based on presented data and analyses from academic papers applying each of these three data analysis methods; (2) Complete many, if not most, of the tasks necessary to analyze text data in Python using each of these three text analytic approaches; (3) Propose (in writing) a text analytics research project with an eye toward increasing the validity of findings given research constraints; and (4) Effectively use resources like Python, Jupyter Notebook, and Python libraries (such as NLTK, Pandas, Sci-kit Learn, and Gensim) for text analysis. Topics for this course may include: introduction to text analytics, text analytics and Python, lexical-based methods (a.k.a. dictionaries) for text analysis, implementation of lexical-based methods (a.k.a. dictionaries) in Python, text classification methods for text analysis, implementation of text classification methods in Python, topic modeling for text analysis, implementation of topic modeling approaches in Python, etc.
EPSY 6601: Methods and Techniques of Educational Research
EPSY 6601 offers an advanced survey of the principal methods employed in the investigation of educational problems, including problem formulation, stating hypotheses, sampling, instrument design, types of research methods and design principles. Students successfully completing this course will be able to: (1) Identify theory, concepts, and terminology pertinent to conducting quantitative educational research; (2) Describe a variety of experimental and quasi-experimental research designs; (3) Identify threats to validity for each of these research designs and propose strategies to minimize those threats; (4) Define a research problem of interest and generate appropriate research questions/hypotheses; (5) Select a quantitative research design to examine specific research questions/hypotheses and evaluate the adequacy of the chosen design; (6) Apply guidelines required for the protection of human subjects in research with identification of the role of the IRB in the protection of human subjects; (7) Evaluate and critique the results of research studies conducted by other researchers within the field of education; (8) Describe the principles of open science. Topics for this course may include: research questions and ethics, introduction to causality and the validity typology, statistical conclusion validity (including confidence intervals, effect sizes, and power), internal validity and randomized experiments, external validity, construct validity, quasi-experimental research designs, matching and propensity scores, regression discontinuity and interrupted time series research designs, mediation and moderation, practical issues in research (open science), etc.
EPSY6611: Hierarchical Linear Models
EPSY 6611 focuses on the analysis of organizational and longitudinal data using multilevel modeling. This course is intended to take students through the steps in a multilevel analysis: deciding which type of model is appropriate, setting up the data file and coding variables, fitting models, evaluating fixed and random effects and/or interpreting covariance structures, predicting between- and within-person variation using covariates, interpreting and displaying empirical findings. By the end of the semester, students should have acquired enough background knowledge (i.e., theory) and technical expertise to apply these methods in practice. More specifically, students successfully completing this course will be able to: (1) Apply hierarchical/multilevel/mixed models in a variety of settings, including for analyzing organizational data and longitudinal data; (2) Conduct and interpret multilevel analyses appropriately; and (3) Use R to conduct multilevel analyses, plot findings, save residuals, and report summary information. Topics for this course may include: the logic of multilevel modeling, organizational multilevel models, building two-level multilevel models, using R to fit multilevel models, centering, model fit, estimation, reliability, proportion of variance explained, hypothesis testing, predicted values, multilevel model assumptions, residuals, longitudinal modeling, time-structured vs. time-unstructured data, person-period datasets, time-invariant predictors, time-varying covariates, interactions among time-varying covariates, modeling discontinuity, piecewise multilevel models, polynomial models, intensive longitudinal models, error covariance structures, multivariate multilevel modeling, heterogeneous variances, mixed effects location scale models, three-level multilevel models, power analysis in multilevel modeling,research design and sample size issues, etc.
EPSY6615: Structural Equation Modeling
EPSY 6615 introduces students to structural equation modeling through: linear models with only observed variables (path analysis), latent variable models without predictive paths (confirmatory factor analysis, CFA), and latent variable models with predictive paths (structural equation modeling, SEM). Students learn to develop, modify, and interpret a variety of structural equation models that are commonly used in social science research. The focus of the course is on conceptual understanding, application, and interpretation of structural equation models rather than on the actual mathematical computations involved in model estimation. Most examples and applications come from the social sciences, especially education and psychology. Students successfully completing this course will be able to: (1) Recognize and distinguish between path models, confirmatory factor analysis models, and structural equation models; (2) Correctly select an appropriate analytic model, based on the type of research design imposed and the data collected from it; (3) Use Mplus to estimate structural equation models according to available data (including recognition and adjustment of default settings imposed by the Mplus software); (4) Interpret parameter estimates from a wide variety of structural equation models; and (5) Evaluate structural equation models in terms of model fit, outcome variance explained, and other pertinent model diagnostic criteria. Topics for this course may include: path analysis, model identification, model estimation in Mplus, confirmatory factor analysis with single and multiple latent variables, factor scaling, comparison of nested models with appropriate criteria, model fit, hybrid models (combining measurement and structural model structures), respecification of latent variable models, mediation, bootstrapping, longitudinal models (e.g., autoregressive models, growth curve models, etc.), mean structures, multiple groups confirmatory factor analysis, multiple groups structural equation modeling, moderation, advanced measurement topics (e.g., multi-trait, multi-method matrices [MTMM], latent variable reliability, higher-order confirmatory factor analysis, bi-factor models, etc.),non-recursive models, etc.
EPSY6619: Advanced Modeling Using Latent Variable Techniques
EPSY 6619 offers a deep dive into the estimation, application, and interpretation of several advanced latent variable modeling techniques, including multilevel confirmatory factor analysis, multilevel structural equation modeling, latent class and latent profile analysis, factor mixture modeling, growth mixture modeling, and Bayesian structural equation modeling. This course also covers common issues and advanced topics related to analysis of real data using latent variable (and multilevel) modeling, as well as Monte Carlo simulation studies using Mplus. Students successfully completing this course will be able to: (1) Independently utilize Mplus for a variety of statistical modeling purposes, especially in the area of latent variable modeling; (2) Run basic Monte Carlo simulation studies using Mplus; (3) Competently utilize advanced modeling approaches, including mixture modeling, multilevel structural equation modeling, Bayesian structural equation modeling, and other emerging latent variable modeling techniques to conduct analyses and to correctly interpret the results of such analyses; (4) Extend, refine, deepen, and consolidate knowledge of structural equation modeling and related latent variable modeling techniques; (5) Read current research within the area of latent variable modeling and incorporate the latest research in the area into their modeling practices. Topics for this course may include: predicting categorical variables (multinomial/ordinal outcomes), mixture modeling (including covariates and distal outcomes), latent class analysis, latent profile analysis, latent transition analysis (including random-intercept latent transition analysis), growth mixture modeling, simulating datasets in Mplus, advanced longitudinal modeling (including dynamic structural equation modeling, latent change score analysis, random-intercepts cross-lagged panel models, and latent state-trait models), bayesian structural equation modeling, multilevel modeling in a latent variable framework, multilevel confirmatory factor analysis, multilevel structural equation modeling, approaches to missing data in advanced modeling applications, etc.
EPSY 6621: Program Evaluation
EPSY 6621 is the first course in the RMME program evaluation course sequence. This course provides students with a basic understanding of evaluation. Students learn about fundamental evaluation topics and concepts in the areas of practice, theory, methods, and profession. Students successfully completing this course will be able to: (1) Explain the history and influences of evaluation in society; (2) Compare and contrast evaluation’s purposes and evaluators’ roles and activities; (3) Discuss theory, concepts, and vocabulary used in program evaluation; (4) Compare and contrast different theories pertinent to conducting program evaluation; (5) Apply basic evaluation practice methods; (6) Debate recent trends influencing the practice of evaluation. Topics for this course may include: ethics and integrity, defining evaluation, evaluation exemplars, history of program evaluation, types of theories used in evaluation practice, evaluation practice in action, evaluation theory (methods theorists, valuing theorists, use theorists), evaluator competencies, current debates in evaluation, etc.
EPSY 6623: Advanced Program Evaluation
EPSY 6623 is the second course in the RMME program evaluation course sequence. In this course, students will build upon the foundational knowledge and skills on ESPY 6621 and transition from learning about evaluation to planning evaluations. Students gain deeper understanding in four areas related to evaluation: program context, evaluators, evaluation methods, and research on evaluation. Students successfully completing this course will be able to: (1) Develop evaluations that maximize the likelihood of their use; (2) Choose appropriate evaluation designs for various programs, processes, systems, organizations, or products; (3) Develop a program logic model or program theory; (4) Choose the most appropriate data collection and analysis methods for specific evaluation studies; (5) Develop an evaluation plan outlining a major evaluation study; (6) Identify the political and contextual factors that affect the practice of evaluation; (7) Determine effective communication and reporting methods for disseminating evaluation information; and (8) Apply the standards and ethical practices of evaluators. Topics for this course may include: evaluator roles & competencies, politics & ethics of evaluation practice, focusing the evaluation & describing the evaluand, culture & evaluation, evaluation designs (choosing data collection methods), planning / implementing / managing / budgeting the evaluation, analyzing evaluation data, and communicating & reporting evaluation processes and findings, etc.
EPSY6636: Measurement Theory and Applications
EPSY 6636 broadens and deepens students’ conceptual understanding of the fields of educational and psychological measurement with discussion of the history of educational and psychological measurement, reliability, validity, fairness in educational and psychological assessment, etc. Students successfully completing this course will be able to: (1) Understand and explain the fundamental principles and concepts of educational and psychological measurement to various audiences; (2) Make meaningful contributions to discussions of educational and psychological measurement issues; (3) Synthesize, compare, and contrast perspectives posited by theorists in educational and psychological measurement on fundamental topics such as reliability and validity; (4) Forge connections between the principles of educational and psychological measurement and personal research experiences/interests; (5) Implement data analyses related to specific measurement techniques (e.g. reliability estimation, classification, measurement invariance investigation, IRT modeling, etc.); (6) Identify/explain current issues in, and future directions for, the fields of educational and psychological measurement; and (7) Conduct a thorough analysis of a selected measurement topic and produce a comprehensive (yet comprehensible) foundational introduction to that topic (as either a written work product or in the form of a teaching talk aimed at engaging and helping fellow students gain perspective in your specific measurement area). Topics for this course may include: defining measurement, classical test theory, reliability (e.g., alpha, omega, etc.), dimensionality and internal consistency, measurement error, generalizability theory, standards setting, fairness in testing, tests and social responsibility, scaling, validity, validity and fairness, differential validity, validity arguments, reliability and diagnostic classifications (e.g., sensitivity, specificity, signal detection theory, receiver operating characteristic [ROC] curves), classification accuracy, assessment engineering, evidence-centered design frameworks and applications, item response models, latent class analysis, mixture models, response time and process models, innovative models for scoring essays and performance tasks, network models, current issues and future directions in educational and psychological measurement, etc. Additional special topics for this course may include: network psychometrics, Bayesian psychometric modeling, Rasch measurement theory, cognitive diagnostic measurement, computational psychometrics, socio-cognitive measurement theory, Bayesian networks in educational assessment/measurement, process data and improved measurement in computerized assessments, scaling forced-choice responses, etc.
EPSY6637: Item Response Theory
EPSY 6637 provides an introduction to item response theory (IRT), a theory of measurement that provides statistical models for explaining and predicting responses on tests of attributes. With application in education, psychology, business, political science, health outcomes research, etc., IRT provides many advantages over the classical true-score measurement framework. Particular advantages arise for testing situations in which we need to compare the scores of individuals taking different assessments or must create assessments that are suitable for individuals at different levels of the trait being measured. This course is relevant for students who: are interested in advanced measurement models; will use IRT in their future work as psychometricians or researchers in areas where the framework yields benefits; and/or aim to learn more about psychological measurement and statistical modeling in research with latent variables. Students successfully completing this course will be able to: (1) Identify the major unidimensional IRT models and explain the meaning of their parameters; (2) Explain the advantages of IRT over the classical measurement framework and identify testing contexts where IRT would be applicable and useful; (3) Use software packages to calibrate item response data and make sense of the results; (4) Carry out goodness-of-fit analyses and make inferences about model-data fit; (5) Construct tests with desired psychometric properties using information functions; (6) Perform linking procedures to place parameter estimates from different calibrations on a common scale; (7) Carry out procedures for detecting differential item functioning and interpret results; (8) Explain how IRT is used in computer adaptive testing; (9) Read and understand technical reports for state-wide assessments that uses IRT; and (10) Use simulated data to investigate the properties and behavior of various IRT techniques. Topics for this course may include: a review of classical test theory (CTT) & rationale for using IRT, IRT models for dichotomous responses, assumptions and features of IRT models, Rasch models, 2- and 3-parameter logistic (PL) models, estimating trait parameters and the information function, estimating item parameters, assessing goodness of fit, item and person fit, test construction using information functions, IRT models for polytomous responses, multidimensional models, differential item functioning, equating and linking, computer adaptive testing (CAT), etc.
EPSY6638: Advanced Topics in Item Response Theory
EPSY 6638 is an extension of EPSY6637, which intends to provide prospective applied psychometricians and academic researchers with the knowledge and skills necessary to successfully implement IRT procedures in applied settings and conduct advanced research on psychometric issues. Students successfully completing this course will be able to: (1) Implement IRT models for dichotomous and polytomous item responses using commonly used commercial and research software; (2) Carry out IRT applications for test equating, detection of differential item functioning, test construction using information functions, and adaptive testing; (3) Design and carry out simulation studies to address psychometric issues of current interest; and (4) Carry out and report on an independent psychometric research project. Topics for this course may include: a review of basic IRT concepts, equating and scaling, differential item functioning, adaptive testing, Bayesian estimation using MCMC procedures, multidimensional models, comparison of IRT software for parameter estimation, effect of model misfit on IRT applications, comparison of DIF detection procedures, survey and evaluation of free software for IRT applications (e.g., R programs, stand-alone programs), etc.
EPSY6651: Introduction to Methods for Causal Inference using Educational Data
EPSY 6651 emphasizes application of quantitative methods and techniques to investigate cause and effect relationships in education and other social sciences. Research involves several stages: developing research questions and hypotheses, planning the research design, collecting and analyzing data, and writing the final report. In this course, we focus on the steps necessary to design and conduct research that will permit valid causal conclusions. We survey four major types of designs that are useful for making causal claims from educational data, identify threats to the validity of causal conclusions that can be drawn from these designs, and examine steps to minimizing such threats. Therefore, this course is simultaneously theoretical and applied. Students successfully completing this course will be able to: (1) Understand and use theory, concepts, and terminology pertinent to conducting quantitative educational research with causal goals; (2) Develop in-depth understandings of the major experimental and quasi-experimental research designs for causal inference; (3) Identify threats to validity for each of the designs and strategies to minimize possible threats to validity; (4) Articulate the conditions under which different types of research designs may be more or less appropriate for facilitating causal inference; (5) Conceptualize mediator and moderator variables and appropriate statistical models for analysis involving these types of variables. Topics for this course may include: descriptive vs. causal research, the Institute of Education Sciences’s approaches to learning “what works” and to education research, Campbell’s validity framework, confounding and Rubin’s causal model, logic of randomization, types of randomized designs, conditional and unconditional inference models, validity threats in randomized controlled trials (RCTs), instrumental variables, moderation, mediation, testing for baseline balance, propensity scores (theory and software implementation), boosted regression for estimating propensity scores, pros and cons of randomized controlled trials (RCTs), regression discontinuity (theory, software implementations, and modeling functional form), difference in differences, Bayesian applied regression trees, etc.
EPSY6655: Advanced Causal Inference with Data
EPSY 6655 explores the implications of two main frameworks for understanding causal inference with data: the Potential Outcomes (PO) framework and the Directed Acyclic Graph (DAG) framework. These frameworks help to illustrate statistical paradoxes of historical interest, such as Lord’s paradox. The course also examines important recent papers related to confounder selection, measurement error in confounders, machine learning approaches to causal inference, a unified theory of mediation, and moderation and causal interpretations of racial disparities. Students successfully completing this course will be able to: (1) Comprehend and critique important research papers in the area of applied statistics and causal inference; (2) Appreciate the combination of technical expertise and thoughtfulness about critical assumptions that is necessary to create high quality quantitative research about important causal questions in the social sciences; and (3) Conduct independent research in the area of causal inference from data. Topics for this course may include: an overview of causal inference topics, experiments; observational studies; Rubin’s Causal Model; reverse causal questions; directed acyclic graphs, directed acyclic graphs vs. the potential outcomes framework, Lord’s Paradox, related paradoxes, pre-treatment variables in propensity score models, confounder selection, measurement error in confounders, machine learning approaches to causal inference, a unified theory of mediation and moderation, mediation/moderation and Rubin’s Causal Model, causal interpretations of racial disparities, etc.