Researcher Guide on Interpreting Impacts

Description IES released a guide to help researchers avoid common misinterpretations of statistical significance and report study impacts that are more actionable for end users. Improving the quality and relevance of education studies is IES Director Mark Schneider’s central goal for the Standards for Excellence in Education (SEER). The guide introduces BASIE (Bayesian Interpretation of Estimates), an alternative framework to null hypothesis significance testing, and walks researchers through the key steps of applying BASIE: Select prior evidence based on the distribution of intervention effects from existing impact studies (e.g., IES’ What Works Clearinghouse database). Report traditional (based only on study data) and shrunken (based on both study data and prior evidence) impact estimates. Interpret impact estimates using Bayesian posterior probabilities (or credible intervals). Examine the sensitivity of shrunken impact estimates and posterior probabilities to what prior evidence is used. The guide includes “express stops” and a simple Excel tool so that researchers can quickly start using BASIE. Detailed “local stops,” technical appendices, and programming code are also provided for evaluation methodologists. View the guide by clicking here. This guide is one of a series that helps researchers implement SEER. Guides on generalizability and sharing study data were recently released, and a guide on implementation research is in development and will be announced on The Institute of Education Sciences, a part of the U.S. Department of Education, is the nation's leading source for rigorous, independent education research, evaluation, statistics, and assessment.

Guidelines for Technology-Based Assessment

Description The Association of Test Publishers and the International Test Commission have collaborated to develop Guidelines for Technology-Based Assessment to promote best practices and ensure fair and valid assessment in a digital environment. These Guidelines are now in draft form and are available for public comment through May 15, 2022.  The draft Guidelines are the product of a multiyear effort that involved dozens of invited authors, ad hoc technical reviewers, and extensive review by ten advisory groups representing practice areas and regions of the world.

Digital Skills Frameworks and Assessments: A Foundation for Understanding Adult Learners’ Strengths and Learning Needs

Description The CREATE Adult Skills Network (the Network) research teams are developing technology-supported learning and assessment tools and implementing curricula to help adult learners build digital skills. Throughout this work, each team has noted the importance of gaining a better understanding of the digital skills learners need to fully participate in the research projects. To that end, this Network Brief will introduce several widely used and relevant digital literacy frameworks and assessment strategies used in adult education. The brief provides high-level descriptions of the following frameworks:   Northstar Digital Literacy standards The ISTE SkillRise Profile of a Lifelong Learner Seattle Digital Equity Initiative’s (SDEI) Digital Skills Framework The Maryland Department of Labor/Adult Education’s Digital Literacy Framework for Adult Learners