Tool or Guide

Description Consistent with ASAP’s assessment principles of offering a justice-oriented, antiracist approach, we prepared a working bibliography of culturally sustaining and anti-racist assessment resources. We plan to periodically update this bibliography, so feedback on the structure, content, and organization of the bibliography is welcome, as are suggestions for resources not yet included (asap@umass.edu). (January 2023)

Description This tool kit is designed to help research teams plan for and develop effective evidence-to-action researchproducts. Although useful for a wider audience, the tool kit was written for research contractors who work for the U.S. Department of Labor (DOL) and for DOL staff who review and provide guidance on research contracts. This guide can help research contractors reach and influence practitioner audiences (referred to in the toolkit as users). This toolkit will help DOL contractors and DOL: brainstorm options for evidence products and services, devise effective dissemination strategies, share expectations on what DOL wants from evidence products and services, build contractor and staff capacity to deliver products and services that lend themselves to E2A, and conduct quality assurance reviews to assess whether they are E2A-ready.

Description This rubric, developed by the CREATE Adult Skills Network team to review technical assistance resources for inclusion in the CREATE Edtech Technical Assistance Library, can help you determine the quality of guidance provided in a technical assistance resource. Use of the scoring is optional; do use it to reflect on the potential of the resource as you comment.  

Description These aggregate frameworks provide a structure to support learning in five subjects relevant to adult learners: Civics Education, Digital Literacy, Financial Literacy, Health Literacy, and Workforce Preparation. They include domains, topics, and subtopics in each subject. The frameworks were developed by aggregating existing curricula, assessments, and frameworks used in adult education and compiling insights from organizations and individual instructors. Use them to support Teaching Skills That Matter-aligned instruction, to locate and use resources in SkillBlox, or as a checklist for planning instruction and assessment. 

Description IES released a guide to help researchers avoid common misinterpretations of statistical significance and report study impacts that are more actionable for end users. Improving the quality and relevance of education studies is IES Director Mark Schneider’s central goal for the Standards for Excellence in Education (SEER). The guide introduces BASIE (Bayesian Interpretation of Estimates), an alternative framework to null hypothesis significance testing, and walks researchers through the key steps of applying BASIE: Select prior evidence based on the distribution of intervention effects from existing impact studies (e.g., IES’ What Works Clearinghouse database). Report traditional (based only on study data) and shrunken (based on both study data and prior evidence) impact estimates. Interpret impact estimates using Bayesian posterior probabilities (or credible intervals). Examine the sensitivity of shrunken impact estimates and posterior probabilities to what prior evidence is used. The guide includes “express stops” and a simple Excel tool so that researchers can quickly start using BASIE. Detailed “local stops,” technical appendices, and programming code are also provided for evaluation methodologists. View the guide by clicking here. This guide is one of a series that helps researchers implement SEER. Guides on generalizability and sharing study data were recently released, and a guide on implementation research is in development and will be announced on https://www.ies.ed.gov. The Institute of Education Sciences, a part of the U.S. Department of Education, is the nation's leading source for rigorous, independent education research, evaluation, statistics, and assessment.

Description The Association of Test Publishers and the International Test Commission have collaborated to develop Guidelines for Technology-Based Assessment to promote best practices and ensure fair and valid assessment in a digital environment. These Guidelines are now in draft form and are available for public comment through May 15, 2022.  The draft Guidelines are the product of a multiyear effort that involved dozens of invited authors, ad hoc technical reviewers, and extensive review by ten advisory groups representing practice areas and regions of the world.

Description This guide will help researchers design and implement impact studies in education so that the findings are more generalizable to the study's target population. Guidance is provided on key steps that researchers can take, including defining the target population, selecting a sample of schools—and replacement schools, when needed—managing school recruitment, assessing, and adjusting for differences between the sample and target population, and reporting information on the generalizability of the study findings.

Description IES has released a guide to help researchers who, to support open science, are making decisions about how to safely and appropriately share study data. This includes deciding which study data to share, how to organize the data, what documentation to include, and where to share their final dataset. Making data open is one of the Standards for Excellence in Education (SEER) that IES Director Mark Schneider identifies as essential to making research transformational. The guide offers researchers tips to address common challenges in sharing study data, such as how to balance privacy with transparency, along with concrete steps to take throughout the research process. Key principles include: The goal of data sharing is to produce something of value for science and ultimately, for the improvement of education. Focusing on sharing a well-organized and well-documented dataset can improve the organization and efficiency of the original study team. Researchers should commit to sharing some data or code to facilitate additional analysis. There is no single approach to sharing study data, and tradeoffs may be necessary. The guide also provides links to other resources, a checklist, templates, and sample materials. View the guide by clicking here. This guide is one of a series that helps researchers implement SEER to improve the quality and relevance of their education studies. A guide on generalizability was recently released, and another guide on how to report more interpretable impact findings is forthcoming and will be announced on https://www.ies.ed.gov. The Institute of Education Sciences, a part of the U.S. Department of Education, is the nation's leading source for rigorous, independent education research, evaluation, statistics, and assessment.