Lies, Damned Lies, & Surveys! Developing High-Quality Surveys for Education and Research
Presented by: Anthony R. Artino, Jr., PhD - Professor of Medicine, Uniformed Services University of the Health Sciences
Location: T-229, Pacific Room, UW Health Sciences Library, Seattle
RSVP (in-person only): Space is limited! https://climeartinoworkshop.eventbrite.com
Surveys are widely used in health professions education (HPE), yet few educators are familiar with the best practices of survey design (Artino et al., 2014). As a result, many HPE surveys fail to adequately capture the very attitudes, opinions, and behaviors they are designed to assess.
This workshop introduces participants to a systematic, seven-step design process for creating high-quality surveys fit for program evaluation and research purposes. The workshop consolidates and organizes the abundance of survey design literature that exists in the social sciences and guides the novice survey developer through the survey design process. This design process is different from previously described processes in that it is inherently collaborative and relies on other experts in the field as well as potential participants. In addition, the process front loads the task of collecting validity evidence by focusing heavily on item development.
Throughout the workshop, emphasis will be placed on conceptualizing relevant HPE constructs, collecting reliability and validity evidence for a survey scale, and identifying common item-writing mistakes. In addition, the benefits of expert validation, cognitive interviewing, and pilot testing will be introduced.
The workshop is intended for any educator or researcher, from novice to more experienced, who is currently developing or thinking about using a survey for program evaluation or research. Given the frequency with which surveys are used in HPE, following a rigorous approach to the design process will greatly improve the chances that survey designers measure what they intend to measure.
Upon completion of the workshop, participants will be able to…
1) Recognize how to use a systematic, seven-step process as the framework for survey design;
2) Demonstrate how to develop an appropriate set of items (a survey “scale”) to characterize the educational construct being measured;
3) Identify common item-writing pitfalls in survey design; and
4) Define the purpose of expert validation, cognitive interviews, and pilot testing.
Anthony R. Artino, Jr., PhD, is a Professor of Medicine and Deputy Director for Graduate Programs in Health Professions Education (HPE) at the Uniformed Services University of the Health Sciences in Bethesda, Maryland. He holds a joint appointment in the Department of Preventive Medicine and Biometrics. A graduate of the University of Connecticut with a PhD in educational psychology, Dr. Artino mentors graduate students and teaches in the HPE degree programs, as well as the Master of Health Administration and Policy and the Master of Public Health degree programs. As a researcher, he is the Principal Investigator (or Associate Investigator) on several funded research projects totaling more than $7.9 million. In this role, Dr. Artino collaborates with a team of scientists trained in a variety of academic disciplines, including psychology, epidemiology, physiology, medicine, and English. This interdisciplinary team studies various aspects of human motivation, learning, and performance in medical education.
Dr. Artino is a Deputy Editor for the Journal of Graduate Medical Education, an Assistant Editor for Academic Medicine, and is on the editorial boards of several peer-reviewed journals, including Military Medicine, Military Psychology, and The Internet and Higher Education. Dr. Artino has published more than 100 peer-reviewed journal articles on learning and motivation, survey design, and online education. Over the last 10 years, he has presented several dozen survey-design workshops at national and international meetings around the globe.