Item-Writing Resources

The Item-Writing Resources on this page are designed to help you understand what makes a good question and how to create one. The guidelines found here should be used when writing questions for all ARRS activities. These guidelines are based on AMA question requirements and psychometric principles.

For detailed instructions, please see the ARRS Item-Writing Guidelines

Takeaways

  1. DO NOT use “All of the Above” or “None of the Above” in the answer options
  2. DO NOT use a negative stem e.g. "Which of the following is NOT"
  3. Include a rationale and a reference for every question
  4. Avoid vague words like "sometimes" and "usually"
  5. Answer options should be grammatically correct from the stem. e.g. "The most likely diagnosis is .... carcinoma."

Glossary

  1. Stem – Background information on case and images
  2. Lead in statement – The question based on the case e.g. What is the best interpretation of these images?
  3. Distractors – The wrong answers
  4. Key – The right answer

Resources about Multiple Choice Questions

  1. Writing Multiple Choice Questions for AJR Integrative Imaging
  2. Prevalence of Flawed Multiple- Choice Questions in Continuing Medical Education Activities of Major Radiology Journals 

ARRS thanks Petra Lewis, MD, for these item writing resources.