Unit Lead: Rob Woods (interim)

Unit Supervisors: Josh Lloyd, Trustin Domes, Lisa Krol, Matt Kushneriuk

This unit focuses on competencies the Clinician Educator (CE) will need to develop and implement a valid system of assessment.  These skills are essential in the development of new curricula and in the management of existing curricula.

Co-requisite: Curriculum Unit 

Competencies Addressed in this Unit

A CE must be able to

1. Apply principles of assessment in developing a system of assessment for a program
2. Select assessment instruments that are relevant and appropriate for the learning context
3. Describe the challenges of self-assessment

Key Documents

• Official Royal College Requirements for the Assessment Unit
• Check-in and check-out forms
• Final Unit Report (to be completed by Unit Supervisor)
• Assessment Task List 
• Assessment Reading List (see below)
• Complete set of documents

Objectives

By the end of this Unit, the CE will be able to:

1.  Demonstrate comprehension of the principles of assessment, by

• recognizing the multiple purposes of assessment in a system of education
• analyzing how a system of assessment fits within the cycle of curricular design
• describing the differences between summative and formative assessment
• applying the concept of “utility” to individual assessment instruments and methods
• identifying and describing the professional responsibilities of educators to ensure that the assessment system meets the needs of trainees and the public
• aligning types of assessment with the corresponding levels of Miller’s pyramid
• comparing and contrasting the terms “assessment” and “evaluation”
• justifying the importance of both self-directed assessment and reflection in the development and maintenance of competence of a clinician
• describing the enablers of and barriers to self-assessment

2.  Conduct a survey of important assessment instruments, by

• comparing and contrasting the following methods or instruments of assessment, with an emphasis on their structure, the types of objectives they can assess, the context for which they are best suited (e.g., clinical setting, classroom environment, etc.), and their advantages and disadvantages:
  • written exams (constructed response and selective response)
  • oral examinations
  • OSCEs and their derivatives (e.g., OSATs)
  • ITERs and other rating-scale–based expert assessment forms
  • instruments for direct observation, such as Daily Encounter Cards, CEX
  • Multi-Source Feedback
  • chart audits and practice assessments
  • portfolios
  • other
• selecting and implementing appropriate assessment instruments within a curriculum

3.  Demonstrate comprehension of the principles of psychometrics, by

• describing modern validity theory and the concept of reliability and applying these to the evaluation of assessment strategies
• applying the concept of utility and analyzing the implications of the assessment system in the context of the candidate’s jurisdiction
• comparing and contrasting classical test theory and generalizability theory
• defining principles of standard setting and selecting appropriate methods for a given assessment instrument and context
• applying the principles of qualitative research in the context of judging the quality of a portfolio as a composite assessment method (using principles of credibility, consistency, triangulation, transferability, etc.)

4. Demonstrating competence in remediation, by

• applying a remediation framework to an encounter the candidate has had with a learner in difficulty, while emphasizing a step-wise approach

Assessment

• Please refer to the Task List for the Assessment Unit for a complete listing of the summative and formative assessments required for this unit.
• Final Unit Report (to be filled in by Unit Supervisor)