Abstract:
This case study aimed to change the construction of teachers’ written tests so that
items were designed to assess competencies in an authentic and challenging way. A
small group of five psychology teachers participated in 10 sessions of an authentic
assessment faculty-training program, to learn to assess problem-solving
competencies for situations typically faced by professionals in the workplace. The
authentic assessment training emphasized the incorporation of three main
characteristics: (1) inclusion of a realistic context, (2) measurement of higher order
thinking skills, and (3) development of evaluative judgment, concerning the quality
of their own performance. Post-training the items´ construction was analysed,
according to their type and authenticity. Mixed effects logistic regression showed a
statistically significant increase in open-response items, and two-way ANOVA
indicated that cognitive challenge improved. The results showed written tests had:
a) more open-response items of higher cognitive complexity, b) fewer items
requiring closed, memorized responses, c) more use of realistic contexts to
measure knowledge in a situated way, and d) improvement in curriculum alignment
of tests and greater consistency in measuring competencies. These advances
validate the authentic-assessment training program for a better written assessment
design.
