Instructors’ professional judgment is at the centre of all assessment practice. From selecting or developing appropriate tasks, choosing assessment tools, giving feedback on writing and speaking performance, to deciding when a learner is ready to progress to the next level, instructors are using professional interpretation and judgment in order to make decisions.
“Instructors’ professional judgment is more reliable and valid than external tests when they have been involved in examining learner work, co-constructing criteria, scoring the work, and checking for inter-rater reliability. . . “ (ARG, 2007).
In this section, we discuss steps in evaluating portfolios, preparing progress reports, conducting conferences with learners.
Evaluating a Portfolio to Assign Benchmarks
The PBLA portfolio tells a story of learner growth and achievement. Telling this story involves gathering evidence, reviewing the evidence and making decisions.We will focus on the evidence and processes that support informed professional judgment when reviewing learner portfolios to make decisions.
How the Portfolio Demonstrates Achievement
To show evidence of achievement of benchmark standards
8 – 10 artefacts in each of the skill areas (Listening, Speaking, Reading and Writing). Portfolio artefacts will include a combination of assessment tasks and skill-using activities. (See: PART B: Portfolio Contents for fuller explanation. )
Informal notes, anecdotal comments or checklists that have been collected over the semester to document what a learner can do aside from regular classroom activities.
Optional: Results from formal exit tasks (e.g., SAM, CLB 5 – 10 Exit Assessment Tasks) may be included if appropriate to what has been taught in the class.
Principles Guiding a Review of the Evidence:
Ask: What benchmarks have been achieved?
Before beginning the review process, there are several critical issues to keep in mind:
It is important to ensure that assessments are consistent with the standards outlined in the CLB. CLB expectations for the level and stage are found in the Profiles of Ability Across a Stage, Some Features of Communication Across a Stage and the specific descriptors for each benchmark level. As well, reviewing the exemplars in the CLB Support Kit will help to keep expectations realistic when assessing learner performance.
Remember that we are not expecting 100% mastery. National Placement guidelines state that “As a general rule, the benchmarks assigned to a learner at the time of placement assessment, summative in-class assessment, or high-stakes language test, mean that the learner has achieved, and demonstrated, the level of communicative ability associated with most or all (traditionally, 70 to 100%) of the descriptors for the benchmarks assigned in each of the four skills. (National Placement and Progression Guidelines, 2013, p.3).Reviewing the evidence in the skill areas to assign benchmarks is not simply a matter of tabulating scores or percentages. The process requires informed professional judgment embedded in a thorough understanding of the CLB expectations and consideration of the factors identified below.
Portfolio review uses a combination of analytic and holistic processes. Portfolio review is analytic – focusing on collecting the necessary artefacts and looking at the performance on individual entries. But it is also holistic which entails taking a step back and widening our lens to look at ALL of the entries in any skill as a WHOLE and exercising professional judgment to answer the question: Does the learner demonstrate success consistently, i.e. most of the time at the benchmark level?
Steps in Reviewing Each Skill Area
Before you begin, ask these broad questions:
Review the evidence. Has the learner met Benchmark level expectations? What progress has been made?
Notice the learner’s strengths and challenges. What suggestions will you make to the learner to help him/her improve his or her skills?
Once you’ve considered these questions, you are ready to begin the review, following these steps:
Check the number of entries. Generally, you are reviewing portfolios after learners have been in class a minimum of 250 hours and you have about 8 – 10 entries for each skill area. (If a learner has recently moved to your class and was not assigned benchmarks upon leaving the previous class, you may consider evidence in the learner’s portfolio since benchmarks were last assigned).
Check the distribution of entries. The entries should be distributed across the four competency areas and reflect a range of tasks and criteria appropriate to the CLB level.
Review the assessment tasks. Look at the tasks over time. More recent assessment tasks should give a clearer indication of current language ability than tasks completed earlier in the semester. Look for trends or particular areas of improvement or concern.
Review the skill-using activities. The skill-using activities also provide evidence of what learners are able to do. Skill-using activities are an opportunity for learners to practise – sometimes learner performance will be weaker on the practice activities and stronger on the subsequent assessment tasks, providing evidence of learning. Sometimes the skill-using activities will be stronger than the assessment task. There may be many reasons for this. For example, consider the amount of scaffolding provided. Scaffolding should be appropriate to the CLB level and help move learners to more independent performance.
Review anecdotal comments. Informal notes about a learner’s performance not directly related to assessment tasks (e.g., the learner’s ability to give an appropriate apology when coming into class late, his/her ability to follow written or oral classroom instructions, the learner’s dependence on an electronic translator, etc.
Consider optional information. You may also include results from tests and formal exit tasks (e.g., SAM, CLB 5 – 10 Exit Assessment Tasks, etc.) if they are appropriate to what has been taught in the class. The results should be viewed holistically as part of the assessment evidence. They should not be given any 0greater weight than other data collected.
Assign Benchmark Levels.
After reviewing the portfolio entries assign a benchmark level. What benchmark has the learner achieved in each skill?
How the Portfolio Demonstrates Growth
To show progress related to learner goals..
About Me section of the portfolio including:
Learner Goal statement,
Review of Goal statement
Review the Evidence:
Ask: What progress has the learner made towards his/her personal language learning goals?
Review the learner’s autobiography or piece of personal writing, needs assessment information, goal statement and review, and learning reflections. This is an area in which the learner should generally take the “lead” in the learner conference but make notes so you will be able to support learners when discussing their growth related to their goals.
Learner Progress Reports
There are two required learner reporting forms: The Learner Progress Report and the Learner Conference Summary.
The Learner Progress Report is used when assigning/reporting benchmarks. The Progress Report is a tool to give specific feedback to learners on their language progress over a reporting period based on sufficient evidence in their portfolio to assign a benchmark level. A separate Progress Report is issued for each course the learner attends.
The Learner Conference Summary is used when learners have insufficient evidence to assign benchmark levels at the time of the Learner Conference. Their portfolios may have insufficient evidence for a number of reasons: they started late, they have sporadic attendance, they leave early, or they are in classes that only meet a few hours/ week. Instructors can use the form to record feedback on progress. A separate Learner Conference Summary is issued for each course the learner attends.
Using the Progress Report
The progress report documents the language-learning outcomes determined through PBLA. The Learner Progress Report is used for both ESL and ESL literacy learners. Two electronic versions of the progress report are available: one with dropdown menus and one without dropdown menus. Check with your program administrator to determine which one to use.
Reporting CLB Levels
Progress reports are intended to be a record of learners’ progress in learning English. The reports are completed after reviewing the learner’s portfolio.
When reporting CLB levels, instructors fill in the COMPLETED CLB level for each skill. Remember, completion means “the learner has achieved, and demonstrated, the level of communicative ability associated with most or all (traditionally, 70% – 100%) of the descriptors for the benchmarks assigned in each of the four skills” (National Language Placement and Progression Guidelines, p3).
For a variety of reasons, learners may make progress in a term without completing a CLB level. This information is incorporated in the Learner Strengths and Progress section. The following language may help you discuss learner progress:
Beginning is used if the learner has demonstrated initial evidence of characteristics (qualities and attributes) associated with performance at this level.
Developing is used if the learner has demonstrated increasing evidence of characteristics (qualities and attributes) associated with performance on tasks at this level but may not be able to do so consistently or may not have performed a sufficient range of tasks and competencies.
Signing and distributing the progress report
The learner’s copy requires three signatures. The administrator’s signature can be an original signature or set up as a computer generated signature. Instructors and learners should sign the Progress Report at the time of the Conference. The learner can then file the Progress Report in his/her Portfolio.
Using the Learner Conference Summary
The Learner Conference Summary is used for both ESL and ESL literacy learners when the learners has insufficient evidence in their portfolio. Two electronic versions of the Conference Summary are available: one with a dropdown menu and one without. Check with your program administrator to determine which one to use.
Signing and distributing the Learner Conference Summary
The ESL Learner Conference Summary is signed by the administrator, teacher/instructor and the learner.
Teacher/Learner Progress Conferences
The brief (10 to 15 minutes) progress conference is an opportunity for the learner and teacher to review the learner’s progress in learning English in relationship to the learner’s specified needs and goals. It should be a culmination of the ongoing dialogues in which the teacher and learner have engaged throughout the term as well as an opportunity to consolidate information about the learner’s language proficiency and to set new directions.
Preparing for the Progress Conference
Progress conferences might be a new experience for the class and may cause the learners some anxiety. You can mitigate anxiety by providing an overview of what will occur during the conference, including some of the topics of discussion. A day or so prior to progress conferences, you might find it helpful to facilitate learning activities to prepare learners to participate in their interview. Working in groups, learners might be asked to do the following:
Review their portfolios and discuss the goals they set at the beginning of the course and the progress they have made towards them
Discuss what they have learned, what they are proud of, and what is still difficult
Talk about what they think the teacher will say about their progress; practise some phrases that they or the teacher might use
Talk about their experience – what they liked, what they wish had been different – or have them write a brief reflection.
Conducting the Progress Conference
Learners should not receive unexpected news about their CLB outcomes in their one-on-one teacher-learner conference. If you have ensured that learners understand the CLB outcomes towards which they have been working and if you have been providing learners with realistic ongoing feedback throughout the course, learners should have a good idea of what they have achieved, as well as their strengths and challenges.
Talking about goals – During the teacher-learner conference, begin by talking briefly about the expectations of the course or program and refer to the personal goals the learner expressed at the beginning of the term or course. If learners can’t remember their goals, their goal statements in their portfolios can provide a useful reminder.
Reflecting on progress – Ask learners to talk about their progress – about what they can do now in English that they could not do before. If learners have been encouraged to reflect on their language learning throughout the program, this activity should not be unduly challenging. Learners who are familiar with the CLB outcomes towards which they have been striving can say whether they think they have met those outcomes. If learners are less familiar with the CLB outcomes, they can talk about something that they could not do before, something they have improved on, and something they think they still need to improve. Ask learners to pick out some samples from their portfolios that support their opinions. If the learners are at a low CLB level, ask them to show you a couple of samples of language use of which they are particularly proud.
Provide learners with the written progress report and discuss the contents. The contents of the portfolios provide examples of language use for discussing the progress reports. During the conference, encourage learners to identify future goals and language-learning objectives and discuss with them strategies that might be helpful in furthering their language development.