Blog 7 - Performance Assessments

In your blog, define performance assessment. Describe how this type of assessment can be used in eLearning. What makes performance assessment truly worthwhile in eLearning? What can you foresee as the pitfalls and problems with performance assessment in the eLearning environment?

What is a Performance Assessment?

According to Oosterhof, Conrad, & Ely (2008) performance assessments involves the learner performing a task to create a product. There task often involves several steps and requires specific skills. These skills are educationally-based and/or on-the-job training and evaluation. The assessment is not the “typical” fixed response (multiple choice, true/false) or constructed-response (fill-in-the blank) assessment. The performance assessment can be written, but does not have to be. For example, a performance assessment can evaluate the student’s ability to complete a tax return or use/apply the English grammar rules in a story – these would be written. On the other hand a non-written assessment might evaluate the learner’s ability to play a musical instrument or throw a football. This is because performance assessments often evaluate motor skills as in the example of speech and language, performing arts, and spots.

To assist an educator in identifying whether an assessment is a performance based assessment or not, they should use the following criteria (Oosterhof, Conrad, & Ely, 2008, p. 144):

1.       Specific behaviors or outcomes of behaviors are to be observed

2.       The behaviors represent performance objectives or goals of the course

3.       It is possible to judge the appropriateness of the learners’ actions or at least to identify whether one possible response is more appropriate than some alternative response

4.       The behavior or outcome cannot be directly measured using a paper-and-pencil test, such as a test involving a multiple-choice, essay, or other written format.

There are two types of performance assessments – (1) single task performance assessments which measure knowledge of a concept or rule and (2) complex-task performance assessments which are used when problem-solve skills are involved. No matter which type is used, they are developed using three basic steps (Oosterhof, Conrad, & Ely, 2008):

1.       Identify the capability to be assessed. This involves identifying the goal to be assessed and the type of capability involved.

2.       Establish the performance to be observed. This involves a summary of descriptions of tasks associated with the goal being assessed, a description of the tasks to be performed, identification on whether to focus on process or product, identification of prerequisite skills, and learner’s instructions.

3.       Define a scoring plan.

Use in eLearning Environment
A critical role in determining whether eLearning learners have learned (or not) are the products they create. Performance assessments play a major role in an eLearning environment, however, “…only to situations where the learners’ actions are being used to directly measure an explicit goal of instruction” (Oosterhof, Conrad, & Ely, 2008, p. 158).  The performance an educator observes is only an indication of the learner’s knowledge; it is not the knowledge the educator hopes the learner is achieving.  Therefore, performance assessments are not used to measure declarative knowledge (Gagne’s reference, http://www.icels-educators-for-learning.ca/index.php?option=com_content&view=article&id=54&Itemid=73#des, Webb refers to as level 1, http://www.aps.edu/rda/documents/resources/Webbs_DOK_Guide.pdf,and Bloom’s Taxonomy refers to as remembering, http://ww2.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm).  They are used often to measure procedural knowledge and problem-solving skills.

A product, versus a process, is favored in an eLearning environment performance assessment. Even though a process is involved, the end result is that the online learner is able to produce a product. For example, a student studying computer programming would produce a program that does “x” and in order to produce that program they must follow the coding process.

An important thought is that the eLearning environment is making educators change from a teacher-centered model to a student-centered model. In this model learners have greater opportunities to demonstrate what they know and what they can do – that’s performance assessments!

Not every content area is ideal for an eLearning environment. Even though a content area is designed in an eLearning environment, not all assessments can be “paper-and-pencil” (or electronic online testing) and/or performance based.  Let’s take an example of a foreign language class. Yes, the instructor can assess the learner’s ability through the traditional “paper-and-pencil” test and this can be electronically with automatic scoring. Grammatical rules, vocabulary and other declarative knowledge can be tested. In a foreign language class it is important for the learner to be able to SPEAK the language and not just phrases but be able to carry on a conversation. A performance assessment can be utilized to assess whether a student can dialog with another learner or even the instructor in a given scenario. Doing such an assessment in a traditional classroom would be relatively easy since the educator can listen and watch the conversation unfold. In an eLearning environment this scenario would pose a challenge. However, with today’s technology of video, video conferencing, Skype, and other two-way communication programs, an educator can observe two or more learners in such a conversation.

There are four elements in online performance assessments (Oosterhof, Conrad, & Ely, 2008, p. 182):

1.       communicating the tasks and expectations to learners

2.       conveying products and other responses from each learner to the instructor and often to other learners

3.       providing feedback to each learner, and

4.       maintaining records.

An educator (and educational institutions) can address these four elements with the selection of an appropriate learning management system, identifying the options for file transfer and the use detached or integrated feedback between the educator and learners, and implementing techniques for efficiency (such as file naming conventions, scoring process, and use of hotkeys).

Advantages and Pitfalls in eLearning
While there are many advantages of performance assessments there are also disadvantages. Performance assessments can measure skills that paper-and-pencil tests cannot measure such as writing, foreign language, music, art, and sports -- these skills are based on motor skills. The educator’s challenge is to figure out which content area can be taught using an eLearning environment and which ones cannot.

Performance assessments promote the teaching and learning of complex skills; this is true for the traditional face-to-face and eLearning environments.

Written assessments traditionally focus on the product that results from doing a task, whereby in performance assessments the focus on the process a learner must do to get to the product. Summative and formative assessments rely on observations of a learner when completing a task. In the eLearning environment this observation is often not feasible.

In the traditional educator-student environment, the educator can easily observe what the learner knows through how the student is performing and their facial and physical appearance. In an eLearning environment, this insight is more dependent on the formal assessments.

In the educational environment, test security is a concern, no matter what type of test is administered. There is much greater test security in the paper-and-pencil tests than performance assessments. In performance assessments, learners are asked to product a product. The scoring plan and even product models (such as past student work) is provided ahead of time to the learner.
There are three limitations of performance assessments:  (1) they are less efficient, (2) subjective scoring, and (3) problems with generalizability.

Performance assessments take more time to prepare, administer, and score (compared to written tests). However, once the assessment is created, that same assessment can be administered for several school years. This is not the case of a written test. It takes more time for a learner to complete a performance assessment than a written test and additional time for the educator to score it. An educator cannot rely on computer algorithms to score the assessment, such as the case of a fixed response or constructed-response test.
As with long constructed response, the educator must be aware of scoring bias. The reliability of the assessment results might be questioned if the educator is scoring their own learners on a performance based or essay assessment. Therefore, it is important to have a well-defined scoring plan that is consistently followed. Scoring options include (1) comparison with a model, (2) checklists, (3) rating scales, and (4) rubrics. The scoring plan can be scored analytically or holistically and may or may not involve numbers. The important part of a scoring plan is that qualitative feedback and numerical score is given to the learner. 
A performance assessment does not generalize across domains since the educator is evaluating the learner’s ability to perform each task. In the case of a meteorology student, can they create a precipitation forecast involving a cold front as well as one involving a tropic storm?

Work Cited

Oosterhof, A., Conrad, R.-M., & Ely, D. P. (2008). Assessing Learners Online. Upper Saddle River: Merrill/Prentice Hall.