Online Assessment – Digital Learning & Scholarship

In a well-designed online course, assessment activities are closely tied to the learning activities. The course syllabus should clearly spell out the learning objectives for the course, and in turn, learning objectives should be defined for each learning activity, including how they relate to the course objectives. Learning assessment techniques should be matched to the learning objectives of an activity, so that instructors can accurately monitor student learning.

What is an Exemplary Assessment?

Exemplary assessments are meaningful, motivational, engaging and should guide the student in the learning process.

Characteristics

Huba and Freed (1999) identified eight characteristics of exemplar assessments as follows:

Authentic – reflects real life experiences
Challenging – stimulates the learner to apply knowledge
Coherent – serves as a guide for the student to achieve the learning goal
Engaging – attracts the learner’s interest
Respectful – sensitive to the individual learner’s beliefs and values
Responsive – includes a feedback mechanism to assist the student in the learning process
Rigorous – requires applied understanding of learning to achieve a successful outcome
Valid – provides information that is useful to meet the intended learning outcomes

It can take some time to design good assessments for an online environment, so instructors should make sure that they use a learner-centered strategy that is appropriate and focused on the:

Learners: Each student (or groups of students) has unique characteristics and needs.
Content: Assessment needs to be customized to the learning objectives of the material.
Instructor: It is important that the instructor use methods where they have the most skills and experience, though they should use techniques that are optimized for the online environment.
Delivery method: What works well in a face-to-face classroom does not always translate well into the online environment.

Rubrics

The use of rubrics or marking guides to assess writing assignments or class projects is encouraged for online courses, especially those that are primarily asynchronous. A rubric allows the student to clearly understand the learning objectives and assessment criteria for the assignment, along with becoming familiar with the skills and proficiency that would demonstrate the highest level of achievement. The use of rubrics also provides an opportunity for the instructor to provide fair, consistent and timely feedback to the students. John Hopkins University provides a good perspective on the use of rubrics in the video 'To Rubric or Not to Rubric' located on their faculty guide to "Assessing Student Learning Online."

Finally, it is important to remember that the addition of technology can offer advantages to the learning environment as compared to the traditional classroom. The 2010 report of the UK JISC names eight advantages for assessment practices that are enhanced by the use of technology:

Greater variety and authenticity in the design of assessments
Improved learner engagement, for example through interactive formative assessments with adaptive feedback
Choice in the timing and location of assessments
Capture of wider skills and attributes not easily assessed by other means, for example through simulations, e-portfolios and interactive games
Efficient submission, marking, moderation and data storage processes
Consistent, accurate results with opportunities to combine human and computer marking Increased opportunities for learners to act on feedback, for example by reflection in e-portfolios
Innovative approaches based around use of creative media and online peer and self-assessment
Accurate, timely and accessible evidence on the effectiveness of curriculum design and delivery

References:

Sewell, Jeanne P, Karen H Frith, and Martha M Colvin. 2010. “Online Assessment Strategies: A Primer” 6 (1): 9.

Conrad, Dianne, and Jason Openo. 2018. Assessment Strategies for Online Learning: Engagement and Authenticity. Athabasca University Press. https://doi.org/10.15215/aupress/9781771992329.01.

Kinds of Assessment

Assessments fall into two categories. Within these categories, there are further subcategories (low-stakes, high stakes, academic, authentic, etc.).

Formative Assessments

Formative assessments have a primary goal of improving the depth and quality of student learning, not to provide final evidence of learning or yielding grades. You will use these assessments to inform your teaching and to provide feedback to students. Students will use these assessments to inform their own view of their mastery and understanding of the subject matter.

Some examples of formative assessments for the virtual classroom:

Pre-tests of foundational knowledge
Muddiest Point discussion forums
Draft submissions of project sections
Peer Review
Self-check quizzes

Dylan Williams, a leading researcher in the field of assessment defines formative assessment like this: “Formative assessment describes all those processes by which teachers and learners use information about student achievement to make adjustments to students learning to improve their achievement.”

Summative Assessments

A summative assessment has a primary goal of providing proof of learning and assigning a grade or defining the level of knowledge a student has attained. You will use these types of assessments to decide if the student has achieved mastery of the course material. Students will use these assessments to judge whether they are capable of continuing in the field of study or similar high-level decisions. These assessments do not generally offer opportunities for revision or improvement.

Some examples of summative assessments for online learning:

Research Papers
Quizzes
Exams
Problem Sets

Another way to differentiate assessment types is by relative value vis a vis their affect on a student’s ultimate assessment, their gradThese two categories are:

High stakes assessments
Low-stakes assessments

High-Stakes Assessments

What is a high-stakes assessment? High-stakes assessments are defined by value more than type. They have six characteristics:

Given infrequently
Is a single, defined assessment
Has obvious, significant consequences for both success & failure
High value relative to student’s final grade
Summative in nature
Feedback, if given is not timely

For example, if you assess your students with a midterm and a final exam that together yield more than fifty percent of a student’s final grade, even if students complete other forms of assessment, you are using high-stakes assessments because failure on these assessments will almost certainly result in failure of the course.

Fear is a primary motivator for students required to take high-stakes assessments. They will do whatever they need to do to succeed because there is so much at stake. This includes cheating, plagiarism, and cramming. These strategies might get them through the exam successfully. but may not yield a student who actually knows what the test claims to prove they know.

Low-stakes Assessments

Like the high-stakes assessments mentioned above, low-stakes assessments are defined by their relative value in relation to a student’s final grade. They also have certain recognizable characteristics:

Given frequently
They individually, have low impact on final grade
Formative in nature
Feedback is timely

As championed by Dr. Scott Warnock of Drexel University, Frequent Low Stakes (FLS) assessments are about feedback. By giving more frequent assessments you dilute the impact of any one grade (good or bad) and reward consistent quality efforts. While high-stakes testing “discourage teachers from using strategies which promote inquiry and active student learning…” FLS methods do the opposite. FLS approaches foster transparency in grading and a collaborative academic atmosphere.

(borrowed from: https://ep.jhu.edu/files/assessment-types.pdf)

Assessment Challenges

Cheating is easier to do (and harder to detect) online.
While it’s not clear whether online students do, in fact, cheat more than face-to-face students (Watson & Sottile, 2010), the truth is that it is more difficult to monitor who’s taking a test and how they’re taking it online than it is in a classroom.

Strategies for adapting assessments for online delivery include:

Timed/open book tests. Online, every test is an open-book test (except those that are proctored). To minimize read-as-you-go test-taking, reduce the amount of time students have to take the test so that only those students familiar with the material can answer the questions in the time allotted. Alternatively, replace selected response tests (such as multiple choice and T/F) with short-answer or essay questions that require students to apply textbook facts to novel scenarios.
Shuffled/randomized test questions. Shuffling questions helps reduce the likelihood that two students sitting in adjacent library carrels (six-feet apart, of course) can take the same test together, one answering the “odds” and the other answering the “evens.” Selecting questions randomly from a large test bank takes this idea one step further, providing each student with a similar (but not identical) assessment.
Plagiarism detection software. Having students run their essays through plagiarism detection service can potentially deter cut-and-paste plagiarism. At the very least, it can start a conversation about how to cite sources properly. Something like Grammarly’s plagiarism checker can detect plagiarism from billions of web pages as well as from some academic databases.
Frequent low-stakes tests, such as short quizzes or self-check activities worth no more than a few points each, help make cheating more trouble than it’s worth.
Performance assessments. Assignments that require students to write, speak, or present to the class are harder to fake—especially if they occur regularly throughout the course.
Coordinated tests. Instructors who teach multiple sections of the same class may want to coordinate tests so that all students take the same test at the same time. (Staggering tests increases the likelihood that the first students to take the test can pass on question details to their peers.)

Online courses need more student-to-student interactions “built in” than do face-to-face courses.
In a classroom setting, students interact—socially (chatting before class starts) and as part of common classroom activities (asking questions for clarification, weighing in on impromptu discussions, etc.). Online, these opportunities to feel connected and learn from each other do not occur naturally; they must be carefully planned and managed. Assessments that incorporate student-to-student interactions—while not appropriate for every course—can play a powerful part in a course’s overall communication strategy:

Peer-review. Asking students to review their classmates’ work (and grading them on their reviews) can help motivate best efforts as well as help students learn from each other.
Group projects. Well-designed group projects help students master both course content and team participation skills.

Online students need more student-to-instructor interaction than their face-to-face counterparts.

While instructor interaction and feedback is important to all students, it’s critical to the success of online learners and—like student-to-student feedback—must be carefully planned and cultivated in an online setting.

Frequent, low-stakes testing (“self-check” quizzes and activities). Options range from short selected response quizzes and watch-and-discuss questions to complex games and activities accessed through textbook publishers’ add-on course cartridges.
Rich, detailed feedback. Strategies for providing rich, detailed feedback vary based on the activities you’ve selected for your online course. The most practical include building detailed feedback into selected response quizzes; actively managing discussion boards; and administering weekly surveys asking students to identify the concept(s) they’re struggling with and then addressing the most-identified concept(s) by using a product such as Jing to create and post quick video clarifications.

Online students need more planned structure—that is, more help in staying on time and on task—than their face-to-face counterparts. The structure that occurs naturally when students and instructor congregate in the same place at the same time (seeing “Test next Tuesday” written on the white board, for example, or overhearing classmates a row over discussing an upcoming assignment) doesn’t just “happen” in an online course; it must be planned and managed. These assessment-related techniques can help:

Frequent, low-stakes tests (ungraded or low-point-value “self-checks”) help students gauge for themselves how well they’re mastering the material.
Graded milestones. Breaking up large projects into smaller graded milestones helps students (and you) identify problems areas early enough to address them.
Graded participation. Using a rubric to grade discussion board participation is time-intensive, as is asking students to review each other’s work. However, the benefits in terms of being able to gauge and guide students’ understanding can often be worth the time investment.

It’s important to note that while implementation strategies differ in an online environment, the pedagogical strategies underlying assessments shouldn’t change when you move a face-to-face course online. Online or off, assessments need to align with course objectives and provide a consistent measurement of student learning. If having students hand-write math proofs so that you can assess understanding and grant partial credit makes sense in a classroom, for example, chances are it makes sense online, too. The pedagogical strategy (hand-written homework) stays the same; only the implementation strategy (having students create and sending handwritten work as a digital file) needs to change.

References:

Watson, G. & Sottile, J. (2010). Cheating in the Digital Age: Do Students Cheat More in Online Courses? Online Journal of Distance Learning Administration, Volume XIII, Number 1. Retrieved online October 1, 2013 from http://www.westga.edu/~distance/ojdla/spring131/watson131.html

(adapted from: https://www.facultyfocus.com/articles/online-education/7-assessment-challenges-of-moving-your-course-online-solutions/)

Assessment Tools

Our learning management system, Moodle, is designed to distribute, collect and store a wide variety of student artifacts and entries. The system is very secure, in that students are only able to see their own work and grades (unless the instructor sets up group activities). Both Moodle Assignments and Quizzes allow the instructor to accommodate accessibility concerns with user or group overrides that can give extra time or access for specified students.

Moodle Assignments - use a Moodle Assignment for a graded essay or final report.

Moodle Quizzes - use a Moodle quiz for a quiz or final exam. Question types include multiple choice, essay, true/false, matching, short answer, plus several other options.

Kaltura Video Quizzes - use a Kaltura video quiz if you want to assess completion and comprehension of a required video assignment. To maximize viewing time, it is recommended to not allow skipping ahead in the video (a recommended setting), along with adding a quiz question near the end of the video.

Graded Discussion Forums - use a graded discussion forum if you would like to assess student asynchronous participation in a discussion forum.

Adding a Discussion Forum
Modify a Discussion Forum for Grading