Remote Instruction Best Practices

Online Assessment

In a well-designed online course, assessment activities are closely tied to the learning activities. The course syllabus should clearly spell out the learning objectives for the course, and in turn, learning objectives should be defined for each learning activity, including how they relate to the course objectives. Learning assessment techniques should be matched to the learning objectives of an activity, so that instructors can accurately monitor student learning.

Assessments fall into two categories. Within these categories, there are further subcategories (low-stakes, high stakes, academic, authentic, etc.).

Formative Assessments

Formative assessments have a primary goal of improving the depth and quality of student learning, not to provide final evidence of learning or yielding grades. You will use these assessments to inform your teaching and to provide feedback to students. Students will use these assessments to inform their own view of their mastery and understanding of the subject matter.

Some examples of formative assessments for the virtual classroom:

  1. Pre-tests of foundational knowledge
  2. Muddiest Point discussion forums
  3. Draft submissions of project sections
  4. Peer Review
  5. Self-check quizzes

Dylan Williams, a leading researcher in the field of assessment defines formative assessment like this: “Formative assessment describes all those processes by which teachers and learners use information about student achievement to make adjustments to students learning to improve their achievement.”

Summative Assessments 

A summative assessment has a primary goal of providing proof of learning and assigning a grade or defining the level of knowledge a student has attained. You will use these types of assessments to decide if the student has achieved mastery of the course material. Students will use these assessments to judge whether they are capable of continuing in the field of study or similar high-level decisions. These assessments do not generally offer opportunities for revision or improvement.

Some examples of summative assessments for online learning:

  1. Research Papers
  2. Quizzes
  3. Exams
  4. Problem Sets

Another way to differentiate assessment types is by relative value vis a vis their affect on a student’s ultimate assessment, their gradThese two categories are:

  1. High stakes assessments
  2. Low-stakes assessments

High-Stakes Assessments

What is a high-stakes assessment? High-stakes assessments are defined by value more than type. They have six characteristics:

  1. Given infrequently
  2. Is a single, defined assessment
  3. Has obvious, significant consequences for both success & failure
  4. High value relative to student’s final grade
  5. Summative in nature
  6. Feedback, if given is not timely

For example, if you assess your students with a midterm and a final exam that together yield more than fifty percent of a student’s final grade, even if students complete other forms of assessment, you are using high-stakes assessments because failure on these assessments will almost certainly result in failure of the course.

Fear is a primary motivator for students required to take high-stakes assessments. They will do whatever they need to do to succeed because there is so much at stake. This includes  cheating, plagiarism, and cramming. These strategies might get them through the exam successfully. but may not yield a student who actually knows what the test claims to prove they know.

Low-stakes Assessments

Like the high-stakes assessments mentioned above, low-stakes assessments are defined by their relative value in relation to a student’s final grade. They also have certain recognizable characteristics:

  1. Given frequently
  2. They individually, have low impact on final grade
  3. Formative in nature
  4. Feedback is timely

As championed by Dr. Scott Warnock of Drexel University, Frequent Low Stakes (FLS) assessments are about feedback. By giving more frequent assessments you dilute the impact of any one grade (good or bad) and reward consistent quality efforts. While high-stakes testing “discourage teachers from using strategies which promote inquiry and active student learning…” FLS methods do the opposite. FLS approaches foster transparency in grading and a collaborative academic atmosphere.

(borrowed from: https://ep.jhu.edu/files/assessment-types.pdf)

Cheating is easier to do (and harder to detect) online.
While it’s not clear whether online students do, in fact, cheat more than face-to-face students (Watson & Sottile, 2010), the truth is that it is more difficult to monitor who’s taking a test and how they’re taking it online than it is in a classroom.

Strategies for adapting assessments for online delivery include:

  • Timed/open book tests. Online, every test is an open-book test (except those that are proctored). To minimize read-as-you-go test-taking, reduce the amount of time students have to take the test so that only those students familiar with the material can answer the questions in the time allotted. Alternatively, replace selected response tests (such as multiple choice and T/F) with short-answer or essay questions that require students to apply textbook facts to novel scenarios.
  • Shuffled/randomized test questions. Shuffling questions helps reduce the likelihood that two students sitting in adjacent library carrels (six-feet apart, of course) can take the same test together, one answering the “odds” and the other answering the “evens.” Selecting questions randomly from a large test bank takes this idea one step further, providing each student with a similar (but not identical) assessment.
  • Plagiarism detection software. Having students run their essays through plagiarism detection service can potentially deter cut-and-paste plagiarism. At the very least, it can start a conversation about how to cite sources properly. Something like Grammarly’s plagiarism checker can detect plagiarism from billions of web pages as well as from some academic databases.
  • Frequent low-stakes tests, such as short quizzes or self-check activities worth no more than a few points each, help make cheating more trouble than it’s worth.
  • Performance assessments. Assignments that require students to write, speak, or present to the class are harder to fake—especially if they occur regularly throughout the course.
  • Coordinated tests. Instructors who teach multiple sections of the same class may want to coordinate tests so that all students take the same test at the same time. (Staggering tests increases the likelihood that the first students to take the test can pass on question details to their peers.)

Online courses need more student-to-student interactions “built in” than do face-to-face courses.
In a classroom setting, students interact—socially (chatting before class starts) and as part of common classroom activities (asking questions for clarification, weighing in on impromptu discussions, etc.). Online, these opportunities to feel connected and learn from each other do not occur naturally; they must be carefully planned and managed. Assessments that incorporate student-to-student interactions—while not appropriate for every course—can play a powerful part in a course’s overall communication strategy:

  • Peer-review. Asking students to review their classmates’ work (and grading them on their reviews) can help motivate best efforts as well as help students learn from each other.
  • Group projects. Well-designed group projects help students master both course content and team participation skills.

Online students need more student-to-instructor interaction than their face-to-face counterparts.

While instructor interaction and feedback is important to all students, it’s critical to the success of online learners and—like student-to-student feedback—must be carefully planned and cultivated in an online setting.

  • Frequent, low-stakes testing (“self-check” quizzes and activities). Options range from short selected response quizzes and watch-and-discuss questions to complex games and activities accessed through textbook publishers’ add-on course cartridges.
  • Rich, detailed feedback. Strategies for providing rich, detailed feedback vary based on the activities you’ve selected for your online course. The most practical include building detailed feedback into selected response quizzes; actively managing discussion boards; and administering weekly surveys asking students to identify the concept(s) they’re struggling with and then addressing the most-identified concept(s) by using a product such as Jing to create and post quick video clarifications.

Online students need more planned structure—that is, more help in staying on time and on task—than their face-to-face counterparts. The structure that occurs naturally when students and instructor congregate in the same place at the same time (seeing “Test next Tuesday” written on the white board, for example, or overhearing classmates a row over discussing an upcoming assignment) doesn’t just “happen” in an online course; it must be planned and managed. These assessment-related techniques can help:

  • Frequent, low-stakes tests (ungraded or low-point-value “self-checks”) help students gauge for themselves how well they’re mastering the material.
  • Graded milestones. Breaking up large projects into smaller graded milestones helps students (and you) identify problems areas early enough to address them.
  • Graded participation. Using a rubric to grade discussion board participation is time-intensive, as is asking students to review each other’s work. However, the benefits in terms of being able to gauge and guide students’ understanding can often be worth the time investment.

It’s important to note that while implementation strategies differ in an online environment, the pedagogical strategies underlying assessments shouldn’t change when you move a face-to-face course online. Online or off, assessments need to align with course objectives and provide a consistent measurement of student learning. If having students hand-write math proofs so that you can assess understanding and grant partial credit makes sense in a classroom, for example, chances are it makes sense online, too. The pedagogical strategy (hand-written homework) stays the same; only the implementation strategy (having students create and sending handwritten work as a digital file) needs to change.

References:

Watson, G. & Sottile, J. (2010). Cheating in the Digital Age: Do Students Cheat More in Online Courses? Online Journal of Distance Learning Administration, Volume XIII, Number 1. Retrieved online October 1, 2013 from http://www.westga.edu/~distance/ojdla/spring131/watson131.html 

(adapted from: https://www.facultyfocus.com/articles/online-education/7-assessment-challenges-of-moving-your-course-online-solutions/)

Our learning management system, Moodle, is designed to distribute, collect and store a wide variety of student artifacts and entries. The system is very secure, in that students are only able to see their own work and grades (unless the instructor sets up group activities). Both Moodle Assignments and Quizzes allow the instructor to accommodate accessibility concerns with user or group overrides that can give extra time or access for specified students.

Moodle Assignments - use a Moodle Assignment for a graded essay or final report.

Moodle Quizzes - use a Moodle quiz for a quiz or final exam. Question types include multiple choice, essay, true/false, matching, short answer, plus several other options.

Kaltura Video Quizzes - use a Kaltura video quiz if you want to assess completion and comprehension of a required video assignment. To maximize viewing time, it is recommended to not allow skipping ahead in the video (a recommended setting), along with adding a quiz question near the end of the video.

Graded Discussion Forums - use a graded discussion forum if you would like to assess student asynchronous participation in a discussion forum.