Assessment in Online Learning | WeCanEduTech
- Jason Matthew
- Jun 16, 2020
- 37 min read
Updated: Jan 13, 2021
Table of Contents
Introduction
If you ask students what their views are on assessment, they will most likely say it SUCKS!!! And to be honest, educators feel the same way – assessments are a pain to create and grade, and most see it as a necessary evil that the institutions that are paying their salaries demand.
But what if I tell you that assessments can play a critical role in learning and teaching.
Misconceptions and common mistakes
'If I am good at teaching, then I must be good at assessing…. Right?'
Just as teaching was a skill that you learned and improved upon with practice, so too is assessment. Unfortunately, many institutions and the educators themselves do not recognize this, and the proper support is either not provided or sought after.
How many of us have created, or should I say collated assessments that consisted predominantly of multiple choice and True and False questions? And how many of these questions came from question banks that publishers provided with the textbooks? Guess what - from my experience, the students have 'acquired' a copy of the textbook's question bank.
You need to consider the following:
Is this the best choice of assessment strategy?
And if it's the best choice, what about the quality?
Should this be used as a knowledge check rather than a summative assessment?
I get it. Assessments are a pain to create and mark. The idea of using ready-made questions and then letting software such as a Learning Management System (LMS) automatically score it is too good to pass up, especially if you have the mindset that assessment is a necessary evil.
Educators, especially those who are new to the online modality, often fall victim to the misconception of trying to duplicate what they do in the face to face classroom on the online platform; assessment strategies included. I am not saying that you can't get good results with the traditional midterm and final exam format. But, we have to be careful that we don't upload the worst aspects of traditional assessment onto the online platform.
As a Moodle (LMS) administrator, I know only too well the logistic nightmares of having high stakes traditional exams online. What happens if there is a power disruption, the internet drops, the software glitches? Do we give the student a second chance? And if so, How much time should be given to complete the exam a second time? Do we give the student a completely new exam? How do we know for sure that the student's claim is legitimate?
Also, traditional face to face assessment requires invigilation; that is all the students are in one physical space at the same time, and the assessment is supervised to prevent cheating. This continues to be a major challenge to replicate in the online setting. One option is to have all the students do the assessment in a computer lab on Campus. But this option comes with technical and logistic challenges, and some would argue, contradicts the very reason learners choose to do online courses. Also, this will not be an option if the Campus is closed due to a pandemic.
Okay, so if we can't have the learners in one place simultaneously, how do we maintain academic honesty if we want to have a midterm and final exam online? This is where things get interesting, and we will probably have a laugh or two in the future as we look back on what we tried. In some cases we require the student to have a camera on them and a mirror behind them while the poor educator monitors from his screen for any signs of dishonesty. Some Learning Management Systems offer a safe exam browser where students can do the exam in that browser. The safe browser prevents students from opening other windows or checking other URLs once the exam begins. Some universities have even employed expensive proctoring software to validate students' identities and monitor students' behavior during the exam.
Unfortunately, students who want to cheat are beating the systems that we are implementing. There is a need for us to rethink our assessment strategies in the online ecosystem. Educators need to develop authentic assessment strategies that allow the learner to engage with the material deeply and personally.
Now that we have established what we are doing wrong, let's figure out how to do it right. First, we are going to look inward and figure out who we are as teachers, then we will discuss the qualities of good online assessment, and finally, we will look at strategies to authentically engage our learners.
What guides assessment creation?
There are so many online technologies out there for educators to choose from. These technologies offer so many wonderful affordances that only being online can provide that sometimes educators are seduced or pressured into letting the technology guide what kind of assessment strategy should be employed. The reality is whether you are in the face to face setting or in an online modality, the assessment strategies you will want to use should only be guided by pedagogical principles.
What is your teaching philosophy? How do you think learning occurs? There are many possible learning theories to choose from such as behaviorism, cognitivism, constructivism to name a few. Once you figure out your teaching philosophy, it gives you better insight into why you teach the way you do. It will also guide you on what criteria you would want in your assessment strategies.
The next step is to start to draft out the type of assessment you want. This can be done using a word processor or in your idea notebook. The intention here is to create a blueprint for the type of assessment you want to design. Only when you have completed your blueprint, then you consider what technologies you have, or will need, to bring your vision for the assessment to life. You can always revise your design if some of the things you initially planned are not possible, but this strategy will yield interesting and creative assessments.
What can technology do for assessment?
Online assessments have many advantages over the paper and pencil exam in the face to face setting. Online assessment can incorporate audio, video, and animation, which can produce project-based assessments that motivate learners to use their creativity and innovation. Other benefits of online assessments:
Immediate feedback for students
Robust plagiarism checker such as TurnitIn
Submission, marking, and feedback can be done from anywhere, anytime
Automated marking and feedback
Grades can be managed in an online gradebook
Reduce the use of paper
What comes first – the content or the assessment?
What is the first thing that you usually focus on when designing a course, whether it is online or face to face? If we did a survey, we would most likely see that the modal theme would be content-driven. This might be instinctive, but instructors, especially novice educators, tend to focus on getting or producing the best content for their course, and significantly less consideration is given for assessment. Often this content-driven approach results in a bloated course that, instead of engaging your students, does the opposite. Another option is to adopt Jay McTighe and Grant Wiggins 'Backward Design' framework.
The backward design framework is divided into three stages. First, you establish the learning goals for the course. In other words, you start with the desired result. Then you decide what assessment tasks and criteria will be used so that students demonstrate that they have achieved the desired learning outcomes. Finally, you create individual learning activities and instructional strategies that will be used.
Learners are more motivated and engaged when they can align the lectures, tutorials, practicals and learning resources, and the course assessments. The way instructors can accomplish this level of integration is by using a backward design framework.
Qualities of good online assessment
The qualities of good online assessment, I describe in this section are by no means an exhaustive list. Neither do I expect for you to apply everything I am suggesting or even completely agree with my list. So please feel free to share your thoughts and ideas in the comments section of the blog below.
Choice and Flexibility
Students, in particular, adult learners, are more motivated and engaged in courses that are flexible and allow them to make choices. Allowing learners to have input in the course design, including the assessment, is crucial for our students to become autonomous and independent learners.
Clear instructions
Students may not have the level of access to their teachers in the online setting as they do in the face to face setting. The assessment instructions must be as unambiguous and detailed as possible but concise enough that the learners will read them.
Formative Vs. Summative
There are two main types of assessment - formative and summative. If you consider formative assessment like an annual medical checkup, then that would make the summative assessment the autopsy. Sure the autopsy is essential. The police, family, and friends would want to know the cause of death. The whys, the hows, prevention strategies will be discussed. But the person who is dead would have benefited from regular and robust checkups. He would probably have made positive lifestyle changes based on the checkups' results and achieved a more favorable outcome.
A common form of formative assessment I use is to provide knowledge check activities throughout the course. The activities typically consist of MCQ questions from the test banks that come with the textbooks. These quizzes are set up so that the learner has unlimited attempts, and the Learning Management System gives immediate feedback. The scores do not contribute to the final grade in most cases, and if it does, it is part of an overall low stake participation mark.
Some other examples of formative assessment include:
Elevator speeches
Concept maps
Polls
One minute papers
Summative assessments tend to have the word final in them - final paper, final project, final exam, etc. Summative assessments tend to be the traditional forms of assessment that we are familiar with. When I am faced with a situation where a course must have a high stake summative assessment, I tend to make it both formative and summative. For example, suppose my students have a final project to submit at the end of the semester. In that case, I create opportunities for them to submit deliverables along the semester to give feedback. That way, they can learn from their mistakes, consider my advice, and submit a much better project at the end. It also has the added bonus that this method also makes it easier for me to mark at the end as, by that time, I am very familiar with the project, and I have an idea of what their grade will be. The formative deliverables along the semester are not graded, and students can choose whether to submit or not. From my experience, once you explain to your students the benefits of working on the project and submitting the deliverables for feedback, you do get buy-in from them.
There are challenges when using either formative or summative assessments. Formative assessments, in most cases, tend to be optional or not for marks. So you might be concerned whether your learners will do them and if they do, what will be the quality of their answers. Strategies that work for me include explaining to learners the importance of formative assessments. If possible, share testimonials of past students. You might also want to consider having the formative assessments contribute to an overall participation mark of the final grade.
Summative assessments are a bit different. For one, your students will be motivated to do it. However, we could segway into the intrinsic vs. extrinsic motivation conversation. But that's for a separate post. One of the major challenges of summative assessment is that the learner and the academic institution may view it as a means to an end. In most cases, the summative assessment is not returned to students due to exam policy, or sometimes the students don't collect them. And I don't blame the students as they have to collect them during their vacation period or at the start of the next semester. It's a missed opportunity for students to use summative assessment feedback to improve their learning. Because let's be real, the grade we tend to focus on is not the sole indication of the learning level achieved. The grade results from many factors, including student's effort/student's cheating, student's procrastination, the teacher's effort, quality of the assessment, etc. I have indicated previously that I tend to incorporate formative components in my summative assessment so there is a mixture of assessment of and for learning. As you read on, we will explore different assessment strategies that move away from the traditional forms of assessment and thrive on the affordances that the online modality provides.
Feedback
There are different avenues for feedback in a course. There is general course feedback, student feedback, etc. In this section, I am specifically looking at assessment feedback from the teacher.
Good instructor feedback is an essential component of student success. This feedback clarifies what good performance is for the student and provides opportunities to close the gap between current and desired performance. It encourages your learners to reflect on their learning. By giving your learners good feedback, it promotes discussions between you and your students about learning.
So what is good feedback?
I want to unpack what I mean by timely feedback. The general rule is, the quicker, the better. Best practice recommends a 24-48 hour turn around if you want your feedback to affect your student's motivation and engagement positively. This will apply to most cases. But you also have to consider the complexity of the assessment and class size as these factors will affect your response time. The idea is to always keep the general rule in mind while at the same time setting realistic deadlines on when feedback will be given.
Based on the community of inquiry model, providing frequent high-quality feedback increases your teacher presence in an online course. I know some teachers, including myself, have found this challenging, especially when faced with high teaching loads and large class sizes. And I want to say it is doable, but you have to be smart about how you do it and use the online modality's digital affordances to your advantage. I wish someone had given me these tips when I started.
In my earlier innocent years of teaching, I would go through each student's assessment and type out my feedback. That's hard and very time consuming and, if I am sincere, the feedback was not of the best quality, especially after you do a few. And I taught year one undergrad, so my classes were huge. Later on, I started giving my feedback as audio clips. That had a dramatic effect on my feedback quality, and my students benefited significantly from it. At least that's what they said in their feedback - no pun intended. The advantage of recording my voice over typing is that I was more relaxed, I used a conversational tone, and it was easier for me to expand on my feedback. So I produced rich high-quality feedback in much less time than it would take to type something inferior. My students also felt that I was more present in the course when they heard my voice rather than just reading text that I typed up. Nowadays, most LMS allows you to give video feedback as well. I have not used it that much simply because I don't think the increase of file size of a video file vs. an audio file justifies a possible minimal increase in teacher presence or other possible benefits.
Here are some other feedback strategies that you can use in your online teaching.
Authentic assessment
Some would say that we need to rethink the use of traditional assessment methods in the online modality. And this is true. The online setting allows us to do things differently. But were the traditional forms of assessment ever really effective at doing the job it intended to do, or has it served a more bureaucratic purpose?
As technology advances, our best students won't be the ones who are good at regurgitating factual knowledge. Instead, learners will be required to apply their learning to arrive at creative solutions to complex problems.
Authentic assessments are ill-defined problems that mimic real-world tasks undertaken by professionals. The learner is in a safe environment where they can create new knowledge and multiple points of view on how to solve real-life problems. Authentic assessments encourage learners to work collaboratively and to think critically. Learners play an active role and are engaged and motivated.
Authentic assessments are particularly effective amongst adult learners. Most adult learners pursuing online learning are also working. These types of learners are continually budgeting between their family, studies, and work-life. For these students to complete their studies, they have to remain engaged and motivated. One way to keep them engaged is to have them use their prior knowledge and work experience to solve real work tasks they can use in their current position at work or in the place of employment they want to achieve.
For your assessment to be authentic, it should have the following characteristics.
Authentic assessments promote a culture of learning and not a culture of earning. The assessment tasks encourage the learner to move away from surface learning and instead take a deep approach to learning where they apply their knowledge and skills in new contexts and creative ways.
I encourage you to design authentic assessments where learners design or create something. My students and I tend to have a lot of fun with these types of assessments and we both learn from the experience. Furthermore, learners can include the artifacts they create from these assessments in an electronic portfolio which they can showcase to future employers.
Academic institutions benefit from the use of authentic assessments. Earlier on, I spoke of the challenge of academic dishonesty when traditional forms of assessments are used in the online setting. It is much more difficult and would require significantly more time and energy to cheat in an authentic assessment. There is also mounting external pressure for educational institutions especially universities to produce graduates that are ready for the world of work. An authentic assessment is an effective way of measuring these types of outcomes.
In the rest of the blog post, we will explore different online assessment strategies that can provide your learners with opportunities to authentically engage with their learning.
Group work
I remembered hating group work as an undergraduate. However, when I did my graduate studies, I appreciated what working in a group brings to the table. This section will discuss what worked and did not work for me both as a student and as an instructor.
Let's start with the good.
One of the principles behind group work is that learners would produce solutions that they would not have been able to derive on their own at the end of the project.
Now we get to the bad and ugly.
Fortunately, there are strategies you can employ to overcome these challenges.
Due to prior negative experiences, many of your learners will have great antipathy towards the idea of working in groups. Just telling your learners that this time will be different will not work. A good strategy will be to ask your learners to tell you about their reasons for having reservations working in groups. Then you explain what you have implemented in your assessment design that would mitigate these concerns. You should also explain the learning benefits of working in a group vs. going solo. There will always be at least one person who will request to do the work independently, so it's best to address it upfront.
Learners sometimes struggle with the soft skills required for working in a group. As the instructor, you may not feel confident to address this or even want to use class time for it. I would recommend that you do, as not addressing these gaps can cause loss of morale and conflict amongst group members. You should at least direct your learners to resources that cover communication, conflict resolution, leadership, time management, etc. skills. You can also provide learners with scenarios of issues that came up amongst group members in the past and how they were resolved.
You must make the expectations of the project clear at the beginning. Create deliverables, which groups submit during the duration of the project, instead of one final submission. This will assist students with proper time management of the project. Also indicate which steps in the project will require the most time so that learners get the heads up and can plan appropriately. Many online students are budgeting work and family with school, and it can be a challenge to find a time for all group participants to be free to work on the project. If you have synchronous sessions with your class then you should dedicate some class time for the students to work on their project. In an asynchronous online course you will need to send frequent announcements concerning the project.
Now we come to the most destructive and unfortunately common challenge presented in group work. What do you do to prevent situations where group members leave most or all of the work to a few more diligent members? The free riders and social loafers. There are many reasons why this can happen - lack of accountability, the group member is not engaged, not motivated etc. Fortunately there are a few strategies you can implement that can help:
Create small groups. Groups should consist of 3 - 5 members. You run the risk that students will not have enough to do or it makes it easier for a student to 'hide' when group sizes are large. The larger the group, the larger the coordination burden on the group. You have more schedules to work around, more opinions to consider, more contributions to integrate.
Assign (either by you or even better negotiated among the group members) specific roles for each group member, such as leader, reporter, communicator, etc. These are the typical roles assigned to group members. I would also recommend that you push your students to think outside the box by including group member roles such as devil's advocate, doubter, the clueless one. Depending on the project's length, you should also rotate these roles within the group, so every member gets a chance in the role.
Ensure that group members are accountable. Have groups submit weekly reports indicating what each group member has done. Make it clear at the beginning of the project that free riders will not be tolerated, and they will be penalized independently of the other group members. Let your students know that they can confidentially report non-performing students and that the matter will be investigated. Likewise it is important to let your students know that they should come to you as soon as possible to discuss any reasons why they might not be participating in a group so that early intervention can be made. An individual might not be performing in a group for many reasons, and we should not assume that we know why. It is essential to discuss this with the student so the appropriate support could be given.
Employ self and peer assessment strategies. This will be discussed next.
There is a lot more to be said about group work. So please add your comments, recommendations, and share your experiences in the comment section.
Self-assessment
As indicated by the graphic above, self-assessment is difficult. It makes learners nervous and uncomfortable.
So what makes self-assessment so challenging? After all, the assessment is about me. And who will know me better than me? It works out that it has to do a lot with what makes us human and our sense of identity. I will encourage you to read the work of Goffman(1959), Bandura(1986, 1971), Snyder (1987), Turkle (1997), and Wenger (1998) if you want to explore this fascinating topic further. I have provided full references at the end of this post.
You can't just throw in a self-assessment activity and expect meaningful results. It has to be organically incorporated in the course design to place the learner on the path of critical reflection and growth, which ultimately leads the learner towards self-directed learning.
For self-assessment to work, the learning environment that you have designed has to provide the following:
Opportunities for knowledge building and knowledge sharing.
Frequent formative constructive feedback.
A common criticism of self-assessment is that there is a disconnect between what the learner thinks he knows and what the learner actually knows. The disconnect tends to be greatest among students in introductory undergraduate courses, while a better correlation is observed with graduate students. Self-assessment should not be used as an evaluative tool i.e., count towards the final grade, and you would want to consider if your course is right for it.
Self-assessment can be a useful learning strategy, and you would want to consider the following if you decide to employ it in your course:
Be considerate of your learners' reluctance. Be open to discuss any questions and concerns the learner may have about it.
Explain the rationale and strategy of the assessment.
Provide clear guidelines and criteria.
Give feedback and support.
Peer assessment
Peer review has become a staple in online courses, and rightfully so.
Discussion forums
Discussion forums are another staple in online courses. And for good reason. Discussion forums facilitate learners' deeper exploration and understanding of topics, and it gives them the opportunities to grow their communication and team work skills.
A significant advantage of asynchronous discussion forums over face to face real-time discussions is that the student has time to think about the question and provide an in-depth answer. The online asynchronous discussion forum allows learners to rewind the conversation and properly analyze other learner contributions before contributing. Discussion forums provide a safe environment for knowledge sharing and knowledge creation.
As with any other activity, you would want to provide clear instructions and expectations to your learners. What is the deadline? How many of their classmates' posts do they have to respond to? What is the word limit for posts? Will there be peer assessment? Provide rubrics or other grading criteria if applicable. Give clear guidelines on acceptable behavior in the online forum and how they should respond to their classmates. Some instructors ask their students to use the RISE model when giving peer feedback.
Learners should be given an early opportunity to get familiar with using discussion forums. This can be in the form of an introductions discussion forum or an ice breaker activity. This way, the learner can practice using a discussion forum without any penalty.
You would want to employ a many to many approach when posting public feedback. Identify the main themes in the forum and encourage interaction amongst learners in the forum. You can make it mandatory that learners reply to other learners' posts. You should not direct feedback to specific students publicly. In most LMS such as Moodle, you can respond to a learner's post privately.
You want your learners to feel relaxed and be focused on discussing ideas. Use a friendly and casual tone. Forego the formality and academic rigor of requiring students to provide references and reference format and all that jazz. There will be other opportunities in the course for your learners to demonstrate those kinds of academic conventions. In the discussion forum, you want your students to explore and discuss.
You will need to moderate the discussion forums frequently. There will be instances where learners may veer off-topic, and you will need to bring them back to what they should be discussing. You can also assign students the role of moderator and rotate it. You will also need to be vigilant about unacceptable behavior on the forum and deal with it expeditiously.
As an instructor, my go-to type of forum on Moodle is the Question and Answer forum. I like the idea that my student will see the question/discussion prompt but will only see what the other learners have posted after she has posted. In this way, my learner's response is not influenced by what other learners have said, but once the learner has given their response, they can dive straight in the discussion with the rest of the learners.
Some challenges to look out for with discussion forums:
Students spend too much time refining their answers that they post close to the deadline and leave little time for responding to other students. This defeats the purpose of the discussion forum.
Students experience discussion forum burn out. There are too many discussion forums in the course.
To maximize the impact of discussion forums on our courses, we must ensure that discussion forums are well integrated with the learning process. That is, it is the best tool for the learning activity it is intended to serve. Discussion forums are great for assessing conceptual understanding but not the right choice for reflections where a blog or journal will be better suited. So that is my general advice on how to make your discussion forums have optimal impact. To address the specific challenges I mention above you may want to consider setting two deadlines. The first deadline is for the learner to upload their post and the second deadline is to reply to other learners.
Social media
You should not just limit yourself to the technical affordances of your Learning Management System. Teachers have created some cool and interesting assessment strategies using social media such as Facebook, YouTube, Twitter, blogs, wikis, and LinkedIn. And the list continues to grow. For instance, students are now saying that TikTok and the Gram are where all the action is.
Most likely, you have experience using some type of social media, so you are well aware that you can design very creative assessments using social media's unique affordances. But incorporating social media in our assessment strategies does come with some danger and challenges. We must be careful that we do not forget the pedagogical intent of the assessment. As I have said before, you should be guided by pedagogical principles when designing assessment and not the technology's bells and whistles. One way to remain on task is to have explicit instructions and appropriate assessment criteria to guide your learners. Also, grading these types of assessments will challenge the way we usually grade. In traditional assessments we will consider things like word limit, grammar, etc. But what mechanics will we assess if we want our learners to produce authentic social media assets? And how does one measure another's creativity? Should you, the teacher, be making all the assessment criteria decisions? I have found that there is greatest student buy-in when they can negotiate the assessment criteria with you. Finally, one of the most powerful social media features is its ability to share and for others to see your work and comment on it. You would want to explain this to your learners and allow them to decide the level of visibility they want for their work.
As I have hinted above, there is a wide range of social media options at your fingertips. Today I will briefly discuss the two most common ones used in assessment - the blog and the wiki.
A blog can be described as a website consisting of several posts organized by the date the posts were made. Blogs foster collaboration, self-expression and exploration, and reflective learning. Students can use blogs to create and share knowledge, and you can use blogging as a form of formative assessment where you assess your learners' growth in the course and provide real-time guidance and intervention when needed. Depending on the assignment's nature, learners can use their blog to network with professionals in their field of study to get feedback on their work. You can guide learners on how to use blogs to develop their professional identity. On a personal level, I have seen students who are shy and introverted use blogging to express themselves (I allow my students to use aliases when blogging so that I am the only one in the class who knows their real identity).
Just as a blog, a wiki can be described as a website but this time the posts are organized by content. Think of Wikipedia, that's probably the best-known example of a wiki. Now, if you understand how Wikipedia works then, you already appreciate the potential of using wikis in your online assessment. In Wikipedia, authors contribute knowledge to a topic globally. And the quality of knowledge is verified by the Wikipedia community. Wikis are excellent at facilitating group collaboration and embraces the social constructivist side of learning. As the instructor, you can easily monitor each student's contribution and see the network of ideas and online resources evolve in front of you.
Social media has not become this global phenomenon because of its education appeal. So as discussed before it is very important that we keep our learners on task. There are other considerations you will have to take into account before you start incorporating social media. First thing you would want to do is to find out what your institution's guidelines for using social media are. In fact there are some educational institutions where social media is totally banned. Also your Learning Management System such as Moodle, Canvas, etc. will allow you to create blog and wiki assignments within the LMS itself. So you would need to decide whether you want to take that approach or use a commercial product. The main advantage a commercial product will have over your LMS is it will be sexier and have more bells and whistles than what the LMS can give, and your students will most likely be more motivated to use the commercial product. These commercial products may come across as 'free' to use but by signing up on these commercial platforms, you are most likely giving these companies the okay to use your data. Again check your institution's policies, especially on how your student's identity can be used. I write this post with a higher education slant so the students will be young adults, but if you are reading this and teaching minors, then the restrictions on what you can do will be even stricter.
Rubrics
Rubrics guide learners on how to succeed in an assessment. A well-designed rubric will give your students a clear idea of what is expected from them and demonstrate the transition from a weak response to an excellent one. It provides learners an opportunity for self-assessment and reflection. Well, designed rubrics provide student feedback that is objective and consistent, which should reduce student queries. Rubrics can reduce the amount of time spent grading, especially if the rubric has been added to the activity in the Learning Management System.
You will not enjoy any of the benefits stated above if your approach to rubrics is that it is a required add-on to an assessment or you use a generic template.
There are two main categories of rubrics - holistic and analytic. For this post, I will focus on the analytic rubric type as that one gives better formative feedback. I encourage you to read up about the two types and their pros and cons.
Rubrics are typically created as a table and consist of:
The criteria that are being assessed (row headings)
Performance levels (column headings)
Descriptors (within each box of the table)
ePortfolios
Using portfolios as a form of assessment is nothing new. The ePortfolio just lets you up your game as your learners can do so much more with the digital affordances such as the inclusion of video and audio with little or no expense. I don't know about you, but I prefer to grade organic, media-rich products over a stack of papers at the end of the semester.
ePortfolios provide meaningful documentation of your learner's abilities and is a record of their learning, growth and change. So you should consider an ePortfolio as not only a product consisting of a collection of digital knowledge artifacts but as a process as well where the learner reflects on creating these artifacts and the overall ePortfolio. Your learners will have to tap into their higher-order thinking skills (shout out to Bloom) to create their portfolios. It is not only an opportunity for deep learning for the students but it allows us to see 'behind the curtain' of our learner's thinking.
Being a social constructivist at heart, I like that ePortfolios allow students to construct their knowledge and reflect about their learning. The ePortfolio tells the story of the student's learning journey. We (teachers, mentors, peers, family members) can engage with the learner by commenting on the student's artifacts and reflections.
Another benefit of an ePortfolio assessment is that it is extremely difficult to cheat due to the long-term effort required in creating an ePortfolio. A student may get someone to do their MCQ quiz or essay for them, but it is unlikely that they will get someone to commit to a long-term project, especially one with formative requirements.
There are many reasons why I would recommend that you give ePortfolios a go. But as with anything that can provide a lot of benefits, so too will it have challenges. Here are some of the main challenges you and your students may face:
Student buy-in. Students may see this form of assessment as 'just another assignment'.
Grading an ePortfolio may be subjective and inconsistent.
Creating an ePortfolio is labor-intensive and time-consuming.
Students are more focused on higher stakes exams in the course.
There is a lack of technical support and assistance.
Students are not sure what they have to do.
Learners cannot access their ePortfolios after they graduate.
Well designed ePortfolios are a great form of assessment. Following the best practices provided below will help your learners engage with their ePortfolios in an authentic way, keep them motivated, and overcome the common challenges they may face when creating ePortfolios.
We should not only see the ePortfolios that our learners create as academic assets, but we should also view them as tools that our students will use in their transition into the world of work. If you agree with me on this then we must ensure that our learners have ownership and access to their ePortfolios even after graduating from the academic institution. This is very important to remember when creating ePortfolios using software within the LMS itself.
The use of ePortfolios continues to grow in popularity in higher education, especially in graduate programs, where it has replaced comprehensive exams. This has not gone unnoticed by software developers, and there is a wide variety of platforms available today to create and host ePortfolios. I have included a list of common software used to create ePortfolios in the additional references section below. Some of these software programs will be free while the more ePortfolio specific software comes with pricing. There are some that are open source and integrate well with your Learning Management System. I would recommend that you spend some time using the different options and see which software best suits your needs. Even the software you have to pay for, should have a free trial offer that will allow you to get a feel of the software. Your student needs, the learning outcomes, the user experience, budget, privacy, and security are some of the things that you should consider when deciding on an ePortfolio platform. Also, are the ePortfolios going to be course-specific, or is it an institution or program-wide initiative? This will also influence the choice of software to use.
Reflective journal
The idea of our learners making sense of their learning through reflection is something I have mentioned throughout this blog post, especially in the self-assessment, blogs, and ePortfolios sections.
Most online instructional design is based on some kind of social constructivist framework. And sometimes, we can be so focused on the social aspect of constructing knowledge that we don't provide enough space for our learners to develop their metacognitive skills. Having your students create reflective journals is one such activity that can help them make sense of their learning.
You are going to get push back from your learners if you ask them to reflect. My advice on how to address this is similar to what I recommend for other activities:
Give clear instructions and expectations.
State the benefits of doing this activity.
Discuss the activity with your students and address their questions and concerns.
Just recently, one of my learners indicated her reservations about doing a reflective journal. She thought that her reflections were personal and should not be shared with anyone. Fortunately, I was following my own recommendation of addressing students' concerns and questions before the start of the assignment and in that particular case I explained the purpose and benefits of the reflective journal and that the reflections would only be read by me. The student found this acceptable and has subsequently done the activity.
High stakes or low stakes? I tend to make my reflection activities account for a low percentage of the final mark. I try to reduce the pressure of marks so that my learners feel free to express themselves in a genuine manner instead of trying to tell me what they think I want to hear. There is a drawback to this approach, and that is that some students feel a reflective journal is too labor-intensive and time consuming for the marks allotted and are tempted not to give the activity the attention it deserves. And then some students love the activity and are motivated to do it. Also, from your end it could be a challenge to monitor this activity, especially if you have large classes. So you may want to consider giving your students options such as reflecting on the entire course or parts of the course or activities that you think are significant.
When to give feedback? This is a tricky one. Normally a reflective journal will run the entire duration of the course. You don't want to wait till the end of the course to give feedback as it will most likely be the least helpful for your student, but you don't want to intervene too early and deny your learner the opportunity to figure it out themselves. Trust in the Force. Let your teacher instincts and experience guide you.
Scaffolding. Common misconceptions students have doing this type of activity are that they feel like the reflective journal is a diary and have to give a daily log of their actions or an avenue for venting. It is vital to guide your students, especially those who are doing this type of activity for the first time. This will help them stay on track. I have included some common reflective type questions that students can answer when making entries in their reflective journal. Feel free to use and adapt the questions I suggest as you see fit.
What's in it for you? It is a win-win situation when a reflective activity is properly executed. Your learners get a better sense of their learning and why they do things the way they do. With that insight, they can make the necessary adjustments to improve. As the instructor, you are there for the journey, and the experience allows you to help your learners in profound ways and lets you make adjustments to help your future learners.
The world is open
This is a topic that is very close to my heart. I believe that education should be open and available to all. At WeCanEduTech we walk the talk and are committed to making all our learning resources freely available to everyone. We are always open to networking with people with similar philosophies.
You should not feel that you have to create everything from scratch, especially as you are teaching online, and digital resources can be easily shared and repurposed. There are many incredible, high-quality open learning resources and activities out there that you can use for your courses. Fortunately, the Open Education Resource movement is strong and continues to grow in popularity and support.
But before you take that deep dive in the open world, there are a few things you should be cognizant about. Firstly and most importantly, just because something is online and you have access to it does not mean free to use. Everything online has some form of copyright. Most legitimate, open education resources will be under some type of Creative Commons license. Creative Commons is an international non-profit organization that provides different types of free licenses that allow others to legally share and sometimes even remix material depending on the license type. Be sure to understand and comply with that type of license's conditions if you are using a resource that is under a Creative Commons license.
Do the research. Ignorance is not an excuse for breaking copyright. Start by sourcing the copyright regulations of your institution. Ensure that the resource or activity you want to use is from a reputable source and the true owners of the copyright. It is possible that someone can post another's material online and say it is theirs and free to use. So always err on the side of caution and don't use it when you can't verify the source or have any doubts. Ensure that you read and comply with the conditions of the copyright license.
I encourage you all to not only be consumers but also producers and sharers and be part of the open movement.
Badges
The use of digital badges continues to gain momentum in higher education. Within your Learning Management System you can create badges that represent a variety of achievements ranging from mundane things such as uploading a profile picture and representing more complex criteria such as engagement, collaboration, and inclusion.
Badges can be assigned at the course level or the site (program) level. Badges can be awarded for completing specific tasks or acquiring specific levels of competencies and skills. Badges allow you to assess learning in chunks and provides multiple opportunities for educational institutions to provide visible claims of student learning.
Badges are transparent. The date of issue and the criteria or outcomes achieved to get them are available. Multiple educational and professional institutions can endorse badges, and the badges are portable, meaning that students can add and display them on an open backpack software system. A link to the collection of badges can be shared with potential employers or on graduate school admissions. Supporters of digital badges predict that badges will play an influential role in graduate school admissions and job applications in the future and even revolutionize the format of transcripts. We'll see.
Please share your experience in the comments section below if you have used badges in your courses. What were the challenges? Were your students motivated by it? Is it something that you would recommend to others, and why?
Serious games
A serious game is a game whose primary role is to go beyond entertainment and teach something, whether it be knowledge or skills or awareness, or even produce an attitudinal change. Serious games can be:
Educational games
Health games
Military games
Political games
Simulations
Serious games can be an exciting assessment strategy as it can incorporate all the qualities of good assessment design. Serious games are very effective at generating high levels of intrinsic motivation and positive emotional experiences.
So if games for learning are so great, why is it not more widespread? The major challenge is creating a game requires advanced technical skills such as coding and a significant financial investment. Fortunately, there are developers out there who are altruistically making their games open source. Unfortunately, to customize the game to match your outcomes would require some coding on your part. Most of the games for learning developed so far, especially the free ones, tend to be simple and cover lower-order thinking skills. There are some really great simulations that train doctors, nurses, pilots, and soldiers, but they are proprietary software.
Presently creating a serious game has a steep learning curve and requires significant resources. Fortunately, technology tends to evolve in a way that tries to organically convert the typical user from a consumer to a producer. Remember, when only people with a particular skill set could create websites. Nowadays, with a few clicks using software such as Google Sites and Wix you can make a reasonably decent website. For FREE!. I predict that the same will eventually happen with the development of serious games, and the learning and assessment landscape will be the new arcade.
Learning analytics
Learning analytics is not a form of assessment per se, but it can lead to formative observations that can inform teaching practice and optimize the online learning environment.
Your learning management system can capture a lot of data about your students. You can apply learning analytics to analyze that data to answer questions about your students, such as:
Are they using the online technologies as intended?
Are they engaging with the assessments as designed?
Which aspects of the course do they appear to have the greatest difficulty with?
Which of them are at risk of dropping out?
Which of them are at risk of failing the course?
Are they experiencing non-academic difficulties?
What are the common trends of successful students?
Which resources are being used by students that get high grades?
How successful have your learning analytics informed interventions been?
There are other ways in which you can probably capture some of this information. You could survey the students. But there are many limitations when you use surveys such as:
The information is self-reported and, therefore, subjective.
Not everyone will answer the survey, and you may be getting an unrealistic representation of the population.
Students won't want to do many surveys. You might get away with doing two surveys a semester.
With learning analytics
Objective data is being collected and analyzed.
Data is collected on the entire student population.
Data is being collected continuously and can be analyzed at any time.
The major drawbacks of using learning analytics
Requires the expertise of a data scientist.
Ethical concerns
If you don't have a data scientist on your team, you have the option of subscribing to learning analytics software that will connect with your Learning Management System and crunch the data for you.
Learning analytics is a very new and exciting field. We will hear a lot more about learning analytics and its impact on teaching and learning in the years to come.
Inclusivity, equity, and accessibility
When designing our learning resources, assessments, and learning environments, we have to keep in mind that our learner population is diverse and that diversity needs to be embraced within our designs.
Diversity in your student population will be manifested in many areas:
Gender
Race
Ethnicity
Nationality
Religious beliefs
Culture
Socio-economic status
Access to devices
Access to and stability of internet connection
Learners with special education needs (both cognitive and physical)
Second-language speakers
Stable electricity supply
Assessments should be gender-responsive and free from racial, ethnic, religious, or cultural bias. As discussed previously, you should create assessments that give your students choice and flexibility. I would strongly recommend that you explore the Universal Design for Learning (UDL) educational framework to get more ideas on designing for diversity. Trying to replicate traditional high stakes summative exams online will significantly bias against learners who are on the lower end of the socio-economic spectrum, and have limited access to devices, unreliable or slow internet connection, and unreliable electrical supply. These students will benefit from authentic assessments spread out in the semester and have both formative and summative components.
You should add alt text (alternative text) to all of your images. Alt text is a brief description of the image and is a foundational web accessibility best practice.
Alt text has three main uses:
Screen readers will read out the alt text to visually impaired users.
The alt text will appear if the image cannot be loaded in the web browser for some reason.
It helps search engine crawlers better index an image.
The good news is that most Learning Management Systems make it very easy to add alt text when uploading images to it. You should see either a description box or an alt text box to enter the alt text during the process of uploading an image.
Adding closed captioning to your videos is another foundational web accessibility best practice. Closed captioning allows the user to turn on a visual display of the audio component of the video; that is, the user can read the spoken dialogue and receive visual feedback of the music and sound effects occurring in the video. Students who have a hearing impairment can still follow the video. Closed captions will also benefit students whose first language is not the language being spoken in the video. There are different ways in which you can add closed captions to your video. You can transcribe the video and then manually add the transcript to the video or you can do what I usually do, which is to upload the video to YouTube and have YouTube transcribe the video and add the closed captions automatically. From my experience, the YouTube algorithm does an excellent transcribing job.
Inclusivity, equity, and accessibility challenges in education are very complex, and there are no easy solutions. Educational institutions, government, private industry, and nonprofits will all have to work together to produce sustainable solutions.
Conclusion
Assessments have chronically been a source of stress, anxiety, and fear for learners. As educators, we can change this. Assessment should be a core pedagogical component of the learning cycle.
Here are some key takeaways:
Assessments don't have to suck.
Assessment is a learning tool.
Pedagogy first, technology second.
Be true to your teaching philosophy when creating assessments.
Design backward.
Provide choice and flexibility.
Be clear.
An assessment is only as good as the feedback it provides.
Authentic assessments are awesome.
You are not alone; the world is open.
Assessment should be accessible, equitable, and inclusive.
Not everything I have discussed in this blog may apply to your discipline of study. The size of your class may limit to some extent what you can do, and you will need to consider that. Also you need to consult with your supervisor and institution's exam policy to see what is allowed.
There are many challenges when creating assessments in tertiary education especially if you are trying to do something new. You are going to get pushback from the very system that you are trying to make better. The people responsible for exam policy at a university, our academic colleagues, were good test-takers themselves and excelled at traditional forms of assessment such as the high stake multiple choice questions and essay combo two-hour final exam. And we tend to forget that and don't appreciate that the type of student we were, represents a small subset of our student population. I encourage you to be that agent of change and work with academic administration to provide equitable, inclusive, and authentic assessments that our learners can benefit from.
This was a holistic look at assessment in online learning, so I could not take a deep dive into any one topic. Please let me know in the comments below what areas I spoke about that you would like me to discuss further. My intention was not to answer all of your questions. In fact, you probably have questions now that you did not have when you started reading the post. So let's not end the conversation here. Please let me know what you think. What you agree with and disagree with. Please share your ideas and experience. Let us learn from each other.
Some questions you can answer in the comments section below:
Do you grade all of your assessments? Why?
Have you seen an increase in the use of different types of assessment strategies at your institution?
What assessment strategies work best for you? Why do you think that is?
Do you think you have enough support to create good assessments? What do you need?
I have included the resources below, which I used to create this post. Please feel free to share your resources in the comments section below. I am looking forward to hearing from you all.
Thank you. You are AWESOME!!!
Resources
I have shared my online assessment knowledge from my experience as a tertiary educator and learning experience designer in this blog post. I would have also researched what the experts would have said on the subject matter before blogging. Please check the references section for additional readings. But before you do that, I would strongly recommend that you read Assessment Strategies for Online Learning: Engagement and Authenticity by Dianne Conrad and Jason Openo. These guys take a deep dive in online assessment strategies and report about it in an easy to understand manner. I guarantee that you will learn something new or get a different perspective on what you already know when you read their book.
Additional resources
ePortfolio resources
You can create your ePortfolios using any of the software below.
We do not recommend or advocate for any of these platforms, and a listing here should not be considered as an endorsement.
The world is Open
Accessibility
Access IT website. Tips on writing HTML with high accessibility
References
Bandura, A. (1971). Social learning theory. Morristown, NJ: General Learning
Press.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive
theory. Englewood Cliffs, NJ: Prentice-Hall.
Ciampa, K., & Gallagher, T. (2015). Blogging to enhance in-service teachers'
professional learning and development during collaborative inquiry.
Educational Technology Research and Development, 63(6), 883–913. doi:10.1007/
s11423-015-9404-7
Conrad, D., & Openo, J. (2018). Assessment Strategies for Online Learning : Engagement and Authenticity. AU Press.
Garrison, D. R., & Archer, W. (2000). A Transactional Perspective on Teaching and Learning: A Framework for Adult and Higher Education. Advances in Learning and Instruction Series. Elsevier Science, Inc., PO Box 945, New York, NY 10159-0945.
Goffman, E. (1959). The presentation of self in everyday life. Garden City, NJ:
Doubleday.
Gulikers, J. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional
framework for authentic assessment. Educational Technology Research and
Development, 52(3), 67–86.
Herrington, J., Oliver, R., & Reeves, T. C. (2006). Authentic tasks online: A
synergy among learner, task and technology. Distance Education, 27(2),
233–248.
Mansouri, S. A., & Piki, A. (2016). An exploration into the impact of blogs on
students' learning: Case studies in postgraduate business education. Innovations
in Education & Teaching International, 53(3), 260–273. doi:10.1080/14703297.2
014.997777
Moon, T. R., Brighton, C. M., Callahan, C. M., & Robinson, A. (2005).
Development of authentic assessments for the middle school classroom.
Journal of Secondary Gifted Education, 16(2–3), 119–135.
Nagel, L., & Kotzé, T. G. (2010). Supersizing e-learning: What a CoI survey
reveals about teaching presence in a large online class. The Internet and Higher
Education, 13(1–2), 45–51. doi:10.1016/j.iheduc.2009.12.001
Roberts, T., & McInnerney, J. (2007). Seven problems of online group learning
(and their solutions). International Forum of Educational Technology & Society,
10(4), 257–268.
Rust, C., O'Donovan, B., & Price, M. (2005). A social constructivist assessment
process model: How the research literature shows us this could be best
practice. Assessment & Evaluation in Higher Education, 30(3), 231–240. doi:
10.1080/02602930500063819
Snyder, M. (1987). Public appearances/private realities: The psychology of self monitoring,
New York, NY: W. H. Freeman and Company.
Spencer, B. (2004). Online adult learning. In G. Foley (Ed.), Dimensions of adult
learning: Adult education and training in a global era (pp. 189–200). Berkshire,
UK: Open University Press.
Turkle, S. (1997). Life on the screen: Identity in the age of the Internet. London, UK:
Phoenix.
Vaughan, N. D., Cleveland-Innes, M., & Garrison, D. R. (2013). Teaching in
blended learning environments: Creating and sustaining communities of inquiry.
Edmonton, AB: Athabasca University Press. Retrieved from http://www.
aupress.ca/books/120229/ebook/99Z_Vaughan_et_al_2013-Teaching_in_
Blended_Learning_Environments.pdf
Webber, K. L. (2012). The use of learner-centered assessment in US colleges and
universities. Research in Higher Education, 53(2), 201–228. doi:10.1007/s11162-
011-9245-0.
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. New
York, NY: Cambridge University Press.
Wiggins, Grant, and McTighe, Jay. (1998). Backward Design. In Understanding by Design (pp. 13-34). ASCD.
Comments