Digital assignments: How shall we grade them?

A couple of years ago, I took the decision to encourage students to submit their assignments in forms other than the traditional, paper based essay. It was about time. Should we persist in assessing students in modes of communication they may never use in the real world? I therefore decided to give them different opportunities to express their learning. Many of the modules I teach at my university deal with alternative methods of learning and teaching, and focus a great deal on new and emerging technologies. It was therefore both opportune and appropriate that I should lock assessment into the mode of learning and the subject matter. That year, one or two students from the group were courageous enough to accept my challenge, and submitted their assignments in the form of blogs. Most played safe and kept to the familiar pathway by submitting standard essays, which was not a problem. Last year, several more students submitted their assignments in blog format, and one or two created videos as their assignments. I believe it's a trend that will grow in pace. Over the next few academic years I predict that submitting assignments in alternative digital formats will become the norm. Then they won't be 'alternative' any more.

Clearly, there are several questions to contemplate here.

The first question is how do you grade these assignments, if they are not presented in traditional essay mode? On this issue, you need to agree with students prior to submission over what the assessment criteria are, and exactly for what the marks are going to be awarded. These criteria must be equalised across all the possible submission formats. How for example, have you agreed an equivalency for wordcount in a video? How should a blog be structured and sequenced, when there may be several non-linear posts contained within it? Would a hyperlink in a blog be equivalent to a reference in an essay? It may be prudent to present your students with model assignments in blog, video, wiki and other non-traditional formats so they can see what they need to be aiming at. Modelling best practice is a very powerful approach and if applied appropriately can offer cognitive apprenticeship to learners.

The second question is how can you ensure that students put the equivalent cognitive effort into say, a video, as they would into a 4000 word assignment? Can a 5 minute video contain the same level and quality of academic discussion as a 2000 word essay? Or is it an easier option? Firstly, you need yourself to be aware of what is possible within the formats and technologies that students will use. How difficult is it for example, to put a voice-over track or a music track onto a video, or overlay captions? If you don't know what the issues are, and the effort involved, you may be fooled into thinking students have worked hard (or not hard enough) to achieve the end product. Secondly, over a course of several months, it may be a good strategy to require students to create assignments in several formats, so they gain an insight into what each can afford, and acquire skills in presenting their academic ideas and arguments in several formats.

Whatever you decide to do, it will be imperative that you ensure all assessment criteria are applied equally across all assignments, no matter what wrapper they are presented in. I'm quite clear with my students. Good structure, good grammar and readability (or watchability), critical analysis and evaluation, good data application and presentation, clear arguments and acknowledgement of sources - must all be evidenced in the assignment I give to my students, in no matter what format it is presented.

There are further, procedural and administrative issues that each institution will need to deal with. What if support services cannot (or will not) accommodate the submission of non-paper assignments? What if your external examiner is unwilling to accept blogs, wikis or videos as legitimate academic evidence of learning? For the first issue, it all depends on how your admin system is set up. Usually a few words or friendly discussion with the relevant manager will be enough to adjust systems to enable admin staff to process non-linear and/or non-paper based assignments. For the second issue - I would advise that you to change your external examiner.

I'm certain this is not complete. Please feel free to add your own ideas and advice in the comments box below.

Photo from Wikimedia Commons

Creative Commons License
Digital assignments: How shall we grade them? by Steve Wheeler is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Comments

Mr. Ziebarth said…
How do you ensure students put in equivalent effort into an essay? I would argue you can't. Each student has different strengths and finds themself in different situations, with more or less time to dedicate to their assignments. As teachers weprovide meaningful feedback during their creative process, helping them reach mastery, regardless of the effort it takes.
Steve Wheeler said…
Thanks for your comment, and I suppose you're right in one sense Mr Ziebarth. I agree that assessment of learning should be about mastery of the subject and about the expression of creativity. But what happens if it's a collaborative assignment? Would you be more concerned about how we measure effort then?
Robin Martin said…
I love that you mentioned at the teacher should have some samples of each assessment form. Many times teachers assign projects such as a movie without even knowing how to make one for themselves. Teachers should have samples of every assessment they offer to their students. It also gives the teacher some idea of the length of time it takes to complete such a product.
Great post, Thanks for the ideas!
Grumpty said…
We have used blogs as a form of assessment for some time and the students put in a great deal of thought and effort. One or two have been brave enough to submit video. They say that they prefer the freedom to express their creativity. I like Robin find that a lot of teachers/academics often ask students to do things that they themselves cannot do. They rely on the support staff to do this for them.
Once again, an excellent and though provoking blog.
Steve Wheeler said…
Yes, I agree that it's important that teachers model practice themselves. Personally, I won't introduce tasks into the classroom unless I have at least tried them for myself. How else am I to understand and appreciate the efforts of my learners? Thanks for the great comments guys!
Grumpty said…
I am 100% with you on that one and it is something that students pick up on. I run a lot of sessions with staff but it is like banging your head against a wall. I never give up though!
Anonymous said…
Two things here – one that it is not just the increased use of new and emerging technologies in learning that warrants the flexibility of submitting assignments in different formats but also the need for individualizing the learning. The learning style and preference of the learner should help them determine whether they want to stick to the standard essay format or submit a video, for example – and two, when it comes to grading the assignment in whatever format, the evaluation rubric should focus on the skills and competencies that we are trying to assess and not the effort or time taken to put the assignment together – so if your assignment is focused on research skills, then the output should evaluate the effort and time spent on research and not on how much time it took to write a 4-page report on the findings vis-à-vis creating a 10 minute video on the same.
Steve Wheeler said…
Interesting comments pjasani, and I agree with much of what you say - but I wonder if you are conflating effort with time taken to complete the assignment. They are not necessarily the same thing, and when I mention effort, I am really referring to cognitive effort - in other words, the commitment of the student to engage with the content and to learn from it.

I'm not too concerned about playing the to 'learning style' of the student - learning style theories are questionable and so is the concept. Rather, as I wrote above, I'm more interested in encouraging students to explore every kind and mode of expression - that way they will acquire new skills they can then apply to a variety of contexts, and hence develop transliteracy. What do others think?
Anonymous said…
I agree with you Steve. If you are referring to cognitive effort, then I am on the same page as you. Regarding, the second part of my comment, in our own ways, again I think we are saying the same thing - learners should be allowed to express in different formats (and a specific format may be better suited to their learning style) and thus acquire new skills.
mrpbps said…
Hi Steve,
I tutor a wholly online unit for pre-service teachers at Deakin Uni, (this is the third year I've done it now). The focus is on exploring tools/spaces that aspiring teachers and teachers upgrading their qualifications might think about using in a classroom as well as considering how they might use them. As a totally online unit everything is mediated via the web using the tools I am encouraging them to explore, note aside from an introductory post on the Uni LMS I do everything else elsewhere. If you're interested you can see the current unit, (we're just about 3/4 of the way through) at http://2013est430.wikispaces.com/ and the 2012 version at http://2012est430.wikispaces.com/

If you check the wiki navigation you'll find the Unit Guide and the Assessment Tasks, (a blog and a Teaching and Learning Space), plus the rubrics that I use to guide their thinking and my assessment. Though there are some aspects of the rubric that aren't as useful as others, I would like to better reward students who are consistent in their work over the time span of the unit as opposed to those who compose all their blog posts in the last week of the time allocated. I'd also like to better encourage and reward meaningful communication between the students whether it be via the blog comments or within our Edmodo group.

The second Teaching and Learning Space task is meant to provide an opportunity for the students to put into a practice some of the skills/knowledge they have accrued in the first part of the unit. Overall over the past three years tutoring the unit I think that the assessment tasks and rubrics have worked in sorting out those students who should be recognised for their learnings.
Simon Ensor said…
Shall be sharing this with colleagues. We seem to be asking similar questions :-)
Lloyd Dean said…
We only assess online. The methods vary depending on the tasks set. I'd imagine we might have a little more "freedom" within an further education institution.

They have used words, soundcloud, Movie maker and screencasting.

In our environment, the range really helps. We've actually found that because the learners are moved away from pure assignment (written fashion) the quality of work has increased.

The feedback is usually similar to the actual task completed. When looking at their academic writing we can give much more detailed feedback.

It's an interesting debate, but I agree that digital assessment will be the norm very soon.
Simon said…
I like your interest in encouraging students to explore and potentially acquire new skills through exploring different modes of expression via digital assignments ... but how do you cater for the student who tries something new and fails? Unless it is part of the assignment and the agreed 'marking scheme' it must, potentially, be difficult not to appear to be rewarding poor work in the eyes of the other students. One way, I guess, would be to make the assignment part a reflective comment on the process of answering - what I set out to do, how I set out to do it, how I did it, how well I succeeded / failed to do it, what I learned from the process ...
Steve Wheeler said…
The answer Simon, is that I negotiate the students' assignments with them and then scaffold their efforts, making sure they keep on track and meet the assessment criteria. I haven't had a single student fail in the last four years since we started this initiative. My students are encouraged to take risks, and my role is to ensure that their risk taking is rewarded, not punished.
Simon said…
Hi Steve (part ii), peer assessment might be another way to go. Other students will know what the assignment is and what should be included in the submission (having been involved themselves) and suitable systems can be set up to anonymise and formalise the process. I'm a bit out of loop now (retired) but I recall some interesting assessment solutions reported by the JISC work on e-assessment: http://www.webarchive.org.uk/wayback/archive/20140615085433/http://www.jisc.ac.uk/media/documents/themes/elearning/effpraceassess.pdf and http://oro.open.ac.uk/11949/. With particular regard to peer assessment: Davies, P. (2004), Don’t Write Just Mark: The Validity of Assessing Student Ability via their computerized peer-marking of an essay rather than their Creation of an Essay, Association of Learning Technology Journal (ALT-J), 12, 3, pp 263-279 and https://at-web1.comp.glam.ac.uk/staff/pdavies/Heriot%20Watt%20Presentations%20May%202007/peer%20assessment/Heriot%20Watt%20Peer-assessment%20Phil%20Davies.ppt.

Popular Posts