Who put the 'ass' in assessment?

I gave a speech to the eAssessment Scotland Conference in 2011. It was an interesting and thought provoking event. Hosted by the University of Dundee, the conference attracted almost 300 delegates from all over the UK, and farther afield and as could be expected, we saw a number of papers presented on all aspects of technology enhanced learning and assessment. These included presentations on the use of blogs, peer collaboration, mobile assessment, serious games, Google forms, Mahara and other e-portfolio applications, audio feedback and personal learning environments.

I was very pleased to have been invited to give the opening keynote, which I entitled: Assessment in the Digital Age: Fair Measures? The slideshow is below:

I started off with a bit of lightheartedness and horseplay on accents and language (I do an impressive Shetland accent, having lived there for two years - but my French accent sounds more like Inspector Clouseau gargling). Although we all had a good laugh, there was a serious point to the funny accents. I made a remark that is still crystallising in my own mind, that accents tend to divide people - they are not only an indication of where we may have spent our time growing up, they are also a cultural marker and a statement of our identity. As such, there can be problems of comprehension and confusion if the accent is strong. 

On the subject of assessment, might it be fair to claim that the accent used by those who are assessing may be confusing or alienating to those who are being assessed? I can't recall how many times I sat down for an exam and turned the paper over, only to be confronted with what seemed to me like a foreign language. Throughout the day, both Donald Clark (another eAS11 keynote) and I showed some hilarious examples of misinterpreted exam answers. The responses given to the answers may have seemed funny, but in fact they were generally correct. The point we both made was that the students weren't wrong, the exam questions were wrong. They were either impenetrable, ambiguous, or simply poorly worded.

I have just reviewed a new book for the Times Higher Ed. It's called Now You See It, and is authored by a well known American academic and brain behaviour scientist called Cathy N. Davidson. In it, she recounts a story of a time she sat a multiple choice question paper. She got very low marks, because she spent most of the time on the reverse of the answer sheet correcting all the errors and ambiguities in the questions. She pointed out that some of the questions could not be answered because none of the options were correct. Surely she should have been given very high marks for demonstrating her creativity and intelligence? No, she didn't answer the questions, and therefore scored a low grade. There were no points for critical thinking or creative solutions. The only reward you can receive in this system is if you play by the rules and regurgitate the facts that were drilled into you.

One of the conclusions of the eAssessment Scotland conference, which very few people argued against, was that examination authorities really need to get their act together if they are to continue to administer exams that shape the future of young people. It's an absolute disgrace and entirely unforgivable when exam boards such as AQA, OCR and Edexcel cannot find the expertise within their organisations to create examination papers that are error free. Let's face it, that's all they are meant to do. Yes, we struggle with understanding people when they have strong accents, but it's more than a struggle when children are penalised because it's impossible to answer exam questions.

Creative Commons License
Who put the 'ass' in assessment? by Steve Wheeler is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.


daja said…
Great slide show Steve, I loved the Maths example with the 5 on its side (is this a type of lateral thinking?) and the Peanuts cartoon. It is sometimes a little depressing to realise that almost everything one wants to say has probably already been said by Charles Schulz. And more wittily.

But I take issue with the idea that pedagogy drives assessment. For me it is the other way around. You can do everything you can think of to encourage deep learning but if the assessment regime is shallow then successful students will adopt shallow learning strategies in order to pass.

You are lucky at University. You have the possibility to design an assessment regime which is authentic and driven by pedagogical principles. I teach in a secondary school. Public exams undermine our attempts at real teaching time and again. Given the politics of education I cannot envisage a near or medium term future where that is not so.
Steve Wheeler said…
Thanks for your comments @daja. I'm of the inclination that pedagogy (that is teaching) and assessment are interdependent on each other. You can't as you say have good pedagogy if the assessment method is shallow. But neither can you have good assessment results if the learning being measured is too shallow. Students will simply not be able to answer the questions. The two components need to work together in tandem and at the same level of responsiveness.

I agree with you though that public exams undermine much of the attempts of good teachers to create engaging learning environments. Unfortunately, I don't think we will ever see a time when governments stop their meddling.
Brad said…
Very catchy title, Steve, and I enjoyed this slide show almost as much as the #RSCON3 one !

You've inspired me to resurrect one of my blog "drafts". It'll be out monday morning and I'll link back to this great post.

By the way... do you know the origins of the word assessment... it actually does have to do with ass... more on monday ;-)
Steve Wheeler said…
Thanks Brad. I look forward to reading that one... :-)
Mark Smithers said…
I really like your slide deck Steve. I have no experience in assessment in primary and secondary education but quite a lot in the tertiary sector. I am reminded that twenty years ago as a course leader in one of the UK's new universities I found the assessment standards to be much more rigorous and consistent than I find them now. In my experience it was common for academic staff to understand the nuances of setting assessment tasks. This was largely driven by highly experienced and rigorous external examiners and the demands of external accrediting bodies. I know I sound like a curmudgeon but the fall in the standard of higher education assessment over the last twenty years really is one of the most dispiriting as aspects of working in the sector.

In regard to your response to @daja; I agree that the two are interdependent but we mustn't make assessment reliant on having been 'taught'. We need to encourage our auto didacts.
daja said…
Good point Mark. As Lucy points out in the Peanuts strip from Steve's presentation, it is not really fair for the student to be assessed on how good the teacher is. Assessment should be contingent upon learning and we should encourage everyone to be a little more auto-didactic.
Anonymous said…
"If you could lead through testing, the U.S. would lead the world in all education categories ... You don’t fatten yr lambs by weighing them." Kozol
Simon Ensor said…
Great timely post. The posting time has nothing with the reading/refelexion time...

I had it on my to read list then I forgot. Then a few days later my memory was jogged by flânerie through tweets and blog links.

Who needs to read lists? It seems to me that much of directive teaching to tests is fruitless.

Deep learning can only be self-directed and self/peer/mko assessed.

It's only now that I realise the importance of what my teachers condemned: doodling, rêverie, careful analysis of their test criteria (so that I would satisfy them a minimum and make them leave me in peace to be me).
UM said…
You state about accents that "there can be problems of comprehension and confusion if the accent is strong."

Are you kidding? You believe that people who speak with accents also THINK with accents?

You are the one with confusion and comprehension problems if that's your premise.
Steve Wheeler said…
Hello U (if that is your real name) and thank you so much for your kind and extremely friendly comments.

No, I don't actually think that people think with accents (that would be absurd. You may infer it, but I can't recall even implying such an idea) but I am deadly serious when I state that some accents are so strong that they can be at times difficult to understand. If you for example have English as a second language, you might be hard pressed to understand someone with a strong regional British accent.

However, the point I'm really making is that the accent or dialect used in examinations may not connect with the dialect younger age groups may be familiar with.

However, I detect from the tone of your comments that you probably hold a strong opinion on this, and if so, we may well have to agree to differ in our opinions ;-)

Popular Posts