assessment & reporting

Monitoring Learner Progress

This post was written at the #idea13 Conference – MCG – 12/11/2013

DSUSDtempHdr
George Araya, from the Desert Sands Unified Schools District in California, talked about supporting student progress with assessment data, but he started by talking about culture. He talked about changing culture by changing tools (for teachers) in this order:

  • first they were given email
  • then to an online gradebook
  • thenk smart slates (linked to electronic whiteboards)
  • then (clicker type) responders
  • then online testing and online assessment

He was very big on standardisation, of moving teachers from basic tools to more advanced tools. Students have Chromebooks and the District had developed a ‘private cloud’. They developed a learning platform focused on measurement and assessment, and gives teachers instant feedback, mostly with teachers preparing assessments and students responding (using clickers called ‘Renaissance Responders’) and getting immediate feedback which is published for parents. Tests are easy to do, efficient, and weekly. Yes. Weekly. The principle is constant assessment and instant, live data.

He talked about using ‘intelligent forms’ to observe teacher performance, and some arguments he had with unions about this. I bet.

He also argued for Chromebooks (they’d just ordered 7000, standardised and cloud based for the whole district) He said that they could run 15000 Chromebooks with one person. It seemed that the Chromebooks stayed in the classroom and students logged into it when they came into the classroom and logged out when they left.

He concluded with a big table of test scores and the great improvements in the test scores.

I was a bit critical. Thinking something like ‘typical American over-testing’ fuelled by by my respect for educators like Will Richardson (https://twitter.com/willrich45) who have pointed out the great divide between what American school systems say (we want great education like Finland) and what they do (test, test, test..) I even tweeted:

2013-11-12_20-45-37

 

They had the tweets on the big screen, and all around the room the twitter stream with the conference hastag #idea13 kept rolling on through. When mine popped up, I saw George looking up at the screen for a long time, reading it, and I felt bad. I mean, he had come all the way from California to tell us about what they were doing, how they were changing culture and raising test scores and I was sceptical and a bit dismissive. And me, from a well resourced school with 99% of students achieving at or above national or state literacy and numeracy benchmarks, and him from a district with huge issues of poverty and second language. Maybe testing was the right thing for them. It sounded like a deadening experience for teachers and students (and he admitted issues with some teacher unions) but maybe I shouldn’t have been so smug and quick to judge.

 

Advertisements

What can we learn from Finland?

Well, maybe one of the things that emerges from this interesting conversation with gurus John Hattie and Pasi Sahlbert, that a colleague put me on to, is that perhaps we are being overly critical and pessimistic about what were doing in Australia. Maybe because pessimism and cynicism can serve a political agenda better than acknowledgement of successes?

Sahlberg talks about teacher quality, equity, funding and a range of other issues in a really reasonable way. I respect the work that Hattie’s done, but do you sense here that he’s talking down our system, and emphasizing his own agenda rather than listening to what he’s being told? He’s being told that Finland values teachers, respects teachers and pays them well, values teacher autonomy, doesn’t over-emphasise teaching … his action plan for Australia … well, it doesn’t really reflect that.

Monitoring (and celebrating) class behaviours with Class Dojo

A little while ago I was lamenting with my Literature class that it wasn’t always easy to document how that class went and establish what to do next, both of which are critical for effective planning, reflection, and formative assessment practice.

I take notes on student progress (formerly in OneNote, latterly in Evernote) but it takes discipline (and time) to do that after every lesson, let alone in that lesson itself. I might do that once a week, so I often don’t even have a record of who I’d had a good learning conversation with each lesson and, importantly, who I hadn’t spoken to 1-1 for a while in class.

It’s something that concerned me again recently when, at the end of term, I asked the students to complete a short online survey of how the course was going for them. Most of the results were positive; the students felt they could learn, were challenged to think and participate and enjoyed the range of activities, though one wag suggested that I could improve my tech skills 🙂

However, the lowest response came from the question: ‘Do you have a good sense of how you’re going in this class?’. It’s something that is difficult for Year 12 students sometimes, especially early in the year when they haven’t done much School Assessed Coursework (SACs) and it’s one place where a short 1-1 conversation can have much more impact than any general or group activity you set up.

But, who have I spoken to recently? Is the problem I began with.

So, I was very interested in the possibilities of the unlikely tool named Class Dojo which promises real-time behaviour monitoring. I saw the program from Roger Zuidema on Twitter and recorded that I was about to invent this!

I was surprised to get a pretty quick reply from the people who make Class Dojo too, proposing a Skype chat sometime about how it could be improved, which I’d like to do sometime.

How it works is you set up your class, who get zany colourful avatars, and you record good behaviour or bad in the class very easily via web page, iPad or iPhone. They get +/s or -/s against their name and the tally adds up. Over the lesson, or the week, or the term or whatever period you decide.

I’m lucky to work in a school where disruptive behaviour is rarely a problem but I was very interested in how this tool might be used to record learning behaviours. Our school’s reports, for example, include a set of ‘Positive learning behaviours’ from ‘Participates in class discussions’ to ‘Is organised and up to date with set work’ that teachers are asked to record in terms of how often they’ve seen their students exhibit those behaviours? Always, often, sometimes, rarely? But how do teachers record those behaviours and could Class Dojo help?

One of the first things I did was enter my students names and then showed them how the class looked via the data projector in class. They’re Year 12 students but they loved the quirky little pictures, immediately wanted to customise theirs (you can’t) or swap with someone else (you can’t).

We then looked at the behaviours section, which is at the heart of the program. We looked at the default behaviours (out of chair!) and talked about what + or – behaviours might look like in our Literature class, and then added them. One of the nice things is that you can set the positive and negative behaviours yourself and can even vary them from class to class. So, I can set up a series of learning behaviours that suit my senior Literature class, which are different to the ones I want to establish with my tutorial group in the wellbeing program. Here’s the behaviours we agreed on.

 

We agreed on these positive and negative behaviours and I was ticking off some of the things in Class Dojo through the class. This year I’ve started using a student as ‘class blogger’, who records the class discussions, rather than doing that myself all the time in OneNote, (they’re put up on the class wiki) and that’s helped free me up more to direct discussions or take records like this. So you click on the student names and assign ‘points’ as you go and it even pops up on the screen with a cool sound if you want.

At the end of the class, you get a report something like this

or you can get a report for each individual student, from any duration (whole year so far, this lesson etc) and you have the option to email that report card to a parent or student or email all the cards (it remembers the email addresses once you’ve put them in once)

 

 

I don’t think our school would approve of zany colourful non-letterheaded reports being emailed to parents, but I was happy to send them to students, always in a positive way to celebrate some ‘points’ they’d gained during the class. I wouldn’t want this to be a negative thing, and I wouldn’t be showing negative scores a lot, or giving them I hope. But, ‘not doing the homework’ is a pretty clear – in a Year 12 year.

Is it perfect? No. Is it a fun, simple and useful way to clearly document and share the good things that are going on in class? Yes. You need to have internet access and more importantly, your students on board; to involve them in the discussion about criteria and be clear and objective about how you’re using it: it’s not an assessment, it’s a tool for improving learning.

I plan to keep using it with my Literature class and wellbeing class this term and see how it goes.

 

Change and continuity in teaching Literature

I’ve been trying to be more actively interventionist in my Literature teaching this year, inspired by some thinking about Personalised Learning I’ve been moved to consciously work on some ‘high impact micro-teaching strategies’ that might help student learning as a follow up to some thinking on formative assessment over the past couple of years.

So, I’ve kept the things that have been working pretty well (the wikis, using OneNote as the default teaching, presenting and note-taking tool) and the blog as the primary means of communicating class news and information.

But I’ve also tried some new things too. I’ve also been up front with the students about that, talking them through my thinking and what the intention/s are. They’re Year 12 students after all, 17 or 18 old most of them, well able to understand these approaches and generally just as keen to do well as I am for them to do well.

We began with a ‘no-hands’ up approach to discussions and I showed them a couple of bits of research about that, including this piece from the BBC.  This approach, coupled with greatly increased ‘wait-time’ has seemed to make the class more generally attentive and receptive. I haven’t had a problem getting discussion going with this group; they’re great about that, but the ‘no hands up’ means that everyone is involved potentially.

I also moved the room around a bit, based on some feedback I got from a couple of teachers who sat in one of my lessons for a ‘classroom observation’ project we’re trialling. I’m stuck with little individual ‘test-style’ tables and, yes I could bundle them into ungainly little squarish pods each lesson, but the next teacher would probably untangle all that and start again. So, I’ve tried a kind of horseshoe arrangement that I use for lots of meetings I run, where students can really make good eye-contact with each other in all the conversations. They’re still all facing the front where the data projector (and teacher) is, but it’s generally more conducive to a good collaborative atmosphere and, importantly, the other teachers who use the room, can mostly tolerate it and don’t shift things back.

I’m going to do more surveys too, shorter surveys more regularly. I generally do an end of semester student survey and end of year but, inspired by a young English teacher who’s been giving her students short surveys using Google Docs (I don’t even know how to do that) I plan to do more surveys online using our own school system.

I did the first survey this week and already it’s given me some good feedback that I intend on acting upon right now, rather than wait until the end of the semester. This is all about helping students improve as they go. I found that they haven’t much enjoyed the poetry cartoon tasks I’ve been setting, which is interesting as I wouldn’t have picked that. I liked them!

And they’re sometimes not so sure about how well they’re going, the kind of progress they’re making. So I want to work on more individual feedback more often, short, focused learning conversations perhaps.

I was also inspired by another teacher to try the “Icy pole sticks”. A simple technique, that you’d often do with younger students, of having an icy pole stick for each student, with their name on it, and selecting the stick at random and asking that student to answer the question. A kind of simple randomiser, and you can just keep selecting sticks at random, or move them from the big pile to a ‘used’ pile to ensure that questions are distributed around the room. I told my class about the idea and got them to name and decorate their stick with some iconographic aspect of themselves. Which was fun.

So, the icy pole sticks, combined with wait time, and the ‘no hands up’, has helped reshape some of the questioning that goes on in the classes so often. And helped make me more conscious of this approach even though, every now and then, I’m drawn to ask the keen student who I know is itching to say something.

Finally, the questioning itself has been sharpened by trying a technique called: ‘Pose, Pause, Bounce, Pounce’, where a question is posed, wait time is added, the question is responded to, bounced to another student, and then a third is asked what they thought of those answers. Sounds more complicated than it is and you can read about it at the Guardian HERE

I’ve just started to try to collect some of these techniques on a Diigo list HERE.  Suggestions are welcome, particularly focused on assessment for learning strategies.

Finally, some traditions are too good to change. Cake day, once every fortnight at the end of the day, is a student-inspired initiative that I’m happy to continue just as it is.

Watch out for those computers: they might break down!

I thought it was great to see a Principal of a government school brave enough to come out this week and remind us all how silly it is (in this day and age) that we actually get students to hand-write long (ish)  pieces for important assessment, and call this a contemporary education.

Actually, Michael Phillips from Ringwood Secondary College didn’t need to remind me. I’ve told anyone who was foolish enough to stop and listen that the Year 12 exam must be about the last time many of these students will ever write anything substantial, thoughtful and structured by hand.

And, it’s not as if they’ve been handwriting through much of their school life.  My students, like the students at Ringwood, have been working with computers extensively and in a sustained way for most of their school assessment through years 7 to 10. Then we get all olde-worlde and put the computers away so students can get their pens out. What?

Or more precisely: why? When I’ve raised this with people from VCAA they’ll argue it’s about equity. Not everyone has had access to a computer so no-one can use one in the exam. And, if pushed, they’re worried about security too in a high-stake exam. But what about equity for students who have been learning, thinking and writing with these tools? ‘Put them away sonny, this is important’. Or equity for those students whose hand-eye coordination results in writing that doesn’t look as good as someone else?

The AGE reported that:

SIX weeks before VCE exams, students at Ringwood Secondary College dump the computer keyboards they have used since childhood and start practising their handwriting.

It’s a forced necessity, given students must write three-hour exams in longhand, that has principal Michael Phillips gritting his teeth. ”Illegible writing has become much more problematic in the last few years because kids are used to working on a keyboard,” Mr Phillips said. ”I think it’s ridiculous that in 2011 we are still doing pen and paper testing … It’s holding the learning back at a time when we’re actually saying there are a whole lot of skills they need for university and the workforce, which involve the use of technology.”

By 2000, we should have found ways of doing work differently.”

In 2007, the Rudd government launched its Digital Education Revolution, pledging every year 9-12 student would have a computer by the end of this year. Given this investment, Mr Phillips said there was no reason why there would be insufficient computers in schools for VCE students to sit their exams. He said assessment should be more sophisticated than students regurgitating facts they have memorised in essays. Instead, exams should test how students solve problems and research and analyse information within time constraints.

VCAA’s response was forthright:

Victorian Curriculum and Assessment Authority chief executive John Firth said in order to be fair, VCE exams needed to be sat under exactly the same conditions across the state. He said there were security risks with the use of technology and the possibility computers could break down. However, using computers was certainly an issue that had been raised, and the VCAA was keen to conduct trials using word processors in exams in lower year levels.

”Our strategic plan over the next three years is we want to make some progress along these lines, but we wouldn’t start with [VCE] English,” he said.

Hasten forward slowly!!

Below: ‘Dont use those modern brass instruments!!! They might break down’*

*Okay, not the best tie-in but I was looking for an olde-worlde illustration and this is the best I could come up with!

NAPLAN knows…

I started off all indignant when I ripped Miranda Devine’s latest piece of folksy wisdom out of the Herald-Sun on Thursday but in the end you just have to laugh at the dross that comes out of the conservative media’s best and brightest day after day as if someone is paying them to do it!

Devine’s latest take on the NAPLAN tests is as fine a piece of persuasive writing as you’ll find, resonating with power. Here’s a bit from the opening:

It is accountability time.

After failing the 20 per cent of children who leave school functionally illiterate, we finally see the truth.

Which students have sat through two years of boring lessons without learning to read and write? NAPLAN knows.

Which teachers are adding little value year on year to the students in their classroom? NAPLAN knows.

Which schools are failing to improve their students’ test results? NAPLAN knows.

NAPLAN knows what evil lurks in the hearts of men. Just like the shadow did! How do they think of this stuff. Has Devine been channeling 1930s pulp fiction? You be the judge!

 

Strike-breakers!!

It’s so silly it’s now almost funny. This week the Australian Education Union decided (for some pretty good reasons) to boycott the forthcoming NAPLAN national tests in literacy and numeracy because of the league tables which (inevitably) emerged from the publication of data on the MySchool site last year.

The response from the Minister for Education was to suggest that parents be brought in, effectively as strike-breakers, to supervise the tests. I’m not sure what the old shearer’s who helped form the Labor Party in the first place, would think about all this. Probably have a chuckle. Certainly most of the parents aren’t all that keen on the idea.

Here’s the AEU’s take on this. There’s a swag of other videos on the site. I can’t find the Gillard statement anywhere. I was going to blog about the growing scandal around the school funding, but enough it enough! I’d love to be talking about curriculum and learning but education is now pretty politicised in this country.