National Methods of Assessment

[John Simkin]

Dr. Joanna Le Metais at the National Foundation of Research has just published an international report on national methods of assessment. It appears that only five countries have compulsory standardised assessment tests (England, Australia, Canada, Singapore, Wales – under review). Only the UK employs league tables to present this information to the public.

Countries such as New Zealand, Japan, Korea, USA, Spain and France use a sampling system in which a representative group of youngsters – usually around 3% are externally assessed. However, the vast majority rely on teacher assessment.

http://www.nfer.ac.uk/

Like the recent OCED survey of educational performance the NFR points out that there is no link at all between national testing and educational performance. Dr. Joanna Le Metais, as the OCED report, likes what is happening in Finland. She points out that “Finnish schools are expected to evaluate the needs of their children and evaluate themselves. Teachers in Finland are also highly qualified – they have to have masters degrees – which is a key factor in favour of success.”

[Guest Adrian Dingle]

John

Firstly a warning, this subject has been the focus of nearly all my professional conversations since arriving in the USA form England nearly four years ago!

To me, this is incredibly simple to analyze. Allow me to give a little background.

In England, kids leave school with a set of standardized test scores (exam results) and little else (I have no idea in what state Record of Achievement is now in). Anybody who understands anything about anything realizes that this set of scores does not tell you all there is to know about an individual child, but educators understand that is does allow you to make certain, perhaps relatively narrow but nevertheless objective, comparisons.

In the USA, kids leave school with a fairly random collection of standardized test scores (SAT - English and Math scores, SAT II's - GCSE like individual subject scores, AP's - A level like individual subject scores) and a very important high school transcript. The high school transcript is an important part of their academic record and is contains all the statistics about the grades they have achieved throughout their high school careers in all their courses. This information weighs in heavily when college's (universities) make offers to kids. Of course, the high school transcript has the potential to be almost entirely subjective. Most schools don't even have an internal standard, so different standards are being applied in the same school, in the same subject, in different classrooms*. This causes no end of problems when parents and kids are assigned a teacher for a subject who is perceived as being "hard". This is important since it has the potential to adversely affect their transcript and, in theory, influence where they go to college.

To get to a state of greater objectivity, it appears that standardized testing is beginning to take on a much more important and higher profile role here in the US. Sanity is prevailing.

*(When I first arrived in the USA I was horrified to learn that there was no internal syllabus for chemistry within the school. The explanation was that, "at this school we hire excellent people who are then given 100% autonomy to pursue excellence in the classroom". To be fair, at the type of school where I work, this works stunningly well. Great teachers, left alone with great kids, do great things, and the academic profile of my school compares to any nationwide. The flip side is that if there is any level of incompetence, the system is open to terrible abuse, with no internal, let alone external, standards. In this situation transcript grades become meaningless and the whole system collapses).

[Maggie Jarvis]

The whole issue of 'assessment' presents problems particularly regarding interpretation of the word itself.

It appears that only five countries have compulsory standardised assessment tests

John's statement seems to be talking about the UK system of SATs (examinations that are endured by students at ages 7, 11 and 14 years). The results of these are published nationally as 'league tables' and are a source of regular and often heated debate at all levels within education! The idea is that this public airing of the 'achievements' of students in every state school will somehow encourage schools to 'do better', and will also inform parents so that they can then select a 'good' school for their children. The question is, are these SATs examinations actually having a beneficial effect on schools? Are they actually of use to the students? Are they of any use whatsoever to employers?

In addition, UK students then sit GCSE examinations at age 16 and A level examinations at age 17 -18. The results of these are also published in league tables for all the world to see (see the DfES website!). Unlike the SATs, however, these examinations do generate nationally useful certificates for the students - something that employers understand as being a useful 'yardstick' by which to have some measure of their potential employees.

In response to Adrian's comment:

I have no idea in what state Record of Achievement is now in

This scheme, which was intended as a sort of CV is now being replaced with a scheme called 'Progress File' - a more flexible and potentially more useful running CV. It does, if properly used and maintained, provide a much broader overview of any particular student and sounds to be something like the 'high school transcript'.

Most schools don't even have an internal standard, so different standards are being applied in the same school, in the same subject, in different classrooms

I think that this situation is still true in some UK schools also! However, I come back to the word 'assessment' again ... the UK National Strategy is trying to address standards within schools at classroom level. Assessment for Learning is one of the key elements - providing students with constructive feedback and information on 'how to improve and move forward', and teachers using their assessments to inform their teaching in future lessons. Simply ticking work and giving a mark out of ten is no longer appropriate ... well, what use is it really in many situations?

As John mentions:

“Finnish schools are expected to evaluate the needs of their children and evaluate themselves.

Assessment for Learning is, surely, about evaluating the needs of the students and evaluating the teaching that follows. A lot of money has been spent by the UK government on the National Strategy - we have to hope that it has been spent wisely on training classroom teachers! Perhaps then assessment will become more meaningful and consistent within and between schools.

I was surprised at being told that some teachers have been told not to use red pens to 'mark' students' work as it gives a negative impression of the feedback from the teacher. Instead teachers are (apparently) busy buying up bottles of turquoise, purple and green ink to use instead. Will this really make a difference?

[Derek McMillan]

Targets

As a result of the ubiquity of testing, there has been a similar blossoming of target setting.

In particular we are continually being urged to "set bold targets". Any one child can expect to have 18 or more "targets" they are supposed to achieve...many of them "bold targets!"

Now personally I am very bad at archery. I am not convinced that if someone were to move the butts twenty yards further away this would encourage me to reach my potential.

Yes teachers did find it useful to set *a* child *a* target once in a while. A formalised system in which each child has a computerised record of her or his targets...well I don't need to labour the point.

The news this morning is that Wales might ditch SATS.....now that is one target worth aiming at

[Guest Adrian Dingle]

Maggie refers to the Record of Achievement thus;

This scheme, which was intended as a sort of CV is now being replaced with a scheme called 'Progress File' - a more flexible and potentially more useful running CV.  It does, if properly used and maintained, provide a much broader overview of any particular student and sounds to be something like the 'high school transcript'.

Actually a typical US High School Transcript shows nothing but a bunch of percentages (grades) that the kid achieved in each class that they took. In that respect it's not much different to a list of exam grades for a kid in the UK. The difference is that grades in the UK are generated as a result of national standardized tests (GCSE's and A levels) but the US transcript grades very often have no standards applied to them at all. They are simply what the individual teacher generated by their own method of assessment in their own classroom. The transcript will also show the results of any standardized tests like SAT's, SAT II's and AP's, but the (non-standardized) classroom grades carry an enormous amount of weight.

[Derek McMillan]

Some things you can assign a numerical value to. Shoe size for example. The failed attempt to assign a numerical value to pupils ("ooh he's a five!") is one which New Labour seems wedded to. They have no reverse gear so it will be necessary to force them into a u turn.

http://satsmustgo.tripod.com

Pupils are alienated, understandably and justly alienated.

Parents report what seem to be unprecedented levels of stress in younger and younger children (our website has a report from the Liberal Democrats on this).

Do ppl think that there is a difference in the attitudes of Math and English teachers on this? How do teachers from neither of these disciplines perceive the problem?

New Labour does not listen to reasoned argument (hence their number one argument: "this is not negotiable") so what must we do to make them change this policy?

[Marco Koene]

What does a number say? I agree with Derek (never thought that would happen soon ) on that one.

At my school we are adopting a system without grades as such. Instead we use learnlines. A student develop oneself along this line. <see debates in education-> natural learning for more details>

This targets get very easy, a child develops at his/her own 'natural'' pace!

[Guest Adrian Dingle]

Marco said,

What does a number say?

This is any easy question to answer that I've already addressed in a previous post.

Anybody who understands anything about anything realizes that this set of scores does not tell you all there is to know about an individual child, but educators understand that it does allow you to make certain, perhaps relatively narrow, but nevertheless objective, comparisons.

[Derek McMillan]

Anybody who understands anything about anything realizes that this set of scores does not tell you all there is to know about an individual child, but educators understand that it does allow you to make certain, perhaps relatively narrow, but nevertheless objective, comparisons.

The problems include:

1) We end up valuing what we can measure.

2) This might be more appropriate to an analysis of ability in Maths or Science than English, History, Drama or ICT

3) Does that mean valuing Maths and Science more highly than other subjects?

4) Graded grains may make finer flour but it is not clear that graded pupils make better learners.

5) Ranking schools according to these scores is blatantly misleading and yet that is the stated aim.

Finally how "objective" is this? The score of the pupil depends on the criteria of the exam-setter. It tells us what the exam-setter values. It is an "objective" measure of what the government thinks is important....and we all know how "objective" they are....(no I will not divert into a discussion of weapons of mass destruction but I will just mention that the commitment to objective truth of this government is questionable!)

Have a nice day.

[Guest Adrian Dingle]

With respect Derek, I think you miss the point.

I openly acknowledge all the faults that you pick in the standardized testing of students, but there is absolutely no doubt that the results are based upon some kind of objectivity. That objectivity may have restricted value, restricted relevance and restricted application, but it IS objective against some kind of standard. Without it, or a similar system, you have zero objectivity. My argument would be a flawed system is better than a free for all with no parameters.

[Graham Davies]

I guess that - for obvious reasons - we are a bit ahead of the game in setting international assessment standards in Modern Foreign Languages. See the Council of Europe's Common European Framework (CEF) for Languages and the DIALANG language testing websites:

http://www.coe.int/T/E/Cultural_Co-operati...Language_Policy - also published in book format by CUP.

http://www.dialang.org

The CEF is an internationally recognised six-point scale of language proficiency that is used as a yardstick by numerous exam boards in Europe - rather belatedly in the UK, however, except by exam boards that offer exams in English to Speakers of Other Languages (ESOL). The CEF is based on extensive research going back to the 1970s. From my experience in teaching students from a wide range of different mother tongue backgrounds, the CEF yardstick is pretty accurate and tells me what to expect. If, for example, I know that a student has passed the Cambridge First Certificate examination in ESOL, which relates to the CEF B2 (Vantage) level, I expect the student to be pretty good, as the First Certificate is usually achieved after around 700 learning hours and tests the four discrete skills Reading, Writing, Listening and Speaking (plus Use of English) very thoroughly.

Our National Curriculum for MFL and related exams have a lot of catching up to do in order to fall into line with the CEF, but there are hopes on the horizon:

"Languages for all: languages for life - a strategy for England"

http://www.dfes.gov.uk/languagesstrategy