Delaying exams; why?

So, a research study has arrived at the conclusion that due to Covid19 students may be 3 months behind in their studies.     The delaying of exams to allow students more time to catch up has also been discussed.   This all seems like rather simplistic thinking.

There are for me a number of issues with delaying the exams.

The first is that we already accept that exams differ each year and therefore there is already tinkering in place to adjust the grade boundaries to keep some consistency across academic years when looking at the statistical outcomes of students in general.   This is why the result show small but steady changes year on year rather than being more volatile. It seems to me to be fairly easy to just adjust this process to normalise the exam results next year should they be, as would be expected, lower than previous years and should it be important to maintain parity in results across different calendar years. And this statistical fiddle would be more acceptable than the algorithm proposed for 2020 results as it doesnt differ from the statistical adjustments of GCSE and A-Level results in 2019, 2018, etc.

Another issue, if we were to delay the exams, is that it simply knocks on to following years.   So, delay the GCSE exams would mean teachers would lose some teaching time they would likely use to start A-Level studies or to start Year 13 teaching of A-Level subjects following Year 12 exams.  As such it doesnt solve the issue, but rather displaces it. Is the focus not on learning rather than measuring learning? As such how can any solution with a knock on to teaching and learning be acceptable.

Also, the point students should be at the end of each academic year has been arbitrarily determined.   At some point the curriculum for each subject was developed and the content decided for each year or stage however it could have easily been decided that more or less content be added.   Why, therefore, is the point students should be at perceived to be so immovable? Why not simply reduce content for the year based on the reduced time available to students? Surely this is an alternative option.

There is also the point that next years results will be compared with this years results, where it has already been reported this years results were significantly up.   This obviously resulted from the use of centre assessed grades, provided by teachers, without any of the normal annual statistical manipulation in relation to grade boundaries.    This comparison is unavoidable. So, despite any delay, etc, there is still a high likelihood of negative reporting in the press with regards the 2021 results, with knock-ons in terms of students/parents being disappointed.

This bring us nicely to the big question I have seen a number of people ask, which is 3 months behind who or what?     Is it 3 months behind where teachers think they would be had Covid19 not arisen?   A prediction based on a predication doesn’t provide me with much confidence as to its statistical reliability.   Is it three months behind in terms of curriculum content covered at the predicted rate that content is covered?   Again this suffers given it relies on predicated rate of coverage of materials plus could the content be covered at a faster rate but in less depth possibly?

Maybe this issue is an opportunity to reassess our assumptions and to question our current approach regarding education and how it is assessed or are we simply going to accept that this is the way things are done around here and that any changes should be limited and only in maintaining the status quo? I believe we have reached a fork in the road, however I worry that we may look to take the route which looks easier.

Advertisement

Exams: A fair system?

criminalatt from freedigitalphotosCovid19 has forced the cancellation of this years GCSE and A-Level exams.   As a result of this schools are being asked to submit grades for their students with a number of people expressing concern over the fairness of the grades which will result.   But were the exams ever fair?

Public exams were introduced in the mid eighteen hundreds, over 150 years ago and I would suggest that they haven’t changed that much.    They are now a foundational element of the world we live in.   Your qualifications, or the grades you get in these qualifications largely determined by exams, dictate your progress and the options available to you as you progress through the education system.  From secondary education to further education then onto university and eventually careers each influenced by the grades you achieve.     To pass each exam a key skill is memorisation.  To memorise key facts, to memorise approaches to using these facts and to memorise and practice techniques to manage exams.   The issue is that in a world of technology, a world where most of us carry Google in our pockets complete with the worlds biggest collection of knowledge plus videos and animations to demonstrate how to use this knowledge, is it really critical that we memorise things?

For some student memorisation is easy as is exam technique and therefore the exam system is perfect for them.   For others memorisation is more difficult and exams bring out exam stress.  In terms of the end outcome both types of students may be capable of the same things, albeit one may need to quickly review Google for assistance.    This seems to be biased in favour of those students who are better suited to exams which is unfair.

Lots has also been written about differing achievement rates in different types of schools, different social-economic backgrounds, etc.   I am not going to expand on this here, only to say that most discussions finish with a lack of equity for students in differing circumstances being identified; that students access to learning and preparation for the all important exams isn’t fair.

Taking “fairness” a stage further, we would expect all students who answer a particular question in a particular way to achieve the same number of marks however, outside of fact-based questions, answers need to be marked by human examiners which introduces variability.  Yes, there are standardisation/moderation processes to try reduce the probability of students receiving unfair outcomes but these only reduce the probability rather than eliminating it.

We also have the annual adjustment of marks boundaries to reduce unfairness where a particular years exam questions might be more or less difficult that the previous years.   On one hand this is good as it seeks to reduce the probability of unfair outcomes, however the existence of this process, and of the standardisation/moderation processes mentioned above both acknowledge the fundamentally unfair nature of exams.

For 2019/20 there will be no exams and instead teachers who have worked with students throughout the year will be making professional judgements as to what students would have achieved.   Schools will then be seeking to check that results across the school are fair, followed by the exam bodies to checking fairness across all schools.   I am not sure how this will be any more or less fair than what we had before.   It may be that the sudden nature of the introduction of this process may be unfair, but outside of this I am unsure that it will have introduced any greater variability than that which existed previously.

I think those asking about the fairness of the 2019/20 results are asking the wrong question.  I suspect some may be invested in the exams machine, while some maybe so used to the exam system that they are scared of potential change and of the unknown.   The big question I find myself with is, if we can issue final qualification grades in 2019/20 without the need for final exams, do we really need these standardised exams in 2020/21 and beyond?

Sync and async remote learning

cog-wheels-2125180_640I have seen a number of posts on twitter pitting Synchronous and Asynchronous remote learning approaches against each other.   Sadly, this kind of binary viewpoint is all too common, if not specifically catered for and encouraged on social media platforms.    As I have often said, sadly the world is not that simple.   So, I thought I would add some of my views:

Synchronous

If we take the SAMR model and the first element of it, substitution, using live video as a substitute for the classroom experience seems to make sense where the classroom experience is not an option.    At a basic level it looks like a simple swap.   Through live video students continue to get access to some of the visual ques present in face to face communication.   They also have the opportunity to engage in the more social side of learning with quick feedback and two-way communications allowing discussion points or ideas to be explored and clarification to be sought where confusion arises.   I believe the social benefits of video-based communication in particular are very important as learning is very much a social activity so the more similar we can make it to “normal” social interaction the better.

The challenge with the above being access to high speed internet to support video plus issues around latency of sound and video which cause problems as soon as multiple people try to talk or where people try to interject with their thoughts or comments.  These issues don’t exist in real time face to face situations in a classroom, or at least they don’t where good classroom management exists.

Another synchronous option might instead be the use of real time discussion or chat solutions.  This doesn’t have the same issue in relation to a need for bandwidth or in relation to video/audio latency.  That said, I believe that typed comments, thoughts, ideas and questions are simply a proxy for spoken offerings, and as a proxy lose some of the detail which exists with face to face real time communications either in real life or via video.  As a result, you can expect higher rates of miscommunication and misunderstanding.

Probably the biggest concern with a synchronous approach is that of workload, stress and strain.   Delivering either real time video or chat isn’t normal when compared with how lessons are generally taught.    This means teaching this way represents a cognitive load in terms of using the technology tools, considering pedagogical approaches and adjusting these, and managing to get feedback from students as an activity or lesson progresses;   Never underestimate the feedback a classroom of students provides through comments, body language, the groups attitude, etc, all of which are more difficult to gather remotely via a screen.  This departure from the current established norm therefore represents an additional load on teachers.

Asynchronous

This focusses more on providing content with students able to work on it in their own time.   From a teachers point of view this is likely to be less stressful as they can plan and develop the required content, including repurposing already produced resources for sharing content.   As it doesn’t put the students directly in front of the students the cognitive load isn’t as significant, as teachers have time to think before responding to students or before posting the next activity.

The challenge here however is that in asynchronous learning the social aspect is lacking.   There isn’t the same interaction between students and teacher or between students and their peers that there is with synchronous or real time activities.    There is also a greater reliance on intrinsic motivation as it requires the students to complete activities in their own time without the teacher prompting in real time.

Sync vs async?

We would never suggest that learning in a classroom, in real time and face to face, was either synchronous or asynchronous.   The teacher might lead a class through some content or a discussion in a synchronous fashion then later in the lesson provide students learning activities to work through in a more asynchronous fashion.   The teacher may then review learning with students back in a more synchronous style.     We would never suggest the teacher just provide resources for students to work through in classroom lessons, or that the teacher and students spend their lessons all working together.

So why is this even being discussed in relation to remote learning?    It doesn’t make any sense to me!

Learning is a complex process best nurtured by experienced educators who know which tools to use and when, who know when they need to work in synchronicity with students or when to empower students to work on their own or in groups in a more asynchronous approach.    It isn’t a question of synchronous versus asynchronous learning.   Its, as is often the case, about finding the right balance between the two extremes, the balance which suits the teacher, the students and learning which is taking place.

Personalising learning

boy-1986107_640Personalisation is a term which is used reasonably frequently in education circles as a goal we should seek.   The ability to provide students with educational experiences which are suited to their needs, abilities and wants.    Technology is helping us progress towards this.

Students can use technology to choose when to learn, using online resources, flip classrooms videos and a multitude of other methods.   Students can choose how to learn or the medium they wish to explore learning through whether this be exploring textual content via Wikipedia, video content from YouTube or audio content via a podcast.   They can also make use of the likes of Skype, Twitter or other platforms to engage in two-way communication and learning with experts, their teachers or other individuals the world over.    This brings us to the global nature of learning in this technological age.    Our learning is no longer limited to our local context.   Instead it can span the earth as I myself know having taken part in a Skype session with a school in Saudi Arabia as just one example of learning across geographical borders.   This opens learners up to new contexts, viewpoints and perspectives which were previously inaccessible, however through technology can easily be arranged.   It also allows them to stretch their own learning and learning experiences beyond that presented formally within their school and by their teachers, exploring those areas which are of particular appeal or interest.

Using technology students can choose the tools they want to use in taking notes during lessons, in revising content and in producing assessment materials.  They can also accommodate for their own specific learning needs using accessibility tools including tools for students with visual impairments, dyslexia or students who are second language learners for example.

Technology is a tool which can have significant impact on learning.   The ability for students to personalise learning and learning content is a key to this potential, putting the learner more in control of their own learning experiences.    It is less about us, as educators, personalising learning for our learners and more about providing them the tools and technologies to allow them to personalise learning for themselves.

 

 

 

Online compliance courses

e-learning-3734521_640Education and schools have to cover a number of risk areas which staff need to be aware of including safeguarding, health and safety and data protection to name but three areas.   The wider world, beyond education, has similar issues which might also include COSSH, lifting and handling and personal protective equipment (PPE).   So how do we address these issues and how do we “train” staff?

Recently I have had the opportunity to see a number of online training platforms, in different contexts, which are being used to address some of the above.   The idea is that these online platforms allow staff to receive training on the areas which relate to them, while maintaining a central record of what training has been done and also sending out notifications and reminders when training has to be renewed.    All sounding good so far?

The issue I have with this is that the focus has almost totally shifted to that of compliance rather than developing learning in relation to the risk area which is being covered.    The platform shows who has done which training courses plus ensures that people do the courses, but does this actually improve the learning related to the particular risk area?

One look at some of the online training content shows multiple ways in which content can either be quickly skipped through or missed out altogether.   I must admit my own urge, when presented with some of these online courses, is to simply get it finished as quickly as possible to allow me to get on with matters I deem to be more pressing.    In addition, the content is not particularly engaging taking the form of video lectures or large amounts of text, with only minimal interaction.   Even the attempts at testing user knowledge at the end of units or modules is superficial in nature plus very much dependent on short term memory of facts as opposed to testing more longer term, or deeper learning of the subject matter.   A user may therefore seem to be proficient in a given area such as cyber security, having completed the relevant online course however may have learned very little if anything from it.

Here we see an example of the focus shifting from developing an understanding of health and safety, for example, to ensuring all have done the health and safety online course.     We stop worrying about understanding of health and safety as we can demonstrate that all staff are deemed proficient having completed to relevant online course.   We have achieved compliance but not competence.   We are considering what we can measure, the completion of online training, as what matters as opposed to trying to measure what matters.

I think we need to take a step away from the compliance culture.  Yes, it is easier to measure an organisations health and safety awareness by the number of people who have completed the annual training, but does this mean the understanding and practice is there?    I believe it doesn’t.    And if it doesn’t why should be spend the time, money and effort on these courses.   Surely, we need to find a better way?

The key for me lies in two areas, the first being how we educate and then on how we measure that learning has taken place.    In the area of education I think it is about making use of multiple delivery methods from short online content to in person training, posters and email awareness programmes.  We also need to continually adapt and revise our approaches which brings me neatly onto measuring.   We need to find methods of measuring whether this is short tests at intervals throughout the year, playing out scenarios, audits or focus group discussions.   This can help inform us as to what has been learned and what has not, and in doing so can help us revise and redesign.   In revising and redesigning we can then seek to build better understanding in our staff.    Yes, this is all much more difficult than simply firing out an online course for staff to do however it builds deeper learning.

Deeper learning is likely to serve a staff member and the organisation much better than a tick against an online training course in the event of a cyber, health and safety, COSSH or other issue.

 

 

e-learning-3734521_640

Some thoughts on educational research

legalbooksWe want our educational efforts to be informed by research as research will be able to show us what works and what does not.   Hattie’s longitudinal study for example indicated the effect size of various educational interventions, drawn from a large number of studies conducted over a significant period of time.   From this research we can identify the activities that we need to do more of and the activities which have little effect and therefore we shouldn’t spend as much time on.   This all seems simple.    Taking a research informed approach seems logical so why wouldn’t you take this approach?

As is normally the case the world isn’t as tidy and simple as we would like it.    Hattie’s study is a good example of research in that it gathered data from across a number of different studies and contexts, plus over a period of time.   It therefore presented findings which could be more easily generalised across educational settings and contexts.   The issue here is the generalisable nature of the findings.    It means that the findings “generally” hold true.   In specific contexts or situations it is therefore possible that the findings may not hold true.   Looking at education in general this is all well in good but teachers are dealing with individual students in their classrooms and therefore should be seeking to find what works for each child.    Holding too strong a view in relation to research findings may lead to practices that don’t work with certain students being applied because the research shows they “generally” work.   Worse still it could lead to practices that do work in a given situation and/or context being labelled as “generally” inappropriate and not being tried.     We need to see educational research as a guide but be careful to understand that in some situations, doing the opposite may equally be effective.

Hattie’s study is based on a thorough and large data set meaning its statistical reliability is reasonable high.   One problem with educational research is that most studies are not based on such a large data set.   They are often based on a very small sample of schools and students.   Studies are often conducted within a specific context such as a certain geographical area, national or region culture, certain age range or curriculum subject.    The validity of the findings when generalised outside the context of the study is often questionable.   I remember my own masters level study when we were guided on the need to state that the findings “suggested” or “pointed towards” as opposed to “demonstrating” or “showing” something to be true.   You will find in most good education research a similar language in the conclusions.    Without a large amount of data gathered from different contexts across a period of time it is highly unlikely any research findings can be generally applied across all or even most educational contexts.    Even where findings are generalizable this doesn’t mean they are replicable in an individual context.

I need to be clear, I am not saying we shouldn’t use educational research in directing practice in individual schools and classrooms.   What I am saying is we should do so with an awareness of the limitations, and bear these in mind.

Football and learning

The World cup has started and I am sure classrooms all over the world will be seeing football related themes, examples, etc. in use as teachers seek to engage students and contextualise learning.    As I sat watching the Spain vs. Portugal game I identified one particular opportunity where football could be used to share an important piece of learning.

It was the 88th minute when Ronaldo stepped up and stuck his free kick round the wall and into the top right corner of the goal.   The Independent described the goal as “sublime”.    I suspect throughout the tournament, and beyond, we will repeatedly see re-runs of the television footage of this goal.

The learning point for me lies in a fact which the commentator shared after the initial shock and awe which immediately followed the goal.    This attempt, this free kick in the world cup, a major tournament, was Ronaldo’s 45th attempt to score from a free kick in a major tournament.   Ronaldo had attempted and failed to score on 44 occasions.

I take two things away from this.

1) Never give up.    Ronaldo had made attempt after attempt and failed to score yet with 2 minutes left in the game which Portugal were losing, he still decided to try a difficult shot despite 44 failed attempts.   He could have gone with easier options such as crossing the ball.   He could have considered the likelihood of success having failed 44 times and judged a direct attempt on goal too risky or too unlikely to success however instead he went with the attempt and saw his 45th attempt sail into the net.

2) Beware of your memory.    We will remember the quality of this goal for time to come.  We will hail Ronaldo as one of the best players in the world if not the best but do we remember the 44 failed attempts?   I doubt it.   This is simply the availability bias at work, in that the goal was recent plus it had a positive outcome, hence it comes easier to mind than the 44 failed attempts.    Students need to be aware of this bias.    One test result or one piece of feedback, whether positive or negative, is not a measure of our ability, knowledge or skill, despite the fact it will come easily to memory.   We need to take care and avoid such strong memories influencing decision making or our perceptions of ourselves and our abilities.

I am sure the World Cup will continue to serve up opportunities for learning as well as providing entertainment.   For now I will get back to watching the Croatia vs. Servia game.

 

References:

FourFourTwo.com, June 2018,  Ronaldo finally scores major tournament set-piece at 45th attempt, https://www.fourfourtwo.com/news/ronaldo-finally-scores-major-tournament-set-piece-45th-attempt#z5avX1ERoRL6kFMc.99

Luke Brown, Independent, June 2018, Cristiano Ronaldo World Cup 2018 hat-trick goal: Portugal star makes history with stunning free kick against Spain, https://www.independent.co.uk/sport/football/world-cup/cristiano-ronaldo-goal-free-kick-hat-trick-video-watch-portugal-vs-spain-world-cup-2018-a8401436.html

Image link: http://www.goal.com/en/news/ronaldo-finally-has-a-world-cup-performance-to-sit-alongside-his-/qvhs25c6727q1qzz72mib4b4t

 

 

Mood music

radio-for-car-2167269_640Popping to Tesco this morning to get some shopping I decided to make use of the wife’s car as it is newer than mine, lighter and easier to drive.    Jumping into the car and starting the engine I was greeted by my wife’s selection of upbeat music, at notably loud volume.    I decided to be kind and not mess with her stereo, instead choosing to listed to her musical selection albeit at a lower volume less likely to be audible from space.

As I drove to Tesco I found myself cheering up as I looked out on the blue sky in between the clouds.   I even found myself rolling down the window.    Something as simple as cheery upbeat music in the car had had the effect of changing my mood.      So how could this simple change impact on well-being if it was part of your daily routine, music in the car or when doing the chores at home, music in the classroom or when marking, etc.

I have decided that before work begins once more on Monday I will create a playlist for my own car purposely selecting up beat music.

I wonder what the implications are for the use of music within the classroom in order to put students in the mood for learning.    I know I have read in various books about the impact of music on mood, emotions and learning.   I also have read various examples of how teachers are make use of this concept however like a lot of approaches which can have an impact they often disappear under the busy activities of the average school day until something brings them back to mind;  For me my drive to Tesco in my wife’s car did just that.

Research based education

researchThere has been a lot of talk over recent months and years about the importance of “research” based practice in teaching and about the importance of research evidence to back up any new technique, approach or fad.   The recent articles following the release of the TIMSS results and the articles which are likely to follow the PISA results due in a weeks time go to show the value which is being attributed to research findings, to quantifiable measures.

The issue is that the idea of a given approach or finding being validated by research make intuitive sense and therefore it seems logical if not common sense that such an approach be taken.     As such we fail to consider the full implications of research and in particular the importance of sample size within the research methodology.

We seek to identify approaches which will be transferable and applicable across the whole of education.   We seek to find those magical teaching methods and learning activities that can successfully be used independent of whether we are in a UK state school in a deprived area or a private school in the UAE.     We seek to make general statements in relation to the state of Maths education, or other subjects, in whole countries or even continents.   The sum total of all children currently in education therefore forms our overall target population.    Based on this any study of 10 schools or even 100 schools makes up a tiny, need I say insignificant, proportion of the overall target population.   Taken on face value the sample size of 600,000 students for TIMSS 2015 sounds impressive however as a percentage of all students within the age ranges covered by TIMSS across all countries involved I suspect it will be a small number.

Daniel Kahneman in his book Thinking fast and slow (2014) discusses the issue of “the law of small numbers” in that, where the sample size is small there is a greater tendency for variance to occur.    He specifically mentions education and how research evidence has suggested, and I am careful to say suggested as opposed to proved, that small schools perform better than larger schools.    He then mentions contradictory evidence which suggests small schools perform worse.    The reasoning behind these contradictory findings Kahneman suggests is the fact that the small sample size used in a small school involved in these studies allows for local variance within the sample which is not mirrored across the target population.   So a small number of high achieving students in one year can result in a significantly positive average, whereas the following year a small number of low achieving students in a year can result in a significantly negative average.   Where the sample size is bigger, such as in a bigger school, the impact of a small number of students is lesser as a result of the total number of students.   So there is a greater likelihood for small schools, those with a small sample size, to appear in either the top or bottom as a result of random variation.

Taking the above into account I wonder about TIMSS 2015 and the fact that Singapore and Hong Kong are both at the top.   These each have a total population according to google of 5.4 and 7.2 million people.   How can we compare these with the UK and USA with populations of 64 and 319 million people?    The smaller sample size allows for more random variation.   Now it might be claimed that the fact they have remained at the top across different years shows this isn’t random variation however as Naseem Taleb suggests in The Black Swan, it only takes a single set of data to refute findings which countless previous data might have appeared to confirm.   TIMSS so far has only seen 6 data sets, 1 every 4 years since 1995, so maybe the next TIMSS data will be the one which provides the Black Swan.

Having given this some thought I wonder if the issue is the viewpoint we are taking which is one of education on a macro level.    Maybe the intuitive pursuit of research based practices is as valid and worthwhile as it feels however the problem lies in trying to look holistically.      Looking at practices in our own school or in a small number of local or very similar schools and at things, practices and approaches that work may be more productive.    We could still use a research based approach however it would be at a micro rather than macro level.       I can also see some linkages here to the teachmeet movement as surely it has been about grassroots teachers getting together to discuss their approaches and what works in their classrooms.

Maybe we need to stop looking for “the” answers and start focusing our energy on looking for “our” answers to the question of how we provide the students in our individual schools with the best learning experience and opportunities possible.

 

Hard Evidence

There is now a strong push on the need for “hard” evidence to prove the impact of technology but also of teaching strategies and other things within education.    Firstly, I wonder what is “soft” evidence however lets park that for now.

Thinking about this I can see where the emphasis on the need for standardized tests has come from as this is hard evidence of the impact of the  educational strategies a given country has undertaken.    But we know it is not that simple as I and many others have previously blogged.

Another impact of this need for “hard” evidence is that teachers seek to ensure they have proof of what they have done.   This leads to the need for forms, checklists and other documents to be created and completed which in turn leads to an increasing workload, another issue which is constantly under discussion.    The need for evidence results in the increased administrative workload.

beakersTaking a scientific standpoint “Hard” evidence, in my opinion, relates to something which is provable by repeatable experiment, however I admit that this is very simplistic and that a full blog or even book could be dedicated to the discussion of hard evidence.

My issue here is that of the number of variables which go into the use of learning technologies, or a particular learning strategy, in the classroom.    These include prevailing national culture, national views on education, available resources, school leadership aims and approaches, teacher qualifications, teacher experience, technologies being used, purpose for the use of technology, etc, and this is just the very tip of the iceberg.   How can any evidence therefore be considered as hard?   It may be that it is “harder” than another source of evidence however, especially where we are looking at generalization on a world or even national level, there will never be any certainty of the ability to replicate a given study and its results.    Having read Talebs The Black Swan I realize it is highly likely that it would be possible to disprove any given study with little effort after all it takes a large number of common studies with the same outcomes to prove something however requires only a single study with contradictory outcomes to disprove it.

Now I am not suggesting that we should stop examining whether given approaches have provable impact.    We must try and check that the actions we take are having a positive impact as otherwise we may undertake initiatives which have no impact or even a negative impact on student learning.   We must however accept that there are unlikely to be educational practices which are so generalizable as to have truly hard evidence which supports their impact.

 

%d bloggers like this: