Thoughts from the Bryanston Education Summit

20180606_091909_resizedI attended the 2nd Bryanston Education Summit during the week just past, on 6th June.   I had gone to in the inaugural event last year and I must admit to having found both years to be interesting and useful.   The weather both years has been glorious which also helps to add to the event and the beautiful surroundings of the school.   Here’s hoping Bryanston keep it up, and run another event next year.

During the day I attended a number of different presentations on different topics so I thought I would share some of my thoughts from these sessions.

The first presentation of the day was from Daisy Christodoulou who was discussing assessment.    She drew a really useful analogy in comparing preparing students for their exams with preparing to run a marathon.    It isn’t something where you can jump straight into a marathon distance on day 1 of training.  You need to slowly build up your preparations, focusing on developing certain skills and approaches.   You need to have a plan and then work to this plan, although amending it as needed as you progress, should injury arise or due to weather conditions, etc.    I found myself wondering about how often we actually spend with our students in discussing this plan, the proposed goal of the subject or year and how we will all, teachers, students, support staff and others, work towards those goals.

Daisy also spent some time discussing summative versus formative assessment suggesting that the use of grades should be kept to a minimum of only once or twice per year.   My first reaction to this was concern as it seemed to disregard the potential benefits of spaced retrieval testing which ultimately would result in a score representing the number of correct answers.   Following further thought my conclusion was that spaced retrieval is very focussed on knowledge plus just indicates where an answer is right or wrong as opposed to grading which is more a judgement of students ability.   As such it may be possible to reduce overall summative assessment grading while still making regular use of testing of student knowledge.   I think this also highlights the fact that assessment and testing are actually different things even although they are often generally used as two interchangeable terms referring to the same thing.

Mary Myatt was the second presenter who discussed how we might make learning high challenge but low threat.    As she discussed Sudoku I couldn’t help but draw parallels with computer gaming.  In both case we engage, of our own free will, in a form of testing.   In both cases the key is the low threat nature of the testing.    For me the question is therefore how do we make classroom learning and assessment low threat.    Mary suggested a path towards this in discussing with students our expectations such as setting reading outside their current ability level, which is therefore challenging, but telling them this and then promising to work through it with them in future lessons.   I think this links to building an appropriate classroom culture and climate such that students feel able to share the difficulties they face and work through them with the class.  It is very much about developing an open culture and positive or warm climate in which mistakes and difficulties are not seen as something to be feared or embarrassed by, but to be embraced, shared and worked through together.   Another thing I took away from Marys session was a list of books to read;  My bookshelf will be added to with some of her recommended books shortly.

The third of the sessions which I found most useful was the session by Andy Buck.    He discussed leadership drawing a number of concepts from the book Thinking Fast and Slow by Daniel Kahneman, a book which is one of my favourites.     I particularly enjoyed the practical demonstrations where he evidenced how we all show bias in our decision making.  This is a fact of being human and the way the brain works, we bring to decision making processes assumptions and viewpoints based on previous experiences, upbringing, etc.   He also, linked to this, demonstrated anchoring, managing to influence a whole room of educational professionals to get a question in relation to the number of Year 11 students in the UK wrong.   Statistics suggest that a percentage of the audience should have got this question correct based on a normal distribution of responses however using anchoring Andy influenced the audience away from the correct answer.   I have since used a very similar approach in a lesson with Lower 6 students to show how easily I can influence their answer and to suggest that Google, Amazon, Facebook, etc. with their huge amounts of data on individuals may therefore be able to influence individuals to a far greater extent.

There was also a presentation on VR in education which has opened my mind up a little to the possible applications of VR.   This might therefore be something we experiment with at school in the year ahead.

20180606_150407_resizedMicrosoft’s Ian Fordham presented on the various things Microsoft are currently working on.   I continue to find the areas Microsoft are looking at such as using AI to help individuals with accessibility and in addressing SEN to be very interesting indeed.   I also was very interested by his mention of PowerBI as I see significant opportunities in using PowerBI within schools to build dashboards of data which are easy to interrogate and explore.    This removes the need for complex spreadsheets of data allowing teachers and school leaders to do more with the data available however with less effort or time required.    I believe this hits two key needs in relation to the data use in schools, being the need to do more with the vast amounts of data held with schools however the need to do it in a more efficient way such that teachers workload in relation to data can be reduced.

I also say a presentation by Crispin Weston on data use in school.    His suggestion that we need to use technology more to allow us to more easily analyse and use data is one I very much agree with.   This partly got me thinking about the Insights functionality in PowerBI as a possible way to make progress in this area.   He also talked about causation and correlation suggesting his belief that there is a link between the two and that the traditional call that “correlation is not causation” is in fact incorrect.   At first I was sceptical as to this however the key here lies in the type of data.    Where the data is simple and results in a simple linear trend line the resulting reliability of an argument that correlation equal causation is likely to be very low.   The world is seldom simple enough to present us with linear trends.    If, however the data over a period of time varies significantly and randomly and the second data element follows this however the reliability that correlation equals causation is likely to be significantly higher.     I think the main message I took away from Crispins session was to take data and findings with a pinch of salt and to ensure that context is taken into account.  If it looks simple and clear then there is something which hasn’t been considered.

Overall the day was a very useful one and the above is a summary of just some of the things I took away.   I must admit to taking 5 or 6 pages of tightly written notes, hastily scribbled on an iPad during the course of the day.

I hope that Bryanston decide to repeat the conference next year and is the quality of presenters and their sessions continues, that it becomes a reliable yearly event.   Here’s hoping the trend of good weather also continues should they decide to run the summit again next year.

 

 

 

Advertisements

PowerBI and School Data

powerBIEver since I started playing around with PowerBI I have found it to be very useful indeed and I must admit that I am most likely only scratching the surface.

I came to experiment with PowerBI to try and address some issues I see with data management.    School data is often presented in colour coded spreadsheets showing student performance against baselines for example.   Different sheets are used to present different views on the data such as showing the performance by subject, by gender or the performance of students by SEN status or by EAL status.   Each additional view on the data, of which there are very many, presents us with another sheet of data.  The data is often presented as flat tables of figures however in some cases may involve pages upon pages of different graphs and charts each showing different views on the available of data.   The logic here being that each additional view on the data gives us more data that we can interpret and therefore a greater opportunity to draw insightful conclusions and from there develop actions.   I believe the reality is the reverse of this.

My belief is that teachers and heads of department don’t have a lot of time to analyse and interpret data, and therefore presenting them with so much data is counterproductive.  Having so many different views on the data presented at once also is difficult to process and to understand.   This in turn leads to either ignoring the data altogether or to giving it only a very cursory glance.   For those that love data it may lead to excessive amounts of time spent poring of the data, to data overload, where time spent planning actions, as opposed to analysing data, would be more productive.    As such I subscribe to the belief that “less is more”.

This is where PowerBI comes in.    PowerBI allows me to take my mountains of spreadsheet data and present it in a very easy to digest graphical format where each of these graphs and charts are interactive.    In PowerBI rather than one sheet by subject and another sheet for gender based data, you have just one set of graphs and charts.   You would just click on a gender or select a gender and all the graphs will change to show the results for that gender.   You might then click an SEN status to see how students who are male with SEN needs are doing compared to students on average.    This means we can combine all our different views which are normally represented by different sheets on a spreadsheet into a single set of graphs and charts.   The user then accesses the various views of the data by clicking on and through these graphs and charts.

The benefit of PowerBI is the ability to dynamically manipulate and explore the data by clicking through various graphs and filters.   You develop an almost tangible feeling for the data as you explore through it.   This is something that flat spreadsheets, even if graphs are included, lack.   Also, as you have less to look at, in one set of graphs rather than pages and pages of them, you have more time to explore and engage with the data.

The one current drawback to PowerBI is simply cost.   It is free to use as an individual both web based or via a desktop application, and you can share via sharing desktop app developed BI files however if you want to share via the web platform or if you wish to publish internally via SharePoint you will need a Pro license for each user.    Where you are sharing with a large number of users, even at educational pricing, this can become expensive.   Hopefully this is something Microsoft will be looking at and can resolve in the near future.

Schools continue to be sat on mountains of data.    PowerBI is a tool which allows us to present this data in a more user-friendly form which then allows it to be easily explored and manipulated, allowing more time to plan actions and bring about continuous improvement.  If you haven’t already done so I definitely recommend putting some of your school data in PowerBI and having a play with its capabilities.

School Data: The tip of an iceberg

Schools gather a wealth of data in their everyday operation, everything from attendance information, academic achievement, library book loans, free school meals and a wide range of other data.    We use this data regularly however I think we are missing out on many opportunities which this wealth of data might provide.

The key for me lies in statistical analysis of the data looking for correlations.     Is there a link between the amount of reading a student does as measured by the number of library loans and their academic performance for example?     Are there any indicators which might help is in identifying students who are more likely to under perform?

The issue here is how the data is stored.   A large amount of the data is stored in tables within our school management system however no easy way exists in order to pull different data together in order to search for correlations.    I can pull out data showing which students have done well, which subjects students perform well in, etc. however I can’t easily cross link this with other information such as the distance the student travels to school or their month of birth.    Some of the data may exist in separate systems such as a separate library management system, print management system and catering system.    This makes it even more difficult to pull data together.

A further issue is that the data in its raw format may not make it easy for correlations to be identified.    Their postcode for example is not that useful in identifying correlations however if we convert this to a distance from the school we have a better chance of identifying a correlation.

In schools we continue to be sat on an iceberg worth of data although all we can perceive is that which lies above the water.   We perceive a limited set of possibilities in terms of what we can do with the data.    Analysing it in terms of pupil performance against baselines with filtering possible my gender, SEN status and a few other flags however given the wealth of data we have this is just the start of what is possible.    We just need to be able to look below the water as the potential to use the data better and more frequently is there, and in doing so we may be able to identify better approaches and more effective early interventions to assure the students in our care achieve the best possible outcomes.

Data: Making better use?

One of my areas which I want to work on over the next year will be that of Management Information.   In my school as in almost all schools we have a Management Information System (MIS), sometimes referred to as a SIS (School or Student Information System).    This systems stores a large amount of student data including info on their performance as measured by assessments or by teacher professional judgement.    We also have data either coming from or stored in other data sources such as GL or CEM in relation to baseline data.   These represent the tip of the iceberg in terms of the data stored or at least available to schools and their staff.

Using the data we then generate reports which do basic summaries or analysis based on identified factors such as the gender of students, whether they are second language learners of English, etc.  Generally these reports are limited in that they consider only a single factor at a time as opposed to allowing for analysis of compound factors.   So gender might be considered in one report and then age in another, but not gender and age simultaneously.   In addition the reports are generally reported in a tabular format, with rows and columns of numeric values which therefore require some effort in their interpretation.    You cant just look at a tabular report and make a quick judgement, instead you need to exercise some mental effort in examining the various figures, considering and then drawing a conclusion.

My focus is on how we can make all the data we have useful and more usable.    Can we allow staff to explore the data in an easier way, allowing for compound factors to be examined?    Can we create reports which present data in a form from which a hypothesis can be quickly drawn?    Can the data be made to by live and dynamic as opposed to fixed into the form of predetermined “analysis” reports?   Can we adopt a more broad view of what data we have and therefore gather and make greater use of a broader dataset?

I do at this point raise a note of caution.   We aren’t talking about doing more work in terms of gathering more data to do more analysis.  No, we are talking about allowing for the data we already have to be better used and therefore better inform decision making.

I look forward to discussing data on Saturday as part of #EdChatMeda.    It may be the after this I may be able to better answer the above questions.

Data, data and more data

waitingroomThis morning it was the turn of the NHS to be the focus of the morning TV discussion about how things aren’t going well.    I suppose I should be partially thankful as this takes the spotlight off education at least for a short while.    That said it also once again shows the superficial use of data.

This mornings TV took some time, along with fancy graphics, to outline how the NHS waiting times had increased.   The specific figure they presented being the percentage of patients at A&E who were seen within 4 hours.   This seems like a reasonable statistic to use from the perspective of a patient as it suggests the likelihood that should I need to turn up at A&E I would be seen in 4 hours of less.   I suspect the fact that it is so potential meaningful for prospective patients, the average TV viewer, is why they picked this statistic over others.

The issue with this is what it doesn’t tell us the additional context which may be important in interpreting the figures.    Over the period under consideration did the number of patients attending A&E remain static or did they in fact increase which may be a contributing factor to increased waiting times?     A briefing report by Carl Baker from November 2016 suggested that in 2016 the number of A&E patients at major A&E departments increased 6.3% over attendance levels in 2015.   Were there any changes in the demographics of patients attending A&E as an increase in elderly people attending may mean that patients are less likely to be able to be quickly seen and discharged, again contributing to increased waiting times.    What about the staffing levels of A&E over the period?   Did this change as a reduction in staffing may account for increased waiting times?   Also the figures look specifically at average data for the whole of England; were there any regional variations?   Personally I live in the South West and feel that it is difficult to access a doctor which may mean that I would attend A&E on occasions where someone with more ready access to a GP would not.    Are there also differences between A&Es serving urban and rural areas?   Are there differences between A&Es serving large versus those serving smaller populations or population densities?

In the current performance indicator and accountability led environment we often focus on specific figures such the percentage of patients seen in 4 hours or the number of pupils achieving A*-C or Progress 8, PISA, EMSA, TIMMS, PIPS or other measures.    Each of these pieces of data is informative and tells us something however equally there are a lot of things that it doesn’t tell us.    We need to ask what doesn’t this data tell us and seek data to add context.

Only with context is data useful.

Accident and Emergency Statistics: Demand, Performance and Pressure, C Baker (2016), House of Commons Briefing Library (6964)

 

Some thoughts on GCSE and A-Level results

criminalatt from freedigitalphotosHaving read various articles following the recent A-Level and GCSE results I cant help but think that schools and more importantly education in general needs to make a decision as to what we are seeking to achieve, and stop acting re-actively to limited data which has been used to draw generalized conclusions.

Take for example the shortage of STEM graduates and students.    This was and still is billed as a big issue which has resulted in a focus on STEM subjects in schools.   More recently there has been a specific focus on computer programming and coding within schools.     In a recent article it was acknowledged that the number of students taking A-Level Computing had “increased by 56% since 2011” (The STEM skills gap on the road to closing, Nichola Ismail, Aug 2016).     This appears to suggest some positive movement however in another article poor A-Level ICT results were cited as a cause for concern for the UK Tech industry (A Level Results raise concern for UK tech industry, Eleanor Burns, Aug 2016).  Now I acknowledge this data is limited as ideally I need to know whether ICT uptake has been increasing and also whether A-Level Computing results declined, however it starts to paint a picture.

Adding to this picture is an article from the guardian discussing entries:

Arts subjects such as drama and music tumbled in terms of entries, and English was down 5%. But it was the steep decline in entries for French, down by 6.5% on the year, as well as German and Spanish, that set off alarm bells over the poor state of language teaching and take-up in Britain’s schools.

Pupils shun English and physics A-Levels as numbers with highest grades falls, Richard Adams, Aug 2016)

So we want STEM subjects to increase and they seem to be for computing, however we don’t want modern languages entries to fall.   Will this mean that next year there will be a focus on encouraging students to take modern foreign languages?    And if so, and this results in the STEM numbers going down will we then re-focus once more on STEM subjects until another subject shows signs of suffering.

It gets even more complex when a third article raises the issue of Music A level Entries which “dropped by 8.8% in a single year from 2015 and 2016”.  (We stand back and allow the decline of Music and the Arts at our peril. Alun Jones, Aug 2016).    Drama entries are also shown to have seen a decrease this year (Dont tell people with A-Levels and BTecs they have lots of options, Jonathan Simons, Aug 2016).  So where should our focus lie?   Should it be on STEM subject, foreign languages, drama or Music?

I suspect that further research would result in further articles raising concerns about still further subjects, either in the entries or the results.   Can we divide our focus across all areas or is there a particular area, such as STEM subjects, which are more worthy of focus?  Do the areas for focus change from year to year?

As I write this my mind drifts to the book I am currently reading, Naseem Talebs, The Black Swan, and to Talebs snooker analogy as to variability.     We may be able to predict with a reasonable level of accuracy, a single snooker shot however as we try to predict further ahead we need more data.    As we predict five shots ahead the quality of the surface of the table, the balls, the cue, the environmental conditions in the room, etc. all start to matter more and more, and therefore our ability to predict becomes less and less accurate.      Taking this analogy and looking at schools what chance do we have of predicting of the future and what the UK or world will need from our young adults?    How can we predict the future requirements which will be needed from the hundreds of thousands of students across thousands of schools, studying a variety of subjects from a number of different examining bodies, in geographical locations across the UK and beyond.

These generalisations of data are subject to too much variability to be useful.    We should all focus on our own schools as by reducing the scope we reduce the variability and increase the accuracy.   We also allow for the context to be considered as individual school leaders may know the significant events which may impact on the result of their cohort, individual classes or even individual students.  These wide scale general statements as to the issues, as I have mentioned in a number of previous postings, are of little use to anyone.   Well, anyone other than editors wishing to fill a space in a newspaper or news website.

 

 

 

 

 

Some thoughts on Data

A recent article in the Telegraph (read it here) got me thinking once more about data.   This also got me thinking about the book “Thinking, Fast and Slow” by Daniel Kahneman which I have only recently finished reading.  The book highlighted a number of issues which I feel have implications for education and need to be considered by school leaders.

Firstly the small numbers effect:  The Bill and Melinda gates foundation commissioned a study to examine schools in search of the most effective schools.    It found, unsurprisingly that small size, in terms of student numbers, schools achieved the best results, over larger schools.   Contradictory it also found that small schools also achieved the worst results.   The reason for this as explained by Kahneman is that where a data set contains only a small number of items the potential for variability is high.   As such, due to a variety of random variables and possibly a little helping of luck, some small schools do particularly well, out achieving big schools.    Other small schools are not so lucky and the variables don’t fall so well, resulting in the worst results.

darts

To clarify this consider throwing three darts at a dart board aiming for the centre.   This represents the results of a school with a small number of students with higher scores being nearer centre and a lower score being those darts ending further from the centre.   In the case of student results an average result would then be calculated for the school and the same can be done looking at the position of the darts.   Assuming you are not a professional darts player you may do well or you may not do so well due to a variety of random variables.     Given the limited number of darts the potential for variability is high hence a high average or low average is very possible.   Next consider if you were to continue and throw sixty darts at the dart board, taking the average across all the dart throws.    Given the number of darts the average will regress towards your mean darts throwing ability.    The increased number of data items means that variability is reduced as each significant good or poor throw is averaged out among the other throws.

Within schools a great deal of value is being attached to statistical analysis of school data including standardised testing however care must be taken.   As I have suggested above a statistical analysis showing school A is better than school B could easily be the result of random factors such as school size, school resourcing and funding, etc as much as it may be related to better quality teaching and learning, and improved student outcomes.

Another issue if how we respond to the results.  Kahneman suggests that commonly we look for causal factors.   As such we seek to associate the data with a cause which in schools could be a number of different things however our tendency is to focus on that which comes easily to mind.   As such poorer (and better, although not as often,) results are associated most often attributed to teachers and the quality of their teaching as this is what is most frequently on the mind of school leaders.    We arrive at this conclusion often without considering other possible conclusions such as the variable difficulty of the assessments, assessment implementation, the specific cohort concerned, the sample size as discussed earlier and a multitude of other potential factors.   We also, due to arriving so quickly at a causal factor which clearly must be to blame and therefore needs to be rectified, fail to consider the statistical validity of our data.   We fail to consider the margins for error which may exist in our data including what we may consider acceptable margins for error.   We also fail to consider a number of other factors which influence our interpretation of the data including the tendency to focus more on addressing the results which are perceived to be negative.   This constant focus on the negative can result in a blame culture developing which can result in increasing negative results and increasing levels of blame.   Maybe an alternative approach which may work would be to focus more on the marginally positive results and how they were achieved and how they could be built upon.

The key issue in my belief is that we need to take care with data and the conclusions we infer from it.   We cannot abandon the use of data as how else would we measure how we are doing, however equally we cannot take it as fully factual.   The world is a complex place filled with variables, randomness and luck, and we need to examine school data bearing this fact in mind.   We also need to bear in mind that data is a tool to help us deliver the best learning opportunities for students;  data is not an end in itself!