Research based education

researchThere has been a lot of talk over recent months and years about the importance of “research” based practice in teaching and about the importance of research evidence to back up any new technique, approach or fad.   The recent articles following the release of the TIMSS results and the articles which are likely to follow the PISA results due in a weeks time go to show the value which is being attributed to research findings, to quantifiable measures.

The issue is that the idea of a given approach or finding being validated by research make intuitive sense and therefore it seems logical if not common sense that such an approach be taken.     As such we fail to consider the full implications of research and in particular the importance of sample size within the research methodology.

We seek to identify approaches which will be transferable and applicable across the whole of education.   We seek to find those magical teaching methods and learning activities that can successfully be used independent of whether we are in a UK state school in a deprived area or a private school in the UAE.     We seek to make general statements in relation to the state of Maths education, or other subjects, in whole countries or even continents.   The sum total of all children currently in education therefore forms our overall target population.    Based on this any study of 10 schools or even 100 schools makes up a tiny, need I say insignificant, proportion of the overall target population.   Taken on face value the sample size of 600,000 students for TIMSS 2015 sounds impressive however as a percentage of all students within the age ranges covered by TIMSS across all countries involved I suspect it will be a small number.

Daniel Kahneman in his book Thinking fast and slow (2014) discusses the issue of “the law of small numbers” in that, where the sample size is small there is a greater tendency for variance to occur.    He specifically mentions education and how research evidence has suggested, and I am careful to say suggested as opposed to proved, that small schools perform better than larger schools.    He then mentions contradictory evidence which suggests small schools perform worse.    The reasoning behind these contradictory findings Kahneman suggests is the fact that the small sample size used in a small school involved in these studies allows for local variance within the sample which is not mirrored across the target population.   So a small number of high achieving students in one year can result in a significantly positive average, whereas the following year a small number of low achieving students in a year can result in a significantly negative average.   Where the sample size is bigger, such as in a bigger school, the impact of a small number of students is lesser as a result of the total number of students.   So there is a greater likelihood for small schools, those with a small sample size, to appear in either the top or bottom as a result of random variation.

Taking the above into account I wonder about TIMSS 2015 and the fact that Singapore and Hong Kong are both at the top.   These each have a total population according to google of 5.4 and 7.2 million people.   How can we compare these with the UK and USA with populations of 64 and 319 million people?    The smaller sample size allows for more random variation.   Now it might be claimed that the fact they have remained at the top across different years shows this isn’t random variation however as Naseem Taleb suggests in The Black Swan, it only takes a single set of data to refute findings which countless previous data might have appeared to confirm.   TIMSS so far has only seen 6 data sets, 1 every 4 years since 1995, so maybe the next TIMSS data will be the one which provides the Black Swan.

Having given this some thought I wonder if the issue is the viewpoint we are taking which is one of education on a macro level.    Maybe the intuitive pursuit of research based practices is as valid and worthwhile as it feels however the problem lies in trying to look holistically.      Looking at practices in our own school or in a small number of local or very similar schools and at things, practices and approaches that work may be more productive.    We could still use a research based approach however it would be at a micro rather than macro level.       I can also see some linkages here to the teachmeet movement as surely it has been about grassroots teachers getting together to discuss their approaches and what works in their classrooms.

Maybe we need to stop looking for “the” answers and start focusing our energy on looking for “our” answers to the question of how we provide the students in our individual schools with the best learning experience and opportunities possible.

 

Advertisements

EdTech and Brexit: some thoughts.

euroIt has been reported that IT budgets will be subject to a squeeze resulting from the Brexit decision and for those who have bought IT items recently this has already become evident.   Prices of Apple devices for example have already seen an increase.   I count myself lucky that we updated our iPad fleet just before the Brexit vote as had we delayed we might now see a bill thousands of pounds more expensive than the cost we actually paid.

Revenue costs will be an issues as we may see some service costs increase during the year ahead.   An example of this might be Microsoft licensing costs.   This will be difficult to deal with as it represents a revenue item with increasing cost.   It may require an assessment of the value of services being used with services of a lesser value being abandoned in order to afford those services which are critical or of a higher value.    If Microsoft licensing costs go up which other licensed products might we no longer be able to afford?

Capital projects are likely to take a significant hit as projects may no longer fall within the originally allotted budgets.   As such some projects may now be cancelled and not progress.    This may also result in some projects which previously may have been considered no longer being considered due to cost or potential future costs.

So what can we do?

The key is that of value.    We need to ensure that all that we do has the highest possible value and return on investment.

This is easier to do where planning is for a new project, new software or new hardware.   Here, if due to the financial situation, the decision is taken to not proceed with a purchase the net effect is zero;  we don’t have the item now so not purchasing it results in no change.     The more difficult situation to manage is where we want to bring about efficiencies by looking at what we have and by removing some items.     This may be removing items to replace them with something else, such as moving from desk based printers to centralised Multi-Function Devices or it might be removing something due to the fact that the cost vs benefit does not represent sufficient benefit given the tightening financial situation.    Any removal or ceasing of support is likely to meet with a negative response from users.

The coming year is likely to be more difficult that the year that has past, a year largely prior the Brexit decision.   Overall in terms of educational technology, the recent Brexit decision will not have impacted on the impact and potential impact on technology, however the cost of this technology has almost certainly seen an increase.    As such when taking a cost vs impact viewpoint, technology may now require a greater level of justification in order to counterbalance the increased cost.

 

 

Coding and ethics.

car-crashI have considered the ethics associated with the use of IT systems in the past.  In a previous series of events in the UAE one of the discussion sections focused on Google and how they use data to help refine and personalize their service.   On one hand this seems like a good thing, however Googles motivation is not altogether altruistic.   Google like most companies are out to make a profit for their shareholders and it is the data that they gather on individuals which allows them to do this.   They use the data gathered on you to allow them to target advertising.   This advertising in turn is paid for by other companies leading to Googles profit.    So one viewpoint may be that Google gather data on you, with your permission, to provide you with a personalized service; this sounds reasonably ethical.    Another viewpoint, however, might be that Google gather data on you, where most people neither understand or appreciate the type and volume of data, for the purpose of selling advertising and making a profit; this doesn’t sound quite so ethical.   When I discussed this with teachers, I did so just to suggest they consider the services being provided and the implications, and that they discuss them with students.

Consider Facebook, I would suggest that new parents starting using Facebook some years ago failed to fully understand the implications of posting every milestone of the children to the world.      Recent articles from the BBC and The Guardian seem to confirm this.

An article shared by a colleague got me to take a different perspective on things.    Considering the Facebook issue my initial thinking had put the error on the end users.   These end users had started using the site without understanding the long term implications.    Looking at google, my discussion with teachers focused on the teacher and their students considering the implications as end users.    But what if the blame, if blame might exist, falls somewhere else?

In an article in The Business Insider it is suggested that programmers need to receive ethical training.   It is the programmers which make the sites and services and define the specific functionality and operation.      If programmers at Facebook had considered the ethics of posting and sharing of an individuals life maybe the security and privacy options would have been more mature at the outset or maybe some warnings may have been displayed in relation to posting photos of your children.

Maybe a better illustration of the issue can be found looking at autonomous cars, which as we know, Google and a number of other companies are working on developing.     Lets assume an autonomous car gets into an accident resulting in damage to someone else’s vehicle and to injury.   Who is at fault?      Would it be the owner of the vehicle who may not even have been in the vehicle?   Would it be the passenger in the vehicle despite the fact they aren’t driving; it is an autonomous car.    Would it be the manufacturer of the car?     Or might it be the programmer who wrote the subsystem which failed to avoid the crash?

In future I will be more aware of the limits of a one sided viewpoint focused on the users as the decision makers; either using the service appropriately, ethically and morally or not.   The fact that a system could be used in an inappropriate or unethical way may indicate a failure of the programmers to appreciate the implications of their code, either now or in the future, or worse that the capability was programmed in, in the first place.

I also wonder about, whether with all the focus on coding in our schools, we also need to spend at least some time discussing the ethical issues surrounding programming.

Hard Evidence

There is now a strong push on the need for “hard” evidence to prove the impact of technology but also of teaching strategies and other things within education.    Firstly, I wonder what is “soft” evidence however lets park that for now.

Thinking about this I can see where the emphasis on the need for standardized tests has come from as this is hard evidence of the impact of the  educational strategies a given country has undertaken.    But we know it is not that simple as I and many others have previously blogged.

Another impact of this need for “hard” evidence is that teachers seek to ensure they have proof of what they have done.   This leads to the need for forms, checklists and other documents to be created and completed which in turn leads to an increasing workload, another issue which is constantly under discussion.    The need for evidence results in the increased administrative workload.

beakersTaking a scientific standpoint “Hard” evidence, in my opinion, relates to something which is provable by repeatable experiment, however I admit that this is very simplistic and that a full blog or even book could be dedicated to the discussion of hard evidence.

My issue here is that of the number of variables which go into the use of learning technologies, or a particular learning strategy, in the classroom.    These include prevailing national culture, national views on education, available resources, school leadership aims and approaches, teacher qualifications, teacher experience, technologies being used, purpose for the use of technology, etc, and this is just the very tip of the iceberg.   How can any evidence therefore be considered as hard?   It may be that it is “harder” than another source of evidence however, especially where we are looking at generalization on a world or even national level, there will never be any certainty of the ability to replicate a given study and its results.    Having read Talebs The Black Swan I realize it is highly likely that it would be possible to disprove any given study with little effort after all it takes a large number of common studies with the same outcomes to prove something however requires only a single study with contradictory outcomes to disprove it.

Now I am not suggesting that we should stop examining whether given approaches have provable impact.    We must try and check that the actions we take are having a positive impact as otherwise we may undertake initiatives which have no impact or even a negative impact on student learning.   We must however accept that there are unlikely to be educational practices which are so generalizable as to have truly hard evidence which supports their impact.

 

A smashing exit: Things not to do in the Middle East.

smirnoff21bottleMyself, my wife and our two sons spent the first two months calling the Hilton home.   It was also the companies head office for when staff weren’t working in schools, for meetings and for all administration activities.

From what I can remember it must have been a weekend when the phone rang and we were informed that we were finally relocating to our company accommodation, accommodation that was to turn out to be our home for the next three years.    The problem with the phone call however was the notice being provided.    We were aware that the accommodation was becoming available soon however we had no details as to when “soon” actually was and therefore everything continued had normal.  The Hilton had continued to be home.

The phone call changed all that as we were pleasantly informed that our apartment was ready and that we should check out of the hotel.    We were to check out by noon.    The issue being that the phone call came at around 10am giving only two hours to get ready to relocate.

Now you wouldn’t think you would have much to relocate from a hotel room however it is amazing what you pick up after two months living in a hotel especially where two children are involved.    And so that morning we set about the manic task of gathering all our belongings from the two hotel rooms we inhabited ready to move.

It was slightly before noon when the knock at the door came as the concierge made himself available to assist in the relocation, at least to the foyer of the hotel if not beyond.    It was like a reverse game of Jenga loading up his trolley when the various bags and items we had, carefully balancing each new item in the hope it wouldn’t fall off the trolley and break.    Upon getting almost everything on the trolley, we picked up the remaining items and made our way to the elevator, where we then travelled down to the foyer.   It was as we crossed the middle of the hotel foyer that the Jenga tower failed.   The one bag we didn’t want to fall to the floor, did just that and fell to the floor with a smash.

And there we were stood in the middle of the Hilton as a smashed bottle of vodka spread its contents across the floor.   Panic does not do justice for the way I felt.    I suspect the concierge may also have felt panic, albeit paired with a quicker reaction time.   Before I knew anything about it he has started to mop up the liquid and glass with the first thing that came to hand, my wife’s Chinese dragon embroidered bath robe.

Thankfully no one made comment as to what had happened.   I am not sure if that was due to the concierges quick actions or due to the busy or quiet foyer;  I can’t remember which it was.   The moral of the story is make sure you decide to carry the alcohol personally as opposed to relying on someone else when moving through the foyer in a middle eastern country.

Books, books and more books

booksAll the way back in December 2015, and at this point it seems both a long time ago and only yesterday, I set myself a target of reading a book per month.     The reason for the target was the feeling that I just wasn’t reading enough.   Prior to this the most reading I had done had been during the period studying for my Masters degree, subsequent to which my reading all but stopped.   Generally I think I have progressed quite well in working towards this target, and I have certainly made progress on my reading habits prior to 2016.

So far this year my reading has included:

  • Black Box Thinking, Matthew Syed, 2015
  • The Dark Net, Jamie Bartlett, 2014
  • The Glass Cage, Nichola Carr, 2015
  • Thinking, Fast and Slow, Daniel Kahneman, 2011
  • The Black Swan, Nassim Taleb, 2007
  • Drive, Daniel Pink, 2009
  • The Invisible Gorilla, 2010
  • Multipliers, Liz Wiseman, 2010
  • How We Learn, Benedict Carey, 2014
  • Resilience, Andrew Zolli & Ann Marie Healy, 2012
  • The Element, Ken Robinson, 2009
  • Adapt, Tim Harford, 2011

And my current book Incognito by David Eagleman (2011)

At this point, in the middle of November I have achieved the target I set myself however the crucial factor is not in meeting the target but in improving from where I was just over a year ago and also in learning from the books I have read.

I found the majority of the books read to be interesting to various degrees and have often started a new book based on its mention within a book I have read.     I feel I have a broader set of ideas and understanding than I may have prior to setting and embarking on this particular new years resolution.

At this point it may seem that this piece is very much about me congratulating myself for the progress made and the books read however this is far from the case.   In reading I have realised how much more there is to know, how many more perspectives there are to every situation, event or concept, how much more I have to read.    Reading has been enjoyable and but also enriching.

Using the idea of Umberto Eco’s Anti-library as mentioned by Nassim Taleb in the Black Swan, I have realised that my bookshelf with its books complete with post-it note annotations is not important.   What matters is the understanding that there is so much more to read; the list of the books I am yet to read.    With every new book I read the list of books yet to read does not decrease, but increases as I add new books to it based on my current reading.  I open up new avenues to explore with each book read.

And with that I will go back to reading Incognito.   I have already added Herd by Mark Earls, Predictably Irrational by Dan Ariely and Nudge by Cass Sunstein to the not yet book list.   Feel free to  share your recommendations.

 

First days, and broken toes

The following a second posting resulting from some of the events of my time working out in the UAE.   It is not the normal fare for this blog however I thought I would share:

 

id-100414457Having arrived as a family in Abu Dhabi in the early hours of the morning, the first thing which we had to deal with was the two hour bus ride to our eventual home in Al Ain.     The four of us, being myself, my wife and our two children, were joined by a number of other families all coming to work with the same company as we boarded a bus, and I hasten to point out that it was a bus, and not a coach.

The conclusion to this journey was our arrival at what was to be our home for almost the next two months.    The Hilton hotel in Al Ain.    Checking in we met with our first significant problem of the trip being that only a double room had been booked for us however as I mentioned a moment ago we were four individuals.    This meant that a single hotel bedroom was a little on the small side.   Sadly given we had arrived on the weekend meant that there was no-one available from the company I had came to work with, who could resolve this issue with the hotel.

This eventually meant that we, and all of our luggage, were helped up to a single bedroom which if memory serves me correctly was on the fourth floor of the hotel.    The bell boy helped us get the luggage into the room as we supervised the children.   Limited in space he made the inappropriate decision to place one of the suitcases on a table.

It wasn’t long before, given the limited space, the table was bumped and the suitcase came crashing down, landing on my wife’s foot.    I can remember thinking to myself, “what a great start this is, only been in the country for less than 24hours and we already have an injury!”

Now at first we didn’t take the injury to be that serious, or at least I didn’t take it to be that serious.     As a result I thought walking it off would be the best course of action, thereby setting us all off in a mission to walk to the local mall.   Now sadly I had misjudged this as the local mall was not as local as I had thought, plus there was the added issue of the change in temperature as only hours earlier we had been in the winter of the UK whereas now we were in the winter of the Middle East, some 15 to 20 degrees warmer.   All of this did nothing to help the now throbbing foot injury which was visibly causing my wife to limp and grimace in pain.

Upon finally getting back to the hotel it was clear that medical attention was required however we knew little of our options in this regards.    As such we spoke to the Hiltons manager who was all to obliging although thinking back this may have been the result of concerns he held with regards potential liability or injury claims.    He pointed us in the direction of a local hospital which we duly got a taxi to transport us to.

Arriving at the hospital our next hurdle was the fact that hospitals here in the UAE expected you to have medical insurance from local companies.    The travel insurance we had did not appear to be something they had to deal with very often and as such the easiest option was to pay “privately” meaning that each doctor we say and each X-Ray, etc had to be independently paid for.   Thankfully at this point I had a reasonable level of cash with me given we had no long arrived in the UAE so this did not pose much of a problem.    Some hours later the diagnosis was provided, that my wife had a broken small toe.

Now the key messages from this are to make sure you know about accessing medical services, etc when going abroad; something I didn’t do.  The second message is to take injuries which occur to your wife seriously no matter how minor you think they are, or otherwise you might still be living to regret it as I do, some seven years later!      And thirdly, “walking it off” is never a good idea either!!

Photo: “Film X-ray Both Foot ( Front View )” by stockdevil from freedigitalphotos.net