The question that goes before ‘What works?’ in education research

This article was originally written in March 2013 in response to a speech by Ben Goldacre and the publication of a UK government report on educational research. 18 months on and the ‘what works ‘ agenda is becoming more embedded in English school culture and has impacted on the practice of education departments in universities with some requiring all their researchers to carry out at least one randomized control trial each year. The issues are still pertinent and the need for debate and response is as great as ever. This blog post is intended to provoke and promote engagement with the state of play in England and the potential implications of this change of direction for educational research elsewhere. The tension between research that can inform policy with its positivist generalizations, and research that has traditionally informed practice with its interpretive approaches is becoming more and more polarized. This author argues that there is a need to re-examine the relationship between policy, research and practice in order that the needs of learners and those who teach and work with them can be better understood and met.

It is heartening to hear an academic who supports, and recognises the value of, teacher training and professional development that involves ‘learning the basics of research’ and understanding how it is done, training and CPD that allows teachers to be ‘critical consumers’ of research, systems for disseminating the findings of research to teachers on the ground, and the inclusion of randomised control trials (RCTs) as a methodology in educational research. (Goldacre 2013). It should be even more heartening when such views are voiced by an academic who has been invited to contribute towards cabinet office thinking about evidence informed policy making in education as it suggests that these views may come to be realised in practice. However, the actions of the government, and the secretary of state for education in particular, since coming to power, suggest that there may be a less encouraging agenda underlying their implied support of this well-intentioned rhetoric. For example:

1)  The General Teaching Council for England, which championed and supported teacher led classroom based research aimed at finding out what works and improving practice, was closed down in 2012. Scotland, Ireland and Wales still have their GTC’s

2)  The Teacher Training Resource Bank (TTRB) a free web based resource making the evidence base for learning and teaching available to teachers, (not just those in training), and facilitating teachers’ critical consumption of research, was privatised in 2010 and subsequently closed down in 2012.

3)  The revised standards for initial teacher education (2012) do not include any reference to learning about research and how to do it. In addition, School Direct allows for trainees to work up to 90% of the timetable of a full time teacher (even though they are not paid at all or are paid on an unqualified teacher’s salary) which raises the question of when and how trainees are supposed to hear about research let alone understand and do it. (All other professions require between 3 and 7 years of training.)Teachers at all levels are appraised, judged and rewarded in relation to these standards that do not mention research.

4)  Professor Andrew Pollard, who has been writing about and facilitating evidence-informed practice (lin ) since at least 2002, and who set up the Teaching and Learning Research Programme which brought teachers and researchers together, was not invited to contribute his extensive knowledge, experience and expertise. Neither were any of his colleagues from the TLRP programme despite the fact that it provided a robust and rigorous set of resources through which teachers could understand what research is about and how to do it. Neither was David Hargreaves, another very experienced and expert education researcher who outlined a national strategy for teacher based research and a national research forum in 1996. There are many, many others, including some teachers and head teachers – the point is that no one with direct educational experience (of any kind other than having benefited very well from their own educational experience) has been consulted, or listened to, by a cabinet office deciding how educational research should inform educational policy.

5)  Professor Marilyn Leask who has set up an online teacher knowledge network that facilitates scaling up promising small scale research, amongst many other things, and who is well known for her work on building the evidence and knowledge base for both teacher education and teachers, has not been acknowledged despite the fact that she has a working infrastructure that is already achieving much of what Dr Goldacre’s vision of a ‘dating service’ for teacher researchers would be, and what the education secretary is advocating should be done.

6)  Funding for teachers to undertake school based study at Masters level, that is, to learn about research and how to do it and to become critical consumers of research, was stopped in 2010 despite an evaluation report by Soulsby and Swain in 2003 that showed such programmes had significant impact upon the development of teachers and children’s learning.

7)  Valid, robust and extensive educational research, such as the Cambridge Primary Review, which tells us a lot about what does work in education, Neil Mercer’s research on dialogue and learning.

which tells us a lot about what is effective in teaching , and the work of Maurice Galton, whose research tells us a lot about transition and expert teaching, have been consistently ignored by successive governments including the current one. This could also be a much more extensive list.

This curious contradiction between intentions and actions can be best understood with reference to the broader policy context. In order to clearly, and with as much brevity as possible, set out what is a fairly complex issue, I shall list a series of points, which together present a view of the perils that might lie ahead unless we can all be critical consumers of education policy as well as critical consumers of educational research.

1)  Edu-business is big business. It is impossible (for me, at any rate) to get any more recent figures to illustrate this than those found in Santos (2004) who notes that ‘Capital growth in education has been exponential, showing one of the highest earning rates of the market: £1,000 invested in 1996 generated £3,045 four years later…. that is an increased value of 240%, while the London stock exchange valorizaton rate accounted on the same period for 65%.’ He also notes that other 2004 data indicate that current comercialised education, incomplete as it is, already generates $365 billion in profits worldwide.

2)  ‘Austerity’ has created opportunities to accelerate the privatisation and marketisation of all public services, including education. Apart from concurrence with neo—liberalism and neo-conservative ideology, this also enables the government to attract new investment, and to generate income through the sale of its public sector, as a means of boosting a declining economy . As noted by Whitfield, the transfomation from public to private has four key parts: ‘…financialisation, personalisation, marketisation and privatisation, all intended to radically change the governance, funding, design and delivery of public services. Peel away the rhetoric and rigorously analyse the evidence, and it is clear these policies are designed to dispossess, disinvest, destabilise, depoliticise and disempower. This amounts to the deconstruction of democracy at local, national and international levels.’

3)  The move from education as a public good to education as a market commodity is most recently evidenced by the Academies Act 2010 and the School Teachers Review Body 21st Report 2010.

Deregulation enables the government to cut its labour costs and weaken the rights of employees as part of setting the field for direct profiteering from education. Neo-liberal economies and neo-conservative ideologies require that the state is strong on controlling education and teachers, and strong on marginalising oppositional democratic forces such as critical teachers and other educators, researchers, students, trade unions and other locally elected democratic representatives. (see, e.g. Gamble 1988)

4)  Whilst the rules of governance and financial management of schools have been relaxed and many restrictions removed, the central control of the curriculum, testing and school inspection has increased, and is set to continue to increase in detail and scope (one illustration of the point above) This includes centrally set minimum targets for achievement in national tests, and judging the effectiveness of schools and teachers against them. In addition, the secretary of state for education has increasingly drawn greater and greater powers to the role, including the powers to direct the closure of a school, make an academy order, and temporarily suspend or modify any education legislation that may be holding back, or stopping, ‘innovative approaches’ to raising standards. With no requirement for any educational training or experience, the secretary of state for education is now able to direct and control any aspect of teaching, learning and school management s/he desires, giving whoever holds the post what Wilby describes as ‘the powers of an elected dictator’.

5)  Eugenic ideas (bear with me) in the first half of the 19th century, were strongly embraced by many countries worldwide, and supported by a range of national and international scientific organisations and publications. (NB there was not a scientific consensus on eugenic ideas but this is not the point here) Hansen and King note that ‘despite the fact that both the U.S and Britain shared similar liberal values, policy outcomes varied sharply: eugenic policies were wide spread in the former and essentially non-existent in the latter.’ (p321) After a detailed analysis they conclude that ‘…ideas are more likely to be translated into policy … when there is synergy between ideas and interests (that is, when actors believe that taking up these ideas will serve their interests), when the ‘’carriers’’ possess the requisite enthusiasm and institutional position, and when timing contributes to a broad constellation of preferences that reinforce, rather than detract from, these ideas.’ (P262) The government in general, and the secretary of state for education in particular, are at this time in need of ideas that will serve the ideological agendas of neo-liberalism and neo-conservatism and, more specifically,  the privatisation of the welfare state.

6)  Reification – giving a concrete (usually numerical) value to abstract or socially constructed ideas is essential for the workings of the market where numerical empirical data is needed in order to give value to, buy, sell, measure and evaluate market commodities. This includes learning, teaching, the effectiveness of teachers and their strategies, learners and their behaviour and so on. You can no more sell a car or house on the basis of a qualitative account of its value than you can a learning intervention or a school. That is not to dismiss all reification out of hand – there is a place for this in evaluation, assessment and research in education, but in all these areas the assignation of numerical values is contestable, problematic and complex. As teachers and researchers we should be engaging with the complexity rather than the reductive simplicities of reification.

7)  RCTs can only work alongside the reification of some extremely interpretive, abstract and/or socially constructed concepts. That is not to say that there is not a place for RCTs in education either – they can have a useful predictive function for some things. As noted by Dr Goldacre, they have to be carried out alongside other kinds of research. In particular, process evaluation, so that we can know whether a particular intervention failed because the intervention was no good, or because it’s implementation was poor. Process evaluation for learning interventions would require some kind of assessment of teaching performance against the requirements of the intervention being tested in the RCT. Think about this for a minute

8)  The debates stimulated in response to the cabinet office paper ‘Developing public policy with randomized control trials’ and Dr Goldacre’s speech have not explicitly addressed, or even clearly acknowledged, the very important distinction between methods (the means of generating, collecting and analysing data), and methodology (an analysis and justification of the principles, rules and criteria that construct the framework for the research, and justification for the approach in terms of the validity of the knowledge it can produce in relation to the question.) Reducing methodology to a ‘simple trick’ of ensuring we ‘find the right methods to answer the right questions’ (Goldacre 2013) focuses purely on the technical and ignores the philosophical issues that underpin any good, reliable and valid research. This point needs a great deal more explanation and discussion and I intend to address this in more detail in a future blog. However, it is at the crux of the debates around the role of teachers and other researchers in education, what counts as good research, and why. In order to briefly expose the tip of the ice-berg here I shall illustrate with a couple of scenarios:

• Two research projects asking the same question about an aspect of classroom behaviour management and student learning. The first is premised on a process-product view of teaching and research and finds that teachers need to bolster their authority over the students. The second is premised on a social-constructivist view of teaching and research and finds that teachers need to learn to mitigate aspects of their authority because learners who are subordinates cannot participate in many of the activities and forms of discourse that would lead to genuine education. (Cited in Donmoyer 2006:22). The methodology is ideologically grounded – which one is best depends upon what we think the role of the teachers is and how we conceive teaching and learning. The same methods can be used with either methodology.
• The question is ‘Which approaches to teaching reading work best?’ person A believes that one of the aims of education is to initiate individuals into the values attitudes and behaviour appropriate to active participation in democratic institutions and processes, and to educate people as critical producers and consumers of knowledge. Person B believes that one of the aims of education is to offer a minority an education appropriate for political leadership, and to initiate the majority into the values, attitudes and behaviour appropriate to their role as producers, workers and consumers, including the market’s need for the majority to be easily influenced by the media and advertising. Would an RCT designed by A be the same as an RCT designed by B?

I fear that there is very little evidence in the current economic and political context to suggest that educational research, wherever and by whomsoever it is carried out, will be little more than a sophisticated market research activity and part of the government’s mechanisms for maintaining a tight control from the centre on teachers, students and others in education. Eugenic ideas were taken a step too far in the Second World War and so their influence and place subsided. Despite the fact that eugenic ideas were premised on some contentious and misconceived ideas about heredity, the science at that time was ideologically acceptable and so dominated. (They were still very dominant in the UK even though they did not enter policy.) Eugenic ideas are now making a come back, albeit as ‘liberating’ eugenics and premised on many advances in the understanding of heredity through such things as the genome project.  What eugenics illustrates is that ‘What works’ in both science and education is more about what kind of society we want, our perceptions of human beings and human nature, who should be privileged when it comes to sharing resources, and who should have what kinds of power over whom in decision making and action, than it is about methods. We ignore this at our peril.

 References                                  

• Donmoyer, R (2006) ‘Take my paradigm…Please! The legacy of Kuhn’s construct in educational research.’ International Journal of Qualitative studies in Education, vol 19, no. 1, 11-34.
• Gamble, A. (1988) The Free Economy and the Strong State: the Politics of Thatcherism. London: MacMillan.
• D Soulsby, D Swain (2003) A report on the award-bearing Inset scheme. London: TDA
• Santos, B. (2004) A universidade no século XXI. Sao Paula, Brazil: Cortez

Share on Facebook

Tweet