Fact checking the free school meal policy

With Free school meals likely to be in the news over the next few days, I have revised my old blog on the UIFSM policy.

Every time you see an interview with a supporter of the UIFSM policy, watch out for these misleading statements and scroll down for the actual evidence.

1  This policy is well designed and not part of some political fix

2  The UIFSM policy is properly funded

3  The pilots showed a Dramatic improvement in attainment

4  There are “health benefits

5  There are “improvements in classroom behaviour”

6  The policy offers value for money

7  UIFSM was recommended by the SFP

8  The policy has cross party support

9  The policy will happen in September

10  The policy is evidence based

As a bonus couple of misleading statements, not downright lies, but not the whole truth.

A. The pilots managed to implement in just 12 weeks

B. Pods are an answer for schools with no kitchens

1.       When Nick Clegg announced his UIFSM policy, there was no mention of funding for infrastructure. Clegg had no idea about the state of school kitchens, or school halls, if caterers could ramp up production, if equipment was available, he had no understanding if what he was promising was actually deliverable?

If it was a well-designed policy why the repeated changes, why the need to find an extra £150mthen £17m and now £22m, why the delays in explaining practicalities?

How well designed can a policy be when they didn’t even realise thousands of schools needed kitchens, a policy that has a red rating and is at serious risk of failing!

As for claims it isn’t part of some sleazy political deal, David Laws in evidence to the education select committee, quite specifically linked the UIFSM policy to the conservatives Married couples tax allowance, as did Dominic Cummings in his blog, to suggest it isnt a fix, is disingenuous at the least, the Lib dems even bragged about it?

After some FoI responses I recently blogged about how little thought went into this policy, if this area interest you, i would recommend it

I also blogged on the political fix over the business case, again well  worth a read

2.        If you hear anyone state categorically that this policy is fully funded, ask them one simple question, “How do you know”?

It has always been clear the infrastructure funding requirement was underestimated. £150m sounds a lot (originally £200m), but in the scale of school buildings it is minute, needless to say, it won’t build 1700 new kitchens. but the Dfe knew this, they knew that in the pilot, Durham (pg 17) found “The average cost per school of extending and equipping school kitchens and dining facilities was reported to be around £20,000”

I could run a page of links to schools and LAs stating there is not enough money to fund all the infrastructure changes they need.

I would just make this simple point, how can anyone claim the funding is sufficient, when councils are only now realising the scale of the problem? The funding was never calculated on the basis of what schools need, so no one can realistically know if £150m will meet the actual need?

How long is a piece of string? Nick Clegg has no idea, he just thinks he has allocated enough on the basis that is how much he was given.

Just as serious in my mind is the issue of the ongoing costs.

On 12 march, David Laws appeared on Radio 4s World at one and said “No schools has said the revenue funding allocated money is anything other than appropriate” I think the latest Guardian article shows that up as selective deafness?

Clearly for many schools the ongoing costs will exceed the £2.30 funding.

The 2011-12 mean cost  (pg 39) of primary school meals was £2.30, meaning many will have paid more.

The Newham Pilot started with a cost of £2.59 in 2009 rising to £2.63 in 2011. (pg 126)

Even in the school food plans own evidence, 5 out of 47 primary schools had production costs above £2.30

The small schools task force provide the only up to date modelling I have seen, I have serious questions about their statistical process, but even with their optimistic projections, almost every scenario is in deficit. (Pg 25 )

Sadly the 2011-12 report  on school meals was the last one available, as Michael Gove ended the funding, however since then, Food prices have increased, utility bills have risen, new charges for disposal of food waste have kicked in and we will shortly see a rise in the national minimum wage.  All of these add inflationary pressures to school meal prices and whilst some of this will be mitigated by economies of scale and optimising working practices, many schools will face an increase in their costs above and beyond the £2.30 funding.  The scale of this problem is unknown, it may be only a few, it may be a significant number, sadly (or fortunately) no one is collating and publishing the figures. One thing is certain, some schools will be forced to subsidise the policy from Other Budgets.

Recently we have had an LGA survey telling us that budgets are being hit and that LAs are £25m out of pocket, i believe that is a gross underestimate and doesn’t include the amount individual  schools have had to find.

3.        Section 4 of the UFSM Pilots report details the academic improvement and the differences observed in infants are decidedly unimpressive; over the 2 years the pilot saw a 1.9 percentage point (ppt) increase in the proportion of pupils reaching the expected level in reading at Key Stage 1 and a 2.2ppt increase for maths at Key Stage 1. Overall, this it barely statistically significant!

It was also noted that for a large number of children UFSM actually have a negative effect on academic achievement.

 “there was no significant effect on the standardised average point score for the population as a whole in area B”
 “the pilot had no impact on pupils who scored in the highest quartile ” the trail actually had a negative impact on the highest performing 25% of children.

Perhaps the most important comment is “It is important to note that the mechanisms underlying the improvements in attainment observed in the universal pilot are not clear“. Ie improvements might not be as a result of UFSM.

It is also interesting to note that there WAS significant improvement for KS2 children, but a decision was made not to target juniors and instead concentrate on Infants. Watch out for politicians claiming KS2 evidence in support of their KS1 policy or pretending improvements over 2 years are comparable to those seen in one year of other interventions.

An issue I have previously flagged and should be of concern to everyone, is the lack of assessments into the implementation and ongoing benefits of this policy. How will we know it is does what ministers promise? 

4.        On page 3, the UFSM pilot report states “There was no evidence that the FSM pilot led to significant health benefits during the two year pilot period. For example, there was no evidence of any change in children’s Body Mass Index.” This doesn’t stop supporters pretending otherwise. It seems to be common now to hint at non existent  health benefits by stating in the pilots, children ate more healthily. Eating more healthy food is inevitable if you give them free healthy food? It is no great achievement and doesn’t mean there were health benefits?

5.       Laws has previously claimed “Schools also reported improved behaviour and atmosphere, as a result of all pupils (and an increasing number of teachers) eating together every day” the report states “The evaluation did not include a quantitative assessment of classroom behaviour” so any claims have no statistical basis and it is inappropriate of Laws to claim causation. (See pg 84)

I have heard people asserting that a school food trust survey found evidence of better behaviors,  but with a sample size of 135 it was not significant new evidence

6.         There are four problems with claiming the policy offers Value for money.

the pilot (pg 143) found only one intervention offered worse value for money, that was the highly targeted one to one tuition ”Every Child a Reader” which was only marginally more expensive at 2011 levels.

The claims that UIFSM had greater impact than literacy hour are also shown up as a lie, KS2 FSM demonstrated a marginally better score, but NOT when compared to Infants who after all, are where the policy is targeting. Worth noting the  huge difference in VFM

The costs involved in the pilot and used in the vfm calculations were £220 per pupil per annum exactly half the £440 the new policy will cost. This dramatically changes any vfm calculations.

The improvements in attainment are dubious at best, as I have explained

We need to look at a wider range of other interventions and the vfm they offer, conveniently there is a full list of comparable fully costed interventions.  It is interesting to compare the VFM of UIFSM at 4 weeks progress per year with today’s rate of around £400 per percentage points.

i touch on this in my blog on the business case used by the Treasury

7.         The SFP recommended (pg 127) “Government should embark upon a phased roll out of free school meals for all children in all primary schools, beginning with the local authorities with the highest percentage of pupils eligible for free school meals”

No infants first, No big bang implementation. No statutory requirement. The decision to ignore the SFP recommendations and implement this particular policy was purely political, there are significant difference between the School Food Plan and this policy, someone must have made a conscious decision to disregard the recommendations, yet David Laws still tries to pretend that the UIFSM policy was in the SFP, at one point, lying to parliament (q 121).

the process involved is expanded in this blog

8.          I need only to point you to the revelations in Dominic Cummings blog to demonstrate how ridiculous this is. Most telling is Goves leaked letter to Danny Alexander, in it the SoS says “I am sure that you will agree with me that we must not risk forcing schools to subsidise meals by reducing their spending on teaching and learning. I would be grateful for your assurance that my budget will be increased to fully cover the cost of this new commitment, as I am unable to fill any shortfall

Schools are having to fund the initial set up costs from existing budgets!

Putting out a letter telling people there is unity whilst at the same time having the Deputy Prime Minister is telling people he ordered Michael Gove to comply, is clear lunacy

9.          We regularly hear glib statements insisting “it WILL happen” along with ultra-positive management speak like ‘good preparation and a cool head can overcome all difficulties’, “Everything is possible where there is a will”

Yet at the same time Nick Clegg acknowledges there will be “teething troubles” at one point the lib dems even suggested schools ignore the new law insisting “schools will be given more time if necessary” this was removed from most sites once the stupidity of the comment was understood.

Meanwhile on BBC Question time (15;15) Paddy Ashdown tells us  5-10% will fail to meet their statutory obligations.

Perhaps it is reasonable to ask what exactly WILL happen? what are the success criteria? what will count as failure? what is being measured and by who?

The help lines can’t help with funding, tell them your school needs a new kitchen and they refer you to the LA, ask for a visit and they charge you. Helplines to spread well published and disseminated best practice  will never solve every problem.

With the start of term imminent, i am sure we will hear how well it is all  going, but much of this will  be despite the politicians, not because of them.  head teachers, staff and governing bodies will have invested massive numbers of hours into getting this into a practical  state where it will work. for some schools the transition will be easy, for a significant minority it will be a nightmare.

Lets remember when Nick Clegg says there will  be teething problems, the reality translates into infant children will be hungry at school

10.        Public Health England reviewed the pilot evidence and concluded

“An evaluation of the Free School meals pilot in England found that the universally extending provision of school lunch eligibility to all students impacted on attainment at Key Stages 1 and 2. However, the study was unable to establish if this was the result of providing free school meals or the wider package of activities associated with the pilot. The extension of eligibility below the level of universality did not demonstrate an improvement in achievement”

So no causation was proven?

The pilot report itself acknowledges there are doubts about any causation

“These effects on attainment could have arisen through the provision of free school meals directly or through the wider activities that accompanied the pilot, such as the promotion of school meals and healthy eating to pupils and parents, or both” (pg 2)

“The source of these improvements in productivity is not clear” (pg 3)

“It is important to note that the mechanisms underlying the improvements in attainment observed in the universal pilot are not clear.” (Pg 14, para 4)

There are significant questions over the methodology used in the pilot scheme, eg no account was made for the cohort effect, concern over the controls used in to calculate improvements, ie the control areas might not actually be that comparable and some conclusions were drawn from self reporting questions to children.

It would also appear that supporters avoided any systematic reviews and ignored any studies which shows up any negative impact.

I would point you to Louisa Ells report; A systematic review of the effect of nutrition, diet and dietary change on learning, education and performance of children of relevance to UK  schools (1.1.5)

“The findings from this review suggest there is insufficient evidence to identify any effect of nutrition, diet and dietary change on learning, education or performance of school aged children from the developed world. Further research is required in settings of relevance to the UK and must be of high quality, representative of all populations, undertaken for longer durations and use universal standardised measures of educational attainment. However, challenges in terms of interpreting the results of such studies within the context of confounders such as family and community context, poverty, disease and the rate of individual maturation and neurodevelopment will remain”

No mention of this damning conclusion was made by the pilot report or the school food plan who instead prefer marginally statistically significant “evidence” to assert UFSM had educational impact.

As a bonus, some misleading statements which though technically not true, are blurring the lines of credibility.

  1.   It seems a standard answer to anyone questioning the timescale to point to the pilots and say they only had 12 weeks.  This is stretching reality to suit. My understanding is that the pilot areas had to bid to be included in the scheme, they had to put forward properly worked out plans well before the start date. It was then just a process of enacting those plans (I accept that was still some challenge) The pilot areas had significant specialist support on the ground, not some lame helpline or poorly attended roadshows, they could call on people to visit schools without charging any fees.

I believe it is wrong to extrapolate the relatively benign circumstances of Newham or Durham as neither are representative of the challenges faced by many areas. (Worth noting individual Durham schools had to match their LA funding allocation)

The big difference however is the scale, currently every supplier is at full stretch working out what is needed, LAs are struggling, resources are stretched much thinner and the high level of funding received by Durham (£17,000 per school) is simply not there.

  1.   Pods are completely unsuitable except in very Very VERY specific circumstances. They will cost schools a minimum £7k PA and then some. Which schools can afford that ongoing? UIFSM supporters like the idea of pods because it is a superficial quick fix that gives them an easy answer to critics who highlight schools with no kitchens. Clearly they have not thought through the cost implications nor understood the economics, but they still promote them as though they were on a commission. Pods are not a panacea!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s