Last weekend, the Mail, Telegraph and Independent all published stories about free school meals making children fatter, given the sensationalist headlines, I though it an appropriate time to make public some analysis I have been doing.
My thought process
Towards the end of last year, we saw huge coverage of the latest HSCIC data on children’s weight. Rightly there was outrage at the fact that around a third of our children were leaving primary school obese or overweight. It is clear that as a nation, we have a huge problem with children’s health and the amount of weight our children put in from the age of four to eleven. What was less well publicised is that these obesity and overweight rates have actually been fairly static for a number of years.
As someone who has long questioned the appropriateness of the state paying for universal free school meals, I started to re-evaluate my position. What if providing a healthy school meal made a difference and cut obesity rates, what if a universal free school meals policy was the elusive silver bullet to address the issue. After all a number of learned Health experts wrote to The Times suggesting the policy could pay for itself.
I looked again for evidence to back up the proposition that universal free school meals improve health and help fight obesity.
The Times letter was frankly hopeless, filled with lots of might’s, coulds and mays. Disappointingly there wasn’t a shred of evidence in there. It seemed little more than a PR exercise designed to garner sensationalist headlines, which is particularly shameful given the names and organisations on there.
I looked again at the only real research, the 2009/12 universal free school meal pilot, which states “There was no evidence that the FSM pilot led to significant health benefits during the two year pilot period. For example, there was no evidence of any change in children’s Body Mass Index.” Not a particularly helpful line if we are hoping to conclude UIFSM improves health.
I couldn’t find anything else, there was no evidence.
But one thing I am not, is a quitter. So I looked again at the National Child Measurement Programme data and discovered they banded children within LAs. This only really becomes interesting when the knowage that two LAs; Islington and Newham, run their own universal free school meal policy and have done for a number of years. The NCMP also helpfully took measurements in reception as well as in year six, providing a baseline for the current cohort, effectively we have a ready made longitudinal study.
If universal free school meals improve the health of children, then within these two LAs, we should expect a gradual increase in the number of healthy weight children over time as they are exposed to the policy for increasing amounts of time.
Equally if UFSM impact on obesity it would seem obvious that the obesity rates in these two LAs should be significantly lower than other similar LAs.
I initially went into great detail explaining my methodology and process, but it’s all a bit boring, so I added it as a note (1)
These are the Yr6 obesity rates for the last 6 years for the two LAs running universal FSM and for London and England as a whole
There are the Yr6 rates for pupils who have healthy BMIs for the last 6 years for the two LAs running universal FSM and for London and England as a whole
Compared to London as a whole, England and to other LAs, I found no statistically significant change in obesity levels for Islington or Newham. If anything Newham has seen MORE obese children, though it’s not significant (though the increase is enough to give the Mail it’s headline)
Compared to London as a whole, England and to other LAs, I found no statistically significant increase in levels of healthy children for Islington or Newham. If anything Newham has LESS healthy children. (there was no significant difference in the number of underweight children either)
In simple terms this is proof that on their own, universal Free School Meals do not make any difference to children’s BMI.
In addition to my own analysis, I asked both councils for any internal assessments done on the policy, Newham and Islington both confirmed they hadn’t done any assessment of any kind. I find this lack of monitoring remarkable, given these LAs spend considerable amounts of money on the project. It doesn’t seem unreasonable to expect someone would check to see if the policy managed to lower obesity rates, especially given that tackling obesity seems to be the main justification for the policy.
Whilst I would never claim universal FSM makes children fatter, I think its fair to state the evidence shows that the provision of universal free school meals have made no significant impact on childhood obesity rates as measured by BMI.
Why does it matter
It shouldnt come as any real surprise that universal free school meals do not make any difference to health or obesity. The drivers of eating badly and lack of exercise will have much greater impact than free provision of 190 meals a year. It should also be noted that these meals are often not as healthy as many would have us believe. The new standards are not nutritional standards, they are food based standards that place no limits on a meals fat, sugar or salt content. Even then, these standards are not applicable to all schools and even if they were, they are not monitored.
Sadly we are stuck with an underfunded UIFSM policy and landscape increasingly run by the multinational catering companies. School catering has become dominated by cheap labour and even cheaper ingredients.
The vested interests within the catering industry are incredibly keen to see this policy continue. They consider Uifsm to be a “Financial bonanza” and have gone to great lengths to pursue and promote their agenda. Whilst I have long shared their newly found concern about lack of evidence and the governments decision not to research benefits, their concern is driven more by a fear that the policy might get cut and damage their profits than any real desire to provide any new insight.
I believe, for the first time we have gone from having no evidence that UIFSM makes a difference on health or obesity, to being able to clearly demonstrate that Universal meals make no difference to children’s BMI.
This entirely contradicts unjustified and misleading claims about the health benefits of UIFSM, which is why this evidence is so important. More so given the recent changes to the SATs tests. meaning there is no longer a comparable baseline, making it impossible to measure any changes in attainment that may accrue as a result of the policy. This leaves “health” as the only justification supporters can use, meaning the catering industry continually jump on the obesity bandwagon to promote the UIFSM policy.
Politicians appear to have fallen for this line, to the extent that in saving the policy the former Chancellor placed UIFSM under the headline “security” as if it was protecting our children (2)
In these days of evidence based policy, we need to look closely at the benefits, I believe this is new and compelling proof that unioversal free school meals does not combat obesity, the next decision now lies Justine Greening ,who needs to set out some success criteria and check it offers VFM before throwing yet more money at the scheme.
(1) Initially I looked at the latest data to see if Newham and Islington had significantly lower obesity rates than anywhere else, as it happens they both have extremely high rates, so I obviously had to compare them against similar areas.
I then looked at a couple of London boroughs with similar demographics and similar 2015 profiles to see if Newham and Islington had significantly different outcomes.
I then went further, looking back at the 2014\15 yr 6 cohorts’ baseline data from 2008/9 when the children were in reception, to see if there were differences.
In all of these exercises it was clear that whilst Newham had amongst the highest obesity rates and lowest healthy children rate in the country, Islington fared slightly better, though still at the high end of the scale.
Finally in order to use a fair start point for any comparison I looked at the 2008/9 reception data, found a number of LAs with similar 2008/9 baseline figures and similar demographics (I looked at transiency, mobility and ethnicity) to both Islington and Newham. In addition I used the 5 LAs used by the UFSM pilot to compare against Newham as well as the data for the whole of England and London. I looked only at the 2008/9 reception data so as not to influence selection based on the year six outcomes I would then measure.
Using the reception data to direct my selection, I then tracked the year six data from 2008/9 to look for any significant differences and to look at trends.
Individually the picture looks fairly chaotic, with various bumps and jumps from year to year, but looking at trends for each of the chosen LAs, it became fairly obvious that they were all fairly level and that overall, the percentage figures for London and for England as a whole give a good indication of overall trend, though with different start points.
It is important to note that this is a substantial sample, with around 95% of all relevant schoolchildren taking part each year. Well over a million children are measured with confidence intervals over 95%. Along with the careful selection of control groups who haven’t offered universal FSM, the scale will mitigate many of the variances, like pupils leaving, or a particular poor or especially good caterer.
Now, this isn’t a full scientific trial, it isn’t an RCT and I don’t clai t to be scientific research, I am not a professional statistician and would add the caveat that I may have missed something, but I have run my process and findings past a number of educational data experts who are all satisfied with my methodology and with my workings.
Saying all that, I have some statistical knowledge and have done my best to be as thorough and unbiased in my choice of data and how it is compared as possible. I am more than willing to provide my files to anyone who wants to go through them.
If anyone has any better suggestions of how we can analyse the benefits of the UIFSM policy, do contact me and explain.
i believe this is as good and thorough an analysis of the data as anyone can do, short of spending millions on an entirely new research program.
One last thing, I understand that BMI isn’t the best way to measure health, but it’s simple, commonly understood and gives historically comparable figures via the base data. I am looking specifically at the benefits of universal FSM on their own, as opposed to being healthy, adding in exercise or school food systems in other countries with different standards and funding.
(2) 2.57 Investing in education and skills will help deliver economic security. In addition to providing 600,000 additional school places, funding for universal infant free school meals will be maintained, supporting healthy eating and saving families around £400 for every infant each year.
This is the full table of LAs I used and their obesity rates, over the next week or so I will post links to all the data.I collated (if there is a demand for it)
|Obese Year 6||2008/09||2009/10||2010/11||2011/12||2012/13||2013/14||2014/15|
|Newcastle upon Tyne||21.9%||21.9%||24.9%||25.0%||22.8%||23.0%||24.0%|
|Hammersmith and Fulham||22.4%||24.0%||23.7%||25.8%||20.1%||22.4%||23.3%|
|Barking and Dagenham||24.2%||23.6%||24.2%||26.9%||24.4%||26.2%||25.3%|