People come off JSA at the same rate, regardless of workfare

- Edit
- Delete
- Tags
- Autopost
People come off JSA at the same rate, regardless of workfare

Evidence matters in politics: the question of whether a policy meets its stated goals is a matter of fact, and separate from the more subjective issue of what those goals should be, or whether the means are proportionate.
"Workfare" or "work experience" for people on JSA has been controversial. David Cameron has said that people on these schemes do well, with half of them off JSA in 12 weeks. Is that good? This question is explored in two good blogs from two good people, FullFact and Jonathan Portes from NIESR.ac.uk
http://fullfact.org/factchecks/benefits_work_experience_scheme_participant_outcomes-3340#overlay-context=node/3342
http://notthetreasuryview.blogspot.com/2012/02/work-experience-does-it-work.html
Bottom line: it turns out people leave JSA at roughly the same rate, whether they're doing workfare or not.
Graph via FullFact and Portes.

I would add one thing:

For all we know, the people who are even offered workfare might be systematically different to the people who aren't. For example, they might be the very people who the job centre think are least likely to get back to work. In this case, we should do a randomised trial. Job centres can identify people who they think should go on workfare, then we split that group in half as we go: half of them get workfare, half of them get "normal" treatment, and we compare how many people in each group come off workfare. Then we'd know for sure. Trials aren't hard and they give you the best evidence.
- 23 February 2012
- Views
- Permalink
- Tweet
3 months ago PeteAtRaspberry (Twitter) responded:

I thought you were against people publishing headlines which make sweeping claims based on incomplete data?
You may well be correct - Workfare may not have a beneficial effect on the rates of people leaving JSA overall - but the people that provided that graph, in the links you provide, state that the data isn't sufficient to draw that conclusion.
It is correct to say that politicians can't use the figures to make the claim the other way either. But a scheme which deliberately targets people who might otherwise struggle to get off JSA, and has results on par with the average, is surely a success...? Please note I'm not stating that interpretation as fact, merely that it would also be potentially correct from the data so far. More work needed here to properly establish a control.

3 months ago thejoeturner (Twitter) responded:

I wonder how many of those have left JSA because they've got jobs, though.

3 months ago Warfreak2 (Twitter) responded:

That's a good point - getting people off JSA isn't itself the goal (otherwise the solution would be trivial: stop paying JSA at all). What are the rates of conversion from JSA -> employment, for workfare and non-workfare?

3 months ago steviesun (Twitter) responded:

Doubt DWP would approve a trial like that. But if true - it could make HUGE savings.

3 months ago niall-l responded:

The trouble is, there is a suspicion that the ones being forced on to workfare are those the providers think are *most* likely to get a job, while they "park" the rest.
If this is the case those not on workfare are slightly (probably not statistically significantly, but I don't know your sample size) more likely to get a job than those on workfare, giving the lie to assertions that many of those on workfare get offered jobs (this may be true, but at a lower rate than would happen if they were looking for jobs).
If this is the case, workfare is hindering those most likely to get a job from actually getting one.
As noted, however, we don't know the sample characteristics, but if the public suspicion is correct, workfare is counterproductive.

3 months ago niall-l responded:

I should have said: "more likely to get a job than those on workfare *during the course of the placement*. The end point is the same.

3 months ago hettypeglerstump responded:

As the author of the chart... The main point is to show that some (or large numbers of) young people leave benefits all the time. This wasn't being made clear in the way it was being presented in the press and in Parliament. In particular, the fact that most work experience placements under the scheme in question last 8 weeks is clearly visible as increased numbers staying on benefits than we would expect. The post-programme element looks as though it returns to no benefit, but that is where both the employers in question - who were sold the idea as a good thing to do to support young people - and policy evaluation - needs to focus. I don't doubt that more work is being done to stand the case up, given how much people have invested in this programme.

3 months ago chrisrodger responded:

The graph is very misleading because it implicitly assumes that the start point of both cohorts (those on JSA and those on the work experience programme) are the same. In fact, according to the DWP (see this document http://www.dwp.gov.uk/docs/eia-work-experience.pdf) the work experience programme is aimed at those who have been on JSA for “3-12 months” (see para 35). So the graph should show the work experience line flat at 100% for 13 weeks before beginning to decline.
It is interesting that the rationale set out in the document is sensible and apparently well thought out, and also clearly acknowledges that most people on JSA leave it relatively quickly – “80 per cent leave within six months” (para 21). The whole (stated) point of the scheme is to target those who do not leave JSA rapidly. It may be that, in practice, that is not happening – but at the very least it would seem sensible to comment on the programme on the basis that it is designed to deal with the “difficult” JSA cases. It is disappointing to see that in fact most (all?) “fact check” and policy commentators have started from the opposite perspective, i.e. that there is a direct comparison that can be made to “normal” JSA churn rates, albeit with half-hearted caveats that in fact the cohort on the work experience may be different.
Finally, having now read the DWP document cited above one can’t help feeling that the programme was established with all the right intentions – as a serious attempt to really help people who otherwise would otherwise struggle to make their way in the world of work. To see it so successfully traduced, and the companies that participated accused of exploiting “slave labour”, is depressing.

3 months ago hettypeglerstump responded:

The graph is not very misleading. At the time these young people went on Work Experience - this time last year - young people at 6-10 months JSA were still going on the tail end of Labour programmes - old New Deal, Six Month Offer, and Future Jobs Fund. We checked that the JSA comparison shown would apply to a 3-month period before these other programmes kicked in - up to 6-9 months. Check the flow rates from 3-6 months and 6-9 months for young people claiming JSA if you want.
Otherwise, it follows the way DWP presented their figures. DWP could justify the stateement if, as Ben Goldacre suggested, they had tried a randomised controlled trial, or even a matched comparison trial (not quite so good). Even so, one cannot assume that people would stay on benefits, whatever their 'barriers' or local conditions.

3 months ago chrisrodger responded:

I am not sure I follow your response – are you saying that the start point for the JSA cohort is not the point when someone starts drawing JSA? And that the weeks shown on the x-axis of the graph are not weeks from that point of first signing on – which is how I, and I suspect many others, had interpreted it? Which begs the question, what is the start point for the JSA cohort – is at some time after day 1 of their claim, at a similar (same?) point in time as the day 1 (from which the weeks on the x-axis are measured) for the work experience cohort – which itself is some time after that cohort began drawing JSA? If so, what is that point in time?
I have not been able to find a definitive source for the JSA data behind the graph – and in fact a number of commentators point out how difficult it is to do (the need to track individuals through the system, etc). The graph gives the Labour Force Survey as a source, but I wasn’t able to access that, so I am none the wiser as to whether that contains the necessary data. Regardless of the source, it would be useful to know whether the JSA survival rate curve is based on a sample, and in fact there is uncertainty around it, or not. Also, if day 1 of the JSA curve is in fact some time after day 1 of the claim it seems odd that the rate of decline is still so high – more than 50% come off in three months, which seems consistent with the comments about the rate claimants flow off from the point they first claim.
So overall I am still confused, and it is still unclear to me that the JSA and work experience curves are directly comparable.
I do agree that if the DWP had set up a trial then most of the debate about the success or otherwise of the programme would have been much better informed.
Ben Goldacre's Space

I like stats. I'm a doctor. I write about dodgy scientific claims in my spare time.
This isn't my main blog. Find me here:
Blog:
www.badscience.net/
Book:
http://www.amazon.co.uk/Bad-Science-Ben-Goldacre/dp/000728487X/?tag=bs0b-21
Tweet:
www.twitter.com/bengoldacre
TED talk:
http://www.ted.com/talks/ben_goldacre_battling_bad_science.html

bengoldacre - secondary blog

People come off JSA at the same rate, regardless of workfare

Ben Goldacre's Space

About Ben Goldacre

Archive

Follow Me