I have written this post principally for people who have started following me (formally on Twitter, or in some other way) because of my somewhat peripheral involvement on the PACE trial discussions.
First off, while I try to be reasonably politically correct, I don't always get all the details right. I've tried to be respectful to all involved here. In particular, someone told me that "CFS/ME" is not always an appropriate label to use. I hope anyone who thinks that will allow me a pass on that, from my position of ignorance.
I've learned a lot about CFS/ME over the past few weeks. Some of what I've been told --- but above all, what I've observed --- about how some of the science has been conducted, has disturbed me. The people whose opinions I tend to trust on most issues, who usually put science ahead of their personal political position, seem to be pretty much unanimous that the PACE trial data need to be released so that disinterested parties can examine them.
But I want to make it clear that I have no specific interest in CFS/ME. I don't personally know anyone who suffers from it, and it's not something I've really ever thought about much. I don't especially want to become an advocate for patients, except to the extent that, having had my own health problems in the last couple of years, I wish every sick person a speedy recovery and access to the finest medical treatment they can get. So I'm not sure I can even call myself an "ally"; allies have to take a non-trivial position, and I don't think my position here is much more than trivial. If the PACE trial data emerge tomorrow, I will not personally be reanalysing them. I don't know enough about this kind of study to do so.
What I do care about is the integrity of science. You can see this, I hope, if you Google some of the stuff I've been doing in psychology. Science, imperfect though it is, is about the only rational game in town when it comes to solving the problems facing society, and when scientists put their own interests above those of the wider community, it usually doesn't turn out well.
So, on to the PACE trial... I want to say that I can understand a lot of defensiveness on the part of the PACE researchers. They have heard stories of others being harassed and even receiving death threats. Maybe some of them have experienced this themselves. For the purposes of this post (please bear with me!), I'm going to assume --- because I have no evidence to the contrary, and people generally don't make these accusations lightly --- that the stories of CFS/ME researchers being harassed in the past are true; arguably, for the purposes of this discussion, it doesn't make any difference whether they are true or not. (Of course, in another context, such claims are very important, but let me try to put that aside for now.) Apart from anything else, given the size of the CFS/ME community, it would be unreasonable not to expect there to be some fairly unpleasant people to have also developed the condition. We all know people like that, whatever our and their health status. CFS/ME strikes people from all walks of life, including some saints and some sinners.
Now, with that said, I am unconvinced --- actually, "bewildered" would be a better word --- by the argument that releasing the data would somehow expose the researchers to (further) harassment. Indeed, it seems to me that withholding the data plays directly into the hands of those who claim that the PACE researchers have "something to hide", and they are presumably the most likely to escalate their anger into harassment. I actually don't believe that the researchers have anything to hide, in the sense of feeling guilty because they did something bad in their analyses. I've seen enough cases like this in my working life to know that incompetence --- generally in the form of a misplaced sense of loyalty to a group rather than to the wider truth and public interest --- is always to be preferred as an alternative explanation to malice, first because malice is harder to prove, and second because it just almost always turns out to be the case than incompetence was behind a screw-up.
About the only reason I can sort of imagine for the argument that releasing the data might lead to harassment of the researchers, is if the alternative were for the question to somehow go away. That's perhaps a reasonable argument with some political issues; for example, there is (I think) a legitimate debate to be had over whether it's helpful to reproduce, say, cartoons that might cause people to get over-excited, when they could just be left to one side. But that's simply not going to happen here. People with a chronic, debilitating condition, and no cure in sight, are not going to suddenly forget tomorrow that they have that condition. So far, none of the replies to people who have asked for the data, and been told it will lead to harassment, have explained the mechanism by which that is supposed to happen.
The researchers' argument also seems to conflate the presence in the CFS/ME activist community of some unpleasant people --- which, again, for the sake of this discussion, I will assume is probably true --- to the idea that "anyone from the CFS/ME activist community who asks about PACE is probably trying to harass us". This is not good logic. It's what leads airline passengers to demand that Muslim passengers be thrown off their plane. It's called the base rate fallacy, and avoiding it is supposed to be what scientists --- particularly, for goodness sake, scientists involved in epidemiology --- are good at.
A further problem with the arguments that a request for the data --- whether it comes from patients with scientific training, or scientists such as Jim Coyne --- is designed to be "vexatious" or to "lack serious purpose" or that its intent is "polemical" (all terms used by King's in their reply to Coyne), is that such arguments are utterly unfalsifiable. Given the public profile of this matter, essentially anyone who asks for the data is going to have their credentials examined, and unless they meet the unspecified high standards of the researchers, they won't get to see the data. (Yes, Jim Coyne --- who, full disclosure, is my PhD supervisor --- can be a bit shouty at times. But this is not kindergarten. Scientists don't get to withhold data from other scientists just because they don't play nice. Ask any scientist if science is about robust disagreement and you will get a "Yes", but if that idealism isn't maintained when actual robust disagreement takes place, then we might as well conduct the whole process through everything-is-fine press releases.)
Actually, in their reply to Coyne, King's College did seem to give a hint as to who might be allowed to see the data, in their statement "We would expect any replication of data to be carried out by a trained Health Economist", with an nice piece of innuendo carried over from the preceding sentence that this health economist had better have a lot of free time, because the original analysis took a year to complete. This suggests that unless you declare your qualifications as an unemployed health economist, you aren't going to be judged worthy to see the data (and if you come up with conclusions after a week, it might well be suggested that you didn't look hard enough). But the idea that it will take a year, or indeed need specialised training in health economics, to determine whether the Fisher's exact tests from the contingency tables were calculated correctly, or whether the results really show that people got better over the course of the study, is absurd. Apart from anything else, science is about communicating your results in a coherent manner to the rest of the scientific community. If you submit an article and then claim that its principal conclusions cannot be verified except by a few dozen highly trained specialists with a year's effort, that's an admission right there that your article has failed. Of course there will be questions of interpretation, over things like what "getting better" means, but nobody should have to accept the researcher's claims that their interpretation is the right one. There needs to be a debate, so that a consensus, if one is possible, can emerge. (Who knows? Maybe the evidence for CBT is overwhelming. There are plenty of neutral scientists who can reach a fair conclusion about that, but right now, they are being deprived of the opportunity to do so.)
A further point about the failure to share data is that the researchers agreed, when they published in PLoS ONE, to make their data available to anyone who asked for it. This is a condition of publishing in that journal. You can't have the cake of "we're transparent, we published in an open access journal" and then eat that cake too with "but you can't see the data". PLoS ONE must insist that the authors release the data as they agreed to do as a condition of publication, or else retract the article because their conditions of publication have been breached. See Klaas van Dijk's formal request in this regard.
These data are undoubtedly going to come out at some point anyway. The UK's Information Commissioner will see to that, even if PLoS ONE doesn't persuade the authors to release the data. As the risk management specialist Peter Sandman points out, openness and transparency at the earliest possible stage translate into reduced pain and costs further down the line.
I want to end with a small apology. I wrote a post yesterday on an unrelated topic (OK, it was also critical of some poor science, but the relation with the subject of this post was peripheral). Two people submitted comments on that post which drew a link with the PACE trial. After some thought, I decided not to publish those comments, as I wanted to keep discussion on that other post on-topic. I apologise to the authors of those comments that Blogger.com's moderation system did not let me explain the reasons why they were not published. I would happily publish those same comments on this post; indeed, I will publish pretty much any reasonable comments on this post.
First off, while I try to be reasonably politically correct, I don't always get all the details right. I've tried to be respectful to all involved here. In particular, someone told me that "CFS/ME" is not always an appropriate label to use. I hope anyone who thinks that will allow me a pass on that, from my position of ignorance.
I've learned a lot about CFS/ME over the past few weeks. Some of what I've been told --- but above all, what I've observed --- about how some of the science has been conducted, has disturbed me. The people whose opinions I tend to trust on most issues, who usually put science ahead of their personal political position, seem to be pretty much unanimous that the PACE trial data need to be released so that disinterested parties can examine them.
But I want to make it clear that I have no specific interest in CFS/ME. I don't personally know anyone who suffers from it, and it's not something I've really ever thought about much. I don't especially want to become an advocate for patients, except to the extent that, having had my own health problems in the last couple of years, I wish every sick person a speedy recovery and access to the finest medical treatment they can get. So I'm not sure I can even call myself an "ally"; allies have to take a non-trivial position, and I don't think my position here is much more than trivial. If the PACE trial data emerge tomorrow, I will not personally be reanalysing them. I don't know enough about this kind of study to do so.
What I do care about is the integrity of science. You can see this, I hope, if you Google some of the stuff I've been doing in psychology. Science, imperfect though it is, is about the only rational game in town when it comes to solving the problems facing society, and when scientists put their own interests above those of the wider community, it usually doesn't turn out well.
So, on to the PACE trial... I want to say that I can understand a lot of defensiveness on the part of the PACE researchers. They have heard stories of others being harassed and even receiving death threats. Maybe some of them have experienced this themselves. For the purposes of this post (please bear with me!), I'm going to assume --- because I have no evidence to the contrary, and people generally don't make these accusations lightly --- that the stories of CFS/ME researchers being harassed in the past are true; arguably, for the purposes of this discussion, it doesn't make any difference whether they are true or not. (Of course, in another context, such claims are very important, but let me try to put that aside for now.) Apart from anything else, given the size of the CFS/ME community, it would be unreasonable not to expect there to be some fairly unpleasant people to have also developed the condition. We all know people like that, whatever our and their health status. CFS/ME strikes people from all walks of life, including some saints and some sinners.
Now, with that said, I am unconvinced --- actually, "bewildered" would be a better word --- by the argument that releasing the data would somehow expose the researchers to (further) harassment. Indeed, it seems to me that withholding the data plays directly into the hands of those who claim that the PACE researchers have "something to hide", and they are presumably the most likely to escalate their anger into harassment. I actually don't believe that the researchers have anything to hide, in the sense of feeling guilty because they did something bad in their analyses. I've seen enough cases like this in my working life to know that incompetence --- generally in the form of a misplaced sense of loyalty to a group rather than to the wider truth and public interest --- is always to be preferred as an alternative explanation to malice, first because malice is harder to prove, and second because it just almost always turns out to be the case than incompetence was behind a screw-up.
About the only reason I can sort of imagine for the argument that releasing the data might lead to harassment of the researchers, is if the alternative were for the question to somehow go away. That's perhaps a reasonable argument with some political issues; for example, there is (I think) a legitimate debate to be had over whether it's helpful to reproduce, say, cartoons that might cause people to get over-excited, when they could just be left to one side. But that's simply not going to happen here. People with a chronic, debilitating condition, and no cure in sight, are not going to suddenly forget tomorrow that they have that condition. So far, none of the replies to people who have asked for the data, and been told it will lead to harassment, have explained the mechanism by which that is supposed to happen.
The researchers' argument also seems to conflate the presence in the CFS/ME activist community of some unpleasant people --- which, again, for the sake of this discussion, I will assume is probably true --- to the idea that "anyone from the CFS/ME activist community who asks about PACE is probably trying to harass us". This is not good logic. It's what leads airline passengers to demand that Muslim passengers be thrown off their plane. It's called the base rate fallacy, and avoiding it is supposed to be what scientists --- particularly, for goodness sake, scientists involved in epidemiology --- are good at.
A further problem with the arguments that a request for the data --- whether it comes from patients with scientific training, or scientists such as Jim Coyne --- is designed to be "vexatious" or to "lack serious purpose" or that its intent is "polemical" (all terms used by King's in their reply to Coyne), is that such arguments are utterly unfalsifiable. Given the public profile of this matter, essentially anyone who asks for the data is going to have their credentials examined, and unless they meet the unspecified high standards of the researchers, they won't get to see the data. (Yes, Jim Coyne --- who, full disclosure, is my PhD supervisor --- can be a bit shouty at times. But this is not kindergarten. Scientists don't get to withhold data from other scientists just because they don't play nice. Ask any scientist if science is about robust disagreement and you will get a "Yes", but if that idealism isn't maintained when actual robust disagreement takes place, then we might as well conduct the whole process through everything-is-fine press releases.)
Actually, in their reply to Coyne, King's College did seem to give a hint as to who might be allowed to see the data, in their statement "We would expect any replication of data to be carried out by a trained Health Economist", with an nice piece of innuendo carried over from the preceding sentence that this health economist had better have a lot of free time, because the original analysis took a year to complete. This suggests that unless you declare your qualifications as an unemployed health economist, you aren't going to be judged worthy to see the data (and if you come up with conclusions after a week, it might well be suggested that you didn't look hard enough). But the idea that it will take a year, or indeed need specialised training in health economics, to determine whether the Fisher's exact tests from the contingency tables were calculated correctly, or whether the results really show that people got better over the course of the study, is absurd. Apart from anything else, science is about communicating your results in a coherent manner to the rest of the scientific community. If you submit an article and then claim that its principal conclusions cannot be verified except by a few dozen highly trained specialists with a year's effort, that's an admission right there that your article has failed. Of course there will be questions of interpretation, over things like what "getting better" means, but nobody should have to accept the researcher's claims that their interpretation is the right one. There needs to be a debate, so that a consensus, if one is possible, can emerge. (Who knows? Maybe the evidence for CBT is overwhelming. There are plenty of neutral scientists who can reach a fair conclusion about that, but right now, they are being deprived of the opportunity to do so.)
A further point about the failure to share data is that the researchers agreed, when they published in PLoS ONE, to make their data available to anyone who asked for it. This is a condition of publishing in that journal. You can't have the cake of "we're transparent, we published in an open access journal" and then eat that cake too with "but you can't see the data". PLoS ONE must insist that the authors release the data as they agreed to do as a condition of publication, or else retract the article because their conditions of publication have been breached. See Klaas van Dijk's formal request in this regard.
These data are undoubtedly going to come out at some point anyway. The UK's Information Commissioner will see to that, even if PLoS ONE doesn't persuade the authors to release the data. As the risk management specialist Peter Sandman points out, openness and transparency at the earliest possible stage translate into reduced pain and costs further down the line.
I want to end with a small apology. I wrote a post yesterday on an unrelated topic (OK, it was also critical of some poor science, but the relation with the subject of this post was peripheral). Two people submitted comments on that post which drew a link with the PACE trial. After some thought, I decided not to publish those comments, as I wanted to keep discussion on that other post on-topic. I apologise to the authors of those comments that Blogger.com's moderation system did not let me explain the reasons why they were not published. I would happily publish those same comments on this post; indeed, I will publish pretty much any reasonable comments on this post.
