Thes Higher Educational Supplement
The Times Higher Education Supplement

Scientists fail to see eye to eye over girl's 'X-ray vision'

Phil Baty
Published: 10 December 2004
http://www.thes.co.uk/story.aspx?story_id=201825
see also published comment, concluding remarks

A Nobel prizewinning scientist has clashed with one of Britain's leading experts on the paranormal in a row over the purported talents of a Russian schoolgirl who claims she uses X-ray vision to diagnose medical problems.

Brian Josephson, a Cambridge University professor who won the Nobel Prize for Physics in 1973, has given his backing to claims by Natasha Demkina that she can see inside people's bodies.

The professor, who has been scorned by colleagues for his enthusiasm for the paranormal, has claimed that an experiment for a forthcoming terrestrial TV documentary that apparently disproves Ms Demkina's claims was "a fix" designed to ensure she failed.

Richard Wiseman, professor of psychology at Hertfordshire University and a key member of the respected Committee for the Scientific Investigation of Claims of the Paranormal, who helped design the experiment, hit back this week.

He said that the attack from Professor Josephson, who is a physicist with no known record of publishing on parapsychology [but see below], "does not carry much weight", as it was posted on his personal website without any refereeing process.

But he admitted that the trial could have been improved.

The experiment to test Ms Demkina's claims was filmed for a Discovery Channel documentary, The Girl with the X-ray Eyes, which is also due to appear on Channel 4 next year.

In the programme, Ms Demkina correctly identifies the medical conditions of four out of seven patients, and misdiagnoses three.

Professor Josephson says, on his Cambridge University-hosted website, that "many viewers ended up with a strong impression that the test... had been deliberately set up with a view to ensuring that she would fail it".

He says that, in difficult circumstances, Ms Demkina overcame odds of more than 50 to one to correctly diagnose four patients.

"Surely a case for celebrating Natasha's success?" he says.

But instead, it was declared that Ms Demkina had failed the test as the experimenters had agreed with her ahead of the test that anything fewer than five matches did not "support any belief in her claimed abilities".

Professor Josephson said: "A statistically very significant result was obtained... but the experimenters concealed the fact with their talk of her failing the test."

Professor Wiseman said he agreed that the number of matches Ms Demkina had to achieve to be deemed a success was set higher than the standard probability of one in 20 normally used in psychology tests.

"We were asking her to jump high, but that was because her claim would present a huge challenge to science if it were true.

"I don't see how you could argue there's something wrong with having to get five out of seven when she agrees with the target in advance."

He added: "I'm not saying that this experiment was perfect or that all Professor Josephson's comments are wrong - like any first-time study conducted under the pressure of time and with limited resources it could be improved - but overall I think the results give us an additional insight into Natasha's claimed abilities."
The following three letters in response to the above article appeared in the December 17th. issue of the Times Higher Education Supplement, in some cases in an edited form:
Your account of the controversy about the girl with "X-ray vision" (December 10) was so obviously biased that I wondered why your reporter felt the need to display his prejudices.
He said Brian Josephson is "scorned by colleagues for his enthusiasm for the paranormal", whereas he called Richard Wiseman a key member of the "respected" Committee for the Scientific Investigation of Claims of the Paranormal. He could have said with at least as much accuracy that CSICOP is an ideologically motivated debunking organisation and that Josephson is much respected for his courage and independence.
It would have been more respectful of readers' diverse views simply to report the controversy without taking sides.
Given his propensity for talking to the press about his own unrefereed work, it is ironic to find Richard Wiseman dismissing my web commentary on his research on account of it being unrefereed (Scientists fail to see eye to eye over girl's 'X-ray vision', December 10th.).  It is precisely because of his tendency to ask subjects to 'jump high' and then to announce that they have failed, as well as his carelessness in experimental design (in this case, some of the medical conditions of the subjects were ones that Natasha Demkina had in the past indicated were ones she had difficulties with), that parapsychologists are doubtful about Wiseman's approach.
Simply to have stated that the investigation had not definitively confirmed Natasha's claims would have been unproblematic, but the Discovery programme showed the investigators also drawing the further, scientifically unwarranted conclusion that they had actually refuted her claims.
I have reviewed Professor Josephson’s arguments, published on his web page, and find them to be scientifically and statistically correct.  In contrast, the statement of Professor Wiseman, of CSICOP, “I don’t see how you could argue there’s anything wrong with having to get five out of seven when she agrees with the target in advance”, demonstrates a complete lack of understanding of how experimental data should be interpreted statistically, as pointed out by Professor Josephson in his web site. 
The experiment is woefully inadequate in many ways.
The chance of the observed 4 successes 7 subjects by pure guessing, is 1 in 78, an indication of a significantly non-random result, as claimed by Professor Josephson.  But, suppose Natasha had a diagnosis rate of 1 in 2, compared with the chance rate of 1 in 7: then there is equal chance of getting 4 or more from 7, or 3 or less from 7.  That is, the probability (power) of detecting a true 50% diagnosis rate on 7 subjects using a 0.01 significance level is only 50% (calculations use a Binomial approximation).  There should have been at least 21 subjects to ensure a 90% probability/power of detecting a true diagnosis rate of 50% (using a 0.01 significance level test).  Only if Natasha had a true diagnosis rate as high as 72% would there have been a 90% chance of detecting the effect using a 0.01 test (i.e. 4 more as the criterion) on 7 subjects. The experiment, as designed, had high chances of failing to detect important effects.  This may have been due to the lack of involvement of a statistician in the design of the CSICOP experiment, rather than its intentions.
Finally, the lack of blinding of Natasha to the sex, age, and state of the subjects means that no inferences are possible about the cause of any increased diagnosis rate that might have been detected.
§  And now some concluding comments.  The fact that Wiseman's main reaction to the critique of his work on my web pages was to suggest that, since it has not been refereed, it "does not carry much weight", may suggest to some (a) that there is little he can find to say in defence of his position, and (b) the name of the game is indeed propaganda (action directed primarily at putting the other side in a bad light), as opposed to proper science. Admittedly, these web pages here are also propaganda, but I hope that the science and the logic are sounder.

In an email, Wiseman explained that he thought it important that the public should understand about peer-review.  I equally think (as noted) that it is important that people understand CSICOP better.  Seeing how Wiseman chooses to respond to my critique may help them do this.

Notice the comment "I don't see how you could argue there's something wrong with having to get five out of seven when she agrees with the target in advance."  I have already indicated on the web page what is wrong with this, thus: 'real science does not work on a basis of getting someone to sign their agreement to a long list of conditions, then later coming back saying "this is what you signed; the challenge goes to us!" '.  How pleasant just to be able to ignore such arguments, on the basis of an assertion to the effect that the web page they were brought up on had not been subjected to peer-review!

By the way, with regard to factual matters, the number of colleagues who "scorn" me for my "enthusiasm for the paranormal" can be counted on the fingers of one hand.  And I have published papers on parapsychology, some of which can, with a little effort, be found on, or via, my home page.
home page