So long, and thanks for all the fishiness

Here's another excellent post by Greg Taylor of The Daily Grail, this time pointing out that James Randi's fabled Million Dollar Challenge will officially end in two years. 

For reasons I explored in a series of posts starting here, it will not be missed. Well, not by me, anyway.

Comments

[The Major]

Well, Michael, I'm glad you brought that up. Someone reporting on that case commented that the twins were "inexplicably drawn to each other". I found this comment interesting, not least because they were trying to explain this in materialist terms.

Why would the twins have a material reason for being drawn to the other? They were separated from birth so their life experiences would be very different. Different parents, friends, education etc. Therefore, their only attraction to each other (in the sense of them being 'inexplicably drawn' to each other) would be if they shared a mental bond, one that perhaps hints at ESP or a dualist interpretation of the brain.

After all, this isn't the first time twins have talked about a different sort of bond between them.

That was a thought I had.

[marcos ferreira]

Then we compare that data with the probability of a church blowing up on any given night. Then we use logic to compare the probability of that church blowing up on the very same night as all the members of the church choir including the choir leader on the same night.

Right, well at least we've found the source of the initial calculation error:

The two alternatives we're considering are (1) The church blows up, and people die as a result versus (2) The church blows up, and no-one is harmed because they were all late.

In both cases, the fact that an explosion occurred is taken for granted so its probability does not come into the calculations.

Just to drive the point home: I just flipped a coin and got heads. What were the odds of that? 1 in 2.

If I started trying to involve the probability that I'd be in this room flipping a coin in the first place, and not elsewhere doing something completely different, we'd end up with an astronomically small probability (if we're being consistent, it would have to take into account the probability that I was conceived and that humanity invented coins!).

And that would be equivalent to involving the probability of an explosion here. We're answering the question, "What's the probability everyone was late for choir practice and thus saved when the church blew up?" The sample space in question splits into "The church blew up, everyone was late" and "The church blew up, at least one person died because they were on time." The probability is a measure of how large that first area is compared to the second; by bringing in the probability of the explosion, you've added a massive area to that sample space labelled "Explosion doesn't happen", and that is what's distorting downwards the relative size of the area we're investigating.

To confirm what I'm saying, try calculating the probability that people would get killed by the explosion using your method. Because of this error, you'll find a similarly astronomically small probability, because "church explodes, everyone late" and "church explodes, someone dies" are both dwarfed by the hugely probable "no explosion occurs that night".

In other words, if your numbers were correct, we'd be just as astounded if most of the choir were wiped out by the explosion!

By the way for you that are not into statistical analysis the control chart is the most robust of the statistical tools used to measure and identify the difference between a common and special cause of variation.

It's a heuristic tool specifically developed to be useful in business and manufacturing processes, and based on the Bayesian principles I'm applying directly above (essentially, you produce a statistical model for common-cause and then ask how unlikely it would be for a given data set to appear under that common-cause model). If there's a flaw in my reasoning, it should be possible to point it out in those terms, instead of an argument from authority: "I used a very reliable tool, so your reasoning must be flawed..."

It's better to use a direct argument in terms that anyone can follow, rather than conferring extra validity to a set of specialised terms which are essentially the direct ones in disguise. That would be like Victor Zammit insisting that only someone experienced in Australian criminal law procedures truly knows how to evaluate evidence, or James Randi saying that because we're not magicians, we're not qualified to assess the likelihood of fraud in psi experiments.

Tim:

The 1 in 250,000 figure comes from multiplying 0.25 with itself 9 times. When I wrote pow(0.25,9), it was just shorthand for 0.25 * 0.25 * ... 9 times. It gives the probability of 9 independent events happening simultaneously if each event has probability 0.25, the same way we'd find that the probability of three simultaneous heads in a coinflip experiment would be 0.5 * 0.5 * 0.5 = 0.125 or 1 in 8. I know 0.25 is just an estimated probability, but it will get us in the right ballpark in terms of "how many zeros on the end".

[marcos ferreira]

Speaking of one-in-a-million shots, whow about this?

In the same week as a http://news.independent.co.uk/uk/politics/article3326209.ece>herald of the coming apocalypse?

[Tim]

Boy, am I out of my league when it comes to statistics. I'll have to rely on you gentlemen with knowledge of it to set me straight. William, what I was referring to in my comment was not just the single event, but the fact that we don't have a database of how many other churches have undergone catastrophic events such as fires, earthquake collapses, etc. at the time when a group event was about to take place. So we have no basis for comparison to see how a) how often such an even occurs and b) if there are recognizable patterns in people's behavior, i.e., surprising absenteeism from a church function prior to an unforeseen disaster. That is why this event cannot be made statistically significant beyond the odds of it as a stand-alone occurrence.

I suppose one could assemble such a database by spending 10 years combing through news archives (a task that, as a fledging journalist, I used to do, much to my soul's degradation), but an online reporting tool would be easier. I wonder if anyone has created such a thing?

As for the twins story, Michael, I think a "D'oh!" is in order.

[william]

There is more here than meets the eye. We are I suspect in soul clusters, reincarnate in groups, and continue to cross each other's paths during this life. I recently saw a movie where a brother fell in love with his half sister before they found out they shared the same father. The mother (mistress) and father tried to break them up but to no avail. Can’t remember exactly the ending but I think they decided to stay together. Passion rules.

I just discovered my auto mechanic whom I have used for ten years and always felt a strong bond with as we had long talks on such things as theoretical quantum physics but we had never discussed where we were born and raised was my paper boy in 1972 lives 1500 miles away from where we both reside now and in the same city.

What are the odds of that considering this country has 300 million people in it. Recently at my mother's funeral when the granddaughter had a get together after the funeral and the family began to realize we were all connected in some way before they married into the family. It goes on and on.

We seem to meet up with some of the same people even 1500 miles away. Just saw a high school friend after 52 years and we discovered we lived ¼ mile apart 1500 miles from where we went to high school for 20 years and shopped at the same stores near our homes.

Newton’s books on life between lives states that his clients under hypnosis continually report back to him we are in intimate soul clusters of 6 to 20 people and in larger soul acquaintances of up to several hundred people.

[william]

I am sure very few people on this blog want us to discuss statistics and logic on Michael’s blog. Very few people understood what Deming was attempting to teach even the famous statistical guru Joseph Juran failed to see Deming’s statistical logic. Deming went to his grave at 93 with little success in America but with some success in Japan, who had just been defeated in a war and they were somewhat open to learn unlike Americans who won the war.

It took a revelation for me to see it. The only revelation of that magnitude in my life and literally changed the way I viewed life. No wonder I find it impossible to share my discovery about the origin of ignorance.

“I know 0.25 is just an estimated probability, but it will get us in the right ballpark in terms of "how many zeros on the end".

Lets see “ballpark” and “estimated probability” and you are challenging a process behavior chart that has proven itself for the past 60 years as a reliable source to separate common and special causes of variation.

Readers you judge for yourselves. Lets look at the data. Even if we use Marcos numbers 1 in 250000 of everyone being late and that’s the night the church blew up. Chance? Could be. The attendance behavior chart would show that there is a 99.8 per cent probability that this was a special cause of variation.

Could the two happen together due to chance and there was not a causal correlation? Absolutely lots of choir practices going on and have gone on in the world in churches and that is Marcos’ point. If Marcos and others want to believe it was due to chance so be it. I suspect life is designed that way (choices within boundaries) to advance faith, hope, compassion, and learning for our souls.

From my point of view for that many people to be late the same night of a church blowing up exactly 5 minutes before anyone arrived I suspect it was more than personal intuition or chance. But one or more spiritual guides doing what they do best even knowing the future like a church about to blow up from a gas leak. Why where those lives saved? I have no idea. Apparently it was not those choir folks time to cross over so guides stepped in.

Marcos if you want to continue this dialog maybe Michael could hook us up by exchanging our email addresses as I suspect he knows them.

Anyhow loved this dialog as it brought back so many memories from my consulting days but I fear we have bored everyone else on here. Little wonder people do not trust statistics, WHO can blame them.

[william]

This discussion that Marcos and I have been having reminds me of the story I saw on TV were a UFO flew beside a commercial airliner and the two pilots and the navigation person saw the UFO around them for several minutes. The UFO even took control of the plane and caused it to bank 15 degrees and the pilot could not control the plane and then the UFO put it right back on course.

Then the TV producers brought on an agnostic ultra skeptic and using his idea of statistical logic stated this proved nothing, as they would have to control the experiment and have at least 30 such experiments (UFO’s control a plane) to validate that UFO's exist. The insanity of that comment is evident to most humans but not all.

[Vicky]

I don't mean to take sides but William I don't quite understand what you mean when you talk about Deming
And I don't think Marcos is saying that he thinks that situations like that are due to chance. He did say that he was agnostic on the cause in his previous comment. And he's not challenging the process behavior chart either but saying that just because it may be right in the way that it is used, doesn't mean another reasoning is flawed on its own.
However. I don't think there is a specific attendance chart (which would be helpful if there were!) recorded for these people and from what I read in the stories of the personal accounts, I was a bit surprised that quite a few of these members would be late for a practice for something like "listening to a song". So I have to wonder at the statement that "they were rarely late". However I do find the whole incident remarkable.
From what I understand, Marcos is just saying that we can't use the statistics to explain any singular event because the statistics for any event can be considered a "one in a million" chance. I mean hasn't there been an incident where someone who has a regular habit of bringing an umbrella every day when it doesn't rain and then forgetting it when it actually does rain? He wouldn't immediately think something strange was going on (although there might be). However yes, this is only the account of one person so yes, the church incident is even more bizarre. I think the point is, is that using the statistic as an argument itself is not convincing.
If however, this same sort of incident occurs more than once at the same church (or even maybe at some others), that would be pretty convincing that something was going on (at least to me). It would be like the plane study that Michael talked about.

About the movie though. What a strange similarity it has to a book called Speaker for the Dead because the samething happens in the plot except they don't stay together because the girl refuses to and the boy is paralyzed.

Anyway...we certainly have steered off the original entry topic although the debate has been very interesting.

Marcos, about the spreading the challenge across a couple of days to reduce the chances of an off day. From what I understood of the rules, it had to be done in one day so I don't know about that.
And just trying to think for the other side although I still remain a bit suspicious. Is it possible for superstar psychics to also be intimidated? I can't imagine John Edward receiving a very warm welcome no matter how much they try to control it. And they could say that he was lying if he tried to bring it to the public that they were hostile kind of like what happened to Natalie, the Russian girl.

[william]

“I think the point is, is that using the statistic as an argument itself is not convincing.”

Vicky that was my original point not much if anything can be proved by using statistical analysis in the field of the paranormal. Swartz at the university of Arizona is using statistical analysis with mediums and how well has that worked to convince anyone of the validity of paranormal events. Radin is using statistical analysis to show the validity of psi and those that have a belief in psi to begin with are the only ones accepting his findings. If we have a materialistic paradigm no statistical data will change that paradigm contrary to what the materialists will tell you.

Concerning the attendance at the choir practice. The story I heard was that the choir people were seldom late as the choir director was very strict. No numbers will satisfy the skeptic, this is about existing beliefs not statistics.

Deming is a whole other story. The big three US automakers failed to understand or implement Deming’ teachings at the leadership level. Toyota leaders understand his teachings the very best. Check the latest edition of consumer reports to see how an understanding and implementation of Deming’s teachings coupled with the Toyota production system worked for Toyota compared to the big three reliability scores over ten years of use. Those best in the world reliability scores are making huge profits and sales for Toyota.

Time magazine several years back listed Deming as being one of the top ten people to have an impact on the world in the last 2000 years; right up there with you know who. Very few in the world understood why he should be listed in the top ten; the exception being Toyota. Ask Marcos if he thinks Deming should be listed in the top ten. Bet his answer will be no.

Deming’s ideas if accepted in the United States educational system would revolutionize our educational system towards excellence instead we go with no child left behind using testing strategies and these strategies go against everything Deming taught. (Although great profits for the corporations that make tests).

How well has no child left behind worked? Absolute failure. Educational excellence is much about understanding variation. And here is the kicker: that understanding of variation has everything to do with what we discuss on this blog about the paranormal, the meaning of life, purpose of life, the suffering in our lives, and why we even exist as perceived separate entities. Go figure.

Most if not all people that read these statements I just made I suspect will arrive at the conclusion that I have gone off on a tangent and my words have nothing to do with the paranormal.

As one blogger commented several months back when I made similar comments: yawn. Oh if that blogger only knew the amount of unawareness embedded within that yawn. Been there myself. I guess what I am trying to say and doing a very ineffective job stating it is that understanding variation has much to do with spirituality.

[william]

michael p: an interesting article that may be of interest to you and your readers of this blog that appeared in:

Network Review: The Journal of the Scientific and Medical Network".

Richard Dawkins meets Rupert Sheldrake for a tv show.

http://ming.tv/flemming2.php/__show_article/_a000010-001911.htm

[Michael H]

William, did you happen to notice the first comment following the Dawkins - Sheldrake article you've linked? To quote:

"There has been absolutely NO evidence that the paranormal is anything but bunk.

Why has no paranormalist won the Randi Prize of 1 Megabuck? Because they can't do what they claim."

I don't think a PhD in statistical analysis would change this guy's mind. He 'knows', and will explain away everything that runs counter to his belief. Before we crucify him though, let's make sure we aren't guilty of the same thing.

[marcos ferreira]

Readers you judge for yourselves. Lets look at the data. Even if we use Marcos numbers 1 in 250000 of everyone being late and that’s the night the church blew up. Chance? Could be. The attendance behavior chart would show that there is a 99.8 per cent probability that this was a special cause of variation.

And as I'm sure you know, crossing the limits on such a control chart happens approximately 1 in 370 by chance (because 99.8% special cause is just the converse of 0.2% common cause).

So even if we could use such a control chart in this situation (we can't, because we don't have enough data, just the single estimate provided by the choir members)...

...and even if it were appropriate to use it here (it's not, because if we had enough data we'd be able to use statistical techniques more suited to the problem domain; measurements taken from complex human behaviour typically fall into a Gaussian distribution, whereas the control chart was developed specifically for use in industrial quality control because machine processes produce chaotic measurements in ways that do not afford us such statistical niceties)...

(pause for breath)

...then the most we could say from using such a chart is that the odds of the choir missing the explosion "by chance" (= common cause) are "1 in 370 or lower" – in other words, my estimate of 1 in 250,000 is more sympathetic to your case than anything you'd be able to conclude from such a control chart.

Michael H:
>"There has been absolutely NO evidence that the paranormal is anything but bunk.

I was tempted to leave a comment there pointing out Sheldrake's experiments with telephone telepathy, because that guy seems to think he's making the claims from anecdotal evidence only.

But what are the odds that guy's coming back to a comment he posted on Friday? (Please, no answers to that involving numbers!)

[william]

To Dr Deming
Well I gave it my best shot but again we have a person that thinks they know what you tried to teach to America for over 50 years and has failed to see or understand variation.

To suggest that a control chart is only for industrial applications is the ultimate in lack of understanding of variation or the control chart.

I will stick with my original statement one can understand statistics and still not understand variation. And we Americans wonder why the big three auto makers and our educational system is failing us.

Deming went to his grave (93) trying to get Americans to understand variation as it applies to all phenomena with little success. His predications and warnings about the future of America have been right on target if Americans did not take time to learn his teachings which much came from Walter’s Shewharts discovery of the control chart.

I worked with some the best six sigma statistical gurus in the business and most of them never had a clue what Deming taught but most were very good at statistics many with advanced degrees in statistics.

“in other words, my estimate of 1 in 250,000 is more sympathetic to your case than anything you'd be able to conclude from such a control chart.”

This statement reveals your total lack of understanding as to what Walter Shewhart called a control chart which that name alone set it back at least one hundred years. The “process or phenomena behavior chart” takes into consideration the systemic performance of phenomena and can be used for predictability of future phenomena. Few people in the world understand that.

“measurements taken from complex human behavior typically fall into a Gaussian distribution”

Nothing could be farther from the truth we tend to force distributions of phenomena into a Gaussian distribution so we can conduct statistical analysis that are based on the Gaussian distribution. The control chart is the most robust at taking these distributions that do not fit into a Gaussian distribution (which most don’t) and make much more accurate predictions.

There is a place for statistics that use the Gaussian distributions such as in design of experiments but that is not my point.

My point is that no amount of statistics will change ones mind when it comes to the paranormal. It is about beliefs and paradigms not statistics or even evidence, no matter how profound the decimal.

[marcos ferreira]

I'll narrow it right down in the hope of a direct answer.

Is it true or false that a measurement falls outside the limits on a control chart with a probability of approximately 1 in 370, assuming that the limits are placed 3 standard deviations from the mean?

[william]

No direct answer. And here is the person that said he would accept statistical significance of one in one hundred or something to that affect and is making a big deal that one in 370 is not significant. hummm

Ever think how far the statistical variation limits would be for an entire choir to be late for church on an attendance behavior chart?

See the difference is that you do not see a causal correlation between a church blowing up at 7:30 the exact time the choir was supposed to start practice and lo and behold everyone was late for practice and we do not have a common cause explanation why everyone was late like poor road conditions. I do for the reasons listed below.

My paranormal research has suggested to me that these types of paranormal events are common and as the Internet grows we will learn about these events more often.

My research into spiritualism and entities coming though mediums state that these entities tell us that spiritual guides or friends or family or whatever are capable of having an influence on someone’s thoughts. I.e. being late for choir practice.

This is why I believe something special may have occurred that night to make the entire choir absent even the choir director. Please note the word “may”.

No amount of statistics will alter one's belief one way or the other contrary to what you stated that statistically significant data would change your beliefs. Been down that road many times with skeptics as most people on this blog have.

I Really did enjoy our dialog I almost always learn new knowledge from these types of dialog but concerned we took up too much blog space that was of little interest to anyone but us.

I taught W Edwards Deming’s material for 5 years believing his teachings were just about SPC (statistical process control) like Juan’s, but then one Tuesday morning in 1991 during one of my seminars I had a moment of revelation (only one of this magnitude in my life) that allowed me see the profound implications of understanding variation as it applies to systems, organizations, educational institutions, government institutions, human performance, wealth generation, societies, and life.

I was able to take it deeper than Deming, not because I am smarter than him, but due to my research into Hinduism and Buddhism. Deming was a Christian so he was unable to connect the dots between ignorance and suffering like the Buddha and many enlighten Hindus were able to do.

What tipped me off that you do not see the profound implications of Deming’s teachings was when you stated these charts were only useful in industrial machine sittings and that most human performance was almost always in a Gaussian distribution. This is a common fallacy of Deming’s teachings so I do not fault your perceptions of Deming’s teachings.

Hey I am no whiz kid here it had to come to me in a revelation. I.e. gift. I used to put down Deming and state that he was just an SPC guru that went to Japan when they were down and out after the war. If I had not had that revelation I would be right there with you on this discussion.

Now what I really would enjoy dialoging about is how variation has a causal correlation to ignorance and ignorance has a causal correlation to suffering and: that later if interested. If we (me included) can get past this “male ego pissing contest” that would be an interesting dialog from my point of view.

I very much respect your statistical knowledge but people continually do not heed my words when I state someone can understand statistics and not understand Deming’s teachings.

[Noah Schwartz]

Hi Michael Prescott and friends; Very interesting conversation you are having here. You are all correct. I think I have the definitive story about coincidences proving the existence of life after death. Regarding the R-101 disaster, I was way into airships before all of this happened to me, I even have the license plate, LZ-172 on my car, the serial number of the Graf Zeppelin, ( sort of ), and I certainly knew nothing about the R-101 seance. I was Googled to your web site because of your essay on the R-101 seance as covered in the book 'The Airmen Who Would Not Die', by John G. Fuller. Back in 1999.

I believe the ghosts of Ray Hinchcliffe, the one eyed pilot pictured on the cover of the book, and Elsie MacKay, his ill fated socialite companion were in my apartment after I saw duplicate rooms of the R-101 in the Industrial Trust Company Building here in Providence at a little after 2:00 PM on October 5th 1999 when the elevator I was on stopped on its own on the wrong floor and went dead. I'm probably the only person within 200 miles of here that would have recognized the Promenade, the Dining Salon, and the famous Lounge of the R-101.

I've read this thread and got a kick out of the link about Time magazine article titled, 'A Long History of Hoaxes', because I always love seeing the word " hoax", and I also knew David Eisendrath. I was about to post about my story yesterday morning, but it got so involved that I deleted the whole thing. I figured I'd go have a cup of coffee, thinking how I could better tell you about the sort of coincidences that happen to me constantly, and and as I got out of my car, I saw this.

What are the odds of that, I ask you ?

Anyway, I really could use some help telling my story. I have a feeling I'm going to end up like this guy.The Amazing Randi didn't care for my story,as you might have guessed. The Ghostchaser people on TV, I haven't heard back from... But you are aware of the R-101 seance, and I have a whole lot more to tell you. I just don't know if this is the proper forum, but I'm not above humiliating myself at this point.

ANYWAY, thanks for your time