Project Blue Book - Project Blue Book Special Report No. 14

Project Blue Book Special Report No. 14

In late December 1951, Ruppelt met with members of the Battelle Memorial Institute, a think tank based in Columbus, Ohio. Ruppelt wanted their experts to assist them in making the Air Force UFO study more scientific. It was the Battelle Institute that devised the standardized reporting form. Starting in late March 1952, the Institute started analyzing existing sighting reports and encoding about 30 report characteristics onto IBM punched cards for computer analysis.

Project Blue Book Special Report No. 14 was their massive statistical analysis of Blue Book cases to date, some 3200 by the time the report was completed in 1954, after Ruppelt had left Blue Book. Even today, it represents the largest such study ever undertaken. Battelle employed four scientific analysts, who sought to divide cases into "knowns", "unknowns", and a third category of "insufficient information." They also broke down knowns and unknowns into four categories of quality, from excellent to poor. E.g., cases deemed excellent might typically involve experienced witnesses such as airline pilots or trained military personnel, multiple witnesses, corroborating evidence such as radar contact or photographs, etc. In order for a case to be deemed a "known", only two analysts had to independently agree on a solution. However, for a case to be called an "unknown", all four analysts had to agree. Thus the criterion for an "unknown" was quite stringent.

In addition, sightings were broken down into six different characteristics — color, number, duration of observation, brightness, shape, and speed — and then these characteristics were compared between knowns and unknowns to see if there was a statistically significant difference.

The main results of the statistical analysis were:

• About 69% of the cases were judged known or identified (38% were considered conclusively identified while 31% were still "doubtfully" explained); about 9% fell into insufficient information. About 22% were deemed "unknown", down from the earlier 28% value of the Air Force studies.
• In the known category, 86% of the knowns were aircraft, balloons, or had astronomical explanations. Only 1.5% of all cases were judged to be psychological or "crackpot" cases. A "miscellaneous" category comprised 8% of all cases and included possible hoaxes.
• The higher the quality of the case, the more likely it was to be classified unknown. 35% of the excellent cases were deemed unknowns, as opposed to only 18% of the poorest cases. This was the exact opposite of the result predicted by skeptics, who usually argued unknowns were poorer quality cases involving unreliable witnesses that could be solved if only better information were available.
• In all six studied sighting characteristics, the unknowns were different from the knowns at a highly statistically significant level: in five of the six measures the odds of knowns differing from unknowns by chance was only 1% or less. When all six characteristics were considered together, the probability of a match between knowns and unknowns was less than 1 in a billion.

(More detailed statistics can be found at Identified flying objects.)

Despite this, the summary section of the Battelle Institute's final report declared it was "highly improbable that any of the reports of unidentified aerial objects... represent observations of technological developments outside the range of present-day knowledge." A number of researchers, including Dr. Bruce Maccabee, who extensively reviewed the data, have noted that the conclusions of the analysts were usually at odds with their own statistical results, displayed in 240 charts, tables, graphs and maps. Some conjecture that the analysts may simply have had trouble accepting their own results or may have written the conclusions to satisfy the new political climate within Blue Book following the Robertson Panel.

When the Air Force finally made Special Report #14 public in October 1955, it was claimed that the report scientifically proved that UFOs did not exist. Critics of this claim note that the report actually proved that the "unknowns" were distinctly different from the "knowns" at a very high statistical significance level. The Air Force also incorrectly claimed that only 3% of the cases studied were unknowns, instead of the actual 22%. They further claimed that the residual 3% would probably disappear if more complete data were available. Critics counter that this ignored the fact that the analysts had already thrown such cases into the category of "insufficient information", whereas both "knowns" and "unknowns" were deemed to have sufficient information to make a determination. Also the "unknowns" tended to represent the higher quality cases, q.e. reports that already had better information and witnesses.

The result of the monumental BMI study were echoed by a 1979 French GEPAN report which stated that about a quarter of over 1,600 closely studied UFO cases defied explanation, stating, in part, "These cases … pose a real question." When GEPAN's successor SEPRA closed in 2004, 5800 cases had been analyzed, and the percentage of inexplicable unknowns had dropped to about 14%. The head of SEPRA, Dr. Jean-Jacques Velasco, found the evidence of extraterrestrial origins so convincing in these remaining unknowns, that he wrote a book about it in 2005.