Data Volume Is Fourth Frontier in Astrophysical Observation

Martin Harwit's article "The Growth of Astrophysical Understanding" (Physics Today, November 2003, page 38) was enjoyable and insightful. The author names three frontiers in observation: angular resolution, timing, and spectrometry. I propose a fourth: expanding data volumes. This frontier has progressed steadily, from paper−and−pencil catalogs, to photographic plates, to large−format CCDs and robotic survey telescopes.

Each advancement brings new discoveries. Some examples: Tycho Brahe's measurements of the planets were precise, but also voluminous; he had 85 data points on Mercury's orbit, for example—a great advancement on the state of the art. The volume, as much as the precision, made Johannes Kepler's analyses possible. John Goodricke discovered one variable star in 1784, but Henrietta Leavitt, working with thousands of stars on photographic plates, discovered the Cepheid period−luminosity relation. Today, in one of many superb examples, a huge observing project called the Sloan Digital Sky Survey is making precision measurements of large−scale structures. The project's resolution and spectroscopy are up to date, but its multiterabyte data set is really new and exciting.

Bigger and bigger surveys are absolutely necessary for advancing our knowledge of astrophysics. They provide the opportunity to discover extremely rare phenomena and to find surprising statistical properties of known ones.

Harwit replies: Benjamin Monreal is quite right. Expanding data volumes have had an enormous impact on astronomical discovery. Although my article could not include it, my book Cosmic Discovery (Basic Books, 1981) provides a plot of the eight orders of magnitude improvement over the naked eye, in sensitivity and data gathering rates, made possible by 1980 through increased telescope apertures, more sensitive photo−response, and increasingly large numbers of spatial resolution elements on photographic plates and photoelectric arrays. The recent discovery, by the DENIS and 2MASS all−sky surveys, of a few dozen extremely low−mass brown dwarfs among hundreds of millions of more ordinary stars, shows the added value of machine data processing.