At the close of the 1920s, the theory of gravitation as outlined by Newton and Einstein was perhaps the most successful scientific theory to date, the major structures of the universe were largely understood and the art of lens-making had advanced to such a degree that astronomers could really get down to the nitty–gritty business of cataloguing stars and galaxies.
It therefore came as something of a shock in the early 1930s when various astronomers (Jan Oort and Fred Zwicky usually get the credit) pointed their powerful telescopes at various structures, both within and beyond our own Milky Way galaxy and declared that they were moving too fast. That is, the observable mass within each system was insufficient, often by several orders of magnitude, to hold themselves together gravitationally given their reckless speeds. Later measurements using gravitational lensing (the bending of light by super-massive objects) confirmed that most objects larger than star systems contain far more gravitational mass than the sum of their stars, free atoms and other known objects can explain. For those who saw it as their job to describe the Universe in detail, failure to account for 80% of its mass was considered unacceptable. The excess mass was termed ‘dark matter’ and that uninformative moniker summarized the state of our knowledge for nearly half a century.
Ordinarily, when astronomers encounter a new phenomenon, they are able to use better and better information to design better telescope searches until a coherent picture begins to emerge. When the new phenomenon is explicitly not visible to telescopes, indirect observations must be employed. The 1960s-1980s saw a number of increasingly precise maps of the speeds of different parts of a galaxy as a function of distance from the center. (The technique is based on redshifting of known chemical signatures.) They could then calculate from Newton's Law of Gravitation how much mass was enclosed within the orbit of each object. They found that dark matter is usually distributed concentrically within a galaxy and maintains substantial mass density well beyond the observable stars. Observations of satellite galaxies, such as globular clusters orbiting our own Milky Way, indicate that the dark matter halo usually extends to many times the radius of the ‘baryonic galaxy’ it supports. 'Baryon' is a category that covers protons and neutrons (and similar particles of known properties), the primary components of 'normal' matter.
With some meaningful data to go on, the 1980s saw a proliferation of theories to explain the missing mass. Many of them stem from finite visibility. Telescopes have limited resolution and there are whole classes of objects which they are simply unable to observe. If galaxies were packed with black holes or planet-sized rocks, they would be difficult to detect using light-collecting telescopes. Several comprehensive surveys of non-luminous objects have now largely ruled out this “dark normal matter” hypothesis. Additionally careful studies of light from the early Universe, when everything was much more tightly packed, suggest that most of the matter that was around then did not interact as strongly as the baryonic matter we see around us today. (See, e.g., the Wilkinson Microwave Anisotropy Probe (WMAP).) The existence of non–baryonic matter strongly suggests that some previously unknown particle account for the locally observed excess mass without interacting in a noticeable way with baryonic matter.
At present there is a strong, though not unanimous, consensus in the astrophysics community that we should be looking for at least one new kind of particle to explain dark matter. We know that it has non–zero mass and that it interacts very rarely compared to atoms. (To help us compare different theories, we quantify interaction strengths by the 'cross–sectional area' that particles would have if they were solid balls flying past each other.) Despite several decades of work, there are still quite a few categories of viable theories. Some of these are impossible to test using present technology, but a number of active experiments are designed to probe different regions of the unknown particle mass vs. cross–section space.
All dark matter detection experiments aim to take in as many potential interactions as possible and then search for very rare events, as few as a handful of interactions scattered across multiple years of operation. However the details vary considerably as different searches rely on different kinds of physics to measure an event. In accelerator-based searches, large numbers of high energy collisions are the relevant scientific currency and new particles are discovered by the mass–energy they carry away. Telescope-based searches often look for characteristic high-energy light bursts from hypothesized dark matter decays. The interaction space is the entire sky, but the events being sought are more diffuse. 'Direct detection' experiments rely on ambient dark matter, which is supposed to be sleeting continuously through the Earth, interacting very rarely with a detector in the lab. Increasing the number of potential interactions means building heavier and larger detector arrays.
Besides the observational questions of missing mass in the early and present Universe, some form of dark matter is required by many theories intended to address how the Universe evolved from a uniform, hot plasma to the structured, yet diverse place we see around us. We are grateful to live in a world where gravity, chemistry and nuclear physics can co-exist, but we strongly suspect that these seemingly unrelated processes are all derived from some unified set of physical laws. And it would be nice to know how that was done. Detecting dark matter and determining its properties would provide a powerful tool to decide which, if any, of the ideas proposed to answer this question is correct. As of yet, we have succeeded only in ruling out sections of the mass vs. cross–section space, but even these exclusion limits have helped to prune the thicket of dark matter theories.
One recent survey identified about 600 people worldwide as 'dark matter physicists', scientists prepared to spend a significant portion of their career searching for a particle which they believe ought to exist, despite precious little theoretical consensus regarding its properties. I am one of those physicists. Strictly, I do not know that there is dark matter in the same way that I know about gravity or God. Establishing the existence of particle dark matter to high confidence would require multiple detections; confidence requires repeated interaction. But if that were the only remaining goal most of us would probably invest our research time elsewhere. I think the case for particle dark matter is fairly strong, but right now I have to exercise a degree of faith. Because the interesting physics, the knowledge that will change how we think about matter and fundamental forces, lies beyond the question of existence. Far more than simple mass accountancy, it is the properties of dark matter that will inform our understanding of physical laws.
This interplay between faith and knowledge is often a part of the scientific process, but is less often explicitly discussed. All scientific knowledge comes with some measure of confidence which improves as many independent experiments verify a particular hypothesis. When viewing established theoretical frameworks supported by mountains of evidence, it is easy to lose the point of view of the people who sifted through that evidence, knowing that the truth was important and believing that this idea was worth pursuing.
The dark matter hypothesis is one case where we have remained 'fairly confident' for an extended period of time while we gather many different kinds of evidence. This situation is a good analogue for the common deity detection problem. The question “Is there a God?” is not readily addressable by observation and, for practical purposes, not nearly as interesting as “What is God like?”. To get from “There might be a God” to actively searching for a God who came to Earth and died on a cross requires an appreciation for the implications of this idea, a measure of faith and enough humility to possibly be wrong. This is not a strange feature of religious knowledge. It is part of how finite people learn new things about a creation beyond our comprehension.