The most efficient way to determine that something is not there is too look for it and not find it. Do that long enough, and it becomes reasonable to assume that the thing you are looking for remains unseen because it really is not there. When Zwicky (
Zwicky, 1933;
Zwicky, 1937) and Oort (
Oort, 1932) first developed the "missing mass" problem, they knew quite well that there was a lot of perfectly ordinary matter that would be simply too dim for them to see, so they did not consider any need for exotica.
However, during the intervening decades, observational capabilities in astronomy have grown enormously. In their day only visible light astronomy was practical, but today we have high resolution and high sensitivity capabilities all the way from long radio to short gamma-ray wavelengths, and we have telescopes with light collecting area as much as 16 times what they could do (i.e, the Keck 10-meter vs. the Mt. Wilson 2.5-meter). We have developed the capability to rule out "ordinary" baryonic matter on the grounds that if it were there, we would see it. So, for instance,
thanks to the Hubble Space Telescope, we now know that there are not enough red dwarf stars in the Milky Way halo to account for the missing mass (and see
Bahcall, et al., 1994). Gravitational microlensing studies can expand that to assure that there are not enough compact objects of any kind, "normal" or "exotic", to account for the missing mass (
Pratt, et al., 1996;
Alcock, et al., 1996;
Yoo, et al., 2004).
When we look at the "bullet cluster" (
Clowe, et al., 2006) we see that the baryonic intracluster mass, visible in X-rays, is not spatially coincident with where we know the mass to be located. only the galaxies are there, but we know that the masses of the galaxies are an order of magnitude too small to account for the observed gravity. We know this partly because we know the stellar mass-luminosity relationship, and we can estimate non-stellar mass by comparing the galaxies to similar type galaxies in the local universe, where non-stellar mass can be directly observed and accounted for. There may be a lack of desired precision, but not a significant lack of accuracy.
In general, gravitational lensing reveals that there appears to be a great deal of gravity in the universe, without the visible matter one would normally associate with mass (i.e.,
Massey, et al, 2007). At some point the constant association of gravity in the absence of detectable mass has to be dealt with seriously. Non-baryonic dark matter is an elegant and reasonable idea, despite your unreasonable objections. It's no more serious than the simple assumption that there is more of the same kind of stuff we already know about (i.e.,
neutrinos), but have simply not detected yet.
So there is good reason to believe that most of the mass is not normal baryonic matter. Of course, that is not the unanimous consensus of the community. Quite a few groups are working on the obvious alternative, that we have the law of gravity wrong, and that by modifying it we can dispense with "dark matter" altogether (and maybe "dark energy" as well). But it is the majority opinion primarily because it is much the simpler idea.