First off, Tegmark writes a lot of very very solid papers on observational cosmology. That part of his work is certainly not crackpot.
Not all infinite sequences are ergodic, although I suspect that "almost all" of them are.
It's difficult or impossible to find real physical systems that are
not ergodic. Generally speaking there are a few conservation laws (like energy, momentum, etc.), but because of quantum mechanics anything that doesn't violate those laws happens. The only way it can not happen is if the weight for it is precisely zero, and the only way I know for the weight to be precisely zero is if there's a conservation law forbidding it.
So Tegmark's view that anything that's not inconsistent will happen in an infinite universe isn't coming out of nowhere, it's coming out of a rather solid physical intuition.
Which means that somewhere there is a universe that consists solely of Royal Daulton figurines except for the exact center, which is a torus four km in circumference and made of caramel nougat.
Consider the "concordance" model of cosmology (that's the simplest model we know of that explains all the observational data). It's got various properties, but the one that matters most for this discussion is that it underwent inflation in its past, and that it has zero spatial curvature (it's "flat"). The latter implies that the universe is spatially infinite, and probably also temporally infinite towards the future. The former implies that the "initial" conditions for each part of the universe were set by quantum fluctuations that happened during inflation.
Quantum fluctuations are governed by a continuous probability density, the one I referred to above. That density is only zero on states that are forbidden by conservation laws. Combining those together, it's not easy to evade the conclusion that there is somewhere a giant tongue made of caramel nougat with "nya nya drkitten" written on it in squid ink.
So how can we escape that absurd conclusion? One way is to drop the assumption that the universe is flat on large scales. There is of course no observation evidence for that, since we can only see a finite piece of the universe. It's just that it's the simplest assumption that is consistent with data.
Another is a much more fancy idea based on "holography", which basically says that the universe doesn't really have a continuous infinity of states in it, even if it's spatially flat, because long distance gravitational correlations reduce the entropy to a finite value. Or in other words, that giant tongue made of nougat doesn't exist in any normal sense of the word "exist". That idea can be made precise in certain contexts, but not so far in cosmology.
Other than that, I'm not sure how to escape it.