That's interesting. I can't see how anyone could miss it. It seems so obvious to me.
However, I'll take you at your word, and I will try to explain.
Why aren't men allowed in women's shelters, in women's locker rooms, in other women's only spaces? Is it because we are mentally ill perverts? No. That's not it. It's because we are men. In the above you referred to "dangerous men". Well, it turns out that it is impossible to tell the dangerous men from the not so dangerous men. So, all men are categorized as dangerous men if they want to be in a women's only space. We're bigger. We're stronger. We can make them pregnant, against their will. We have attributes that make us dangerous.
Most of us wouldn't actually rape or assault women, but it's impossible to know which of us would and which of us wouldn't. You can only make that judgement after the fact, and then it's too late.
So, all men are banned. Not some men. All men.
And, well, as much as many people don't like it when people say that, trans women can make cis women pregnant. Trans women are, statistically, bigger and stronger than cis women. Trans women have every single characteristic that makes men be treated as dangerous.
It's not a moral judgement. It's not a values issue. It's a practical reality.