Not true. Your side, with virtually one voice, has acknowledged that the opposite bio sex goes through sometimes without issue. I do, too.
You don't do nuance, do you?
The critical question is whether anyone of the opposite sex is
legally permitted to be in the wrong-sex space. If the answer to that is no, then the people to whom that space rightfully belongs have control. If they see somone transgressing the rule, they can ask that person to leave. What happens next will vary according to how intransigent the interloper is, but the rational move at that point would be to say excuse me, and go. End of incident, nobody died.
There are two situations where that doesn't happen. One is where the interloper's disguise is so perfect that the deception is not clocked. Bear in mind the behaviour has to be unremarkable as well as the visual appearance. The likelihood of this happening is greater for a transman in the Gents than a transwoman in the Ladies, simply because testosterone is a one-way street so it is more likely that a woman will succeed in masculinising herself than a man will succeed in feminising himself. Also, I understand men don't look at each other or make eye contact in the Gents, which makes it even more likely they will overlook a reasonably-passing transman. In the Ladies women engage with each other, make eye contact, and strike up small-talk conversations. It's a lot harder for a male person to negotiate past that, but still, some probably do.
The Supreme Court recognised this, but of course could not give any legal ruling that such people are
allowed in the wrong-sex space. Their suggested remedy is slightly complicated but we needn't worry about that. The pragmatic situation is that very well-passing people will not be challenged because nobody will know.
The second situation is where the rightful occupant of the space recognises that there is an interloper present, but decides not to challenge him (or indeed her) for some reason. In a hurry, the person looks inoffensive, don't want a confrontation, whatever. In the past this was far from unheard-of, as Emily's Cat has said. Whether that sort of latitude will continue to be afforded an interloper after all the grief this tolerance has brought us is anybody's guess. Personally, I'm not in a very forgiving mood right now.
These things will happen. Maybe it's not perfection, but we live in an imperfect world.
What we absolutely do not want, will not countenance and will fight tooth and nail to prevent, is any
legal right for a man to be in the female space or vice versa. That shifts the power to the person who is trying it on in the wrong place. If challenged, he (it's going to be a he, let's face it) can claim to be one of the special exceptions group, and then we're into "prove it!", and it all escalates from there. Not at all desirable.
If nobody at all is permitted in the wrong-sex spaces, then people who pass perfectly will simply go on doing it and nobody will be any the wiser. People who don't are taking a risk. It's on them. The critical thing is that anyone who is legitimately in that space and objects to the interloper has the right to ask him to leave. If he has any consideration for others, or indeed sense, he will quietly apologise and go.