And furthermore, I strongly suspect that this is where we will eventually end up in any case. Transwomen (and transmen, for all that) will be allowed to use all public and public-service facilities which match their trans gender. I will - and do - heartily applaud that.
What counts as a transwoman?
That isn't a flippant question, BTW. I'm being quite serious. I presume that you do not want cismen to be able to access all facilities of any gender, for example. But how do you distinguish between a cisman and a transwoman? There are a number of possible answers, and they have considerable consequences if transwomen are allowed access to women-only spaces but cismen are not.
One possible answer, and the one preferred by certain posters here, is that people get to self-declare what they are. This position is problematic, to say the least. It essentially means that there are no women-only spaces, because anyone can declare themselves to be a woman (trans or otherwise), so anyone can access women-only spaces, and so they are not actually exclusive in any real sense.
Another possible answer is that people have to get some sort of official declaration of trans status. I believe that's the case in the UK, at least officially if not in practice. But depending on the requirements to get such a declaration (and apparently it's trivially easy in Canada), this may end up being little different than our first scenario. But if you were required to get a full evaluation by experts who can and do reject applications, that would be considerably different.
Another possible answer is that people have to be passing. In other words, a transwoman has to look like a woman (maybe not perfectly, but at least in large part) in order to access such spaces. So no walking into the locker room with a beard, for example. This one has fuzzy boundaries, which will cause problems for judging edge cases.
Another possible answer is that you require people to have medically transitioned. A fully post-operative transwoman is likely to be viewed by most females much differently in a shared clothing-free space than a transwoman who has not transitioned at all. But that answer isn't acceptable to many trans activists.
In other words, even if I agree with your statement as phrased, that's far from the end of the discussion. It's really only the start. So how do you think transwoman should be defined for the purpose of access? And there doesn't even have to be one answer. It can even be different answers for access to different things.