Why all this talk about feelings? Let's talk about harm.
What harm would it do to let trans people use the facilities of their choice?
There appear to be two arguments.
One is that people might see genitals they're not expecting. I'd imagine that most trans people who want to pass for their target gender aren't going to publicly display their genitals, especially since there's a lot of anti trans violence, and they don't want to out themselves to random strangers. But I agree, it is at least possible, though I'm not sure how it would constitute actual harm.
The other argument is that rapists, who up until now have been either too scared or too polite to enter women's locker rooms, will claim to be trans in order to somehow justify attempted rape. That argument is just ridiculous.
Now, what's the harm if we make people use the facilities intended for their birth sex?
Well, people who look like men will be in the ladies' room, and vice versa. This won't do anything to lessen the fears of people who are afraid of people of the opposite sex in their facilities, because superficially, they won't look like they belong there and people won't respond well to the perceived 'intruders'. And again, it's probably less safe for the trans folks to be forced to out themselves like this.
What's the harm of doing as Dragonlady suggested and expecting people to not use public facilities if they don't conform to the norms? Well, that would mean trans people could never change their clothes or use a toilet outside their own homes. I don't think it's fair to deny people access to public facilities because they look different.
So I'd say, for the least amount of harm, go with option one. You'll probably never see anyone's genitals anyway, and the rape argument is nonsense.