I'm not sure your quote is quite accurate. I think females in a female-only space have reason to fear seeing male genitals.
Now, I don't mean to say that genitals are in and of themselves something to be scared of. But there are common taboos against exposing your genitals to people of the opposite sex. If someone knowingly exposes their genitals to you in an inappropriate context, that indicates that they are disregarding that social taboo. And it is that disregard which represents a possible threat. If someone disregards one set of social taboos, that indicates a higher risk that they will disregard other social taboos, such as the taboo against sexual assault. So inappropriate nudity is a proxy indicator for other antisocial behavior and thus danger. The correlation need not be perfect in order for the fear to be a rational response.
In regards to your question, though, it doesn't work the other way round to the same extent. First, in regards to my above comments about antisocial behavior, women pose less of a physical threat to men, so a woman who is willing to do bad things is less dangerous (in general) than a man who is similarly willing. Second, in regards to the fear of someone of the opposite sex seeing your genitals, men are more sexually aroused by visuals than women are. So to the degree that people prone to misbehavior because of sexual arousal, men are more at risk of that than women.