Okay. Let's try a different approach here. Let's take the language off the table as much as possible and still be talking.
100 people consisting of people with both penises and vaginas (that's various people who have either a penis or vagina not people with both oh you know what I mean...) all enter an athletic competition. At the end of the competition the top 10% of people all have a penis. The people with vaginas feel they are not being represented in the sport and start a seperate league where they have a chance of being in the top 10%.
Without using the words man, woman, male, female, trans(X), or any other oblique referencing to the labeling of the sexes and genders answer the following the question.
Is the complaint and actions of the vagina havers valid? Why or why not?
There's another word that I would like to avoid. That's "fair". I won't use it as I provide my answer.
For simplicity, I will address only sports as spectator driven events, but nearly idenitcal arguments apply in educational settings, and to a lesser extent, recreational leagues.
People watch sporting competitions because, for whatever reason, we love to watch displays of athletic ability. We want to see a combination of skill that has been developed practice, natural talent, and physical fitness that comes through training. Specifically with respect to that last element, we want to see athletes that have done everything in their power to maximize their bodies' potential to perform.
Having a vagina is correlated with other physiological, including hormonal, characteristics that limit performance. We want to see people who have developed their bodies to peak athleticism, but a vagina-possessor who has developed their body to peak performance levels will not be able to perform to the same level as a penis-possessor at the equivalent level of training. Indeed, a penis-possessor of mediocre ability can frequently outperform a world class vagina possessor.
In order to showcase the very best, most athletic, performers, there has to be separate leagues for people with penises, and people with vaginas.
If it can be arranged, the same can also be said for physical stature and other characteristics of athletes. For example, in sports where height is an advantage, leagues for short people may be appropriate. However, there often is not enough spectator or athlete interest in, for example, a short person's basketball league, in order to sustain the activity, so having a league specifically for other disadvantage categories (height, weight, age) may not be practical. However, there is almost always support of a league for vagina possessorts, so it is a good idea.
In summary, what people want in spectator sports is an opportunity to see great displays from the very best athletes. The very best athletes with vaginas are likely to underperform lots of the best athletes with penises. Separate leagues should exist wherever possible to showcase these great athletes that are born with physiological conditions that limit their absolute level of performance.
ETA: To answer in the words of the original question, the actions of the vagina havers is valid, i.e. it serves a good purpose, and is the best way to serve that purpose. I don't know if there is a "complaint" as such. They just observe that the best vagina possessors almost always lose to not just the best penis-possessors, but most of the penis possessors.