Gender isn't assigned at birth, it is socially inculcated over time. Sex is assigned (some would say documented) at birth.
As to who is supposed to use what, I don't really have a dog in the fight. You could at least specify whom you're arguing against. I'm guessing gender-critical feminists?
This. Though it would be more accurate to say sexual phenotype.