http://www.planet-science.com/about_sy/events/pdfs/Steve_Jones_Transcript.pdf
It's basically due to the difference in reproductive systems. Men can spread their sperm around all over the place, which tends to result in a few powerful men having lots of children, while the less powerful have significantly fewer. That's not possible for women because of the whole pregnancy thing, so the distribution of children tends to be more spread out across many women.
If you look at this diagram (from here), it shows how the idea of the most recent male or female ancestor can be constructed. That picture shows for women, but if you do the same for men, there will tend to be more than just two or three children at each point, and so it will converge back to the common ancestor much more quickly.
The important point to remember is that this does not say that there were no other people alive at the time, or that this is the ultimate ancestor, both of which are obviously inconsistent with the male and female one occurring at different times. Instead, it is simply the most recent ancestor that all men or women share. Lots of people will also share many ancestors from both earlier and later, and some of the earlier ones will also be common to everyone.
The bit being missed in the question is that it it is last demonstrated common ancestor.
This is shown by looking at y-chromosomes for men, which are only carried by men, and by mitochondrial DNA which comes only from the mother.
None of my daughters will carry my Y-chromosome. My son will carry his mother's mitochondrial DNA, but he won't pass it on.