My view is that the vast majority of sports should be separated by sex for children that have entered puberty.
I think that elementary school sports can be co-ed in terms of competition. Although some children will have begun puberty by the end of elementary school, the majority have not. Additionally, elementary school competitions rarely extend beyond the specific school itself, and are generally more for fun and exercise than for competition.
Generally speaking, by the time we get to middle school, the majority of kids are in the throes of puberty, where the changes to their bodies become significant - and they do so in a relatively short period of time. Additionally, for most schools, middle school is the first time that sports will have varsity, junior varsity, and general categories, and the competitions begin to include other schools within the region and the state.
I also think it's perfectly fine to have non-competitive "open" leagues at all ages which do not separate by sex.