Economic and social concerns, for the most part. Those reasons are just as good as genetic ones, perhaps even better than.
Economically, it makes more sense to gain resources by reserving your offspring for marriage outside your family.
In the father/daughter dynamic, no, you can't have sex with them first; they need to be presented at full value in the transaction, and no one wants your leavings.
With the mother/son dynamic, the son is usurping his father's property. Gee, that can't be good. Get your own woman, kid; I was literally here first. And besides, there could be a significant power imbalance: a son obeys his mother, but a wife obeys her husband...so how does it work if your lover is also your mother? Probably not too well. Besides, he's needed to form new social and economic alliances, just as his sister is.
With brother/sister pairing, we actually do see that in some systems, but usually amongst the royalty, the elites. Not so much among the commoners, because such a pairing interrupts the economics again. Better to have your kids making new alliances and drawing in more economic and social benefits than wasting their potential on each other.