most civilised stated recognize the equality of the sexes, i.e. they make laws and practices illegal that discriminate based on gender. (Same as with race, religion, etc.)
I know of at least one, although one could argue about "civilised". In the United States of America, it is permissible for the law to discriminate based on gender. (Example: Laws related to women in combat.)
I am not aware of any place that has laws that make discrimination on the basis of "number of people" illegal, hence this argument simply does not work for polygamous relationships.
While other people have applied my argument to polygamy, I haven't, (ETA: lately) and I don't think it works. There's a reason I limit my argument to incestuous relationships.
I am also not aware of any place that has laws that make discrimination on the basis of "biological affinity" illegal, hence this argument does not work for incesteous relationships. (In fact, there could be no laws against incest in such a place to begin with.)
There is nothing in the US Constitution, or the constitutions of most states, that prohibit discrimination based on either biological sex or sexual orientation. All such laws are restricted in their application, such as a prohibition on hiring preferences based on sexual orientation.
On the other hand, there is a statement in the Constitution that all citizens are entitled to equal protection of the law. A law that allows you to marry my sister but prevents me from marrying my sister treats you and me differently. Therefore, it is unconstitutional. At least, that seems to be the reasoning applied.
Just to avoid confusion, I want to make my personal position clear. I believe there should be no laws prohibiting incestuous relations. Indeed, I believe that such laws should be considered unconstitutional, using exactly the same reasoning as applied in Lawrence v. Texas. I do not believe that there is a constitutional right to incestuous marriages. For emphasis: sexual relations, yes. Marriage, no. Your milage may vary.