Let me express myself in another.
Heterosexual marriage is pair bonding. It's 1-1 attachment. Homosexual marriage is also pair bonding (and involves people that probably shouldn't be getting married heterosexually anyhow).
The polyamorous dynamic is fundamentally different mathematically. So the arguments for homosexual marriage can't be carried over trivially. For instanc,e if 50% of the population engages in polyamorous attachments, then the insitution of pair bonding IS damaged, as people who prefer that have trouble finding mates (I'm not attached a moral evaluation here, just stating a fact). Indeed, the dynamic is different in a lot of respect on a psychological level as well. A small group can be dominated by a charismatic individual, leading to other members of the group being neglected. It also can mean that when there is a problem in the relationship, instead of focusing on that problem and fixing it, a person can instead just shift to other relationships and is therefore inattentive of the problem. Granted, it is possible for such things to happen in a pair bond, but it is my contention they are less likely and less damaging. There are other potential abuses that could make things which we consider bad now (trophy wives) even worse (multiple trophy wives). One can't trivially say these concerns have no substance, though I admit it is possible they'd come to naught.
Since marriage is something of social value, one shouldn't change the fundamental dynamic without careful thought and consideration. I would therefore advocate that before any law allowing multiple marriages ever came to pass, careful research must be done to evaluate the short term and long term consequences of such a change. It may end up being a positive thing, or it may not, but we should not blithely assume one or the either. (I'll grant my personal feeling on the matter is that it would not be good, but I admit that my reasoning would need to be put to the test to either validate or invalidate it).