There is a huge gap between universal morals and scientifically determined morals. And in what works is subjectivism the old fashioned belief?
And even then, look at murder. Everyone has the rule not to kill... Some people in some situations with extreme variation.
This is the problem with welfare based arguments. If you are talking about the welfare of a group that I am indifferent to or have animosity for, or if my or my groups welfare increases from their suffering... Who cares? There is zero value.
You argument falls down immediately upon examination. If your argument was correct then you would agree that the actions of mass murderers, sexual offenders, thieves, etc, are moral because they are indifferent to the suffering of their victims.
Individuals don't decide morality, societies/populations do. The problem you point out is one that has been addressed many times as in-groups have grown. When clans gave way to tribes and then to cities, nations, and finally empires, moral viewpoints were forced to change.
Harris does nothing but extend this to the point that all humanity is the new in-group so it doesn't matter if you like a group or not, it only matters that they are a part of the group that morals apply to.
It was science that showed us humans are human. That there are no races, no difference between the people of the bear clan and the eagle clan, the Maori and the Zulu, Russians and Americans.
Harris extends this to say that based on this, science is the only way to remove bias when making moral decisions. Traditional methods, steeped in culture, are very prone to making poor moral judgements based more on what is beneficial to the in-group than what is right. Your argument is a perfect example for his case.
Harris also argues, as does Shermer, that it is time for people to start looking at this field from a scientific perspective. That science is the only proven system we have for deciding anything so why shouldn't it be applied to morality?
They both have a point in my view. We've seen what happens when you let a leader of a small in-group (religion) decide the moral issues of the day for a larger population. Harris' ideas can't be any worse than those results.