Yes.
The concept of moral relativism is really in play here, and something I think is a valid viewpoint. If there is a society where holding slaves is considered normal and decent, I would not say that every single person in that society was some sort of monster. They were part of a system that I would say is terrible, but that doesn't mean that every single person who was part of that system is terrible.
I'll give an example of someone with quite a few statues, although not in the United States. Ever heard of Cuauhtémoc? He's got a fair number of statues to him in Mexico. He was the successor of Moctezuma II, and was the last emperor of the Aztecs. Statues can be found in Mexico. I don't know of any in the United States.
If you've never done it, I recommend doing some reading on the Aztecs. If ever there was a society more firmly evil than the Nazis, it was the Aztecs. Incredibly, horrible, awful, people. I grew up thinking that Hernan Cortes showed up with guns and overthrew the empire. Nope. He rallied a bunch of Indians who absolutely hated the Aztecs. Those guys not only owned and traded in slaves, they cut out their still beating hearts to bring rain. If there were good guys and bad guys in the battle of Tenochtitlan, the Spaniards were the good guys.
So, when we're demanding statues come down, I assume Cuauhtémoc is on the list?
Or do we just say, perhaps, that he is an important figure that represents a heritage significant to the Mexican people, and that the statues represent that, and not some sort of endorsement of slavery and human sacrifice?