I've personally always found it incomprehensible how the Southern elite managed to get the rest of their countrymen to support them in a war that was against their interests.
Because it was about a whole lot more than just slavery. The protectionists tariffs hurt the poor farmers and families a lot more than it did the elites. In fact, the biggest reason for the increasing rift between the northern and southern states throughout the first half of the 19th century was the tariff issue.
Then there was the issue of creeping Federal power, which the northern states liked because they were the majority in both houses and the southern states didn't like for the same reason.
But even then, they ultimately didn't cry war, just secession. If Lincoln hadn't sent in the troops, and had surrendered Fort Sumter like he did the rest of the military bases, no one would have been attacked. The South was on the defensive.
ETA: Hmmm. I don't always express myself in a tactful way and I just realized that my post probably could have been written more diplomatically considering that you said you traced your family roots back to the 1700s and that chances are high, that even though they didn't own slaves, they still fought for the Confederacy.
No, you're fine. No problems here.