This.
As individuals we are necessarily selfish. But there's only so much any one person can achieve. We can get more for ourselves by working together for the common good, which then benefits everyone. The problem is how to reach a balance between the desires of the individual and those of the group. People can have very different ideas on where that balance should lie, and most are wrong because it's impossible to know ahead of time (or even after) exactly where that point lies.
That's why humans developed empathy - evolution 'bred' into us the ability to imagine how we would feel in another's situation so we would act in the interests of the group. There's a lot of variability though, which helps us find that balance under differing circumstances. This flexibility is part of what has made humans so successful.
But there's a downside to variability - politics. People of like minds find it advantageous to combine forces so they can 'force' their views onto others. So 'hard minded' selfish people gravitate towards one party, while 'soft-minded' empathic people gravitate to the other. In a two party system that unfortunately leaves no room for other spectra such as conservatism (maintaining traditions and opposing change) vs liberalism (embracing change and discarding old ways). This leads to many being dissatisfied with their party, both on the 'left' and 'right'.
Libertarians are selfish, but that doesn't necessarily make them conservative. Being selfish means they think they should be allowed to do whatever they want, but they recognize that this won't work unless restricted to things that don't harm others in the group. They believe that with a minimum of 'fair' laws in place everybody can be selfish while also benefiting the group as a whole. However in practice this is very very difficult to achieve without significant restrictions, which makes a practical 'libertarian' society not much different from our current system.
You can lack empathy and yet still recognize that a society which doesn't keep its members happy won't last long. But (unlike empathy) that requires both intellectual and emotional intelligence. When selfishness is combined with stupidity you have a problem. When logic is clouded by emotion it's worse. You say Ayn Rand was intelligent. Perhaps she was, but that wasn't enough. The same applies to her fans.
