This is the crucial distinction. Someone has no choice about what skin color they are born with, and slurs based on skin color are racist by definition.
But all whites are not crackers any more than all whites are skinheads or clansmen or Nazis.
Plenty of whites manage to be born poor in the South and manage somehow not to fit the description of 'cracker'. And some are born to privilege beyond their peers and work hard at touching all the bases that would make them fit the description to a T. Big pick-ups with Confederate battle flag decals, gun racks, etc. And, yes, disparaging blacks at every opportunity.
Yes, it is a derogatory figure of speech, and intended as one. But it is not a racial slur. It is a character description. It is in no way by any stretch of the imagination any sort of an equivalent to the "N" word. And any efforts to make it seem so reek of the most base and desperate apologetics.
People are not born fitting the description as 'crackers'. They earn it. There is little doubt that the McMichaels have earned it. Even worked for it.