What makes "hate crime" legislation specifically different enough to make your skin crawl?
Well, for one, the discretion that it gives prosecutors to prosecute a crime according to subjective can lead to abuse. Now, prosecutors always have some discretion, but in the case of hate crimes, that discretion can get coupled to identity politics, which can make things messy in ways which may not favor actual justice being done.
But perhaps more importantly, at least in the US, hate crime legislation is often a mechanism for changing jurisdiction for the prosecution of a crime from a more local to a broader level. So for example the feds can get involved in crimes which would normally simply be local matters. If you believe in the idea that local government is generally preferable to national government, that's a minus.
Now, there's a damn good reason that change of jurisdiction was implemented in the first place: hate crime laws were originally a way for the feds to step in to prosecute crimes that the local governments were intentionally failing to prosecute because of racism. The feds couldn't prosecute the original crimes since they have no authority to enforce local laws, so new crimes had to be created in order for the feds to have the legal ability to step in. Those crimes were originally tailored fairly narrowly to try to only cover those crimes the local governments were likely to refuse to prosecute.
But while this approach was quite understandable given the restrictions that our federal system of government has, it was never a perfect solution. The original problem is now mostly absent, but instead of shrinking, hate crime legislation did what so many aspects of government did: it grew instead. It started to cover
more crimes, not fewer. And to justify this, rather than pointing to the original purpose (the failure of local government), new reasons are now given. But it's not clear that these new reasons really justify such jurisdictional shifts anymore. And that, alone, is cause for concern. If nothing else, such legislation can smack of bureaucratic over-reach.