Skipping over the rest, the problem with assuming all these to be distortions by his enemies is that most of those folks were actually not his enemies. The Slavic chronicle for example is very much in his favour, and pretty much sees him like you: a great warlord, and justified in his cruelty. Yet about half the stories about him in that source corroborate the tales of cruelty in the German sources.
And even to the Saxons (and most of the rest of Christian Europe, the Pope included) his homicidal raiding the Turks actually made him HUGELY popular. He was pretty much a hero of Christianity at the time, which might also be one reason why Matthias Corvinus actually treated him very well after he supposedly threw him in jail (it was actually more like a house arrest in the Corvinus Castle), but then again treating captured kings well was standard in the middle ages. At any rate, pretty much nobody wanted to believe the evidence that Matthias presented against Vlad, because Vlad was such a hero of Christianity, slayer of many Muslims, and all that.
Since you say you knew all along that he ruled in Wallachia, you probably also know that the Saxons were in Transylvania, where they had been colonized by the King Of Hungary, not in Wallachia, which was outside that jurisdiction. They didn't move outside much. In the 17'th century, for example, long after Dracula's time, their distribution looked like this:
http://upload.wikimedia.org/wikipedia/commons/d/de/Saxons_Transylvania.svg
Notice how it stops at the Carpathians.
Although there was some conflict between Vlad and the merchants of Transylvania, some of which may have been Saxons (especially around Sibiu), and a couple of raids of his against them (oddly enough, despite being allied to Matthias, their liege), mostly they had no problem with each other and they had finally signed a peace with each other in 1460.
Not saying there weren't people there who hated him, but it's hard to see more of a reason to defame him than for any other ruler in Europe.