A lot of cynics here. Here's a brief run-down of what gave the LDS church its momentum:
From the 1810's through the 1840's, give or take a decade or two, there were a number of "revivalist" movements along the East Coast religious folk, up and down the coastline. Smith was definitely a huckster in his youth, using "seeing stones" to hunt for treasure, and he eventually stumbled upon a local revivalist group in Pennsylvania. He'd already had plenty of priming during his huckster days in telling tales and maintaining a narrative, and he apparently was a sucker for attention-- even Mormon stories of Smith's youth state he was quite the storyteller as a child. This was also around the time that the Rosetta Stone was discovered, which among intellectual circles (some of whom were familiar with Greek) was quite an interesting find. Additionally, it was a time where Andrew Jackson's anti-Native-American campaigns were fresh and contentions between Americans (and American settlers) were quite high. What Joseph Smith managed to do was take the common anti-Native-American sentiment, the revivalist/millennialist movements growing at the time, and his penchant for telling tales about his ability to speak unknown languages, and build a narrative of religious exceptionalism that connected his story to old and ancient people in a manner relating to the religious movements he was growing more influenced by. He chose what were at the time completely unprovable criteria to fill the gaps in his claims, which is where the Israelite "tribe" came from, as well as his fictional "reformed Egyptian" language, and built a tale that was a fuzzy mirror image of the Exodus story and the Gospels-- two of the more common Sunday School tales that most religious children knew-- into his tale of these unprovable people he was claiming special knowledge of.
So, yes, by all rights, Smith can easily be considered a huckster and a scam artist. However, he built his scam on already-existing religious movements and, over time, adjusted his story and his budding religion accordingly. Many of the people who followed him genuinely seemed to be "true believers," and not simply because Smith was so adept at selling a scam. Many of these earliest members were already involved in the revivalist/millennialist gatherings taking place, and Smith was absorbing members through these other groups. We hear little of these groups because, on the whole, they died down over time or were absorbed into other Protestant denominations (Pentecostals are an example). Further, Smith managed to stifle investigations into his claims by having many early members sign affidavits affirming Smith's claims, and even though one or two early members broke off from the church they never refuted their signed affidavits (they never re-affirmed them, either, as far as I know). People believed Smith not only because he told a fantastic tale that was believable enough for them, but because he had signed statements from people affirming that Smith's claim was true. As far as they could reason, Smith couldn't possibly have been lying since he went through such secular steps to verify his religious claims and had them affirmed with evidence-- not very strong evidence, and certainly not strong enough to validate his claims, but ignorance is bliss. Most of the early decades of the Mormon church grew with this built-in cognitive hiccup, and Smith rode that wave of growth to what can be read of the LDS church history from various sources. Smith himself wasn't too prejudiced against black people-- Smith mostly made up crazy crap about Native Americans-- but Brigham Young (his successor) definitely was. Thousands of otherwise well-meaning individuals starting from the rantings of these leaders of the early Mormon church have basically established the foundation of weird perspectives and beliefs in LDS doctrine, both official and cultural. Their whole point-of-reference has been skewed by the cognitive dissonance necessary to find Smith's original tale of being visited by an angel who was once an ancient Israelite living in the Americas in the pre-Columbian period, who happened to speak this previously unknown language which no one has seen or read outside of Smith himself.
Sure, it's hogwash, and Smith was a scam artist. However, the cultural movement that became the Mormon church was already present and growing, and it's certainly reasonable to assume that had Smith not come along with his story, someone else would likely have come along and formed their own millennialist sect. In fact, others have done just that: the Jehovah's Witnesses are another millenialist sect with similar roots, as are (I believe) the Seventh-Day Adventists.