It's curious, of course, that the Bible suggests that Joseph suspected (perhaps because he was not a total idiot) that Mary had been with someone else (Matt. 1:18-19), and he decided to divorce her quietly. But Joseph changed his mind after being given information from an angel that the fetus's father was the Holy Ghost (Matt. 1:20), and not, say, Matthias the pizza delivery boy. Joseph was told that the gender of the child would be male and that his name ought to be Jesus. How Joseph knew this messenger was an angel is not explained, nor that he knew who the Holy Ghost was, nor why he thought that the perp being the Holy Ghost excused the whole business, nor that Joesph had any inkling about something being a wee bit strange about a ghost impregnating a girl. And to top it all off, the information was conveyed in a dream, where most people realize they experience a whole bunch of really weird stuff but they shrug it off when they wake.
Joseph didn't shrug it off. So maybe he was a total idiot after all.