This is indeed a contradiction, and I'd love to see what cj has to say about it.
The limitations of the Bible are limitations of language. It's for Christians to either choose to treat the Bible as being a perfect expression of God's word, without fault and complete - or to recognise, along with mainstream Christianity for two thousand years, that there's a necessary imperfection even in the fact of being written words, and that texts need to be interpreted in this light - that treating a moral guide as if it were a physics/history textbook is a gross misuse.
I can well see that even viewed in this light, many atheists will find fault with the Bible, with Christianity and with religious belief as a whole. However, it seems that this is not enough - a subgroup of atheists continue to regard the bible as inerrant. I suspect that it is based on their path to atheism. Back when Joe Soap the atheist was a Christian, he was repeatedly told that the Bible was the basis of Christianity, that it was perfect and accurate in every detail. When he came to see that this was false, he rejected Christianity and became a happy atheist. However, if he accepted that the Bible wasn't inerrant, then the very basis for his lack of faith would be overthrown. Joe should relax, and accept that there's a wide range of Christian belief for him to reject. He can discard the whole thing.
As for the Bible being the sole basis for Christianity - imagine a Gentleman's club, that's existed for 200 years. They have an imposing headquarters, membership lists, all sorts of traditions and rules - and then a non-member pokes his head in the door and shouts that the whole thing is based on the 1954 rulebook, and without that, the club couldn't even exist.