Well, I know you weren't asking me (not one of the naysayers), but there IS quite a lot of religious baggage associated with the position... but that's true of the other side, as well (fate, destiny, etc.). My point is that you can't throw out what you can see right in front of your face (the fact that we make arbitrary choices all the time) without much clearer evidence than has ever been presented.
The woo is on the other foot, as far as I'm concerned. It seems to be primarily a rebellion against the "soul" concept... failing to consider that you can throw out all that extra religious baggage without denying what you can see right in front of your nose.