My understanding is that it's a question of consent. It goes something like the following.
If I'm drugged at night and hooked up to a concert violinist (you know the analogy), then I have the right to demand to be disconnected even if it kills the violinist.
But if I volunteered to be hooked up to the guy -- if the guy is only in this situation because of my own actions -- then I don't have the right to disconnect and kill him. Since I'm responsible for his position, I don't have the right to abrogate responsibility and let him die.
This viewpoint says that a person can be reasonably expected to care for the children that his actions are responsible for creating. It's the same argument that we use for mandatory child support payments, extended to the notion that the baby is already in need of something (use of the woman's womb) even before birth. And the woman, responsible for the kid being here, has an obligation to allow it.
But a raped woman is not responsible for the baby, and therefore has a right to deny it her womb.