That is probably a fair comment. But in the future, who knows what better, foolproof technology might come along to track innocent citizens? In the meantime however everyone cheerfully carries mobile phones around with them, usually without thinking twice that they could be tracked - and those things have tracking ranges of kilometres. Sure, you can turn your phone off. You could also leave your ID card at home. Unless you planned to use it, in which case the location of that usage could just as easily be logged when the card is verified.
I just fail to see the special difficulty that RFID chips present, versus all the other things we do on a daily basis, unless you start invoking improbable reading scenarios, or future technology which may or may not materialise, and which in any case isn't available for the foreseeable future.
Edit: To be a bit more on-topic, I generally feel uncomfortable about ID cards on the whole. It has a bit too much a of "Show me your papers!" feel to it. But in the USA where people often seem to have state issued ID cards, it seems to make sense to standardise them for the sort of reasons that FullFlavorMenthol gave.
RFID isn't even effective as an anti-counterfeit measure. Check Bruce Schneier's blog about the RFID technology in ID's and bus passes that has been cloned successfully, in seconds, by thieves.
