A Jewish Chaplain routinely counsels on matters of faith with non-Jewish Soldiers for example, and that is something that an atheist could not do. At least, not with any credibility. No matter how well educated an atheist might be about religion, a belief that there is no god is a disqualified.
But the Jewish Chaplain in your example does not believe in the aspect of god that is core to the Christian, that of Christ.
He would answer the same as an atheist if asked; "Do you believe the only path to salvation is to accept Jesus as your Lord and savior?"
A solid NO.
What if the troubled asked about the Trinity? Or the Holy Spirit? The two members of the trinity besides the God figure are central to the beliefs of Christians, after all denying the existence of the holy spirit is the core of the so called blasphemy challenge. So if the rabbi is in agreement with the atheist on two thirds of the trinity he's okay but the last bit is the deal breaker?
As this site shows often, atheists are often just as familiar with the contents of holy books as believers, often more so. As Bill Maher discovered while making Religulous, there are plenty of clergy that are already atheists, they are where they are to help people and they don't see their own lack of faith as a barrier to that.
Also, atheists don't have a favorite in the race, they can treat the Catholic as identical to the Protestant as neither is on 'their team' so they can be a less biased, more equitable choice.
As for the trappings of religion, anyone can learn the rituals, there is no magic and an wouldn't be constrained to only being able to do what his own religion allows in that area.
As for lacking credibility, how would that be an issue? Are the current chaplains allowed to run around proselytizing all the time for their religion? If an atheist chaplain kept his beliefs to himself, hell if ALL the chaplains kept their own PERSONAL beliefs out of the business of giving support and comfort to others, how would anyone know which chaplain was the atheist?