Even if you mean "country" in the sense of our population (as opposed to our government), it's still a slightly awkward thing to say, even though it is basically true. Yes, Christianity is a comfortably solid majority and a very important part of the lives of pretty much all Americans past and present, it's only about three-fourths of all Americans
according to adherents.com. In a sense, one could say that America is a
white nation just as much as it is a Christian nation, as there are even more white Americans than Christian Americans (
80% apparently) and whiteness certainly has played a large role in American history.
Sometimes when people say things like that, of course, they are merely stating statistical facts, but to use those terms has a tendency to gloss over the bigger picture. And of course, sometimes (not always, but sometimes) when people gloss over things like that, they have ulterior motives. One should not assume too negative intentions of people, and one should also tolerate a certain amount of political incorrectness about things, but the phrase is certainly problematic.
(Note: I feel that by comparing the phrase to race I may be committing a semi-Godwin, but it's the only good example that comes to mind.)