I have no idea what that means (especially what a "world of givens" is and what it means for a statistician to live in one). Maybe there is some sort of evidence that is outside the realm of statistics. My point is that there is evidence that is in the realm of statistics that your preoccupation with philosophy has confused you about.
In this case, philosophy has a better grasp on reality than the math. At least in the abstract way you seem keen on using the math.
There are theorems in statistics that show that the larger the sample size, the more information the sample provides about the parameter it is estimating. If you doubt this, then you're confused, and I think we can trace your confusion to your preoccupation with philosophy.
Oh, let's try another story. Why not? They are ever so much fun.
Two ancient scientists are asked by the King to investigate the properties of water. Specifically, the King suggests that water is cold - for some understood meaning of cold, perhaps "causes numbness when a finger is placed in it."
Scientist One walks a half mile to a mountain fed stream and tests the water. It is cold. He tells the King about the evidence he has found.
Scientist Two travels several days away to an oasis he has heard about and finds the water there to be tepid, if not actually warm. He returns and reports to the King as well.
"Ah," says the King, "We have evidence both for and against my hypothesis. What are we to do?"
"Simple," says Scientist One, "I shall return to the stream and draw another sample. In fact, I shall return every day for a week and have seven tests, one each day. For, I have been told by a statistician of merit that resampling counts as new data and provides further evidence."
"Wait," says Scientist Two. "I could do the same, but I have to travel too far. I can never keep up with the number of samples Scientist One can accumulate."
"Tough bananas," says the King. "I don't need you to keep checking anyhow, I can just keep accumulating new evidence from the local stream until we are all convinced by the mounting evidence in favor of my hypothesis. Water is cold, and that's that."
"But," says Scientist Two, "If you are just resampling the same data set, you are going to keep getting the same information over and over again."
Turning to the guard, the King says, "Put him in the dungeon. I suspect he is no scientist at all, but a dreaded philosopher!"