I heard it put this way by a biographer of Newton, and I have to agree: Can you imagine being person X? For example, he opined that he could imagine being Einstein. While he happily admits that Einstein is a lot smarter than him, one can sort of 'project' how it would be to be Einstein. The same with people like Darwin. But Newton? He is just at another level entirely - Newton is not me x2, or x8, or xanything. Sure, I could imagine, maybe, getting his results in optics. But then to invent the Calculus, figure out the laws of motion, resolve fundamental issues about gravity and the motion of the planets, and then knock off the Principia in my spare time? Never. Humans aren't capable of such things.
And, of course, that was only a small part of his output, as others have mentioned. He did tons of stuff in alchemy (to no end, of course), wrote something like 20 books worth of history and biblical scholarship, was in politics, became master of the Mint, led the Royal society for years, etc. I'd be friggin proud of myself if I made dogcatcher one day.
eta: another way to put it. Suppose Newton had only did his optics work, or only mechanics, or what have you, we'd still be putting him on this list.