Masonry with fitted stones is nothing high-tech, new, or impossible. Pre-columbian Americans did it too. You rough the rock to shape, try it, then refine the fit, try it again, then refine it some more.
Yes I know, these are also puzzling, however they are generally smaller stones. Which would be considerably easier.
I am a cabinetmaker and fit pieces of wood together accurately. It becomes increasingly difficult the larger the pieces are to a dramatic extent. I would say exponential at least.
In my opinion to use the refine and fit method you mention it would become extremely difficult with stones over 2 or 3 tonnes, even with an unlimited labour force and highly skilled craftsmen.
The Egyptian stuff you're talking about, I presume was using rocks cut to very regular shapes (like large bricks).
I was looking at stones which were irregular shapes, often there would be a corner along an edge. One stone in particular near the current entrance to the pyramid of Cheops was over 20 feet long 10 feet wide and 3ft deep.
This would weight over 40 tons.
I'm no authority, but I think the notion that the stones were always "perfectly" fitted is a myth. (I'm not even sure of how you'd define a "perfect" fit compared to one that is simply very good.)
To me a perfect fit on something this size would be no gaps over 1 or 1.5 millimeters.
On many of the blocks approximately over twenty tons in the sphinx temple are fitted far more accurately. I could see no perceptable gaps, I would say over .02mm wide. You certainly wouldn't be able to push a razor blade between them. Yet the sphinz temple is the oldest building on the site.
I am not claiming any extra terrestrial involvement here, my point is that they had technology capable of constructing buildings out of rock which we would be hard pressed to reproduce today, with the most advanced technology at our disposal.
On one of the oldest buildings in the world.