The "buildings collapsed at near-freefall speed" is, for me, one of the most frustrating claims of the entire TM. If we look at WTC1, the upper block coming down at 1 g reaches the ground in about 8.56 seconds. If it comes down at 1/2 g, it reaches the ground in a bit more than 12 seconds. So dialing the acceleration due to gravity back by a full fifty percent only adds about 3.5 seconds to the collapse time.
On the other hand, if you want to make the argument in the opening post that the upper block shouldn't accelerate at all, you're saying that the upward force of the building resisting the collapse is exactly cancelling out the downward force of gravity acting on the upper block, for the entire duration of the collapse. And that's simply absurd.