Agreed.
This is a misunderstanding of acceleration and velocity. I am also not claiming that the downward force is less than the upwards force (even though it actually was as it constantly accelerated, but this is a different concept).
Based on your first paragraph, we both agree that the lower floors would have and should have exerted an upward force on the falling mass. We are in 100 percent agreement on this.
Here's where the issue is. As the falling mass impacted the lower floors, the lower floors would provide resistance to the fall during each impact. It does not matter how great the resistance is, and it doesn't matter which object exerted the greater force. What matters is that whatever the magnitude of this resistance was, it was in the opposite direction of the force exerted by the falling upper floors. Newton's third law says this.
The force exerted by each floor during each impact would not be constant. The force would be temporary, and since it was in the opposite direction, it would cause the floors to decelerate. The magnitude of the deceleration is insignificant. What matters is that any temporary force in the opposite direction of the falling floors is going to change the acceleration of the floors. If there are any temporary changes in acceleration, the acceleration, by definition, can not be constant.
The floors should have exerted an opposite and equal force on the falling mass. This means that the deceleration could not be constant, since the resistance was temporary and only felt with each floor impact. Since acceleration was constant, this means there were no impacts with lower floors. This means the floors were removed. The only conclusion from this is obvious.