A few things to consider that might help out, as well. I admit this is overly simplified, others can flesh it out / correct mistakes as needed.
Remember those "Newton's Cradel" desktop toys? the 5 metal balls suspended In a line, and when you pull one away, and let it go, it hit's the line and the ball on the opposite end gets sent away? (and note that it takes practicly NO time for that transfer of energy to take place) That's your conservation of momentum at work. If you pull two of the five balls back, and release them, two go flying off from the other side.
Now, the tricky part - Pull back 3 balls. Since you have to conserve the momentum, when they collide with the other two, those two balls plus 1 of the three that were pulled back continue on. The mass has to be equal for the momentum to be conserved.
What does this have to do with WTC? Well, as the floors above the crash site drop, they have momentum. They impart that momentum onto the floors they crash into, so what you have isn't so much a collapse but a pile-driver breaking the floors below. You have "X" number of floors hitting one floor, some of the energy is lost breaking stuff, and the ball at the end of the line (in this case, the floor at the top of the Stack) loses it's momentum, but the rest of the mass keeps moving, it doesn't "Stop at each floor" like we've seen on some posts.
You don't see the top of the stack flying off at each floor because it's still effected by gravity, and still somewhat connected to the rest of the stack. So continues to be dragged down with the rest of the pile.
Now, your thinking, "wait a minute..It doesn't hit one floor, it hits the entire remaining mass of the tower". Well...yes and no.
Ever see a martial arts demonstration where they break many boards at once? Notice that they don't just stack the boards on top of each other, they have small "Spacers", maybe half an inch thick, between the boards? That's so you don't have to break the strength of all 10 boards (or however many), you just need the force to break 1 board 10 times. Some momentum is lost at each board, but the martial artist is still using his muscles to Compensate.
The WTC Towers are like that - each floor is a board, and are connected to the outside walls, the central colum, and various interior walls and such that act as the spacers do, making it easier for the X-number-of-stories mass to break through each floor. Now instead of trying to break the entire 70-some odd floors remaining, it only needs to do it one floor at a time. While a mathmatical amount of momentum is lost in each colision, Gravity was still accelerating the mass. Also, with the exception of what is ejected from the collision, each floor colided with adds its mass to the force comming down. even if it's pulverized, it still has mass, wich adds to the force applied on the floor below it.
Hope that makes sense, any corrections that need to be made, let me know. like I said, it's somewhat (allright, a lot) simplified. But the idea should be pretty clear.