Moiré technique for dummies
Here's an explanation on how NIST got horizontal sub-pixel resolution out of the WTC7 video. That should help understand how they derived their sub-pixel data and why it can't be applied in the other direction.
I know this forum likes animated GIFs so I'll use that...
(Sorry about the gifs, I couldn't find a better way to explain it)
Let's say we have a dark band almost vertical, but with a certain angle, that may move left and right.
But that movement may be subtle when looked from the distance. This is how it might look at a distance, as captured from a camera, cropped to our region of interest:
Barely noticeable movement, isn't it? Well, that's exactly the problem. We can't really appreciate the movement clearly, and the displacement from that perspective is less than one pixel in both directions.
Now let's zoom into what the camera caught:
Yes, it really is the same, just zoomed 24x. Notice that seemingly up-down movement? That is precisely the moiré effect exploited by NIST.
In order to quantify that up-down movement, observe that the 3rd column is basically black, and the 2nd and 4th columns have gradients. The one in the 2nd column goes from dark to light, and vice versa for the 4th column.
Note also that each of the pixels on the 2nd column is either darker than the corresponding one from the 4th column in the same row, or lighter.
And note also that there's a row where they cross: in one row the one in the left is darker than the one in the right, and in the next row it's the opposite.
That's the characteristic that NIST exploited to measure the vertical movement of the moiré pattern.
In the following image that crossing row is marked with a triangle, and the center of the original bar is marked with a cross:
It can clearly be seen how the vertical position of the arrow is related to an horizontal position of the center. Therefore, determining that vertical position will give us information about the horizontal movement. Note also how the horizontal movement all happens within one pixel. That means that the vertical location of the crossing point is giving us sub-pixel information about the horizontal one.
Now, that amplification is bigger the straighter up the line is, as long as it is not perfectly vertical. The technique depends on finding a spot where such a pattern can be found, and that is at only a slight angle with respect to the horizontal or vertical pixels.
The technique has a bunch of assumptions: that the camera CCD is more or less linear if the colors of both sides are not equal, that the façade doesn't flex, and that the variation of angle is insignificant.
How was that applied to WTC7?
Figures C-4 and C-5 of the NIST report are pretty much self-explanatory but just in case let me add an explanation:
That's a frame from the video with a grid and pixel column numbers added. In my example the pixel columns were numbered from 0 to 4.
That's a zoom-in from figure C-4 showing their choice of pixel columns, which were 517-519, with vertical arrows pointing to pixel column 518 and horizontal arrows pointing to the vertical position where the intensity of the pixel at column 517 crossed with the intensity of the pixel at column 519.
Note also how the angle with the vertical was much smaller than the one I've used for explanatory purposes. That means much bigger amplification of the horizontal movement.
It should be clear from the above explanation that NIST found no such horizontal reference line with the same characteristics and the right inclination to amplify the movement in a way that let them apply the same trick.
Yet femr2's tracking program was able to reproduce these results and applies equally to the horizontal and vertical direction.
