Merged Discussion of femr's video data analysis

[tfk]

Where is the camera position? Distance from the tower? What angle is the view from the location to the top of the WTC?

The setup has to be specified or the data is worthless.

You can actually get around that if you know the dimensions of various features in your image. And that building is such a metal & glass Cartesian coordinate system, that it'd be pretty easy to be pretty darn accurate.

For example, if you know the actual window to window index height (12' 9" or whatever it might really be), then you can measure it on the image.

You can't assume that all scaling factors are constant, but if you get several points, you can generate a (usually linear) transform between pixels & height.

For example, if you measured the pixel location for the top of, say every 2nd window as near to as possible the fall line that you'll be measuring, then you'll get a good transform between pixels & height.

Even for way off-axis camera shots, like the Camera 3 video.

If you enter the vertical scale factors for vertical lines along the left edge, the center & the right edge, (along with their respective x- coordinates), you'd have a 2 dimensional transform matrix that would allow you to go pretty easily from pixels -> absolute position, velocity or acceleration anywhere on the image.

You gotta know actual dimensions on the building, tho.

tom

 

[tfk]

Again, careful Tom. You've generated *an* acceleration function. Mine is slightly different...

I think that going from where we were to where you appear to be now is a leap premature.

We should have spent at least a couple of weeks looking at various smoothing and noise reduction methods.

On that basis, I'm treating your current acceleration curve as *tfk acc #1*, not *the* accel function.


What time interval does each row span ? 0.1s ?


Obviously results from my accel curve differ to this. (with over 1s averaging over-G)

We will have to thrash the procedures around until we both converge on similar results.


Would be a tad useful if your processing steps from position/time data are described in detail, along with inclusion of resultant data. Quite happy to do the same of course.

You are where I was a week ago.

You aren't gonna get there from here with a single polynomial function of any degree.

That approach fights itself. You add more degrees to the polynomial to try to capture more detail, but it just adds more non-existent artifacts when you do the differentiations. You'll generate wildly oscillating curve that tries to pass itself thru all (or most of) the data points. This is the opposite of what low-pass filtering does. What you'll find is that the acceleration calculation becomes way, way too sensitive to slight changes in your polynomial. And it becomes almost impossible to tame.

The right way to do it with discrete data is to use the discrete interpolation function. This fits a relatively low order polynomial (say 5th to 7th order) to successive sets of data points, making sure to match the absolute value, and 1st (and for spline fits) 2nd derivatives at the blend points between the polynomials.

This techniques works well, and can be tamed. Plus, it allows the function to follow the actual data thru zones where different effects are predominant. The single high order polynomial technique that you are using doesn't follow the displacement curve into the "pre-collapse" zone at all. The interpolation function approach handles this easily.

I've run thru sufficient permutations of filtering to feel comfortable that the curves I've posted will be very close to best practice.

You want to use the minimum amount of filtering that tames the noise, while preserving as much frequency response as possible.

I've worked my way down from lots of filtering, and worked my way up from very little filtering to find out where there is a pretty good compromise.

And I also found out that the results did, in fact, converge. I wasn't sure at first that they would.


tom

 

[femr2]

You are where I was a week ago.

Nah. Have been looking at trace data, and the various ways of treating the noise for many months.

You aren't gonna get there from here with a single polynomial function of any degree.

The order 50 poly behaves very well, and results in very similar curve shape to one done with more simplistic sequential symmetric differencing and moving average smoothing.

That approach fights itself. You add more degrees to the polynomial to try to capture more detail, but it just adds more non-existent artifacts when you do the differentiations. You'll generate wildly oscillating curve that tries to pass itself thru all (or most of) the data points.

The resultant curves are not wildly oscillating at all Tom...

Velocity...

Acceleration...

I included the huge position/time image to show clearly how close the poly fit was to the underlying data. It's pretty good. Again it would be useful if you could provide the data for your derived velocity and acceleration plots so a subjective comparison can be made. A picture without the underlying numbers is not much use at this point.

This is the opposite of what low-pass filtering does. What you'll find is that the acceleration calculation becomes way, way too sensitive to slight changes in your polynomial. And it becomes almost impossible to tame.

That's not what I find though Tom. I find velocity and acceleration curves which are very similar with all of the smoothing methods I'm using, and also similar results between the Dan Rather data and the NIST Cam #3 data.

From simple observation of your graph it looks like it's showing it's low-order basis quite clearly. I'm sure you would agree that that behaviour is very unlikely to be a true representation of the actual event. Our graphs are not *that* different, but the oscillation in mine is certainly less *wild*.

The right way to do it with discrete data is to use the discrete interpolation function.

Again, I don't think it's appropriate to make such conclusions on the *right* way to this. There are literally hundreds of methods available to smooth data, and without looking at a wider range (here) there's no subjective right way yet. Most of this thread should really be dedicated to developing and justifying the variable *correctness* of various techniques, not jumping to *this is the way*.

It's easy to generate velocity and accel graphs with simple averaging techniques, though they contain more noise than the smoothed results. If the output of a set of smoothed results don't *fit* the average based derivations, then I'm inclined to bin that curve.

The single high order polynomial technique that you are using doesn't follow the displacement curve into the "pre-collapse" zone at all.

The Dan Rather data has a rather annoying pre-release *wobble*, but I've generated the derived graphs from the peak immediately pre-release, so it's not affecting the curves. I agree that poly-fitting has problems with the very earliest moments.

I've run thru sufficient permutations of filtering to feel comfortable that the curves I've posted will be very close to best practice.

Me too, though for mine. Oh, what to do...

You want to use the minimum amount of filtering that tames the noise, while preserving as much frequency response as possible.

Absolutely, which is why I've compared symmetric differencing and high order polys. Both result in a similarly shaped acceleration curve, without the obvious low-order oscillations you're resulting with. I've looked at numerous other methods too and the next step is to split the curve into overlapping segments and redo them all again. If the end result is yet again similar to my current accel curve I'm going to tend towards being a bit more rigid on that.

I've worked my way down from lots of filtering, and worked my way up from very little filtering to find out where there is a pretty good compromise.

I guess we'll have to look at ways of qualifying each procedure eh.

I'll gather a few methods' velocity and accel data together, and suggest you do likewise.

 

[tfk]

femr,

In your excel spreadsheet, you've got > 2:1 difference in horizontal vs. vertical scaling factors. (3.44 ft/pixel vertical / 1.63 ft/pixel horizontal).

This has got something to have something to do with interlacing, right? Such that the effective vertical scaling becomes 1.72 ft/pixel after interlacing?

You can't possibly have a 2:1 aspect ration in the finished video.

tom

 

[femr2]

This has got / to have something to do with interlacing, right?

Correct. Unfolded interlaced video results in two half-height images per interlaced frame.

 

[jaydeehess]

As Tom requested, please refrain from these more generalised kind of thought trains on this particular thread.

Difficult as that is,
I'll do so.

No, the error margin is significantly amplified through derivation of acceleration data from noisy position/time data. What I hope results from this thread more than anything else is clear progression towards methods which deal effectively and accurately with significantly reducing the noise inherent in the raw data, making derived velocity and acceleration much more accurate.

What good is the whole exercise if the error margin is increased? Goes to tfk's senario #1 above and my first questioning the ffa that Chandler's own hacking about found. FFA in a first order approximation would indicate a margin of error at least as large as the difference between the acelleration and 'g'. I understand that there are situations that can have parts of an object moving with ffa but that is not being dealt with in any of this handling of the data. Istead we are simply being told, with greater detail, the same thing that Chandler said, that there was faster than free fall. The only difference is that Chandler assumed that ffa was impossible and that by waving his hands he could claim that it indicated at least free fall.
BUT
How then can you or tkf now say that, yes, the NW corner did experience a short period of ffa if you do not quantify the margin of error? Why can the margin or error not be large enough to have the acelleration of the NW corner never exceed 'g'?

Perspective has not been taken account of, though it probably will be. It's omission does of course affect the *accuracy* of the data, though from the Dan Rather viewpoint I would suggest the effect is quite small. Modifiers would lekely be linear, so the *shape* of derived data should not be seriously affected.

I should have said 'precision' rather than 'accuracy' perhaps?

A quick calc indicates that at 500 meters distance between camera and wall point that a difference of a one meter depth (to/from camera)would create a 0.002 meter error in height(two centimeters, less than one inch) measurement so I suspect you will be right. Though how a slight change in the angle of the object will affect the depth of colour of a pixel and thus affect the SE handling of that pixel is also of interest.
If a line of pixels changes colour due to tilting away from the camera might not SE assume that that point has moved downward more than it actually has?
You might be able to track the difference between two rows of windows to try to take this into account, if that distance changes it indicates a tilting, except it could also indicate a crushing or deformation of the floor.

Can you understand that my original question about the margin of error has simply not been answered yet, nor, from, my perspective, does it seem that anyone with the wherewithal to quantify it , will be getting to that point soon?

 

[tfk]

Let me try to phrase it in terms of "degrees of freedom".

When you use a single polynomial to try to describe the entire curve, small adjustments in one location (to try to get a better fit) will produce changes all along the curve. The algorithm is trying to get a "best fit", but every term has an effect everywhere along the curve.

When you use an interpolation function, the algorithm only worries about fitting the curve locally. Adjustments to get a better fit in one area have no effect on the curve fit anywhere else along the curve. You have in essence relaxed a boatload of constraints. You've got lots more degrees of freedom to adjust your constants to fit the LOCAL data points.

Which means that you can do so with a relatively low order polynomial.

I am also saying that you cannot quantify "by eye" the quality of fit 2 derivatives away. Saying that it fits the displacement "pretty darn good" does not give a valid metric for the local accelerations.

Which is what this exercise is all about.

After all, the results that you are getting is a step up from Chandler's - where he FORCED the acceleration model to be a constant. A single horizontal line. He did this by creating a model with rigid constraint: it had to fit a linear velocity profile.

You've relaxed some constraints to allow your curve to follow the actual data. But you've still got a bunch of constraints jerking your curve around.

We'll see when we get there.


tom

 

[femr2]

What good is the whole exercise if the error margin is increased?

There's no way to avoid amplification of raw data error when performing first and second order derivations of that data I'm afraid. It's a mathematical certainly. W.D. Clinger made a good post a while back providing good detail on the point. I'll dig it out.

What we can do, if we can agree an amount of error for the position/time data, is generate a good estimation of error magnitude at first and second order derivation levels.

Why can the margin or error not be large enough to have the acelleration of the NW corner never exceed 'g'?

There has to be significant error in the derived data to make significant changes to the derived accel data, but it's a reasonable question. Suggest step 1 is to agree the noise level in the position/time data. I'm comfortable with +/- 0.2 pixels.

Can you understand that my original question about the margin of error has simply not been answered yet

Yep.

, nor, from, my perspective, does it seem that anyone with the wherewithal to quantify it , will be getting to that point soon?

I'm sure suitable estimations will be provided in time.

 

[jaydeehess]

femr,

In your excel spreadsheet, you've got > 2:1 difference in horizontal vs. vertical scaling factors. (3.44 ft/pixel vertical / 1.63 ft/pixel horizontal).

This has got something to have something to do with interlacing, right? Such that the effective vertical scaling becomes 1.72 ft/pixel after interlacing?

You can't possibly have a 2:1 aspect ration in the finished video.

tom

Correct. Unfolded interlaced video results in two half-height images per interlaced frame.

Aspect ratio should be 4:3 in the interlaced video should it not? With 1.63 feet per pixel horz, that would give a vert of 2.17 feet per pixel then.

Each feild then is a line of pixels 2.17 feet per pixel but only half as many lines.
There is half as much information per feild.
Properly each feild is a same height picture but with blank lines (no information)between each row. feild one begins at top left of screen while the second feild begins one line down from top left of screen, and 1/59.94 of a second later in time.

 

[femr2]

When you use a single polynomial to try to describe the entire curve, small adjustments in one location (to try to get a better fit) will produce changes all along the curve. The algorithm is trying to get a "best fit", but every term has an effect everywhere along the curve.

Aiii.

When you use an interpolation function, the algorithm only worries about fitting the curve locally.

Sure, though of course you're at the mercy of the particular decimated points used, and the data inbetween is effectively ignored.

Adjustments to get a better fit in one area have no effect on the curve fit anywhere else along the curve.

Adjustments ? Manually ?

You have in essence relaxed a boatload of constraints. You've got lots more degrees of freedom to adjust your constants to fit the LOCAL data points.

Which means that you can do so with a relatively low order polynomial.

Yep, though I'd be interested with seeing the results of a range of decimation intervals compared.

I am also saying that you cannot quantify "by eye" the quality of fit 2 derivatives away. Saying that it fits the displacement "pretty darn good" does not give a valid metric for the local accelerations.

Of course, though it would be perhaps useful to compare R2 values between my order 50 poly and your interpolation functions. Will have to dig it out, but it's above 0.999999 if my memory serves.

Which is what this exercise is all about.

Indeed. Hopefully more than just us-two will throw methods into the pot for comparison.

We'll see when we get there.

Cool.

 

[femr2]

Aspect ratio should be 4:3 in the interlaced video should it not?

Depends. The metrics provided are relative to known building measurements horizontally and vertically, and so are independant of aspect ratio. Accounting for camera perspective is another kettle of fish of course.

Properly each feild is a same height picture but with blank lines (no information)between each row. feild one begins at top left of screen while the second feild begins one line down from top left of screen, and 1/59.94 of a second later in time.

In terms of live projection, kinda true, yes, however in this context the scaling methods employed are fully appropriate.

 

[tfk]

femr,

This is the key:

I agree that poly-fitting has problems with the very earliest moments.

This is exactly right. It cannot follow the discontinuity (going backwards in time) from "falling" to "not falling".

But, guess what...

It is not only at the earliest moments that the polynomial has a problem. It is anywhere that there is a discontinuity in the forces applied. The polynomial can't easily follow local discontinuities, because it is nailed to the entire curve.

ETA: Can't follow it as easily as an interpolation function.

From simple observation of your graph it looks like it's showing it's low-order basis quite clearly. I'm sure you would agree that that behaviour is very unlikely to be a true representation of the actual event.

First off, I'm not sure of any such thing. I don't have any expectation that this is unlikely behavior.

Do you have a basis for this assertion?

This is expressing a bias in what you want to see. It doesn't matter what we think is right or wrong or expected. What matters is "what does the data show?"

And it is tempting, but poor technique, to fiddle with parameters until we see what we want.

That's not what I find though Tom. I find velocity and acceleration curves which are very similar with all of the smoothing methods I'm using, and also similar results between the Dan Rather data and the NIST Cam #3 data.

Similar results with the other Cameras is a VERY good thing. One would say, "required".

Now let's see if it's real, or merely an artifact of our technique.

Our graphs are not *that* different, but the oscillation in mine is certainly less *wild*.

Wild does not equal right or wrong.
Neither does tame.

Again, I don't think it's appropriate to make such conclusions on the *right* way to this.

I am not drawing conclusions yet. I've got (surprise) strong feelings for where this is headed.

I'll gladly go where ever the analysis takes us.

The Dan Rather data has a rather annoying pre-release *wobble*, but I've generated the derived graphs from the peak immediately pre-release, so it's not affecting the curves.

That's not "an annoying wobble". It's a real motion of the building.


tom

 

[tfk]

femr,

What does "Aiii" mean?

 

[femr2]

It is not only at the earliest moments that the polynomial has a problem. It is anywhere that there is a discontinuity in the forces applied. The polynomial can't easily follow local discontinuities, because it is nailed to the entire curve.

I think we're dealing with diminishing variance as the order increases, but sure. There's a different set of problems with interpolated curves though, and again I suggest a good way to quantify is to look at the actual poly-v-interpol data itself and compare it to the base position/time data. I have a few graphs ready but will upload them at a more appropriate juncture.

First off, I'm not sure of any such thing. I don't have any expectation that this is unlikely behavior.

Do you have a basis for this assertion?

Yes. The correlation between the shape of the curve for the high-order poly function and the simple averaging/differencing curve shape. Very similar. Again I have some graphs but will do some prep before uploading.

This is expressing a bias in what you want to see. It doesn't matter what we think is right or wrong or expected. What matters is "what does the data show?"

Not at all. I have no reason to *want* to see anything at all in the accel curve. Near-g, at-g, above-g, whatever. Makes little difference to me to be honest. I'm after a kickin' way of reducing the noise. Preferably a way which makes my poor old PC emit steam for a while whilst it works out the absolute bestest approximation curves.

And it is tempting, but poor technique, to fiddle with parameters until we see what we want.

Absolutely. Though I do think that trying many different methods and comparing the results is a good thing to do.

Similar results with the other Cameras is a VERY good thing. One would say, "required".

Agreed.

Now let's see if it's real, or merely an artifact of our technique.

Okay.

I'll gladly go where ever the analysis takes us.

Okay.

That's not "an annoying wobble". It's a real motion of the building.

Hmm. Are you sure ? Tis not the same in the Cam#3 data. More likely a video artefact at the beginning of movement. As I said earlier, I moved on from the Dan Rather data to the Cam#3 data as it is of justifiably higher quality.

ETA: Aiii = Yes.

 

[Major_Tom]

tfk, in the OP you mention your problem with femr's claim to sub-pixel resolution.

Have you changed your mind or do you still have a problem with that?

Has femr explained it well enough?

 

[pgimeno]

Aspect ratio should be 4:3 in the interlaced video should it not? With 1.63 feet per pixel horz, that would give a vert of 2.17 feet per pixel then.

Image aspect ratio is not to be confused with pixel aspect ratio. A 4:3 picture usually has 4/3 times more horizontal pixels than vertical (e.g. 640x480, 1024x768), so that the pixel aspect ratio is 1:1, i.e. the pixels are square. For DVDs that's not usually the case, though: the image is usually 720x480 projected in 4:3 format (ETA: i.e. it has 1.5 times more pixels horizontally than vertically, not 1.333), meaning an 8:9 pixel aspect ratio, or 704x480, meaning 10:11 pixel aspect ratio. For example, 1.63 ft/hpx translates to 1.83 ft/vpx with 8:9 pixel aspect ratio. See http://en.wikipedia.org/wiki/DVD_Video#Frame_size_and_frame_rate for common DVD video formats.

Deinterlaced images, on the other hand, double the vertical part of the aspect ratio. In that case, the pixel aspect ratio is 8:18 for originally 8:9 pixels, or 10:22 for originally 10:11 pixels.

Anyway, as femr2 said, in this case he calibrates both dimensions separately, which should allow for better accuracy provided the real world measures are accurate. I am thinking that that would be a good way to estimate how accurate they actually are. I don't think perspective distortion is playing a significant role in the Dan Rather video, so a comparison of the theoretical and practical aspect ratios of the vertical vs. horizontal ft/px can give information about how accurate they are (unless they both are biased by the same proportion, which I doubt).

E.g. if the image is 720x480 (I haven't looked at the actual resolution yet) and the horizontal ft/px ratio is 1.63 ft/px, how close to 1.83 ft/px are the vertical ft/px?

I think that can help in establishing figures for the errors in the measurements.

 

[femr2]

Image aspect ratio

Always a minefield.

Here is some good lower-level technical information on digital video aspect ratio...
http://www.iki.fi/znark/video/conversion/

a comparison of the theoretical and practical aspect ratios of the vertical vs. horizontal ft/px can give information about how accurate they are (unless they both are biased by the same proportion, which I doubt).

I can post some numbers, but there are so many variables between the original source equipment and the resultant mpeg-2 file format video instance I'm using that it'll be impossible to say what is correct. Anyway...

Output Video resolution (my copy of the video) - 704*480 (dvd)

Assuming the original camera adhered to standards (ITU-R BT.601)...

Pixel Aspect ratio (x/y) - 4320/4739

Active picture size - 710.85 * 486


The first issue there is that one would expect a 704 pixel width post-convert format from a 625 line system, and so a 576 pixel vertical output. Instead our output video has a 480 pixel vertical output, which corresponds to a 525 line system and would therefore be expected to have a 720 pixel output horizontal resolution (so that the full 710.85 pixels of the actual active picture size can be captured).

I'll have a look at the most probable convertion procedure if it's of enough interest. i.e. asked for by a few folk.

There's an effective 16 pixel (10+6) black border on the output video, which further complicates determining the conversion process, and it looks like there was a slight pixel clock timing issue between interlace fields (as there is noticable combing of vertical features).

Then we'd have to decide if the video container was intended for 4:3 or 16:9 output...

I suggest folk simply check the actual distance measurement-to-pixel values provided.

For example, the 100 pixel mesurement provided for vertical could be 100 +/- 1/8 (or even higher), but I've kept the value to integer pixels to indicate that I'm not suggesting sub-pixel accuracy.


(A single interlaced frame unfolded - click for actual size)

Minefield.


Quick additional note whilst I remember it's there...59.94 Hz is only a conventional approximation; the mathematically exact field rate is 60 Hz * 1000/1001.

 

[pgimeno]

Always a minefield.

You've convinced me.

I'll have a look at the most probable convertion procedure if it's of enough interest. i.e. asked for by a few folk.

I'm not interested any longer. There are many more hidden variables than I expected. It would probably be futile for a determination of error.

 

[tfk]

tfk, in the OP you mention your problem with femr's claim to sub-pixel resolution.

Have you changed your mind or do you still have a problem with that?

Has femr explained it well enough?

What's your hurry? You got a stagecoach to catch?

Has he explained it well enough? Hell no.

Perhaps the response of a perfect geometric shape, against a perfectly white background, with a total of 4 shades of gray, with perfectly balanced geometry (equal, 100% white on all sides of the black dot), represented in a way that incorporates none of the features of how a CCD actually gathers & process light info convinces you...

As usual, I've got my suspicions. But we've still got a few things to sort out.

 

[tfk]

Always a minefield.

Here is some good lower-level technical information on digital video aspect ratio...
http://www.iki.fi/znark/video/conversion/


I can post some numbers, but there are so many variables between the original source equipment and the resultant mpeg-2 file format video instance I'm using that it'll be impossible to say what is correct. Anyway...

Output Video resolution (my copy of the video) - 704*480 (dvd)

Assuming the original camera adhered to standards (ITU-R BT.601)...

Pixel Aspect ratio (x/y) - 4320/4739

Active picture size - 710.85 * 486


The first issue there is that one would expect a 704 pixel width post-convert format from a 625 line system, and so a 576 pixel vertical output. Instead our output video has a 480 pixel vertical output, which corresponds to a 525 line system and would therefore be expected to have a 720 pixel output horizontal resolution (so that the full 710.85 pixels of the actual active picture size can be captured).

I'll have a look at the most probable convertion procedure if it's of enough interest. i.e. asked for by a few folk.

There's an effective 16 pixel (10+6) black border on the output video, which further complicates determining the conversion process, and it looks like there was a slight pixel clock timing issue between interlace fields (as there is noticable combing of vertical features).

Then we'd have to decide if the video container was intended for 4:3 or 16:9 output...

I suggest folk simply check the actual distance measurement-to-pixel values provided.

For example, the 100 pixel mesurement provided for vertical could be 100 +/- 1/8 (or even higher), but I've kept the value to integer pixels to indicate that I'm not suggesting sub-pixel accuracy.

http://femr2.ucoz.com/_ph/7/125286220.pnghttp://femr2.ucoz.com/_ph/7/2/125286220.jpg
(A single interlaced frame unfolded - click for actual size)

Minefield.


Quick additional note whilst I remember it's there...59.94 Hz is only a conventional approximation; the mathematically exact field rate is 60 Hz * 1000/1001.

Thanks.

That is the most detailed description that I've seen you give of the raw video file.

You are the one who has spend countless hours analyzing these videos. Those questions asked are very sensible questions for someone wanting to follow along with your work.

If you are considering publishing your data, you would be smart to have all of those questions (& any other that anyone could think of - the value of collaboration) readily & conveniently answered in an appendix to your paper.

It is really annoying to read thru a paper, only to find that a bunch of obvious questions are not addressed. Gets you pissed at the authors. And then thinking that he has something to hide. (I'm NOT saying this about you. You've been forthcoming with your data.)

There is a spectrum of rigorous to sloppy. Better to err on the former side.


tom

PS. I'd like to see those questions answered.