T'ai Chi : Some math/stat questions for you

[BillHoyt]

Yeah, I can. How long is the impulse response?

Good one, jj! Testing to see if we're awake?

 

[jj]

It's very satisfying, though, how this thread turned around 180 degrees to bite the biter back. I wish I could vote again on it.

Um, Sundog, I wouldn't be quite so quick there.

Tai Chi's questions are more linguistic than mathematical.

Unfortunately, I don't find several of his/her responses encouraging, but I can't say that I can reach anything that I'd be willing to publically say was definitive.

 

[BillHoyt]

Gang,

Most especially sundog and tachump,

Randi's rules of engagement here are simple. If someone makes a claim, you can ask them for evidence about that claim. You can, in fact, fairly pepper them with questions about it until:

o they answer, or
o they state they refuse to answer, or
o they state they believe whatever they claimed despite having no evidence.

This forum isn't to do kids' homework assignments or to keep people busy asking about Ron Fisher or the Central Limit Theorem or anything similar.

You (sunny & tachump & some others) seem to think you can mimic this approach to play some sort of game here. You seem to think the skeptics here can pepper believers with questions and that, therefore, the beleivers can pepper the skeptics back and expect answers. That ain't how it works, kiddies.

Tachump made claims about degrees. He offered to scan in his degrees and challenged me to do the same. I told him he was deep in the midst of the fallacy of argument from authority. After that, I suggested (based on some specious statistical things he said) that he burn his degrees and spank his professors. He then started a bonfire, painted hoggrease on his naked body and started chanting "scan! scan! scan!" I refuse to engage in such arguments from authority. I am, simply, a bouncer at a local strip club.

I started this thread in direct response to Whodini's claims. That is all. I have made no such claims. That is all.

And, that is all.

 

[jj]

Good one, jj! Testing to see if we're awake?

Actually, I'm digesting the 1- gaussian.

It ought to work, of course, but the narrower the frequency sigma, the wider the time sigma....

It is at least arguably optimum in some sense. I'm not sure if a custom-design process would be better as far as rejection, though.

 

[jj]

Actually, I'm digesting the 1- gaussian general idea.

Oh, the statement isn't quite that...

And that singularity e^ (1/ zero) suggests that the transform doesn't fall off very fast...

And of course 1 - e ^ -inf is not exactly zero, either. Not sure why that's a notch filter. e^-inf = 0...

(foolish me, I had read it as the other way around in the exponent of the exp ....)

 

[jj]

JJ,

I have constructed a new type of notch, which essentially takes the form f(w) = 1 - exp[-(1/x)^2], where x = 0 is the location of the notch. This seems to give a nice smooth notch, which can destroy any singular peak without being much wider than the peak. The question is, can you think of any potential problems using such a notch could present?

Thanks,

Dr. Stupid

Ok, first time I misread this as 1 - gaussian... That would be a band-reject filter consisting of an impulse and a guassian. In some senses an optimum solution.

But it's not, it's got the 'x' in the denominator of the thing.

It has a singularity at zero, at least in its derivitive, yes?

And, well,

at x= +-inf (assume you're using bilinear warping here?)

1 - e ( - (1/inf))= 1-1 = 0

at zero
1 - e (- inf) = 1 - 0 = 1.

Remind me how this is a band STOP, ok?

 

[davefoc]

BillHoyt,
I think I understand the logic of what you said.

You are saying that T'ai Chi has made a claim of statistical knowledge.

Under the rules you put forward, you are allowed to challenge T'ai Chi to prove his claim. The technique you chose was to ask him some statistics questions that presumably only an individual knowledgeable of statistical technique could answer.

By doing this though, you were making a kind of claim that you are capable of judging the validity of the answers to these questions and have statistical knowledge yourself.

So, I think, T'ai Chi is applying your rules when he chose to challenge you on your implied claim.

As I understand your summary of Randi's rules, you can at this time refuse to answer which apparently you have done. Still, I think there is some inherent unfairness here. T'ai Chi in good faith took a shot at your questions that challenged his claim, but you have decided not to answer his.

 

[T'ai Chi]

Randi's rules of engagement here are ...

You are questioning my statistical knowledge. There are several possibilities here:

1. you are questioning my statistical knowledge and you know some statistics
2. you are questioning my statistical knowledge and you don't know statistics
3. you are questioning my statistical knowledge because you believe me to be some troll

If your answer is 1., then I can quiz you.
If your answer is 2., then admit it. You obviously hint 2. by your non-responses to the statistical questions I posed to you.
If you answer is 3., then you are arguing over personalities, not the topic of statistical knowledge.

Or, feel free to add another number to the above list of possibilities if I missed anything.

This forum isn't to do kids' homework assignments or to keep people busy asking about Ron Fisher or the Central Limit Theorem or anything similar.

Ahh, there's the thing. If I was asked questions in an area I was familiar with, I wouldn't view it as busy-work, rather it would be easy to answer them. Now, me trying to answer jj's FIR question, for example, would keep me busy for about a month or two, because I don't even know what the basic terms mean, let alone the mathematics and the relationships between terms.

..tachump ..

That's "T'ai Chi". You seem to be spelling it wrong, Billy Ho.

After that, I suggested (based on some specious statistical things he said) that he burn his degrees and spank his professors.

So, exactly how did you determine that the "statistical things" I said were "specious"? This is obviously implying that you have statistical knowledge, Bill, if you are able to diagnose and critique the statistical sayings of others.

Also, besides your implied knowledge of statistics, one can ask questions to someone even if they don't claim anything. Where on Earth did you learn that you can't ask people questions unless they make a claim??

He then started a bonfire, painted hoggrease on his naked body and started chanting "scan! scan! scan!" I refuse to engage in such arguments from authority. I am, simply, a bouncer at a local strip club.

Fine, but are you a bouncer at a local strip club who knows enough statistical theory to critique my statistical knowledge, but not answer my statistical questions to you? Or do you know enough to critique and answer my questions?

That is all. I have made no such claims. That is all.

And, that is all.

It is your right to refuse to answer the questions. I have no problem with that.

 

[Stimpson J. Cat]

jj,

I see you are using a slightly different definition of "even" than I am. I have always understood the term even to mean that f(t) = f(-t), for all t, which implies that it is symmetric about t = 0.

Ahh. Ok. For FIR filters, the number of taps of length (talking in discrete-time here) turns out to be relevant to a number of things. An even-tap filter has a half-sample delay, which can be annoying or very useful, depending Because of the limit conditions at Pi and 0, one or the other may be much more useful in a given application.

I see what you are saying. For an even number of points the 0th point does not correspond to t = 0. With frequency domain stuff we never have to worry about this. Since we can assume periodicity, we let x[0] be the t=0 component. The "middle" component, x[n/2] is assumed to be aliased, meaning that it corresponds to both t = (n*dt)/2 and t = -(n*dt)/2, where dt is the time step.

Are you sure you don't have the terms "symmetric" and "even" mixed up? For example, a function for which f(t-a) = f(a-t) would be symmetric about t = a, but would not be even.

Even number of taps is what I mean. Symmetry means symmetry around the center point of the filter.

Got it. Completely different use of the term "even" than I had in mind.

I have recently had to rewrite all the filtering software we use here in our lab, because the software we were using is not only buggy, but poorly documented, so that we don't know exactly what it is doing. All of our filters are frequency domain, zero phase-shift, zero time-delay, Butterworth filters. There are two issues which have somewhat puzzled me.

Well, butterworth filters have phase shift. I'm assuming since you're talking about sampled data you're taking the magnitude response of the butterworth and using it at zero phase as a filter in a convolve via transform scheme?

Yes, exactly. I am just using the magnitude response of the Butterworth filters. Incidentally, I am only doing this because the filters our old software used were Butterworth filters (that's about all we know about them). Do you know of a better functional form of the magnitude response for frequency domain filtering. I am, of course, concerned with finding the optimal balance between minimizing the artifacts introduced by having the filters too sharp, and maximizing the ability of the filter to get rid of what we don't want, and keep what we do.

Works, but you have a LOT of padding to do.

I know. The time series we are filtering are pretty long, though, so I don't see it as a big issue, other than finding the right amount to use.

1) Zero padding: Since the signals being filtered are not periodic, some wrap-around effects will be present. This problem is usually resolved by padding the end of the filtered signal with zeroes. For time domain filtering with an FIR filter, this is simple. The amount of padding needed is just 1/2 the total length of the FIR filter.

The length you need is N+M-1. Not half, length - 1 of the FIR added to the signal. Some signal will be before zero time, then, too. That's just how it is.

Of course you are right. That is what my program currently does. FOr some reason I was thinking of padding at both ends when I was writing that.

Any idea how to estimate the appropriate padding length for a frequency domain filter, other than the brute-force method of inverse Fourier transforming the frequency response function, and seeing how long it takes it to go to (effectively) zero?

There is no simple answer. You need to figure out how long the impulse response that YOU CARE ABOUT is.

I was thinking in terms of setting some objective criteria, and then deriving a relationship between the filter parameters and the required padding.

Better to custom-design an FIR of the right type.

Unfortunately that is something I don't know how to do.

Yeah, I can. How long is the impulse response?

It may be way way long...

Well, at least that I have tested for. It does not seem to pose a problem unless the notch is very narrow. From a theoretical point of view I guess this makes sense. The total length of the impulse response should be (roughly) inversely proportional to the bandwidth of the narrowest structure in the filter, right?

Again, try using a custom-designed FIR. You have MATLAB handy?

We have it. I can't use it for the filtering, though. I thought about using Matlab to construct the filters, and then using the filter coefficients it produces to do the filtering. I had really hoped to avoid having to use Matlab at all.


Dr. Stupid

 

[davefoc]

T'ai Chi,
How about the Sherlock Holmes question? Were you SH?

 

[Stimpson J. Cat]

jj,

Ok, first time I misread this as 1 - gaussian... That would be a band-reject filter consisting of an impulse and a guassian. In some senses an optimum solution.

But it's not, it's got the 'x' in the denominator of the thing.

It has a singularity at zero, at least in its derivitive, yes?

And, well,

at x= +-inf (assume you're using bilinear warping here?)

1 - e ( - (1/inf))= 1-1 = 0

at zero
1 - e (- inf) = 1 - 0 = 1.

Remind me how this is a band STOP, ok?

My mistake. I wrote all of that from memory. It is not 1 - exp[-(1/x)^2], but exp[-(1/x)^2]. I was misremembering the "1 -" part because what I had originally tried was 1 - Gaussian.

As for the singularity, that is actually kind of interesting. This particular function is infinitely differentiable, and at zero, all of its derivatives are zero. The thing is, it is not analytic. Its complex derivatives are undefined at zero.

That's the neat thing about it. It is about the smoothest function you could want, but its reciprocal goes to infinity faster than any singularity you would ever expect to see in a physical signal. That's why I chose it.


Dr. Stupid

[edited: The normalization factor I refered to was for something else. The above function is zero for x = 0, and 1 for x = +-infinity.]

 

[Sundog]

Gang,
You (sunny & tachump & some others) seem to think you can mimic this approach to play some sort of game here. You seem to think the skeptics here can pepper believers with questions and that, therefore, the beleivers can pepper the skeptics back and expect answers. That ain't how it works, kiddies.

Tachump made claims about degrees. He offered to scan in his degrees and challenged me to do the same. I told him he was deep in the midst of the fallacy of argument from authority. After that, I suggested (based on some specious statistical things he said) that he burn his degrees and spank his professors. He then started a bonfire, painted hoggrease on his naked body and started chanting "scan! scan! scan!" I refuse to engage in such arguments from authority. I am, simply, a bouncer at a local strip club.

This defense falls flat on its face because your questions had nothing to do with the paranormal.

Therefore your assertion is that skeptics here, just because they are the skeptics, can pepper what you call "believers" with questions whether the questions address their beliefs or not.

This is bogus on the face of it. If you can demand T'ai prove his competence in a field, he can certainly question your competence to judge him. To think otherwise means that you are arguing from authority.

 

[BillHoyt]

It is your right to refuse to answer the questions. I have no problem with that.

Ah, goody. Then you are at least part way to understanding the JREF ground rules.

 

[BillHoyt]

This defense falls flat on its face because your questions had nothing to do with the paranormal.[/quote
What are you smoking, sunny? There was nothing in the rules of engagement that restrict "claims" to "claims about the paranormal".

Therefore your assertion is that skeptics here, just because they are the skeptics, can pepper what you call "believers" with questions whether the questions address their beliefs or not.

This is also not what I said. I said JREF had certain ground rules for engagement. This is from a previous post of mine on the subject:

But the forum has certain rules of engagement; the very rules Claus and others play by and the very rules you continue to whine about.

Randi has made these rules clear on several occasions:
o If someone says they believe something and have no evidence to back it up, let it go unchallenged.
o If someone says they believe something and have evidence for it, call on them to marshall that evidence
o If they say they refuse to answer, let it go.

These are excellent rules of engagement. But, Clancie, it is you, not Claus, who won't play by these rules of engagement. We hear this constant whine of yours because you recognize the believer's trap built into these rules:
o You must either admit you have no evidence, or
o You must state your refusal to answer

That is pretty clear. Somebody makes a claim about something. Anything. They can be called upon to marshall evidence for it.

I have made no such claims. Woodini is simply asking questions. Sorry, sunny, but my time is too valuable to let people randomly ask questions and demand answers of me. How does this differ from Woodini's case? He claimed authority. He offered scanned in documents to back it up. (Laughable really.) He then challenged me to scan in my degrees. I claim no degrees. I am simply a bouncer at a local strip club.

It is about time woodini (and you) deal with a simple fact of skepticism: authority doesn't matter. If a janitor wrote here, and was able to back up her claims with proper citations and excellent logic, she would be welcomed here. She would be showered with praise. Probably offers of marriage as well. We skeptics are weird folk. We're driven more by the quest for truth than for scrutiny of degrees.

I am, simply, a bouncer at a local strip club.

 

[Sundog]

This is also not what I said. I said JREF had certain ground rules for engagement. This is from a previous post of mine on the subject:


That is pretty clear. Somebody makes a claim about something. Anything. They can be called upon to marshall evidence for it.

I have made no such claims. Woodini is simply asking questions. Sorry, sunny, but my time is too valuable to let people randomly ask questions and demand answers of me. How does this differ from Woodini's case? He claimed authority. He offered scanned in documents to back it up. (Laughable really.) He then challenged me to scan in my degrees. I claim no degrees. I am simply a bouncer at a local strip club.

It is about time woodini (and you) deal with a simple fact of skepticism: authority doesn't matter. If a janitor wrote here, and was able to back up her claims with proper citations and excellent logic, she would be welcomed here. She would be showered with praise. Probably offers of marriage as well. We skeptics are weird folk. We're driven more by the quest for truth than for scrutiny of degrees.

I am, simply, a bouncer at a local strip club.

Do you honestly think I'll let you get away with that?

You have just proved my point for me.

The "rule" you quote says:

"If someone says they believe something and have evidence for it, call on them to marshall that evidence"

Read that again, Bill. Believe. That absolutely does not translate into a general claim!

You are entitled to question beliefs. NOT any incidental claim a woo woo might make, just because they're on that side of the fence.

Incidentally, I happen to agree with you completely about the degree issue.

And also incidentally, notice that enen though I have shown that it is YOU who misunderstood the rule, I have not resorted to things like asking you what you're smoking... beats me why you think this is proper behavior for a critical thinker

 

[BillHoyt]

Do you honestly think I'll let you get away with that?

You have just proved my point for me.

The "rule" you quote says:

"If someone says they believe something and have evidence for it, call on them to marshall that evidence"

Read that again, Bill. Believe. That absolutely does not translate into a general claim!

You are entitled to question beliefs. NOT any incidental claim a woo woo might make, just because they're on that side of the fence.

Incidentally, I happen to agree with you completely about the degree issue.

Holy cow, sunny. Is your only reference book a dictionary? That's pathetic; another argument about wording and definitions...

 

[Sundog]

Holy cow, sunny. Is your only reference book a dictionary? That's pathetic; another argument about wording and definitions...

You just cannot admit when you're wrong, can you? It's incredible.

Explain to me how questions about statistics are in any possible way interpreted as questions about someone's "beliefs". You are the word twister in this case.

Shall we ask for a judgement call? I think we should.

 

[Sundog]

Holy cow, sunny. Is your only reference book a dictionary? That's pathetic; another argument about wording and definitions...

The chief reason I can't respect you is the ease with which you drip into the gutter.

Accusing a person of having only one reference book is rather an odd position for someone who pooh-poohs the value of degrees.

Calling me pathetic fools no one. Why do you think such words have any place in a logical discussion?

 

[jj]

jj,



My mistake. I wrote all of that from memory. It is not 1 - exp[-(1/x)^2], but exp[-(1/x)^2]. I was misremembering the "1 -" part because what I had originally tried was 1 - Gaussian.

As for the singularity, that is actually kind of interesting. This particular function is infinitely differentiable, and at zero, all of its derivatives are zero. The thing is, it is not analytic. Its complex derivatives are undefined at zero.

...
Dr. Stupid

[edited: The normalization factor I refered to was for something else. The above function is zero for x = 0, and 1 for x = +-infinity.]

Now we're getting somewhere. Yep, that is a bandstop filter. I hadn't actually done the derivitives, but if the first few derivitives are continuous (what I really want to know) that's good.

Consider:

Function has discontinuity: Rolls off at 1/f (square wave)
Function's first derivitive has discontinuity: Rolls off at 1/f^2 (triangle)
and so on....

And so on.

That's not all there is to it, but it sets a "fastest rolloff" limit by seeing where the first discontinuity shows up.

Do you have a matlab access with the signal processing toolbox? You really, really need to look at "remez", "fir1"... try "help signal".

You may find it extremely useful.

No, you won't use MATLAB to filter things. Yes, you can use MATLAB very handily to create coef's. It beats what you're doing for most apps, I think.

 

[T'ai Chi]

Ah, goody. Then you ...

Bill, you ignored my questions. Here they are again:

You are questioning my statistical knowledge (and claiming it is specious). There are several possibilities here:

1. you are questioning my statistical knowledge and you know some statistics
2. you are questioning my statistical knowledge and you don't know statistics
3. you are questioning my statistical knowledge because you believe me to be some troll

Or, feel free to add another number to the above list of possibilities if I missed anything.

Which is it? 1, 2, 3, or "other" ?