Trayvon Martin, Vigilante Justice

[RandFan]

Someone posted this link last night:

http://cs.joensuu.fi/pages/tkinnu/webpage/pdf/speaker_recognition_overview.pdf

It has a pretty nice overview of the field, plus a good bibliography. It's somewhat technical, but pretty interesting.

Thanks.

 

[Skeptic Ginger]

Sorry, I should have been clearer. I understand what the purpose of the software is, which is what it does at a very high level.

I'm not even disputing that it works. But as someone who works in a related field and knows the math involved, it frustrates me that they don't at least say what the models are, and how accurate they are.

There may be good reasons why they do this. But I still find it frustrating.

See my last post above.

 

[totoro]

That might be an ad hom argument, but not an ad hom attack.

Your point is? Since an ad hominem isn't a proper argument, calling it an "atttack" doesn't seem all that unreasonable.

 

[Skeptic Ginger]

I think that's fair. BTW: I found an excellent article, Voiceprint Identification.

It also says:

In the federal court system, we are aware of 30 trials in which the question of aural-spectrographic voice evidence was addressed. All but three admitted the evidence based on Frye or McCormick. On appeal, the Second, Fourth and Sixth Circuits held the evidence admissible, applying McCormick, while the District of Columbia did not, applying Frye. See United States v. Williams, 583 F.2d 1194 (2d Cir.), cert. denied 439 US....

...Several state appellate and supreme courts have admitted the evidence, as have three of four federal appellate courts.

 

[sgtbaker]

Yes because they owe all these people on Internet forums their time.

I'm sure if you were interested in purchasing this software the company would be happy to provide you with whatever evidence you asked for.

As with most software, the program is only as good as the programmer. I think it's very important to understand how the program works to decide if it is, in fact, scientifically reliable. In any other situation where skepticism comes in, we generally don't take the word of the person making the claim on the validity of their claim; we call that circular reasoning. The fact that they are looking for outside validation for this guys expertise is not handwaving. I admit that I haven't read his peer reviewed papers and I don't even know what his claims to expertise really are, which is why I am not arguing for or against him.

Like I said, it's hard to ignore two independent tests coming to the same conclusion but those questioning it have far more experience in the sound field than I. It's likely that they would know the questions to ask, that would never even have dawned on me.

Right now, I see it as really potential damning evidence against Zimmerman but I will reserve full judgement until the special prosecutor comes back with her decision. If she decides to press charges, you can bet you're butt I'll be tevo'ing that case. If she decides not to press charges, I hope she lays out exactly why.

All this arguing over the legitimacy of the tests done by people who, as far as we know, have absolutely nothing to do with the actual investigation is kind of a waste, if you ask me.

 

[totoro]

Yes because they owe all these people on Internet forums their time.

I'm sure if you were interested in purchasing this software the company would be happy to provide you with whatever evidence you asked for.

Maybe they would. Hard to say. Since their target customer base seems to be non-technical, they might not even be prepared to do so if I were.

In my neck of the woods, using fancy machine learning techniques is usually a selling point, not something to keep hidden completely, but they know their business, I suppose. If there are standards that such software is supposed to meet, it's a little surprising to me that this is also not mentioned at all (meets standard blah blah blah). But I come at this stuff from a different angle than their customers.

Given that someone could easily raise these questions in court, I guess that they might have some answer ready though. Hmm, maybe I'll send an email asking. If they reply back, I'll post whatever they send me (they probably won't bother, but you never know).

 

[Skeptic Ginger]

....

All this arguing over the legitimacy of the tests done by people who, as far as we know, have absolutely nothing to do with the actual investigation is kind of a waste, if you ask me.

It is revealing how people think and for me, it helps me refine and examine my own thought processes. For example, my comment, "I don't need to cite evidence X just because you dismiss evidence Y. Evidence Y supports my conclusion", was the result of trying to articulate what was wrong with the arguments dismissing the forensic expert's conclusion.

 

[Skeptic Ginger]

Maybe they would. Hard to say. Since their target customer base seems to be non-technical, they might not even be prepared to do so if I were.

What do you base this on? Who do you think they are marketing this software too, forensics hobbyists?

I have a medical practice and I get catalogues all the time that have sophisticated laboratory analysis equipment for sale, in about the same price range. The catalogue and the web pages are typically as non-technical as this web site for this product is. That doesn't mean the potential customer is "non-technical".

 

[crimresearch]

Your point is? Since an ad hominem isn't a proper argument, calling it an "atttack" doesn't seem all that unreasonable.

Ahhh... but in the life or death world of combat on the skeptic's forum it is the difference between a Cloak Of Invisibility, and a mere potion of blurry vision...

 

[totoro]

What do you base this on? Who do you think they are marketing this software too, forensics hobbyists?

I have a medical practice and I get catalogues all the time that have sophisticated laboratory analysis equipment for sale, in about the same price range. The catalogue and the web pages are typically as non-technical as this web site for this product is. That doesn't mean the potential customer is "non-technical".

Hmm. Maybe. The software and libraries I use tend to have pretty exhaustive technical documentation available freely.

The fact that they don't led me to believe that their target customers aren't interested in such things. Also, I wouldn't necessarily expect forensics people to have graduate degrees in EE, which looks like the background you'd need to write this kind of software (at least from the papers I skimmed). The definition of non-technical user I'm using here is "doesn't know the math underlying the software he/she is using". Given the kinds of high paying jobs people with this kind of background can get in Silicon Valley, I'm guessing that most of the people doing this kind of forensics stuff don't have this background.

I would actually be pretty surprised if the people their videos are aimed at are interested in whether they are using vector quantization, support vector machines, or Gaussian mixture models, how their feature selection works, etc. Of course, I could be wrong. I have been wrong about many things in the past, am certainly wrong about things now, and will be wrong about things in the future .

We're quibbling at this point, since I'm not claiming that this software doesn't work. As you pointed out, they don't owe _me_ anything.

 

[WildCat]

That makes no sense at all. If a DNA test rules out a match between two samples do you think that is indistinguishable from the result if the system doesn't work at all?

If your method is incapable of making a DNA match then you will never get a match either. It doesn't mean there is no match, only that your method is flawed.

And AFAIK it has never been shown that any method of voice analysis can match a voice with a scream. It always requires the voices to repeat the same or similar phrase, and it needs to be of good quality. This clown claims to have done what hasn't ever been done before.

 

[Dancing David]

All this apologizing for an inconvenient piece of new evidence goes beyond skeptical weighing of the evidence.

I did no such thing, these are legitimate sampling questions, that will be brought up if the is evidence is presented at trial.


When we know the phone that made the call and the technique used to store the dispatcher call, then we can talk about what sampling artifacts there might be. But no assumptions about the audio quality can be made at this time.

 

[Dancing David]

^^^ This 100X ^^^! One of the things that makes me instantly skeptical here is that real scientists are extremely unlikely to render an opinion without enough information and it's almost always those that are not practicing good science that are eager to rush in and make pronouncements far too early.

I am willing to be wrong, but I find it highly doubtful that anyone can come to meaningful conclusions based on the recordings we've heard.

There would be a number of ways to check on the technique rather quickly however.

 

[Dancing David]

Thank you. That last video seems like a pretty good refutation of the hand waving going on.

There are legitimate questions to be answered about audio compression and sampling. Did the video discuss samples of normal voice vs, screaming as well?

The way digital audio is handled could create serious issues, the baud rate of the phone, its compression, the transmission if it was VOIP or over a cordless and the digital storage all raise legitimate questions, they may have answers.

ETA: While one test address a bandwidth compression, it does nothing to address the issues of digital transmission compression, sampling over a cel phone, or the storage compression.

Nor does it show that a yelling voice is at all similar to a normal voice.

As stated before these would all be very easy to verify, very quickly, in two days at most. Given knowledge of the phone used, the placement of the phone vs. the yelling, the transmission method and the storage method.

It would really only take two days to run tests, have different people in that apartment complex yell, capture the voice recordings at the site using a similar phone at a similar distance, then run samples vs. their speaking voices. Given say twenty volunteers randomly chosen, a variety of yelling and screaming samples (ten at least per individual), then we could find error bars very quickly.

 

[AvalonXQ]

An example of Tom Owen's views on science and forensics:

Report of the meeting of the SC-03-12 Working Group on Forensic Audio of the SC-03 Subcommittee on the Preservation and Restoration of Audio Recording, held in New York, NY., US., 2009-10-11Guy's obviously a woo pushing flake.

Still not saying anything different than what the polygraph experts say.
Industry credentials and "expertise" do not reliable science make. Reliable techniques are developed for controlled conditions. I've not seen any papers indicating the reliability of determinations with this degree of acoustic distortion. If you know of any, share them; your arguments from authority are of little evidentiary value.

 

[Kevin_Lowe]

If your method is incapable of making a DNA match then you will never get a match either. It doesn't mean there is no match, only that your method is flawed.

That's not what they said though, I thought. I though they said they could rule out a match.

And AFAIK it has never been shown that any method of voice analysis can match a voice with a scream. It always requires the voices to repeat the same or similar phrase, and it needs to be of good quality. This clown claims to have done what hasn't ever been done before.

Can any method of voice analysis rule out a match without them repeating the same or a similar phrase?

Everyday experience indicates to me that it's much, much easier to say for certain that two voices do not match, than it is to say that two voices are definitely the same person.

 

[monoman]

As with most software, the program is only as good as the programmer. I think it's very important to understand how the program works to decide if it is, in fact, scientifically reliable...

We don't need to understand how the program works - the maths will be way above most people's heads. We just need to know that it's been extensively double blind tested with normal voices versus screaming voices over a phone line.

 

[Dancing David]

Moss, do you agree per my post above that double blind tests with the tester being able to correctly make matches are an absolute must if we are to give credibility to this field? How can we trust it with anything less?

Thats why i stated what we have is possibly indicative but not definitive.

 

[RandFan]

I've not seen any papers indicating the reliability of determinations with this degree of acoustic distortion.

This is simply setting a standard based on.... what? What is your expertise to make these assumptions? What is your basis for where the bar gets set? Isn't this a bit like the crockaduck? Perhaps, and I'm spit-balling here, perhaps the expert whose testimony, unlike polygraph experts, was allowed into court actually knows something that you don't. Perhaps he doesn't have to prove to you anything in order to testify in court? I could be wrong, perhaps you are the expert that judges turn to for advice in these matters.

 

[Dancing David]

Care to answer my question? If it's proven to be Martin screaming, is that game over for Zimmerman?

If it can be shown to 95% reliability, that would be damning to say the least. If they can show it is not GZ to the same degree.

It could be a home run.