• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Pattern recognition

Ed

Philosopher
Joined
Aug 4, 2001
Messages
8,658
I have an application that is as follows:

I have a number of images, consider them to be like a new alphabet. I would like to compare an incoming image to the stored ones and create a match. It is basically an OCR task and such software would be ok if I could edit the library.

Anyone have any ideas?
 
Isn't this the sort of task a lot of people spent a lot of time designing AI to do in the 80's, with practically no success?
 
Soapy Sam said:
Isn't this the sort of task a lot of people spent a lot of time designing AI to do in the 80's, with practically no success?

Dunno. But if my OCR program can pretty much recognize an "a" why could it not recognize a "***==^^" ?

I am not asking it to learn, I am asking it to match an incoming pattern to one in a library.

I am a bit rusty but I suppose I could program something. Do they still make ALGOL compilers?
 
You could try an open source OCR such as this one and see if you can change the font to your custom alphabet. Ask the developers.
 
I did a lot of research on pattern recognition during my PhD time (about 10 years ago).

C4.5 was always hard to beat and from what I read its succesor See5 is even better.

The major problem though is the extraction of suitable patterns from your set of images (feeding bitmaps in the classifier for training and recognition won't do ypu no good).

Having a go with an open source OCR seems good advice to me.

Zee
 
I dont know what you mean by "learning". This app is sorta different. It does not have to take just any old pattern. For example, one might sort of standardize the size to x by y pixels.

A mark looks like this



mark_thenenin.gif


Now, it might not be this clear. Because of a number of factors it might be less distinct or parts of it may be missing but the size is sorta the size.

Does this change anything?

edit: it might be 1/4" by 1/4" or so. Could be a bit smaller or a bit larger but not much.
 
Ed said:
I dont know what you mean by "learning". This app is sorta different. It does not have to take just any old pattern. For example, one might sort of standardize the size to x by y pixels.

A mark looks like this



mark_thenenin.gif


Now, it might not be this clear. Because of a number of factors it might be less distinct or parts of it may be missing but the size is sorta the size.

Does this change anything?

edit: it might be 1/4" by 1/4" or so. Could be a bit smaller or a bit larger but not much.

In order to give you reasonable advice, we need to know the exact setup of your problem. Is it somewhat like this?

1. you have a set of k different characters like the above.

2. each character is given in the form of a bitmap x pixels width and y pixels height, with x and y being the same for all bitmaps. Each pixel can be one of c different colors.
3. For any given x by y bitmap with c-colored pixels, you want to know to which of your k characters it is the most similar?

4. For each character, do you have exactly one prototype or do you have a number of slightly different examples (written by different persons), but the characters are already correctly classified?

The size of the numbers k, x, y, c and the sizeof your sample set matter a big deal regarding what classification algorithm might suit you.

Zee
 
Originally posted by ZeeGerman
In order to give you reasonable advice, we need to know the exact setup of your problem. Is it somewhat like this?

1. you have a set of k different characters like the above.

Yes. These marks might occur on examples in museum collections or private collections

2. each character is given in the form of a bitmap x pixels width and y pixels height, with x and y being the same for all bitmaps. Each pixel can be one of c different colors.

Actually 0/1 would be sufficient for color. They are stamped in metal, usually steel.

3. For any given x by y bitmap with c-colored pixels, you want to know to which of your k characters it is the most similar?

Yes

4. For each character, do you have exactly one prototype or do you have a number of slightly different examples (written by different persons), but the characters are already correctly classified?

Aye, there's the rub, matey. Since the "quality" of the exemplar figures can vary one might wish to make a composite of some sort or, alternatively, call those variations a "class" and determine the hits/class. I was attempting to simplify by suggesting one example in the library.

The size of the numbers k, x, y, c and the sizeof your sample set matter a big deal regarding what classification algorithm might suit you.

Ultimately the library examples might number in the few thousands. The sample (ie. that which is to be identified) will be one at a time. It will never be a batch operation.

Incidentially, this has been a bee in my bonnet for some time. I believe that various museums would be interested in helping to assemble the database


Zee [/B][/QUOTE]
 
Ed said:
Originally posted by ZeeGerman
In order to give you reasonable advice, we need to know the exact setup of your problem. Is it somewhat like this?

1. you have a set of k different characters like the above.

Yes. These marks might occur on examples in museum collections or private collections

2. each character is given in the form of a bitmap x pixels width and y pixels height, with x and y being the same for all bitmaps. Each pixel can be one of c different colors.

Actually 0/1 would be sufficient for color. They are stamped in metal, usually steel.

3. For any given x by y bitmap with c-colored pixels, you want to know to which of your k characters it is the most similar?

Yes

4. For each character, do you have exactly one prototype or do you have a number of slightly different examples (written by different persons), but the characters are already correctly classified?

Aye, there's the rub, matey. Since the "quality" of the exemplar figures can vary one might wish to make a composite of some sort or, alternatively, call those variations a "class" and determine the hits/class. I was attempting to simplify by suggesting one example in the library.

The size of the numbers k, x, y, c and the sizeof your sample set matter a big deal regarding what classification algorithm might suit you.

Ultimately the library examples might number in the few thousands. The sample (ie. that which is to be identified) will be one at a time. It will never be a batch operation.

Incidentially, this has been a bee in my bonnet for some time. I believe that various museums would be interested in helping to assemble the database


Zee
[/B][/QUOTE]

that's still not clear enough.

How many diefferent characters (i.e. classes) are there? e.g. our alphabeth has 26, the set of digits has ten etc.

Generally, buildinan classifier you do the following:
(lets stick to the alphabeth example, i.e. we have 26 different classes: A,B,C, ..., Z)

First you need a training sample (we call it T) with say relatively uniformly distributed 260 bitmaps, i.e. 10 A, 10 B, 9 C, 11 D, 8 E etc. and these bitmaps are already correctly labelled, i.e. classified.

We assume, that the bitmaps are all of the same size (x by y) and pixels are either black or white and are not randomly rotated.

Second, we need to extract relevant features from the bitmaps in T. The simplest thing would be to regard each bitmap as a bitstring of x*y bits with values 0 or 1. So the feature vector of each training bitmap in T would have x*y two-valued features. Assuming that the bitmaps are e.g. 64*64 pixels, that would give 4096 features. Way to much for the common training algorithms to succeed. What we need to do is to compress the information in the bitmaps in a way that leaves you with 5 to 10 features.

Now you choose a training algorithm e.g. C4.5 or a neural network of some kind and present it the training patterns again and again to give it time to adapt.

For your unknown bitmap you want to classify automatically, you apply the same feature extraction and present the feature vector to the trained classifier and it will give you the class with the highets probaility (according to the information embedded in the training sample).

The trick is really, to identify and extract the right features. once you got those, a decision tree algo like C4.5 will happily calculate a classifier within a couple of seconds or minutes and the classification of a unknown pattern will take no time at all.

Now, I never did OCR but you could try some simple spectral analysis of your bitmaps like a histogram along x yn y. Divide each bitmap in say 4 16 pixel wide columns and 4 16 bit wide rows and count the number of black pixels in each column and row respectively. That would yield a feature vector with 8 features, each of which would take values from 0 to 1024.
Might already work. Or not :D

Sadly, pattern recognition is not a straight forward technique and I spent some happy hours watching a couple of workstations crunching on my problems with different approaches and algorithms.

Zee
 
I recall someone here mentioning a visual search program that is being worked on- ie a file manager able to search for an compare records by what they looked like rather than by attributes or text search.

Anyone remember what or where?
 
Let me try to explain the problem again.

I have an images in my database. Let us call all database images "Index Images". These images have been evaluated and are either identified (as to date, countery of origin and so on) or not.

So, we have our first Index Image:

38e8bb49.jpg


It was found on a sword dated about 1560 and is from Germany. Excellent start. Let us call this image "Index Image 1" or II-1-1.

Now we come across another image... a new one:

mark4.jpg


Cleverly, our algorithm suggests that it matches II-1-1. Upon examination, we agree (see note, below). It is entered as II-1-2. It is, however assigned to the "Class" of Images known as II-1. So, we have now, what we believe, two versions of a mark made in the same shop in Germany in the mid 16th century.

We are rolling.

[note: The process cannot be completely automated since we may have information above and beyond that contained in the image per se. For example, two similar marks may exist where we know with certainty one was made in Spain and the other in Germany. This is a question that can be resolved with some sort of hierarchy. For the moment let us only concern ourselves with unique Classes]

In short order we come up with a new, faint image:

4dcc64f3.jpg


Though faint, our algorithm matches it to class I-1 and assigns to it the identifier II-1-3.

Finally, we examine this one:

mark5.jpg


Hmmmmm...... no gizmo above the "T". Well, we may be given a number of choices for classifying it. Or, we may say "gee, this one is definately from England and must date around 1450" In which case it becomes II-x-1.

Does this description help the process?
 
A trainable OCR program might make a good start, but if you're dealing with makers marks, they could be in the thousands. Besides, OCR is designed to work with blacks of text, not individual characters and sigils.

I think what you're looking for is more along the lines of the AFIS fingerprint matching system, where it looks for commonalities between structures in the images, like a juncture here, this particular shape here, etc.

I'm guessing you're working on a mark identification system where you can feed it an image and have it spit out likely matches. One way to do this would be to develop a consistant descriptive terminology set to describe the marks (just like they have for fingerprints) and have a database of the descriptions for each mark. Then, filteroff the mark to be identified and see what pops up as likely.

Beanbag
 
Beanbag said:
A trainable OCR program might make a good start, but if you're dealing with makers marks, they could be in the thousands. Besides, OCR is designed to work with blacks of text, not individual characters and sigils.

I think what you're looking for is more along the lines of the AFIS fingerprint matching system, where it looks for commonalities between structures in the images, like a juncture here, this particular shape here, etc.

I'm guessing you're working on a mark identification system where you can feed it an image and have it spit out likely matches. One way to do this would be to develop a consistant descriptive terminology set to describe the marks (just like they have for fingerprints) and have a database of the descriptions for each mark. Then, filteroff the mark to be identified and see what pops up as likely.

Beanbag

hmmmm....no images at all, just some number of classifyiers.....very interesting
 
Ed said:
hmmmm....no images at all, just some number of classifyiers.....very interesting

For the example you gave above:

letter: T
star:
6 points
curved rays


Actually, in interesting thought just occurred to me. There already exists a well defined descriptive system for describing shapes and their positions on a field. It's called heraldry (you know, coats of arms, family crests, that stuff). Rather than having to invent a whole new system, you could use heraldic terms to describe the objects and their positions. Colors don't matter in this case.

Of course, heraldry is such an arcane subject with the most peculiar terms, so it might be easier just to make up your own terms.

Just a thought.

Beanbag
 
This sounds less and less like something that an-off-the-shelf classifier could deal with, and more and more like a PhD project.

If this is a professional venture, I'd suggest approaching an academic and proposing funding some research. Not exactly a swift return on the investment, though.
 
The more I think about it, the easier it gets. Simple descriptive system, using text descriptions. The description, plus an image of the mark itself could be stored as a single record in a database file, such as MS Access. You just do successive queries on the features to drill down to the most likely candidates, then look over the images until you find the one you like.

For your example, you query for records with the letter T. Then filter again for a star. If there are too many, then filter for 6 points. That should get you close enough for a visual match.

Beanbag
 
Beanbag said:
For the example you gave above:

letter: T
star:
6 points
curved rays


Actually, in interesting thought just occurred to me. There already exists a well defined descriptive system for describing shapes and their positions on a field. It's called heraldry (you know, coats of arms, family crests, that stuff). Rather than having to invent a whole new system, you could use heraldic terms to describe the objects and their positions. Colors don't matter in this case.

Of course, heraldry is such an arcane subject with the most peculiar terms, so it might be easier just to make up your own terms.

Just a thought.

Beanbag

As with so many other arcane pursuits, I am familiar with Heraldry. It is a good call except that the terminology is quite arcane and one might not expect the people who would use such a tool as we are describing to know it well enough. Also, the terminology is not very precise. That is to say, "star" might have a number of pictoral representations.

For a hoot, I have attached an image of a reference. It is Der Neue Stockel, a standard reference of a couple of thousand pages.
stockel.jpg


I was hoping, eventually, to scan the images and let them speak for themselves, so to speak. I think that you can see that coming up with a scheme for catagorizing might be a bit time consuming. Also, there are references in auction catalogs and a variety of books. A pretty big project I fear though the Royal Armouries have (I am told) a pretty comprehensive reference though they have not published it.
 
Actually, it doesn't look that difficult to do. Time consuming to produce the database, yes.

From the page you've provided, I can immediately split the marks into two categories: those with text characters (letters or script) and those without. That's an obvious classification point.

Working with marks that have text, the next division point is, text only, or are there added elements (shields, plaques, figures, etc).

Going with text only, then you can take it further by number of letters present, then take it to the actual letters.

It's decision tree logic. The whole thing could be implemented in software as yes/no or multiple choice questions.

The point to make here is while it would be wonderful for a program to take you to the actual one-and-only match, getting that degree of accuracy would be costly. The better approach would be to let the software get you close, presenting items that are reasonably close, and then letting the human eye make the final choice. The person will have the final approval, anyway. The more precise you try to make the siftware, the greater the chance that due to maybe a poor image to work from or a poor choice in selecting an option, the correct choice is hidden from the user by the software. Write the program to get you close, say within a few percentage points. Trying to get those last few percentage points eliminated will triple or quadruple the effort required to write the program, and really isn't necessary.

Beanbag
 
JamesM said:
This sounds less and less like something that an-off-the-shelf classifier could deal with, and more and more like a PhD project.

If this is a professional venture, I'd suggest approaching an academic and proposing funding some research. Not exactly a swift return on the investment, though.

Not professional. Maybe a mitzva for my Curatorial and collecting chums.
 
I was never a programmer for anyone but myself in my own business and I would get professionals very amused with my efforts. That said, let me ask a simple question.

If I have bitmaps that contain the same number of rows and columns, how close would I get to a match if I did a series of correlations and simply ranked the r^2's?

I thought that was the format for a .bmp file but when I looked it seemed that there were other odd characters. The concept could be tested in Excel for small images, I suppose. The thought is that since I need images anyway, why not use them for maybe honing in a bit and then going to a decision tree.
 

Back
Top Bottom