When ignorance of computer tools hampers public accountability

Blue Mountain

Resident Skeptical Hobbit
Blue Mountain
Joined
Jul 2, 2005
Messages
8,336
Location
Waging war on woo-woo in Winnipeg
While the main story is political and quite concerning, I'm posting this in technology due to the apparent ignorance of the Ontario Information and Privacy Commissioner and her staff.

Ford government used code words to make it 'unduly difficult' to search Greenbelt records: report
CBC said:
Political staff were using code words to thwart document requests, says IPC commissioner

Staffers sometimes referred to the Greenbelt project in messages as "special project," or "GB," or "G**," with references to G** being next to impossible to find. Those terms and their inconsistent use made it "unduly difficult" to search for Greenbelt-related records, [Ontario Information and Privacy Commissioner Patricia] Kosseim wrote.

"Worse, the use of the code word "G**" made it virtually impossible to find relevant records, given that the asterisk ("..") is used as a technical wildcard when conducting text searches, returning any word starting with "G," she wrote.

That meant having to forego using the code word "G**" as a search term, so some Greenbelt records may have been missed, Kosseim wrote.
Click to expand...
Full story at the CBC

I have no idea what search software the commissioner and her staff were using, but any search software worth its salt can “quote” wildcards to say “do not treat this as a wildcard but as a character.”

For example, in grep:
Code: 
grep 'G*' *.txt
Will find for you *any* line where there is a capital G on the line.

However,
Code: 
grep -E '\bG\*\*?\b' *.txt
Will find only those lines that contain “G*”" or “G**” as a whole word (\b tells grep to take word boundaries into account.)

I usually have faith that government organizations know what they're doing, but to have an entire office not even ask the question “is it possible to search for "G**" without interpreting the asterisks as wildcards?” is disheartening.
 
Last edited:
CBC said: Political staff were using code words to thwart document requests, says IPC commissioner
Click to expand...

Sounds like it was the opposite of ignorance of the tools they were using.
Implication is that this was a deliberate effort to use metadata that obfuscated document searches on a particular subject that they did not wish to supply to the public.

Basically the opposite of Hanson’s Razor.
 
Last edited:
EHocking said:
Sounds like it was the opposite of ignorance of the tools they were using.
Implication is that this was a deliberate effort to use metadata that obfuscated document searches on a particular subject that they did not wish to supply to the public.
Click to expand...
That's exactly what the article says. The commissioner was pointing out that's what the government was doing: using code to make document searches difficult.

My complaint is no-one in the commissioner's office had the technical skills to get around the obfuscation. It is possible, but no-one apparently knew how to do it.
 
Blue Mountain said:
…
My complaint is no-one in the commissioner's office had the technical skills to get around the obfuscation. It is possible, but no-one apparently knew how to do it.
Click to expand...
I didn’t see anything in that article about lack of skill to undo the obfuscation?
There didn’t seem to be any discussion on remedying the issue. Unless I missed something?

That’s why I thought the thread title was a little misleading. To misquote Hanson,
“Never attribute to ignorance that which is adequately explained by malice.”
 
EHocking said:
I didn’t see anything in that article about lack of skill to undo the obfuscation?
There didn’t seem to be any discussion on remedying the issue. Unless I missed something?

That’s why I thought the thread title was a little misleading. To misquote Hanson,
“Never attribute to ignorance that which is adequately explained by malice.”
Click to expand...

It's in part I quoted:

"Worse, the use of the code word "G**" made it virtually impossible to find relevant records, given that the asterisk ("..") is used as a technical wildcard when conducting text searches, returning any word starting with "G," she wrote.
Click to expand...

That tells me neither she nor anyone in her office knew how to search for the literal text "G**" and instead threw up their hands in despair when they realized their naive search returned every instance of "G" in the documents being searched.

Also, in case you missed it, the Information and Privacy Commission was conducting an investigation of the government, not trying to support it. Unless you have reason to believe that a commission is in the pocket of the sitting government. Here in Canada that tends not to be the case.
 
Blue Mountain said:
It's in part I quoted:



That tells me neither she nor anyone in her office knew how to search for the literal text "G**" and instead threw up their hands in despair when they realized their naive search returned every instance of "G" in the documents being searched.
Click to expand...
Ah, missed the implication of that comment.
Blue Mountain said:
Also, in case you missed it, the Information and Privacy Commission was conducting an investigation of the government, not trying to support it. Unless you have reason to believe that a commission is in the pocket of the sitting government. Here in Canada that tends not to be the case.
Click to expand...
No. All my comments were directed at the government department.
I’d misread the comments being made by the commission officers.
Title and OP are quite valid!!!
 
Blue Mountain said:
All too true. And it's not limited to government people, either. Reading some tales from tech support, doctors and lawyers can be challenging people to work with as well.
Click to expand...
I worked years in tech support, exclusively in government departments, and my experience there was that people - individuals - generally knew how to use the computers on their desks pretty well. My post said that governments - as a whole - don't know how to use computers. They don't know how to deploy them, they don't know how to keep them up to date, they don't know how to... uh... leverage their synergies...?

Government departments are by nature conservative and slow to move. Much slower to move than the internet, as it turns out. The fastest moving areas of government, in my experience, are IT Security areas, and that's because they have had to learn how to move quickly. If IT Security moved at the pace of other areas of government it would be disastrous. Not that they can actually keep up, but they're running the fastest.

Blue Mountain said:
That tells me neither she nor anyone in her office knew how to search for the literal text "G**" and instead threw up their hands in despair when they realized their naive search returned every instance of "G" in the documents being searched.
Click to expand...
It's not actually what it tells me. The article you're citing represents a snapshot in time - you don't know what happened the day after the interview was taken. Rather, I think someone found out that there was this huge loophole in their code, and immediately started to work on fixing it. Forgoing "G**" as a search term was a workaround until the next update could be developed and deployed. The CBC doesn't report that bit. In a month's time, I'll bet they can do the search the way they intended.
 
arthwollipot said:
It's not actually what it tells me. The article you're citing represents a snapshot in time - you don't know what happened the day after the interview was taken. Rather, I think someone found out that there was this huge loophole in their code, and immediately started to work on fixing it. Forgoing "G**" as a search term was a workaround until the next update could be developed and deployed. The CBC doesn't report that bit. In a month's time, I'll bet they can do the search the way they intended.
Click to expand...

Unfortunately, the quote is not from an interview. It's from the commissioner's report.
 
I find the premise of the Privacy Commissioner's office thinking the documents "unsearchable" unbelievable in the extreme. I have dealt with Ontario's Privacy Commissioner (a previous incarnation) and the staff there. They are not that stupid. Annoying and rule ridden, but not unintelligent.
 
Just by the by...

The term for searching for special characters, as characters, rather than using their special functions, is called 'escaping'.

For any sensible product, there will be a method to 'escape' special characters.

Typically, the method will be the addition of another special character, in front of the character/s you're search for.

Most commonly, the escape character is the backslash: \
(It's practically the standard.)

So to search for 'G**' you'd actually search for 'G\*\*'

I'm adding this, in case anyone runs into this common problem elsewhere, searching for 'escape character' and the system/product you're using, will find you the info you need.
 
novaphile said:
Just by the by...

The term for searching for special characters, as characters, rather than using their special functions, is called 'escaping'.

For any sensible product, there will be a method to 'escape' special characters.

Typically, the method will be the addition of another special character, in front of the character/s you're search for.

Most commonly, the escape character is the backslash: \
(It's practically the standard.)

So to search for 'G**' you'd actually search for 'G\*\*'

I'm adding this, in case anyone runs into this common problem elsewhere, searching for 'escape character' and the system/product you're using, will find you the info you need.
Click to expand...
Congratulations. You repeated what I said in the OP.
 
novaphile said:
The term for searching for special characters, as characters, rather than using their special functions, is called 'escaping'.
Click to expand...

To be more precise, 'escaping' is the term for specifying that you mean the actual special character to stand for itself, not to have its special meaning. One of the common cases for doing that is when searching, but that's not the only one.
 
theprestige said:
Stipulated. I'm still interested in the technical aspects.
Click to expand...
There's an investigation by the RCMP underway so maybe all will be revealed in due course.

The Ontario Government has strong Records Management procedures in effect. When the perps used the in-place IT/communications structure much data would be automatically recorded and archived.
 
You must log in or register to reply here.

Back
Top Bottom