Information, what is it, then?

[John Hewitt]

This question has haunted me for a year now. What is information? I mean, we can measure it, we can use it, we can cuddle it, but I do not know of a sensible definition of what the universal phenomenon that which we call "information" is.

So far I have only come up with one important aspect of it: Time. Or causality. Without either (they are related) information would be impossible.

So I'm asking you. Maybe one of you has stumbled across a good definition. What is Information?

Data is what is measured in bits and bytes and, so it seems to me, it is also what Shannon defines in his statistical mechanical definition of information. (In fact, in Shannon's day, the words data and information seem to have been used more or less interchangeably so I doubt that he was making any big point when he used the word information.)
Today, the IT curriculum for schools and such like tell us that information is "interpreted data." (This is not really my field of expertise, you understand, but I got lumbered with some IT classes at one time.)

This implication of this is that information cannot be measured in bits and bytes. On the other hand, it does mean that information can be encoded as data provided that the transmitter and receiver share an agreed interpretative mode.
This is the definition of information, and the distinction between information and data, that I now use. However, I find that the more senior and expert a person is, the less inclined he seems to be to follow it.

 

[Soapy Sam]

There's a danger we mean varied things by the same word.
Shannon's information has always seemed to me to be almost the exact opposite of the general understanding of the word.

 

[Paul C. Anagnostopoulos]

Information is a measure of the decrease in uncertainity (entropy) after reception of a message.

Righty-o!

Shannon's information has always seemed to me to be almost the exact opposite of the general understanding of the word.

Then maybe you like Kolmogorov complexity better, which is defined as the amount of computation required to specify an object.

~~ Paul

 

[Dymanic]

I would say yes, it could make sense to treat the snowfield as a communication channel

My question was whether doing so would make the snowfield a communications channel. You yourself provided what I believe to be the operative phrase: "where you wish to transmit". I don't know about time, or causality, but one aspect of information that I would consider indispensible (again, in the universal sense) is: intent; it cannot be defined independent of some goal-derived context. Part of the human perspective seems to include teleological assumptions that are so pervasive and so deeply embedded as to be completely transparent. The pattern-recognition modules return positive results, and subsequent action instantly defaults to "adopting the intentional stance" (if it looks like it might be capable of purposeful intent, treat is as if it is). Before you know it, virgins are being sacrificed to the appeasement of angry gods.

Not everything that could be treated as a communications channel is in fact a communications channel, and treating one as if it is may make sense, but only if it actually is -- otherwise, you may be up all night trying to decipher morse code messages in banging pipes, or micro-hieroglyphics on pebbles you've found (I know someone who actually did this once after being up several days on crank).

Shannon's information has always seemed to me to be almost the exact opposite of the general understanding of the word.

Because what is generally considered most important is content, something that is explicitly ignored under Shannon.

 

[Cuddles]

I'm far from confident that a DVD contains information if no dvd player exists.
It contains a series of holes. Those holes only contain meaning in the appropriate context, which is a dvd player. Even then, they only contain meaning when viewed by a human. What is the meaning of information if it is not in a human head.

But where does the information go? At what point do you draw the line and say there is no longer any information? A DVD contains information when it is made, there is no argument there. If we put a DVD in a box and hide it for thousands of years, at what point does it no longer contain information? When DVDs are not popular so most people can't use it? When the last DVD player breaks and no-one can read it? When the last person who built DVD players dies so no-one knows how to build something to access the information? When the last person who has used a DVD dies so no-one is sure what it is for? When the last piece of history describing the use of DVDs is gone?

I would say that even in the last case it could still be possible to determine that there is information encoded, even if it is not readable, much like trying to translate ancient languages. This raises a similar point to whoever mentioned Chinese earlier. I can't get any information out of Chinses writing, but I still know that it contains information. I don't agree that information depends entirely on context. The question of accessing information is not the same as the question of whether it exists. I think that information exists whether I, or by extension anyone else, can read it. A cyborg from Planet X might not be able to read a DVD, or even work out that it is supposed to contain information, but the information is still there, whether he knows it or not.

 

[Dymanic]

But where does the information go? At what point do you draw the line and say there is no longer any information?

It's a variant of the traditional conundrum: when a tree falls in the forest and there's no one around, does it make a sound? And while I experience an involuntary eyeball-roll upon encountering the term, I think l0rca may have nailed it in post #4 above with: "Information is a form of qualia."

 

[Soapy Sam]

Then maybe you like Kolmogorov complexity better, which is defined as the amount of computation required to specify an object.

~~ Paul

How much computation does a rock take to fall down a mountain? Yet the path it ends up taking may be incredibly complex, due to trivial stuff like the compressibility of another rock it hits , the rate of spin etc , etc. Absolutely impossible to compute in advance, yet the rock generated this path with consummate ease.

In Shannon's sense, that course may be impossible to express simply, so it's information heavy. While I appreciate the value of Shannon's POV in his original context (signal / noise) - it seems pretty inapplicable here, as does any notion that the path requires information to define it at all. It doesn't. It can be described in terms of information. It can possibly be described in terms of the will of the gravity gods. It just seems equally perverse to do so in cases like this. I think information is a metaphor- useful in some cases, not in others.

Because what is generally considered most important is content, something that is explicitly ignored under Shannon.

I'd say "meaning", but I think we mean the same thing.

But where does the information go? At what point do you draw the line and say there is no longer any information? A DVD contains information when it is made, there is no argument there. If we put a DVD in a box and hide it for thousands of years, at what point does it no longer contain information? When DVDs are not popular so most people can't use it? When the last DVD player breaks and no-one can read it? When the last person who built DVD players dies so no-one knows how to build something to access the information? When the last person who has used a DVD dies so no-one is sure what it is for? When the last piece of history describing the use of DVDs is gone?

I would say that even in the last case it could still be possible to determine that there is information encoded, even if it is not readable, much like trying to translate ancient languages. This raises a similar point to whoever mentioned Chinese earlier. I can't get any information out of Chinses writing, but I still know that it contains information. I don't agree that information depends entirely on context. The question of accessing information is not the same as the question of whether it exists. I think that information exists whether I, or by extension anyone else, can read it. A cyborg from Planet X might not be able to read a DVD, or even work out that it is supposed to contain information, but the information is still there, whether he knows it or not.

I'd argue with your starting assumption. I'm not sure it's meaningful to say there is "information in" a dvd at any point. There are holes in a dvd. Put those holes in the right situation and you'll get pictures. Put those pictures in the right situation, you'll get physical changes in synaptic growth. I'm still looking for the information. What I'm seeing are processes.