Weak Signal® Research
Part II: Information Theory
Bryan S. Coffman
January 16, 1997
back to Part I: Introduction
Signals
Since we’re using the term "weak signal" it might be useful
to dip into some information theory to define some terms and examine some
concepts. Perhaps a side trip like this into a distantly related field
will shed some light on a systematic process for searching out weak signals.
What is a signal?
John R. Pierce has written a wonderful book entitled An Introduction to
Information Theory: Symbols, Signals and Noise. He defines a signal as
a variable electric current deliberately transmitted by a communication
system. We’ll take a broader, more metaphorical view, and call a
signal an event in which some living system or other element in the environment
transmits a message in the course or as a result of its actions or behavior.
In this sense, "everything speaks." The relative order of items
on a person’s desk sends a message. A gesture with the hand, posture,
or facial expressions communicates. One company buys a controlling interest
in another, seemingly unrelated one—what does it mean? Articles published
in trade or professional journals contain signals. The crime rate in a
city decreases. These are all signals.
Signals contain messages. Really the two are inseparable:
message is one way of looking at a communication event and a signal is
another. Message is content, signal is process. We use a variety of senses
to detect signals: eyes, ears, touch. The message is a string or collection
of symbols that comprise the transmission. The message by itself does
not MEAN anything. It has no intrinsic value whatsoever. The message is
simply the sequence of symbols received over time. The entire value of
the message is determined by the receiver.
We are absolutely deluged by signals during our waking and
sleeping hours, and these sort themselves into three groups.
Signals
beyond our perception
Most signals we are incapable of receiving because they are transmitted
at frequencies or across channels that we simply do not have access to.
None of us sees in infrared or hears ultrasonic sounds, for example. Radio
waves stream constantly through the environment but we are unaware of
them because we are not built to receive them. Or, even if we could perceive
the signals, we are not in a physical position to do so; activities on
the other side of the building, or city go on without our knowledge. To
gain access to signals outside the design and range of our personal senses,
we have to construct devices that can capture them. This sounds trivial
but it may be the most profound lesson to learn in weak signal research.
Most of our organizations are not designed, nor have they evolved, with
the proper receptors to recognize valuable weak signals. We must redesign
our organizations (and ourselves) so they can "tune in" to different
frequencies. Without this redesign, we’re sunk. How can you take
advantage of a great new idea if you’re incapable of hearing it?
How serious is this deficiency? Here's a quote from Megamistakes
by Steven Schnaars:
"Video games did not come from the leaders in the
board game industry. [They] were introduced by Nolan Bushnell, a 'work-in-the-garage'
inventor...
"Disposable diapers came from Procter & Gamble,
an industry outsider...
"The list goes on and on. Digital watches were ignored
by the Swiss watchmakers. Rough-terrain or 'city' bikes did not come
from the leading bicycle manufacturers. Running shoes did not come from
the sneaker giants. Both diet and caffeine-free soda did not come from
Coke and Pepsi... In every instance, as well as in myriad others, market
leaders were amazingly myopic in their perception of emerging markets
in their own backyards."
Signals
within our perception but unrecognized by our mental models
The next category of signals are those that we are able to receive and
process but which are ignored for one reason or another. When we drive
cars, we’re aware in general of the traffic around us, the condition
of the road, the weather, the scenery that we pass through. But we are
not aware in detail of these things, and the more familiar the trip, the
less we see and acknowledge. This is good! Imagine the traffic jam in
your brain if you tried to consciously process everything you saw on the
road? It’s impossible. Instead, we all construct and employ models
to filter out most of these signals so we can focus on the few that truly
make a difference. Like brake lights illuminating on the car in front
of us. We tend to focus on signals that provide us with opportunity or
represent a threat. But all of this is based on our mental models of how
traffic works. Take your mental model to France or Italy or Singapore
and you’ll experience an explosion of conscious awareness of what’s
going on around you (or you’ll have a tremendous repair bill). New
environments trigger the construction and expansion of new models, or
filters. When we bring these new models back into an old, familiar environment,
we will see it differently than we did before. Our models tell us what
signals to discard out of hand, to the point where the signals might just
as well not exist. Plants and animals in nature use this phenomenon in
the form of camouflage for protection or stealth. They depend on other
animals having models and sensory apparatus that will ignore the signal
and move along or come within range.
Not only must we redesign our organizations to see signals
that they’re not built to perceive, but we must also redesign the
models they use in filtering out what they are built to perceive, but
tend to ignore.
Signals
recognized by our mental models that we use to modify our behavior
The third category of signals are those that we can perceive, that our
models tell us to pay attention to, and which we use to adjust our behavior
(what a physicist would call the state of the system in phase space).
When a system receives a message it assigns value or meaning to the message.
This happens always at the receiver, never at the transmitter. In a sense,
the message is only a signal until it is transduced and decoded by the
receiver. Then it becomes a message. Again, this may sound trite or obscure,
but it points to at least two possibilities for organizational redesign:
transduction and decoding. What kinds of signals are allowed to cross
the organization's membrane, or boundary, and what mechanisms and languages
are used to decode them? To put it more bluntly, what kinds of ideas are
permissible to discuss and which are subject to dismissal or ridicule?
In how many languages or disciplines is the organization versed? What
is the danger of a new development taking the enterprise by surprise only
because the language surrounding it is incomprehensible jargon and incapable
of being decoded by insiders?
Information
We use the term so unconsciously and automatically that it's easy for
two individuals to talk about information, then go their separate ways
believing that they are in agreement, when they are really at odds. To
one person, information is synonymous with knowledge; to another it means
data. Many people would say that a message or signal is information, or
at least that it carries information.
Shannon, Wiener, and Two Definitions of Information
Claude Shannon and Norbert Wiener both spent careers thinking about information
from a very mathematical perspective. Wiener is one of the modern fathers
of Cybernetics, the study of control in organisms and machines. He focused
on information as an organizing property of living systems. Terms like
"feedback loops", "lag", and "systems theory"
emerged from his work into the vernacular. Peter Senge's The Fifth Discipline is
a structured application of Cybernetics to the business world and organizational
dynamics. Shannon was a mathematician who focused more on the communication
aspects of information, in particular on algorithms for sending signals
across noisy channels. The difference in how these two men defined information
may yield valuable insight for those working to design organizations that
excel in identifying and exploiting weak signals.
Wiener sees information as a measure of the degree to which
a system is organized; a measure of its order. The DNA molecule with its
structured nucleotide chains and the mechanism for translating these into
proteins demonstrates a high degree of order, and therefore, possesses
a large amount of information. In this sense, information is a measure
of the decrease of entropy--or randomness--in a system. A pile of papers
strewn on top of a desk has a higher entropy than the same set of papers
arranged in labeled files or catalogued in a knowledge base. There's more
information resident in the labeled files than there is in the strewn
papers. This is definition of information that most of us can relate to,
and that makes sense.
The scientists working in the field of complexity theory
might amend Wiener's sense of information, and say that information is
a measure of the degree of complexity inherent in and exhibited by the
organism. A bacteria contains more information than a stone, and a human
being more information than a bacteria due to the increase in complexity
from one to the other.
Shannon sees things a little differently. He defines information
as a measure of uncertainty as to the next message to be received in the
communication or messaging event. The higher the uncertainty or surprise,
the greater the information, and the greater the entropy. This is the
key: Wiener sees information as a decrease in entropy and Shannon thinks
of it as an increase in entropy. To be fair, They're not talking about
the same kind of entropy, so there's not true dichotomy. But pondering
both vantage points from the point of view of the organization may be
enlightening. Still, it's hard to understand how an increase in information
is a function of the level of uncertainty in communication. Since this
line of reasoning is counterintuitive, a few examples are in order.
Imaging a telegrapher sending consecutive letters in the
English language. Also assume that the letters combine to make ordinary
English words. Acronyms, place names and abbreviations are excluded. You're
the receiver. The first letter that comes across the line is an H.
A quick glance at the dictionary reveals that the second letter in words
that begin with an H is always one of the five vowels
plus Y. So, there's not much information that the second
letter will convey when it gets transmitted. In fact, many English words
are easily deciphered if all of the vowels are removed from their spelling.
Imagine that the second letter that's transmitted is an O.
Now think about what the third letter might be. It could be most any consonant
in the alphabet. We're far more interested in what the third letter might
be than we were in what the second letter was. There's more uncertainty,
and therefore, more information in the anticipation of the event.
For an extreme example, imagine that the first letter that
gets transmitted is a Q. Now, there's no information
inherent in the second letter, because there's no uncertainty as to what
it might be. Every English word that begins with a Q
is followed by a U. There's no point in sending the second
character at all, because it's predetermined.
Notice that before a message gets transmitted, there is
some degree of information, or uncertainty associated with it. After the
transmission, this measure of information collapses to zero because there
is no uncertainty associated with a message that's received (assuming
it is received error free). In a way, Shannon sees information as "potential"
messages and Wiener sees it as the ability of the organism to apply these
messages to generate a higher degree of order in itself and its progeny
(learning and evolution).
Here's another, completely non-scientific example I like
to use. Picture a couple who have been married for thirty or forty years.
They've settled into a routine, and often, if you watch them or listen
to them, you can predict with a high degree of certainty their course
of action following some remark or activity. There's no information contained
in their habits. There's nothing wrong with this at all unless these behaviors
comprise the whole of the relationship. Habits are a learning tool. They
make certain acts automatic so that we can be free to accept new information
and undertake new, more challenging experiences. Habits are the expression
of the past work of what James Miller calls the relationship between the
Associator, Memory and Decider. But what happens if habits
become not only the center but the circumference of our being? There's
too much order expressed. Too much homeostasis. Not enough entropy in
our communications. If we read the same magazines, hold the same discussions
with the same people, look at the same numbers, read the same reports,
scoff at the same ideas then we have left no window in our lives for information
to intrude--to stir things up and make us doubt, challenge, practice,
fail, and learn again.
As it is for us individually, so it is with our organizations.
In thinking about weak signal research it helps to remember the two kinds
of information that we seek. We want signals and the messages they bear
to accomplish two things for us:
-
Provide us with the raw materials out of which we can
learn, grow and evolve--to create ever increasing levels of "order"
in our systems.
-
Surprise us. If all we ever hear is what we expect to
hear, then we can be fairly certain that we will miss opportunities
and fall unsuspecting prey to threats.
Strong signals may surprise us, but their strength at least
puts them on our radar screens. The more interesting surprises come from
weak signals. This leads us to the problem of how to detect the weak signals
in the first place. To do so, we will face one major opponent...
Noise
Recall that Shannon focused on the problem of sending a signal over a
noisy channel with some assurance that it will be received intact. John
Pierce describes the impact of Shannon's solution on the community of
communications engineers at the time. The solution is elegant and unexpected.
First I'll recap some of the solutions that were tried before Shannon's
work.
If you knew there was a faint signal being transmitted on
a certain channel, but you couldn't separate it from other, unwanted signals
on the same channel, what options might you have?
First, you might consider boosting the power, or amplifying
the signal so that other signals would fall into the background. That
would solve the problem, but adding power to signals can be expensive,
and if you put too much power into the signal, you might deform the channel.
[The first transatlantic cable was destroyed by applying this technique.]
You could ask the transmitter to repeat the message over
and over until you were sure you had received it without error, by checking
successive transmissions against each other. This is also expensive, time
consuming, and not a guarantee of success.
Or, you might reconstruct the channel of more pure materials
so that it delivers a cleaner transmission. This will also solve the problem,
but again, at a high expense. Any other options? [There are several that
will not be discussed here.]
Shannon's answer was simple, but profound. In fact, it was
a true weak signal--an answer that was staring everyone in the face, and
seems almost trite today, but was inscrutable at the time. All you have
to do is understand the noise in the system and then you can make
the signal stand out. You don't have to strive to eliminate all of the
noise in the channel or keep asking for a repeat of the transmission,
or make the transmitter boost the power. Here's how John Pierce puts the
discovery:
"'Suppose that I told you that by properly encoding
my message, I can send it over even a noisy channel with a completely
negligible fraction of errors, a fraction smaller than any assignable
value. Suppose that I told you that, if the sort
of noise in the channel is known and if its magnitude is known,
I can calculate just how many characters I can send over the channel
per second and that, if I send any number fewer than this, I can do
so virtually without error, while if I try to send more, I will be bound
to make errors.'...
"Indeed, the whole problem of efficient and error-free
communication turns out to be that of removing from messages the somewhat
inefficient redundancy which they have and then
adding redundancy of the right sort in order to allow correction
of errors made in transmission."
Aha! There's a problem. It's all well and good to talk about
understanding the noise in the channel, but how can an organization have
any control over adding the right sort of redundancy to the transmission
of a weak signal when it has no influence over the sending of the signal?
It can't.
Unless it's the one transmitting the signal. Recall our
working definition of Weak Signal® Research. One of
the things our organizations must learn is how to redesign themselves
to take advantage of the possibilities implied by ecosystems or sets of
weak signals. Sometimes the best way to take advantage of a weak signal
is to broadcast it to a wider community, to lobby for support, as it were.
To invoke the law of increasing returns. Propaganda, fads, gossip, news,
jokes all propagate in this manner and we're familiar with this process.
They get shared in an exponentially expanding circle of transmitters and
receivers. Many times they get transmitted with amazing accuracy. But
remember the little game of "telephone" where children sit in
a circle and pass a fairly complex sentence from one to the other by whispering?
After the message goes around the circle, the children hear the original
message and the often humorously corrupted final version. If our organizations
are going to encourage the growth of ecosystems of weak signals by rebroadcasting
them, they must learn how to encode the signals so that they propagate
error-free. The ideas grow in a complex environment, but collapse in a
chaotic one.
But we talk about detecting weak signals, it's true that
the only part of the equation we can control is understanding the noise
in the channel. But that's sufficient. Metaphorically I think of ourselves
and our organizations as the channels. The maxim then becomes, "know
your noise." Think about what ideas you
easily scoff at or discard out of hand and hunt these down as sources
for good weak signals. Be suspicious of things that you believe will never
come to pass. What ideas challenge your most fundamental beliefs? Why
should they be such a threat; is there nothing that can be learned from
playing 'Spoze with them?
Remember the discussion above about the three kinds of signals?
- Signals beyond
our perception.
- Signals
within our perception but unrecognized by our mental models.
- Signals
recognized by our mental models that we use to modify our behavior.
Detecting weak signals of type one require somewhat expensive
modification of the sensory apparatus of our organizations. Detecting
weak signals of type two, however, only ask of us that we examine our
mental models and challenge them for the purpose of making them more global,
flexible, strategic, and therefore more resilient.
Summary of Part II
Our enterprises receive signals bearing messages every day. They assign
value to these messages through a complex comparison of the message with
other similar messages stored in memory and through the application of
an Associator that helps make decisions on what behavior the message should
invoke. Some of the messages we receive are very unexpected, or surprising
and may demand a more focused response. We say that such messages are
high in information content and entropy. Other, routine messages are still
necessary to process out of habit and for purposes of maintaining homeostasis,
but they will not be sources of good weak signals. The enterprise uses
messages to increase the degree of organization or order inherent in its
structure and behavior. This is the other sense of information--a highly
ordered organism is said to contain a high degree of information. To enable
our organizations to see potentially valuable ecosystems of weak signals,
we can modify the sensory apparatus, or change our mental models. The
process of changing our mental models can be called, "know your noise".
Part 3: Sampling, Uncertainty and Phase Shifts
in Weak Signals
Part 4: Evolution and Growth of the Weak
Signal to Maturity
Part 5: A Process Model for Weak Signal
Research
Other material on Weak Signal Research on this website
Here are some references on the world wide web for you to check
out:
- What is Information,
by Karl Erik Sveiby discusses the difference between Shannon and Wiener
in depth
- Cybernetics,
Human-Computer Symbiosis and On-line Communities: The Pioneering
Vision and the Future of the Global Computer Network, by Ronda Hauben
serves as a quick overview of Wiener's work, and John Pierce's book,
An Introduction to Information Theory
- You can download a primer on information theory using simple mathematics,
titled, Information
Theory Primer With an Appendix on Logarithms by Thomas D. Schneider.
- The Millennium
Clock, by Danny Hillis. It's short; just read it to help you get
above all of the technical stuff for a few minutes.
Here are some books on the subject of information theory:
- Cybernetics: or Control and Communication in the Animal and the
Machine by Norbert Wiener
- The Human Use of Human Beings: Cybernetics and Society by
Norbert Wiener
- Invention: The Care and Feeding of Ideas by Norbert Wiener
- An Introduction to Information Theory: Symbols, Signals and Noise
by John R. Pierce
- Living Systems by James Grier Miller
 
copyright © 1997, MG Taylor Corporation.
All rights reserved
copyrights,
terms and conditions
19970116170412.web.bsc
|
