Weak Signal® Research

Part I: Introduction

Bryan S. Coffman

January 15, 1997

Explanatory Note
In this series, I will spend a bit of time examining the scientific theory of information, cybernetics, and some topics under the heading of complexity, or self-organization. Then I'll apply this examination via metaphor, simile and analogy to the human venture, or enterprise. The purpose of this application is to stimulate thought, to cause it to shift perspective or basis, and to make it more elastic by consequence. The theory of information and communication deals with the transmission and recognition of electronic signals; complexity and self-organization derive many of their conclusions from computer simulations. Results from these fields may be instructive in the realm of the enterprise, but I do not suggest that the principles of the science of electricity, for example, maintain any one-to-one correspondence with or transference to the art of growing organizations.

I do believe that the prevalent beliefs that society holds as to the nature of the universe and reality largely govern its expression of culture. Even though, for example, the principles of quantum mechanics belong expressly to that atomic and sub-atomic realm, the popularization in business of terms such as "quanta", "the uncertainty principle", and "the dual nature of light", has led people—rightly or wrongly—to apply these principles to their business and look for their metaphorical representation.

It may be that these principles apply to business in more than metaphorical ways. Take, for example, the quantum jump. In physics it refers to the transition from one energy level to another, and the release of a packet of energy equivalent to that of the transition. Energy is not radiated or absorbed on a continuous scale, but discontinuously, in these discrete packets called quanta. This governs how electrons move between orbits in the atom and how many electrons can inhabit each orbit. An electron can either be in one orbit or another, but it doesn’t create new ones, or inhabit space in-between. Using this knowledge as a filter, might we spot the same phenomenon in our organizations? Perhaps enterprises are tuned to operate at discrete levels of performance but not in-between. Perhaps there is not a linear correlation between investment and performance in some cases, rather, increased performance occurs in sharp, discontinuous spurts—you invest a lot of money into an idea over time and all of a sudden, it "pops" into viability. Once you wear the "quanta filter" you may see your enterprise in a whole new way.

Or maybe not. And that’s the point, oddly enough. Creativity is full of blind alleys and dead ends. Creativity is the elimination of options.

So as we go forward, the reader will understand that I’m not blindly applying science to situations where it does not belong, but using the principles of various sciences as windows or filters through which we may gain some illumination on the challenges facing our various enterprises.

What is a Weak Signal and What is the Purpose of Weak Signal Research?
In organizational dynamics, and for the purpose of these papers, a weak signal is

  1. an idea or trend that will affect how we do business, what business we do, and the environment in which we will work
  2. new and surprising from the signal receiver's vantage point (although others may already perceive it) [To learn about one mind bender that you may not be familiar with--nanotechnology--click here]
  3. sometimes difficult to track down amid other noise and signals
  4. a threat or opportunity to your organization
  5. often scoffed at by people who "know"
  6. usually has a substantial lag time before it will mature and become mainstream
  7. therefore represents an opportunity to learn, grow and evolve.

Once you perceive a weak signal and understand it, a whole host of other signals may become visible. These comprise the complete ecosystem of ideas and trends that will support each other in the journey from dream to manifestation. No weak signal ever rises to dominance by itself, but is accompanied by shifts in political, economic, technological, and social thought and invention.

Weak Signal Research refers to those organizational traits and organic components that enable the enterprise to detect weak signals as a matter of course, build models and stories that illustrate the possible effects of whole sets of signals over time, and redesign itself efficiently to take advantage of these possibilities.

There are some real kickers in this definition. First, for most organizations, weak signal research is not a natural function. Most living systems are interested in excluding and rooting out radical ideas that threaten to infect them like viruses. Mutations may be an evolutionary driver, but we work hard to keep them from happening. In order to maintain organizational integrity and stability (otherwise known as homeostasis) the enterprise can't possibly entertain and play 'Spoze with every new idea that appears on the horizon. So the weak signal research function has to be approached with discipline and creativity. And new tools must be invented to help us entertain large numbers of ideas at once and view their interactions as a synthesis. We can prepare ourselves for a synthesis of ideas more easily than we can for a million different contingencies. Another skill that's required is storytelling, which has become a lost art. Synthesis and forecasting of weak signals can't rely on numerical analysis alone (although the more nonlinear forms of analysis that are emerging show some promise). It relies instead on the complexity inherent in building a story that illustrates the interaction of characters, plot, and settings over time. [click here for a University of Houston overview of forecasting and click here for a Sloan Management Review article on scenario building.]

There are a couple of broad categories of weak signals. One type identifies those signals that threaten homeostasis or offer incremental improvements but don't require the complete overhaul of products, processes, projects or organization. The other type offers large leaps in productivity or threatens catastrophic loss of capacity, and in this case, the organization must undergo radical redesign and evolution.

A Weak Signal Research Story
In the late 1970’s I worked as a junior geologist for a gas company in Colorado. I was assigned to investigate a persistent leak in one of the company’s storage fields. During the summer months, we pumped gas back into wells to store it in porous and permeable sandstone reservoirs deep underground. In the winter we pumped it out to meet the increased demand of that season. Some of this gas was leaking somewhere, as evidenced by the changing pressure at various well heads around the field. As a part of my investigation, I pulled the logs from all of the wells in the reservoir and from neighboring wells outside our field and mapped the paleogeography to plot the overlapping strata of ancient river beds nearly 100 million years old. After sorting out the various pumping cycles I uncovered a faint pattern in the data. The gas was migrating in one particular direction, following an ancient river bed. It was capped at this end by a fault, but ancient flooding had left a trail of sand that had broken the banks of our river bed and connected to a small pocket of sandstone outside our property. I checked the log of a well that bored into this pocket and discovered that it had been dry since it was drilled but within the last couple of years it had suddenly become a producer. They were selling our gas! A minor adjustment in our pumping cycles fixed the problem and saved the company a tidy sum of money.

I had been on the hunt after a weak signal. The engineers in the field had first picked up signs of it in the records of a trickle off of well head pressures. I picked up the trail in my analysis of the subsurface geology. After sifting through and understanding all of the noise in the data, the pattern of the signal stood out in sharp relief and I could address the problem with confidence. Notice the pattern here:

  1. Something just "feels funny" about the behavior of a particular system. There’s something different happening and we can’t quite pin it down. Usually this is confirmed only by a hunch or occasionally stray pieces of data call attention to themselves.

  2. In order to uncover a full picture of the situation, a great deal of noise must be processed. Note that the noise is frequently eliminated only by understanding it. I had to understand what was happening underground and also had to sort out various pumping cycles. Once I did this analysis, the data began to sort itself into groups—into context. In fact, I built an entire ecosystem in context so I could see relationships between these sets of data.

  3. The signal now stands out clearly against the noise and can be mapped in relation to other factors influencing the problem.

  4. Once mapped, a solution can be designed and implemented.

Numbered steps 1, 2, and 3 are called "creating the problem," and step 4 is "solving the problem." These are the two halves of the creative process. [Click here to read more about our Seven Stages of the Creative Process Model.]

In the example, the problem was not "we’re losing gas." That was the condition or situation. The true problem was, "overpressure in the field is causing the gas to flow through a lower permeability sandstone into an adjoining layer, from which it is being pumped by another operator." Conditions, however vexing, have no solution. They are not specific enough to allow a solution. They await the application of creative vision, determined intent, and the serendipity of insight so that they can be crafted into problems that can be solved.

Likewise, the solution to the problem was not, "stem the loss of gas from the field." This was the desire that fueled intent, perhaps, but it lacked insight to meaningful action. Once the problem was formulated, a number of true solutions could be tried in the engineering and testing phase of the creative process. Taking legal action as a solution was untenable, and suboptimal from a geological point of view. Instead, gas was piped out of well heads at the periphery of the field and pumped back into well heads in the center of the field. This kept the pressure at the boundary low enough so that the low permeability sandstone leading to the adjacent pocket would resume its role as a barrier to the flow. The solution was incorporated as an option to implement once pressure at periphery well heads reached a certain level—it was built into the operating procedures of the system. Then it was actually used by the engineers in the field and its results evaluated.

Another Example--Innovation and Evolution
That’s a pretty simple example. We have all collected and analyzed data in order to spot malfunctions in the system before they grow large enough to cause real trouble. This is one use of weak signal research—to maintain system homeostasis and stave off threats to this balance. But there’s another whole realm of weak signal research whose purpose is to allow systems to evolve and innovate. It’s to this realm that we turn our attention. It’s the realm where uncovering a weak signal before anyone else does, gives you an edge in development and may allow you to witness the emergence of an entire ecosystem of interlocked, collaborating ideas, inventions, and enterprises.

In a narrow sense, we’re talking about tracking trends but more generally, we’re interested in spotting non-linear, hard to predict ideas long before they reach mainstream recognition. And we’re not looking for ideas or trends in isolation. It’s rather meaningless to hunt down and map a potential trend for automobiles powered by hydrogen fuel cells, for example. For such a trend to materialize, a whole host of other factors—psychological, political, social, technological, economic—must emerge. How will petrochemical companies respond to the possibility? What about the public’s connection of hydrogen with disasters like the explosion of the Hindenburg (unfounded in the case of fuel cells)? What infrastructure of fueling stations, repair shops, and trained mechanics must be in place and how might this infrastructure emerge? Now, what are the probabilities of all of this occurring in an interconnected way? There's no way to apply metrics and measurements to such a question. The best way to approach such complexity is through systematic, disciplined story telling, or scenario building. The trick in such story telling is to suspend disbelief long enough to play 'Spoze with the idea. In other words, assume that the ecosystem of ideas and trends actually does emerge at some future date. Then build a complete picture of that future ecosystem and extrapolate BACKWARDS to fill in all of evolutionary landmarks reached along the way. In doing so, with rigor and some depth of research, you will likely uncover true weak signals hidden in the periphery of your scenario. These become your target. The original weak signal that you used to play 'Spoze with may only be a foil through which you can discover more surprising possibilities.

So now we can modify our process somewhat for weak signals that portend great innovation, growth or evolution. The steps are slightly different than for weak signals that indicate a system is out of homeostasis.

  1. Some particular idea or set of half-conceived ideas are hanging around the periphery of your comprehension. There’s something different happening and we can’t quite pin it down. Usually this is confirmed only by a hunch or occasionally stray pieces of data call attention to themselves.

  2. In order to uncover a full picture of the new idea, a great deal of noise must be processed. Note that the noise is frequently eliminated only by understanding it. In this case, you’ve got to understand the biases and ignorance that keep you from seeing the new idea clearly. You’ve got to put yourself in a position to play ‘Spoze intelligently. This means suspending biases (even cherished ones) temporarily and doing some research to more clearly understand the language, behavior and position of the new idea. You can’t play ‘Spoze if you have too little data.

  3. The signal now stands out clearly against the noise and can be mapped in relation to a complete ecosystem of ideas at some future time. This is advanced storytelling and scenario building.

  4. Once mapped, one of several outcomes may be realized:

    1. The new idea can be discarded as a low probability occurrence.
    2. The new idea may have potential and the organization must be redesigned to take advantage of it.
    3. The process of building the scenario uncovers a number of other weak signals that have at least as much potential as the original one.
    4. The process may lead the designers to CREATE their own new weak signal and redesign their organization to take advantage of it.

Most weak signals that enable innovation and evolution are not so profound as the development of an entirely new technology, or a radically different economic system. On a daily basis we scan the environment for clues on how to do our work much better, much faster, for greatly reduced cost, and with results that appeal to customers in a more lasting and satisfying way.

The most valuable weak signals don’t start, and can’t be found in whatever industry we happen to be working in. Great ideas that are born within an industry for use in that industry tend to mature quickly and are shared broadly. No one can generate more than marginal gains from them. Or, worse, they are ignored in a blindness that comes from a survival instinct to maintain the status quo. There are many stories about industry leaders failing to detect and implement new ideas that threatened entire product lines and ways of doing business, or, worse yet, just seemed outlandish and silly to someone who had been in the business for years and was obviously an expert. The most valuable weak signals usually come from someone working outside of the field who happens to invent a solution in search of a problem, or a solution to someone else’s problem. The book Megamistakes by Steven P. Schnaars is full of examples. [click here for a Star Tribune article on forecasting] And sometimes, before the idea can be made useful, it must undergo several transformations, so it’s not simply a matter of spotting a ready made solution lingering in the bowels of some unrelated field of endeavor.

What if there were a process that could be followed to systematize the hunt for weak signals? Perhaps there are techniques or strategies that can be employed in the four stages that were mentioned above. That’s our hypothesis, and we invite you to experiment along with us to see if such strategies and methodology exist.

Part 2: Information Theory
Part 3: Sampling, Uncertainty and Phase Shifts in Weak Signals
Part 4: Evolution and Growth of the Weak Signal to Maturity
Part 5: A Process Model for Weak Signal Research

Other material on Weak Signal Research on this website

copyright 1997, MG Taylor Corporation. All rights reserved
copyrights, terms and conditions


© MG Taylor Corporation, 1995 - 2002

iteration 3.5