The Signal and the Noise: The Problem Facing Growers Struggling with Too Much Data

Apr 27, 2017

The Signal and the Noise: The Problem Facing Growers Struggling with Too Much Data

Apr 27, 2017

 

Pretty decent article here from the Capitol Press on how growers are struggling with how to make sense of the really large amounts of data so easily available to them in our increasingly technological age.

http://www.capitalpress.com/Profit/20170406/growers-struggling-to-make-use-of-overwhelming-data-experts-say

The fact of the matter is that reduced computing costs have created an enormous wave of information, and in a recent article in the Wall Street Journal written by Michael Milken and Igor Tulchinsky, caution us to buckle up because this already large tsunami of data "doubles in size every few years". The two authors, while conceding that this is a challenge, also recognize it as an opportunity.

As many are finding, the problem of understanding the meaning of lots and lots of data is deeper than just pressing the whole undifferentiated mass into Google and getting an actionable answer.  I would suggest it is rather more a matter of sorting out the unimportant data from the important, and then having the mastery of that body of knowledge to which the data refer and only then be able put it all together to arrive at a good decision. 

The book titled "The Signal and the Noise",  written by statistician Nate Silver makes some progress on this issue. A lot of the data available to decision makers and prognosticators in a wide range of fields, from weather, to markets, to sports events, to elections, and yes to agriculture is not that useful, is not worth listening to and can be called noise, while those bits that are really useful, the signals, are where we should be spending our time and attention.

Complicating this however is the fact that it's very rarely just one signal that merits our attention, but rather it can be a multiple or even further an interaction of these signals which is most meaningful, and yet not all are as equally important.  Take for the example the malnourished plant with a compromised root system.  Is the malnourishment truly just the roots, or do we also face some deficiencies in the soil?  What of the soil pH or CEC which might be impeding the transmission of these nutrients to the plant?  The knowledgeable person is going to know what compromises a root system, what soil nutrient deficiencies look like, what a pH of x means to the whole shebang and weighs its value, pieces the important parts together, discards the rest and then makes the call on how to proceed.

In short, it is a deception to think that simply having access to ever greater amounts data effortlessly bestows upon one the ability to make better and more accurate decisions.  Really it takes some accomplishment, experience and quite frankly a lot of hard work as an individual to sort out the signals from the noise, and further be able to put this concert of signals into a comprehensible whole.