Making Noise to Drown Out the Signal
In case you haven’t noticed, there’s a big fight going on in North Dakota over access to Native American land. The Washington Post has a better explainer than any I could ever write. That’s not what this post is about. This post is about something interesting that happened yesterday and that is going to seriously limit how we use social media to track events (and diseases and conditions) through it in the future.
A rumor started circulating yesterday that the local police in North Dakota, where the standoff if happening, was using Facebook check-ins to track the people who were traveling to the reservation to support the protestors. In an attempt to disrupt those efforts, tons of people — including myself — checked in on Facebook, saying that we were at the reservation. A “check-in” is when you tell your friends and followers where you are in order to make them jealous, encourage them to join you, or tell other people there that you have arrived and that you are bringing your best game to the party.
They’re kind of useless, really.
Anyway, a few weeks ago, the ACLU in California revealed that police in Baltimore were using social media and some private software to identify people involved in the riots last year. They would go social media sites like Twitter or Facebook and look at posts from people at the riots and what they were posting about the riots. A lot of the posts were pictures and photographs.
So this case in North Dakota would not be the first time that law enforcement uses social media to keep track of what is going on, and who is participating in these events. On the one hand, it can be a very valuable tool because you get to identify “bad hombres” among the otherwise peaceful protesters. You could act to stop them from putting everyone at risk and keep any demonstration peaceful. On the other hand, you’re probably breaking some ethical rules… If not laws against surveillance of innocent people.
But what happens when everyone starts making noise to mess with your surveillance system? What if your signal gets drowned out?
This was one of the concerns that we had when we first started to use residents in Maryland as sentinels of influenza activity. We worried that people would go online on purpose and report bogus presence or absence of symptoms. We also worried that non-influenza symptoms, like allergy symptoms, would be interpreted as actual signals by the system… Or would drown out the real signal.
Most epidemiological surveillance systems work this way. There is always some signal that is lost in the noise. You then try to adjust your system to account for the noise and end up losing something that was very close to the noise and not quite a full signal. This all makes you go back and forth on choosing what kinds of signals you want to detect versus the ones you’re willing to lose in order to ignore the noise.
In all of this, one of the things that not even the best surveillance system can account for is human ingenuity. Human ingenuity is a big reason why quarantines don’t work. It’s also a big reason why our best predictions of things like traffic and crime — and other strictly human endeavors — always end up being slightly off from what ends up happening. We’re kind of predictable and unpredictable in that way.
So what do you do if the signal gets drowned out on purpose? Well, the number one thing would be to have a backup system to rely upon if yours is rendered useless by some human interaction. That should be basic knowledge. The other thing you can do is to calibrate your system from time to time by injecting noise into it and telling it how to handle it. (It’s the reason why we have pandemic influenza response exercises. Talk about noise when everyone and their sister all of a sudden becomes a “worried well” casualty.) The last thing you could (and should) do is to present the findings of your surveillance program and include the caveat that there are missing/latent variables in the whole thing because something/someone went and made a lot of noise.
Good luck getting your bosses to be okay with that last one.
There have been a lot of advances in how we systematically collect, analyze, interpret and disseminate data. During the collection process, there will be times when a lot of noise drowns out your signal. For those days, the kind of days you wish you had stayed home, there are different methods to deal with what is happening. One thing you should always know how to do, however, is to recognize noise. Then you’ll truly be able to tell apart the noise from the signal yourself, technology be damned.