I’ve just started reading Learning Systems Thinking by Diana Montalion. I’m not far in and don’t feel super qualified to define exactly what Systems Thinking is yet, but it made me think of a recent criticism of the polling from this year’s US Presidential election.
When we think about polls we use “linear thinking”. The raw polling data goes in, a model is applied, and a pristine and immutable result is emitted.
A beautiful linear (and closed) system.
However in contradiction to this model, Nate Silver just published about the “herding” he sees in polls out of swing states in the US. Pollsters have a perverse incentive for their results not to be outliers. You don’t want your poll dragged online for being a rogue poll, and you don’t want your poll to be down-ranked by aggregators. You want your poll to “stick with the herd”.
The polls for the US Presidency are so close they stretch credulity. Given a particular sample size you should expect a certain amount of variance. There should be outliers. But there aren’t—polls in swing states all sit within a 2.5 point range. Assuming a binomial distribution, Silver places the chance that this would happen naturally at 1 in 9.5 trillion.
It’s worth pointing out that the polls in non-swing states have the the expected variance. Curious.
It turns out, the polling system is actually in a system of systems.
Poll results feedback back not just into the answers poll responders give, but also into the way modellers think about their models.
Analysing this is Systems Thinking as far as I can tell. It’s zooming out and recognising the relationships between the different systems, and acknowledging that the very act of measuring the system inadvertently affects the system.
As far as I can tell the purpose of Systems Thinking is not to provide tools that allow us to re-encapsulate the system-of-systems back into a box that will conform to our linear thinking. It’s to give us tools to understand the mess as it is, not as we wish it was.
Red Bearing 