The Surprising Messiness of our Data World
Watch any sort of present day or futuristic spy movie or show and there’ll inevitably be a scene with a room full of screens, big ones and little ones. People toiling away and a main character telling the shadowy figures to bring up some sort of data on the target. They zoom in, sometimes hacking into video surveillance or a really cool screen shot with all kinds of data tat’s spiffy and ultra detailed. The reality is far more messier than we often think.
Just this past week, two Facebook (Meta) engineers were questioned in court about where user data is kept and how it is kept as part of the 2018 Cambridge Analytica scandal case. Their statement? They don’t think anyone at Facebook knows where all the data is. The court even found that when someone downloads their own data, much is missing. This supports the engineers comments.
We may have it in our noggins that companies like Apple, Facebook, Google, Oracle and so on, have these massive, perfectly organised, exceedingly well curated data centres. That they can perform amazing feats of data wizardry. We often think the same about federal governments. While they are very good with data and invest millions in data lakes, data warehouses and such, they’re far more messy and convoluted in reality.
Perhaps unsurprisingly, this all comes down to the usual suspect. Humans. And how we’ve built data storage technologies. If you work in a business that uses an ERP or even a CRM, even if it’s a small business, chances are, you’ve complained more than once about just that CRM or ERP. The larger the enterprise, the more different types of databases there are; customer, logistics, financial, marketing, sales and so on. So why then, given all these amazing tools and beyond humans, is it all so very, very chaotic?
In large part it’s because there are so very many different types of databases. Some we don’t think of databases, such as an ERP program or a website, but they are. A website is simply a collection of data sets packaged up with a nice design for humans to interact with.
On top of all these types of databases and the software designed to access them, comes along humans and how we set rules and policies around how that data is collected, stored, managed and analysed. And businesses today, as they have been for hundreds of years, are bureaucracies. While we tend to ascribe that word to governments, it is the same for businesses. They just happen to generally, be more efficient than government run bureaucracies.
A key cultural aspect of bureaucratic behaviours is information power. We see this in business and government and we often call it “working in silos”. And as much as we talk about the collaborative work environment of today and sharing data, which we do more than ever, there are still silos of information power. Not everyone believes that sharing is caring.
Senior leaders in any type of organisation know that knowledge is power and knowledge starts with data, then it becomes information, that in turn, becomes knowledge. Human cultures have, for thousands of years, used data and information as power tools. To change this would mean fundamental shifts in human culture. Cultures are built around information, it is the knowledge we use to live our daily lives. So it is unlikely to change.
Some data must, naturally, be kept siloed. For privacy reasons, intellectual property protection and so on. Data collection too, is messy. Especially when humans are doing the data entry. Companies spend a lot of time and money trying to get this right. It is a constant tension with no easy solution.
Another issue is the increasing amount of data we generate. It’s almost impossible for the wrinkly grey database and processors in our skull to comprehend. So much now flows through multiple countries and across multiple organisations. Governments have been talking about taxing some of that data. If they do, they’ll only ever tax a minuscule amount.
Some pundits have declared data to be the new oil. It is and it isn’t. It’s a good analogy, but oil is finite. Data is not. The term infonomics has even been created to start putting economic rules around data. This is brilliant. We may well find when it comes to data, we humans will always be Sisyphus, pushing that ever growing ball of data up an eternal slope that never ends.
So the next time you watch some dramatic show with scenes of wonderfully designed interfaces pulling up vast amounts of data instantly on the target. You can relax and think just how opposite reality is.
