Fifty Shades of Data
Don’t lose control of your analytics
Maybe you’re a bit curious. Or a bit scared. Maybe it’s a bit of both, and that’s what makes it exciting. You know there are people around you that are into it… like really into it, and from the outside, they seem so normal. But dig a little deeper beneath their spreadsheets, and you find yourself stumbling into the strange and scintillating world of data.
Fair warning, the preceding intro is about as juicy as this blog post is going to get (at least from a pulp romance novel perspective), and in case you need it, just remember the safe-word “cloud confidence”. From here on out, we’ll be taking a closer look at the kinds of data we collect and share, how we transform that data into insight, and how data is changing just about every aspect of business – which, come to think of it, is actually pretty juicy in its own right.
Big Data: a rose by any other name?
About five years ago, the term “big data” started to appear – almost out of nowhere – on every CIOs list of their top technology priorities. Five years on, it’s still on their lists (though it’s fallen off the Gartner Hype Curve this year). And while it may have slipped from being the top priority for many CIOs (in favour of issues such as cloud computing and cyber security), it still is a perennial favourite when it comes to issues that dominate discussions of technology strategy. Where did this big data phenomenon come from? And why do so many organisations seem unable to harness the power of big data? In short, “big data” is a set of data so large that current technology and understanding struggles to process it. It’s not new. The idea of being able to collect data that taxed processing power can be traced back to the creation of the Hollerith machine in the late 1890s. We’ll examine these issues throughout the post, but first let’s take a look the different names we’ve used to describe the idea of turning your data into insight.
- Data mining
- Analytics
- Business analytics
- Business intelligence
- Big data
How does data become insight?
For a lot of business executives, the idea of analytics can be daunting. There may be a vague understanding of how databases work (those are the cylinder-shaped things in presentations about IT, right?) Even for IT executives who aren’t directly involved in analytics, the structure of a data program can be confusing. For the most part, it’s fairly straightforward: you’ve got (usually) separate systems to capture your data, to arrange that data into a useable format, to store that formatted data, to structure that data in a way that’s easy to “ask it the right question,” to present the answers to those questions logical and compelling ways, and ultimately to apply those answers to solve business problems. So from end to end (from data to insight, if you will) those systems are:
- ETL: allows you to extract, transform and load your data
- Data warehouse: where your enterprise data is stored
- OLAP: lets you select bits of data for comparison (elements like time, revenue, costs & location)
- Data Visualisation: presents data in charts, graphs and other visual ways
- Analytics: the interpretation (human or machine) of that visual data to gain insights and make business decisions.
Styles of analytics
So now that the journey your data take (from extraction through to analytics) is clear, how are you presented with this information? Below, are styles of analytics you’re likely to encounter from the most basic reporting (e.g. sales figures by region) to the most advanced analytics with machine learning (e.g., incorporating real-time environmental data from multiple sources to enable a driverless car). It’s worth digging a bit deeper into each of the styles as each has a set of use cases for which it’s best suited, but in order of complexity, they allow you to see what has happened, what is happening, what’s happening against expected or desired measures, what’s likely to happen, and how to interpret and act on what’s happening.
- Reporting
- Adhoc query
- OLAP (slice and dice)
- Alerts and notifications
- Dashboards
- Scorecards
- Predictive Analytics
- Machine learning
Sources of data
One of the reasons that big data has become such a large issue to organisations in recent years is the simple fact that the sources of data keep changing and getting more complex. Before the internet, data came from fairly fixed systems with fairly structured and standardised data. Information from ERP, CRM or POS systems didn’t really change the type of data they captured. But in the past decade, there’s been an explosion in sources of data. Companies are capturing unstructured data from places such as social media and call centres. Loads of data is being generated by tags and sensors in smart devices. As this variety of data continues to grow, the challenges of being able to make sense of all of it continues to grow as well. Here are the main sources of data your organisation is likely to encounter:
- ERP systems
- CRM systems
- Social Media
- Call centre logs
- RFID tags
- Sensors
Location of data
So where does all this data reside? The short answer is all over the place – which in and of itself is a problem when you’re trying to bring together all your company’s data for a single version of the truth. With the rise of cloud computing though, “all over the place” isn’t necessarily a bad thing. And while there’s much to be said about cloud computing and data storage, we’re just about halfway through the list. So if you need a break (or just want to read more about the cloud), remember the safe-word: Cloud Confidence.
- Desktop (Excel files, Access database)
- On-premises
- Cloud
- Hybrid
Top people to listen to on Social Media
If you’re still reading… I’m glad you stayed. Because while the first half of this post focuses on the basic structures that make up analytics, the second half focuses on the juicier bits: the bigger questions about where data and analytics are going, and what barriers need to be overcome to get there. The best way to keep a finger on the pulse of the analytics community is to get connected with the top minds in the space. From data science, to machine learning, to prescriptive analytics, and more, here are some of the big names in big data:
- @claudia_imhoff: Claudia Imhoff, Founder of the Boulder BI Brain Trust
- @idigdata: Jen Underwood, Microsoft BI & Analytics guru and thought leader
- @jameskobielus: James Kobielus, IBM Big Data Evangelist
- @jenstirrup: Jen Stirrup, PASS Board of Directors for Business Analytics
- @jlwoodward: Jonathan Woodward, Microsoft UK Lead for BI and Analytics
- @josephsirosh: Joseph Sirosh, Microsoft machine learning expert
- @kdnuggets: Gregory Piatetsky, big data and data science legend
- @kirkdborne: Kirk Borne, Principal Data Scientist at Booz Allen
- @stevedunbar: Steve Dunbar, Microsoft UK Lead for IoT
Real world examples of data in action
Can you use data as a transformational force for your organisation? Absolutely. How it’s used is entirely up to you. We’ve got customer stories showcasing big data from a wide variety of industries from a football club that have used analytics to improve fan engagement across the globe, to a manufacturer using data from IoT to improve lettuce production, to a charity that uses advanced analytics to connect people with causes that matter the most to them. Here are just a handful of our customer stories:
- Financial Services: MetroBank uses data to improve the customer experience.
- Wholesale and Distribution: JJ Foods uses predictive analytics to improve sales.
- Manufacturing: Fujistu uses IoT data to improve crop yield.
- Sports: Real Madrid tracks social data to improve fan engagement.
- Charity: Just Giving uses advanced analytics to increase giving.
Top challenges of data
While there are a lot of great examples of companies leveraging data in all its forms, quite frankly there aren’t enough. Only a fraction of companies feel as though they are successfully leveraging their data. This happens for a number of reasons. Some are technological challenges (e.g. having legacy systems and data or just having the wrong portfolio of analytics tools in place), but more are human challenges. There’s a dearth of qualified data scientists available in the market (it’s why they command top salaries), but more importantly leveraging data to make business decisions isn’t part of the business culture for many companies.
- Historical estate and legacy data
- Not having the right technology portfolio
- Dearth of available talent
- Data-driven decisions aren’t part of business culture and strategy
- Not knowing the right questions to ask (or how to ask them) of data
Future Trends of Data:
So where do we go from here? For those organisations who have overcome the current challenges of data, they’ve got their eye on the next phase of analytics. While we could write a separate post on 50 trends of data, many of these ideas fall under a few umbrella concepts. The cloud will be the source of significantly more data as more devices and systems are connected, but the cloud – with its infinite scalability and flexibility will also be crucial in being able to make sense of all these new sources of data. While traditional analytics (what happened and why) will still have an important part to play, we’ll start to move beyond predicting what will happen to being prescriptive with analytics – systems automatically taking action on insights gleaned through data. And our systems continue to improve their ability to anticipate, learn and respond, we will reach a point – aka the singularity – when this technology surpasses human control or understanding. Depending on your point of view, this could be thrilling or terrifying. Maybe a bit of both.
- The Cloud
- Prescriptive Analytics
- The singularity
Now with all this talk of data, are you curious? Is your data rewarding you or punishing you? If you want to find out more, have a look at our Data Culture Series. Whether you’re a data expert or a business professional, we’ve got an event to help you take control of your data. Laters.
