As companies get smarter at collecting and storing information, they have also become increasingly aware of the need for effective tools for managing and analysing that information. This has created a significant business opportunity for tech firms in the UK and globally to develop new business solutions as well as opening up new roles within these firms for much sought after data scientists.

The three key considerations around big data are analysis, skills and the future landscape.

 

Analysis

When identifying key big data trends for this year, Forbes declared 2014 as the year that big data would “transition from hype to actionable insights”. In the recent tough economic climate, there has been an increased focus on value for money and return on investment. The potential value of a company’s big data is huge – but this can only be unlocked if this data can be managed and analysed to provide commercial insight to inform strategic decision making.

Data science is a relatively new term and many smart tech firms are now employing data scientists and investing in upskilling their existing teams in order to develop new solutions around big data analytics. From diagnostic analytics (gaining insight into a current situation) to predictive analytics (identifying future market conditions, challenges and opportunities), the successful deployment of analytical tools is moving towards a semi-automated approach – with the development of predictive and prescriptive models to increase productivity combined with human intervention to identify anomalies.

 

Skills

As far back as 2011, McKinsey predicted a shortage of relevantly skilled people to harness the potential of big data. This is now an urgent issue. The Wall Street Journal reported earlier this year that the shortage is especially severe in the U.S. where 80% of new data scientist jobs created between 2010 and 2011 have not been filled, according to their analysis.

The pipeline of talent for this role is also uncertain – the level of qualification needed takes many years to achieve and the career path is still not well understood outside of the professional world. As the pure technology to manage big data rapidly advances, so too does the need for people with the appropriate skills to design the intricate analytical models, to help companies gain insight from huge volumes of data they hold.

 

The future of big data

One thing is certain – the significance and prominence of big data will only increase in coming years.

However, there are changes ahead. Research recently published by EMC shows that the geography of the digital universe will change. Currently, 60% of data in the digital universe is attributed to mature markets such as Germany, Japan, UK and the United States, but by 2020, the percentage will flip, and emerging markets including Brazil, China, India, Mexico and Russia will account for the majority of data.

The source of the growth of data will also change as billions of everyday objects are equipped with unique identifiers and the ability to automatically record, report and receive data – a sensor in your shoe tracking how fast you run or a bridge tracking traffic patterns.

According to IDC the number of devices or things that can be connected to the Internet is approaching 200 billion today but only14 billion are already connected generating 2% of the world’s data.  IDC forecasts that by 2020, the number of connected devices will grow to 32 billion and they will represent 10% of the world’s data.

Much of this data will be “useful data” ie data that could be analysed and used for business and social benefit which will also present challenges as at the moment even though the EMC report says that only 22% of information in the digital universe is considered `useful only 5% is currently analysed. By 2020, more than 35% of all data could be considered useful data, thanks to the growth of data from the Internet of Things, but it will be up to businesses to put this data to use.

The other challenge will be keeping this 'useful' data secure and again a huge improvement in performance is necessary according to IDC who estimate that 40% of the data in the digital universe requires some level of protection, from heightened privacy measures to fully-encrypted data but today only half of that data is actually protected.

And finally and very importantly the question of who owns the data and who controls access to it is already becoming the most pressing policy and legal question for governments, citizens and businesses around the world.

We have to get this right particularly in Europe to ensure that the technical capabilities of Big Data actually deliver for the UK economy and society.