Big Data: The Future of Innovation, Competition, and Productivity

Big data is poised to become a central driver of competition, productivity, and innovation. However, it can only reshape industries if the right policies and enablers are implemented to harness its value effectively.

The volume of data worldwide has surged dramatically, and analyzing large datasets, known as big data, will become an essential pillar of competition. According to research by McKinsey Global Institute (MGI) and McKinsey's Business Technology Office, big data will fuel new levels of productivity, innovation, and consumer benefits. Every industry leader, not just data-driven managers, must address the implications of this data explosion. Data growth is expected to continue rapidly, with more detailed information captured through social media, multimedia, and the Internet of Things (IoT).

Deep Analytical Talent: Where Are They Now?

Research from MGI and McKinsey’s Business Technology Office dives deep into the current state of big data and identifies the substantial value that can be unlocked. MGI’s study focuses on five domains: healthcare in the U.S., the European public sector, retail in the U.S., and global manufacturing and personal location data. The potential for big data to create value is vast. For example, retailers leveraging big data effectively could increase their operating margins by over 60%. The public sector can also unlock enormous value: U.S. healthcare, if optimized through big data, could save over $300 billion annually, mainly by reducing healthcare costs by about 8%. In Europe, governments could save more than €100 billion ($149 billion) just from operational efficiencies, not to mention reducing fraud, improving tax collection, and minimizing errors. Personal location data could create $600 billion in consumer surplus. These examples demonstrate the massive potential for big data to transform industries.

Seven Key Insights on Big Data

1. Data Is Now a Core Factor of Production

Data has become an integral part of every industry, alongside traditional factors of production like labor and capital. By 2009, nearly every U.S. sector with more than 1,000 employees had accumulated an average of 200 terabytes of stored data per company—double the size of Wal-Mart’s data warehouse in 1999.

2. Five Key Ways Big Data Creates Value

Big data can generate value in a variety of ways:

• Transparency and Frequency: Big data can make information more accessible and usable at higher frequencies, allowing businesses to act faster.

• Improved Performance: The vast quantities of transactional data allow companies to track and analyze everything from inventory to employee performance, uncovering variability and enhancing overall efficiency.

• Narrower Segmentation: Data allows businesses to target customers more precisely, leading to better-tailored products and services.

• Improved Decision-Making: Sophisticated analytics help improve decision-making by identifying patterns and trends.

• Next-Generation Products: Big data can fuel the development of new products and services. For example, manufacturers use data from embedded sensors to offer proactive maintenance services before problems arise.

3. Big Data Drives Competition and Growth

The use of big data will become a primary factor in business competition and growth. Companies must take big data seriously to stay competitive regardless of their industry. From established players to new entrants, all firms will use data-driven strategies to innovate, outpace competition, and extract value from timely, detailed information.

4. Big Data Fuels Productivity and Consumer Surplus

Big data will be instrumental in driving productivity gains and generating consumer benefits. For instance, retailers utilizing big data could boost their operating margins by more than 60%. Additionally, big data has the potential to enhance consumer experiences, with services powered by personal location data offering an estimated $600 billion in consumer surplus.

5. Sectors Set to Gain the Most from Big Data

The value potential from big data varies by sector. Comparing historical productivity trends with the potential for significant data-driven gains, sectors like computer and electronics, finance, insurance, and government are expected to benefit the most. These industries have tremendous opportunities to use data to improve operations and deliver more excellent value.

6. Talent Shortage in Big Data

The shortage of skilled talent is a significant challenge to fully unlocking big data’s potential. By 2018, the U.S. alone could face a shortfall of 140,000 to 190,000 individuals with deep analytical skills. In addition, there is a growing need for about 1.5 million managers and analysts who can leverage significant data insights to make effective decisions.

7. Addressing Key Challenges to Unlock Big Data’s Full Potential

To realize big data’s full potential, several issues must be tackled. Privacy, security, intellectual property, and liability policies must evolve to protect stakeholders. Companies must have the right talent, technology, and workflows to capitalize on big data. Access to data will also become more critical as businesses increasingly need to integrate information from multiple sources, often provided by third parties. The right incentives and structures must be established to make this data sharing feasible and beneficial.

Big data holds enormous potential to drive innovation, enhance productivity, and revolutionize competition. But businesses can only fully unlock its transformative power with the right policies, talent, and strategies in place.