The Big Challenge of Massive Data

Saturday, 21 May 2011 22:06
Print PDF

Data is flowing into every corner of the global economy today; companies large and small produce massive volumes of transactional data and capture information about their customers, suppliers, and operations while millions of networked sensors embedded in devices such as energy meters, automobiles, and industrial machines sense, create, and communicate data in the age of the Internet of Things. Social media sites, smartphones, PCs and laptops have allowed billions of individuals around the world to contribute to the amount of data available. Additionally, the growing volume of multimedia content has played a major role in the exponential growth of data volumes.


Researchers have gathered strong evidence that these massive volumes of data can play a significant economic role to the benefit not only of private commerce but also of national economies and their citizens. Hence, the ability to capture, store, aggregate, and combine data and then use the results to perform deep analyses has become ever more essential for businesses and governments as cloud computing continue to lower the cost of storing data and the capabilities of sophisticated analytics software increase by the day.


Many pioneering companies are already employing massive data to create value, and others need to explore how they can do the same if they are to compete. Governments, too, have a significant opportunity to boost their efficiencies and to better utilize their finite resources.


However, companies, governments and policy makers need to address considerable challenges if they are to capture the full potential of massive data. The first challenge is the shortage of the analytical and managerial talent necessary to make the most of massive data. Research suggests that the United States alone faces a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts to analyze massive data and make decisions based on their findings. This acute shortage of talent is just the beginning. Other challenges include the need to ensure the availability of the right infrastructure and that incentives and competition are in place to encourage continued innovation and that safeguards are in place to address public concerns about massive data security and protection. Most of us remember that only last month the personal details of over 100 million subscribers were electronically stolen from the Sony Playstation Network.


Analysts fully anticipate that this is a story that will continue to evolve as technologies and techniques of manipulating massive data develop and data, their uses, and their economic benefits grow (alongside associated challenges and risks).


The effective use of massive data has the potential to transform economies, delivering a new wave of productivity growth and improve the life of citizens around the world. Using massive data will become a key basis of competition for existing companies, and will create opportunities for newcomers who are able to attract talented employees that have the critical skills for the massive data world. Leaders of organizations need to recognize the potential opportunity as well as the strategic threats that massive data represent and should address any gaps in their current IT capabilities and their data strategy.
Policy makers need to recognize the potential of harnessing massive data to unleash the next wave of growth in their economies. They need to develop institutional frameworks to allow organizations to easily create value out of data while protecting the privacy of citizens. They also have a significant role to play in helping to mitigate the shortage of talent through education and immigration policy.


From a regional aspect and apart from the most developed banks and mobile service providers it is hard to see where else in the Middle East have organizations considered massive data analysis as a strategic source of operational efficiency or competitive gains. Moreover, universities in the Middle East do not seem to be participating in the race to produce talent in this sector. Couple that with the forecasted global shortage of massive data analysis skills and you will get even more worried. It is reasonable to assume that IT vendors will bring their global knowledge, experience and best practices to support such projects much like they have done when the region started adopting technologies like ERP and CRM but business executives and policy makers hold the keys to such investments and CIOs have a major role to play in terms of promoting the expected gains.






Add this page to your favorite Social Bookmarking websites
Reddit!! Mixx! Google! Live! Facebook! StumbleUpon! LinkedIn! TwitThis Joomla Free PHP
Last Updated on Sunday, 22 May 2011 00:29


0 #5 Bharathi Baskar.B 2015-03-23 10:07
Thanks For Your valuable posting, it was very informative. Am working inCloud Erp In India
0 #4 Ahmed Taher 2011-10-02 11:12
Alarming shortage of skills .. we get themn young and help them develop their skills though.

Nice piece thanks
0 #3 Hafez Awad 2011-07-29 16:48
Big data .. massive data .. will only happen with bug vendros IBM and Oracle .. maybe Teradata ..
But I want to say that not all smart decision making sucess stories are based on big/massive data. Many of those wonderful stories do not rely on big data as a foundation.
0 #2 Samy Mahfouz 2011-06-26 20:56
Very nice piece, thanks. So what do we collectively do in order to bridge the skills gap in the Middle East?
Is the academic world listening? What are your plans?
0 #1 Turam 2011-05-22 08:38
Right this is becoming a very important topic, I just noticed that IBM is going to invest $100 Million in big data research .. see it here:

Add comment

By commenting on our blog, you are automatically confirming that you have read and agreed to abide by our editorial policy found under About CIO2CIO - CIO2CIO Editorial Policy

Security code

Anti-spam: complete the task
  • Poll
  • Login
  • Search

Does your organization use big data?

Follow CIO2CIO on: