This article first appeared in Bloomberg Businessweek Middle East on June 21, 2018.
Where is big data at in the Middle East at present?
It’s important to understand that Big Data is really starting to hit its sweet spot for three big reasons:
Firstly, social media and digitization: the rate at which data is created today is unprecedented and growing exponentially. 90% of all data ever had been created just in the past couple of years. With the proliferation of IoT and sensors, these numbers will explode. So a lot more data is available.
Second, computing power and cost of computing: Computing power has been rapidly growing and with distributed computing and cloud computing capabilities, the ability to process big data has become immensely cheaper and feasible. So, processing of the data has become a lot more accessible.
The third reason is that Artificial Intelligence algorithms are maturing, providing data analytics capabilities like never before. With large scale collaboration enabled through ubiquitous connectivity, analytics toolkits have rapidly evolved and AI algorithms are providing new ways of interpreting and learning from big data, making it much more useful. So analytical capabilities have made big data outputs much more valuable.
And this is only the starting point. The growth is going to be white hot for the foreseeable future.
This view is not lost on corporates, nor on governments. And hence we see the faster moving governments and companies in the Middle East already actively working on Big Data initiatives.
Why is Big Data so important for government and businesses, especially in the Middle East?
Big Data and Big Data Analytics (BDA) is seen by many as the inevitable working model of the future. Customer and citizen segments are becoming more and more sophisticated. For businesses, for example, differentiation – and hence value in businesses – is increasingly driven by micro-segmentation and micro-targeting that BDA enables.
Much of the competitive threat for businesses is being driven by disruptors being able to target sub-segments better or more customized – and big data is a powerful counter to that.
In terms of operational efficiency, big data provides powerful optimization opportunities.
For governments, the ability to cost effectively deliver high quality service is greatly enhanced by Big Data Analytics. For example, smart metering, real-time data traffic analysis and cost tracking. Let’s look at each of these three a little more closely:
- Smart metering solutions: data on utility consumption levels at a household level can be continuously analyzed to identify patterns, and inform the implementation of dynamic tariffs
- Real-time data traffic analysis: to improve traffic routing by dynamically redirecting traffic flow based on data collected from intelligent traffic systems
- Cost tracking: the ability to register all public payments and analyze that information centrally can provide feedback on the most efficient use of funds, for example health
The ability to keep nations and cities safe and secure is also significantly upgraded with Big Data analytics.
Digital image analysis on terabytes of CCTV recordings can provide incident flagging, tracking of listed persons-of-interest, identification of suspicious patterns.
Surveillance networks and monitoring and the resulting data analytics can provide high-value advanced diagnostics to prevent breakdowns of key public infrastructure.
Advanced algorithms are becoming more and more effective in identifying financial fraud, suspicious transactions and unwanted financing activities.
A relatively new dimension of information is being added in to policy making, where social media analytics, large scale usage trends and other big data insights can be inputted in to the policy design process for governments.
In the Middle East, Big Data is likely to play a very important role, in some ways more so than most other markets across the world, for a number of reasons: Firstly, it’s a region in a state of flux. Major changes, major transformations and major policy reform cycles are underway in several markets – all of these can make healthy use of Big Data Analytics.
Secondly, security continues to be a major challenge and advanced surveillance analytics can help identify pain points better for more effective solutions.
And thirdly, for businesses, there is a lot of opportunity to innovate. Simple steps forward on big data can provide large dividends in terms of customer pull. A more customized travel plan, a more targeted insurance package, a better segmented home maintenance offer can all command significant premiums in GCC markets – often more than what they would in other markets.
Where are we at in terms of Big Data in the Middle East at the moment?
As per IDC, revenues for big data and analytics (BDA) in the Middle East and Africa (MEA) totaled $1.98 billion in 2016 and are projected to grow at an annual rate of 10% up to 2020. This is good starting point for the region given the market context. In comparison, the Asia Pacific region for instance registered $13.6 billion in revenues for BDA. The United States remains by far the largest market globally, with total revenues of US$78.8 billion.
While revenues tell one story, it’s important to also look at ambition and capabilities. Initiatives such as the ones taken by Dubai government to develop an AI hub in the region are expected to translate into the development of a thriving and technically advanced BDA market on par with global leaders. Independent analysis observes that BDA investment is picking up, and that regional executives plan to implement Big Data technologies on a large scale.
In many Middle East countries, there are certain advantages that are fueling the Big Data revolution and could drive it faster than most other markets in the near future. These include the faster government decision-making seen in several GCC countries which has allowed governments to launch new, well-funded initiatives at an incredibly short turn-around. This is expected to continue to be the case for multiple countries in the GCC block.
Many Middle East countries also have young, tech-savvy populations, increasing the efficacy of any big data initiatives. The target customer or citizen base is much more digitally switched on, providing more information that goes in to the Big Data engines, and also has greater affinity to the outcomes that big data applications provide – be it targeted offers or more customized service.
What are the main challenges associated with developing and using Big Data in the Middle East?
Talent gap is the primary challenge: Big Data Analytics, is above all a talent play. Typically, it takes decades to build strong ecosystems that attract global talent into technology hubs. Governments in the Middle East need to not only adopt new technologies but also develop new capabilities and skills. A part of it would need to be a resident local talent pool, and in the short run that would also mean the temporal import of a significant volume of talent.
GCC countries have shown great agility in the past in being able to bring the right talent set during high-need periods and they might need to do the same for Big Data. At the same time, it would be equally critical to retain a significant share of that talent in the long run to continue evolving the frontier of BDA in the region.
Another challenge is the legal framework: The underlying data used in BDA can be varied in nature, but often contains personal information. With large computing and storage capabilities, the accumulation of this information requires regulatory intervention to prevent abuses, and enable people to remain in control of their data and of how it’s being collected, stored and treated. The use of big data therefore needs to be adequately regulated, and governments are increasingly acting on this, with the EU’s General Data Protection Regulation (GPDR) a prime illustration of this concern. In the Middle East, the maturity of data protection laws varies significantly. While in some jurisdictions they can be weak or only apply to citizens and not expat residents, there are countries with comprehensive regulatory frameworks in place.
Infrastructure for data collection, processing, storage, and sharing: there is significant legacy to overcome on the data collection and sharing infrastructure. Governments and businesses will need to re-think their working models and revamp infrastructure and cultures to overcome the historical data challenges. That being said, we see very real intent across players to do this. Whether the right investments will be made in line with the intent, would be worth observing in the near future.
Cyber-security: With greater data comes greater threat of data-theft. Cyber-security will be a critical determinant of people’s faith in sharing data and in the value they see in Big Data applications. Corporations and governments will need to work hard and work together to mitigate and minimize the threat of cyber-attacks.
Can you give any examples of how companies in the region are leveraging Big Data in a particularly successful way?
In no particular order, I would highlight the following:
- Smart city initiatives in UAE or Saudi Arabia are actively leveraging big data, through ubiquitous sensors monitoring traffic, water and electricity consumption, air quality, at other metrics.
- Reports suggest that Dubai Airport is increasingly leveraging big data for its operations. It integrates real time data such as air-traffic, passenger flows, luggage transit and dispatching, ambient temperature etc. It also uses historical data at security control to predict passenger volumes, which can then be used to make processes more efficient during peak times.
- The Dubai Health Authority is said to be integrating real time data from the emirate’s hospitals, allowing awareness of bed-availability at each location. This can be invaluable at time of crisis when emergency services need to rapidly decide where to dispatch victims.
- Saudi Arabia has recently become a member of the UN Big Data Global Working Group alongside Eurostat, OECD and others
- Big data projects in companies are not always publicized to retain the competitive edge, but sources suggest new projects being internally launched for telecoms, banks, conglomerates and others. We should see more and more applications emerge in the next couple of years. The demand for data scientists in the region, appears to have increased manifold in the past three to four years
Globally, private sector companies working on Big Data include well-known global majors, but we also see a parallel trend to internalize the Big Data function for organizations which produce or have access to large data-sets such as telecom operators, credit card companies, transport and logistics operators, online retailers and oil & gas firms. This is manifesting itself through organic creation of dedicated departments, but also through M&A activity. For instance, last year we saw smart meter manufacturer Itron acquire Comverge, and pharmaceutical giant Roche buy Viewics, both software companies with cloud-based big data platforms. Big pharma, big oil and utilities are increasingly on the look-out for such opportunities, realizing the value they can bring to their organizations.
How many companies are typically involved in the end-to-end Big Data cycle?
There is no ‘typical’ number of companies involved. The big data value chain can incorporate a very diverse set of players depending on the Big Data application being considered, and also on which players are involved. Data-capture can require sensors, in the form of hardware; data-processing requires computing power and software, the latter which is often customized; data storage is again hardware, but increasingly involves cloud providers, or cloud infrastructure. And then there is the commercialization of that data, or of its application, involving the data marketplace, application provides, marketers and end users.
