IT Pro is supported by its audience. When you purchase through links on our site, we may earn an affiliate commission. Learn more

How the cloud can add value to Big Data

Early adopters are already demonstrating how to get more from analytics via the cloud

CGI representation of a cloud containing light, signifying data held in the cloud

Cloud computing and big data systems have without doubt been among the most disruptive technology trends of the past decade. These developments have irrevocably changed the way that technology companies operate, and how value is delivered to stakeholders.

With cloud computing, firms have been able to optimise IT operations through significant reduction of the need to invest in on-premise hardware and software, as well as provide businesses with more strategy options than ever before. Digital transformation and the shift to the cloud continues to show momentum, with the public cloud services market growing 29% from the previous year in 2021 according to IDC

Simultaneously, big data technologies have provided enterprises with an unprecedented avenue of data value from assets. In the past, high-volume, diverse data proved troublesome for enterprises that were used to working with traditional database technology. But great technical developments allowed businesses to reduce the overhead required to transfer raw data into sorted information within a data store, as well as drastically increasing the efficiency, accessibility and speed of mass data processing.

These innovations have allowed for actionable analysis on a range of once-challenging data sources, including web logs, documents and machine sensors. Even so-called ‘dark data' has been revitalised through these technologies.

Big Data systems help organisations solve the trickiest problems, but they generally require a significant up-front and ongoing IT investment, in addition to dealing with the sheer amount of data in more ambitious projects.

Related Resource

2021 Gartner critical capabilities for data integration tools

How to identify the right tool in support of your data management solutions

Whitepaper cover with title, text, dark header banner, and logoFree Download

Therefore, it makes sense that enterprises are turning to cloud providers with expertise in managing and maintaining scalable, and flexible, computing and storage infrastructure. While on-premise data systems are far from defunct, organisations are starting to effectively push the limits of big data analytics at scale by tapping into big data systems hosted on cloud infrastructure.

While many of these early cloud projects involve high volumes of structured data, there are 3 key components that are already enabling extraction of value from massive, diverse data sets on cloud infrastructure:

Cloud Analytical Databases

These cloud-based services, such as Amazon RedShift, are elastic data warehouses optimised for analytics with existing Business Intelligence tools. This type of analytical database includes management as well as solutions monitoring for the provider.

Hosted Hadoop Services

Hadoop clusters are often hosted in the cloud, which negates the need for on-premise infrastructure. Some Hadoop cloud offerings come with additional managed services, like job troubleshooting, software installation, testing and other such services.

Data Integration and Analytics

Databases built for high-performance analytics can make use of cloud techniques like compression, column-based storage and high-speed inserts of structured data, perfect for sophisticated queries and truly insightful business analytics.

Early adopters are already demonstrating how the cloud can improve upon the value proposition of big data, delivering elastic and cost-effective solutions for analysing data at unprecedented scale.

This article was first published on 12/09/17, and has since been updated

Featured Resources

Mastering retention

Turning user behaviour insights into retention strategies

Free Download

Dell PowerEdge with AMD

IT applications and infrastructure are the prime catalyst for new revenue creation

Free Download

Building for success with off-premises private cloud

Leveraging co-location facilities to execute your cloud strategy

Free Download

Cyber resiliency and end-user performance

Reduce risk and deliver greater business success with cyber-resilience capabilities

Free Download

Recommended

The secure cloud configuration imperative
Whitepaper

The secure cloud configuration imperative

7 Mar 2022
The secure cloud configuration imperative
Whitepaper

The secure cloud configuration imperative

7 Mar 2022
Hybrid cloud storage maturity model 2021
Whitepaper

Hybrid cloud storage maturity model 2021

1 Feb 2022
Hybrid cloud storage maturity model 2021
Whitepaper

Hybrid cloud storage maturity model 2021

1 Feb 2022

Most Popular

46 US states call for Meta monopoly lawsuit to be reinstated
mergers and acquisitions

46 US states call for Meta monopoly lawsuit to be reinstated

20 Sep 2022
Anonymous hacks Iranian government and state broadcasters
cyber attacks

Anonymous hacks Iranian government and state broadcasters

22 Sep 2022
Why collaboration is key to digital transformation
Sponsored

Why collaboration is key to digital transformation

13 Sep 2022