NinjaOne has announced a new integration with ServiceNow, in a move the IT operations vendor said will give IT teams greater visibility across their environments while simplifying IT service desk workflows.

The move will enable organizations to share real-time endpoint data between the two platforms, helping IT teams automate incident management, improve asset visibility, and resolve issues faster without needing to switch between applications.

Texas-headquartered NinjaOne’s unified IT operations platform combines endpoint management, patch management, backup, and remote access through a single console.

According to the vendor, the offering supports more than 40,000 customers across 140 countries.

By integrating the offering with ServiceNow’s service management platform, NinjaOne said it is addressing the information gaps that can emerge when organizations rely on multiple IT management tools.

“NinjaOne’s integration with ServiceNow eliminates this problem by continuously updating IT data in ServiceNow as conditions change, providing real-time accuracy, and ensuring devices remain healthy,” explained Rahul Hirani, chief product officer at NinjaOne.

“As a result, our customers save time and resources, simplifying daily operations so they can deliver best-in-class IT support and services.”

Real-time visibility and proactive support

The new integration enables endpoint data from NinjaOne to flow directly into ServiceNow, allowing organizations to automatically update asset information, configuration management databases (CMDBs), patch status, endpoint records, and other configuration data.

According to the companies, alerts detected within NinjaOne can also automatically generate incidents in ServiceNow to help IT teams investigate and resolve issues before they affect end users.

Additionally, technicians can access and manage devices directly from within the ServiceNow platform, further reducing the need to move between multiple management tools.

NinjaOne said the move will provide customers with a single, real-time view of their IT environment that will help eliminate manual updates and improve data accuracy across service management and endpoint operations.

Commenting on the partnership, Alix Douglas, group vice president of global technology partnerships at ServiceNow, added that the integration will help organizations adopt a more proactive approach to IT support.

“The most resilient IT organisations are the ones that eliminate blind spots, and that starts with connecting endpoint intelligence to service management in real time,” he said.

“NinjaOne’s integration with ServiceNow gives IT teams a unified, current view of their environment, so they can move from reactive firefighting to proactive resolution.

“That’s the kind of seamless workflow intelligence that drives real productivity and delivers better experiences for employees.”

FOLLOW US ON SOCIAL MEDIA

Databricks and Microsoft are forging deeper ties with an expansion to the duo’s decade-long partnership, with a fresh focus placed on data management.

As part of the move, Databricks said it plans to expand use of Azure Databricks in core business operations and build a new, unified data lakehouse.

The move will also see the firm adopt Microsoft’s Arm-based VM infrastructure, Azure Cobalt, to “improve performance and efficiency”.

Microsoft, meanwhile, plans deeper integration of Databricks’ Data and AI platform across core product lines in a bid to provide customers with new tools such as Genie, the company’s flagship AI chatbot.

Databricks CEO and co-founder Ali Ghodsi said the expansion builds on long-standing ties and aims to accelerate customer data transformation efforts.

“For nearly a decade, Databricks and Microsoft have helped enterprises innovate with data and AI,” he said.

“Today, our partnership is stronger than ever. With Databricks Genie and Unity AI Gateway deeply integrated across Microsoft’s products, we’re helping enterprises unify their data and ground AI in business knowledge.”

Delivering business context

The partnership expansion aims to address key recurring hurdles for enterprises dabbling in AI innovation: data preparation and management.

AI systems, particularly agents, require fine-tuned data grounded in domain-specific context, according to the duo. Simply put, this means AI systems need to understand enterprise products, customers, operations, and general business processes to run efficiently.

A significant portion of enterprises are still struggling to bridge these gaps, however, and it risks derailing adoption processes, according to Judson Althoff, CEO of Microsoft’s commercial business.

“The next generation of AI will be defined by how effectively organizations turn their unique knowledge into intelligence,” Althoff said.

“Microsoft and Databricks are helping customers connect data, AI, and business context to accelerate decision-making and drive measurable impact.”

Databricks targets performance gains

A key focus of the partnership rests on expansion of Azure Cobalt for Databricks. The company said this will help improve performance and efficiency for agentic AI and ‘data-intensive’ workloads for customers.

The data analytics firm currently uses Cobalt 100. Looking ahead, the company plans to adopt Cobalt 200, which Microsoft noted offers up to 50% better performance and includes memory encryption by default.

Expansion of core business operations through Azure Databricks will also deliver performance improvements, the duo noted, enabling the firm to scale up capabilities for customers.

Genie integration

Deeper integration of Databricks’ Genie AI service within Azure also aims to simplify data management capabilities for customers, according to the firm.

Running the AI system directly within Microsoft environments will enable customers to ground and operate agents on data hosted within the cloud service. This means the AI tool will be “deeply integrated” with key Microsoft products, including:

• Azure Data Lake Storage
• Power BI
• Microsoft Foundry
• Microsoft 365, Teams, and Copilot

“These capabilities bring governed, real-time data and AI into business workflows, giving organisations the context, control, choice, and cost efficiency needed to drive impact,” the company said in a blog post.

FOLLOW US ON SOCIAL MEDIA

Data is treated as a source of untapped riches for businesses, promising a world of better insight and easier management. What's less discussed is how it's possible to drown in this ocean of information – more data sometimes just means more noise, and this is as true in IT as in any other profession.

Why doesn't more data produce better results? Is there a better way forward? And what does this mean for the end users? On this episode of the ITPro Podcast, Bobby Hellard and Ross Kelly talk to Fernando Castanheira, the CIO of Riverbed.

Castanheira's career has struck a balance between tech and finance, and how data is used to power businesses. He talks us through the mismatch between the amount of data available and how it's being used.

Highlights

"If I'm a service desk person, my job is to kind of make sure that everyone's able to do their jobs. There's no interruption, etc. I can come in and say, we have just introduced our third iteration of AI. We call it a Q. Q is basically you're going away from dashboards, which everybody's got to go look at dashboards, to come across as more conversational. I can come in and say, "Hey, I'm the desk guy. What are the three or four items that I should focus on today so that I can improve my you know my my environment? And then based on the data that we've collected over the last 24 hours, a week, whatever it is, we come back with some suggestions."

"These dashboards all look green, right? All look green on the outside, but you open things up on the inside, a lot of red. Because a lot of things are not working, even though the dashboards are looking green. So I think our approach is to make sure we kind of solve that by having very specific data, very specific recommendations, or very specific automation."

"The key thing is to have that foundation in place, and then we can act and enable things as you go forward, but it's a journey. You don't like go from zero to 100 overnight. You start with the easier stuff and then continue on that journey."

Links

• Data quality worries are holding back AI adoption among manufacturers, despite optimism over its growth potential
• AI projects are faltering as CDOs grapple with poor data quality
• Everpure wants you to get your data AI-ready

Stephen Wood, chief operating officer at Rathbones Asset Management (RAM), faced a challenge. His organization wanted to build strong data foundations to help deliver fresh insight to its professionals and embrace emerging technologies, such as agentic AI.

The solution to this challenge came via technology specialist Snowflake’s data platform.

“We wanted to create a golden source of data for each of our foundational domains,” he says.

“We wanted to overlay the data quality checks, the business rules, all those good things, and then everything would be built off that data lake. That's where we found ourselves working with Snowflake’s AI Data Cloud.”

Taking a long-term view

Wood, who spoke with ITPro at the recent Snowflake Summit 2026 conference in San Francisco, says this process began two years ago, when RAM, which is part of the Rathbones investment group, was looking for a way to manage its information assets more effectively.

The shift to Snowflake is part of a broader digital transformation to support business growth. The organization’s assets under management grew quickly before the move to Snowflake. However, RAM was still using a portfolio management system that was customized for the wealth management element of the wider Rathbones group.

Fortuitously, a solution came from within the broader Rathbones group. The firm was in the first phase of its Snowflake deployment, and Wood suggested to senior executives that the technology could help his organization manage its data headache:

“We said, ‘Look, we can take advantage of this technology,” he adds. “We want to lead the way in terms of building out our core foundational data domains in a Snowflake environment.’”

Transformational stages

Stage one of the transformation project involved a point-to-point data share between the existing Charles River investment management solution and the Snowflake platform. Wood says this data-sharing process covered a trading view of the organization’s funds, such as securities bought and sold.

Stage two, meanwhile, involved building out RAM’s data store and the static reference information for its funds. Wood says this process allowed his team to build the Open Funds template, which external vendors use as a standard for consuming the organization’s data.

The success of this initial work meant senior executives could see further opportunities, with Wood adding: “Then we started to get into conversations where we were saying, ‘Okay, now we need historical fund performance, our quant risk calculations, what we call subscriptions and redemptions, and the cash flow in and out of our funds from clients.’”

Today, RAM has nine domain-related data sets established in its Snowflake data platform. These data sets cover a range of areas, including performance, risk, sustainability, and unstructured information, such as the decisions behind the firm’s investment activities.

Foundation for future success

Wood says the integrated data sources provide a foundation for change, including potentially embracing emerging technology. Work undertaken so far means his team can start exploring AI, including Snowflake’s agentic services. At the San Francisco event, Snowflake announced new features for CoWork, its personal agent for knowledge workers, and CoCo, its coding agent for developers and data engineers. Wood can see the agentic technology’s potential.

“When you combine all the information, you can start to do some powerful things,” he says.

“We're now in a scenario where we can say, ‘Looking over all this data, we can start to analyze the performance of our investment decisions and give product specialists new insights.’”

RAM’s agentic explorations are already underway. The organization has a test environment for CoCo and CoWork that’s running with millions of rows of data. “At this stage, the outputs look very decent, so it's a near-production environment,” Wood says of the explorations, before outlining other potential applications.

He adds: “The future direction is that you start to get into sales and distribution. The most valuable data in our CRM system today is the call notes written up after every meeting that say clients are interested in some funds and pulling away from others. No one could analyze that data until now, because, after you've had two or three calls with the client, the information disappears down the chain in Salesforce.

“Now we're looking at the idea of using agentic technologies to consider the long-term buying trends of clients. The salespeople know their clients, and they know what they're buying. But if we see the general trends, then that insight can go into a product cycle that makes us think, ‘Do we need to be looking at new funds, do we need to broaden our horizon? Are we at risk in certain areas?’”

Fuelling innovation

The work Wood’s team has undertaken means they can investigate native applications and emerging technologies with confidence, rather than focusing on point-to-point data shares with Snowflake. That investigatory work is crucial because, as a smaller business, RAM will never have a team of engineers to build out a big AI platform.

“That approach doesn't make any sense,” he says. “What does make sense is that we've got Snowflake technology that is central to our data strategy. Now, our work is about how we can find the tools that interplay with the platform, so we reduce the Frankenstein's monster of connectivity and integrations that can slow projects down.”

Wood reflects on the digital transformation program he’s overseen and says that, as with any financial services firm, the key challenge is ensuring exploratory projects are prioritized in an industry that often contends with a regulatory burden and technical debt. Sometimes, due to these intractable challenges, innovation gets stifled because of people and budget constraints. Wood says his organization has circumvented these issues.

“I think Snowflake, and how Rathbones operates, has allowed us to take control of our environment,” he says, referring to the split between the group implementation of the platform and his organization’s deployment. “There's a broad Snowflake environment, and then we've got a ring-fenced environment, with all our data sets, which means the implementation has been built in a way that allows more self-service capability.”

Wood suggests the key to long-term success will be staying on top of Snowflake’s product advances. For example, his team received buy-in for its CoWork and CoCo explorations by explaining how a trial of the agentic technologies could benefit the group.

“There are good ways to try and ensure that you get that prioritization if you need to,” he says.

“When you've got a real-world business case and value proposition that is going to materially benefit the business, versus just being a nice technology idea, the investment ultimately comes through.”

Amazon Web Services (AWS) has announced a series of changes to Amazon OpenSearch aimed at streamlining data log analysis.

A new purpose-built log analytics engine for the service will provide users with marked performance gains and up to 70% lower storage costs, according to the firm.

OpenSearch allows users to search, analyze, and visualize data in real-time, and is typically used for application monitoring, observability, and AI/ML search applications.

The hyperscaler said the update comes in direct response to surging volumes of log data handled by enterprises in 2026, largely due to AI.

Research from AWS shows that log volumes have grown between 30-40% year over year, which is stretching budgets and creating bottlenecks.

Amazon OpenSearch overhaul

As part of the update, Amazon revealed OpenSearch will now store data in the Apache Parquet format. This is a column-oriented data file format designed specifically for more efficient data storage and retrieval purposes.

The service previously relied on inverted indexing. Amazon noted the shift will deliver significant efficiency gains and faster search and retrieval capabilities.

“Apache Calcite parses and optimizes each query, then routes operations to the engine best suited to execute them: Apache DataFusion for analytical operations on columnar data, or Lucene for search predicates,” the company explained in a blog post.

“The two hand off mid-query, so a single query can search log content and aggregate the results without additional roundtrips.”

Amazon noted that OpenSearch now also supports the Piped Processing Language (PPL) and SQL query languages, both of which “execute natively” through the platform.

Amazon eyes optimized log analytics

From a performance perspective, Amazon noted users can expect significant improvements alongside lower costs.

OpenSearch now boasts up to four-times price improvements compared to the existing engine, based on internal benchmarks.

Users can also capitalize on two-times faster analytics queries due to the fact the engine processes data in columnar batches, even across larger data sets. A two-fold improvement on ingestion rates also aims to reduce bottlenecks.

“The optimized engine sustained 1.78 million documents per second at matched concurrency, approximately 2x the throughput of the Lucene baseline, while consuming less CPU,” the hyperscaler explained.

“Both domains ran with zero write rejections. For teams ingesting terabytes per day, the throughput advantage translates to fewer nodes for the same volume, or longer retention on the same infrastructure.”

How to access Amazon OpenSearch

The new log analytics engine is available now for enterprises in AWS regions where OpenSearch Optimized instances are supported, the company noted. Similarly, there will be no additional premium for the optimized engine.

It’s worth noting that the new engine is a setting selected at creation time, meaning users can’t add this to an existing domain or enable it within individual indices or fields within a general-purpose domain.

“To adopt the optimized engine, create a new domain and migrate your ingestion pipelines into it,” the company explained.

“Create a new domain in the Amazon OpenSearch Service console and select Observability as your use case. The optimized engine is enabled by default.”

The in-built console will provide users with a side-by-side comparison of capabilities based on individual needs.

“After your domain is ready, ingest JSON documents through the same Bulk API and client libraries you use today. No changes to your ingestion pipelines or application code are required.”

FOLLOW US ON SOCIAL MEDIA

Everpure has announced the launch of a new Data Intelligence platform aimed at streamlining storage and data management capabilities, marking its latest post-rebrand pivot.

Unveiled at the company's annual Pure Accelerate conference in Las Vegas, the new platform aims to simplify data visibility, control, and management.

The platform boasts three key capabilities, according to Everpure, including "universal" data discovery features providing heightened visibility into structured and unstructured data, regardless of storage format, the company said.

Elsewhere, automated governance features aimed at bolstering compliance capabilities, along with data mapping to drive "AI-ready context" for agents, were also touted as key differentiators by Everpure.

"Everpure Data Intelligence discovers, classifies, and contextualizes enterprise information at its source," the company said in a blog post.

"It works on all data, inclusive of the Everpure Platform, public clouds, SaaS applications, and third-party storage."

Unlocking "trapped meaning"

A key focus for the Data Intelligence platform lies in unlocking the trapped meaning in enterprise data, which is often hosted in rigid applications or siloed storage infrastructure.

Analysis from IDC earlier this year identified data siloes as one of the key hurdles to enterprise AI innovation, and Everpure's position on this front is that they're missing out on vital context by failing to centralize and consolidate data sources.

This 'context' directly impacts AI outputs, with models and agents relying on poor quality data, separate research from IDC found. The vast majority (94%) of respondents to a recent survey found data quality is "important or very important to AI project success".

Notably, redundant data, siloed storage architectures, and obsolete data were ranked among the three key contributors to poor data quality.

With agentic AI adoption continuing at pace, the need to feed agents accurate and relevant data is now critical, Everpure noted, helping to ultimately improve response accuracy and thereby reducing token costs.

Everpure's data management pivot

The move by Everpure builds on its recent acquisition of data management firm 1Touch, embedding key features and capabilities of its own platform.

In a briefing with assembled media ahead of the event, company executives noted the announcement marked the latest shift in its data management pivot. Everpure first signalled its intention on this front at last year's conference with the launch of its Enterprise Data Cloud (EDC).

That's not to say storage isn't still the company's key focus, but rather represents a shift based on how enterprise storage architectures are changing in the AI era.

Prakash Darji, general manager for Everpure's digital experience business unit, told assembled media that "data readiness" is now a key focus for enterprise customers.

Meanwhile, traditional application-centric architectures, which saw data fed into centralized applications, often trapped critical data and context. Darji highlighted specific functions, like sales or finance, as key examples here.

"Architectural paradigms have fundamentally started with a single building block over time, and that building block was an application. The data comes from the application, like an ERP or CRM system," he explained.

"The context, the business rules, the pricing logic, everything's encapsulated within that unit of architecture."

The result here for enterprises is that they're regularly faced with data bottlenecks and blind spots, requiring a shift toward a "data-centric" architectural model, referred to by the firm as a "data primacy model".

"In a data primacy model, information is liberated from individual applications to become a shared and governed system of record," the company noted in a blog post.

A key focus here is to ensure that data is "self-describing" and will "carry its own meaning and logic".

Rob Lee, chief technology and growth officer at Everpure, said this represents flipping these traditional architectures on their head – and it comes at a time when enterprises are increasingly entering an AI-accelerated world.

"In a data primacy world, what we're saying is 'hey, Mr and Mrs customer, take control of your data, you own your data, you own how to store that and how it's used, and then the applications need to fit into that," he told assembled media.

"Data readiness" has quickly become one of the key challenges facing enterprises dabbling in AI innovation, according to research from IDC, and Everpure wants to help enterprises sharpen up their capabilities.

The flash storage and data management company confirmed the general availability of a new platform aimed at streamlining paths to production for AI, helping them prepare and refine unstructured data for use in AI applications.

Data Stream, which was first announced at Nvidia GTC, is based on the Nvidia AI Data Platform design and aims to reduce raw data preparation from "months to minutes".

Speaking to assembled media ahead of the event, Everpure's chief technology and growth officer, Rob Lee, said the solution fundamentally aims to make data "more usable" and cut down on what has traditionally been an extensive, manual process.

"One of the problems that we're trying to solve is, how do we automate, how do we make just the whole process of enterprise customers tapping into their vast amounts of data and making that more usable, more easily plumbed into AI workflows," he said.

Lee noted that Data Stream will allow enterprises to connect "all their data sources" and data streams, with the platform thereafter automatically ingesting data, curating it, classifying it, and indexing it.

Data Stream will "basically automate the process, which today is altogether too manual".

Data readiness in the age of AI

The move by Everpure comes amidst a sharpened focus on data management for the firm, following a recent rebrand and acquisition of data intelligence firm 1Touch.

Everpure unveiled the Data Intelligence platform at its annual conference in Las Vegas. And, like Data Stream, it aims to simplify and streamline data management and bolster AI readiness for firms.

A common recurring hurdle for enterprises on this front has been data quality, per IDC research. Analysis from the consultancy found 94% of IT leaders identified this as "important or very important to AI project success".

Similar research from Riverbed also highlighted data quality as a major factor in AI project failures. In a poll last year, 87% of respondents said that high quality data is critical to AI success, yet more than two-thirds (69%) said they had serious doubts over how effective their firm is at maximizing the use of data.

The Everpure Data Intelligence problem addresses this challenge directly, as ITPro reported, helping enterprises to map data dependencies and therefore providing AI systems with relevant, contextualized data.

On a broader front, both Data Stream and the Data Intelligence platforms could play a vital role in helping enterprises push AI projects into full production – an area which many have struggled with in recent years.

Research from Dynatrace in January this year found roughly half of agentic AI projects were stuck in proof of concept (PoC) stages, with IT leaders failing to push initiatives into full production.

Snowflake and Anthropic have announced an expansion of their strategic partnership aimed at driving broader AI adoption for enterprises.

According to the duo, the move comes as both report increased demand from enterprises looking to move AI from pilot projects into full production.

The initial partnership, signed in December last year, saw Anthropic’s Claude models integrated within Snowflake’s Cortex AI platform.

This allowed customers to use Claude with Snowflake data through the platform, helping to streamline access, support agent deployments, and provide users with a range of safety and governance controls.

So far, the partnership has proved beneficial for customers and highlights a growing demand for AI that “works directly on their governed data, not in isolated systems”, according to Christian Kleinerman, Snowflake’s EVP of product.

“We’re seeing strong demand across our AI products, with Snowflake Cortex Code becoming the fastest growing product in Snowflake’s history,” he said.

“Together with Anthropic, we’re helping organisations move from experimentation to product faster and laying the foundation for the agentic enterprises, where AI, data, and governance work together to drive real business outcomes.”

Snowflake and Anthropic target deeper integration

Building on this, the duo has expanded access to Claude within the Cortex AI platform, now enabling enterprises to build AI agents and applications directly on in-house data.

The Cortex Agents framework, for example, allows customers to build custom agents spanning a range of use-cases, including customer support, data analysis, and core operations.

Elsewhere, Claude will also now power Snowflake Cortex Code and Snowflake Intelligence, the companies confirmed.

According to Snowflake, a host of customers, including Block, have recorded marked improvements using Claude across its core platforms.

Block engineering lead Arnaud Weber said the company has built an “AI-native operating layer” which connects data across different brand ecosystems, spanning CashApp, Afterpay, and Square.

Weber noted this has helped bolster security, data governance, and developer efficiency.

“By combining Anthropic Claude with Snowflake’s governed data platform, our teams can investigate compliance and security issues in real-time, trace controls and requirements, surface operational insights, and automate workflows grounded in trusted enterprise data,” Weber commented.

“Developers are also using Snowflake Cortex Code to build and operationalize these capabilities directly within Snowflake, creating a unified layer where AI can move seamlessly from analysis to action.

“This approach helps us reduce friction across investigations and decision-making, while maintaining the governance, performance, and scalability needed to apply AI responsibly across financial services and commerce.”

Moving from pilot to production

The continued collaboration between Anthropic and Snowflake targets a recurring pain point for enterprises - mainly the inability to move projects out of the pilot stage and into full production.

Research from Google as far back as 2024 identified this as a key challenge for enterprises dabbling in AI adoption, describing the trend as “pilot purgatory”.

Crucially, the situation has remained much the same over the last two years. Analysis from Informatica in January 2025 found just over one-third (38%) of companies had successfully transitioned AI projects into full production, for example.

This problem has been further exacerbated with the advent of agentic AI, with integration of agents proving far more complicated. Research from Dynatrace in January this year, for example, found around half of AI agent projects are still stuck in proof of concept (PoC) stages.

A key factor here, the study noted, lies in the fact enterprises fail to efficiently govern agents, which prevents them from scaling projects safely. More than half (52%) of IT leaders highlighted concerns around security, privacy, and compliance as key hurdles.

FOLLOW US ON SOCIAL MEDIA

Alex Read, senior enterprise product manager for data at energy provider EDF UK, is charged with helping his organization make the most of its information assets. Perhaps unsurprisingly, the hottest technology topic and the issue his bosses raise the most, is exploiting AI.

“It’s at a point where there are visions at an executive level of where AI is infused throughout our technical services and capabilities,” he says. “Our job as data and technology experts is to understand where we can deliver the most value, and also make sure that what we're doing, from an enablement perspective, is designed to be adopted and operationalized at scale.”

Read says it’s a responsibility he relishes, describing the demands of his role as fast and varied. As part of the enterprise IT function, Read helps over 1,000 users across the business to make the most of the EDF UK’s data tools. His team sits at the heart of a federated hub-and-spoke model, where they focus on enablement and business units concentrate on value.

“I'm accountable for the tooling, architecture, and common services-type capabilities in data and analytics,” he says. “Then in the federated business units, they're on the hook for data products and value extraction. They have teams of data engineers, data scientists, and ML Ops engineers, who are building data, AI, and ML products that create value for EDF UK.”

Read adds the federated approach gives his business “the best of both worlds” – one data platform across EDF UK that creates a consistent approach to information, and product teams in organizational functions that are naturally closer to business value.

“That's the sweet spot we've found,” says Read, who explains how the Snowflake AI Data Cloud is the heart of his organization’s data approach. The platform provides a single, consistent one-stop shop for hundreds of potential data sources.

“The data needs to be available at a cadence that can be used to operationalize near-real-time use cases,” he says. “So, we've got things like dynamic tables and event-driven architectures that enable that data to be brought onto the platform.”

Snowflake forms part of an integrated data stack that includes dbt, Matillion, Amazon Bedrock, and Sagemaker. Read says people across the organization can use the tools within the stack depending on their role, function, and projects: “Data stacks serve multiple personas – that’s the easiest way to describe it.”

With solid foundations in place, Read’s team has started enabling the structures that allow the rest of the business to exploit AI and ML. The next step is to ensure enterprise data is high-quality and well-defined so that emerging technologies know its context.

“We’ve got Snowflake services we're using, like Semantic Tables and Horizon Data Catalogs, whereby we're defining our data assets and making sure they're understandable to AI agents,” he says. “And then, we're building capabilities via Cortex Analyst and Cortex Code on top of that platform.”

The results of this construction process are becoming visible across the organization. The data platform already supports data initiatives across EDF UK’s Retail, Business, Wholesale Markets, Finance, and Zero Carbon Homes divisions. In Wholesale Markets, the company is rebuilding its volume forecasting platform, the analytics engine behind purchasing and hedging decisions in an energy market worth more than $13.53 billion (£10 billion), on Snowflake.

“That's about taking data from industry flows, weather providers, and internal retail and business solution consumption patterns,” he says. “We're essentially building forecasts in AI models, whereby we will be able to forecast the amount of energy required across the market. As these are such big numbers, even small percentage improvements in the margin of error and your ability to forecast, price, and hedge can create significant financial benefits.”

While those data-enabled efforts take place behind the scenes, Read’s team is also pursuing significant customer-facing initiatives. One major driver for these initiatives is the Market-Wide Half-Hourly Settlement, a program of industry reforms led by energy regulator Ofgem, which aims to provide a more accurate picture of electricity consumption across the UK.

Read says the regulations have inspired EDF UK to engage with data and explore innovative billing techniques. FreePhase, for instance, offers three pricing bands that reward customers who shift their usage to lower-cost periods. “You can't create those kinds of products without the ability to access data at a fine granularity,” he says.

Another example of an innovative billing technique is the Sunday Saver Challenge, EDF UK’s smart meter-enabled program that rewards customers with up to 16 hours of free electricity on Sundays when they shift weekday energy use from peak demand periods, easing pressure on the grid.

Customers contact the firm with questions about the challenge, such as its rules and benefits. Now, Read is using agentic AI to help service center staff deal with these queries effectively. His team is developing an agent using Snowflake Cortex technology that pulls information from the data platform and presents insight in the staff’s Slack user interface.

“The agent is being built as we speak, and is pretty close to being productionized on Cortex,” he says. “That agent will automate a lot of the questions being answered by our energy specialists. A customer query will be automated against Snowflake. So, essentially, they’ll be talking to the data on the platform to answer questions.”

At a time when many businesses struggle to turn agentic explorations into production services, the developments from Read’s team demonstrate the potential power of AI to boost internal operations and customer services. The key lesson for other digital leaders is to spend time on delivering the right outcomes.

“Snowflake comes highly integrated out of the box, and a minimal number of vendors are required to get to value,” he says. “And that's what our focus has been, and that has been the key lesson from this transformation – remove any barriers that get in the way of delivering value quickly.”

The result is that a business eager to reap the benefits of AI-enabled innovation is beginning to adopt and operationalize data-powered services at scale. Read reflects on progress so far and paints a picture of data and AI-enabled services at EDF UK in the future.

“Snowflake will be serving near real-time use cases via its platform with data that is of acceptable quality, defined, catalogued, and accessible to AI models, which have touchpoints out into the wider operation,” he says. “People of all skill sets and personas will have AI tools in their hands, fed by Snowflake. I think that's what good looks like for us.”

One of the oldest pieces of data we have is a customer complaint about substandard copper, engraved on clay. At around 3,500 years old, it’s fascinating to see a) what the day-to-day lives of people were like millennia ago, and b) how well preserved the clay is.

However, will future generations look back at the data we’re creating and feel the same way?

According to Statista, the amount of data created, consumed, and stored worldwide is projected to increase from 149 zettabytes in 2024 to 394 zettabytes by 2028. That’s the equivalent of 394 billion 1TB storage drives worth of data – a mind-blowing number.

AI-generated content and images are adding to this mountain of data at a phenomenal rate. Enterprises are struggling to cope with the exponential growth in data storage demands. The channel is uniquely positioned to help here.

Balancing the creation and storage of data

Consider the amount of data that a single high-resolution security camera can generate while in operation 24/7. Add to this all of the data created by IT systems for security, resilience, or regulatory reasons, and it can feel an insurmountable problem for companies to deal with.

What’s more, as channel partners are aware, many organizations are only now starting to deal with AI-generated data, and it’s clear that a scalable and flexible strategy is needed to cope with the modern-day data deluge.

While it’s doubtful that future generations will look back at the data we’re saving with fascination as we do with the clay tablet, it is generally accepted that we need to keep as much data as possible in case it is needed in the future. This could be for regulatory reasons, records, or following, say, a security incident.

Dealing with the data deluge 

Here are some strategies that channel partners can help organizations implement to better handle these challenges:

• Hybrid placement and mobility: Most large organizations have a hybrid approach to cloud, with some data sets on-premises and others in the public cloud. It is key to understand the various implications in terms of cost, security, and resilience, and to deal with change as soon as an issue is identified, because as data grows, it will only become harder to move. One should always ask, “Are my decisions still valid if this dataset grows 10x?” While 10x might seem a lot, a growth rate of 40% equates to 10x growth in just seven years. It’s also important that data can be moved as easily as possible without severe egress charges.
• On-demand consumption for new requirements: on-demand consumption of on-premises storage systems can help deal with not just unplanned requirements, but also understand the current needs and growth profile of new applications and the associated data sets.
• Security and the growing need for data resilience: The rise of ransomware attacks, as well as broad regulatory requirements, has resulted in a stronger focus on data resilience. This can result in greater data growth, as more resilience generally means more copies of data and more systems to track and manage them. Look for solutions that provide cyber data resilience with a low overhead in terms of storage requirements but also with fast recovery, as resiliency requirements call for ever-lower recovery time objectives (RTO), generally beyond what traditional backup solutions can provide. 
• Sustainability: power efficiency of data storage systems, as well as their “carbon cost,” needs to be taken into account. Power efficiency should be evaluated as a function of capacity per watt and performance per watt, while carbon cost should include the entire lifecycle, including manufacturing, transport, and decommissioning.
• Long-term archiving: keep your eyes wide open when evaluating archiving and long-term data retention solutions (whether on-premises or in the cloud) as their cost profile will be very different if data is accessed (even rarely) versus completely cold. Take into account the total cost based on a realistic ratio of data being accessed. That includes retrieval costs for cloud object storage, but also off-site storage fees and physical retrieval, transport, and time to restore for media such as tape.
• The special case of video data: Video makes up an estimated 50% of the world’s data, so it makes a lot of sense to employ specific strategies to deal with it. Advances in the field of data compression, sampling, and optimization in general have helped keep the storage requirements as low as possible; data access requirements have also changed. Traditionally, a lot of video data was an archive, but advances in AI mean that people want to analyze and understand it. High-capacity flash storage can facilitate both cost-effective storage requirements while allowing data access with the performance levels required for AI on video.

How the channel can help overcome the data deluge

What enterprises need most of all to manage the data deluge is expertise and advice in order to form a clear strategy. Here are three ways the channel can add real value in relation to the challenges posed by AI and data storage:

1. Become a strategic adviser on AI and data storage, demystifying technologies and helping businesses understand the impacts that AI initiatives will have on their business.
2. Assess data workloads and AI readiness, helping customers understand data types and volumes (e.g., structured vs unstructured), AI use cases and growth projections, and recommend high-performance solutions based on needs. Consideration should be given to scalable object storage and storage-as-a-service (STaaS) solutions, which have distinct cost and scalability advantages.
3. It’s not all about AI. The trend towards exponential data growth was underway before generative AI arrived on the scene. Customers also need advice on how to maximize the efficiency of data storage. One way to achieve this is through high-density flash storage technologies that enhance storage density, allowing for higher capacity and better space utilisation. Further efficiencies can be gained through data storage technologies that utilise always-on data deduplication, which eliminates the need for post-process scanning, reducing latency and increasing performance.

The channel must remain relevant as data storage solutions evolve

Now more than ever, there’s an opportunity for the channel to play a key role in the way customers deploy data storage solutions for the AI era. The channel must stay on top of rapid advances in this space. For example, we’re now seeing innovation in the field of ceramic storage – a technology that has proven its ability to protect data - even the most mundane customer complaint - over millennia.

As such, to ensure your customers’ data storage is truly future-proof, look for data platforms that enable flexibility and are really able to tackle today’s data access, security, and resiliency requirements.

Being a Managed Service Provider (MSP) no longer means providing reactive support or solving clients’ technology issues as they happen. Instead, the most successful MSPs now mine the data available to them to anticipate their clients’ requirements, optimize their business operations, and proactively address challenges before they arise.

More business processes than ever before live on digital platforms. Remote Monitoring and Management (RMM) and Professional Services and Automation (PSA) platforms are ubiquitous in the MSP space, while some providers also have separate Customer Relationship Management (CRM) systems, billing, accounting, and finance software, and IT documentation solutions.

Together, these tools hold a wealth of data that can give MSPs valuable insights into how well their business is doing, as well as crucial information on how to improve client acquisition, service delivery, profitability, help desk productivity, and more.

Too many MSPs, however, never find the time to review these metrics. Overloaded with the day-to-day running of their business, they miss the chance to take a step back, analyze the data available to them, and act on it. Business decisions are made on gut instinct rather than hard evidence. This reliance on guesswork comes with risks and lost opportunities.

Flying blind

A lack of insight is particularly evident in sales and marketing, traditionally a weak spot for many MSPs. Although most have a CRM system or use CRM features in their PSA platform, only a few actually study the data. As a result, an MSP may not know where their sales come from, whether their marketing campaigns have been successful, or whether their lead generation activity is delivering. They may be able to track the number of leads but have no insight into lead quality, or no understanding of the cost of acquiring new customers.

Without this awareness, much of an MSP’s growth planning will be based on speculation rather than hard facts.

The issue is not a lack of data. It’s about not taking the time or making it a priority to query it. The data could, for example, reveal that it would make more business sense to outsource certain lead-generation activities or automate marketing processes.

Equally, analyzing support tickets alongside customer retention and satisfaction could flag risks of customers churning as well as upsell opportunities, allowing MSPs to act early on these to strengthen customer relationships and protect or even boost their bottom line.

How to become more data-driven

Mining the data already available to them to understand strengths and weaknesses and pinpoint areas for improvement could be a game-changer for many MSPs. Better business insights lead to better decisions, more predictable growth, and ultimately, a healthier business. Plus, keeping a close eye on service desk activity can help predict and pre-empt client needs and measure outcomes, in turn allowing the MSP to prove the value they deliver.The most data-driven providers already use analytics, automation, and AI insights, as well as industry benchmarking, to be smarter. They base business decisions on verified facts, regularly review key performance metrics, and hold their teams accountable to agreed standards.

Deciding what data to track will depend on each MSP’s individual circumstances. A good starting point is focusing on metrics related to business health, such as growth rates and monthly recurring revenue (MRR) projections, productivity statistics such as ticket close rates and time spent on each ticket, sales and marketing success figures, and financial data.

Evaluating these numbers and comparing them to how peers perform will allow an MSP to identify areas for improvement, verify perceived issues, and review whether their current business strategy needs adapting.

Where the data displays unexpected patterns or unusual trends, such as a spike in help desk requests or recurring service tickets, it is worth carrying out root cause analysis. Does the data suggest bigger problems down the line? Are there bottlenecks that could be removed? Does a client’s infrastructure need an upgrade? And, are there any recurring tasks that can be automated or supported by artificial intelligence (AI)? By using data insights to anticipate issues and optimize labor-intensive tasks, MSPs can free up valuable engineer time.

Adding AI to the mix

For MSPs who are not sure where to begin, AI can be helpful in deciding which KPIs to monitor. From reconciling and modelling financial data to summarizing support ticket histories and finding patterns in the smallest details, GenAI tools are powerful at analyzing and making sense of data. By interrogating AI, MSPs can review where their challenges are, diagnose business symptoms such as cash flow shortages or declining profit margins, and determine relevant statistics to track.

Many MSP tools already have built-in AI capabilities as well as consoles for monitoring vital metrics and creating automated alerts. In addition, customizable dashboard solutions help aggregate data from multiple sources. Once established, keeping an eye on the important business parameters won’t take much time. The key is to decide where to focus – and to keep the desired business outcomes in mind.

The question should never be whether there is time for data analysis, but how to make it a priority. Those MSPs that invest the time and resources will gain a significant competitive advantage by using the insights to operate more efficiently, fuel growth, and deliver a better service.

AI has given hackers tools to wage more sophisticated attacks – and they're doing it faster than ever.

To help companies fight back, Databricks has unveiled a preview of its agentic SIEM (security information and event management) tool that's powered in part by Anthropic's Claude models.

The average time from spotting a flaw to developing an exploit has fallen from 23 days last year to 1.6 this year, the company noted, citing data from ZeroDayClock.

That means manual testing is no longer fast enough, and such techniques become a bottleneck. Research from CrowdStrike backs that up, showing AI-enabled attacks are up 89% from last year – and they're happening faster than ever.

"Security teams can no longer rely on manual workflows to outpace AI-driven attacks," said Ali Ghodsi, co-founder and CEO of Databricks.

"With Lakewatch, we are giving enterprises a new open data architecture and agentic capabilities to replace stagnating SIEM tools," Ghodsi added. "Defenders must have even better visibility and speed than today’s agent attackers."

AI is changing the game when it comes to security, and has led to a "fundamental shift" in data privacy and governance, according to a Cisco study. A separate Zscaler report revealed how attack surfaces are expanding at a rapid pace because of enterprise AI adoption.

Databricks has also announced the acquisition of Antimatter, a UC Berkeley spin-out which specializes in agent authentication, and SiftD.ai, created by Splunk lead architects for large-scale detection engineering.

Open for data formats

Designed with "open formats" and an "open ecosystem", Databricks said Lakewatch lets users avoid vendor lock-in while also pulling in a range of data types for analysis against attacks.

That includes video and audio, key to spotting social engineering and insider threats, the company noted. These capabilities come at a crucial time, Databricks noted, amidst the rise of AI-powered attacks using the cloned voices of executives to fool victims.

The increase in data volumes is another problem facing security professionals. Large companies generate huge amounts of security data daily – which comes with a cost as traditional SIEMs create a "financial penalty" by tying storage to compute.

"Teams respond by limiting ingestion, filtering data through routing layers, deleting historical data, and ignoring multimodal sources like chat logs and video entirely," the company said in a blog post.

"This creates a dangerous asymmetry: attackers use AI agents to analyze everything and attack anywhere, while defenders see only a fraction of their own data."

Lakewatch addresses that by applying the idea of the "lakehouse", allowing data to be analyzed where it's stored.

"You can ingest and retain 100% of your security telemetry (including multimodal data), analyze it alongside all your business data, and deploy AI-powered agents for detection and response at a fraction of legacy costs," the company added.

FOLLOW US ON SOCIAL MEDIA

Last year was a big year for McLaren Racing, as the team won its second consecutive Constructors' Championship and tenth overall. British driver Lando Norris won his first driver's title, while Oscar Piastri finished third.

With the new F1 season looming, the team will be aiming to repeat their recent successes.

ITPro sat down with Dan Keyworth, executive director of performance technology and systems at McLaren Racing, on site at the McLaren Technology Centre (MTC) in Woking.

This is the headquarters of the McLaren Group, including its racing and automotive businesses, a Norman Foster-designed campus that houses some of the team’s most famous race-winning cars. Against this historic backdrop, Keyworth reflects on the pace of change in Formula 1 and says the influence of technology and data continues to increase.

“Even in the last decade, the trend is almost parabolic,” he says, referring to the steep and rapid acceleration in the use of technology in the sport. “Every year, there's more data that we can digest and gain insight from.”

Keyworth says the modern F1 car includes about 300 sensors that capture information from practice and race-day sessions. The aim is to ensure the technologists and data professionals in his team are dialed into these insights and looking for signs of a potential competitive advantage.

“That work might include correlating those insights with data from computational fluid dynamics, the wind tunnel, and the driver simulator,” he says. “These are all indicators of performance that you want to make sure you recognize at the track.”

The challenge for McLaren is arguably greater this year than ever before, with a raft of new rules and regulations covering power units, chassis, tyres, and fuel creating a step change in the sport.

The good news is that McLaren team principal Andrea Stella recognizes the scale of the challenge and the role technology will play in helping McLaren to maintain its advantage. To that end, Keyworth was recently promoted to his current position, having formerly held the role of director of business technology.

“That change is something that’s naturally been happening over the last couple of years, which is just the convergence of technology in all areas, across engineering, production, and the race team. You've got to evolve in this sport. You stand still; you’re going backwards,” he says.

“What's clear to us as a team is that technology is another battleground in the sport. Ultimately, we're designing the car, building it, and then going racing. We've got 1,200 people here at Woking doing the first two things, and we need the best technology and systems to drive performance into the car and give the guys at the track the best possible product.”

Data analytics has revolutionized McLaren

Having spent nine years with technology specialist Fujitsu, Keyworth joined McLaren in September 2017. He lives locally and was a fan of Mika Häkkinen driving for the team as a child. Now approaching a decade with McLaren, Keyworth describes the role of overseeing technology as a privilege.

“It's a very historic brand, which comes with pressure, because it's 60 years of huge success, particularly in the last two years,” he says. “But that pressure is also the thing that energizes me. I'm quite a stable person, quite calm. So, I like the pressure cooker of Formula One. And I like the intersection between the sport and my interest in technology.”

While the team can bask in the glow of its recent successes, Keyworth recognizes there have been harder times along the road to victory. When he joined, the team was struggling at the back of the grid. Technology has played a crucial role in supporting progress on and off the track.

“I worked with the commercial team to bring in some great technology partners, such as Google, Dell Technologies, and Cisco. Some of those relationships are now very long-standing. These partnerships are not just about putting a sticker on the car for a couple of years. This work is about longevity,” he says.

“The other achievement is getting our technology genuinely focused on lap time and performance. There's been a huge amount of support from the board and from our leadership team. We put investment into our tools and infrastructure, which has given us an on-track advantage.”

One of the team’s key technology partners is IT service management (ITSM) specialist Freshworks. Keyworth describes ITSM as the stable technology layer that supports the team. “It’s the beating heart,” he says. “If you lose any IT during the race, we’re completely out of business, so it's a critical component.”

Keyworth says the lightweight nature of Freshworks’ technology means the team can make ITSM changes as circumstances demand. That capability is critical given the fast-paced nature of F1 and the raft of new rules and regulations.

“An F1 team doesn’t run like a traditional business,” he says. “You've got to have flexibility. Freshworks has been a good advocate of moving fast and being adaptable, and that's the DNA of our team. Everything we do is supported by technology, and it must roll at the same pace as our organization.”

This season’s new regulations mean the changes are coming more quickly and in greater numbers than usual. While the rules are a big shift for the team’s drivers on the track, Keyworth recognizes that the modifications also create new demands for people off the track.

“Like any change, good preparation means good execution,” he says. “This team's awesome. We've all rolled up our sleeves and embraced the new regulations, rather than felt defeated by them. It's been good to see our culture challenged by the change, and I'd say we're stronger than ever.”

It’s not just regulatory changes that present a challenge to Keyworth. While digital leaders in F1 tend to avoid bleeding-edge developments in favor of tried-and-tested technologies, he recognizes the need to keep an eye on AI through partners such as Google Gemini and Freshworks.

“There’s the ability to make processes more optimal by pairing AI with traditional machine learning and asking natural language questions, like, ‘Show me the best lap between Lando and Oscar in a practice session per sector,’ and suddenly there's a graph in front of you,” he says.

“Through our partnership with Freshworks, we've been using Freddy AI, which is an overlay into our service management processes for things like grouping tickets and spotting problems. A lot of this work was quite manual in the past. That overlay of intelligence means our people can focus on fixing problems.”

Data center operators should admit whether infrastructure projects will lead to an increase in greenhouse gas emissions and impact the UK's net zero ambitions, campaigners have told the government.

A group of environmental and social justice activists have written to the UK technology secretary Liz Kendall saying the build-out to power AI could disrupt plans to decarbonize the country's electricity grid.

Because of that, new projects should be forced by the government to show how projects won't lead to an increase in emissions or cause water shortages.

At the moment, the UK government is working on a national policy statement (NPS) on data centers. Activist groups including Foxglove and Friends of the Earth would like that document to include a framework to assess the broader environmental impact of data centers.

This would include assessments of water and power use, Scope 1, 2, and 3 emissions, as well as the introduction of new rules aimed at banning data centers from increasing emissions or water scarcity.

Campaigners expect the NPS to be published for consultation in the coming months, but hundreds of new data center development projects are already underway.

“Without these commitments, such vast electricity use will inevitably generate vast climate emissions,” the campaigners said in a letter to Kendall and shared with ITPro.

The call for stricter environmental and energy rules for data centers comes amid a massive rush to build infrastructure to power national AI ambitions.

On both sides of the Atlantic, national and local governments have faced significant pushback from campaigners over the environmental and economic impact on local communities.

Stricter data center rules needed

Campaigners want the requirements to apply to major data centers above 1MW, noting that AI infrastructure's demands differ from previous designs.

"The new generation of ‘hyperscale’ data centers currently being planned across the UK are on a very different scale to the facilities in place today," the letter noted, pointing to plans for infrastructure in Elsham in Lincolnshire and Cambois in Northumberland, which will each have a demand for electricity of 1GW.

"By way of context, the peak winter electricity demand for the entirety of the UK is around 47GW.”

"This additional demand – unless meaningfully matched with new, renewable energy generation and storage – poses a serious threat to efforts to decarbonize the UK’s electricity grid," the letter said.

Google's proposed data center in Essex, for example, could lead to a “net increase in GHG emissions of 568,727 tonnes CO2е per year during the operational phase”, according to the company’s planning documents.

To avoid increasing emissions, data center developers should prove how they will increase renewable energy supply – by building generation or by funding it – and to fully describe how cooling systems will work.

"Measures to prevent the use of ‘greenwashing’ in claiming the data center is carbon-neutral, for example through the purchase of renewable energy certificates or the use of power purchase agreements which do not result in the construction of additional generating capacity," they added.

The letter was also signed by Global Justice Now, Global Action Plan, Opportunity Green, and the Green Web Foundation.

Government response

Whether the government listens to such concerns remains to be seen, but signs suggest environmental and energy impact of this infrastructure build-out is raising concerns, at least for some politicians.

Indeed, MPs last week launched an inquiry into the environmental impact of data centers in the UK, in particular energy and water use.

In response, energy secretary Ed Miliband last week said the UK's grid decarbonization efforts should already mitigate any emissions from increased electricity demand, but admitted that energy use from data centers remains "inherently uncertain."

A government spokesperson told the Guardian that such data centers would help boost the economy and address environmental issues, too.

"Datacenters will increasingly be powered by renewables and our AI energy council is exploring opportunities to attract investment in new clean power sources for the industry, while the planning system takes water scarcity into account," the spokesperson added.

FOLLOW US ON SOCIAL MEDIA

Pure Storage has announced a new acquisition – and a new image – as the company pivots away from its traditional hardware focus and towards data management.

The acquisition of data intelligence and orchestration firm 1Touch will bolster the company’s data intelligence and orchestration features, which will be incorporated into its Enterprise Data Cloud (EDC) platforms.

“We are going beyond just pure data storage and adding context to the information and pulling it together into an enterprise data cloud,” CEO Charlie Giancarlo said in an announcement video.

“The announcement we made today to acquire 1Touch is a major advance toward that goal.”

As part of the move, Pure Storage will also rebrand to ‘Everpure’ in a move Giancarlo said better “reflects the company we have become”.

“The name Everpure represents the next step as we continue to not only redefine storage, but now redefine data management at scale,” he said.

The rebrand marks the culmination of a sharp pivot by the company over the last two years away from “pure” storage offerings toward data management capabilities for customers.

As ITPro reported at Pure Accelerate 2025, the launch of the firm’s Enterprise Data Cloud (EDC) signaled a heavier focus in this domain.

With EDC, the company now offers enterprises an all-encompassing environment which incorporates both hardware and software.

In the video announcement detailing the rebrand, CTO and founder John Colgrove echoed Giancarlo’s comments on the company’s new focus, noting that the “brand expansion” will better serve customers.

“We started the company and we built a fantastic data storage platform, but now as we expand beyond data storage into data management, understanding the data, helping people use their data better, it just makes sense to expand the brand,” he said.

Everpure targets deeper data insights

A key factor behind the EDC launch last year was the problem of data silos, the company said – an issue that has only been exacerbated by the arrival of generative AI.

“The key thing for AI is being able to make better use of one’s data,” he said. “But actually, the data that companies have is highly fragmented in their operational silos, and that data doesn’t really have the context necessary to be used for AI.”

Shawn Hansen, VP and GM at what is now Everpure’s Core Platform Business Unit, told ITPro last year that the company’s motivation for launching EDC came as a result of surging customer demands for flexible storage solutions and more intuitive management features.

FOLLOW US ON SOCIAL MEDIA

Snowflake and OpenAI have announced a $200 million multi-year partnership aimed at driving enterprise agentic AI adoption.

The tie-up builds on an existing partnership between the two companies, and according to Snowflake, will help customers derive “greater value" from in-house data and simplify agent adoption.

Key to this will be the integration of OpenAI models within the company’s flagship Cortex platform, which allows customers to build and deploy custom AI applications.

OpenAI models will now be available natively within the Cortex platform for the company’s 12,000+ customers, for example. Elsewhere, models including GPT-5.2 will also be accessible through Snowflake Intelligence, the firm’s enterprise agent platform.

Snowflake CEO Sridhar Ramaswamy said the partnership will allow customers to take advantage of their “most valuable asset”.

“Customers can now harness all their enterprise knowledge in Snowflake together with the world-class intelligence of OpenAI models, enabling them to build AI agents that are powerful, responsible, and trustworthy,” he said.

“Together, we’re setting a new standard for AI innovation, helping businesses transform with confidence, while maintaining strong security and compliance standards."

The deal with OpenAI comes just weeks after a similar tie-up with one of the company's key rivals in the generative AI space, Anthropic.

In early December, Snowflake announced a multi-year partnership with the firm – also valued at around $200 million – to integrate Anthropic's Claude models across the Cortex and Intelligence platforms.

What can Snowflake customers expect?

According to Snowflake, the partnership will help simplify custom agent development, an area OpenAI has focused heavily on over the last year.

In March last year, the firm announced the launch of a new ‘Agents Software Development Kit’ (SDK) aimed at streamlining the creation, deployment, and monitoring of agents.

The tool includes observability features that allow developers to see how agents are performing, allowing them to fine-tune and tweak based on individual needs.

Snowflake noted that the duo will work closely to integrate new SDK-based features within core products, including Cortex. Agents run directly on enterprise data, but many businesses face significant hurdles unlocking value from that data - largely due to poor quality or siloed environments.

Elsewhere, integration of GPT-5.2 within Snowflake Intelligence will also give users more granular insights into enterprise data, the company noted. The platform allows users to query and interpret both structured and unstructured data using natural language prompts.

“This partnership brings our advanced models directly into that environment, making it easier to deploy AI agents and apps, so businesses can close the gap between what AI is capable of and the value they can create today,” said Fidji Simo, CEO of applications at OpenAI.

FOLLOW US ON SOCIAL MEDIA

Europe is rushing to adopt AI, but many organizations don't trust their data and almost all say staff need more training.

Findings from Informatica’s annual CDO Insights study show almost eight-in-ten businesses will have adopted generative AI by the end of this quarter, and 68% will have started agentic AI pilots.

Businesses in the UK are moving at a slower pace, however, with just 61% planning to introduce agentic AI.

CDOs are hoping that AI will improve business decision-making and enhance employee collaboration, both cited by 32%, while 28% are looking to optimize internal processes and 27% to enhance customer experience and loyalty.

However, there's a big paradox when it comes to trust. While data leaders believe that 61% of employees - 52% in the UK - trust ‘most’ or ‘all’ of the data that their organizations are using for AI, 96% say that staff need more training in AI or data literacy to use the technology responsibly.

"Although employees generally trust the data used for AI, many are lacking in data and AI literacy skills, and organizations lack underlying AI governance structures for achieving the responsible and ethical outcomes they desire," said Krish Vitaldevara, chief product officer at Informatica from Salesforce.

This poses significant risk exposure and hurts confidence in AI initiatives."

Most European businesses (55%) are opting to buy vendor-supplied agents, though the figure is just 44% for UK firms. Meanwhile, 45% of European organizations, and 55% in the UK, are developing and/or managing agents in-house. Of these, 21% are choosing to build using no- or low-code platforms.

Too much, too fast

Notably, Informatica warned enterprises may be moving too fast, with more than three-quarters of respondents saying their company’s AI visibility and governance has not kept pace with employee use of AI technology.

When it comes to AI literacy, data leaders believe that the majority of employees trust ‘most’ or ‘all’ of the data that their organizations are using for AI. However, 96% reckon employees need more data or AI literacy training in order to use AI or its outputs responsibly.

There is, though, money to deal with these issues: 85% of European businesses are increasing their data management investments in 2026, with 23% expecting to significantly increase their spend.

The top drivers for this are upskilling employees to improve data and AI fluency, improving data privacy and security, and enhancing data and AI governance, all cited by 44%. In the UK, 49% are planning to invest in data literacy.

For 57% of European data leaders and 60% of UK leaders, poor data reliability is delaying moving generative AI initiatives from pilot to production. Half are ‘very’ or ‘extremely’ concerned that new AI pilots will move forward without addressing the data reliability problems evident in previous initiatives.

Other major blockers include security concerns, lack of agentic AI expertise, observability issues, insufficient tools for managing AI agents, and lack of safety guardrails.

Data reliability problems persist

When it comes to dealing with data reliability, 58% of organizations are improving workflows around data/AI, and 56% are investing in data and metadata collection and management.

More than half (55%) are increasing the frequency of data checks while a similar number (54%) of respondents said they are increasing investments in data quality.

"Encouragingly, there are early signs of maturity in how organisations approach AI and agentic systems”, said Emilio Valdés, SVP sales international, Informatica from Salesforce.

"Data leaders are recognizing the risk and increasing investment in data governance and compliance foundations."

FOLLOW US ON SOCIAL MEDIA

Autonomous IT is here and it is transforming how organisations across the world work - increasing resource and technological efficiencies whilst championing security. Today we are discussing a critical first step in freeing up resources and laying secure foundations for transformation… Patching.

Patch management is a crucial part of any business. From chaotic, ad-hoc beginnings to the regular cadence of Microsoft’s Patch Tuesday, how IT teams patch vulnerable systems has evolved considerably over the past decades. Yet there are still inefficiencies in the patching process.

This Video Q&A, brought to you in collaboration with ITPro and Tanium, explores a new answer to this problem: autonomous patch management. With the ability to save companies time and money, while freeing up IT professionals to do more valuable tasks, autonomous patching could be the answer your business is looking for.

Join Rob Broughall, Kate Jefferyes and Dan Jones from Tanium as they discuss the history and complexities of software patching, as well as explore how automation can help lift the drudgery of this crucial but repetitive task.

Watch now to find out more about the business benefits of autonomous IT and unified endpoint management delivered by Tanium.

Featured Tanium Speakers:

Rob Broughall

Rob Broughall helps enterprises modernise endpoint management and transition from manual patching to autonomous, data‑driven workflows across Windows, macOS, and Linux. As part of Tanium’s Domain Architecture team, he focuses on Endpoint Management, leveraging deep expertise in modern workplace, collaboration, and M&A integration to enhance employee experience while reducing operational risk. A frequent speaker and technical thought leader, Rob advocates for Autonomous IT as the key to resilient, self‑healing environments — enabling organisations to achieve security and efficiency at scale. He holds multiple certifications, including Microsoft Certified Master (Exchange 2010), PRINCE2, and Tanium credentials.

Kate Jefferyes

Kate Jefferyes is Tanium’s UK & Nordics Regional Vice President and helps organisations gain real-time visibility and control over every endpoint in their IT environment. With over two decades of experience in technology and cybersecurity, Kate has held senior roles at Microsoft, ServiceNow, and Symantec, driving strategic initiatives across healthcare, public sector, and enterprise markets. At Tanium, Kate is passionate about enabling businesses to adopt Autonomous IT — delivering automation and unified control that transforms security and operations into unstoppable business outcomes.

Dan Jones

Dan Jones brings extensive cybersecurity expertise following a distinguished career with the UK Ministry of Defence. At Tanium, he advises organisations on managing digital supply chain risks, focusing on the interplay between people, processes, and technology. A recognised thought leader on regulatory frameworks such as NIS2 and DORA, Dan champions Autonomous IT as the foundation for unstoppable business — enabling organisations to achieve resilience, agility, and security at scale through automation and unified control.

Appian has launched a new studio of AI agents for business automation, but it wants you to know these are not your usual agents – they’re better.

Agent Studio, first unveiled at Appian World 2025, is now generally available, with its beta program coming to an end this week at its Appian Europe conference in London.

Appian is a native AI company – a bona fide automation specialist – that takes business processes and shrinks them down from days, weeks, even months, into mere minutes.

It takes B2B data and uses it to automate business processes, so it is an AI company, but it wants to stress to you that it isn’t like others, and its agents aren’t like all the other agents.

For one thing, it has a slower approach. Agent Studio holds the mantle of the “longest, largest beta program” Appian has ever conducted, according to CEO and founder Matt Calkin.

Part of the reason for the slow process is compliance, the company told attendees. Appian is a fan of the EU’s data regulations and not a ‘move fast and break things’ innovator, Calkin said.

It also wanted to offer agents that were “dramatically different” from those already out there in the ecosystem. Agent Studio is a suite that lets businesses control and deploy agents at scale.

These are agents that can ‘reason’, according to Appian, and they can handle unexpected conditions and or act on enterprise data to automate complex workloads.

This is achieved through Appian’s ‘data fabric’ architecture, which is largely where Appian believes its agents are better, as it offers a holistic view of a company's business data.

This, it said, makes for smarter decisions, real-time adjustments, and easy governance.

Misapplied AI vs serious AI

Calkin began the event talking about the current state of AI and how his company was different.

Referencing a recent MIT study that stated 95% of AI projects fail, Calkin noted a key factor behind this is that people misunderstand that AI isn’t a standalone technology.

Indeed, it's a very dependent one, requiring other elements such as good quality data and meaningful processes to fully reach its potential.

For Calkin, AI is only as good as the data you give it, but also only as good as the work you make it do. This is how you get to “serious AI”.

“Serious AI is happening now, but it may not be happening where you think it is,” Calkin told attendees. “It’s happening in some pretty unglamorous places, like procurement or government contracts.”

Where businesses are inundated with documents, serious AI is emerging. Invoices, receipts, bills, transcript logs, the list goes on across both digital and physical documents, and it is here where serious AI is helping.

And, as Calkin said, “the returns are not boring”.

MORE FROM ITPRO

• Is enterprise agentic AI adoption matching the hype?
• Businesses are being taken for fools with AI agents
• Enterprises are worried about agentic AI security risks

The role of data engineers is changing, according to new research, and they’re becoming increasingly crucial to businesses thanks to the rise of AI.

Three-quarters of business leaders increasingly believe data engineers – those who manage databases and extract information for actionable insights – are now integral to the success of their company, according to a survey from MIT Technology Review and Snowflake.

That rises to 86% for the largest companies with revenue over $10bn, the survey found.

"Senior leadership is quickly realizing that if they’re not using data in their decision-making processes, they risk lagging behind their competitors," said Chris Child, VP of product data engineering at Snowflake.

In a post detailing the survey results, Child noted he saw two trends contributing to the increasingly prominent profile of data engineers.

"The first is the sheer volume of information being generated and made available today – a function of data being easier to collect and cheaper to store than ever before," Child wrote.

"The second is a growing realization among top executives that without using data to inform decisions, their organizations are essentially flying blind. This is driving demand for AI to enable the use of data in more and more complex decisions than ever before."

No wonder then that the survey found three-quarters of companies expect their data engineers to face ever increasing workloads.

Data engineers are swamped at work

Indeed, the survey found that the amount of time spent by data engineers on AI projects has doubled from 19% of their day in 2023 to 37% this year – and those polled expect that to hit 61% of their work day within two years.

But they're not just putting work into AI, however, and data engineers are also getting a benefit. Three-quarters of respondents said AI has boosted the output of data engineering teams over the last two years, with a similar figure reporting an improvement in quality of work, too.

While half of those surveyed plan to begin deploying agentic AI in their workflows in the next year, with 20% already there, respondents reported expectations of continued productivity boosts.

But that optimism is tempered by concerns about complexity and exacerbating tech stack fragmentation, Snowflake noted. Because of that, the role of data engineers may become more about management.

"Over time, I think the role of the data engineer will be less about writing code for every single pipeline and more about managing the infrastructure that AI agents are operating in,” Child noted.

Shifting roles for data engineers

The importance of data engineers in AI projects means the C-suite has finally come to realize and acknowledge the importance of the field.

"This has really brought data engineers to the attention of senior business leaders, with data engineers increasingly serving as essential partners for shaping strategy and driving business outcomes," said Child.

Indeed, data engineers jobs are evolving – and the role's influence is expanding.

Two-thirds (66%) of respondents said data engineers are influencing decisions about what tools to invest in and from which vendor. Meanwhile, half said engineers now have an impact on overall data strategy and how AI is used in the business.

"AI is pushing leaders to reconsider data processes as a result of a digital transformation," said Child.

"As AI agents take on more work, data engineers will spend less time on extracting, transforming and loading work and more time on the data and AI strategizing that will help take the business forward."

Child advised data engineers to take advantage of the opportunities offered by the rise in AI by upskilling in the subject, ensuring they have a strong understanding of key business objectives, and building strong communication skills to be able to expand their influence further.

MORE FROM ITPRO

• Databricks wants to train 100,000 people in AI across the UK and Ireland
• Data storage is dead, long live data management
• Engineering firms see little productivity benefit from use of AI

Databricks has announced plans for a new investment round, bringing the analytics company’s valuation to over $100 billion.

In a statement published 19 August, the company said it expects the funding round to “close soon” and includes backing from existing investors.

Databricks said the latest financing round will be used to accelerate its AI strategy, including an expansion to its Agent Bricks service and investment in its new database solution, Lakebase.

The company added that funding will also help drive further “global growth”.

Ali Ghodsi, co-founder and CEO of Databricks, said the latest funding round highlights the “tremendous investor interest” in the company as a result of its AI push.

Databricks launched Agent Bricks at its annual conference in June. The service enables enterprises to build AI agents trained specifically on in-house data.

Lakebase, meanwhile, is a new operational database (OLTP), built on open source Postgres, again aimed at supporting agentic AI development.

The sharpened focus on agentic AI comes amidst a flurry of activity on this front across the industry, with big tech providers including Salesforce, Google, and Microsoft all launching their own agentic AI solutions suites over the last year.

“Every company can securely turn its enterprise data into AI apps and agents to grow revenue faster, operate more efficiently, and make smarter decisions with less risk,” Ghodsi said.

“Databricks is benefiting from an unprecedented global demand for AI apps and agents, turning companies’ data into goldmines. We’re thrilled this round is already over-subscribed and to partner with strategic, long-term investors who share our vision for the future of AI.”

Databricks has been building for growth

Databricks has raised significant investment in recent years, and this latest round brings its valuation up 61% compared to December 2023. In December last year, the firm raised $10 billion in venture capital (VC) funding – one of the largest investment rounds in tech history.

This marked a high point in what was a whirlwind year for the burgeoning data analytics giant. The firm opened a regional hub in London, for example, and confirmed a major acquisition.

June 2024 saw the company announce plans to acquire data management firm Tabular in a deal believed to be worth around $1 billion. This marked the second major acquisition for Databricks in the space of a year, having snapped up LLM startup MosaicML for $1.3 billion in July 2023.

Both these deals were forward-thinking, strategic acquisitions for the firm as it prepared for a major push in the generative AI space – and it appears to have paid dividends.

The company said it has recorded “strong momentum” in the last two quarters, and is on track to surpass $3.7 billion in annual recurring revenue as of July 2025, which represents 50% year-on-year growth.

Closer ties with industry stakeholders have also been secured, with the company having announced – or expanded – partnerships with an array of tech giants, including Microsoft, Google Cloud, SAP, Palantir, and Anthropic.

“More than 15,000 customers around the world use the Databricks Data Intelligence Platform to democratize access to data and AI, making it easier to harness the power of their data for analytics and AI apps and agents,” the company said.

MORE FROM ITPRO

• Databricks continues EMEA growth drive with double leadership appointment
• Databricks just launched an open source large language model to compete with Llama 2, Mixtral, and GPT-3.5
• Databricks expands AWS partnership to drive generative AI capabilities

Data sprawl is the uncontrolled – and often undocumented – spread of data across an organization’s systems, platforms, locations and formats. This includes on premises infrastructure, user devices, SaaS applications and cloud providers.

But we’re probably doing data an injustice describing the issue as such, says Tim Nelms, senior director analyst at Gartner, as the principal challenge is with content sprawl. “Content – unstructured data – represents 80% of corporate information,” he points out.

Whether coined as content or data, this sprawl issue most commonly arises when organizations rapidly adopt cloud services that enable different teams or departments to generate, store and use data independently, and don’t have a process of filtering out duplicate and unneeded data effectively.

“In an effort to be agile and make use of modern technologies as fast as possible, this often happens without central oversight or data governance,” explains Tom van Aardt, chief data officer (CDO) advisor at consultancy firm BML. “The result is a fragmented data landscape that increases risks related to security, compliance and inefficiency, and makes it difficult to discover, trust or leverage data effectively.”

This is in addition to a steady buildup of data in recent years due to the widespread adoption of multiple cloud platforms, decentralized procurement of tools, shadow IT and the rapid growth of unstructured data across collaboration platforms. Throw into the mix the race to use AI and advanced analytics and you’re left with duplicated, fragmented and poorly catalogued data, he notes.

The pandemic must take some of the responsibility for the exponential growth in data, as this was a key reason for the rapid roll-out of digital workplace technologies that in some cases overlooked recommended governance and lifecycle controls in favor of speedy implementation.

“Microsoft 365 alone has gone from 13 million active monthly users to over 330 million today. Most of those deployments happened quickly,” Nelms notes.

The more data the better, no?

In an era when data is one of the world’s most valuable commodities, it’s fair to ask what’s the harm in having so much of it? Well, the real issue isn’t the volume of data, but the lack of visibility and observability around it.

This opens up the organization to security vulnerabilities, as potentially sensitive data could be sitting in places it shouldn’t. “This could be on end-user devices, in a cloud storage bucket that has misconfigured security, in poorly protected transactional systems,” says Brent Ellis, principal analyst at Forrester. “While traditional ransomware encryption is still a threat, exfiltration and the movement of data outside of the perimeter that a business can control means it is threatened by risks to compliance, IP theft or loss of customer confidence.”

While businesses can use programs specifically designed to detect data sprawl, it’s most common for them to discover they have an issue when a security incident occurs. According to security firm Rubrik, enterprises are facing a cloud security crisis fueled by data sprawl, with consequences of breaches including reputational damage, compliance-related fines and forced leadership changes.

It can also significantly hamper incident response and recovery when the worst occurs, by obscuring where sensitive and critical data resides, delaying breach detection, complicating forensic investigations and slowing containment efforts, notes van Aardt.

Data sprawl just magnifies the overall burden of recovery, Ellis agrees, adding that more generally, in terms of operations, data sprawl means longer back-ups and higher storage costs – essentially eating up time and money that could be better used elsewhere.

How can businesses handle data sprawl?

In today’s complex hybrid and multi-cloud environments, full control over all data is rarely achievable. It’s more realistic therefore, to focus on effective mitigation rather than complete elimination of data sprawl.

“One might have a goal of complete control, but companies must be agile and the environments where data must be used will change over time. The big thing to establish is a culture that respects data and understands that the business has to manage it responsibly,” Ellis advises.

An area of focus should be on improving observability of your data, which can be achieved through a mix of tools and practices. Start with implementing a data strategy, setting up data governance and then implementing tools to support these, advises van Aardt.

“Following on from having a solid strategy and governance framework, businesses can then improve observability of data across hybrid or multi-cloud ecosystems by implementing centralized data inventory tools, standardizing metadata and tagging practices and using cloud native monitoring and logging services integrated into a unified data catalogue or governance platform.”

Building standardized ways of deploying infrastructure across cloud and on prem is an important step so that you can use consistent tooling to understand where data is and what policies are applied to it, continues Ellis. “Exceptions occur, but those should be controlled and clear practices for making sure data in a new SaaS vendor, for instance, is kept in compliance to internal data controls. This may mean negotiating with vendors to get data visibility that’s not currently available as part of the service.”

“Enhancing observability also requires automated data discovery, classification and lineage tracking alongside clear ownership models and policy enforcement to ensure visibility, compliance and control over all data assets – regardless of where they reside,” adds van Aardt.

The human factor

While automation and AI can also help support the management of data sprawl, it’s important not to overlook the human side of the equation. Strong leadership and ownership from the top are required to build a successful culture of data management and it’s important to break down silos and foster communication and collaboration across IT, security and business teams.

Control over data sprawl looks set to become even more important as near-future regulations are likely to demand greater transparency on where data resides, how it’s processed and who has access.

“Ultimately, evolving regulations will force businesses to embed privacy-by-design and security-by-default principles into their data strategies, turning regulatory compliance from a checkbox exercise into a core operational discipline,” van Aardt concludes.

Simplicity was the war cry ahead of Pure Accelerate 2025, and by all means it appears Pure Storage has come away from the conference with a huge win.

The storage vendor has made significant headway in recent years in its quest to build an all-encompassing ecosystem for customers, combining high-performance flash hardware with a broad software stack to manage and optimize infrastructure.

Notably, in recent months its storage as a service (STaaS) model has begun paying dividends, as I noted ahead of the conference, but the announcements last week highlight its razor sharp focus on even more simplicity - with a dash of automation, of course.

The flagship update from Pure Storage centered around the launch of its Enterprise Data Cloud, the firm’s new combined approach to data storage and management.

Underpinning this is the company’s Purity operating system, providing enterprises with a unified software platform, alongside Pure1 and Pure Fusion - the latter two now make up the ‘management’ layer, so to speak, within the Pure estate.

Pure Fusion was a point of particular focus for the storage vendor last week and forms a critical part of the Enterprise Data Cloud. The company has taken its time fleshing this out, having first announced this in 2021, and new updates mean that customers can now view their storage assets in a single pool.

No more disjointed environments and disparate datasets: it’s all there in front of you, visible and easily accessible.

Combined with Pure1, this is the ‘intelligent control plane’ of the EDC, and brings a level of automation to data storage that Pure has been striving for over the last few years.

During two extensive hands-on keynote sessions, attendees were shown how workflows and configurations can now be easily automated at the click of a few buttons.

It’s more than just simple automation, however. The company repeatedly emphasized that the automation features span an array of areas, with compliance and security of particular note.

Pure Storage is playing the long game

The EDC represents the fruits of more than a decade’s labor to consolidate the Pure Storage ecosystem, founder John ‘Coz’ Colgrove told ITPro at the conference. CEO Charlie Giancarlo waxed lyrical about the fact that this represented a “paradigm shift” in the data storage industry.

In pulling together a comprehensive, finely curated architecture, the company will no doubt make it easier for enterprise customers to optimize infrastructure, but more importantly, manage the data which now represents the lifeblood of their business.

Patrick Smith, Field CTO for EMEA at Pure Storage told ITPro ahead of the conference the company wants customers to essentially sit back and let Pure handle the drudge work with storage optimization while focusing on data management.

“Don’t manage the storage, manage the data,” he quipped in a call ahead of the annual event. Pure’s EDC launch certainly appears to hit the nail on the head on this front, but is it truly unique? Not exactly.

Prakash Darji, general manager at Pure Storage's digital experience business, told ITPro on the ground that the EDC is “evolutionary, not revolutionary” having previously hit out at a question from the press on the novelty of the approach.

It does bear some similarities to other options on the market - just don’t mention that to them - albeit with a more fleshed out portfolio that’s been made possible by Pure’s late-entrant status within the broader storage industry.

Simply put, it hasn’t had to contend with decades’ worth of legacy solutions, infrastructure, or most importantly, customers like competitors have.

That’s the advantage the company now has in the broader marketplace. With the EDC, the final building blocks are in place, all that’s required from here on out is to let customers tinker and experiment.

Pure Storage will take care of the hardware, whether enterprises can use the service to get their data in order will be the real test.

SAS has announced new synthetic data capabilities and support for digital twins in a push to equip enterprises with concrete platforms for data-driven decision making.

Synthetic data is artificial data generated to resemble real datasets in every way that’s meaningful. These datasets are typically used to train AI and machine learning models without exposing sensitive data.

While generating synthetic data has benefits, obtaining it can be difficult, research shows. Organizations need access to the right algorithms to generate quality data, particularly if they’re looking to train a domain-specific AI model for very targeted use cases.

This is where SAS Data Maker is intended to come in. The service is a low-code/no-code platform that can quickly generate synthetic data in the quantities needed and to a high standard that’s auditable.

SAS first announced Data Maker in 2024 as an in-preview product with an unspecified release date, and has since acquired the UK synthetic data startup Hazy to improve its data generation software. It has now announced a targeted general release of Q3 2025.

The announcement was made at SAS Innovate 2025, the company’s annual conference held in Orlando, Florida. At the same event, SAS elaborated on work it’s doing at the other end of the data pipeline to make better use of existing business data for advanced data simulations, such as digital twins.

It gave the example of a recent project it has undertaken with the paper and wood-based product manufacturer Georgia-Pacific (GP), in collaboration with Epic Games.

The trio have leveraged Epic Games’ 3D Unreal Engine, originally developed for video games, combined with GP's enterprise information and SAS data technology, to simulate an entire factory at Savannah River Mill in Georgia.

Using the factory’s digital twin, GP could analyze the efficiency of its on-site automated guided vehicles (AGVs), as well as lower the barrier to entry for workers seeking analytics data on the factory.

SAS said its cloud-native Viya platform was integral in getting the project up and running, with GP having used tit to translate unstructured factory data into informations systems that formed the backbone of the digital twin.

“With the help of SAS and Unreal Engine, we can create realistic simulations of factory operations,” said Roshan Shah, GP’s vice president of AI & Products.

“Imagine watching AGVs navigate through a bustling factory floor, reacting to proximity alerts, obstacles and rare adverse events in real time," Shah added.

"The powerful analysis and photorealistic simulations delivered by SAS’ enhanced digital twins can enable decision making and boost output at Savannah River Mill. And they hold great promise for improving productivity, safety and efficiency at other GP facilities.”

MORE FROM ITPRO

• What to watch out for at SAS Innovate 2025
• Why AI researchers are turning to nature for inspiration
• Are AI digital twins a match made in heaven?

That's a wrap for SAS Innovate 2025, with the formal proceedings of the event having all but drawn to a close. You can still catch up on everything as it happened at the event below, with our beat-by-beat record of the announcements available in chronological order.

You can also find ITPro's comprehensive coverage from the event here:

• SAS wants its AI agents to supercharge workers, not replace them
• SAS leans on synthetic data and digital twins to support business data demand
• SAS thinks quantum AI has huge enterprise potential – here's why
• “Public trust has become the new currency for AI innovation”: Why SAS is ringing the alarm bell on AI governance for enterprises
• SAS Innovate 2025 live: All the updates from day two
• What to watch out for at SAS Innovate 2025

Hello from Orlando, where the sun is not yet in the sky. While we wait for the conference proper to get underway, why not read ITPro's analysis of the biggest talking points at SAS Innovate 2025.

It's bright and early in Orlando and attendees are gearing up for the day-one general session. Based on the products and demos on show in the expo hall, we're expecting to hear lots about AI in SAS Viya and doubtless some customer testimonies for how SAS has benefited their business.

On that note, assembled media were last night shown around the Orlando Magic Kia Center, the practice and exercise space for the Orlando Magic NBA team.

We were told how Orlando Magic has worked with SAS to improve customer outreach, processing game data to deliver targeted emails to fans. The same partnership has also improved ticket revenue by giving season ticket holders an easy way to let Orlando Magic sell their tickets if they can't attend a game, in return for virtual credit.

Just under ten minutes to go until the day-one keynote begins. As one would expect, we'll hear from Jim Goodnight, CEO at SAS.

But a little more unusually, we're also expecting a virtual address from Satya Nadella, CEO at Microsoft. We'll bring you that as it happens.

And we're off, with an opening welcome from Dominque Moon, customer success manager at SAS.

Moon says attendees can expect news on groundbreaking technology and thought provoking hands-on opportunities, as well as "lots and lots of fun surprises".

To begin with, we're told, we'll have a keynote from Bryan Harris, CTO at SAS, looking at the specific outcomes that SAS' technology can unlock.

Next, we'll have a series of speakers running us through the newest announcements from SAS – more on those as they come. Rounding it out, we can expect to hear from Frank Abagnale, of Catch Me If You Can fame, on how we can tackle fraud.

We're now off in earnest, with a sizzle reel beginning with a word from Goodnight, who stresses that organizations have more data to handle than ever before.

Harris has now taken to the stage. He begins by noting SAS is nearing its 50th anniversary and has achieved a lot along the way – billions of lines of code, its own programming language, and many generations of technology.

The latest innovation, Harris says, is AI and this can be harnessed with decision intelligence from SAS. This, he says, can cut through the difficulties and confusion associated with adopting AI.

"It really feels like all you need to do is sprinkle a little generative AI in your enterprise and all your problems will magically disappear," he says.

"Can you feel it?" he asks the audience, to laughter.

Harris says generative AI is a powerful technology but comes with limitations. For example, he cites a recent study which found that LLMs recommend more denials and higher interest rates for black applicants than identical white applicants. Harris notes this systemic issue was found in models from OpenAI, Anthropic, and Meta.

“On average, applicants need credit scores 120 points higher than white applicants to receive the same approval rate,” he explains.

The fix to this issue? A prompt urging stressing the importance of removing bias. It's a great example, Harris says, of why organizations need to take governance and implementation of AI very seriously.

Agentic AI can help address these concerns, SAS believes, and Harris is now giving us a practical example of SAS AI Agents in action.

In a perfect system, he says an AI agent could autonomously detect and denies fraudulent transactions in a finance system. But it's not always that straightforward – which has led SAS to a 'human in the loop' approach.

In the Viya dashboard, we're shown, a user tracking mortgage applications can interrogate the decisions agents make, check the model they're running on, and see every step it's taken to come to its decision.

This can be especially useful, Harris says, when a user is looking to understand how an agent made by another department in their company works.

Controlled sectors such as banking might want even more information on model governance. So at the end of the process, users can use see a ‘lineage’ for the models, from SAS Intelligent Decisioning, to SAS Model Manager, SAS Model Studio, SAS Information Catalog, and SAS Studio.

We're now moving on to digital twins, which Harris says give organizations the power of 'what if' when it comes to their data.

SAS wants digital twins to be the most sophsticated simulated environments they can be, he says, to enable complex systems to be better understood.

To bring this to life, SAS has partnered with Epic, the games company behind Unreal Engine, to simulate real-world and create synthetic data in extreme detail. You can read more in our detailed coverage here.

SAS has worked with the paper product manufacturer Georgia-Pacific, which has been using the digital twins to simulate the movement workers, vehicles, and autonomous vehicles around its manufacturing environment. We're told it's already worked out more efficient ways to route traffic through its sites.

The digital twin can also be used to improve safety. “According to US labor statistics, a person dies every 90 minutes in the workplace,” Harris says, explaining how realistic simulations like those SAS and Epic Games are making can be used to train computer vision models to make them safer.

Manufacturing is just one sector that can benefit from digital twins, Harris says, with others including financial services, healthcare, and the supply chain.

"I don't know, maybe see what the impact of tariffs are, that might be helpful now," he adds, to audience laughter.

SAS is committed to solving some of the most complex problems in computing, Harris says, and is increasingly looking to quantum AI to overcome this. Here to tell us more is Krista Comstock, director of product and innovation at P&G.

P&G has been doing business with SAS for 45 years, Comstock says. The firm faces the constant issue of creating all its various products while ensuring some ingredients never mix with one another – to illustrate her point, Comstock encourages the audience to walk down their grocery aisle and consider all the different kinds of shampoo and the different ingredients they have.

P&G looked at a hundred ingredients across five manufacturing tanks and worked out the total number of mixture combinations possible was 100 to the 14th power – more than the number of atoms in the universe.

Using traditional 'solver' algorithms, Comstock says P&G was able to break this problem down in around six hours. Using quantum AI with SAS, it cut this down to two minutes but introduced some unwanted results. Finally, it settled on a hybrid approach, in which quantum AI was mainly used and the traditional solvers were only used for the final calculations – for an overall processing time of 12 minutes.

With that, we're moving onto some of the innovations coming to the SAS platform. To hear more, we've welcomed Jared Peterson, SVP Platform Engineering at SAS, to the stage.

"At SAS, we really understand how transformative Al can be. But we also understand the challenges that you face and the problems that you are trying to solve,” Peterson says.

“It's why we've created products that give you more data, more flexibility, and more control over how you get your job done whether you're a developer or a business decision maker, the things that you will see this morning can change the way that you work and the way that you run your organizations.”

We're starting off with a demo of Viya Workbench, the SAS cloud-based development environment. Peterson says SAS Viya Workbench was almost canned in the development phase but a developer “cornered me in a hallway” and urged him to consider the potential of the platform.

That developer was Joseph Henry, distinguished software developer at SAS, who's here now to show us SAS Viya Workbench live.

Henry says performing sentiment analysis, stemming, and limitization within Viya Workbench is straightforward, requiring just a few lines of code.

In his practical example, analyzing review data from Amazon products, Henry starts in Python and moves to R, showing the complexity of the process, then shows that the workflow in SAS is much simpler.

Peterson wants to know if more advanced models can be used in SAS Viya Workbench. Henry says you can, giving the example of the textbert model being used to score sentiment in text.

To round his time off, Henry shows off a simple app he created within Viya Workbench, to rate the sentiment of SAS Innovate 2024 attendee reviews.

We're talking about Data now, with Peterson underlining the importance of available data to power AI models.

Synthetic data is one route to meeting these needs, Peterson says, which is why SAS announced its synthetic data generation tool Data Maker last year.

Also in 2024, SAS acquired UK synthetic data startup Hazy. Here to tell us how its software has improved Data Maker is Harry Keen, founder and former CEO at Hazy and product evangelist at SAS.

To start, Keen says, users select their source data and let Data Maker automatically work out the structure of the data. In the latest version, Keen says, users can also highlight multi-table and time-series data.

Users can then select the specific model they want to use to generate the data, whether or not they want the enhanced privacy of differential privacy – "smart noise" added to the data to prevent it from being reverse-engineered – and then generate the data.

Once you've completed the initial model setup, Keen says, you don't have to ever again. Users across an organization can simply use the trained model to repetitively generate data going forward.

We're moving fast here in Orlando. Peterson notes that not everyone is a data scientist and this is where SAS Viya Copilot in Model Studio comes in.

Here to explain more is Ben Tomlinson, principal software developer at SAS. He demos how Viya Copilot can be used to build out a data pipeline, or explain an existing node.

In an example showing Viya Copilot's summary of a forest model, made up of decision trees, Tomlinson shows how a user can ask for more information on the node's receiver operating characteristic (ROC), using RAG to draw on enterprise documentation so the information is as accurate as possible.

Users can also ask Viya Copilot to 'improve' a specific node.

"By going beyond just answering questions and actually performing actions, SAS Viya Copilot unlocks the potential all types of users," Tomlinson says.

Peterson says SAS wants to inspire its users to be innovative and to empower its AI and modelling teams to cut down time consuming tasks.

Here to tell us more directly about AI is Manisha Khanna, senior manager product marketing at SAS and Udo Sglavo, VP Applied Al and Modeling at SAS.

Adopting AI can be challenging, the pair say, and this is where SAS' ready-made AI models come into play.

Sglavo says the models are lightweight and easy to implement, as well as built with specific industry applications in mind. They're also, he says, quick to initialize.

SAS has announced a range of industry-specific models and has more in the pipeline aimed at specific industries. Read more about SAS' new AI models in our dedicated coverage here.

SAS is using the session to stress how much it's already trusted, with decades of experience working hand-in-hand with protected industries. We're welcoming John Boyd, VP Solutions Product Management at SAS, onto the stage to tell us more about how it helps its customers address the biggest challenges.

"For decades, we've worked with domain industry experts to solve everyday business departments," Boyd says. "Think credit scoring, manufacturing, drug safety. We are working throughout your entire value chain to make your jobs easier."

Boyd adds SAS has been the "gold standard" for drug discovery for decades, was the partner of choice for banks in the wake of the Patriot Act, and a key player for vaccine development during the pandemic.

Banking customers rely on SAS Solutions predictive models for credit scores, fraud detection, and governance, he says.

For example, CNG Holdings was able to get over 90% of its new customers through the application process within 90 days, using SAS Solutions, a result Boyd says was previously "unheard of" for the customer. The work also reduced the financial solution provider's false positive rate to near zero and reduce fraud by around 30%.

Fraud is a key focus for SAS, as we're now hearing from Stu Bradley, SVP Risk, Fraud and Compliance at SAS.

“You never know the face of a fraudster,” Bradley says, necessitating advanced technology and a need to get “inside the mind of a fraudster”.

To find out more, we're playing a game of 'Former Fraudster vs Fraud Fighter'. Our contestants are Tatiana Zambrano, head of fraud extraction at Truist and Frank Abagnale, bestselling author and subject of the movie Catch Me If You Can.

“50 years ago the fraudster was a conman acting domestically,” Abagnale says, adding that there was a chance for fraudsters to get emotional and decide to hold back from completely ruining victims.

“But today you’re dealing with someone thousands of miles away, they never see you, they never see your victim, you never see the perpetrator, so there is no emotion and they will steal everything from you.”

Zambrano concurs, adding that this is the reason for defense at scale.

"We're seeing a need to deploy technology that can outsmart them."

With the game in full swing, Abagnale says that attackers can carry out the kind of fraud he did with much more ease these days, given the technology at their disposal.

He adds that attacks will only become easier through technological innovation.

Asked about the future of fraud, Zambrano says that defense will rest on adopting and using generative AI faster than adversaries.

We're now handed back to Harris, who rounds out the opening session with a commitment to meeting customer needs.

"We are here to listen to your business goals and apply the right technology to you for you to achieve that," he says.

"And we are here to help you thrive in your role. SAS is building Cutting edge Al capability so you can deliver outcomes that outperform, so that you can gain a decision advantage."

And with that, the general session has come to a close. Stay tuned for all the latest from SAS Innovate 2025, right here on ITPro.

The sun has risen on Orlando once again and the second keynote is due to kick off in just under 30 minutes. We're expecting to hear more about agentic AI use cases today – off the back of a slew of AI announcements yesterday.

We're seated in the Orlando Ballroom, as the room fills up with freshly-caffeinated customers, partners, and staff (plus a select group of plucky technology journalists). Energy on the ground has been high throughout the event, with strong attendance expected for this session.

And we're off, welcoming back Dominque Moon, customer success manager at SAS, to introduce today's speakers.

Moon says that while day one focused on how data and AI can make business operations more efficient and powerful, today will center around making responsible and trustworthy AI.

To start with, Moon says, we’ll be joined by Gavin Day, COO at SAS, to discuss SAS’ relationship with Microsoft. Next, we’ll be shown a discussion between Jim Goodnight, Co-Founder & CEO and Satya Nadella, chairman and CEO at Microsoft. After that, we’ll have some in-depth looks at responsible AI deployment.

With that out of the way – and having been treated to a sizzle reel showing some highlights from day one – we're welcoming Gavin Day, COO at SAS to the stage.

Day begins by expressing excitement for the upcoming Goodnight and Nadella conversation. He then pays tribute to SAS' customers, with whom he says he's lucky to be able to work.

Day says that these are "volatile and uncertain times – perhaps the most volatile we've faced in our career," a reference to recent economic upsets such as the US' tariffs.

Amid all that, Day says, SAS stands ready to support its customers. And in this role, he says, its work with partners is all important.

2025 marks the fifth year of SAS and Microsoft's strategic partnership, Day says, with a commitment to "push the limit of what's possible to drive the best decisions, and the best outcomes for you". With that, we're passing over to Goodnight and Nadella.

Nadella begins by saying Moore's law is now "in hyperdrive", with AI innovation happening at speed.

"The real trick", he says, "is to be able to parlay that advance in modern capability into actual useful products."

"The scale of data involved has also increased at an incredible speed, and the amount of information companies have to deal with nowadays must be met with technology that can handle the complexity and scale of these workloads," Goodnight says, noting that generative AI cannot meet all needs. He asks Nadella where he thinks we are on the curve of AI and how leaders can stay ahead.

Nadella agrees, noting that Microsoft's work with SAS is testament to the fact that while generative AI may be used to make a plan, for example, the execution can still be done with deterministic workflows, ML, and statistical models.

He adds that bringing SAS solutions to Microsoft Fabric is critical, and will help customers remove friction in their workflow and cut the time to deploy AI.

We're moving onto quantum now, with Nadella mentioning Microsoft's recent breakthrough with Majorana particles. He says that quantum computing will be necessary to properly simulate nature, even as AI is used as an "emulator" for nature.

"Going forward, whether it's computational chemistry, biology, or what have you, it will be truly accelerated with the combination of quantum and AI," Nadella adds.

Both leaders say they're honored to be working together, and with that we're handing back to Day.

Day says though it wasn't previously called 'decisioning', SAS has been a leader in using data and business rules to accelerate decision-making for nearly 50 years. In practice, this looks like helpijng banks prevent fraud, hotels to forecast demand, public sector organizations identify at-risk children and mitigate flooding, and empower pharmaceutical companies to bring drugs to market faster.

"Ultimately, it's the decisions that drive your organizations, not data," Day says, adding that trust is the most important thing for SAS.

Here to tell us more is Reggie Townsend, VP, Data Ethics at SAS.

Townsend starts by acknowledging that some of SAS' announcements have been complicated, not least quantum. So he's starting simple with the analogy of staring into a mirror – a simple object, that at one point in history was considered "a revolutionary innovation".

"Like those first reflective surfaces, AI is changing how we see ourselves and how we envision our future," he says.

When AI uses data, Townsend adds, it sometimes reflects ourselves with clarity and sometimes with distortion.

"So, when Al produces a biased result, it's not a technical failure, we should stop saying that," he says.

"What it's doing is it's showing us the biases and the beneficiaries that we've embedded in our cultural and our organizational practices."

Townsend says we all have a responsibility to be responsible innovators, from industry, to government, academia, civic organizations, and individuals. He urges the audience to consider whether we want the norms of the past century to drive the algorithms of the next century.

Townsend is digging into some of the numbers around AI governance now, with data showing leaders do want to be responsible AI leaders. To read more on this, check out our detailed coverage of Townsend on trust in AI.

Townsend advocates for 'response-ability', in which leaders predict and simulate harms, to get ahead of potential risks and improve AI oversight.

"The only thing better than a quick decision, is a decision you can trust," Townsend says.

Government has a role to play, Townsend says, and here to tell us more is Mubaraka Ibrahim, chief artificial intelligence officer & CIO at the Emirates Health Services, UAE.

SAS has partnered with the organization, which Ibrahim explains is steaming ahead with AI adoption.

The UAE appointed its first AI minister in 2017, Ibrahim says, and has established a detailed AI 2031 plan. As part of this, the UAE government has now appointed 22 chief AI officers across all sectors, who work with the AI minister to collaborate on secure AI deployment.

EHS is looking to implement AI to improve healthcare outcomes, build trust in AI, and make it accessible to the population of the UAE. Ibrahim says it's already been shown to reduce provider burnout and cut costs, while improving patient engagement.

Internally, EHS has built more than 40 models for predictive analytics, having used these to forecast ICU mortality risks and model infections during the pandemic. It has also used machine learning to improve the connection between providers and patients.

We're now moving onto shadow AI, with Townsend stating that of the 58% of employees already using AI for work, 60% are doing so without company-approved tools. SAS knows governance is key to success, he says, and wants to "earn the right to help you with your approach".

Townsend is now handing over to Alyssa Farrell, director, Global Industry Strategy Health & Life Sciences at SAS, who will tell us about how ethical AI is reshaping healthcare.

Farrell begins by acknowledging healthcare data is incredibly sensitive, with organizations in the sector faced with the double challenge of innovating without compromising on responsibility.

SAS can help heathcare providers to analyze drug safety, run medical sumulations, and assess the cost of care. But trust is the number one factor here, she adds.

To give a real-world example, Farrell introduces a partnership SAS has established with two Dutch organizations: Erasmus University Medical Center (Erasmus MC) and Delft University of Technology (TU Delft).

Together, they have launched Responsible and Ethical AI in Healthcare Lab (REAHL), to tackle the responsible use of AI in healthcare.

To tell us more, we're now hearing from Michel van Genderen, an Internist-Intensivist doctor at Erasmus.

Every organization and thus also every hospital always needs to know which models are currently in production, but maybe more importantly, what is their intended use," he says. With SAS and the World Health Organization, van Genderen says his team is helping to establish a registry to track deployment and ensure models follow best practice when it comes to transparency and ethics.

Bias is also a critical issue, van Genderen says, with SAS helping to identify where data changes could negatively impact model trustworthiness. For example, he says, two years ago his team tested a model that would determine when it was safe to discharge a patient following major oncology surgery, but found a standard hospital IT update shifted the datasets that fed directly into the model's decision. It was lucky, he adds, that it was only a controlled test and not a live product.

SAS ModelOps helps deliver this real time oversight and model assessment, he adds.

One of van Genderen's major tasks right now, he says, is to help his peers adopt AI and make meaningful change with it. A recent success, he says, was the introduction of a dashboard to track decision-making for patients on mechanical ventilation, care in which his hospital specializes. The dashboard drove a marked improvement for patients, he says, and also provided doctors and nurses with clear feedback.

Whether it's patients or customers, SAS wants attendees to know it's focused on positive outcomes. Here to tell us more is Kate Parker, senior director, Customer Intelligence Practice at SAS.

"Your most powerful asset is your data," says Parker. "And SAS Customer Intelligence Solutions provide you with the ability to connect directly to your data environment and turn that data into action. Because data is the foundation for decisions that are made to positively impact the customer experience."

We're moving quick today, with many customers coming onstage to give practical examples of SAS in use. Here to tell us more about customer experience is Bennett Fox-Glassman, SVP Customer Journey at Macy's.

Fox-Glassman says Macy's is focused on the customer experience and improving enagement, which leads to profit and sustainable growth.

It's all about knowing their customers, Fox-Glassman says, while keeping it natural.

"Nobody ever comes up to me and says, 'Hey, l'd like more personalization', right? That's a term we use as data individuals, as marketers, but customers do come, and they ask for context," he says.

In practice this means making sure you can recommend relevant products to customers, send them relevant marketing emails, and ensure every interaction is right for each specific customer.

Fox-Glassman adds that Macy's is excited to drive business outcomes with its investments in technology, but urges all firms to be "extraordinarily use-case-driven".

This circles back to trust, he says, in which in Macy's case means earning brand trust from its customers every day.

The organization is still exploring generative AI, Fox-Glassman says, giving the example of a terrible output he recently got from an AI agent. For the moment, Macy's is mainly using AI internally as it gets to grips with how it can deliver the most value.

Parker's moving on now, noting that many data scientists spend 80% of the time preparing data and only 20% performing analysis – an issue SAS is working to fix.

"Providing marketers with the solutions they need, to lighten the lift of IT resources drives tremendous efficiencies across the organization and will ultimately help improve your bottom line," she says.

We're now being shown a demo, in which Parker is able to use marketing analytics within SAS Customer Intelligence 360 to track annual subscription churn. She can raise insights into the most common attributes of customers prone to churn, to better target them with marketing and improve customer retention and satisfaction.

Within the platform, Parker can also see metrics driven by AI analytics, with recommendations for improving the customer experience. For example, in the demo the platform recommends changing the eligibility for an organization's car loan approvals, as the model has identified the current guidelines are harming sales.

We're now handing back to Day, who states that agentic AI has major potential but only with strong framework in place.

Without this, he says, organizations will be impacted by all the downsides of LLMs such as hallucinations and otherwise unpredictable behavior.

Here to show how SAS helps leaders orchestrate agents is Diana Maris, product manager for Intelligent Decisioning at SAS.

Using an orchestration agent which decides which agent to call for which task, Maris says, a user can use natural language to ask for detailed data breakdowns. An example user, Sarah, asks for details on customer complaints, triggering a complex series of agents running on decisioning frameworks, to draw together the data and present it in plaintext.

The orchestration agent can call a series of specialized agents, including a Complaints Agent, Churn Agent, or Policy Agent.

Users can review the decision trees the agents follow, to ensure they fit the expected outcomes and ensure organizations aren't deploying agents they can't predict.

"You are able to stress test your agents against dozens of scenarios, making sure you validate their performance as well as the path the data takes through the decision phase, ensuring full trustworthiness and confidence before the agents make it into production and go live," Maris concludes.

To summarize the event's key themes, we're back to Day.

Day says the AI journey is simply a continuation of the mission SAS has been on for nearly 50 years and is only becoming stronger through its collaboration with partners such as Microsoft.

And with that, Moon is back onstage to officially announce the second day's proceedings have begun. Stay tuned for more analysis from SAS Innovate 2025, right here on ITPro.

Business leaders are under immense pressure to back up their decisions with data, but data skills and literacy issues are creating a crisis of confidence.

Research from Salesforce, released as part of a new product launch, found that more than three-quarters of business leaders think the growth in AI increases their need to be data-driven.

Unfortunately, though, many are struggling. More than seven-in-ten said they were under pressure to use data to demonstrate business value. Indeed, they said their career depended on their ability to use facts, metrics, and data to drive decisions.

86% went a step further, saying their careers are reliant on how data literate they are in terms of using methods and tools to extract actionable insights from data. This drive for data literacy was felt by nine-in-ten leaders in sales and marketing.

"There is an increasing amount of pressure on business leaders to use data in today’s hyper-competitive environment," said Southard Jones, chief product Officer of Tableau, Salesforce's analytics platform.

"Yet fragmented enterprise data and complex analytics tools are common obstacles leaders face that can significantly hinder confident decision-making."

Confidence in data quality is dipping

Meanwhile, business leaders don't trust the data itself. Confidence in data’s relevance to business objectives is down by 18% since 2023 while confidence in its accuracy down 27%.

Fewer than half said their data strategies fully align with their business priorities - another big drop since 2023. The answer, said Salesforce, is agentic analytics.

"Agentic analytics offers a solution by unifying data and empowering AI agents to proactively deliver trusted, real-time insights to everyone, enabling data-driven decisions for everyone – even those who don’t consider themselves ‘data people," said Jones.

These claims come amidst the launch of new features for its Tableau Next product, including three AI agents. Data Pro helps convert raw data into business language, automating extract, transform, load (ETL) processes.

Concierge is a natural language interface allowing users to ask questions in everyday terms and receive visualizations and recommendations - "What are my best sales opportunities?", for example.

Similarly, the firm said its Inspector tool continuously monitors data for key changes, identifies patterns and trends, and surfaces insights that humans might never think to look for.

"We’re shifting from basic reports to a world where AI is a collaborative decision-making partner," said said Ryan Aytay, CEO of Tableau.

"By combining AI agents with trusted data and easy-to-use tools, we’re making data accessible to everyone and transforming the data-to-action process into an automated, proactive, and insight-driven cycle. This empowers users to improve decision-making, boost efficiency, and manage risk more effectively."

MORE FROM ITPRO

• Are we facing a generational decline in digital literacy?
• A quarter of firms still don’t have a formal data strategy – and it’s hampering AI adoption
• What is data democratization and how will embracing it benefit your business?

IT decision-makers are ramping up automation efforts in a bid to tackle internal ‘data divides’, new research shows, but they’re facing significant hurdles.

According to a new survey from Jitterbit, seven-in-ten enterprises still don’t have a unified data platform, which is creating challenges in consolidating data silos.

The firm’s research showed two-thirds of enterprises are now using more than 500 applications - creating an overwhelming 125,000 potential integrations that require continuous oversight, updates, and maintenance.

With data silos growing in scale and complexity, the study noted that IT leaders are re-evaluating budgets to prioritize data integration and process automation. For two-thirds, these initiatives are soaking up 30% of their budgets, with 34% saying they account for more than half.

"The path to success is clear: businesses must break down data silos and automate workflows to thrive in the age of AI," said Jitterbit president and CEO Bill Conner.

"While many organizations still struggle to find the resources across IT, IS, and line-of-business teams to bridge this 'data divide,' the opportunity for those who can is immense. We're on the cusp of a new era of efficiency and innovation, driven by true end-to-end AI automation."

One key lingering issue, however, is security. Half of respondents said that potential vulnerabilities in third-party integrations are their top data security concern.

Meanwhile, more than a third cited regulatory compliance as a major constraint to full deployment of AI-driven automation.

When it comes to resources, 70% of the load is falling on IT teams. But among those organizations that have given line-of-business (LoB) units — finance, HR, and operations, for example — the right tools and autonomy to drive their own initiatives, there's a very positive result.

These business segments gain significant value from end-to-end automation, especially for operational efficiency and customer experience, with supply-chain management and finance set for substantial benefits.

Enterprises eye AI automation gains

More or less all of the enterprises surveyed said they'd already integrated AI into their operations - and those that were early adopters are now looking to agentic AI as the next frontier.

More than three-in-ten are already working on plans for agentic AI, aiming for autonomous decision-making enterprise AI solutions through the use of end-to-end AI automation.

"Legacy automation, designed to execute isolated tasks, is no longer sufficient enough to keep up with modern business demands," said Jitterbit CTO Manoj Chaudhary.

"Agentic AI is driving a fundamental shift — moving from task-based automation to intelligent automation with adaptive workflows that drive real business outcomes. By leveraging AI-driven decision-making, enterprises can break free from data silos and IT bottlenecks, enabling seamless end-to-end automation."

MORE FROM ITPRO

• What is synthetic data?
• Businesses need to realize automation is not about cutting costs
• AI is causing a data storage crisis for enterprises

Google Labs has made Data Science Agent available to all Colab users in a bid to help automate data analysis processes.

Google Colab is a free, cloud-hosted Jupyter Notebook where users can write and execute Python code.

According to the tech giant, the Data Science Agent can help users cut research and data analysis times from weeks to minutes, as there's no need for setup tasks like importing libraries, loading data, and writing boilerplate code.

"In fact, a scientist at Lawrence Berkeley National Laboratory working on a global tropical wetland methane emissions project estimated Google’s Data Science Agent reduced their analysis and processing time from one week to five minutes," said the firm.

Powered by Gemini 2.0, Data Science Agent users can describe the analysis they want to see in plain language. The necessary code, import libraries, and analysis are then generated in a working Colab notebook.

Data Science Agent provides fully functional Colab notebooks, Google said, and not just code snippets, but complete, executable notebooks. Solutions can be modified, with the generated code customizable and extendable.

"We’ve heard awesome feedback from trusted testers like 'Data Science Agent generated concise and high-quality code, effectively corrected errors, and proved to be user-friendly' and ‘I've already started exploring Data Science Agent and think it's a great product. This access will greatly help me streamline my data workflows and uncover valuable insights'," Google said.

Under the hood of Google's Data Science Agent

Data Science Agent works by orchestrating a composite flow that mimics a typical data scientist’s workflow, using the large language model (LLM) for task decomposition and planning.

A flow consists of individual atomic flows or subflows that specialize in concrete data science tasks such as data cleaning, data exploration, and data plotting.

Each atomic flow is a sequence of individual steps, using code execution and execution output feedback to complete a subtask and communicate downstream.

Combining natural language to code, the planning and reasoning capabilities of LLMs and code execution allow agentic capabilities such as self-refinement, error correction, and summarization, Google said.

Notebooks are generated dynamically by AI, including generated code, code outputs - for example plots - and text cells. They can be tailored by specifying the libraries, visualization types, algorithms, or evaluation metrics using natural language in the task description.

Users can also generate code for handling missing values, outliers, inconsistencies, and formatting issues, and create visualizations and summary statistics to understand the distribution, relationships, and characteristics of their data.

Similarly, they can implement hypothesis testing, correlation analysis, and other statistical techniques to draw meaningful insights from their data.

Meanwhile, predictive modeling allows machine learning models for regression or classification tasks based on your data and objective to be built and evaluated.

Google said future plans for Data Science Agent include interactive elements for user feedback and customization within the notebook generation process and enhancing natural language understanding for more flexible input descriptions.

It said it also plans to support additional data types for more complex tasks, along with the range of supported data science tasks and algorithms and larger file size upload support.

MORE FROM ITPRO

• Everything you need to know about OpenAI's 'Operator' agent
• Workday wants to make it easier to manage AI agents
• GitHub's new 'agent mode' feature let's AI take the reins from devs

Data analysts around the world remain dependent on spreadsheets for preparation tasks that take up a significant portion of their day, according to new research.

A survey commissioned by Alteryx, spoke to global data, IT, and operations analysts across five industries to determine how the role is changing in the face of new technologies and rising automation.

Three-quarters (76%) of respondents said they still used spreadsheets to clean and prepare data for analysis, with just over half (56%) stating they used dedicated data preparation tools.

The largest proportion of analysts (42%) said they typically spend between one-to-five hours per week preparing and cleaning data, whereas with 40% this takes roughly up to 10 hours a week.

One-in-twenty analysts revealed they spend more than 10 hours every week on data preparation and cleaning, however.

When combined with data collection, the global average time spent on preparation was 10.57 hours, this was the second most time-consuming activity in the analysts’ day just behind the actual analysis of the data, which came in at 11.23 hours.

Half of the respondents said the complexity of data was the greatest challenge bogging them down, whereas data quality, security, and privacy were also cited as hurdles they frequently face when preparing it for analysis.

AI and automation will drastically change the data analyst role

AI tools have seen widespread uptake in the analyst profession, with 97% of respondents stating that they have integrated AI tools into some aspects of their workflow.

Tools for analytics automation are not far behind, with 87% of analysts adding they have also integrated these solutions into their daily activities.

More than half (59%) said AI tools increased their efficiency and productivity while 46% said it had reduced their workload and stress, citing an improved job satisfaction as a result.

On a weekly basis, the analysts said that utilizing AI tools saved them 8.6 hours on average.

The time saved using these tools looks to be shaping the analyst in new ways, with strategy becoming more heavily emphasized as AI removes some of the more repetitive tasks from the analysts’ daily workflow.

The vast majority (94%) of analysts agreed that their role impacts strategic decision-making, with 87% stating that their influence on business decisions has increased in the last year.

When asked how their role has changed in the last year, 85% of respondents agreed that their ability to respond to changes in the scope or direction of a project has improved.

Nearly four-fifths (79%) also said their ability to combine multiple sources of data compared to a year ago had improved since the previous year.

Overall, 86% agreed that AI had changed their role at least to some degree, with half of these going further to say it has reshaped their role “to a great extent” over the last 12 months.

This has improved slightly over recent years, with just 34% of respondents saying the same about the last 3 – 5 years, and 16% adding that they didn’t think AI had changed their role and had been affected by AI at all during this period.

MORE FROM ITPRO

• The data analytics market is booming – here's why
• AI projects are faltering as CDOs grapple with poor data quality
• Best data science bootcamps

Only a third of businesses are making meaningful progress in AI adoption, and chief data officers (CDOs) are pinning the blame on poor data quality.

More than two-thirds (68%) of CDOs said data quality was their top challenge, according to the Ataccama Data Trust Report 2025. Meanwhile, four-in-ten struggle to maintain consistent data quality, directly hindering AI outcomes.

The report noted that trust is a critical factor when leveraging data in daily operations. Without it, organizations face inefficiencies, poor decision-making, and the risk of compliance failures, ultimately limiting their ability to achieve business objectives.

However, the report found most organizations still struggle with siloed systems.

"Untrusted data erodes every decision it informs. Without real insights into data quality, businesses risk cascading failures, from unreliable AI outputs to stalled growth," said Krishna Cheriath, chief digital Officer at Thermo Fisher Scientific.

"Trust must permeate every layer — data, models, and decisions."

Bad data leads to bad insights, the report noted, affecting decision-making, slowing down operations, and wasting valuable resources. Similarly, it jeopardizes compliance, leaving organizations vulnerable to regulatory and financial risks, and diminishes ROI.

Knowledge gaps are hampering digital progress

Knowledge gaps around data trust and governance are slowing progress, the report found, with a lack of unified standards leading to inconsistency. Without guidelines for data formats, definition, and validation, CDOs find it hard to establish a centralized system of control.

A third of organizations experience processing delays because there are too many barriers that stand in the way of integration.

"Fragmented systems bleed efficiency and inflate costs," said Andrew Foster, chief data officer at M&T Bank.

The report also found legacy systems are still a major barrier to innovation, with CDOs finding that outdated systems are ill-equipped to handle increasing data volumes.

Many systems are designed to provide period data updates, for example, rather than continuous, real-time streams. It’s this area in particular that’s causing serious trouble for CDOs ramping up AI adoption, with only a third of enterprises reporting meaningful progress on this front.

The report called for new national standards for data quality in the UK, and suggested that the proposed National Data Library - a core goal within the UK’s AI Action Plan - could play a key role in bolstering national data governance benchmarks and best practices.

These standards would ensure clear compliance guidelines while supporting the UK’s pro-innovation regulatory goals.

"The report makes one thing clear: enterprise AI initiatives rely on a foundation of trusted data," said Jay Limburn, chief product officer at Ataccama.

"Without addressing systemic data quality challenges, organisations risk stalling progress. The UK’s approach to AI regulation shows how aligning data trust principles with national standards and infrastructure modernization can deliver tangible results."

The global data analytics market is growing fast, driven by advancements in IoT, cloud computing, and AI, according to new research from GlobalData.

With data volumes projected to exceed 175 zettabytes by 2025, the total data analytics market is set to grow at a compound annual growth rate (CAGR) of 11.1% between 2023 and 2028, hitting $190 billion in 2028.

While the data analytics space is a relatively mature market, it has seen significant innovation in recent years across the four layers of its value chain - hardware, data management, applications, and delivery.

Prescriptive analytics is the most advanced innovation, according to GlobalData, telling organizations what to do next rather than just describing what has happened and why.

Machine learning techniques can now provide data-driven recommendations by parsing large amounts of data and assessing 'what if' scenarios.

The traditional data analytics vendors such as SAS, IBM, Oracle, and SAP are being disrupted by AI-native vendors, such as Cognitive Scale and H2O.ai, which aim to help companies automate operational decision-making using ML.

Similarly, the emergence of generative AI tools has led data analytics vendors to embed solutions in their platforms. Microsoft's launch of Copilot, for example, embedding ChatGPT into analytics products such as Excel and PowerBI, represents a step change for enterprise capabilities.

"The traditional data analytics vendors are being disrupted by AI-native vendors that aim to help companies automate operational decision-making using machine learning," said Isabel Al-Dhahir, GlobalData principal analyst, strategic intelligence.

"Furthermore, the emergence of generative AI tools has led data analytics vendors to embed those solutions in their platforms, democratizing access to data science capabilities. For instance, Microsoft has launched Copilot, embedding ChatGPT into analytics products such as Excel and PowerBI."

Governance, security, and skills are key concerns

The report warned that the rapid growth of data volumes and the expectation of advanced analytics brings great pressure in terms of management, governance, and security.

"The ability of GenAI to create highly sophisticated models and simulations from vast datasets raises significant concerns about the potential misuse of personal information," said Al-Dhahir.

"The risk of exposing sensitive data increases as these AI systems become more adept at generating detailed, realistic outputs. This calls for stringent data governance frameworks."

Managing data analytics pipelines means a need for more skilled workers, including data scientists, data engineers, and other data specialists. Meanwhile, data analysis requires skills in visualization, reporting, and communication.

"Organizations must attract, train, retain, and upskill their current workforce to fill these roles. Companies may also attempt to plug the data-skills gap by encouraging less technical employees to act as citizen data scientists," said Al-Dhahir.

"Businesses must build up their in-house capabilities, but in some cases, strategic acquisitions or partnerships with small data analytics service companies may be preferable," Al-Dhahir added.

"As the market evolves, innovation across data management, AI integration, and governance will shape the future of data-driven strategies."

With AI and other groundbreaking technologies growing in popularity and capabilities, more and more data is becoming available to employees who may not have traditionally had the skills or expertise to use it in their day-to-day workloads. This is data democratization – the idea that we can make data more accessible beyond simply available to data scientists or engineers – so that more people across a business can use it to derive valuable insights and shape decision-making.

There are plenty of benefits to embracing data democratization, as well as risks. But instigating a data democratization drive is a complex strategic process that requires not just a technological shift, but also a cultural and mindset shift.

The rise of generative AI has also had a massive influence on data democratization. Embracing this paradigm shift requires leaders to pursue dedicated programs across various disciplines so that their businesses can truly gain a competitive edge and make better, quicker, and smarter data-driven decisions at several levels.

What is data democratization?

Democracy refers to the distribution of power to the people of a society – ensuring that a single person or entity is not dominant without the consent of the masses. In exchange, people agree to live within certain rules and structures. Data democratization follows the same pattern, director of data at Penguin Random House, Pete Williams, tells ITPro.

"Excepting certain security and confidentiality needs, access to data should be decreed open and equal for all members of an organization who could find a use for it," Williams says. "Those colleagues should receive appropriate access to tools and training to unlock potential data use cases."

It opens up data to everyone in an organization regardless of technical expertise, removes silos, increases data literacy, and allows self-service so that employees across the breadth of an organization can use data responsibly. A central team may be responsible for gathering, governing, and distributing data, Williams adds, but structures and controls ensure people can trust the data they access.

What are the benefits and risks of data democratization?

There are several benefits to data democratization. These include reduced reliance on IT resources – allowing users to identify trends and make real-time decisions – enhanced collaboration, and improved employee engagement by allowing them to contribute to improving customer experiences, says Forrester analyst Jayesh Chaurasia. "These factors directly contribute to a higher return on data investments and enhanced market competitiveness," he tells ITPro.

Williams says that by instigating data democratization, you have a much larger pool of well-informed decision-makers across the organization, who collectively take more key decisions than were previously possible.

"Data-enabled organizations are more innovative, more agile, and have a better sense of the connectivity across domains or pillars that enable business processes," he explains. "This DNA exposure helps them understand how the organization truly functions and where it could become more efficient. I’d expect lower costs, higher productivity, more innovation, and a better connection of the product to the customer."

On the flip side, there are a handful of risks. An organization may not appreciate the loss of control over decision-making that data democratization permits, Williams adds. It can also feel that different delivery models may occur when the pace of change outweighs the control systems in place – in short if there are no guardrails or governance measures to manage this shift, it may feel like things are spinning out of control. "To use the racing analogy, 'slow down to speed up', as if you don’t brake sometimes, you’re going to spin out at the corner."

Data democratization and AI: A key trend in 2025

Has AI influenced data democratization trends? It's complicated if you ask our experts. Chaurasia describes AI as a "game changer" thanks to the way it has helped users look for and access data using natural language processing. "Interaction using natural language has also lowered the technical barriers for users to query and analyze data," he elaborates. "It is helping improve the understanding of data by creating summaries and explanations about data, queries, analytics, results, and visualization.

Williams attributes a more nuanced impact on generative AI and says in some ways it has hurt data quality. "To the non-technical user, it looks too easy, and they expect the simple provision of useful, consistent, accurate information to be solved by 'adding ChatGPT’," he says.

"But it has driven curiosity and expectation, which can be helpful to the data democratizers if they can maintain the momentum," he continues. He agrees with Chaurasia on natural language, adding the biggest impact will be the "short-circuiting of data literacy needs" and by making "deeper analytics a conversation rather than an equation".

How to implement a data democratization strategy

To become data-driven, businesses must embrace a new management paradigm, according to Harvard Business Review. Everyone within the organization must be empowered to work with data, regardless of how far outside their comfort zone it may be or their experience levels, but doing so requires several measures that need to be taken in parallel – not just new just making new technologies, tools, and systems available.

Fundamentally, argues Williams, the C-suite should want to democratize data and then focus on "instilling a culture of measurement and asking better questions".

On the technology side of the equation, says Chaurasia, organizations should invest in data fabric architecture, unified governance tools, data quality and observability platforms, and cloud-native applications. If these tools are embedded with machine learning (ML), automation, and generative AI features, then all the better.

But, as he and Williams agree, technology is the easy part. Williams adds: "It’s simple to initiate and join up a modern data stack where you can gather, process, and send data in almost any way you want; I do not see that as a barrier."

"There should be dedicated resources for change management to address user concerns and resolve resistance," says Chaurasia on the cultural changes. "Organizations need to promote a data-driven approach and a data product mindset. This helps employees understand the importance of data and the roles they play in leveraging data for growth."

This must come in addition to a data literacy program that focuses on not just tool usage but on processes, workflows, and responsibilities. Users should understand frameworks, data ownership, and the importance of data quality, he adds, with governance and quality seen as everybody's joining responsibility.

Steve McMillan, CEO at Teradata, wants you to forget what you think you know about his company – and the way that you approach AI.

From reducing egress fees by keeping data in one place, to raising our expectations on the potential for generative AI, McMillan believes it’s better to accommodate customers in the cloud rather than dictate what they use.

“When we thought about how to architect a platform, especially for the cloud, we recognized we were not going to be everything to all people,” McMillan tells ITPro.

“We recognized that there were best-of-breed tools, technologies, and capabilities out there that people want to bring to the platform. So we’ve had a kind of ‘bring your own model’ capability for a long time.”

Teradata was formed in the 1980s and is largely known for data warehousing. But that doesn’t tell the whole story, or where it is now, or even where it wants to go. McMillan says it has been on a journey over the last four years, building itself as a cloud company, but even that fails to really explain what Teradata is about.

“We think about Teradata as a platform where you can bring in the right capabilities, rather than us forcing you to use what we have,” McMillan explains.

“Although we’re known for being an enterprise data warehouse, the platform is so much more than that now because it’s got a ton of analytics built in, but it's also got ModelOps. So you can bring your own model into the environment, use Python, use languages that you want. You can put it into our ModelOps capabilities, you can do AB compares against the model as you train the model, work out which is going to have the best results for your organization, and deploy it right there, run that against production data at scale.”

The thinking here is that AI can do almost anything, at an equal pace to our imaginations. It’s of course true that ideas can be impeded by caution, a lack of understanding, and technological limitations. But as the founder of Shazam can attest, businesses can excel if leaders combine innovation and persistence.

Shazam and the application of creative persistence

The history of Shazam is almost hard to believe. The company not only began development in the late 90s and launched in 2002 – six years before iPhones and apps and a full year before iTunes and digitized music databases. You were able to get the original form of Shazam on a Nokia 3210. How did one of the most popular apps ever exist years before we even had apps?

“Defiance” is part of the answer, according to Shazam’s found Chris Barton. Speaking on stage, Barton explains that his idea, to identify any song from a short audio sample, was challenged at every step of the way and even branded ‘impossible’ by people with far more knowledge and experience than he had.

Barton isn’t a technical person – he doesn’t code, he doesn’t build programs or systems – but he had such unshakable belief in his idea that it drove him through every challenge he faced. To be able to recognize every song from a short clip required teams of people recording and labeling every available song, which they achieved by striking up a partnership with companies that provided CDs to retail outlets. Rather than buying every CD available, they spent nine months recording everything in a warehouse, creating a database for themselves and the provider.

Despite developing its own noise cancelling technology and building out a digitized database, long before either were fully mature technologies, Shazam wasn’t an instant success. Barton admits that there were six very difficult years from 2002 onwards, with a constant threat of bankruptcy looming over them and mass layoffs. But his persistence paid off and Shazam eventually became the most used app on the iPhone.

While a disruption like Shazam may look unlikely today, Teradata believes that companies should take inspiration from its story. And that message is aimed everyone.

“We want to set in the audience's head, this whole thing about breaking down barriers from an innovation perspective, inside your organization,” McMillan adds.

You can tell a lot about a company by its partners and in the case of Teradata, it has the cloud industries grand seal of approval. The big three hyperscalers (AWS, Azure, and Google Cloud) all support and work with Teradata services.

What's perhaps more telling is the services themselves. When Steve McMillan talks about breaking down barriers, he only needs to show us Teradata’s AI Unlimited service as proof. Using a business' chosen cloud environment and bringing their own large language models (LLMs), it accommodates seemingly everyone.

If there is a point of caution to add, it's that skills and training will still be a barrier. While Teradata offers all this possibility, businesses still need the right people to fully appreciate it.

UK businesses are increasingly waking up to the importance of data governance, according to Experian, but many are having trouble putting principles into practice.

More than eight-in-ten British businesses agreed that data governance should no longer be an afterthought, and that it could give them a strategic advantage. A further 86% agreed that data governance will become more important over the next five years.

With AI changing the way businesses are run and being seen as a key differentiator, nearly three-quarters told researchers that data governance is the cornerstone to better AI.

"In a world where everything is increasingly based on data, the importance of data management is crucial," said Andrew Abraham, global managing director, data quality, at Experian.

"The research shows that British businesses understand that and, by leveraging data effectively, they can gain the competitive edge and ability to deliver better services as well as ensure they remain fully compliant and secure."

The research found that nearly eight-in-ten organizations currently have a senior executive who oversees their data governance program, and 69% have a specific data governance team. More than half had other representatives from business operations or a steering committee involved in data governance.

Almost three-quarters are managing data governance in-house, and 84% have a specific data governance budget, with more than a third expecting it to increase in the next 12 months.

Implementation of data governance policies are a significant challenge 

However, respondents cited difficulties in integration and scalability, as well as poor data quality, saying this made it harder to manage data effectively and responsibly through its lifecycle. 

Similarly, the study found the three most common barriers to good data governance are having to embed data governance into existing ways of working and processes, cited by 72%; improving data quality and scalability, cited by 71%; and ensuring it keeps pace with existing technology and business models, an issue for 71%.

As a result, only 15% of organizations think their data governance efforts are fully meeting expectations and delivering the outcomes they need, while 22% see considerable scope for improvement.

Experian advised businesses to make sure they have appropriate talent to ensure a smooth implementation process. The most successful businesses, the study found, often set up a cross-departmental data governance council to identify, prioritize, and execute data-based projects.

Meanwhile, data governance investments should be chosen wisely. Almost every business surveyed said it planned to invest in its data governance approaches over the next two years. This includes investing in high quality technologies and tools and improving data literacy and skills internally.

Finally, data governance approaches and policies should be reviewed regularly, with the data governance team meeting at least once a month to discuss key items.

"While data management is crucial, a one-size-fits-all approach won't work," Abraham said.

"Businesses need to develop data governance strategies using flexible tools that align with their own unique opportunities, risks and regulations to unlock the true potential of their data."

As Splunk .conf24 draws to a close, partners and customers may be breathing quiet sighs of relief. 

While the event has touched on the expected topics of AI, observability, and security in good measure, it’s in its successful messaging around corporate continuity under Cisco that its real success lies.

Cisco announced its $28 billion acquisition of Splunk in September 2023 and the deal closed on 18 March 2024. In the months since, it has gradually revealed more of its vision for Splunk’s future.

Before the conference began there had been some speculation that this could be the last .conf, with some predicting Cisco would simply roll Splunk into its Cisco Live events. But having seen the continued energy of the crowds and leaders at the event firsthand, along with the pledges made by Cisco throughout, it seems safe to say that these rumors are incorrect.

It was also clear that Splunk would need to dedicate significant time to explaining the benefits of being a Cisco company at .conf24, tackling potential customer and partner concerns head-on.

No one knows this better than Cisco, it seems, given the company spent the entire event stating and restating that it’s not looking to rip up Splunk’s existing strategy.

Chuck Robbins, CEO at Cisco, told the audience in the first keynote that Cisco’s job is “not to screw up” what customers already have with Splunk. The loud applause and cheers that this remark elicited from the crowd speaks to the febrile nature of the event up to that point.

Despite reassurances at Cisco Live 2024 that Splunk was to remain an important player in its own right, customers and partners turned up to .conf24 needing clear acknowledgment of what will and won’t be changing under Cisco’s control.

“We believe in, we love, and we will embrace the Splunk community,” said Gary Steele, president of go-to-market at Cisco and GM at Splunk in the .conf24 opening keynote.

“That means the Splunk brand stays. That means this community that fosters innovation and coolness, that gets the most out of Splunk, we will continue to drive that and encourage it in every way we can.”

A mutually beneficial Splunk portfolio under Cisco

The phrase “Better together’ was used throughout .conf24, referring to the benefits that both Cisco and Splunk can leverage from the acquisition.

The pair have announced a raft of new product integrations across their portfolios, with benefits flowing in both directions. For example, the integration of Cisco Talos threat intelligence across Splunk security products including Splunk Attack Analyzer, Splunk Enterprise Security, and Splunk SOAR to enrich security investigations and make links between detected activity and known threats rooted in Talos’ extensive threat expertise.

On the other hand, Splunk’s pedigree in data analytics puts it in prime position to supplement and supercharge Cisco’s existing products and to act as a powerful backing force as the companies move ahead with AI.

Acknowledging Splunk’s tenth time being acknowledged as a Gartner Magic Quadrant leader for SIEM, Steele told the audience that its security and observability drive “only gets better with Cisco”.

“We'll get greater visibility, we'll get greater insights across the network, through the endpoint, across devices. And I feel like my excitement and enthusiasm is that I fundamentally believe we can drive better outcomes for you, that's what I think about every single day,” he said.

Both Splunk and Cisco have taken something of a cautious approach on AI, a shared trajectory and strategy that has helped keep the acquisition smooth. Splunk has been somewhat slow to adopt popular or generic generative AI assistants within its products, choosing to focus instead on improving its in-house model used for turning natural language prompts into data reports within Splunk.

The new AI Assistants for observability and security, alongside an improved AI assistant for producing Splunk’s Search Processing Language (SPL), are the fruits of this approach.

“I don't think you're going to see AI in itself as a thing, I think you're going to see everything powered by AI in the roadmap going forward with Cisco,” says James Hodge, GVP and chief strategic advisor EMEA at Splunk, in conversation with ITPro.

“There’s a lot more coming in the future by combining the best of Cisco and Splunk,” said Hao Yang, VP of AI at Splunk in the opening keynote. Noting that “data is the fuel for AI,” Yang stated that there’s an unprecedented opportunity for AI expansion in Cisco and Splunk working together.

“As part of Cisco, we’ll have more data relevant to security and observability than any other company and we’re confident that together we will be able to deliver amazing outcomes with AI to make you more digitally resilient.”

Once again, all of these announcements were grounded with repeated claims that Cisco is not looking to change any of Splunk’s AI roadmaps.

Tom Casey, SVP and GM of products and technology at Splunk told ITPro that Splunk’s approach to AI has been “domain-centric AI”, rooted in three core principles that set it apart from the competition:

• Domain-specific
• Human in the loop
• Open and extensible.

“While we’re integrating with Cisco and that fits really well, so you can chat from that chat experience right into the workloads … we remain open and extensible, so that continues to work and you can bring your own AI models for the industry as well.”

In its continued commitment to these principles, partners and customers can find more cause for reassurance. Cisco is pushing ahead with its AI focus, with ample access to data grounded in its detection platform ThousandEyes, and it seems both it and Splunk are in lockstep on the importance of an open approach to AI in the long term.

Encouraging and educating Splunk partners

The relationship between Splunk and its partners is a clear indicator for how successfully it is navigating the acquisition. When ITPro attended Splunk’s Global Partner Summit keynote at .conf24, it was clear that the mood in the room was one of excitement rather than concern.

Even one of the larger ballrooms within the Venetian Conference Center was no match for the attendees, with latecomers forced to stand to the side for lack of seats.

Alexandra Turbitt, GVP for alliance & channels, EMEA, tells ITPro that achieving this goodwill has been a mixture of listening carefully to partner concerns and taking advantage of the good deal of crossover already present in Cisco and Splunk’s ecosystems.

“For many of our partners they are now Cisco and Splunk partners,” Turbitt tells ITPro, noting that around 75% of Splunk’s partners fall in this overlapping category. In feedback sessions, she adds, Splunk partners have praised Splunk’s observability go-to-market, expressing that they no longer feel the need for vendors outside Cisco and Splunk to meet their observability needs.

For the remaining 25%, Splunk is encouraging them to get stuck in with Cisco and introducing Cisco team members into their contact with Splunk to ease this process. Turbitt says that there’s also “huge interest” among Cisco partners who aren’t yet acquainted with Splunk to get certified on its solutions and join the Splunk ‘Partnerverse’

There’s no doubt that Splunk is being incredibly careful with how it’s communicating the acquisition to partners, having doubled down on the message of there being no dramatic changes planned under Cisco.

With the disclaimers and logistics out of the way, Splunk put great emphasis on how its products can work together to achieve overall digital resilience. As partner strategies go, this is excellent. Having focused so much on its observability and security achievements in both keynotes, Splunk has ensured that the idea of resilience is front of mind for customers and that partners are equipped with tools tailor-made to solve this problem.

Turbitt tells ITPro that she particularly likes Splunk’s Hidden Cost of Downtime report for the same reason. The report reveals that the top 2,000 firms worldwide lose $400 billion per year due to downtime incidents and Turbitt says it provides all the relevant context to explain why Splunk’s observability, security, and AI solutions are so badly needed.

“I got really good feedback on the report in terms of numbers like this really help with conversations on a CFO level, for example, to simplify what the technology impact is on their business.”

“So they were very appreciative for helping to really get the value in front of their C-suite.”

Earlier in the year, Splunk wasn’t certain that the deal would clear before .conf24. In another world, this would have resulted in an event with the Cisco acquisition looming over it unspoken rather than up in bright lights on the keynote stage.

We don’t live in that world and anyone with a stake in Splunk should be glad that’s the case. In arguing why it deserves to keep its identity under Cisco – and repeatedly giving a platform to Cisco executives who explained how much they value Splunk as a complete package – Splunk has walked away from .conf24 having very clearly explained what being a “Cisco company” means for its future.

Welcome back to ITPro's live coverage of Splunk .conf24 in Las Vegas. It's day-two here and we have another busy morning ahead of us with the product innovation keynote. 

This morning's session will take a deep dive into how Splunk and Cisco are collaborating with customers to bolster digital resilience. Splunk SVP and GM of products and technology, Tom Casey, will lead the session, giving us a glimpse into how Splunk and Cisco are aligning closely to support enterprise customers. 

Casey will be joined by Jeetu Patel, Cisco EVP and GM of security and collaboration, to give us an overview of Cisco's now-combined portfolio following the Splunk acquisition. 

Splunk is now a Cisco company and this week's announcements come hot off the heels of Cisco Live 2024, held from 2-6 June in Las Vegas. Throughout its event Cisco emphasized the value of AI and its acquisition of Splunk, while Gary Steele talked about how both companies have benefited from the $28 billion deal.

Highlights of ITPro's Cisco Live 2024 coverage can be found below:

• Why AI matters so much for where Cisco goes next
• Exclusive: Splunk will play a key role in Cisco's AI plans, claims former CEO
• Cisco: “AI is changing everything” – including security
• Cisco unveils first product integrations since Splunk acquisition
• Cisco just launched a $1 billion investment fund for AI startups — and it's already backing Mistral, Cohere, and Scale AI
• Cisco CEO Chuck Robbins thinks AI adoption is going to be like the cloud transition “on steroids”

Cisco and Splunk's joint approach for the channel

Ahead of tonight's keynote, ITPro got a chance to catch the Global Partner Summit keynote. It's a popular event frequented by Splunk and Cisco partners both new and old – this year's talk was so full that we sat on the ground alongside other keen attendees.

"No matter how you engage with your customers, we want our partner programs to help you on that journey," said Gretchen O’Hara, channel chief at Splunk. O'Hara welcomed Rodney Clark, SVP partnerships and small & medium businesses at Cisco, who praised Splunk's innovation and partner approach and said Cisco's partner-led approach would blend well with Splunk's strategy to date.

Cisco is running a pilot program in which some of its large Cisco partners which hadn't previously been invested in Splunk are becoming integrated in the platform, which Clark said is intended to signal to its customers that Cisco is invested in Splunk in the long-term through continued investment.

O'Hara noted that around 75% of Splunk partners were already Cisco partners.

"It's on us to make this move as seamless as possible," said Clark. "We've got teams of people that are working together, I mentioned that earlier, and dedicated in every way, shape, and form to back-end systems and tools on how we rebate and ultimately go to market together."

Teasing AI innovation for Splunk's partners

We also heard from Tom Casey, SVP products and technology at Splunk, who teased “more announcements than we can possibly cover on the main stage” for partners and a unified approach toward AI in collaboration with Cisco.

The bulk of these announcements are under embargo – ITPro will be bringing you all of them as they're formally announced on the keynote stage.

All across the event, attendees are reassured that Cisco and Splunk will benefit each other and that the acquisition is the start of a bright new future for the both of them. Prior to the .conf24, I wrote that the event is Splunk's pivotal moment to explain where it sits as a Cisco company and how this modifies its strategy going forward.

Under 20 minutes until the evening keynote now – the ballroom at Caesars Forum is filling up quickly. 

The keynote is set to begin very shortly and crowd are still pouring in. Las Vegas is experiencing a heatwave right now but you wouldn't know it in this air conditioned, mood-lit keynote hall.

And we're off, with an atmospheric video montage of the kind you'd expect at a high-profile keynote like this. Splunk has taken a step away from the normal with the addition of dancers clad in LED lights.

"Bring on the skills, bring on the connections, bring on the future", the screen reads, and with that we're welcoming Gary Steele, president of Go-to-Market at Cisco and GM at Splunk to kick us off.

Advancements in the past year

"We're going to give you the skills to help you drive digital resilience within your organization," Steele promises the crowd, before telling us that tomorrow's product keynote will go into more detail on the specific announcements at this year's event.

Steele pays tribute to the advancements over the past year, which he says are in no small part down to customers like those assembled in the crowd. But for all the advancements in genetics, or VR and AR, he notes that there have been steps forward in the threat landscape that put everyone at risk. 

There's a balance here, he notes, and nowhere is this more true than in AI – which Steele describes as "probably the most transformative technology that we will see in our lifetimes".

"The speed with which we're seeing things is fundamentally changing," Steele says, adding that he firmly believes it will improve work, "make your job that much more fulfilling". 

While there are risks associated with AI, Steele says, "I do believe that AI will be an amazing catalyst for good and growth for all of us".

Rejecting siloed security

While the security community has worked hard to deliver point solutions to customers, Steele says that this has led companies down the wrong path and into siloed workloads.

Steele references Splunk's new report The Hidden Cost of Downtime, which found that $400 billion per year is lost among just the top 2000 firms due to downtime and lack of resilience.

"Without comprehensive visibility, trying to ensure the digital resilience of your entire digital footprint is a guessing game".

In response, Steele says, Splunk has committed itself to delivering digital resilience across its customers' entire digital footprints. He points to the fact that Splunk is the only company to be in Gartner's magic quadrant for both SIEM and observability, and has been for 10 consecutive rankings.

Splunk + Cisco as a recipe for success

We've just been shown a short video on the benefits of Splunk and Cisco working together, with a promise that Splunk's high-quality observability platform and Cisco's prestige with security and networking are a surefire recipe for success.

We're now welcoming Chuck Robbins chair and CEO at Cisco to explain more alongside Steele.

"Our job is not to screw up anything you do really well today," says Robbins, drawing applause from the audience.

Robbins says Cisco sees 400 billion security events per day, with massive oversight of its networks with ThousandEyes and sees "everything going on on the internet, everywhere".

"We can actually predict where there are going to be outages. So when you talk about digital resilience and you take all the insights that we have, that Cisco had before and we feed it into what you already do with Splunk... That just makes it better."

Robbins promises that in addition to innovating on its core technologies such as networking, Cisco is investing in AI and infrastructure to accelerate innovation.

"We've been building a pretty meaningful software business, we have a software business in excess of $20 billion."

He also points to Cisco's first enterprise stack for AI alongside a partnership with Nvidia, announced at Cisco Live 2024, as well as its investment across its portfolio from the data center to collaboration.

Robbins also highlights some of Cisco's most recent innovations including Hypershield, a security architecture that can detect anomalous activity to drive zero-trust changes in an enterprise's network. 

Steele notes that there are "a lot" of connections already in place between Splunk and Cisco, such as between Cisco's XDR platform and Splunk Enterprise as well as its new observability portfolio tied to Cisco's AppDynamics platform.

“What do you want people to remember?” Steele asks.

“We want to be the Splunk you love, only better,” Robbins replies, pledging that customers will feel better about Splunk in a year’s time than they do today.

On what else isn’t changing, Steele notes that Splunk is staying on course with its roadmaps and that “brand stays”. Splunk may be a Cisco company, Steele seems to be saying, but that doesn’t mean it’s watering down what it already has.

A flexible but consistent roadmap

In line with its commitment to flexibility, Steele announces the private preview of Splunk native to Azure. 

Moving quickly, Steel also announces federated access to Amazon S3 and notes that this falls under the umbrella of ‘GDI’: ‘get data in’.

All of this falls under four main points for investment and development, he says:

Striving for the best with security and observability

Steele is whipping through his slides and notes, taking in AI, security, and observability. He says that Splunk wants to create the best security and observability platforms possible, grounded in its experience and newfound connections through Cisco.

"We want you to have a single platform for threat detection, investigation, and response and make the person that's sitting in the SOC, make their time that much more effective. 

"You're going to see the automation and advancements that we've made from an AI perspective. We still believe in human in the loop, meaning humans ultimately make that final decision, but can we make it that much easier by reducing noise and driving a AI into the fundamental workloads?"

This cuts across Splunk's observability strategy too, Steele says, pointing to its federated data approach, Cisco's "best in class" traditional application management, and Splunk's Observability Cloud for modern applications.

"No one else on the planet can deliver the Insight that we can," he adds, including on AI which Splunk wants to use to drive customer outcomes even faster than before.

Steele says he wakes up every day seeking to serve customers, ensuring they're being delivered for and not adversely impacted by Splunk changes. He points to Splunk's 20 year legacy and predicts "the years ahead will be even better".

"That's a challenge for us and I'm excited to take on that challenge. So, let's bring on the future together," Steele finishes, introducing Hao Yang, VP of AI at Splunk.

Splunk's AI advancements

“Many of you don’t know me yet,” Yang says, explaining that he joined Splunk last year having previously led the AI team at Visa and on teams at Google and IBM

Yang says that Cisco and Splunk joining forces will give customers a “unique” approach to AI and empower them to solve their issues. He adds it’s the result of a year of development. He says together, customers can enjoy “unparalleled access” to the data enterprises need, as well as expertise on AI and critical security and privacy for AI.

Today, Splunk is announcing three new AI systems:

• Splunk AI Assistant for SPL
• Splunk AI Assistant in Observability Cloud (now in preview)
• Splunk AI Assistant in Security (available for preview later this year)

Together, the tools will enable users to write SPL queries using natural language, receive insights on security posture explained in understandable language, or kick off observability analysis based on contextual data.

We're shown a demonstration in which Sonal Pardeshi, senior director of product management AI at Splunk, uses the Splunk AI Assistant in Observability investigates an error with Kubernetes nodes, with recommended steps for fixing the issue.

It's not all generative AI, Yang notes. Today Splunk is also announcing Advanced AI in its IT Service Intelligence (ITSI) offering. Additions include machine learning (ML) assisted thresholding within the system, as well as a new configuration assistant which can monitor the health of established thresholds.

People at the back of the arena just screamed and shouted in excitement at the AI in ITSI announcement – “There you go,” laughs Yang.

A customer security story with Splunk

Splunk has made a lot of noise about its commitment to customers during the keynote to this point, but we've not heard from any directly. That's about to change, as we welcome Stephanie Franklin-Thomas VP and chief information security officer (CISO), Medtronic and Patrick Coughlin, SVP of Global Technical Sales at Splunk.

Franklin-Thomas says that understanding her risk as a CISO is her number one priority, followed quickly by convincing all 95,000 of Medtronic's people to be the firm's front-line of defense.

"It's really making ourselves available to everyone," Franklin-Thomas notes, adding that the phrase "never let a hack go to waste" can be put to good use. For example, leaders with their eye on the mistakes other firms are making can pass this information onto their employees to provide real-world examples of why being security-conscious is so important.

"If you're a security person out here and you haven't spent time with your infrastructure side... you need to be having coffee with those people," Franklin-Thomas adds.

Medtronic uses Splunk to look for anomalies, as well as understand its global supply chain, but Franklin-Thomas says the "coolest" thing it uses Splunk for is real-time dashboards that help it balance labor to deliver its products and "save lives".

Prompted on the potential benefits of AI by Coughlin, Franklin-Thomas suggests that AI could be used for early detection of illnesses. Medtronic is already using AI for incident response and for a chatbot for business functions though not with Splunk – room for another partnership down the line, she suggests.

The Splunkies

It's time for The Splunkies, Splunk's annual awards show. Our hosts for this evening are Toni Pablovich, CCO at Splunk, and Alexandra Turbitt, GVP of alliance and channels at Splunk.

The awards come in five categories: 'The Visionary', 'The Inventor', 'The Builder', 'The Champion', 'The Change Agent'.

The hosts are joined onstage by Buttercup, Splunk's horse mascot.

This year's winners are:

• The Visionary: Chloe Foulkes, HSBC.
• The Inventor: Benjamin Maes, technology solutions manager as SaskTel.
• The Builder: Harold Murn, senior systems engineer at Atlassian.
• The Champion: Craig Woolley, CIO at Louisiana State University.
• The Change Agent: Marin Todorov, director of IT at HelpMeSee.

Everything you need to know about day-one at Splunk .conf24

Yesterday's keynote session was a rapid-fire experience for all involved. 

Former Splunk CEO Gary Steele and Cisco chief exec Chuck Robbins took to the stage to give attendees a run down on all the latest product developments post-acquisition. 

We heard extensively from both at Cisco Live 2024 last week on how the two companies are combining their shared expertise to provide a more comprehensive portfolio of solutions for enterprise customers, and yesterday we were given yet more details. 

Robbins promised attendees that while Cisco plans to continue ramping up investment in core technologies such as networking, the tech giant is drawing upon Splunk's extensive data expertise to accelerate AI and infrastructure innovation. 

It seems that Splunk and Cisco are very much a match made in heaven. The duo showed a short video to attendees outlining a lot of the work that's ongoing at Cisco since the acquisition. 

Observability is critical in the age of AI

Something that stuck out for attendees yesterday was the sharpened focus on observability. Steele told the crowd that while the security sector continues to provide intuitive solutions for enterprises, many have fallen into the trap of building siloed workloads, which is having a negative impact on operational cyber resilience. 

Splunk's report The Hidden Cost of Downtime, published this week, found that around $400 billion per year is lost to downtime among the top 2,000 firms globally. A key factor in this is a lack of resilience, the report concluded. 

"Without comprehensive visibility, trying to ensure the digital resilience of your entire digital footprint is a guessing game," Steele said yesterday.

ITPro caught up with Splunk execs to discuss the findings of this recent study, and what enterprises can do to prevent costly downtime events - which you can find below. 

• Downtime costs enterprises $400 billion per year – and Splunk says AI and security are key to preventing this

Splunk product announcements from day-one

We also had a slew of product announcements from Splunk yesterday, and as you might've guessed, they were very much centered around its current work in generative AI. 

Three new AI systems were unveiled by the firm, including:

• Splunk AI Assistant for SPL

• Splunk AI Assistant in Observability Cloud (which is now in preview)

• Splunk AI Assistant in Security (available for preview later this year)

Splunk said these tools will enable users to write SPL queries using natural language, and gain detailed - yet easily digestible - insights on their security posture. 

Attendees were shown a demonstration in which Sonal Pardeshi, senior director of product management AI at Splunk, used the Splunk AI Assistant in Observability to probe an error with Kubernetes nodes, with recommended steps for fixing the issue.

The product keynote will kick off in less than one hour and Caesars Forum is already buzzing with attendees. This is a bumper keynote session running for 90 minutes and will unpack all the latest announcements in detail.

There's a small fleet of minibuses running to and from the Venetian, where the rest of the conference is taking place, as it's too hot to walk the distance between the two. At 7:40am, it's already 30°C (86°F) in Las Vegas and set to hit a high of 107°F (42°C) today.

Attendees have been greeted with a giant version of Buttercup, Splunk's pony mascot. Those who wander the .conf24 show floor will see other representations of Buttercup, along with Splunk brand shirts, hoodies, capes, and even fezzes.

As this year's event is taking place during Pride Month, many attendees are also wearing pride pins. In December, Splunk received its fourth perfect score of 100 on the Human Rights Campaign Foundation’s 2023-2024 Corporate Equality Index (CEI), which assesses LGBTQ+ workplace equality in US workplaces.

We're just minutes away from the start of today's keynote – the ballroom is almost full. The talk is running slightly behind, but we've been told it's about to begin.

Greater resilience as a mission

And we're off, beginning with the same video montage as yesterday themed around the phrase "Bring on the ____" with the blank filled with words like "innovation" or "future".

Here to kick us off is Tom Casey, SVP and GM of products and technology at Splunk. He's straight into musing on the opportunities and challenges of AI.

"We need to catch the issues when they're small so we can focus on our work. We need stronger analytics so we can find new opportunties as well.

Casey gives the example of HSBC, which is currently processing more than 60 billiopn events per day across 150 data sources using Splunk. He also notes that customers such as Intel and BMW are using Splunk for observability and resilience.

Casey says that true digital resilience only comes from seeing the full picture, rather than just managing one surface, as incidents could arise internally, from third parties, from CSPs, or from the public internet."

"Today, you're going to hear how we're changing the game around how you manage your data with Splunk, how we're bringing together metrics, logs, traces, so that it's more manageable for you in an increasingly federated and distributed environment."

With Cisco, Casey says, Splunk can use "networking leadership" in the form of Cisco's services and products such as endpoint visibility. Here to explain more about how Splunk and Cisco can deliver the SOC of the future is Jeetu Patel EVP and GM, security and collaboration at Cisco.

It's Patel's first time at .conf and he jokes that he's got extra pressure as his wife is in the audience. Casey and Patel acknowledge that Cisco and Splunk have worked hard together to 

Cisco's new approach to security began two years ago, Patel says, with a focus on an integrated approach to security.

"The one who has the most amount of data that's the most effecitvely correlated is the one that can best detect breaches and threats and keep the world safe."

Patel says Cisco's threat intelligence service Talos processes around 550 billion security events per day, and Cisco has been keen to plug it into Splunk data to make these even more relevant to its customer base.

Cisco Talos will become fully integrarted within Splunk, Patel announces, with "all of the data enriched so that you can have a far better ability to go out and do detections and compress the time to investigation."

Casey and Patel touch on the issue of telemetry to catch lateral attacks within networks, which is impractical to converge into a single location. Its solution is to use Cisco's XDR to filter the signals and connect this with Splunk for more targeted detections.

Moving on, Patel highlights the issue of identifying vulnerabilities. Cisco Hypershield, which can detect vulnerabilities, roll out patches, and run analytics across a highly segmented and distributed portfolio. Casey describes it as a "fabric rather than a fence, noting that you can bring security to your workloads rather than the reverse.

Patching a vulnerability can take up to three days across verticals, but Hypershield can cut this down to "days and hours", says Patel.

"Great products get created by companies, but movements get created by communities," Patel says to round off. "We are not going to screw up Splunk, and more importantly we are going to nurture this community because it's all of you that should be proud of what you've created."

'Changing the game' for data management

Casey says that data is the real differentiator for enterprises and that Splunk recognizes it needs to continually innovate in this area.

To this end, Casey announces new Splunk Unified Data Ingestion. This will enable customers to ingest data across their metrics and feed it into Enterprise Security, the Splunk Observability Cloud, and in Cisco AppDynamics in the future.

Splunk's mission for data is based on three main points:

• Access the right data
• Apply the right analytics
• Accelerate the right actions

"Ultimately, all of this is designed to help you be reactive when you need to be but to be efficient and effective in doing so. It gives you more time to find new value to and to differentiate your company in the kind of services that you provide to customers and your employees."

Here to give us a more practical example of how this can be realized is Faya Peng, VP of Platform Product Management at Splunk and Nimesh Bernard, head of observability at Fannie Mae.

Bernard says Fannie Mae set out to make its data more available for stakeholders through Splunk, having struggled with contextualization and data resiliency.

Using Splunk, Fannie Mae has been able to consolidate its tools "without compromising on the capabilities," says Bernard. He adds that it has already achieved a return on investment with its licensing cost.

"At the end of the day, it's all about the business," says Bernard. "If a security issue happens, how do you connect that to the business impact? If latency is happening or cloud services ae down, how do you connect that to a business impact?"

In the future, Bernard says Fannie Mae will be pursuing AI through Splunk for adaptive thresholding, remediation recommendations, and proactive alerting.

Data management for resiliency

We're moving through announcements quickly. Peng acknowledges that data management is an exponential issue and can be especially overhwleming and chaotic across different cloud providers and legacy systems.

Peng says that customers can use Federated Analytics to power enterprise security detection, or for monitoring to improve performance.

Here to show us more is Lizzy Li, principal product manager at Splunk.

Li shows us an example in which an admin can use Splunk Data Management and Federation to optimize where its data lies. Using Ingest Processor, Splunk's cloud data processing solution, the admin can look at their virtual machines hosted in AWS, and choose which data should reside in Splunk and which can be sent to a long-term storage option such as Amazon S3.

Through Federated Analytics, the admin is shown a Windows Defender security alert noting that a user's account has bee compromised. Using Splunk Enterprise Security, the admin can see that MFA has been disabled and the user's IP originates in China. They can then use Splunk's SOAR capabilities to disable the account to address the incident.

Splunk's security focus

It's Splunk's tenth year as a Gartner Magic Quadrant leader for security and here to tell us more about security is Mike Horn SVP and GM of security at Splunk.

Horn says that SOCs face hundreds if not thousands of points to investigate and track, with the data landscape constantly getting more complex with more silos and cyber security skills in short supply.

"What if you could empower your teams to have a streamlined investigation experience to more auickly and easily find the threats that matter? We call that the SOC of the future and we're omn a mission to help your teams navigate the overwhelming volume of alerts you face every day and help you find and mitigate those threats.

To give firsthand experience, Horn welcomes Sean Mason, MD, Cyber Defense at United Airlines.

Mason says he's facing the 'usual suspects' of state-backed attacks from Russia and China as well as ransomware units, along with more unexpected attacks such as from paparazzi trying to figure out where notable people are flying to or from.

Mason says that using Splunk, United has been able to customize its security posture  and already uses products like Splunk Attack Analyzer to detect phishing and malware, describing it as "amazing".

He adds that United is already seeing AI attacks in the wild and that it's one of the things that "keeps me up at night".

"Honestly, I think that space is very nascent from a defender standpoint. So we're trying to figure out how we can combat this stuff but on the flip side how can I enable my defenders to use this stuff to be stronger, faster, more efficient?"

New security products

Horn segues from Mason's praise of Splunk products to announce some new security products. First up is Splunk SOAR 6.3, which automates common analyst tasks.

Splunk Attack Analyzer, which is generally available, can be integrated with ES and SOAR to enable end-to-end threat response and analytics, Horn says, noting that it's already very popular with customers.

Another new product available in early access is Asset and Risk Intelligence, which can speed up security investigation rooted in the context of specific assets. Once the tool has identified an asset it continually updates it with information such as its IP address.

"We've all heard the old adage that you can't protect what you don't know about," Horn says.

As a final security announcement, Horn unveils Splunk Enterprise Security 8.0, which rolls the Mission Control investigation feature directly into the platform.

Horn welcomes Katie Brown, director, Technical Interlock at Splunk, to show us an example of Enterprise Security 8.0 in practice.

Brown says Enterprise Security 8.0 can reduce incident response down significantly, to as little as “a matter of minutes”.

New risk-based alerts trigger whenever it detects risks that exceed a specific threshold within an environment. From the Enterprise Security 8.0 dashboard, the user in Brown’s demo can see details such as related dashboards to the finding, adaptive response actions, and the finding’s owner.

From the same page, the user can analyze the PowerShell string associated with the finding to confirm that it’s a true positive, or in other words has been identified as malicious. “This is really, really powerful,” Brown says.

The system can rate the risk based on real-time information, and new detection capabilities in Enterprise Security 8.0 allow users to check when findings have been modified – in the example, the security analyst can flag a finding to be checked by detection engineers.

Horn rounds his section off by announcing that the new integrations for Cisco Talos in Splunk, which he says will "automatically enrich information" for customers with indicators of compromise. It will be available for customers who already have Enterprise Security and SOAR at no extra charge.

We're now moving onto observability, the second plank of today's keynote – here to run us through it is Patrick Lin, SVP and GM of observability at Splunk.

The complexity of observability

Lin opens by saying that complexity is not going away for customers, as enterprises look to repatriate cloud workloads and hybrid workloads have to be overseen.

"There are endpoints, applications, third-party resources you tap into, infrastructure residing in your data center or in the cloud – and what happens when that doesn't work?"

Across all this, Lin says, companies have to maintain observability to know what's working and what's not. At Splunk, he says, there's a "better way" to eliminate stress from a single point of access.

"We are the first company to deliver observability for the entire enterprise," he adds. "Splunk, now supercharged by Cisco, is bringing you full-stack observability that you need to see across your digital foorprint."

Customers know Splunk for log analytics, microservices through APM cloud, and infrastructure monitoring but Lin says that combined with Cisco, Splunk can "redefine the game" and extend this observability to traditional applications.

We're being shown a customer testimonial now, with word from insurance firm Progressive on how Splunk observability has helped the firm. Here to tell us more is Jonathan Moore, domain architect at Progressive and Mimi Shalash, observability strategist at Splunk.

"People are prepared for what's predictable because you measure what's easy," says Shalash, noting that while Splunk knows "a little data can go a long way", it's also keen to show that context can go even further.

Moore says that when Progressive's customers have an accident, it's imperative that they have the least stressful experience possible. Using Splunk Synthetic Monitoring, Progressive ensures its customers can access the digital services they need, when they need them without downtime or network issues.

We're shown an example in which a user can see errors through the Splunk Cloud dashboard, which they can then check in more detail through the APM overview to see the upstream and downstream consequences of the error in question.

All of this is rooted in OpenTelemetry, which can be used for free out of the box.

In the example, the user can confirm that despite the error customers are still able to make transactions despite the error. To investigate it in more detail, APM can get more granular to surface the Kubernetes cluster causing the issue. 

A color coded visualization of the node shows that the issue lies with memory pressure, caused by a single pod consuming too much memory. From here, Moore says, "I have everything I need. I have to go reach out to these teams that can remediate this issue quickly and efficiently". 

Moving onto AppDynamics, Moore says that the tool is impressive in the speed and standardization that it gives to developers.

"It gives us a new power that we didn't have before, to get that context to reallt understand what's going on fast."

Moore adds that with AI Assistant for Observability, architects can use these tools with confidence and easily run processes such as input lookups.

Lin is back onstage and notes that observability isn't just about keeping your eye on one silo but making sure you know what's going on everywhere to mitigate emerging problems.

Splunk is committed to innovating based on customer feedback, Lin says. He gives an example of checking the Splunk Cloud dashboard in the evening and seeing a whole list of errors. Using Splunk Observability Cloud, Lin can see metric and trace data directly within the dashboard next to logs.

We're getting more detail on Splunk AI Assistant in Observability Cloud, which Lin says can deliver insights to users based on natural language input. For example, a user can identify a Kubernetes cluster that's down and ask the system why – in the given example, it quickly provides the same answer discovered manually in the earlier demo.

"This is just one of the many things we're doing around AI and in this case, we're allowing you to just chat with your data and from there surfgace insights about what's going on, service correlations that might be very helpful to you."

To round it off, Lin says that if the audience remembers one thing from his section it's that "observability is critical to your organization's security resilience".

Casey's back on stage to end the keynote, calling for a round of applause for all of the product development teams who worked on the day's announcements and the customers who shared their stories onstage.

"Let's keep learning and developing greater resilience together," he says.

And with that, the day-two keynote is complete! Stay tuned for more detail on all the latest products and updates from the rest of the event.

Snowflake Data Cloud Summit 2024, which is taking place this week, is set against the backdrop of rising interest in AI and data collaboration. While it would be hard to call AI Snowflake’s bread-and-butter, the booming field is reliant on the company’s home turf of data. 

With more than 500 sessions on the agenda, the summit is positioned to offer valuable insights in data and the cloud, as well as more emergent technologies.

Snowflake has made great in-roads in the generative AI space over the last year, having unveiled a slew of plans at its 2023 conference. Since then, however, it's ramped up its activities, having announced the launch of its new enterprise-grade large language model (LLM). 

This year, we can expect the company to build on its plans and further articulate its Ai strategy for 2024 and beyond. 

Here are four things to look out for at the summit.

Artificial intelligence

AI's role in enterprise technology is expanding, particularly in areas like predictive modeling and analytics. The agenda at Snowflake Summit reflects this trend, dedicating over 140 sessions to generative AI and large language models (LLMs).

Even the keynote has an AI glow to it, with Snowflake's CEO Sridhar Ramaswamy being joined by NVIDIA co-founder and CEO Jensen Huang during his keynote. The chip maker has been making a name for itself in the AI space for some time, notably with its GPUs and, more recently, NeMo toolkit for conversational AI.

The two companies announced a partnership at last year’s summit, so it will be interesting to see how this year’s event builds on that.

We will also likely hear more about the company’s recently-released Arctic LLM, which was revealed in April this year.

Data collaboration and privacy

With the rise of data collaboration comes the challenge of maintaining privacy and security. The Snowflake Summit 2024 will delve into privacy-preserving collaboration techniques, addressing the complexities of sharing data without compromising security. Sessions will explore best practices for data governance, ethical AI innovation, and maintaining data quality to ensure trustworthy analytics.

This all comes against an unfortunate backdrop however. 

The company was recently forced to deny its systems had a vulnerability following successful cyber attacks on two customers, Santander and Ticketmaster. Instead, CISO Brad Jones claimed the fault lay with organizations not using two-factor authentication.

While it’s unlikely that the issue will be discussed at the summit – at least not by Snowflake executives – it will doubtless be in the back of some delegates’ minds.

Real-time analytics and streaming data

Real-time data processing is becoming increasingly feasible, prompting businesses across industries to reconsider its value. Advancements in cloud technology and stream processing engines have significantly reduced latency, making it possible to analyze data as it's generated. 

This capability is particularly appealing in sectors like finance, where real-time fraud detection can save millions, or in manufacturing, where instant machine health analytics can prevent costly downtimes. 

With Snowflake’s core business being data warehousing and analytics in the cloud, it’s certain that this will be another area of focus for the conference when taken as a whole, even if it’s not high up the agenda on the keynote.

Customers galore

Every conference features at least some customer case studies, but at Snowflake Summit delegates will be spoilt for choice. Household names like Deliveroo, CNN, and SurveyMonkey are just some of those holding breakout sessions to talk about how they are using Snowflake’s platform in their businesses. 

Notably, however, while the keynote may be wrapped in AI fanfare, these sessions are still firmly rooted in more traditional data management and analytics.

ITPro editorial director Maggie Holland will be attending Snowflake Summit from 3 – 6 June. Click here to catch up on all the news and analysis from the event.

We’re living in the age of data. From live sales dashboards to cloud-based data platforms and API integrations – not to mention data from vendors, distributors, and other external sources – channel partners have never had access to so much information.

As well as presenting an opportunity, the data age poses a series of practical and operational challenges for partners – not least how to capture, unify, and make sense of all the numbers in order to generate actionable insights that improve business performance and ultimately drive revenue and profit.  

Data deluge

Westcon-Comstor’s new Bridging the Gap report reveals that many partners are in the thick of these challenges, and wrestling with how to use the wealth of information they have at their fingertips to transform and grow.

Indeed, after surveying more than 500 decision makers at value-added resellers, managed service providers, and systems integrators across nine countries including the UK, we found that access to the right data is the biggest obstacle facing partners in their pursuit of recurring revenue models.

Handling multiple diverse data types emerged as partners’ biggest practical challenge in the application and utilization of data, with more than half of those surveyed (56%) highlighting this as an issue.

This is to be expected given that partners often deal with a deluge of data from multiple vendors, customers, and internal systems, increasing the chances of inconsistencies in format, structure, and quality. Integrating and analyzing such diverse data sets, while introducing automation, can be time consuming, technically complex, and require specialized skills and tools.

As Samuel Cordery, head of Sales Operations, Client Success, and Lifecycle at UK-based ITGL put it, “[o]ur biggest challenge with data is the volumes we receive, and second to that would be that it resides in so many different places and it often conflicts.”

The data opportunity

Despite these challenges, there is  widespread recognition among partners of the importance of data in their current and future business models – as well as a firm desire to unlock data’s full potential. 

For example, many respondents acknowledged the need to bolster their internal capabilities to address gaps in how they manage and use data for lifecycle selling.

There is a strong desire among partners across all four continents where the research was conducted to become more sophisticated in their use of data in order to both maximize cross-sell and upsell opportunities and drive adoption of new products and solutions.  

Investing for the future

Almost all partners (95%) marked data as a key investment focus for the next 12 months, with a perhaps surprisingly high 70% stating that they already have an in-house data and analytics function.

This willingness to invest in data capabilities demonstrates a recognition amongst partners that data analysis can help them accurately identify customer segments, predict buying behavior, and offer proactive customer support – leading in turn to improved customer satisfaction, retention, and sales.

Most tellingly, 97% see data as a central component in their journey to a subscription, software, and services-based future.

Bernardy Suhendra, head of Data and AI at Indonesia’s PT MII, summed up the sentiment perfectly when he told us, “[a]ccess to the right data, at the right time, for the right users is one of the most important aspects of organization in this digital era to handle business challenges. Without data, decision-makers can only make decisions or analyze their performance based on gut feel.”

JP Smith of Kenya-based Dimension Data added, “data is crucial for making calculated business decisions – only if you know how to use it.”

Less noise, more insights

The overarching story of our research is that partners see data as the new gold. In fact, more than one used that exact phrase. So, the question becomes, how can partners overcome the challenges they’re facing and succeed in mining this new gold?

A major piece of the puzzle lies in improved collaboration.

Many partners are looking to distributors to help them close gaps in data and help complete their journey to a subscription, software, and services-based future. In fact, 59% strongly agree that the future of distribution lies in the provision of meaningful customer and market insights.

In this context, it is vital that those across the channel ecosystem, including distributors and vendors, step up and provide partners with the actionable insights they need. Crucially, this must be done in a measured way that simplifies matters and helps partners to see the bigger picture, rather than simply adding to the noise.

Nicole Oliveira from Altron Digital Business in South Africa said the channel can derive significant benefits by harnessing the power of data.

“Distribution can add big value through data. Not only just data on existing install bases and what was previously purchased, but also looking at white space analysis and new opportunities."

Striking gold

Despite challenges with data utilization, there is widespread acknowledgement of the strategic, pivotal role data plays in shaping the future of the channel.

Partners know that quality, actionable data will help them identify new opportunities, improve their forecasting, efficiency, and decision-making – all resulting in an increase in sales and revenue.

Now it is down to the channel as a whole to turn the data challenge into an opportunity, and strike gold in doing so.

Data has long been a vital part of sports, helping coaches understand how teams are performing and how to get the most out of players. More recently, high-quality video has made it possible to analyze the smallest aspects of a game or a player’s performance. But it also brings with it some problems – like how to store all the data in the most accessible and cost-effective way.

The Boston Red Sox Major League baseball team is at the forefront of the data revolution in sports. “It’s a pretty dynamic environment and one where technology is really the underpinnings of a successful sports franchise,” says Brian Shield, SVP and CTO at the Boston Red Sox.

“You are only as good as your data and so there’s a big focus here, whether it’s traditional data, analytics data, video data, or fan-level data, really ensuring we are doing a very competent and capable job of capturing that and managing that,” he said.

“Data is becoming our lifeblood. In sport it’s growing at a highly accelerated rate, especially video data,” Shield tells ITPro.

Baseball teams generate a huge amount of video during games which creates the challenge of storing and retrieving it. The Red Sox can have up to 50 cameras across seven locations, shooting hundreds of frames a second, for two to three hours of a game. That can create terabytes of data for every game.

“In baseball, we have very high-performance camera systems that we use for analytics. In sports you can do analytics in a lot of different ways and in baseball the vast majority of that is done via camera analytics,” Shield said.

“If you’ve got 40-plus cameras shooting hundreds of frames per second for any given game, that’s not going instantly to the cloud. It needs to be stored and processed, so we can perform analytics you’re going to need some kind of tiered storage to be able to support that model.” 

The team is using Hot Cloud Storage from Wasabi to store its assets in a readily accessible archive which also helps to keep its cloud costs down. Wasabi’s team will use its technology for everything from Microsoft 365, historical and real-time video archives to Google Cloud Platform and Azure backup and recovery, as well as Internet of Things data, and for surveillance data from cameras and devices throughout the ballpark. 

To prepare for upcoming games, teams will use high-resolution, multi-angle video which means a lot of storage. Wasabi claims the economics of its offering will allow the team to expand the range of content that it can analyze to improve its decision-making.

Wasabi Hot Cloud Storage is cloud object storage, which the company sells advertises as offering one tier of highly-secure, low-latency storage. It’s intended for use-cases such as active archives, offsite backups, and hybrid cloud storage. Unlike many other cloud options.

“What we were looking for was a partner who could help us tier our storage,” Shield tells ITPro. “We were looking for an underlying technology that could be our storage repository, be cost effective and support a model where we could have fairly rapid retrieval for some of the key assets.” 

Beyond huge volumes of video data that the team creates with every new game, the club also has lot of other video assets from across its long history. “The Red Sox is a storied franchise that goes back over a hundred years; we’ve got a lot of video, some of that video has been ingested with the appropriate metadata. But we have lots of other video content that has not,” he adds. 

Applying metadata to swathes of archival footage is something that has become more feasible through AI tools. 

“There’s a whole host of different things that are now within our reach that historically would have been challenging for us,” Shield says. “So, I look at our solution as not only does it solve for a lot of our more traditional IT challenges, such as ‘how do I cost-effectively store and retrieve content’ but also how it aligns with our ongoing challenge in a video world.”.

“What Wasabi does is it creates a common data store for a broad range of assets that we have. We are using Wasabi as the underlying storage and retrieval environment horizontally across those different vertical environments,” Shield adds.

Big data stores that need to be kept long term but also where fast access is required can be expensive in some cloud settings. Shield says that it wouldn’t have previously been feasible to keep that scale of video data in the cloud.

While some might think that moving data to the cloud is the end of the problem, that’s not necessarily the case.  

Shield argues that it’s always more complex in reality: “any CIO or CTO who just decides ‘let’s to move everything to the cloud and it will just sort itself out’ is really abdicating their responsibilities”. This also means the issues you have to consider with storage are different.

“It’s not just how efficiently can I operate on any one cloud, now that you are pulling all these things together – you’ve got on-prem, edge, private and public cloud storage and underlying partners like Wasabi – it’s a real opportunity I think for organisations to more cost-effectively know how to allocate resources where it makes most sense,” Shield says.

Running the tech at a sports franchise means thinking about three different important groups at the same time, encompassing the players, the employees, and the fans. 

“You have not only the challenges like any traditional business of running a proper business with [enterprise resource planning] (ERP) systems, sales systems, Salesforce, and all these traditional tools, you also have a whole other side of your business which is around player performance. ow do we get the most out of our players, how do we support them, how to support the team that supports those folks, all of the medical and wellness work, and everything else that goes into it,” he says.

“Finding the right combination of how we can service and support those is a lot more complicated than running a tradition business with traditional assets,” he adds.

Professional services specialist PwC UK has announced a new partnership with London-based data and analytics software provider Quantexa.

Quantexa’s Decision Intelligence (DI) Platform is designed to connect siloed systems and visualize complex relationships to provide a single, streamlined view of an organization’s data.

To date, the company says it has over one trillion siloed data records unified, with its platform enhancing operation performance with over 90% accuracy and 60 times faster analytical model resolution when compared with traditional approaches.

The collaboration with PwC UK will see the DI Platform installed across the professional services firm’s infrastructure to pull together multiple sources of third-party data on UK companies. 

As a result, PwC said it will be able to serve up detailed insights to clients about their customers, which they can then use to support their own data analytics and make faster, more informed decisions that will help protect, optimize, and grow their business.

In an announcement, the company said the move will help tackle ongoing industry challenges surrounding supply chain and procurement risk, financial crime, fraud, Know Your Customer (KYC), as well as data management and customer intelligence.

“I am very excited that PwC UK will be joining forces with Quantexa to help our clients move from strength to strength,” commented Scott Samme, financial services partner at PwC UK. “Supporting our clients on their journey to become a data-driven organisations means embracing automated decision-making as an integral part of everyday operations.

“PwC UK will be leveraging the latest technology and AI alongside Quantexa to give our clients a 360-degree view of the companies they are doing business with and how they are connected to them.”

Samme said this information will help organizations slow down or prevent fraud, enforce compliance, dispense benefit funds, and secure borders. Clients will also be able to leverage the data to identify fraudulent activities, accelerate revenue growth, and enhance the overall customer experience.

“Comprehending these challenges is daunting and costly,” he continued. “By instituting a unified, integrated, and dynamic view of confirmed entities and networks, these challenges can be tackled with greater speed and accuracy, higher performance, and at scale.”

Databricks has launched a new Data Intelligence Platform it says will enable customers to more efficiently manage data using artificial intelligence. 

The announcement represents a shift beyond the firm’s long-established Lakehouse data architecture, integrating technology from recently acquired MosaicML to leverage AI on its all-new intelligent data platform. 

AI will be the foundation of the platform, Databricks said, and will use AI models to understand the semantics of enterprise data and provide a raft of features to make day-to-day data operations more efficient.

Introduced by Databricks in 2018, the flagship Lakehouse software provided users with a unified system to manage a wide variety of data types across an enterprise through a singular data management experience. 

The Databricks Lakehouse brought new functionality to enterprise users that traditional data warehouses and data lakes could not. This included providing a single solution through which they could store and manage unstructured data, semi-structured data, and data with high variety, velocity, and volume. 

As an increasing number of businesses continue to rely on collecting and analyzing vast amounts of raw, unstructured data, this functionality has become extremely valuable.

Using data lakehouses, businesses could query all data sources and govern the operations that use this data, such as business intelligence (BI) or AI. 

These platforms were not without challenges, however. One being a significant technical skill barrier limiting who could use the service, as querying data requires proficiencies in SQL, Python, or BI.

Other weaknesses included requiring extensive data curation, needing to ensure adequate governance and privacy of the data, and requiring highly skilled personnel to manage the complex system in order to avoid poor performance.

All of these challenges stem from the platforms themselves lacking an understanding of the data and how it is used.

Databricks eyes deeper customer insights

Acquired for $1.3 billion in July 2023, MosaicML provided Databricks with the ability to integrate sophisticated AI models into a data intelligence layer – DatabricksIQ – which underpins its new data intelligent platform.

The DI platform will use AI models to automatically analyze the data and how it is used, offering organizations new capabilities. 

For example, users – regardless of their level of expertise – will be able to work with internal datasets using natural language prompts. The intelligent platform is able to learn specific company terms and respond to complex questions users may have about the data.

Furthermore, AI systems will be able to optimize how data is organized based on its usage, as well as automatically detect and alert enterprises when their data is being misused. 

Databricks’ integration of AI into its data platforms means end-users will be able to gain insights by easily querying the data while helping data teams manage the data more effectively.

The DI platform will also simplify the development of enterprise AI applications, the firm said. DatabricksIQ will be directly integrated with Mosaic AI, enabling enterprises to create AI applications that can understand their data. 

This will give enterprises the ability to train custom models on their own proprietary data, including building chatbots powered by these models.  

Databricks sees AI as a transformative technology, particularly for data management.

“[T]he impact of AI on data platforms will not be incremental, but fundamental: massively democratizing access to data, automating manual administration, and enabling turnkey creation of custom AI applications.”

It believes the AI-powered features offered through its Data Intelligence Platform will revolutionize data science, where anyone can gain access to data insights, and any business can leverage their data effectively.

“Historically, data platforms have been hard for end-users to access and for data teams to manage and govern. Data intelligence platforms are set to transform this landscape by directly tackling both these challenges – making data much easier to query, manage and govern. In addition, their deep understanding of data and its use will be a foundation for enterprise AI applications that operate on that data.”

A government-led coalition has launched the Smart Data Challenge: An open call for the sharing of new ideas that could underpin further innovations in UK businesses using smart data. 

Smart data is digital information that is organized at the collection point so that it offers actionable insights in real-time to the user. It enables consumers and small businesses to securely access and share data with third parties who can provide them with helpful services that can improve their operations.

The coalition, which includes the Department for Business and Trade, Challenge Works, the Open Data Institute (ODI), and Smart Data Foundry, hopes to attract leaders in business and academia to offer their proposals on the opportunities smart data can provide.

The initiative hopes to build on the success of open banking as a tool to drive innovation in the financial services industry. In February 2023, Open Banking Limited (OBL) reported that for the first time, seven million consumers and SMBs used open banking services such as financial management and payment tools during January.

The aim is to foster similar innovations that span a number of sectors including energy, retail, transport, and home buying. 

Chris Gorst, challenges director at Challenge Works commented: “A smart data economy could unlock transformative innovations just as open banking did for fintech. It is a new frontier and would represent a massive change to how individuals and small businesses take control of their data.”

According to the organizers, the ideas this program is hoping to attract and develop could be used to address current challenges facing society, such as the cost of living and climate crisis.

Kevin Hollinrake, minister for enterprise, markets, and small business, said: “Smart data has the potential to help create solutions to real life issues that consumers face on a daily basis as well as supporting small firms across the country and helping to grow the economy.” 

The challenge will identify leaders in the smart data space and let them share their ideas on how this technology can benefit businesses and consumers alike. These leaders will also be involved in developing a potential Smart Data Prize, which could offer a share of up to £750,000 to successful participants.

By helping individual innovators turn their ideas into viable business concepts with a view to moving them into development, the most promising ideas will be supported with both guidance and cash prizes, the organizers said. 

“We want creative people from across industry, academia and civil society to join the conversation about opening up services using smart data to make people’s lives better, and enrich our national data infrastructure” commented Louise Burke, CEO at the ODI.

The Smart Data Discovery Challenge’s initial phase calling for ideas will run from 26 October to 8 December. To get involved, innovators can propose cross sector solutions and explain how they will benefit consumers, small businesses, and wider society. A longlist will then be selected with a pitching event set to take place in March, where the best proposals will receive cash prizes.

Increasingly complex data storage requirements mean that businesses have to learn lessons from previous examples of “monolithic” data transformation tactics, according to NetApp CEO George Kurian.

Speaking to ITPro, the chief executive said previous issues with Hadoop, a distributed big data storage framework, show that organizations risk data-fueled bottlenecks which harm innovation if storage considerations aren’t flexible enough to match evolving needs.  

Hadoop was once hailed as a key tool for organizations processing large datasets ranging from gigabyte size to petabytes. Thousands of organizations used - and continue to use - the framework, including Meta. 

However, Hadoop’s limitations have often been widely bemoaned by users, many of whom went “all in” with the solution, Kurian said during his opening keynote at NetApp Insight. These limitations included the lack of real-time data processing and the ability to efficiently handle new data types, including AI and ML models. 

Kurian said that, with the emergence of generative AI, the need for efficient and high-performing data processing capabilities should be a critical priority for firms hoping to capitalize on their data and maximize value. 

"Many organizations said ‘let’s take a top down, monolithic approach to solve my data problems and put all of my data into this one solution called Hadoop,” he told ITPro at a press briefing.  

“Hadoop, meanwhile, had its own proprietary analytics framework, its own data schema, its own proprietary computing models and storage,” Kurian added.

“And suddenly the world changed and Hadoop was no longer capable of dealing with new data types. It couldn’t deal with AI or ML models and so on.” 

Kurian noted that instances of “top down”,  sprawling data transformation projects more often than not resulted in longer-term headaches for organizations, many of whom were forced to fundamentally alter their entire data architecture.

“So now you’re trying to replatform,” he said. “And trying to replatform a Hadoop landscape is brutal. You have to replatform the analytics framework, your data schema, your computing architecture, your schedulers and your storage.”

Kurian’s comments to ITPro formed part of a broader message on data transformation and leadership throughout the duration of the NetApp Insight conference.  

In his opening keynote, Kurian emphasized that data leaders have to evolve to accommodate for rapidly changing business conditions and the emergence of technologies such as generative AI. 

Kurian pointed to research from Boston Consulting Group that found “data laggards” - leaders who are slow to modernize data architectures for changing requirements - could risk falling behind their more agile, forward thinking competitors. 

Far from calling for a widespread rethinking of data architectures, both Kurian and NetApp have been keen to emphasize that businesses adapting to modern data management requirements should act “flexibly” in their transformation practices. 

“What we are saying is you want to be able to preserve flexibility in certain parts of your architecture,” he explained. “In your analytics framework, for example, you want to be able to preserve flexibility because the technology change is so fast right now.”

“You don’t want to have all of your layers of your architecture to be slammed together in one monolith. You want to have a layered model where some are stable and some are flexible.”

Copenhagen Airport is colossal. Spread across multiple terminals and passenger concourses, covering an area of 4.6 square miles, the airport employs 1,700 people and there can be up to 83 operations (take-offs and landings) every hour. Some 30 million people travel through the airport each year and this is expected to increase in the coming years. To meet the growing passenger throughput, Copenhagen Airport needed to expand.

Physical expansion isn’t always possible, as airports are constrained by the physical limitations of their site and planning regulations. But space isn’t always used as effectively as it could be. 

And if an airport were to operate more efficiently, it could increase this capacity. For Copenhagen Airport, the answer lay in unlocking the power of big data.

Joining disparate systems under the AIRHART project

Airports are information-dense environments, but data is spread across multiple systems and often in incompatible formats, making it difficult to gain a complete overview of what’s happening. The result is inefficiency and conflicting priorities being introduced.

“Imagine buying a car, and the only tires you could add to that car are from a specific manufacturer,” says Mehdi Motaghiani, CEO of Smarter Airports, a joint venture between Copenhagen Airport and Netcompany.

“We wanted to be able to add in and attach multiple systems and data to one platform that would help orchestrate those processes and create one source of truth that everyone could rely on for decision-making,” he adds.

For ten years, Copenhagen Airport examined how it could combine previously incompatible digital systems into a single platform, enabling total airport management. The solutions they were initially presented with, however, didn’t fully meet expectations, so they looked into creating their own.

The AIRHART project developed a digital platform unifying all information streams. This offered improved airport traffic management, for example enabling them to better predict passenger flows and the associated baggage handling requirements. But this was no easy task. Managing multiple data streams is challenging, especially when they’re generated by specialist software.

Why it was hard to define the project scope

Airports are constantly evolving. As such, the development team was unable to lock the project scope during its three-year development. A level of flexibility was maintained in order to be able to adapt to meet shifting demands.

“You cannot lock the scope, otherwise you would need to do all your thinking up-front; but that cannot be done, because you don’t actually know the details of what you want to achieve,” says Motaghiani. “You need to ensure that you can keep being innovative, while still having control over timelines, systems, and the stakeholder landscape.”

Airports also necessitate high-level security, which Netcompany ensured by developing AIRHART with a methodology based on security by design; they used continuous testing, authentication safeguards, and good programming practices to make AIRHART as impervious to attack as possible. AIRHART’s security was tested prior to launch by penetration testing teams.

The team then rolled out AIRHART with minimum levels of disruption thanks to the ‘shadow production’ distribution model. The existing platform and the new AIRHART were running simultaneously across the networks; by not enforcing full adoption, Copenhagen Airport enabled stakeholders to transition to the AIRHART system at their own pace. 

How the project met regulatory requirements

Airport regulators are understandably risk averse, so one AIRHART’s key challenges was meeting the strict regulatory requirements necessitated by governing bodies. A particular challenge was they were developing AIRHART according to new regulations – which hadn’t been previously mandated.

“There are a set of laws around air traffic management systems, which are new, as the systems put into production after a given date need to comply with this new set of standards and regulations. As there had not been anyone going through this process, we needed to understand that and make it tangible: what did it actually mean when it comes down to software testing and security?”

With hindsight, they'd have liked to be more involved with regulators. Greater collaboration would have let them create a platform with compliance embedded at its core. “What we tried to do was not only develop a platform but to say ‘This is what we think should be a standard’,” explains Motaghiani.

The next stage for AIRHART will be using machine learning to predict events and recommend actions. Utilizing machine learning in this way could help Copenhagen Airport further minimize disruption and improve efficiency. There are also plans in the works to offer the core functionality of AIRHART to other airports and potentially similarly complex sites, such as hospitals.

Energy management and automation specialist Schneider Electric is bolstering its support for IT channel decarbonization with the launch of its new Zeigo Activate platform.

The new subscription-based software as a service (SaaS) offering has been designed to calculate, measure, and analyze carbon emissions, set accurate decarbonization goals, and build a tailored sustainability strategy, the firm said.

As well as track energy emissions, small-and medium-sized partners can leverage Zeigo Activate to create a customized implementation roadmap to help minimize environmental impact, as well as connect to solution providers that specialize in energy efficiency and renewables.

The solution sits alongside Schneider’s other digital sustainability offerings, Zeigo Power, which helps European users to digitally tender renewable energy power purchase agreements (PPAs), and its Zeigo Network, which is designed to connect like minded members of its 600+ user base to help with their decarbonization push.

In an announcement, Schneider’s vice president of IT channels, David Terry, said organizations remain determined to tackle the sustainability issue despite ongoing global challenges.

“Channel businesses continue to face a host of energy and sustainability challenges, but despite a turbulent macroeconomic environment, ESG, energy efficiency and ecosystem decarbonization remain key areas of focus for partners,” he said.

"Simplified" data collection

Zeigo Activate has been built around the concept of simplified data collection, Schneider said, meaning partners can apply its functionality to build an energy consumption profile, as well as track baseline progress of carbon emissions.

Partners can use the software to build a customized roadmap specific to a certain target or business strategy – such as achieving 100% renewable status - and use the software’s streamlined dashboards to both track and report progress.

The platform also facilitates inter-business connections, with partners able to connect directly with an ecosystem of vendors for competencies that are required of their action plan and to view estimated costs, savings, and return on investment.

Additionally, the offering promises to help reduce Scope 3 emissions through engagement with the supply chain, providing mid-market members of the chain with the tool to assist with their own decarbonization efforts. 

Schneider says the guided and simplified nature of the platform will allow businesses to navigate the time pressures that are usually associated with complex software and limited sustainability expertise. 

“Through Zeigo, partners can access a suite of world-class SaaS tools and identify specialist businesses that help them reduce energy consumption, emissions, and costs, as well as strategic processes to accelerate business’ sustainability journeys,” Terry added.

While businesses may already be drowning in data to some extent, that doesn’t necessarily mean that the information organizations use and hold is useful. Indeed, the stuff that stacks up uncontrollably is almost by definition of low value: something rare is far more likely to be monetizable than something commonplace. 

The term datafication, coined in 2013 by Kenneth Cukier and Viktor Mayer-Schönberger, is about capturing data in a deliberate, directed way. Before then, the term was linked with analyzing aspects of our lives captured through data, but the impact of big data and the massive potential of data analytics shifted its significance. When we use datafication, we still refer to translating aspects of our lives into data points, but on a much, much larger scale. 

What's the difference between datafication vs digitization vs big data?

There’s a similarity between datafication and digitization, but datafication isn’t the same thing as, say, scanning in physical documents or media. Where datafication is relevant is more around, say, the metadata of physical materials. A recycling company, for example, doesn’t need to read every piece of paper it sends to the incinerator – unless it’s in the intelligence or cyber security business – but it might well want to know how much those documents weigh, and relate that to the heat and power produced and consumed by their destruction.

Datafication and big data are two different things entirely, meanwhile. Admittedly, Cukier and Mayer-Schönberger’s book is entitled Big Data: A Revolution that Will Transform How We Work, Live and Think – but the classic big data methodology involves working with the huge volumes of unstructured, unexamined data that are already sitting around on servers. Datafication is more about collecting specific information – perhaps just a few bytes a day – that wouldn’t otherwise be captured.

The idea is actually more similar to the Internet of Behaviours (IoB), but IoB instead involves analyzing and responding to customer behavior. This can involve a healthy element of datafication, via techniques like using sensors to record footfall. But datafication is a broader idea, with applications across a whole range of processes and environments.

What data should businesses aim to collect?

That entirely depends on the business and the industry in which they’re in. Businesses should go through a desirability exercise before jumping in and spraying the place with data collection sensors, CCTV cameras, door-actuation microswitches, and so forth.

Ideally, you’re looking for something that can inform your workflows, without alerting competitors or customers to your superior process control.

The specifics may not become clear until you’ve collected and crunched the data. Then, what do you do with it?

Since Kenneth Cukier writes for the Economist, it’s not surprising he sees creating new data as a way to create new value. You might use the data you generate to monitor and improve your processes – or you might be able to sell it on. Again, though, that sounds like an idea you’d hear from an economist, rather than a business operator.

What are the benefits of datafication?

Since the parameters of datafication are broad and fuzzy, it’s fair to be wary of jumping in. But remember, in the world of marketing, only 1% of passing punters need to react to a billboard for it to be counted as a roaring success. 

Any sensible datafication effort should provide a much more tangible benefit than that. And the very looseness of the idea means it can be applied in a huge number of scenarios. Whether you’re interested in productivity, sales data, or customer contacts, it’s a good bet that as-yet unexploited sources of data can help identify shadowy patterns and correlations.

What are the data protection implications for datafication?

Some areas, for example, have lately been filling up with smart litter bins, with massive advertising screens on their sides, and built-in Wi-Fi transmitters that provide convenient internet access for paid-up customers.

They also happen to log the MAC addresses of all passing devices, such as the phones in people’s pockets, whether subscribed or not. By tracking devices as they pass different bins around the area, the data collector can build up some quite sophisticated models of people’s movements.

This is a classic piece of datafication, but it raises questions about consent and data protection.

The data collector doesn’t know exactly who you are, but they can tell – for example – that most mornings you walk a particular route on your way to work, and they can tell if one day you go to the hospital instead. 

As always, such issues need to be carefully considered in any datafication project. In fact, all the smart bins in such neighborhoods have recently gone dark – perhaps because the information they were logging was deemed too sensitive.

It’s not a revelatory statement, but one that is true nonetheless: Technology vendors really can live or die by the health of and focus on their partner ecosystem. 

Data cloud firm Snowflake is no exception. But it’s not simply a numbers game, as Dan Waters, vice president of partners and alliances in EMEA told ChannelPro at the company’s annual conference, Snowflake Summit, in Las Vegas at the end of last month. 

“We don't have a customer success team. Our customer success needs to be delivered through our partner community. It's everyone's responsibility to make sure that our customers are as successful as they possibly can be and realize the value of Snowflake Data Cloud. Our partner ecosystem is central to delivering that,” he says. 

“[If we want to] realize our goals as a company, we can only do that with a strong and diverse partner ecosystem.”

Recruiting at the top

That win/win sentiment is definitely echoed company-wide and Snowflake is putting its money - and its talent - where its mouth is right here and right now. Indeed, earlier this month, the firm appointed Tyler Prince - who’d spent the previous 10 years at cloud giant Salesforce and many more before that in key industry roles - as its senior vice president of worldwide alliances and channels. 

“Snowflake’s vast ecosystem of partners are absolutely essential to our mission of mobilizing the world’s data,” Chris Degnan, Snowflake’s chief revenue officer said in a press release announcing Prince’s appointment in mid-June. 

“I’m thrilled to welcome Tyler to Snowflake. His deep industry relationships and extensive experience leading exponential growth across the channel will accelerate our partner enablement and go-to-market strategy globally.”

Prince joined the firm on 5 July and appeared to be laser-focused on the partner opportunity from the get-go. 

“Customers are at the heart of everything Snowflake does, and the Snowflake Partner Network is a world-class example of how delivering choice in the tools and solutions available to customers drives our customers’ success,” Prince said at the time of his appointment becoming public. 

“I look forward to continuing to grow the Snowflake ecosystem, engaging more deeply and broadly with partners to accelerate their own businesses.”

Prince is in good company in terms of experience, vision and execution, certainly as far as EMEA is concerned. Waters is a 24-year technology industry veteran, with experience at Arrow ECS and Bell Microproducts under his belt prior to joining Snowflake. His primary focus right now is to ensure words turn into action, specifically in EMEA. 

“My role is to make sure that we've got the best and most capable and competent partner community that we can possibly have to really serve the needs of our customers, in each of the markets,” he adds. 

“That isn't a one size fits all [approach]. The UK market is very different to our market in Israel, which is very different to the market in the Nordics, which is very different to the German market. So it is very much [a question of] what's needed locally, and making sure that we've got the right partners, doing the right things, and the right connections into our sales teams to give customers the best possible experience working with Snowflake.”

‘Partners’ is an overarching umbrella label with many different types of organizations sitting underneath, myriad of sizes, and focused on different specialisms and sectors. Ultimately, Snowflake has many different types of partners flying the firm’s flag, from small to large, from services and technology and everything in between. 

What’s more, while Snowflake is a global business with a global partner ecosystem to match that status, many markets around the world have more differences than they do similarities. And that calls for a unique, market-by-market approach to ensure success for all concerned. 

“It takes local to local intentional engagement and planning and execution, to actually move the needle in any given location, especially in EMEA,” Waters adds. “I think that's what's really unique about our market. It's about local-to-local engagement, local customer, local commitment, and local capability. We call it the three Cs.”

Building a global community

Snowflake was founded 11 years ago in July 2012 and launched publicly just over two years later in October 2014. Three data warehousing experts were behind the firm’s inception: Thierry Cruanes, Benoît Dageville, and Marcin Żukowski The former made up of two ex-Oracle alumni and Żukowski’s background being that of co-founder of Vectorwise, a Dutch start-up. 

What has happened during that time has been an exciting ride for industry watchers as well as Snowflake’s assembled cast of internal and external players. Growth has been phenomenal and success sustained. Indeed, as of May 2023, the firm’s platform boasted a 3.1 billion average daily query volume, which is up 59% year-on-year and 7x the average number of daily queries if we compare April 2020 (the company’s FY21) to May 2023 (the company’s FY24). 

Furthermore, right now Snowflake has 40 regional deployments and 599 customers featured in the Forbes Global 2,000 list, the company cited during its Investor’s Day in June 2023. 

In the firm’s FY19, it generated $97m in product revenue. In FY23, that figure had grown to $1.9bn. For FY29, Snowflake has set its ambitions high with a target of $10bn in revenue during that financial year. 

Partners - in their many forms - and customer attraction, as well as retention, will clearly be key to achieving such lofty ambitions. 

“We’re trying to bring the community together a bit more. Both visibly and effectively. We've got a phenomenal global community, but I think the EMEA market is definitely different and complex. It's not-a-one-size fits all [thing] - every market is different,” Waters emphasizes. 

Snowflake and NTT Data: A flourishing partnership

One such partner is NTT Data, which has a longstanding relationship with Snowflake that goes above and beyond the sort of standard channel unions that have become typical of late. 

Snowflake has recently put in new partner management in the UK for NTT to help strengthen and further support the relationship, Waters explains. 

“That really makes a difference,” he says. “We've been very focused on making sure we have the right talent to work with the right partners.”

NTT Data is what Snowflake describes as a ‘Services’ partner and it’s certainly a relationship where both benefit from collective as well as individual experience. 

“It’s the customer conversation again. It's about [looking at] real-life pain points for your customers as it pertains to service industries. That then gives the partners an opportunity to really differentiate,” Waters says. 

“We verticalized our sales teams in EMEA in the core markets in February of last year. I think that's really helped us get closer to our partners and mine their specialization and capabilities as it pertains to their vertical skills. It’s a fairly new approach for Snowflake having verticalized relatively recently, but partners like NTT have been verticalized for years.” 

Gareth Lyon, head of data and intelligence alliances at NTT Data UK, says the firm does around $100m of business with Snowflake each year.

“There was recognition that if we didn't get on the Snowflake train, then we’d get left behind. We very much see Snowflake as being one of our key software partners that will allow us to accelerate our clients’ route to value,” Lyon says.  

“We primarily look at Snowflake - it's very much a way for us to generate three to four to five to 10 times the professional services revenue compared to the Snowflake consumption that is sold,” he adds. 

“On engagements, we've been brought in after the sale. So my remit and objective for this year is for us to not only co-create but to also co-sell. And to do that in the spirit of truth, a true partnership, where we bring opportunities to each other.” 

All has definitely been well with the NTT/Snowflake partnership thus far. But, as Lyon explains, the firm is keen to expand its horizons. 

“We've seen the results of it so far. The pipeline has tripled in size. It's now about making sure that we close some of those deals, and then continue to deliver volume,” he says. 

“I get approached probably on a weekly basis by another partner. I see it very much like [as a case of] ‘Where can I drive the best value for the client?’ And Snowflake is [the answer]. We've seen it with the engagements where we've delivered in the past.” 

The value of two-way relationships

Snowflake also has strong ties with the ‘big three’ in AWS, Google and Microsoft and those relationships are likely to get stronger rather than weaker over time, as Waters alludes to. 

“We don't have an on-prem product. So our partner relationships with Microsoft, AWS and Google are absolutely critical in terms of how we go to market,” he says. 

“That actually connects really well into our services partners as well, where we have what we call a power of three play.”

Collaboration is definitely the name of the game as far as Snowflake and its growing number of established and new partners are concerned. And both Lyons and Waters concur that communication is critical.

“We had an open and candid conversation earlier on talking about how the partnership is going and how we can improve it. A partnership for me is about walking into the room, shoulder to shoulder. If you do that, then you are being successful,” Lyons adds. 

Waters adds: “In any relationship, there are always ups and downs and peaks and troughs. You've got to hold each other accountable, to be the best that you can be. A lot of the conversations I'm having with partners over this week [at Snowflake Summit] is [asking them to] tell us how we partner. That's not just on us and it's not just on the partner. It's a two-way thing.” 

The corporate networks of today are not your grandmother’s corporate network, to coin a phrase. Where once everything was limited to a single primary site and perhaps a couple of branch offices, business footprints have steadily grown to incorporate numerous non-traditional environments. This could be something as mundane as a user’s home or a coworking space, to more exotic locations like oil rigs or research vessels.

This is what is meant by computing at the edge, and this is often where vast amounts of useful – even critical – data is generated.

Edge computing is intimately entwined with the Internet of Things, notably devices like sensors, robots, and computer vision systems. These can generate a constant stream of data that in a traditional network environment is transferred to the public cloud or a private data center for storage and processing.

While this doesn’t pose a technical problem, it can decrease productivity due to the time taken to transfer data from one location to another for analysis, and potentially back the other way for action. In a busy environment such as a manufacturing floor or transport hub, milliseconds of processing time can mean the difference between a slow operation and one defined by efficiency.

By 2025, 80% of data is expected to be processed at source, according to the European Commission, and the majority of this will come through edge computing. However, businesses need to implement networks that can support this if they want to make full use of the data they generate.

Edge computing in practice

Often, edge computing is talked about from a very high level, abstract point of view, so it’s worth considering some concrete examples of where the technology can make a real difference.

Consider a standard warehouse environment, where the stock it contains must be kept within a certain temperature range. Smart thermostats could monitor the temperature, analyze if it’s within the defined parameters and, if needed, engage the heating or cooling system to warm the air up or cool it down. If there’s a regular problem, such as the temperature always going out of range at 2pm on a Wednesday, an alert could be automatically generated for an engineer to come and investigate the cause.

Another widely used example is monitoring machinery on a production line, wind turbine, oil rig, or another situation where failure or degradation could be disruptive or even dangerous. Analyzing data from sensors and other monitoring equipment in real time means alerts can be raised as soon as parts start to weaken and before a complete failure occurs. This in turn minimizes risk to both the facilities and the people working there, as well as reducing the cost associated with unscheduled downtime.

While these possibilities are exciting, an intelligent edge is only possible when the right infrastructure is in place.

The right infrastructure in the right place

The basics of creating an intelligent edge network are largely predictable. Wireless access points, which can be installed both indoors and outdoors, are needed to provide wireless capacity across a site with low-latency connections.

A software-defined wide area network (SD-WAN) can be an excellent solution for edge connectivity. This is a virtual service for connecting disparate devices and locations using network technologies such as broadband, 5G, and LTE, and is more suitable for cloud-based applications.

SD-WANs can be especially useful for edge networks as they reduce latency compared to legacy WANs, and allow IT teams to remotely manage routing protocols.

As well as SD-WANs, a network of capable wireless switches will also be necessary for the infrastructure to function.

For real time analytics, though, specialist infrastructure is required. Hewlett Packard Enterprise (HPE) has an array of products and services that can help businesses achieve the advantage they need through edge computing.

HPE Edgeline converged infrastructure, for example, are optimized for use in edge settings, while Aruba Edge Services Platform (ESP) can help with automation and analytics.

To make the most of their intelligent edge, organizations should also consider using HPE GreenLake, the company’s hybrid cloud platform.

An edge-to-cloud approach

HPE GreenLake allows you to take advantage of public cloud features like scalability, automation, analytics and artificial intelligence (AI) while still maintaining the benefits of private cloud such as data sovereignty and security.

Data protection is an important part of any edge strategy that should not be overlooked. The data generated and analyzed at the edge should be treated as sensitively as intellectual property (IP) or personally identifying information (PII) such as payroll details, as it could both give away details of how a business operates and open it up to disruption from malicious actors.

HPE GreenLake allows organizations to keep control of their sensitive data and offers protection such as identity and access management, as well as  providing controls for data location and movement. Through HPE GreenLake Platform, IT professionals can manage all elements of their edge network and hybrid cloud in one place. This includes networking services from Aruba, an HPE company, which can be managed through the Aruba Central console, as well as data analytics applications like Splunk that can be provisioned through this single pane of glass.

Additionally, when the data that’s being created at the edge no longer needs to be analyzed immediately, it can be moved to another part of the organization’s hybrid cloud estate for warm or cold storage. Thanks to HPE GreenLake’s elasticity and quick provisioning, it’s easy to increase storage capacity in the business’ private cloud as needed without having to worry about overprovisioning or running out of space.

Edge computing offers up a new world of insights and possibilities. With HPE GreenLake, businesses can make the most of their data in a way that suits them with a true edge-to-cloud hybrid platform.

For more information on edge computing, visit  Connect your edge with HPE GreenLake

Data is the lifeblood of modern business, regardless of sector or size, and organizations have been tapping data to gain valuable insights for years. It’s much easier than ever before, thanks to modern platforms, architectures, and cloud-powered services. Organizations have, as a result, gained a better grasp of their customers, addressed supply chain inefficiencies, and made smarter decisions – especially with real-time insights. 

But as data analytics has increased in popularity, new challenges have begun to emerge. Businesses may, for example, be wrestling with a mixture of legacy and modern infrastructure, or even be struggling to contend with data silos dotted throughout their IT estate. Maximizing the potential of data is difficult in itself, let alone while such issues are prevalent. These might also include ballooning datasets, distributed data, and information tangled up in a variety of systems and applications. 

The main problem isn’t how to collect or process data, but unlocking genuine value from it to reach the best decisions in the quickest time. Thanks to technologies like precision analytics that streamline the workflow, businesses can begin to navigate these challenges and kickstart a data-first modernization journey.

How data management has become more complex

The volume of information businesses are harvesting, with a view to extracting value, is rising exponentially. The total amount of data generated was 64.2 zettabytes (ZBs) in 2020, with this figure expected to rise to 180ZBs by 2025, according to Statista.

But collecting or processing more and more data doesn’t automatically translate to deriving higher quality insights, and not all businesses are equipped with the tools to make the most of their data collection. The picture has certainly become more complex ever since organizations first started data-collection and data analytics schemes internally. 

The average organization uses 23 different tools to manage the data environment, according to Enterprise Strategy Group (ESG) research, with 68% of data leaders saying the complexity of the IT environment has slowed down their operations and digital priorities. 

Despite data-first leaders demonstrating their organizations were generally more productive than nascent businesses, they were also twice as likely to agree their data management capabilities were falling short. A large majority of respondents (74%) also acknowledged their data management capabilities couldn’t keep pace with business needs.

The business benefits of smarter data

Data-first leaders, who are charged with reducing the complexities of data management in their business, drive a number of key benefits. These individuals, who normally have cloud operational experience, improve IT and business results, according to the aforementioned ESG research, and improve technology outcomes generally.

Data-first leaders are, for example, 20 times more likely to beat competitors to market by several quarters, while being twice less likely to be in line with or behind the competition. These leaders are also more than 11 times more likely to beat revenue goals by more than 10%. Moving faster than the competition can naturally lead to more revenue opportunities, with data-first leaders at the heart of this drive.

Following a ransomware attack, meanwhile, data-first leaders are roughly five times likelier to recover within seconds, and twice as likely to recover within minutes. This is because less business data is at risk, and they can use more optimized recovery services to get back on track following a cyber security incident. 

There are several differences in this approach, including delivering data infrastructure as a service, as well as simplifying the configuration, ordering, and deployment of data infrastructure. There’s also scope to use artificial intelligence (AI) tools more effectively to create a cloud operational experience. Under this model, however, data leaders must also find ways to strip out the potential for hidden bias, to ensure the insights derived through automation and other tools are genuinely valuable, and not skewed. 

Indeed, from bias in word associations to bias in hiring practices, there are a number of alarming ways in which these deficiencies manifest when AI is deployed. One of the main causes is flawed data, according to Brookings. There are technologies out there like HPE Swarm Learning, however, that counter the ways bias manifests in datasets across an organization. This framework is a decentralized machine learning system that allows collaborators to share data insights, without revealing the data itself. By allowing organizations to share insights beyond their own walls, they can improve the quality of the data and the application of AI by sense-checking with the wider field. 

Embracing data-first modernization

The business-as-usual approach isn’t working, and organizations must streamline the process by which they gain insights and act on them. The cloud is instrumental to this, and services like HPE GreenLake are essential to unifying the data regardless of where it resides. This means that valuable enterprise data your organizations has been collecting can be put at the heart of the modernization process. 

A unified approach to managing enterprise data also means the proportion of data never analyzed falls. There are countless hidden insights within corporate datasets, and compiling these sources into a single unified platform gives businesses a much higher chance of gleaning valuable insights in real-time, which is then fed into smarter decision making.

HPE GreenLake, which manages more than one exabyte of customer data, can unify data at the edge by securely integrating all your apps and infrastructure without friction. It means you can go from the edge-to-cloud with the benefit of scalability; you can access the data wherever it lives, eliminating migration headaches in the process. This edge-to-cloud architecture, combined with an AI that is embedded with tools to strip out bias, offers real-time intelligence. The intelligent edge environment you tap into would certainly give data-first leaders that competitive edge when optimizing the data across the IT estate, and attempting to feed this into their data-first modernization journeys.

To learn more about data insights, visit Turn your data into intelligence with HPE GreenLake.

Databricks has launched a raft of new features for its flagship Lakehouse data lake platform, including artificial intelligence (AI) and natural language improvements as well as data governance features.

With Lakehouse AI, customers can develop generative AI applications, including large language models (LLMs), within the platform itself, with tools spanning the breadth of the AI lifecycle. This includes data collection, model development, and monitoring.

“This gives you control over your intellectual property,” said Databricks CEO Ali Ghodsi during his keynote address at Databricks Data + AI Summit 2023. He added every company in the next five years “will be an AI and data company” and would need the tools to create their own LLMs to be competitive, rather than rely on third-party services.

What is LakehouseIQ?

LakehouseIQ is the firm’s ‘knowledge engine’ that’s designed to make natural language interfaces, like chat bots, more accurate and relevant to the needs of a particular organization.

Sitting as a fabric between data and a front-end interface, the tool uses generative AI to understand a business’ unique culture, jargon, and data usage patterns, among other factors.

It ingests a vast quantity of internal material, including organizational charts, notebooks, and more, in order to understand how best to answer questions users ask via any chat interface.

It’s designed to be specific to each business, so natural language interactions are far more accurate and context-sensitive than classic chat bots. While not a chat bot or virtual assistant itself, it’s designed to improve the reliability and performance of these tools.

Databricks has made Lakehouse IQ available as an API, and the firm plans to integrate this into open source libraries.

New AI features in Lakehouse

Vector Search gives developers the capacity to improve accuracy of generative AI responses. The features creates vector embeddings based on files found in Unity Catalog — the central hub that gives customers access control, auditing, lineage and data discovery capabilities across their workspaces.

It creates vector embeddings from Unity Catalog files, and automatically updates them through integrations with Databricks Model Serving. Developers can also add query filters to provide better outcomes.

Improvements to AutoML also adds low-code functionality to finetuning LLMs. Any tweaks customers make to their LLMs will be owned by the enterprise, with data kept in-house and not shared with any third parties. This model can also be sahred within an organization with Model Serving, Unity Catalog and MLflow integrations.

Finally, Databricks has launched several open source models through its marketplace including MosaicML’s MPT-7B and Hugging Face’s Falcon-7B, as well as Stable Diffusion.

On the LLMOps front, MLflow AI Gateway lets customers centrally manage credentials for software as a service (SaaS) models or APIs, and provide access-controlled querying. This also allows for prediction caching to track repeat prompts and rate limiting to reduce costs.

MLflow Prompt Tools, meanwhile, is a no-code feature that allows users to compare different outputs from various models based on prompts that are automatically tracked within MLflow.

New governance features in Lakehouse

Also new in Lakehouse is LakeHouse Federation capabilities, which has broken down silos between data systems that previously existed on the platform. As a result, businesses are now able to discover, query and govern data across all their systems without the need to copy or move data. 

Query federation comprises catalog and querying functionality to consolidate and map out data asses from different platforms beyond the confines of Databricks, including MySQL, PostgreSQL, Redshift, Snowflake, Azure AQL Database, Azure Synapse, BigQuery, and others. Users can secure, audit and access data from one interface rather than having to move the data between different platforms.

Additional governance tools in Unity Catalog also lets customers apply access policies on any data asset that’s registered, including tables, rows, columns and tags. This comes ahead of future plans to define data access policies and push them out to other data warehouses.

The Lakehouse Federation's capabilities also include giving users a common approach to discovering and exploring both structured and unstructured data with a single query. Domain-owned data sources for analytics and AI use cases are also rapidly exposed, offering faster access to data. Finally, the changes make it possible for users to apply a single permission model for the data estate.