Data silos are still blocking the path to digital transformation success

Data silos remain a key hurdle to digital transformation success, new research shows, with IT leaders bemoaning outdated infrastructure and disparate data sets as a lingering challenge.

Software company MuleSoft’s 9th annual Connectivity Benchmark Report polled 1,050 IT decision makers around the world on their investments in IT and the struggles they contend with to achieve digital transformation goals.

Virtually every IT leader (98%) reported they were facing challenges regarding their digital transformation efforts.

Of these challenges, 81% of IT decision makers said the biggest barrier blocking their organization from meeting digital transformation targets was data siloing. 

Jamil Ahmed, distinguished engineer at middleware company Solace, told ITPro data silos are holding companies back from getting a clear picture of their data consumption, representing an antiquated organizational system.

“The challenge companies face in accurately measuring data consumption lies in overcoming the mass amounts of siloed data within their system,” Ahmed said.

“These data silos not only worsen productivity but also bury these insights, compiled into a mountain of data that is difficult to identify and analyze. Data silos are a result of organizational infrastructure built for a bygone era, one with limited technological adoption, and limited pathways for dataflows. Over time these have created complex organizational barriers.”

Mikael Kopteff, CTO at software development consultancy Reaktor, echoed Ahmed’s comments, noting that disparate data sets and the silo effect has been a long-running challenge for IT leaders to overcome. 

“From employees’ unwillingness to learn, to siloing potentially usable data into places they cannot access, IT leaders have faced an enormous number of blocks when it comes to digital transformation.”

Digital transformation relies on the “unification of data”

Leaders believe businesses need to implement a smarter approach to unifying unstructured and structured business data, the research revealed. 

Kopteff emphasized the important role data has in any businesses digital transformation.

“Almost all digital transformation hinges on data, it is one thing not having the data to begin with, but it is another to not be able to utilize a wealth of data to inform future decisions. Businesses can achieve significant improvement by carefully analyzing past mistakes and learning from them.”

Just under two thirds of respondents said their organization isn’t yet equipped to harmonize its data systems in order to leverage AI tools, impeding the overarching digital transformation.

Organizations need to have a robust data integration strategy in order to deploy technologies such as artificial intelligence (AI) and edge computing, which will be core components of any digital transformation.

This is significant because it means businesses are unable to harness the productivity benefits AI can unlock. For example, 85% of IT leaders said they expect AI to increase developer productivity at their organizations over the next three years. 

Speaking to ITPro, James Fisher, chief strategy officer at business analytics specialist Qlik, claimed optimizing their data infrastructure should be a priority for businesses looking to benefit from new technologies such as generative AI.

“Businesses now need to focus on optimizing their data fabric in order to embrace AI fully, and break down silos to ensure data from different sources can be combined and made available for analysis” Fisher said.

“Supporting the variety and veracity of data needed for gen AI will put new burdens on current architectures as organizations seek to make efficient and better decisions and fuel reliable and responsible AI use. In short, we need to get data out of silos and go from big data, to better data to support AI”. 

Siloing also creates cultural barriers to digital transformation

Julie Smith, director of data analytics at software company Alation, suggested data siloing can also have a detrimental effect on employee attitudes towards data, ultimately creating further setbacks for a company’s digital transformation ambitions.

“Organizational structure and its underlying company culture play significant roles in the formation of data silos. When individual departments cultivate distinct data cultures and perspectives on data sharing, it can foster an environment of mistrust and reluctance to collaborate across departmental lines, exacerbating the issue of data silos,” she said.

“When teams lack confidence in consistently and properly managing their data, it becomes challenging to encourage effective data sharing throughout the organization. This ‘silo mentality’ not only leads to the creation of information silos but also significantly affects both data culture and performance.

“This environment hinders collaborative efforts and can negatively impact the overall efficacy and productivity of the organization.”

Smith said IT leaders should make a combined effort to address both data siloing and data literacy at their organization.

“The data leaders who successfully nurture a data-centric mindset and remove data silos will not only drive innovation and agility but also gain a competitive edge in their markets and, in turn, take their business on a successful digital transformation journey.”