Moving generative AI from proof of concept to production: a strategic guide for public sector success

It’s undeniable in 2025 that generative AI can streamline knowledge work and help improve productivity in the workplace by reducing manual processes like form filling. This technology has public sector appeal as it promises to save money and improve time to resolution – freeing up time for experienced civil servants to support citizens.

There’s a huge gap between exciting plans for generative AI adoption and actually putting it into production, however. In a recent report, the UK Department for Science, Innovation and Technology estimated that generative AI tools can save up to 40% of time in the public sector but noted strategic hurdles to unlocking this productivity.

A recent MIT study suggests as many as 95% of AI pilots fail, as reported by Fortune, with researchers pinning the blame on poor enterprise adoption strategies.

Elsewhere, Gartner predicts that over 40% of agentic AI endeavors will fall by the wayside by the end of 2027, citing a widespread lack of strategic thinking before deploying AI and pursuing costly proofs of concept.

Plugging the gap between appeal and adoption will come down to setting out AI plans in collaboration with trusted partners, alongside more long-term thinking by public sector leaders to ground AI initiatives in organizational goals.

How can organizations implement generative AI?

First and foremost, all public sector organizations looking to bring generative AI into production need firm, clear plans for adopting it in the manner best-suited to their core mission. This means leaders must align any potential AI programs with the objectives of their respective organizations, both in the short and long term.

The starting point for adoption must be strategic alignment between one’s organizational objectives and any AI initiative. Instead of being driven purely by technological outcomes, leaders should look to solve specific problems using AI, with these goals as a guiding light throughout the adoption process.

In order to succeed at this task, public sector organizations and their digital leaders must also promote a culture of innovation among their workforces, with transparency around how new AI systems help achieve long-term goals.

For example, the UK’s Financial Conduct Authority (FCA) identified significant potential for AI to help greatly reduce the time its analysts have to spend on manually reviewing tens of thousands of regulatory submissions received every year. In collaboration with AWS, it deployed a generative AI prototype that uses large language models (LLMs) grounded in retrieval augmented generation (RAG). This is a methodology for ensuring LLMs refer to a trustworthy, domain-specific source of knowledge to produce more accurate outputs. The pilot demonstrated that generative AI can support faster and more targeted document review by surfacing critical issues and freeing up time for human experts to focus on deeper analysis and judgement.

Inward-facing recognition of where skills and technical capabilities may be lacking is also key. This is where the AWS’ One Government Value Agreement (OGVA) 2.0, a specialized program created through a memorandum of understanding (MOU), with Crown Commercial Service (CCS), comes in.

Through OGVA 2.0 Scaler program, UK public sector organizations can access up to $25,000 in AWS credits to kickstart transformation efforts and proofs of concept (PoCs) in a low cost, low risk way. Upskilling is also a pillar of the program, with five complimentary AWS Skill Builder licenses provided per organization, to those that qualify.

All of this helps lay the groundwork for moving generative AI projects into production, ensuring that an organization’s people and technologies can work with AI tools in line with long-term objectives.

ZNotes is an educational platform, primarily used by high school students from emerging countries. To scale its operations and meet the needs of users around the world, the organization got involved in the AWS EdStart program, which put it in touch with EdTech firms around the world. It also pinpointed the specific technologies and services within AWS, including Amazon Bedrock,that help Znotes enhance user experience, while maintaining high availability during peak exam periods and ensuring data security for underage users. Through this process, it has evolved from a WordPress blog to a sophisticated platform that now serves six million students across 190 countries.

In the immediate future, ZNotes will localize its content using generative AI, leaning on its decade of data and educational methodologies to meet the needs of national curriculums faster.

The second most important factor for successfully translating AI proofs of concept to AI successes is possessing the right AI infrastructure.

As many as 70% of respondents to the DSIT review said their data infrastructure is poorly coordinated and not interoperable, which makes adopting centralized AI systems a more time consuming prospect.

As organizations digitize services and coordinate data, they must ensure they build their new data foundations on efficient, reliable infrastructure such as AWS’ open data lakehouse and zero extract, transform, and load (ETL) pipelines. These help public sector organizations to quickly connect existing data to AI systems.

Scalable compute is also of paramount importance for enterprise-grade AI, particularly for meeting variable customer demand.

For example, Hillingdon London Borough Council worked with AWS and PwC to deploy an AI tool for call and web customer service to improve resolution rates with local residents. The result provided the call fielding equivalent of 26 full-time staff and reduced cost-per-call by 95%.

This system is available to residents 24/7 and stands ready to deal with a large number of concurrent calls during peak hours, underlining the benefits of an elastic compute approach to AI deployment.

Governance is another major consideration for deploying AI in the public sector and beyond. Leaders must assess potential damages to stakeholders that can arise from AI deployment, be they commercial customers or, in the case of the public sector, citizens relying on them for trustworthy and effective services.

Of course, cybersecurity is an essential pillar in any governance framework and generative AI in production should always be paired with strong data protection built in at the ground level.

For example, cybersecurity leaders may work together with figures overseeing a public sector AI project to ensure that new tools abide by strict access controls, as set out by environment-wide services such as AWS’ AI security services

Embedding governance and human oversight at every stage requires stakeholders to take a hands-on approach that considers the right protocols for AI behavior from every angle and examines how it will fit into a given enterprise in detail.

The next evolution in AI implementation involves autonomous agents capable of reasoning, planning, and executing complex tasks. These systems will move beyond conversational interfaces to become active partners in organizational operations. PwC tracked AI Agents as being adopted in 79% of US companies in a recent survey. Amazon Bedrock AgentCore helps you transition your agentic applications from experimental proof of concept to production-ready systems in easy steps.

The journey from proof of concept to production requires a balanced approach combining technical excellence with organizational readiness. As we move toward autonomous AI agents, this journey becomes even more critical. To make the most of generative AI and emerging technologies such as sophisticated AI agents, public sector organizations need to follow their goals and principles as a north star, in collaboration with trusted service providers like AWS.