Otter.ai wants to bring agents to all third party systems – with transcription just the start

Otter.ai has unveiled new agentic AI features to connect its app to more third-party applications, alongside autonomous features to streamline the project management workflow.

A new public application programming interface (API) for Otter will now allow developers to connect data from Otter with their existing enterprise systems such as a customer relationship management (CRM) offering.

Sam Liang, co-founder and CEO of Otter.ai, told ITPro that the firm has built integrations with systems such as Salesforce and HubSpot, as well as project management systems such as Asana and Atlassian.

“The data can flow both ways,” Liang explained.

“The data can automatically be put into Otter so that the Otter Agents can make use of the CRM data, or project management data in meetings, or the data can flow out of Otter to whatever system enterprise customers use.”

For example, Otter’s new agents can be configured to automatically populate an Asana project with action items based on a discussion between coworkers held on Zoom.

In addition to the new API support, Otter has also created a dedicated model context protocol (MCP) server for Otter, allowing its agents to connect with AI platforms such as Claude and ChatGPT.

Admins will be able to configure what data is passed to their given tool, with granular controls over the data they expose to third party tools and protections against the contextual enterprise data Otter captures from being shared with unwanted parties.

Otter’s AI agent is capable of automatically taking screenshots of slides during a virtual presentation, as well as pulling data from the slides so users can easily search for terms in the transcript even if they weren’t spoken aloud.

Another of the new features is Advanced agentic AI chat which can be used to analyze hundreds of individual audio recordings, as well as to allow users to ‘chat’ with a transcript in real-time, for example to catch up on the content of a meeting after joining ten minutes late.

It can also autonomously create documents and schedule meetings based on action points and follow-ups discussed within a call.

Liang told ITPro that data security was a chief benefit of using the Otter.ai desktop app, with enterprise customers given a choice over how much of their enterprise conversation data is shared with third parties.

As of July, Otter.ai has become Health Insurance Portability and Accountability Act (HIPAA) compliant and Liang stressed that admins would be given advanced controls over which meetings Otter will and won’t have access to, as well as how data will be stored and shared.

Otter.ai’s USP

Otter’s transcription software is already compatible with virtual collaboration software such as Microsoft Teams, Google Meet, and Zoom and can be run as a standalone desktop app for macOS (the Windows release is forthcoming).

Liang told ITPro that while most people think of Otter.ai as a meeting transcription company, it sets itself apart from competitors with its ability to connect meeting transcripts according to a customer’s unique environment and context.

“Most meeting note takers are working on a single meeting note basis,” he explained.

“They can transcribe and summarize but they don't have a system to organize, share meeting notes with the the whole team or the whole company, and make everything searchable, make everything chatable or organize the meetings in a structure like, for example, the Slack channel structure, so that the whole team can get maximum value out of the meetings.”

With the new AI features, Otter.ai aims to broaden this context database to streamline business processes. This will allow workers to quickly access information like previous interactions with specific clients and to ask company-specific questions about prior conversations or targets.

The firm is also encouraging use of its desktop app, which can capture audio from meetings without an Otter bot needing to join and will soon be released for Windows.

“[With] the high level strategy, again, this is where we are different from Zoom, we're different from Microsoft Copilot.

“We allow you to create this corporate meeting knowledge base and then enable workflows based on who you are. It could be a salesperson, could be a recruiter, could be a project manager, then Otter will create all these agents using the corporate knowledge base to optimize your workflow.”

Otter uses a hybrid approach to AI, with some reliance on OpenAI and Anthropic models for its features as well as Otter.ai’s proprietary models. But Liang stressed that for text to speech, speech recognition, and speaker ID, it only uses its own models, which it finds are more accurate across different accents.

He added that it has had to fine tune its proprietary models because transcripts have very different characteristics from written documents, on which most large language models (LLMs) have been trained.

“Transcripts based on meetings number one, have multiple speakers. This is very different from a written document. Multiple speakers, they take turns and everyone speaks maybe a few sentences or a few minutes.”

At present, users can enter ‘customer vocabulary’ into the tool, so it can accurately transcribe industry-specific terms or acronyms. Going forward, Otter is developing its models to complete this process automatically.

The firm estimates that its average customer can save the equivalent of one full-time employee’s workload for every 20 users, a 10:1 return on investment.

With over a billion meetings now having been processed with Otter, the company is working to make its tool even easier to connect to the most popular enterprise software.