Why enterprise search is not internet search

Google's Search Appliance can crawl SharePoint, Documentum, FileNet, OpenText and LiveLink content management systems as well as Lotus Notes, Domino, databases like Oracle and SQL Server and a wide range of document formats. But for other systems, unless there's a third-party connector, it relies on you creating an XML feed.

Even though Google has added Kerberos support for authentication in the most recent Search Appliance, using feeds can make it hard to limit who sees what. Enterprise search needs to respect the roles and access permissions that all those systems use and present what's relevant for the job. "What works for one person in HR in London may not work for someone in R&D in Switzerland," says Recommind's Carpenter.

"When someone in the HR department searches on one of their employees who happens to be a researcher, it's likely they're wanting to know when they were hired, where they come from and whether there's an agreement on file for them. If that same query is done by someone in the research group, not only will they typically not care about that information, they should not have access to it either; it's sensitive information that's sitting on a UK server and subject to data protection regulations."

Search in context

Enterprise search tools can also use information about the user running the query to improve results. Recommind looks at where the user is based, what it knows about them from group membership, job title and even their CV, where they're logged in from when they run the search and whether they have frequently accessed documents by one particular author, to prioritise results which are also organised by author, expertise and relationship to make it easier to drill down into results in a logical fashion.

Similarly, Trampoline's Sonar system builds a network of users who will be interested in particular topics. Google's Search Appliance integrates with LDAP and Active Directory but administrators will need to create policy groups that specify that the sales department will be more interested in pricing than engineering specifications.

Google's main appeal may be to users who feel comfortable with a search page that looks familiar. If you're looking for a simple search experience on a budget, IBM's free OmniFind Yahoo Edition packages up the open source Apache Lucene text search engine with the Yahoo Search interface - but for secure access to enterprise content repositories IBM suggests upgrading to the pay-for Enterprise Edition. That's a platform' for building semantic search on top of, so you can expect to need consulting to get features comparable to other enterprise search tools.

Even with fully featured tools, enterprise search is often just the first step to getting more value out of data.

"Companies get it in their head that they need search," says Recommind's Carpenter. "The CEO reads about it and says we need this - go and get it'. But they also need to understand their data; they need to clean it up and get it into a format that allows them to do what they need to do. Initially they want a search system to sit on top of their data but they end up using it to figure out what data they have and to get it into shape. They're focussing search more specifically, to solve a specific business problem. "