Is Keyword Search Dead?

Search is Dead says Stephen Arnold at the Enterprise Search Center (Jan 7, 2009)

Among other reasons, there are these startling two:

Keyword search is often a turnoff. Users say that hitting on the right combination of keywords to get the information is too difficult.

As many as two-thirds of enterprise search system users report that other means must be used to find needed information

Silver bullets promised by enterprise search vendors have been taxonomies, natural language processing, "intelligent systems that discover the meaning of documents using latent semantic indexing", and various hybrids.

Studies have shown that people expect the following:

Offer a web page that gives users specific suggestions and options with hotlinks to topics, categories, and key subjects.

Include a search box, but provide the user with point­ and-click options and a way to get started on the quest for the needed information without requiring the user to frame a keyword query.

Allow the user to drill down or jump across topics. The technology should make it easy to explore the available information and—equally important—to backtrack or find a previously displayed piece of information.

Arnold lists four companies he has tested that "point to the future".

Autonomy Update

It Ain’t Over By Jason Stamper, Computer Business Review (Dec 2008)

Autonomy CEO Mike Lynch talked to Jason Stamper about his company and the challenges in enterprise search. There is a good review of the competition in enterprise search and mergers that have taken place in the last few years such as Microsoft's purchasing FAST. More directly, Mike Lynch outlines the direction Autonomy is taking, and quotes IDC's Sue Feldman about the company and industry.

"IDC’s Feldman though says that, “At this point, it is clear that Autonomy should no longer be considered purely a search vendor. It builds search-based applications to answer market demands for better information-centric software.”"

"What does that mean? Autonomy’s website explains: “Autonomy's software powers the full spectrum of mission-critical enterprise applications including pan-enterprise search, proactive information risk management, information governance, e-discovery, consolidated archiving, call centre solutions, rich media management, security applications, customer relationship management (CRM), knowledge management (KM) and BPM [business process management].”"

There is also a sidebar on the rise of e-discovery.

Importance of Information Architecture

Enterprise Information Architecture: A View From The Legal World by Kate Simpson, FUMSI (Nov 2008)

Information Architecture is the context for a taxonomy for the enterprise. Kate Simpson identifies a need for law firms to review and improve "a firm's Enterprise Information Architecture (or Firmwide IA) through an information housekeeping initiative". She looks at processes, systems and tools (where taxonomies fit, information assets, and governance as components to Firmwide IA and recommends a coordinated approach.

Building an ECM Taxonomy

Forrester has published a guide to creating taxonomies -- How To Build A High-Octane Taxonomy For ECM And Enterprise Search Systems (Nov 17, 2008). Forrester makes the point that "... it's vital to enrich the content that flows through these systems with more meaningful and structured metadata. Enriched content is simply easier to isolate, promote, find, and control." Table of contents is provided in the document excerpt.

Cost is $379 US but there is a promise of full refund in the first 3 weeks if you aren't satisfied.

Social Tagging

Many have examined the value of user tagging and compared it to the use of controlled vocabulary.

These are the topics in "The Value of Social Tagging in a Corporate Setting" (April 2006) by Stephanie Lemieux in the TaxoCop wiki. It includes a description of the use of social tagging at Raytheon where "people submit website suggestions (URLs) along with recommended tags/keyword which are subsequently verified and approved by librarians."

"Why does it work? Chiefly because the sites submitted are specific to a group or discipline, and no matter how hard we try, having a degree in library science does not give you a degree in engineering (insert discipline here). We do not speak their vernacular. We do well enough to add value with controlled terms, but these folk tags have a life of their own."

The 2008 Conference on Human Factors in Computing Systems had two papers on social tagging available through the ACM Portal:

• Social tagging roles: publishers, evangelists, leaders
• What drives content tagging: the case of photos on Flickr

Abstract and references are available. Access requires subscription to ACM.

Social Tagging at Museums

There are some interesting postings about tagging and folksonomies at conference.archimuse.com site hosted by Archives and Museums Informatics.

The site is described as "a collaborative space for professionals creating culture, science and heritage on-line"

J Trant posts entries to the archimuse blog about tagging. One refers to her presentation on Access to Art Museums Online: A role for social tagging and folksonomy?. Findings come from STEVE:The Museum Social Tagging Project.

Considerations for Enterprise Search

Why enterprise search is not internet search by Mary Branscombe, IT Pro (Dec 4, 2008)

Google has proven to be terrific on the Web, but will it do as well searching resources in an organization? Google uses link analysis in relevancy ranking which doesn't work as well inside an organization where there are no links between documents and files. An enterprise search tool needs to do more - as this article points out - working in large part with metadata. Several companies are mentioned - Recommind, Autonomy, Microsoft and InQuira - and Google's Search Appliance is also described.

The enterprise search engine must also "integrate with document repositories, corporate databases, ERP and CRM systems, email, call centre and customer support systems, directories like LDAP and Active Directory, your HR and accounting systems and everywhere else".

There are some figures in this article on the costs of looking for information - "businesses waste the equivalent of 10 per cent of salary costs (says the Butler Group) or information workers waste around three to four hours a week – a total of five weeks a year - because they don’t find the information they’re looking for a third to a half of the time (IDC and HP)."

Explaining Taxonomies

Waffles and Taxonomies by Rich Payne, AIIM Infonomics Magazine

Rich Payne orients readers to the concepts of taxonomy through an analogy with food orders at a Waffle House. The example works - somewhat.

I'm not sure he made the point about eggs completely clear. Let's imagine the Waffle House needs to sales records and other information about the product line. It would see that different kinds of eggs are big business for them. To help people find the information about eggs, they might set up a "taxonomy" for eggs with sub-types of scrambled, fried, poached, boiled. This resembles a thesaurus, where the scrambled, boiled etc are the Narrow Terms.

Two basic rules are interesting:

• Users must understand the "product of taxonomy"
• Almost everything has a relationship or is part of a hierarchy

Controlled Vocabulary in Folksonomies

Editorial - Folksonomies: Why do we need controlled vocabulary? by Alireza Noruzi , Webology (June 2007)

Answers the question "what can a thesaurus do for a folksonomy-based system?" Also makes clear how the folksonomy differs from a classification system.

"Folksonomy is a bottom-up approach where users themselves join the classification, compared to top-down taxonomy and library classifications. By this nature, folksonomy classification can reflect users' actual interest in real time (Niwa et al., 2006). In contrast to hierarchical library classifications (e.g., DDC or LCC) and thesauri, there is usually no limit for choice of tags in folksonomy; so many similar tags are generated. "

Taxonomies and ECM

Paul Quigley, Editorial Director at Enterprise Content Management 365, makes some interesting comments about enterprise taxonomy and folksonomies in the wake of Google's Index Now site search application.

"In the week that SEO behemoth Google took the wraps off its new indexing for content application 'Index Now' site search, the whole sphere of enterprise taxonomy and folksonomies is conjured as the role of knowledge management and context become ever more critical to effective content management strategies. ... The world of semantics, ontology and Kantian epistemology comes to mind in the vast 'contextualisation' process of content management, a veritable holy grail for all umbrella content in a massed digital asset world."

Off-the-shelf taxonomy will not work, but it is suggested that business process management can help in creating a classification / organizing scheme.

These are comments from the November 28th, 2008 newsletter. Subscribe to the newsletter to get updates and ECM industry news.