Taxonomies to enhance search

Two excellent articles by Penny Crosman in Intelligent Enterprise on the use of taxonomies to enhance search.

Search in Focus: Implementing a Taxonomy (Dec 2006)

" Search engines don't know the difference between reading glasses and drinking glasses, but a taxonomy puts your query in context. We outline several ways to build taxonomies, ranging from the tough but potentially more accurate approach of building from scratch to the easier but potentially compromised approach of buying a prebuilt taxonomy or using automated clustering software. "

Select from http://www.intelligententerprise.com/
toc/?day=01&month=12&year=2006

The Perfect Search (March 2006)

"Google-style search is all right for some, but an enterprise search demands a mix of technologies and techniques that lead to better accuracy."

Select from http://www.intelligententerprise.com/
toc/?day=01&month=03&year=2006

Taxonomy Warehouse

Factiva Expands Taxonomy Warehouse Partner Network , Press Release (Dec 19, 2006)

"Factiva, a Dow Jones and Reuters Company, has added three new partners to its Taxonomy Warehouse Partner Network. Austin, Texas-based Cycorp, Los Altos, Calif.-based Ibuki, and Paoli, Pa.-based Intellisophic join the Taxonomy Warehouse Partner Network, which currently includes Thomson Gale and WAND, to bring together a collection of taxonomies, thesauri, and classification schemes to its global customer community."

Taxonomy Warehouse is at www.taxonomywarehouse.com

Seamark Navigator 4.1

Siderean will organize unstructured data by extracting the "metadata". It is notable for its navigation. The new release, Seamark Navigator 4.1 " features a number of visualization enhancements designed to interactively present relationships, analyses and trends." Of interest, this version introduces "Tag Library" support for enhancing user customization."

The Librarians' Internet Index uses Siderean to provide navigation to its subject directory.

See Easing navigation in KMWorld (Nov 8, 2006)

Factiva Taxonomy Warehouse

Taxonomically speaking in KMWorld (Oct 23)

"Factiva has introduced an enhanced version of its Taxonomy Warehouse, a community directory and information source for taxonomies, thesauri and classification schemes. It is said to provide enterprises, as well as academic and government organizations, the information required to categorize internal and external data collections and ensure that relevant answers to search queries are found with speed and precision, Factiva reports."

View and search the Taxonomy Warehouse at http://www.taxonomywarehouse.com/index.asp

Endeca Moves to Version 5.0

Endeca Information Access Platform 5.0 Released, by Avi Rappoport, Newsbreaks (Oct 16)

Endeca has been long known for its faceted search. Online shoppers will know it from the Walmart and Home Depot websites. With version 5.0 Endeca is adding "relationship discovery, extended data source integration, and greater business control."

"Version 5.0 has a new feature called Relationship Discovery, which leverages a richer metarelational index that can support ad hoc dynamic connections for related content. This content can then be displayed using visualization tools pioneered by business intelligence, such as charts and graphs."

Does Tagging Work?

User tagging is much talked about on the Internet, and is extensively used for social bookmarking (del.icio.us, magnolia, rawsugar), blog posts (Technorati), community news sites (Digg.com, Newsvine), photo sharing sites (Flickr) and video sharing (YouTube). Subject directories such as Yahoo and Open Directory Project with their subject classification structures are considered very old Web 1.0 and nearly obsolete.

This presentation (Sept 2006) by Thomas Vander Wal (InfoCloudSolutions) on Understanding Folksonomy: Tagging that Works is a very good overview of what tags are and why some people love them.

In tagging, he explains, "People are not so much categorizing, as providing a means to connect items (placing hooks) and provide their meaning in their own understanding."

There are good examples from the main social bookmarking sites on how to use the tags to dig into a topic.

Online Taxonomy Seminar from Stellent

Stellent is hosting an online seminar on August 23, 2006 on "Practical Strategies for Taxonomy and Metadata Design" as part of its ECM Expert Series. Zach Wahl from Project Performance Corporation is the speaker. His expertise is in "the development of business taxonomies and ontologies and [he] has deployed Enterprise Information Portals for a variety of organizations."

Finding through tags

Findability with tags: Facets, clusters, and pivot browsing by Rashmi Sinha (July 27) - Sinha considers three types of browsing: facets such as at wine.com; clustering and recommendations; and "pivot browsing" -- "When everything (tag, username, number of people who have bookmarked an item) is a link, you can use any of those links to look around you. You can change direction at any moment. Its a lightweight browsing mechanism, less structured and more freeform than browsing traditional hierarchies or even browsing faceted systems."

Librarians talk about folksonomies

The Library 2.0 Folksonomy Gang in Talking with Talis (July 27) - a group of librarians talked with Paul Miller at Talis in the UK about folksonomies and user generated tagging and contrasted these to the Dewey and Library of Congress Subject Headings. The main question - is there a place for both? Karen Schneider of LII.org especially endorsed the use of a user-created folksonomy at LII. Others commented on the benefits of capturing new terminology through user tagging. To the question of problems in variety of terms and spellings, one presented the solution of pulling suggestions from a controlled vocabulary. There are many points to consider thanks to this podcast of their telephone conference.

LII.org Will Adopt Faceted Navigation

Librarians' Internet Index (lii.org) announced at the ALA conference that it would be adopting faceted classification to assist in navigating its collection of quality web sites. The technology is provided by Siderean Software.

""One of the key reasons for selecting Siderean's Seamark Navigator was its front-end navigation features," said Karen G. Schneider, director of LII. "By replacing our existing, cumbersome search capabilities with faceted navigation, our growing community of users will be able to quickly and easily browse our vast collection of peer-reviewed content. With more than 10 million hits per month and a repository of 18,000 items categorized in 14 main topics, 300 primary topics, and 5,000 subtopics, we needed a way for people to rapidly browse these collections to improve discovery. We also needed a way to highlight collections and feature new and custom collections. Faceted navigation allows us to illuminate these items. Users also will be able to avoid standard search 'dumps' or result lists, which lead to dead-ends. Instead, Siderean will present results in logical categories to enable users to navigate selections through LII's rich metadata. This is a tremendous benefit to our users.""

Librarians' Internet Index Selects Siderean Software to Improve Access to High Quality Web Content, Business Wire via MarketWatch (Jun 26)

Categorizer from Inxight

Categorize completely, KMWorld (June 28)

"Inxight Software has announced Categorizer 5.0, a new, hybrid categorization engine with automated techniques and editorial controls to ensure accuracy. Version 5.0 can also deploy third-party and industry-standard taxonomies and includes a "workbench" feature designed to facilitate the creation of testing of taxonomies."

Semantic Web Part 2

The idea of having a grand taxonomy (or several smaller ones) as the backbone for a semantic web seems to still have a glimmer of life to it judging from this report from the W3C (World Wide Web) conference taking place in Scotland.

Researchers look to semantic Web to drive Internet "Computer scientists discuss ideas for organizing the Internet's growing mass of data" By Jeremy Kirk, IDG News Service (May 24)

"Much of the discussion is centered on the "semantic Web," the term for how researchers believe information on the Web can be intelligently labeled, interpreted, and linked through applications that can draw relationships and discover buried information."

Adding keywords to a metatag for description won't do it. "Labeling information on the Internet involves tagging it with code and then classifying it into a taxonomy. Customized taxonomies and ontologies, or data models, could be created for different subject matters to connect disparate, rich information tucked away on servers."

"In the long term, businesses need to realize that developing ontologies for data is an asset, said Richard Benjamins, director of innovation and research and development at Intelligent Software Components in Madrid."

But who will develop and use these on the public Web?

Google Clusters with Inxight

Google is teaming up with Inxight to add clustering and entity extraction to search results from its Enterprise Search Engine.


Inxight Search Extender for Google Brings Sophisticated Clustering and Entity Extraction to a Familiar Enterprise Interface by John Blossom, Shore Communications (May 23, 2006)

"... using the interfaces provided into the Google Search Appliance and Google Desktop Inxight Software has announced the integration of its content extraction and content clustering capabilities to content retrieved by Google's enterprise search technology. As demonstrated at the Enterprise Search Summit conference today in New York City, the Inxight Search Extender clusters search results into dynamically generated categories, providing both topic-oriented views and extracted content in categories such as "People, Places, Date/Time, Product, Currency" and so on. Right-clicking on the results can allow one to hook into content from subscription services such as LexisNexis or Hoover's or to online services such as Google Earth."

Taxonomy for the Intranet

Cathy McKnight, a business consultant at Prescient Digital Media, has some advice for people building the web interface for the corporate intranet -- Don’t forget to add the Tax(onomy). People know that the web site should be attractive but might not realize that part of good design is to make the information findable.

"In this world of ultra-usability and user-centric websites, many great intranet sites go unused because of poor, or altogether lacking, site taxonomy.

So what is taxonomy? How does it differ from information architecture? And how do you create an effective taxonomy for a successful intranet?"

Conferences 2006

Information Today has announced two important conferences for people working with taxonomies and enterprise search.

Taxonomy Boot Camp Returns This Fall -- "Taxonomy Boot Camp is an intensive 2-day conference for taxonomists, info pros, and anyone in charge of categorizing, organizing, or managing content. The event will be at the San Jose McEnery Convention Center in San Jose from November 2 to 3, 2006. "

Go "Beyond the Basics" with Enterprise Search Summit -- "Enterprise Search Summit, the 2-day intensive conference designed for information managers, IT and search professionals, is in New York City from May 23 to 24, 2006." Peter Morville, author of Ambient Findability, will be the keynote speaker.

Keeping up with Information Design

InfoDesign: Understanding by Design is an excellent weblog and resource for news and information done by Peter J. Bogaards. Peter Morville, president of Semantic Studio, says in his testimonial on the site that he uses it everyday. Major categories include information design, information architecture, user experience, and metadata. There is not a category for taxonomies but a search will bring up several entries posted to other categories.

Taxonomy vs Folksonomy

Ontology is Overrated: Categories, Links, and Tags from Clay Shirky's Writings About the Internet (2005)

Clay Shirky has written an excellent review of formal categorization versus user categorization. It is based on two talks he gave, one about ontologies and the other on folksonomies and tags. He identifies many failings in the traditional classification schemes and especially takes Yahoo to task for its categorization. Classification systems tend to be biased, he argues, and be rigid. User tagging, on the other hand, is more democratic and self-regulating. The article is something of a polemic in defense of user tagging. We may not agree with all the points but it makes us appreciate the weaknesses of classification schemes and some of the potential of user tagging.

As it happens, I found this article by searching for taxonomy at Keotag, the new all-in-one meta search for locating tagged items, and selecting Blinklist as the target source. There is something to be said for this style of serach.

Taxonomies at CiL2006

A Tale of Two Taxonomies , Information Today blog (March 24, 2006)

Taxonomies were on the program at the Computers in Libraries Conference 2006. Jennifer Evert, Taxonomy Product Manager at LexisNexis and Marjorie Hlava, President of Access Innovations presented their views on the importance of taxonomies and indexing.

Intellosophic Taxonomies

New intelligence for Documentum CIS, KMWorld (Mar 13, 2006)

"Intellisophic reports it has integrated its pre-built taxonomies with EMC Documentum Content Intelligence Services (CIS). The move gives Documentum CIS customers access to Intellisophic taxonomies covering a variety of specific industries including: automotive, chemicals, defense, energy, finance, high-tech, information sciences, legal, life sciences, military and others."

Healthline's Taxonomy

People interested in examples of taxonomies may find the health taxonomy developed by Healthline useful. A recent press release described the work of Healthline.

"To present this information in medically accurate context, Healthline has developed the largest consumer health taxonomy of its kind, featuring an information classification system that delivers precise, relevant open Web results, related physician-reviewed articles and HealthMaps® – Healthline's unique visual navigation tools – to optimize the search experience for individuals seeking healthcare information."

From the press release - Info.com Adds Health Vertical Search to its Search Platform (Feb 28, 2006)

Enterprise Search Mix

The Perfect Search By Penny Crosman, Intelligent Enterprise Magazine (March 1, 2006)

Article surveys the enterprise-search tools available to companies and identifies key technology developers and vendors.

Google-style search engines are not suitable for some situations. "Where many repositories of non-Web content and documents need to be searched or critical information must be found quickly, companies need to design searches that approximate human reasoning."

Key components to improved enterprise search will be tools for structuring content - now easier with XML tools; building taxonomies - or possibly using light-weight approach of user tagging, or clustering tools; and analyzing text to extract names (entities) or concepts. IBM's Unstructured Information Management Architecture is being adopted by many vendors as a "backbone" for these approaches.

Companies where this is most needed may include e-commerce sites, pharmaceutical companies and government agencies, and call centers.

"The sophisticated search pioneers are e-commerce sites, pharmaceutical companies and government agencies, which have the most to gain: greater sales, faster drug development, detection of terrorist activity. Call centers are getting search makeovers so that multiple search tools can mine unstructured content and databases together and give reps all the information they need to close calls."

Folksonomies - stay openminded

Pros and cons of tagging are examined in Folksonomies: Tidying up Tags?, a new article in DLib (January 2006) by Marieke Guy and Emma Tonkin, two writers from UKOLN, the UK centre for digital information management.

From the abstract: "In this article we look at what makes folksonomies work. We agree with the premise that tags are no replacement for formal systems, but we see this as being the core quality that makes folksonomy tagging so useful. We begin by looking at the issue of "sloppy tags", a problem to which critics of folksonomies are keen to allude, and ask if there are ways the folksonomy community could offset such problems and create systems that are conducive to searching, sorting and classifying. We then go on to question this "tidying up" approach and its underlying assumptions, highlighting issues surrounding removal of low-quality, redundant or nonsense metadata, and the potential risks of tidying too neatly and thereby losing the very openness that has made folksonomies so popular."

Makes the point that folksonomies that we know at the popular sites such as del.icio.us and flickr are trying to serve two worlds - the individual and the collective. This may affect quality but the authors ask people to be patient - in time the folksonomy will reflect some consensus and evolve with changes in language.

ERIC Thesaurus

The thesaurus used by the Education Resources Information Center (ERIC) is another excellent resource that can be viewed online. ERIC updates this quarterly. Browse or search for terms.

Learn from MeSH

The U.S. National Library of Medicine has created an educational video to help people understand and use the MeSH vocabulary in searching Medline and PubMed.

Branching Out: The MeSH® Vocabulary describes the structure and content of the Medical Subject Heading thesaurus, shows how indexers use it to index documents, and instructs searchers on how to utilize it to find documents and understand the topic.

This is an excellent introduction to the nature of controlled vocabulary and its value in enabling information retrieval.

Constructing a Subject Gateway

For the technically inclined, this article on Constructing Web subject gateways using Dublin Core, RDF and Topic Maps offers some guidance on the technical database aspects of building a taxonomy.

In this research project, the authors built a software tool that would make it easier to build a subject gateway, essentially a taxonomy for subject access, and using the metadata definitions of Dublin Core. The article described the relational database structure.

Constructing Web subject gateways using Dublin Core, RDF and Topic Maps by Jesús Tramullas, Department of Librarianship and Information Sciences, and
Piedad Garrido, Department of Computer Science and Systems Engineering
Zaragoza University, Zaragoza, Spain; Information Research, Vol. 11 No. 2, January 2006

http://informationr.net/ir/11-2/paper248.html