Sunday, July 12, 2009

Text Mining for Meaning

Many are pointing to the emergence of a Web 3.0 that brings together Web 2.0 collaborative qualities with a semantic web of connections and semantic tools for understanding.

Nstein, the digital publishing technologies company, held a webinar titled From Metadata to Meaning: Intelligence in the Semantic Era on June 25, 2009, moderated by Diane Burley.

Seth Grimes of Alta Plana presented the case of an exploding digital universe where individuals are creating 70% of the content. Search based on keyword matching won't be sufficient. The next generation of tools must make sense of the data to deliver meaningful answers. Text mining combined with semantic analysis is one direction. Google has some new capability at picking out data and its context - eg the employment data for a US state. Newssift from the Financial Times shows more capability at identifying organizations, places, people and topical themes in its aggregation of news. Grimes noted that semantically enriched content, linked data, context sensitivity, and location awareness are now recurring themes. Text mining / analytics is enabling Web 3.0 and the Semantic Web.

For designers and users this means:

• Automated content categorization and classification.
• Text augmentation: metadata generation, content tagging.
• Information extraction to databases.
• Exploratory analysis and visualization.

Jean-Michel Texier, Chief Technology Officer at Nstein Technologies, described Nstein modules for mining and analyzing data - essentially to make sense of vast stores of text by extracting concepts, identifying and normalizing entities, categorizing content, extracting relevant sentences, analyzing sentiment, and finding similar content.

The presentation is available in PDF on the dowload section of the Nstein website: http://www.nstein.com/en/downloads.php?sourceId=236 - registration required.

2 comments:

Seth Grimes said...

Thanks for attending the webinar.

Google has actually sucked in A LOT of data from the US Bureau of Labor Statistics. Google isn't waiting for LinkedData and the Semantic Web. All this data is on the Web, now, without semantic mark-ups, and Google can and does index it. That's a good thing!

Please note that the term "exploding digital universe" and the 2007 stat that "individuals are creating 70% of the content" come from an IDC report, per my webinar attribution.

Seth

Gwen Harris said...

Thank-you for your comments, Seth.

You included an example in your presentation about Google's use of data. For others reading this, Google describes "Searching for public data" at http://www.google.com/support/websearch/bin/answer.py?hl=en&answer=144522

Queries like washington unemployment rate do retrieve US public statistical data. A note to Canadians - this only works at google.com - not google.ca.

For access to data, tho not necessarily linked data, there is the new Wolframalpha.com "computational knowledge engine".

Also, fyi for people reading this - the digital universe is growing at a mind-boggling rate. It is forecast to be 5 times larger in 2012 as it was in 2008.

EMC Corporation has a page with the IDC studies from 2007 and 2008 and other materials.

http://www.emc.com/leadership/digital-universe/expanding-digital-universe.htm