Apache UIMA and Semantic Search

Editor's Notes

1. Hello everybody, I am Tommaso Teofili, I am from Rome and I work in Sourcesense and I’m an Apache UIMA committer. In this talk I am going to tell you about Apache UIMA & Semantic Search.
2. Apache UIMA stands for Unstructerd Information Management Architecture, it's an architectural framework to manage large volumes of unstructured data. It was an IBM project donated to Apache. It's currently an Incubator podling and the new 2.3.0 release is coming. It's also been approved as an Oasis standard. By the way, I became Apache UIMA committer on August.
3. UIMA is made of components, which can be put in a pipeline to be executed. Every Apache UIMA component is described via XML descriptors. The most important of which is the Analysis Engine, that is responsible for the very analysis of documents. Each AE can be aggregated using an aggregate XML descriptor. To manage large volumes of data UIMA can scale using Asynchronous Scaleout. Each UIMA pipeline is made of AEs, one or more flow controllers and CAS consumers, the component which is responsible of treating the output (usually CASs).
4. Apache UIMA is not inherently a semantic search tool. The semantic search tool package for UIMA is not open source. Lucas is a CAS consumer for UIMA capable of putting UIMA annotations inside Lucene indexes, a semi-semantic approach.
5. UIMA is well suited as a metadata generation engine for CM systems. It can enrich data with annotations, entities, categories and so on, eventually linking them. For those who know the US Jeopardy game, IBM it's building a system able to answer open-domain questions against humans and it's based on UIMA-AS. We'll see two examples of the power of UIMA to boost semantic search.
6. Here's the first one: Recently during my college studies I used UIMA to extract metadata to build semantic searches on a real estate market analysis tool called Macpi. zone statistics -> UIMA to find zones, price, filter out bad matchings frequent announces -> disambiguation, data filtering
7. Another little demo I’m going to show is about integrating Apache UIMA and AlchemyAPI from Ochestr8. AlchemyAPI provides, you guess, services to extract Entities from text, but what we are interested in here is using the linked data, to exploit, for example DBPedia, Freebase and so on data clouds. So the power of UIMA here is again putting together in a pipeline many components, once my system learned I can easily substitute this component with a UIMA Dictionary using the ConceptMapper. I get Alchemy component then I can put the OpenCalais component, my personal dictionary of specific terms, a machine learning engine based on what UIMA extracts and so on. Here we’ll see indexing documents by concepts.
8. Ok, let’s go to the demos!