We recently presented at the Turning Statistics Into Knowledge conference, jointly organized by the US Census Bureau, the OECD and the World Bank, on July 15-16 in DC at the Census Bureau. Here is a pdf of it including our speaking notes, or you can click through it below, sadly without notes (goto the slideshare site to see the notes).

I promised a number of people links to non-technical to technical primers, here we go:

Non-technical

1. Very straight-forward overview, very good for sharing with non-technical people:
2. Very quick one-pager, “The “Semantic Web”: a low-fat, no-nonsense introduction” by @ajubdeen
3. Original article from Tim Berners-Lee showing his vision.
4. CNN Making sense of the ’semantic Web’
5. Tim Berners-Lee 2009 TED talk on Linked Data

Semi-technical

1. Largely non-technical, injects a few buzzwords, “Quick Linked Data Introduction” by Michael Hausenblas
2. Understanding the New Web Era: Web 3.0, Linked Data, Semantic Web by Richard MacManus, summarizing a multi-part overview by Greg Boutin

Technical

1. An excellent overview “Linked Data Tutorial” by Michael Hausenblas
2. 30 Minute Guide to RDF and Linked Data by Ian Davis and Tom Heath

After that, you’re off to the races!

(but as I said, just send us an email if you have questions or wanted to talk about it some more)

RDF

For those who are unfamilar with RDF a short description would be:

The Resource Description Framework (RDF) is a language for representing information about resources in the World Wide Web. This Primer is designed to provide the reader with the basic knowledge required to effectively use RDF. It introduces the basic concepts of RDF and describes its XML syntax. It describes how to define RDF vocabularies using the RDF Vocabulary Description Language, and gives an overview of some deployed RDF applications. It also describes the content and purpose of other RDF specification documents.

For more information about RDF read the following primer hosted on the W3C’s website.

RDF Stores

An RDF store allows for flexible definition of content which is a shortcoming of traditional relational database. An example of this would be the need to add a field within a database. In a relational database one would have to define a new column in the database table which is not usually achievable from a program given that most systems are not foolish to allow database changes from code. However, in an RDF data-store things are stored as nodes and relationships between nodes. Therefore to add a new field simply requires that a new node be created and an establishment of a relationship between the existing nodes and the new node. This flexibility is extremely handy in cases where the data being captured by a system changes often. Add to this the creation of an RDF schema or an ontology and one has the ability to semantically query the data you have within your data-store and are able to make use of meta-data in a way that can not be done within a relational database.

I was thinking that this type of flexibility would be of great benefit for a project we hope to be working on in the new year. Given Zephira’s backing of Mulgara I thought I would investigate whether it could be ready for prime time and the level of complexity required in using it.

Mulgara

Mulgara was initiated in 2006 and was a fork of the Kowari project which died or became unsupported as of 2005. Their claim is that Mulgara has the following featured:

• Native RDF support
• Multiple databases (models) per server
• Simple SQL-like query language
• Small footprint
• Full text search functionality
• Datatype support
• Supports and tracks W3C Specifications and guidelines
• Large storage capacity
• Optimized for metadata storage and retrieval
• Multi-processor support
• Independently tuned for both 64-bit and 32-bit architectures
• Low memory requirements
• On-disk joins
• Streamed query results

The remainder of this post will cover my experiences with using Mulgara.

Rough Start

Unfortunately things didn’t start off well. Though the downloading and installation of the Mulgara server was painless there were a number of issues that I came across that would deter many from considering the use of it in a production level system:

Connectors are not included in the default download

Currently there are only a few means of accessing Mulgara to perform CRUD operations:

• A JRDF connector
• A Jena connector
• Straight RMI (remote method invocation)

In the default download from the Mulgara website, the JRDF and Jena connectors are not provided. After a lot of struggling, I resorted to downloading the source and building the server from scratch to ensure that the connectors were included within the jar files.

After getting to this stage I tried to follow the tutorials to try and connect and interact with the server. It seems that the documentation on the mulgara website is stale and that they have not been able to keep their tutorials up to date because the code provided in them does not work unless one uses an in-memory database which is not specified in the tutorial. I posted questions to the various user group email lists and receive no responses.

Once I figured those aspects out things went more smoothly though there is a significant amount of boiler plate code that needs to be created to allow for simple CRUD operations.

Creating a connection to the server

As noted above, one can not simply use JDBC to connect to the server but have an option of using the majority of the popular RDF creation frameworks such as JRDF, Jena or Sesame. I initially tired to use Jena but it seems that support for the JenaMulgara connector died quite a while ago and the connector does not work for the latest version of Mulgara. Therefore, I moved on to the JRDF connector given that the Sesame connector is quite immature.

// Create the URI of the server
java.net.URI serverURI = new java.net.URI("rmi", hostName, "/" + serverName, null);

// Create a new session factory, ensure that it’s local
SessionFactory sessionFactory = SessionFactoryFinder.newSessionFactory(serverURI, false);

// Get a local JRDF Session (local)
Object o = sessionFactory.newJRDFSession();
org.mulgara.server.JRDFSession session = (JRDFSession) sessionFactory.newJRDFSession();
[/sourcecode]

Create a model/database

The above give you a connection to the Mulgara server. To create a model/database requires the following section of code:

java.net.URI modelURI = new URI("rmi", hostName, "/" + serverName, graphName);
java.net.URI modelType = new URI("http://mulgara.org/mulgara#Model");
session.createModel(modelURI, modelType);
org.jrdf.graph.Graph graph = ClientGraph(createGraphProxy(modelURI, session));

Create nodes/relationships

In order to create RDF triples one needs to specify a subject, predicate and an object. There are three types of Java objects that can be used in this in building these relationships:

1. BlankNode – a blank node that is a node used to group other relationships. A BlankNode can be used as either a subject or an object
2. Literal – a literal value of some sort (String, number, etc.) A literal can only be used as an object and not a subject or predicate
3. URIReference – a URIReference is used to define the predicate or relationship between the subject and the object.

org.jrdf.graph.GraphElementFactory elementFactory = graph.getElementFactory();
org.jrdf.graph.BlankNode blanknode = elementFactory.createResource();
org.jrdf.graph.URIReference predicate = elementFactory.createResource(new URI());
org.jrdf.graph.Literal literal = elementFactory.createLiteral(value);

Lastly to create and insert a triple one simply needs to do the following:

org.jrdf.graph.Triple triple = elementFactory.createTriple(subject, predicate, object); // Create the triple object
graph.add(triple); // Store the triple in Mulgara server

For sample code on how to do everything from connect to query and delete see the end of this post.

Performance

I wrote a simple application that took the contents of a relational database and converted it into RDF and stored it in Mulgara. The database I was using was big but not huge, roughly …. All these numbers are based on running the application on a MacBook Pro with a 2Gh Core Duo processor and 2GB of RAM. I initially wrote the app to do inserts one at a time which was obviously inefficient but I wanted to test out the speed of an insert. Each insert of a single triple took roughly 0.17 seconds. In round 2 I started doing inserts in batch. In batch mode it seems that inserts took roughly 0.08 seconds per insert of a node. Both speeds are not particularly fast but like I said, this is running off my laptop so one can’t expect superb performance.

Conclusion

Given that RDF stores are competing with regular relational databases I would have hoped that there was an easier means of connecting to the database and that RM wasn’t being used under the hood. Given the push for RESTful interfaces within the semantic world, I was surprised to see that the Mulgara server does not have a RESTful interface which would relieve the need for RMI completely. Once I got over my dependence on the abhorrent tutorials and documentation things went fairly well though I have performance concerns when dealing with large volumes of data. The main area of concern for me is the requirement of RMI which has been well documented as being not very performant. It pains me to have to use RMI when running both the server and the application on the same server. What is nice is that the server is completely transactional and any failure results in a roll-back. The last concern I have is the fact that the user community/group around Mulgara does not seem to be very active or dedicated to ensuring that the product is supported to the extent that one can rely on getting up to date documentation and a steady stream of bug fixes.

Sample Code

The following code sample shows how to perform inserts, updates, deletes and selection.

package org.twopaths.jrdf;

import java.net.InetAddress;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.UnknownHostException;

import org.jrdf.graph.BlankNode;
import org.jrdf.graph.Graph;
import org.jrdf.graph.GraphElementFactory;
import org.jrdf.graph.GraphElementFactoryException;
import org.jrdf.graph.GraphException;
import org.jrdf.graph.Literal;
import org.jrdf.graph.Triple;
import org.jrdf.graph.URIReference;
import org.jrdf.util.ClosableIterator;
import org.mulgara.client.jrdf.AbstractGraphFactory;
import org.mulgara.query.QueryException;
import org.mulgara.server.JRDFSession;
import org.mulgara.server.NonRemoteSessionException;
import org.mulgara.server.SessionFactory;
import org.mulgara.server.driver.SessionFactoryFinder;
import org.mulgara.server.driver.SessionFactoryFinderException;

public class Sandbox {
	public static void main(String[] args) {
                Graph graph = null;
		try {
			// Create the host name
			String hostname = InetAddress.getLocalHost().getCanonicalHostName();

			// Create the URI of the server
			URI serverURI = new URI("rmi", hostname, "/" + "server1", null);

			// Create a new session factory, ensure that it's local
			SessionFactory sessionFactory = SessionFactoryFinder.newSessionFactory(serverURI, false);

			// Get a local JRDF Session (local)
			Object o = sessionFactory.newJRDFSession();
			System.out.println(o.getClass().getName());
//			LocalJRDFSession session = (LocalJRDFSession) sessionFactory.newJRDFSession();
			JRDFSession session = (JRDFSession) sessionFactory.newJRDFSession();

			//create a new Model
			URI modelURI = new URI("rmi", hostname, "/" + "server1", "exampleGraph");
			URI modelType = new URI("http://mulgara.org/mulgara#Model");
			session.createModel(modelURI, modelType);

			//create a JRDF Graph for the model
//			graph = new JRDFGraph(session, modelURI);
			graph = AbstractGraphFactory.createGraph(serverURI, modelURI);

			//get the Factory
			GraphElementFactory elementFactory = graph.getElementFactory();

			//create resources
			URIReference person = elementFactory.createResource(new URI("http://example.org/staffid#85740"));
			BlankNode address = elementFactory.createResource();

			//create properties
			URIReference hasAddress = elementFactory.createResource(new URI("http://example.org/terms#address"));
			URIReference hasStreet = elementFactory.createResource(new URI("http://example.org/terms#street"));
			URIReference hasCity = elementFactory.createResource(new URI("http://example.org/terms#city"));
			URIReference hasState = elementFactory.createResource(new URI("http://example.org/terms#state"));
			URIReference hasPostCode = elementFactory.createResource(new URI("http://example.org/terms#postalCode"));

			//create values
			Literal street = elementFactory.createLiteral("1501 Grant Avenue");
			Literal city = elementFactory.createLiteral("Bedford");
			Literal state = elementFactory.createLiteral("Massachusetts");
			Literal postCode = elementFactory.createLiteral("01730");

			//create statements
			Triple addressStatement = elementFactory.createTriple(person, hasAddress, address);
			Triple streetStatement = elementFactory.createTriple(address, hasStreet, street);
			Triple cityStatement = elementFactory.createTriple(address, hasCity, city);
			Triple stateStatement = elementFactory.createTriple(address, hasState, state);
			Triple postCodeStatement = elementFactory.createTriple(address, hasPostCode, postCode);

			// Add triples to graph
			graph.add(addressStatement);
			graph.add(streetStatement);
			graph.add(cityStatement);
			graph.add(stateStatement);
			graph.add(postCodeStatement);

			//get all Triples
			Triple findAll = elementFactory.createTriple(null, null, null);
			ClosableIterator allTriples = graph.find(findAll);
			while (allTriples.hasNext()) {
				System.out.println(allTriples.next().toString());
			}

			//search for address (as a subject)
			Triple findAddress = elementFactory.createTriple(address, null, null);
			ClosableIterator addressSubject = graph.find(findAddress);
			while (addressSubject.hasNext()) {
				System.out.println(addressSubject.next().toString());
			}

			//search for the city: "Bedford"
			Triple findCity = elementFactory.createTriple(null, null, city);
			ClosableIterator bedfordCity = graph.find(findCity);
			while (bedfordCity.hasNext()) {
				System.out.println(bedfordCity.next().toString());
			}

			//search for any subject that has an address
			Triple findAddresses = elementFactory.createTriple(null, hasAddress, null);
			ClosableIterator addresses = graph.find(findAddresses);
			while (addresses.hasNext()) {
				System.out.println(addresses.next().toString());
			}

		} catch (UnknownHostException uhe) {
			uhe.printStackTrace();
		} catch (URISyntaxException urise) {
			urise.printStackTrace();
		} catch (SessionFactoryFinderException sffe) {
			sffe.printStackTrace();
		} catch (NonRemoteSessionException nrse) {
			nrse.printStackTrace();
		} catch (QueryException qe) {
			qe.printStackTrace();
		} catch (GraphException ge) {
			ge.printStackTrace();
		} catch (GraphElementFactoryException gefe) {
			gefe.printStackTrace();
		} finally {
                        try {
                            graph.close();
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                }
		System.out.println("DONE");
	}
}

There was one vendor showcase in particular that I got a great laugh out of. I was sitting in the very front row, right in front of the presenter who was about 4 feet in front of me. Before I go on I should give you some information about myself, I am a Muslim who looks very Muslim…ie the full beard, middle eastern looking, etc… I, like all the other attendees has a name tag on with my full name written in large print which is visible from several feel away. So he starts his talk about the tool that they are selling. Mid way through the presentation he starts a demonstration on how the tool was used to prove that two people who claimed to not know each other did in fact know each other and that they were smuggling funds between them.

As he went through this presentation, he paused, saw me and my name and became really uncomfortable. You see, the names of the people he was talking about had the exact same last name as me and I don’t think he really saw me sitting there until that point. At that point he tried really really hard to come up with another example to demonstrate the tool but for the life of him he could not. I had to fight the urge not to stir the pot to see how much more uncomfortable I could make him feel.

Regardless, you know this technology is going to take off when the US department of defense is interested in it. It seems that they are dumping large sums of money into it’s development and use.

1. Linguistics.
   • the study of meaning.
   • the study of linguistic development by classifying and examining changes in meaning and form.
2. Also called significs. the branch of semiotics dealing with the relations between signs and what they denote.
3. the meaning, or an interpretation of the meaning, of a word, sign, sentence, etc.: Let’s not argue about semantics.

What we are talking about here is devising a means for a computer to be able to understand the meaning of language. Once it can understand language. When you enter a question into Google such as “What is the best way to exercise your lower abs?” google goes through and analyzes the phrase/sentence you entered into the search box and breaks it up into keywords. It then uses these keywords to search their repository for the information within a web page that best matches the keywords you entered. Therefore, in the example I provided above, it will go through and extract words like ‘best’, ‘exercises’, ‘lower’ and ‘abs’ and use these keywords to search their repository and return matches that have a high likelihood of containing the information you’re looking for. Google has little to no idea what the sentance “What is the best way to exercise your lower abs?” means.

What semantics hopes to archive is to extract information from the internet such that a computer or set of computers could perform a search understanding exactly what “What is the best way to exercise your lower abs?” means and would therefore return search results matching exactly what you are looking for. In the above example it’s fairly clear what I’m looking for but what if I entered something like “paris hilton”? What exactly am I looking for? Am I interested in the Hilton hotel located in Paris, France or am I interested in information about the trash TV celebrity? By now I’m sure you get the idea.

Needless to say, getting a computer to understand meaning isn’t an easy task and therefore the community that is hoping to make semantics a reality holds two conferences a year on the topic to gather like minds and to showcase how the technology has progressed along with any real world uses of the technology. The conference has been held for the past four years and has grown steadily each and every year. This year there were over 1100 attendees which is remarkable given the downturn in the US economy. The remainder of this post will outline the technologies of interest and some real implementations of the technology.

Technologies and Specifications

Gleaning Resource Descriptions from Dialects of Languages (GRDDL)
GRDDL is a W3C specification to allow people to make their content semantically accessible. For example, one can have a website containing all sorts of data within it that may or may not be structured. The GRDDL specification allows one to embed a link to an XSLT transform that will take the XHTML of the webpage as an input and output a set of RDF triples representing the content of your page.

Sesame
Sesame is an open source RDF framework with support for RDF Schema inferencing and querying. It can be deployed on top of a variety of storage systems (relational databases, in-memory, file systems, keyword indexers, etc.), and offers tools to developers to leverage the power of RDF and RDF Schema.

ELMO
Elmo is a toolkit for developing Semantic Web applications using Sesame. Elmo wraps Sesame, providing a dedicated API for a number of well known web ontologies including Dublin Core, RSS and FOAF. The dedicated API makes it easier to work with RDF data for the supported ontologies. Elmo also offers a set of tools related to the supported ontologies, including an RDF crawler, a FOAF smusher and a FOAF validator.

JENA
JENA is a Semantic web framework that can be used to access RDF, RDFS, OWL and SPARQL and a rule based inference engine. Included within the framework is an RDF API, an OWL API and a SPARQL query engine.

Topaz
Topaz is a RDF persistence and querying service framework.

NetKernel
1060 NetKernel is a resource oriented microkernel and RESTful application server created from the convergence and unification of the powerful fundamental concepts found in the World Wide Web and Unix. NetKernel is being used by a large number of major semantic applications.

Persistant URLs
A PURL is a Persistent Uniform Resource Locator. Functionally, a PURL is a URL. However, instead of pointing directly to the location of an Internet resource, a PURL points to an intermediate resolution service. The PURL resolution service associates the PURL with the actual URL and returns that URL to the client. The client can then complete the URL transaction in the normal fashion. In Web parlance, this is a standard HTTP redirect.

Implementations

The following are a list of applications that have been created using semantic technologies in part or as a whole. Examples of implementations such as these provide direction as to where this technology can be used to solve real world problems.

Powerset
Powerset is a website that has sucked in the content from Wikipedia and Freebase. They have taken this content and have translated it into a very very very large RDF store using NetKernel (or so I believe). Therefore, using their interface, one can search the data using semantic technology. If one is doing research on a given topic and has confidence in the data contained within Wikipedia, then using powerset to do your research would be of great use. Search result and content related to a particular item are easily retrivable using powerset. For example, if I were researching the pouring of concrete and were to enter “How do you pour concrete” you would see a result of all content that matches the information you are looking for understanding as best they can the meaning of what you are asking. Upon selecting the first result you would not only be given information about concrete and a slew of related content.

Twine
Twine is a social network focused on sharing your interests. In other words, creating a grouping of people interested in the same stuff. It is currently in private beta but hopes to go into public beta sometime in the late summer/early fall. The concept revolves around the idea that if you are interested in a certain topic, twine can offer information/articles/etc that fit within that area of interest. This is where the use of semantic technology fits in. They use semantic technology to search for and filter content related to what you are discussing and so on within your group. I was extremely disappointed with the fact that they made no attempt to showcase their technology by providing a walk-through of their application, but rather spent the entire time talking high-level about the application and showing a stupid video created about the website by some pre-teen. A real waste of an opportunity.

Garlik
Garlik is the darling of the semantic world. They are an example of where semantic technology can be used to solve real world problems. In this case, the use semantic technology to help protect your identity online. Currently they offer their services within the UK. Information is sketchy as to how exactly they do this using semantic technologies. I was really disappointed that they sent a non-techy to speak about their product but that seemed to be the running theme of the these types of companies within the conference, keep your secrets guarded regardless of whether it would further the proliferation of the technology.

Zepheira Remix
Now this tool is something that follows along the lines of Garlik in that it provides a solution to a real world problem. In this case they are focusing on sharing data. They have created a tool that significantly reduces the complexity of using semantics to share data. Remix allows you to import two files and merge, query, generate content between them with no need to understand anything about semantics or even the underlying technology. Remix features an AJAX front end that allows users to massage the data as they see fit and extract a meaningful summary of the data in an HTML format. Zepheira’s goal is to advocate for the use of semantic technologies while encouraging others to attempt to reduce the barriers of entry to harnessing the power of the technology. ie lets make it easier for people to use semantics through the use of frameworks and applications.

All in all, the conference was great. I have to admit, before attending the conference I was a skeptic about the technology but after the conference I think there are some great possibilities for this technology in the very near future. I look forward to attending next year. The only thing that I believe would improve the conference is an increase in details about the technology and/or implementation/usage of the technology rather than just skimming over things at a very very high level.