“The plan is to have all sorts of speakers speak about all sorts of stuff. Not brands, advertising, blogging and twitter but interesting, unexpected, original things. I’m hoping to find fascinating people and to just ask them to speak about something they care about. I want to replicate the experience of clicking from one really good blog to another, ranging across sciences, arts, musics, jokes and whatever.”

—Russell Davies, founder of Interesting

The format is quite cool – a smallish number of speakers give 5 minute talks, and the rest of the night is unstructured. The speakers are from quite the range of life and the topics look extremely thought provoking.

Apache Cassandra
Cassandra is a fairly recent addition to the “no-sql” movement. Initially developed by Facebook, Cassandra was open sourced in 2008 and is now housed and maintained by Apache.

Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google’s BigTable. Like Dynamo, Cassandra is eventually consistent. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.

Cassandra was open sourced by Facebook in 2008, where it was designed by Avinash Lakshman (one of the authors of Amazon’s Dynamo) and Prashant Malik ( Facebook Engineer ). In a lot of ways you can think of Cassandra as Dynamo 2.0 or a marriage of Dynamo and BigTable. Cassandra is in production use at Facebook but is still under heavy development.

Data Model
The key difference between Cassandra and many of the other key-value stores is that it possesses the concept of a “Super Column.” For more information on what exactly a super column is (and an overview of the Cassandra data model in general) read the following blog post “WTF is a SuperColumn? An Intro to the Cassandra Data Model”

Performance
Many may be curious about the performance of these key value stores. We found an article that compared Cassandra to Voldemort

Interacting with Cassandra
Cassandra is developed in java however it does not allow access via JDBC. Instead it provides access to it’s repository via the “Apache Thrift” framework which basically allows developers in any major language to gain access to a given service (in this case the Cassandra data store). For those like me who are used to JDBC, the thrift interface can be a bit confusing given that there isn’t much documentation for them. The Java client made available on the Cassandra website is rather rudimentary with no support for things like transactions, therefore many people choose to make use of a library called Hector that provides many of the features necessary for any production worthy application making use of Cassandra.

The Project
What I wanted to to do was use the Cassandra data store on a pet project that I’m working on in my spare time. I intend to use the datastore to do the following:

1. Store messages being sent from one user to another
2. Track queries made by a given user
3. Store user complaints

Installation
Installation of the server is easy enough. Install Java 1.6 if you haven’t already. Edit /bin/cassandra.in.sh and change JAVA_HOME if you run a different version of java as your default JAVA_HOME. By default the server has an RMI port set to 8080, you can change the value in the /bin/cassandra.in.sh file as well.

Configuration
Edit /bin/storage-conf.xml to define various key spaces and column families. The concept of a Keyspace is similar to the schema in a relational database, where as the ColumnFamily is analogous to a table. In defining the ColumnFamilies you are defining the key groupings and how they are sorted. Therefore, I defined the following:

  <Keyspaces>
    <Keyspace Name="Nikahfied">

      <ColumnFamily Name="Queries"
                    Comment="User queries"/>

      <ColumnFamily Name="Complaints"
                    Comment="User complaints"/>

      <ColumnFamily Name="Messages"
                    ColumnType="Super"
                    CompareWith="UTF8Type"
                    CompareSubcolumnsWith="UTF8Type"
                    RowsCached="10000"
                    KeysCached="50%"
                    Comment="Message threads between users of the system"/>

    </Keyspace>
  </Keyspaces>

I would be creating a schema named Nikahfied and three table are Queries, Complaints and Messages. As you will notice, there is no real structure to the ColumnFamilies other than what type of column and the CompareWith strategy as outlined in the “WTF is a SuperColumn? An Intro to the Cassandra Data Model” article. In this case we have one super column and two regular columns. In code I will ensure that each of the rows fore each ColumnFamily contain the same content however there is nothing forcing you to do so. As you will notice, we can configure each of the ColumnFamiles individually. Messages will need to handle heavy reads and writes where as Queries and Complaints will be more write intensive. Messages therefore have a caching scheme defined where as the others do not as they will primarily handle writes rather than reads.

When initially defining the column structure I was a bit confused. I was taking the definition of Cassandra as a “key value store” literally and was confused about there being a primary key and the secondary keys. When defining a basic Column I though it would simple have a key and a value. So things would be defined as Nikahfied.<ColumnFamily>.key = value where this is actually not the case. For a basic ColumnFamily, the structure is as follows: Nikahfied.<ColumnFamily>.<Primary key>.<Secondary Key> = value. In the case of a super column, this goes one level further Nikahfied.<ColumnFamily>.<Primary key>.<Secondary Key>.<Tertiary Key> = value. That had me puzzled for some time…upon re-reading the “WTF is a SuperColumn? An Intro to the Cassandra Data Model” article I realized my mistake so I though I’d point it out to anyone else who was confused about this construct.

Data Structures:
Queries:
The structure of queries would be simple enough. I would store each query by date/time. To track which queries the user made when.

Nikahfied.Queries.UserId.<Date/Time> = <Query>

Complaints:
There is only one type of complaint wherein a user can complain about another user. So I wanted to track who received the complaint, who complained and when. The structure of a complaint would be defined as follows:
Nikahfied.Complaints.<Violator Id>.<Complainant Id> = <Date/Time>

Messages:
If there was ever a case to use a super column, a messages would be it. A user has a number of threads that contain one or more message. A user can receive and send messages. Messages themselves are the same, I simply need to track which threads contain messages that a user sent. I also need to keep track of which threads have unread messages within them. My initial design was the following:
Nikahfied.Messages.RecipientId.threads = A JSON Object that contains Threads defined in JSON Objects themselves. A thread being a group of
.sent_ids = An array of thread id's that contain messages that the user has sent
.unread_ids = An array of thread ids that contain unread messages

However, with this structure there was no need for a super column and I was determined on making use of one . So I expanded out the thread structure as follows:

Nikahfied.Messages.RecipientId.threads.<thread id> = A JSON Object that contains an individual Thread
.sent_ids = An array of thread id's that contain messages that the user has sent
.unread_ids = An array of thread ids that contain unread messages

Now that we have the content structure defined I now need to create a hook between my application and the Cassandra key value store. For development I have decided to use the Grails Framework to speed up development. I have included the Hector library to access the framework but needed to define a CassandraService to be the central point of contact between various controllers and other services to the Cassandra key store. Both Cassandra’s Thrift interface and Hector have pretty poor documentation so I created a Grails service that makes the interaction with Cassandra a bit clearer (at least to me). Note: You will need to be using Java 1.6.

import me.prettyprint.cassandra.service.CassandraClientPool
import me.prettyprint.cassandra.service.CassandraClientPoolFactory
import me.prettyprint.cassandra.service.CassandraClient
import me.prettyprint.cassandra.service.Keyspace
import org.apache.cassandra.thrift.Column;
import org.apache.cassandra.thrift.ColumnPath;
import org.apache.cassandra.thrift.NotFoundException;

import org.apache.cassandra.thrift.SuperColumn

class CassandraService {

    boolean transactional = true

    def servers=["localhost:9160"]
    def defaultKeyspace="Nikahfied"
    private static final String NOT_FOUND = ""

    private execute(keyspaceName=defaultKeyspace,block){
        CassandraClientPool pool = CassandraClientPoolFactory.INSTANCE.get();
        CassandraClient client = pool.borrowClient(servers);

        try {
            Keyspace keyspace = client.getKeyspace(keyspaceName)
            return block(keyspace)
        } finally {
            pool.releaseClient(client);
        }
    }

    /**
     * Get a single super column
     * @param columnFamily
     * @param secondaryKey
     * @param key primary key
     * @return Matching super column
     */
    public SuperColumn getSuperColumn(String columnFamily, String key, String secondaryKey) {
        ColumnPath cp = new ColumnPath(columnFamily)
        cp.setSuper_column(secondaryKey.bytes)
        return execute {Keyspace keyspace ->
            SuperColumn sc = null;
            try {
                sc = keyspace.getSuperColumn(key, cp)
            } catch (NotFoundException nfe) {
                sc = null;
            }
            return sc
        }
    }

    /**
     * Get multiple super columns
     * @param columnFamily
     * @param secondaryKey
     * @param keys primary keys
     * @return matching super columns
     */
    public Map multigetSuperColumn(String columnFamily, List keys, String secondaryKey) {
        ColumnPath cp = new ColumnPath(columnFamily)
        cp.setSuper_column(secondaryKey.bytes)
        return execute {Keyspace keyspace ->
            Map scMap = keyspace.multigetSuperColumn(keys, cp)
            return scMap
        }
    }

    /**
     * Get multiple columns
     * @param secondaryKey secondaryKey
     * @param keys primary keys
     * @param columnFamily column family
     * @return results if any
     */
    public Map multigetColumn(List keys, String secondaryKey, String columnFamily) {
    	ColumnPath cp = new ColumnPath(columnFamily)
        cp.setColumn(secondaryKey.bytes)
        return execute {Keyspace keyspace ->
            Map cMap = keyspace.multigetColumn(keys, cp)
            return cMap
        }
    }

    /**
     * Get a single column and it's values
     * @param columnFamily
     * @param key primary key
     * @return matching column
     */
    public Column getColumn(String columnFamily, String key, String secondaryKey) {
        def cp = new ColumnPath(columnFamily)
        cp.setColumn(secondaryKey.bytes)
        return execute {Keyspace keyspace ->
            Column c = null;
            try {
                c = keyspace.getColumn(key, cp)
            } catch (NotFoundException nfe) {
                c = null;
            }
            return c
        }
    }

    /**
     * Sets the new value for this column path
     * @param cf Column family
     * @param secondaryKey secondary key
     * @param key primary key
     * @param value value
     */
    def setColumnPathValue(String cf, String key, String secondaryKey, String value){
        def cp = new ColumnPath(cf)
        cp.setColumn(secondaryKey.bytes)
        return setColumnValue(cp, key, value)
    }

    /**
     * Sets the new super column value for this column path
     * @param cf Column Family
     * @param sc Super Key Id
     * @param secondaryKey Secondary Key
     * @param key primary key
     * @param value value
     * @return the value of the Column requested
     */
    def setColumnPathValue(String cf, String sc, String key, String secondaryKey, String value){
        def cp = new ColumnPath(cf)
        cp.setSuper_column(sc.bytes)
        cp.setColumn(secondaryKey.bytes)
        return setColumnValue(cp, key, value)
    }

    /**
     * Set a regular column
     * @param cp Column path
     * @param key primary key
     * @param value value
     */
    private setColumnValue(ColumnPath cp, String key, String value){
        return execute{ Keyspace keyspace ->
            keyspace.insert(key, cp, value.bytes)
        }
    }

    /**
     * Batch insert content for a give key and it's values
     * @param key primary key
     * @param columnMap column key values
     * @param superColumnMap super column key values
     */
    public batchInsert(String key, Map<String, List> columnMap, Map<String, List> superColumnMap) {
    	return execute{ Keyspace keyspace ->
        	keyspace.batchInsert(key, columnMap, superColumnMap)
    	}
    }

}

Using this class I have created a MessageService, ComplaintsService and a QueriesService that interact with the CassandraService to retrieve and store data. There is an adjoining Message and Complaints controller where as the QueriesService is referenced by the SearchController. Hopefully this blog post will provide you with a better understanding of how to get started with Cassandra the CassandraService class should provide you with a jump start on integrating Cassandra into an existing Grails or Java project.

• Parent object has a collection of Child objects
• We want the Parent’s create and edit GSPs to allow us to add/remove/update associated Child objects
• The controller should correctly persist changes to the collection of Child objects, including maintaining Child object ids so any other objects referencing them don’t get screwed up

I found a really nice solution that avoids adding a lot of code to the controller to sift out added/changed/deleted collection members. The original page seems to have disappeared, so here are copies from archive.org (easier to read) and Google cache (PDF).

I was disappointed that the original page is gone, and I found some small errors in the sample code, so I thought it would be nice to document here.

Here’s a sample project I created to go through this. Source code: one-many.tar.gz

The original example used Quest objects that can hold many Task objects. I’ll follow the Grails docs and use Author objects that can hold many Book objects.

First, create the Author class.

    import org.apache.commons.collections.list.LazyList;
    import org.apache.commons.collections.FactoryUtils;

    class Author {

        static constraints = {
        }

        String name
        List books = new ArrayList()
        static hasMany = [ books:Book ]

        static mapping = {
            books cascade:"all,delete-orphan"
        }

        def getExpandableBookList() {
            return LazyList.decorate(books,FactoryUtils.instantiateFactory(Book.class))
        }

    }

(Here’s a minor correction I had to make to the original document’s code. They declared getExpandableBookList as returning a List, but that gave unknown property errors. Using a plain def fixed that.)

This adds a bunch of useful behaviour right away. The mapping block declares that books will be deleted when they’re removed from the Author.books collection, so we don’t need to clean up anything manually. By initializing books to an empty ArrayList when an Author object is created, and by using the getExpandableBookList() method, we can easily add and remove Book objects to the Author.books collection.

Next, the Book class is pretty simple.

    class Book {

        static constraints = {
        }

        String title
        boolean _deleted

        static transients = [ '_deleted' ]

        static belongsTo = [ author:Author ]

        def String toString() {
            return title
        }

    }

Nothing too fancy here, but pay attention to the _deleted property. That’s what we’ll be using to filter out Book objects that need to be removed from the Author.book collection on updates.

For the views, I like to combine the guts of the create and edit GSPs into a template that they can both render.

    <div class="dialog">
        <table>
            <tbody>
                <tr class="prop">
                    <td valign="top" class="name"><label for="name">Name:</label></td>
                    <td valign="top" class="value ${hasErrors(bean:authorInstance,field:'name','errors')}">
                        <input type="text" id="name" name="name" value="${fieldValue(bean:authorInstance,field:'name')}"/>
                    </td>
                </tr>
                <tr class="prop">
                    <td valign="top" class="name"><label for="books">Books:</label></td>
                    <td valign="top" class="value ${hasErrors(bean:authorInstance,field:'books','errors')}">
                        <g:render template="books" model="['authorInstance':authorInstance]" />
                    </td>
                </tr>
            </tbody>
        </table>
    </div>

That uses _books.gsp to render the editable list of books.

<script type="text/javascript">
    var childCount = ${authorInstance?.books.size()} + 0;

    function addChild() {
        var htmlId = "book" + childCount;
        var deleteIcon = "${resource(dir:'images/skin', file:'database_delete.png')}";
        var templateHtml = "<div id='" + htmlId + "' name='" + htmlId + "'>\n";
        templateHtml += "<input type='text' id='expandableBookList[" + childCount + "].title' name='expandableBookList[" + childCount + "].title' />\n";
        templateHtml += "<span onClick='$(\"#" + htmlId + "\").remove();'><img src='" + deleteIcon + "' /></span>\n";
        templateHtml += "</div>\n";
        $("#childList").append(templateHtml);
        childCount++;
    }
</script>

<div id="childList">
    <g:each var="book" in="${authorInstance.books}" status="i">
        <g:render template='book' model="['book':book,'i':i]"/>
    </g:each>
</div>
<input type="button" value="Add Book" onclick="addChild();" />

And that uses _book.gsp to render the individual records. It’s a bit overkill to call out to another template for only a few lines of HTML, but that’s how the original example did it and I’ll do the same for consistency.

    <div id="book${i}">
        <g:hiddenField name='expandableBookList[${i}].id' value='${book.id}'/>
        <g:textField name='expandableBookList[${i}].title' value='${book.title}'/>
        <input type="hidden" name='expandableBookList[${i}]._deleted' id='expandableBookList[${i}]._deleted' value='false'/>
        <span onClick="$('#expandableBookList\\[${i}\\]\\._deleted').val('true'); $('#book${i}').hide()"><img src="${resource(dir:'images/skin', file:'database_delete.png')}" /></span>
    </div>

Here’s where I changed a bit more from the original example. I used jQuery because the selectors make things easy. Basically we render the books from the already-persisted author object, and keep track (using the _deleted field) of any that the user wants to remove. We also keep track of new objects to add.

One of the reasons I really liked this technique was how little impact there is on the controller. We just need to add this to the update method in AuthorController.

    def update = {
        def authorInstance = Author.get( params.id )
        if(authorInstance) {
            if(params.version) {
                // ... version locking stuff
            }
        authorInstance.properties = params
        def _toBeDeleted = authorInstance.books.findAll {it._deleted}
        if (_toBeDeleted) {
            authorInstance.books.removeAll(_toBeDeleted)
        }
        // ... etc.

The original example added similar code to the save method, but I don’t think it’s required for new objects (since they don’t have any already-persisted books to delete, only new books to create) so I only put it in the update method. I also changed it from find{} to findAll{} to guarantee that we get a list, and checked that we have objects to remove before calling the removeAll().

And it works great! Let’s look at some screenshots of the application in action.

First, we can create a new author and add some books right here instead of creating them separately and then matching them up.

Hit “Create” and it creates the Author and Book objects.

Edit the author we just created and see how we get a form that looks the same.

However, it’s worth noting that the books displayed here are the already-persisted ones, so the form is keeping track of their ids and whether we should keep them or delete them on update. Let’s delete the first one and add two more new books.

Now when we hit “Update” the controller has to be smart enough to remove that first book from the Author.books collection, then create two new Book objects and add them to the collection. And naturally, it is.

In addition to creating and destroying Book objects, we can update them. For example, let’s change the title of that first book to be the long version.

No problem!

So that’s one-to-many relationships in Grails forms. I hope it’s useful.

We recently presented at the Turning Statistics Into Knowledge conference, jointly organized by the US Census Bureau, the OECD and the World Bank, on July 15-16 in DC at the Census Bureau. Here is a pdf of it including our speaking notes, or you can click through it below, sadly without notes (goto the slideshare site to see the notes).

I promised a number of people links to non-technical to technical primers, here we go:

Non-technical

1. Very straight-forward overview, very good for sharing with non-technical people:
2. Very quick one-pager, “The “Semantic Web”: a low-fat, no-nonsense introduction” by @ajubdeen
3. Original article from Tim Berners-Lee showing his vision.
4. CNN Making sense of the ’semantic Web’
5. Tim Berners-Lee 2009 TED talk on Linked Data

Semi-technical

1. Largely non-technical, injects a few buzzwords, “Quick Linked Data Introduction” by Michael Hausenblas
2. Understanding the New Web Era: Web 3.0, Linked Data, Semantic Web by Richard MacManus, summarizing a multi-part overview by Greg Boutin

Technical

1. An excellent overview “Linked Data Tutorial” by Michael Hausenblas
2. 30 Minute Guide to RDF and Linked Data by Ian Davis and Tom Heath

After that, you’re off to the races!

(but as I said, just send us an email if you have questions or wanted to talk about it some more)

Graph Layout

After looking at various existing graph layout algorithms, I decided to reinvent the wheel and developed my own approach. Each node has a mass based on its size, and I simply used gravity calculations to determine the (inverse, so they repel each other) force each node apples to each other node. You can see this in action when you mouse over a node, it gets bigger and other nodes retreat from it as its mass increases.

This allows each node to position itself in a way so it is visible. It’s not perfect, but it works okay and makes the graph feel responsive to the user.

How to use it

Double click on a node to load its relationships and related nodes. A node must be an rdf node to load properly, I haven’t implemented any way to handle normal web links yet. Basically, only click on links that look like http://dbpedia.org/resource/something, or that you know are rdf.

click for demo of the wikipedia article for baseball

If you’re just interested in the pretty pictures, stop here, if you want more technical information read on.

Implementation

I just used javascript and followed the HTML5 docs for using the canvas element. One thing I ran into was the fact that Firefox doesn’t yet have a full canvas implementation, specifically it doesn’t yet render text on the canvas. I tried doing some voodoo with absolute positioning divs containing text over the canvas, and after a brief descent into insanity, I found the excellent typeface.js library which emulates the missing canvas calls for text.

As for the javascript code itself, I implemented a simple object graph that can contain vertices or edges, then a render loop just advances the object graph by a given amount of time and renders the scene to the canvas.

The back end serving up rdf was done quickly in grails using the Jena framework to do all the heavy lifting. There is some minimal inference going on using the RDF data to determine if something is an image.

Disclaimer

Keep in mind that this is a work of progress that I have not spent much time with. It has bugs, and it’s nowhere near usable for anything useful, but it serves its purpose as a proof of concept. I’m not sure if it will even run under IE, but it should run on any browser that has a decent javascript + canvas implementation.

Conclusions

The canvas element is cool, and works well.

Basically I hope you like it. Please leave any suggestions or feedback as comments. I realize it’s probably full of bugs, but that’s the nature of the beast.

The course was laid out in a novel way – as a scrum project itself! As Product Owners we were asked to compose User Stories capturing what we hoped to learn from the course, and at the end of each “iteration” within the two-day course, were asked to re-prioritize these, knowing time was tight and we may not get to all topics. This helped us to have hands-on experience with Story creation and re-prioritization, with the added bonus of us each being personally invested in the outcome instead of it being a theoretical scenario.

One of the topics I was interested in was how to adapt the scrum process for a Product Owner that is not only not on site, but is overseas and in a different time zone. At 2Paths, one of our larger projects that was recently completed involved a client overseas. This project was very successful, and we accredited much of its success to our client’s buy-in and participation to our scrum process. We had, however, modified the traditional scrum process to accommodate our client’s schedule and time availability. Because they were quite busy with other projects, and due to the large time difference, the client didn’t attend our daily scrum as is recommended. Instead we had weekly phone check-ins which augmented our regular email communication to discuss issues as needed.

I was hoping in the course to see if there would be a more recommended way for this sort of Product Owner to be involved in a project, and how we could try to more closely follow the traditional scrum process. However, the course was more geared towards an internal Product Owner. This type of Product Owner would be a lot more hands on, would physically attend the daily scrum, and spend time sitting within the dev team to facilitate clearer communication. It also seemed that many of the other course attendees had similar project roles where the Product Owner was in-house. From what I could gather, I think we handled our process as well as we could given the circumstances.

The course was well organized, and the two presenters Bryan Stallings and Skip Angel were quite knowledgeable and helpful. We had many good group discussions and participants actively participated, sharing their own war stories. I’d recommend this course for other ScrumMasters wanting to get a better sense of what is required by a Product Owner, and how they fit into the scrum process.

My primary goal was to get a handle on the vocabulary so that I could ask semi-intelligent questions to get more information. That worked out well, because Bethany gave us an excellent high-level introduction to the field.

• supervised learning – the system uses a reference set of labelled data to classify new data points
• unsupervised learning – no reference data set
• reinforcement learning – the system gets feedback, positive or negative, from its decisions and optimizes its behaviour accordingly

We talked about classification problems a lot. Imagine an 2D space like the one below with a bunch of data points that we can intuitively group into clusters. This could represent some set of objects you’re trying to group into categories based on two measured properties, say a voice recognition system that figures out words based on pitch and length (a bad example). The question is how to draw boundaries around those clusters, especially when some points are reasonably close to two or more clusters.

Some unsupervised learning algorithms:

• hierarchical clustering
• k-means
• expectation maximization

The k-Means algorithm made the most sense to me. It works by first  picking cluster centres at random and associating each data point with a cluster according to the nearest centre. It then refines the cluster centres by computing the centroid of the points within each cluster. These two steps are repeated until the solution converges (i.e. the centres stop moving around). Unfortunately, you need to tell the algorithm how many clusters there are at the beginning and the system doesn’t always converge. These problems are general to unsupervised learning. K-means is cheap enough that you can run it multiple times with random starting centre points and compare the outcomes to increase confidence in your solution.

Some supervised learning algorithms:

• k-nearest neighbours
• bayesian classification
• decision trees
• neural networks/support vector machines

The k-nearest neighbours algorithm seems pretty straightforward here. The training data are a bunch of labelled data points. For each test input, you find the k nearest labelled neighbours. The test input is then assigned to the same cluster as the majority of those k nearest neighbours. If you put the newly labelled input into the pile of reference data (i.e. let the system learn as it goes), this can make your clusters drift over time. I suspect this can get dangerous if you’re not careful.

Bayesian classification uses P(C|x)= P(x|C)P(C)/P(x) which sounds vaguely familiar from the old stats and physics courses. Best to think about that some more before discussing it. Decision trees looked more straightforward to read, but I think I’d have a hard time implementing something that creates its own. I’ve had very mild introductions to neural networks a few times and they always seem like a whole mess of complexity that I don’t want to deal with.

Bethany talked about some of the difficulties with supervised learning, such as the amount of training data and time required, along with the associated problems of overfitting or underfitting the data. If your training data are too closely coupled, you risk misclassifying input data that’s a bit further out. Similarly, if your training data are too loose, you risk problems of cluster boundaries overlapping. On the topic of the amount of data and time required, Bethany made the excellent point that people don’t like the effort required for supervised machine learning, but we spend orders of magnitude more on training babies and then marvel at how much they learn. That’s not to say that we should spend more time with machines instead of babies; rather, it’s just important to realize that even an exquisitely refined and capable machine like a human brain still needs a lot of training input before it can do anything useful.

Reinforcement learning algorithms fall into two categories:

• model-based
• model-free

Those reminded me of model predictive control and other feedback systems like PID from the control theory courses I took in school. In the machine learning context, you also want the system to guess when it encounters unknown data. In reinforcement learning, the machine gets a particular reward or punishment for classifying/misclassifying each new input, and it classifies new input based on its own self-interest. In the model-free algorithms (e.g. Q-learning, function approximation, policy iteration), the machine has some way of learning based only on the rewards and punishments of its classifications. For unknown data, they do things like guess randomly (Q-learning) or work out probabilities of reward/punishment based on past decisions (policy iteration). In the model-based  algorithms (R-max, KWIK), the machine has some model of the environment that gets refined based on input. This can take a lot of work from the programmer to determine a good model, but it sounds like a properly tuned algorithm can classify very reliably.

I hope I’ve described that reasonably accurately. I did a bunch of probability in various math and physics courses back in university, so while reading about these topics, they dance between understandable and just plain Greek. I also built an autonomous robot for a course, so I appreciate just how hard it is to get machines to do anything remotely intelligent-looking in the real world.

A big thanks to Bethany for her informative talk! I’m sure it will fuel many arguments around the water cooler.

We experienced what could best be described as an illuminating and transformational year at 2Paths.   From changes to our processes and tools to the diversification of our team demographic, to teaming up with some fabulous new valued partners and clients, change abounded.  And there’s not a soul among us who would argue that it wasn’t for the better.  As our year concludes, my 2008 retrospective blog finds no shortage of things to reminisce about.   Sit back and enjoy.

From a people perspective, the team welcomed 2 new babies into our extended family.  Congratulations to Michal and Lori on the birth of baby Cohen, and congratulations to Ricky and Lin on their second addition to the family, Lucas.  I personally bought a house and got married to the love of my life, Willy the caveman, in a small Whistler wedding. Garret married the love of his life, the beautiful Priscille, in grand fashion Winnipeg-style.  2Paths gladly welcomed new members to the team Omar Khan, Sandra Widjaja, Gary Meehan, Tim Oxenford, and. Kari Braaten.  And last, and very important – we celebrated Michal Urbanski’s 5-year anniversary with the company by sending him on kite-board lessons in Squamish.  He returned exhilarted and intact, thank goodness.

Processes and tools were areas that 2Paths focused heavily on in 2008, recognizing that these will differentiate us from our competition in the future.  We honed in on our existing skills by cultivating 3 certified scrum-masters (Lorill, Geoff and Omar).   Scrum (a brief daily morning developer meeting) has been pivotal to our successful software projects.   Someone likened it to inviting your mother-in-law to live with you – she may not fix everything, but she will most certainly point our everything that needs to be fixed.  Then it’s just about the fixing.    Next, the team continued to examine open source software tools and added some more to our repertoire, including Grails, Hudson, and Groovy, to name a few.   To ensure we got the latest on emerging technologies and uses of technologies, we also sent 2Pathsians to attend conferences on Geo-spatial technology, the Semantic Web, QCon (a conference of technical depth and enterprise focus), No Fluff Just Stuff, and Agile Methodologies.

In the spring, we moved our office to the Arthur Erikson Waterfall building by Granville Island from Gastown.   In addition to loving the view, we have been enjoying lunch at Go Fish, the NoodleBox, and Sushi Go, not to mention whipping up concoctions in our gourmet kitchen.  After our first pay cheque in the new location, our Synesso Cyncra coffee machine made its proud appearance in our kitchen providing java nirvana for everyone to enjoy.  There has been talk of a group membership to Caffeine Anonymous, but so far we’ve been holding out alright.

From a team-building perspective, we did a TON of fun things together this year.  Our long-standing tradition of playing together, letting off some steam, and laughing is our real secret to strong teamwork.  Here’s the illustrious list of our 2008 fun:

-    We bonded over some spring-skiing/riding at Grouse mountain, followed by beer on the patio in the glorious sun;
-    We learned to Curl at Marpole Curling;
-    We tested our trust in each other during rock-climbing;
-    We golfed at Gleneagles and BBQed at Judi’s new home;
-    We enjoyed some fine-dining at Chambar;
-    We participated in Bike to Work week
-    We enlisted a yoga instructor to lead us through weekly hatha yoga practices on the new roof-top patio in the summer, and in our boardroom in the winter.

One of our major projects has been with the “Insert name of international organization which we will shortly be revealing” on an application that has global impact and countless social good benefits.  It gave us an opportunity to go globetrotting to the likes of Paris and London.  Better yet, it presented a chance to work in concert with partners, Gagan Diesh and Kaaren Schulz of DesignStamp, http://www.designstamp.com/,whom we’ve grown to love and cannot do without.  We were also glad to continue our solid working relationship with our valued client, Carlson Wagonlit Travel, http://www.carlsonwagonlit.com/en.  And, we played pretend, and acted like a client to ourselves, dictating our own requirements towards an internal initiative which we will may be talking about a lot more in 2009.

So, as you can see we have been very busy this year.  Not just building web applications that make complicated problems palatable and bite-size, but also in being good to ourselves and to each other.   And we look forward to an exciting 2009, filled with market improvements, more work with our valued partners and clients, and at the very least, some good conversations, hopefully with you, over a fancy-pants coffee from our cool espresso machine.  Have a safe and happy holiday season, everyone!

2Paths Team