We recently presented at the Turning Statistics Into Knowledge conference, jointly organized by the US Census Bureau, the OECD and the World Bank, on July 15-16 in DC at the Census Bureau. Here is a pdf of it including our speaking notes, or you can click through it below, sadly without notes (goto the slideshare site to see the notes).

I promised a number of people links to non-technical to technical primers, here we go:

Non-technical

1. Very straight-forward overview, very good for sharing with non-technical people:
2. Very quick one-pager, “The “Semantic Web”: a low-fat, no-nonsense introduction” by @ajubdeen
3. Original article from Tim Berners-Lee showing his vision.
4. CNN Making sense of the ’semantic Web’
5. Tim Berners-Lee 2009 TED talk on Linked Data

Semi-technical

1. Largely non-technical, injects a few buzzwords, “Quick Linked Data Introduction” by Michael Hausenblas
2. Understanding the New Web Era: Web 3.0, Linked Data, Semantic Web by Richard MacManus, summarizing a multi-part overview by Greg Boutin

Technical

1. An excellent overview “Linked Data Tutorial” by Michael Hausenblas
2. 30 Minute Guide to RDF and Linked Data by Ian Davis and Tom Heath

After that, you’re off to the races!

(but as I said, just send us an email if you have questions or wanted to talk about it some more)

Because the OECD already had a mature system with a soap interface to query data, we not only didn’t want to reinvent the wheel, but we also didn’t want our solution to be intrusive or impose changes to the legacy system.

When coming up with our design, we strived to keep a clear delineation between the legacy system and the new application via an abstraction layer. We made use of the existing legacy heavyweight soap API and wrapped it with a lightweight restful API. This way we were able to create a stand-alone API sitting on top of the legacy system with loose coupling between the two. We were able to create our solution without interfering with the existing infrastructure.

Of course, integration even in this loose-coupled way was not without pain points. Because we did not have any control over the underlying legacy soap service implementation, the solution was tricky to optimize. Fortunately, we were able to identify some areas of concern early in the process and work with the OECD to fix these issues.

Another important element of legacy system integration is the need to closely communicate with the legacy system maintainers to coordinate rollout of any legacy changes. Because our application was mostly designed to be independent of the legacy system, this did not greatly affect our project. However, an in-person visit to the OECD headquarters for a meeting with the legacy system maintainers to get buy-in for our project was invaluable for opening the flows of communication in this regard.

Legacy system integration can be challenging, but by prioritizing loose-coupling between the systems and maintaining good communication between all parties, legacy system integration can be attained very successfully!

We recently had the pleasure of working on a project with the OECD, a large international non-profit organization based in Paris. This was a year-long project with a team of five to seven developers to create a user-friendly web-based query wizard interface to the OECD’s Development data. The OECD had an existing legacy application allowing users access to their data, but it required expert knowledge of the underlying datasets in order to access the information. The query wizard was to hide this complexity from the user and to provide one unified view to the various Development datasets.

An integral part of SCRUM is that there is one Product Owner that is available and vested in the SCRUM process. As the OECD is a large organization based overseas and a nine-hour time difference, this was seemingly going to be difficult.

The OECD Development Co-operation Directorate presented two excellent Product Owners – one representing the technical side and one representing the business side – who provided a unified front to the OECD’s vision and user requirements. 2Paths decided upon a two- week iteration schedule. Because of the nine-hour time difference and the busy schedules of the OECD employees, we decided upon a schedule of weekly video-assisted teleconferences with the OECD team, the third-party funders, and the 2Paths project team. We conducted bi-weekly iteration reviews, alternated with bi-weekly checkins in lieu of face-to-face stand-up meetings.

In addition to the weekly teleconferences, we established a good email rapport with the Product Owners, and corresponded via email throughout the iteration to clarify any outstanding issues or requirements. This process worked well for the most part, as the Product Owners were very decisive; they knew what they wanted or decided fairly quickly what they wanted. We also kept a well-organized Wiki site to hold all relevant information and documentation surrounding the project to which the OECD had access. This assisted in coordinating work and kept information in a known central location. We at 2Paths took care not to bombard the OECD with technical terms, keeping the language appropriate to their technical savviness.

Occasionally the time zone difference was a hindrance to our process as this necessitated a twenty-four hour turnaround in email responses, which sometimes was not quick enough for our liking. However, for any critical issues that needed to be resolved, we were mostly able to set up impromptu after-hours calls to the OECD to sort them out on the spot.

Having a team of two Product Owners instead of just one Product Owner also didn’t end up presenting any problems. They seamlessly provided one unified voice to represent the OECD. They were careful to come to agreement amongst themselves before presenting any decision on any issue to 2Paths.

The OECD admitted that if they had their choice, they would have preferred to have a 2Paths developer on-site in Paris throughout the life of the project (I would have volunteered!), but in general the teleconferences and email discussions were sufficient to get through any issues that arose. It was helpful that 2Paths representatives were on-site in Paris both for the initial requirements- gathering phase, and for the final project handover at the end of the project. The OECD Product Owners were also able to meet with 2Paths in person in Seattle mid-project.

When embarking on this project, we were unsure of how receptive the OECD would be to our agile processes, as traditionally large organizations can be more inert and resistant to change. We were very pleasantly surprised as the OECD embraced our processes wholeheartedly. They were very glowing about our “refreshing” agile process, and how they could quickly and often see tangible results of our development. They felt they got value from their ability to have constant input into the evolution of the software. They were use to having longer rollout times due to heavier weight processes in-house, so were very impressed by the progress each iteration.

The OECD also got some extra fringe benefits. Through our software development process, the OECD was able to identify and work on areas in their own process that had room for improvement.

Another key to the success of this project was that the two chosen Project Owners were very knowledgeable of their business and their requirements. 2Paths spent a good chunk of time working with them at the start of the project to establish a good solid high-level design of the product. There was also up-front time invested into the User Design of the application, and 2Paths with the aid of sub-contractors DesignStamp, came up with detailed user personas. Feedback revealed that OECD found this process extremely useful.

Take aways for 2Paths from this project were that it IS possible to work in an agile way with large organizations, especially when knowledgeable and open- minded Product Owners are chosen to represent the organization. Providing tangible results early and often to generate feedback can be a refreshing change for employees working in a large organization. Instead of arrogantly preaching the Agile software development process, being sensitive to the way that large organizations work and working WITH them as a team will ensure buy-in and aid in the success of the project.