QCon SF 2008 – Day 3

November 24, 2008 | by Omar

I was pretty excited about the “Architectures you’ve always wondered about” stream for day 3 of the conference. It started off horribly. Dan Pritchett of MySpace was up first and basically took the change to showoff his personally little project that allows events from their thousands of servers to notify them what’s going on within them in a centralized manner. How he figured people were interested in hearing about the log files of MySpace.com baffles me…not only were people rolling their eyes but some were falling asleep (myself included). He really didn’t do anything to help the tech community have a better view of MySpace. He finished his session in 25 minutes and then opened the floor for questions…crickets for the most part. The moderator tried valiantly to get people to ask questions but what the hell do you want us to ask about log files? There went 30 minutes of my life I will never get back.

Joe Stump from Digg.com was up next and his session was awesome. He talked about the evolution of their architecture from the point he joined to what they are planning on rolling out early next year along with some of the bumps they’ve encountered. MySQL’s scalability had major issue so they moved to IDDB and are considering using MemcacheDB which is MemcacheD mixed with Berkley DB.  Joe pointed out their use of MogileFS for the management of images on digg images which I will be taking a look at for our current project. Joe, you rock, maybe you can give some pointers to Dan…then again hopefully MySpace will send somone with a clue next year.

Aditya Agarwal from facebook was great. He talked about all the different aspects of facebook and the use of PHP/MySQL being the underlying architecture and how for them it . They tend to use MySQL simply as a key value store and profile content is randomly scattered across their thousands of servers. They use LAMP for the most part but have created a framework that allows them to use services written in other languages within their application.

The discovery of the day was CouchDB. One of the lead developers/designers was sitting at the same table as me during the Digg session. Tim Bray was quite excited about the technology as well having mentioned it in his keynote on day 2. I attended one of two sessions about that covered what it’s about from a high level. It has it’s roots in Lotus Notes database design given that it was created by one of the people who worked on Lotus Notes in the past. It has some very cool features:

  • RESTful so it doesn’t require any connectors
  • Stores all content in JSON
  • Querying the database is done via javascript functions for performing map reduce like functionality
  • Very efficient and performs versioning and synchronization when distributed

Jan made some good arguments was to why it has some strengths that traditional RDB’s don’t such as no need for schema management, etc. It’s a technology that I plan on taking a serious look at and may possibly use in the future.

In conclusion, QCon was a great experience. The hosts tried their best to make sure things were as tech focused as possible. There is a need to ensure that speakers cater their presentations to the audience and keep to the subject of the presentation. The lagging economy did have an effect on the conference, there were 200+ people that had been registered that didn’t attend (you could see all of the uncollected badges at the registration desk) and if the economy improves I think that next year will be even better. It’s truly a conference for developers by developers which I truly enjoyed. It was great to meet several of the authors of my favourite tech books. I would have preferred more opportunities to network with the people at the conference, an overall guest list would have helped with that or a formal networking event given that a large percentage of the nerds there were stereotypical introverted nerds who didn’t like having a strange person walk up to them and make conversation. Maybe I came off as sleazy or creepy or a threat to homeland security ;) . Regardless, I hope to attend next year.

Update: This blog post has been featured on the infoq.com website

Bookmark and Share

Tags: , , , , , ,

Leave a Reply