MongDB at Visibiz

Last Tuesday, I had the pleasure of giving a webinar about how we're using MongoDB in our application.  It's a brief presentation that outlines why we made the choice to use MongoDB, and how it fits into our application architecture.  You can find the slides here on Slideshare.net.  I'll be giving the same presentation at MongoPhilly and the Emerging Technologies conference at the end of this month.

I was thrilled that 10gen gave me the opportunity to help spread the word about MongoDB.  It's been great for us so far, and I'm looking forward to learning more about it as we begin to scale up our operation.  We're in private beta now with a limited number of users, but hope to open up the application to more users soon.  Stay tuned.

Mongo Happenings

In an earlier post, I mentioned that MongoDB is the one component of our architecture that I was a little unsettled with.  Mongo is new and different, which can be good for many things, but with your database manager, you usually want something a little more established.  It was first released in early 2009 so it's only been around for about 2 years.  And it's not your traditional relational, SQL-based datastore.  The whole document-based storage paradigm is new to most of us.

But...two recent events have me really stoked about MongoDB.  First, Grails recently announced support for Mongo under the GORM framework.  This is just awesome.  It means that the Grails community recognizes Mongo as a popular and viable datastore.  Adding GORM support makes Mongo a first-class citizen in the Grails world so now you can treat it just like all the other relational datastores that Grails supports under Hibernate.

Second, our company attended the MongoDC conference held in Washington D.C. last week.  Sponsored by 10gen and Intridea, the conference brought together MongoDB experts and users.  It's always good to know who the experts are, but it was really cool to see how many other folks are on the Mongo wagon these days.  And it's not like MongoDC was the only conference this year...10gen has run several already this year and more are on the schedule.  At the after-party, we were talking to Mathias (one of the presenters and a developer at 10gen) and he said that he's presented at "about 7 or 8" this year.

So the moral of this story is that Mongo is alive and well.  Community support is a great thing.  It certainly feels like Mongo is going to be a player for long time.  If you're wondering if a NoSQL DB is right for you, take a look at the use cases that Mongo is well suited for.

JSON-centric architecture

In my last post I mentioned that I'm now working at a startup.  I was one of the first employees, so I get to help set a lot of the technical direction.  I don't claim to be a genius, but I do like that part so I take the time to research what's out there.  We've been developing on this stack for just over 4 months and so far, so good.  We don't have a production environment with users yet, so it's too early to claim victory.  But we haven't come across any concerns yet, and everybody feels really productive.

I won't bore you with the things that we investigated but dismissed.  What's important is what made the cut and how we're putting things together.  Here's the highlevel view:

The main component in our architecture is Grails.  We love the fact that it's based on the JVM (we're all experienced Java programmers), but it doesn't carry the weight of the traditional Java EE world.  The fact that it's backed by SpringSource was a big factor for us.  We weren't comfortable with a platform/framework that didn't have significant support from the community or a real company.  SpringSource is now owed by VMWare so we know they're not going away any time soon.  And SpringSource is all about Java, so we feel confident that they'll continue to invest in Grails.  The quality of the plugins developed by SpringSource has been fantastic.  Can't say it enough...I love Grails.

Another decision we had to make was how to expose our services to various client devices (browsers, smart phones, other applications, etc.).  We quickly settled on JSON over REST.  We started out being "REST-like", just exposing HTTP GETs.  But we then realized (thanks MikeD!) that we should really follow the REST protocol.  I'm so glad we did, because it opened up the possibility to plugin standard components that support REST (we're using Jersey).  It also simplified our discussions about "what should our URLs look like?".  If you follow the REST protocol, most of those questions are answered for you.

JSON was an easy choice over XML simply because JSON, well.....JSON is simple, and XML is verbose.  Sure, XML has great tool support.  But JSON is gaining momentum in the community and is so easy to create, store, parse, debug, etc.  And the integration between Grails maps and JSON documents is dead simple.  Simple == good.

The next big choice was for the database manager.  We chose MongoDB.  I'll admit that we're taking a flyer on this one, and I'm a tad nervous about it.  One of the main reasons we chose Mongo is because it is a document-based datastore.  JSON documents, in fact.  Given that we're using JSON documents on the front-end (exposed through the RESTful services), and Grails as the app server, having a JSON-document datastore allows us to deal with JSON documents everywhere.  Again...dead simple.  The idea of transforming data from one format to another is almost non-existant in our architecture.

Mongo is not without risks and trade-offs.  Mongo doesn't support transactions, so we may have to get creative with our processing to ensure we don't lose updates.  We've already made some decisions to work around it, but the impact has been minor.  And Mongo isn't supported by a big name player.  10gen is doing a great job with it so far, and I hope they continue.

There are more components to discuss but I've rambled on long enough for now....

Jumping into the startup world

I recently joined a startup company in the role of tech lead. I came from a big company where I worked for more than 20 years. I had it good there, but I finally made the decision that I wasn't doing myself any good by staying there. Sure, the money was nice (I took a hefty pay cut to join the startup), but I wasn't growing and challenging myself the way I wanted to. So I put some trust in my abilities and made the leap to the world of startups.

We're starting from scratch which is something that most developers love. You don't often get the opportunity to jump on a bunch of new technology. Many times the projects that you're working on are well established, and carry the baggage of the existing components and technologies. If architected well (loosely coupled), there's a better chance of introducing newer tools, languages, frameworks, etc.

But there's nothing like starting from scratch. It can be fun (OK, it IS fun) if you love technology and learning. With the freedom of the blank slate, comes the responsibility of doing it right. A big part of doing it right is to constantly challenge yourself and stay up to date with the latest tools, techniques, and patterns.

Our platform and architecture is loaded with stuff that's new for me. And I love that. I'm having a blast digging into it. But they're not paying me to have fun (how do I get that job, anyway?). So I'm trying to set a productive direction for our team by blending components, tools and methodologies that have worked for the team members in the past with current technologies and trends.

We have a great team so far. Each team member brings a unique set of strengths that complements the other team members. And I learn from them every day. I don't kid myself that I know everything, and can make every decision. I rely on the team to bring their perspectives and experiences to the table and enjoy briefly debating the issues that come up. We haven't had many missteps so far, so I feel good about our collective knowledge and our ability to apply it to the real world.

I'll talk more about our architecture in future posts. But for now, I'm excited to hang on for the ride.