We've been busy, very very busy

by Greg Olsen

You've probably noticed we've been pretty quiet with respect to product announcements over the last several months, and it was for good reason.  The entire engineering and product management team has been heads down implementing a set of significant improvements to the Coghead service that will provide both short and long term benefits to service users (these updates will appear on the service by mid-December).  The engineers have put in an amazing effort that we're all very proud of (and soon they will be able to re-establish relationships with their friends and family members whom they haven't seen much of for months). 

The Analysis

Back in June we saw an opportunity to improve two critical aspects of the Coghead service:

- Client performance, robustness, and usability: The Coghead client (the user interface that runs in the browser) was very feature rich, which was good, but was slower for many type of interactions than it needed to be.  Sluggishness was significantly degrading usability, particularly for users on client hardware with less memory or slower cpus.  Some of these users were also experiencing timeouts on forms that were rich in content.  We were also generally unsatisfied with the speed at which we were able to make UI changes (both for performance and usability) and our ability to automatically test the UI.

- Cost effective support for very large scale expansion of the service: Interest in the platform was high and we saw that we would need to support rapid expansion in both direct users of the service and in Coghead affiliate related users.  It was also clear that many Coghead users were outside of North America, and that international growth was going to be earlier than we expected.  Our hosting infrastructure was adequate, but it would have required significant ongoing investment of resources and focus to handle our expected growth profile.

The Decisions

We decided to make two significant upgrades to the service:

- Change the underlying client-side technology infrastructure from openlaszlo to Adobe's Flex:  We'd been following Flex since its introduction, and after Adobe changed its licensing model and introduced Flex 2.0 and the Flash 9 runtime, it became very interesting.  The runtime behavior of Flex-based code in the Flash 9 plug-in was significantly better and the available development and testing tools were also better.  We determined that by changing to Flex, our UI would perform significantly better and it would be much easier to develop and maintain.  The future prospects of the Flex platform (e.g. Flex 3, Flash 10, and AIR) and the rapidly growing community of Flex developers worldwide were also compelling.  The challenge was that switching to Flex basically meant rewriting the entire UI implementation.

- Move our server-side hosting infrastructure to a service-based grid deployment using Amazon's EC2 and S3 services:  Like Adobe's Flex, we'd been watching the evolution of Amazon's Web Services offerings for a long time.  Back in February, we began experimenting with deployment of the Coghead service on an EC2 instance.  The Amazon EC2 and S3 services were appealing for multiple reasons. The cost efficiency was certainly compelling.  Our early analysis showed a significant reduction in projected costs for comparable cpu and storage.  The network infrastructure was also compelling with excellent peering and future prospects of geographic distribution.  Finally, the federated deployment architecture we could deploy on EC2/S3 would allow us to deploy a very robust service that relied much more on automated recovery and management than on data center staff or on hardware reliability.  Our service could be more agile and cost efficient as we adapt to changing needs of customers.  The challenge was that work was required to adapt our deployment architecture to a federated model that took full advantage of the EC2 and S3 services.

The Implementation

After several months and many late nights, we completed the Flex rewrite of our UI implementation.  For the most part, the new UI is a reimplementation of the old UI - although we did take the opportunity to slip in a few key improvements to usability based on feedback from our user base (details on these changes will be described in a separate post). Overall the result is even better than we expected.  UI performance is significantly improved, and our development velocity is significantly increased.  Not only are our developers more efficient, with better tools and automated test infrastructure; we've been able to more rapidly ramp up new developers and have more than doubled the size of our UI dev team. Over the next several months, we will be implementing a wide variety of new features that will further enhance the user experience for both direct users and affiliates.

The move to EC2/S3 was executed through a series of steps dating back to February.  We designed a load distribution, failover and instance management scheme that leveraged the EC2 and S3 services and that would provide high service levels for Coghead users.  The design supports dynamic failover to minimize service distruption and it allows us to allocate compute and storage resources with fine granularity to match customer usage needs.  The implementation started in late June and has gone through several iterations and through extensive simulation testing.  The result is a server infrastructure that will provide an immediate improvement in service levels (uptime and latency) and will support customer growth far into the future.

Conclusion

Very shortly next month, the Coghead service will experience two major upgrades that will provide significant benefits to Coghead users. There are certainly more improvements to be done, but our users will immediately experience a much better service that will improve much more rapidly going forward.  I'm extremely proud of our engineering team and the amazing result they've achieved in a relatively short amount of time.

Comments

[nick]

interesting you moved away from OpenLaszlo, as your app was fairly sluggish. It will be interesting to see the difference in performance.
We've been working on a big and complex openlaszlo app with lots of data, but we've ironed out any performance issues and it runs very quickly - like a desktop app does. You'll see Openlaszlo's webtop app runs very quickly too. I think there are a few tricks with Openlaszlo, something they'll help you out with for $, whereas the Flex community is large and Adobe probably makes it easier to code with best practices.

[Mike Potter]

I'd love to see the new Flex based interface. Is it available publicly yet?

Mike

Mike Potter
Adobe Flex Marketing Team

[grego]

Hi Mike, The new Flex UI is not available yet, but will be within a couple of weeks. Stay tuned.

[Ephraim Cohen]

Will you be including some basic dashboard capabilites with charts and graphs in this release?

[Max Rudman]

I have similar experience with OpenLazslo and Flex. I had used OpenLaszlo for a past project, and while it worked I wasn't quite satisfied with it. I got introduced to Flex through my work building applications on Salesforce.com platform. And based on what I've seen so far I find it a much more polished platform. The UI model is richer, ActionScript supports strongly typed objects (ie: compile time type checking) and the entire framework promotes good design patterns like MVC. I am anxious to see the new release.

[grego]

Ephraim, this release will not include charts and graphs. Those should show up early next year. Flex has an excellent chart & graph component library that we can leverage.

[Ben Tremblay]

Interesting how this has become an OL discussion ... front of mind with a lotta folk!
It's been a sentimental favorite of mine for a long time ... that said, it'll be real good to see how your Flex implementation pans out; the folk at OL strike me as the sort who'll make best use of substantial feedback.

[Kelly A. Shaw]

I've just written a review of Cogheadin the context of Business Mashups. (http://businessmashup.blogspot.com/2007/11/coghead-lets-you-build-simple-web-apps.html) Unsurprisingly, I conclude that you are a good choice for quick development of applications in the long tail, but you aren't yet a platform to build business mashups. When you solve your performance issues, when you expose your REST services, and finally when you can call out to external services, then you will have a very special platform that can be used by non-technical 'business mashers' to create those pesky applications that never seem to get IT funding or attention.

Look out ActiveGrid.

Regards,

Kelly A. Shaw, Ph.D.
Analyst
Serena Software