Velocity: The Ultimate Web Performance and Operations Conference

Last month, I attended O’Reilly’s Velocity Web Performance and Operations Conference and, without fail, the information I came away with reinforced existing VMIX initiatives and strategies, and inspired new ones.

Attending the Velocity Conference, I had three goals:

  • To listen to relevant industry-leading speakers, regarding how to improve performance and operational efficiency, taking this information and disseminating it to my colleagues who couldn’t attend.
  • To meet and discuss with peers from companies facing the same challenges and hurdles when it comes to performance and operations. Understanding their approaches and pitfalls while offering my own experiences.
  • And, finally, being away from interruptions at work, to simply be inspired by the incredibly smart people around me to draft new strategies for increasing scale, efficiency and performance to bring back and implement.

It’s All About the Data

At Velocity this year, as well as the previous two, I became aware of underlying themes that went beyond specific technologies or tools.

The most important, which in some cases may seem obvious, is to make decisions based on data.

A lot of times, decisions are made based on theory, instinct, wishful thinking and other motivating factors.

Validating these factors with data collected from your applications, infrastructure and clients will allow you to make the most informed decision. The more data points you collect, the higher the odds are that your decisions will result in the kind of outcomes you think they should.

One important method of data collection that is overlooked at times is the human factor. No matter how much our hardware computing evolves, humans still decide which methods or strategies to implement in order to provide a complete solution. And the human brain is still the highest performing computer on the market. Going to a conference and interacting with peers and colleagues on operations and web performance allows us to complete the system.

Learning and Implementing

As a result of attending the Velocity Conferences over the past three years, we have been able to implement the following technologies and optimizations, which have greatly impacted our efficiency at VMIX.

Velocity 2008

I met Brian Moon and Alan Kasindorf (dormando) and, after discussing file-system optimization, proceeded to implement some file-system tuning which, in turn, helped increase our Apache Web Servers running mod_disk_cache efficiency by 3 times.

Velocity 2009

After several BOF (Birds of a Feather) sessions, and speaking with various attendees, I was convinced to give Puppet (Configuration Management Framework) a try, along with Cobbler for rapid Linux installs. We’d previously used CFengine and Standalone Kickstart and felt they were not necessarily the right tools for our operation. After proof of concept (POC), we deployed Puppet and Cobbler and are now able to build and control systems at a quicker rate, with more management and less complication.

Velocity 2010

I attended a talk by Leif Hedstrom on Apache Traffic Server and realized it was open sourced from Yahoo to the Apache Organization. I had known about the Traffic Server back in the Inktomi days (1990’s), as well as from a company I worked for that licensed the code. However, I hadn’t realized it was openly available. After hearing that Yahoo uses the Traffic Server for their own CDN network, I decided to give it a shot and plan on using it to possibly replacing our Squid servers at VMIX.

Currently, we are running a bit of a bake-off between Squid and Apache Traffic Server, and preliminary results for Traffic Server are so favorable that, by next month, Squid might be calamari.

In quick closing, hopefully O’Reilly and the Velocity Program Committee keeps up the good work and my list of things learned and implemented grows!

Phil Chen is Director of Systems Engineering at VMIX.

8 Responses to “Velocity: The Ultimate Web Performance and Operations Conference”

  1. everyz says:

    Have you tried using Varnish instead of Squid? Facebook successfully uses it in their infrastructure.

  2. Phil Chen says:

    I have read about Varnish but haven’t taken it out for a test run yet. Thanks for the heads up, I will let you know what I think when I do give it a whirl.

  3. Greg says:

    Velocity was fun this year. I enjoyed the talks that I was able to attend. John Allspaw’s was good, for example.

    If you’re interested in a free java performance tool, my company (which was a sponsor) offers it here: http://www.appdynamics.com/free

  4. Phil Chen says:

    Hi Greg, it was definitely fun. John Allspaw did give a good talk, I am in the middle of reading his new book now. I will check out your app, and give it a whirl when a get a chance. Thx.

  5. Leif Hedstrom says:

    So, did you end up giving Traffic Server a test drive? Curious to hear experiences with Squid vs Traffic Server vs Varnish.

  6. Phil Chen says:

    Hi Leif,

    Yes I did give Traffic Server a test drive, and have since replaced half my Squid servers with it. The connection handling of Traffic Server is pretty amazing, along with its caching abilities. I haven’t tested it against Varnish just yet since I needed a solution quickly and the performance increase was so large. I plan on writing a complete blog post on the implementation, use case, and performance differences I saw between Squid and Traffic Server when I get a chance. I will give you a heads up when I finish it.

  7. Excellent! We’d love to hear about your experiences on users@ and/or dev@ , the ATS mailing lists. Also, if you have a chance, play around with the latest “dev” releases, v2.1.2 was just released, and v2.1.3 should be only a few weeks out (I hope). This will become ATS v2.2.0 later this fall, and it has some very significant improvements in performance and features over the v2.0.x releases.

    Shameless plug: More information about Apache Traffic Server is available on http://trafficserver.apache.org/.

  8. Nimitz says:

    This test compares “Varnish” and “Apache Traffic Server”:

    http://nbonvin.wordpress.com/2011/03/24/serving-small-static-files-which-server-to-use/

    Performances, memory and CPU usage are charted.

Leave a Reply