Barry on WordPress

Load balancer testing

Over the past week here at wordpress.com, we have been doing some experimentation with various software load balancers. In the latest test, we are using Pound with a weighted round-robin algorithm between all of the web servers in a given datacenter. The individual weight takes into account hardware differences and any other tasks that server may be doing. So, a dual processor server would receive more traffic than a single processor machine. The tests have been going really well so far.  Once the testing is complete and we have decided on a solution, I will post about the various things we tried, pros and cons of each, etc.  The graph below shows the load averages of the various web servers over a one week period. I’ll let you guess when the test began 🙂 Pretty dramatic difference, eh?

totals-web_load-week.png

Barry

, , ,

11 responses to “Load balancer testing”

  1. WordPress.com Load balancer testing « Avatar
    WordPress.com Load balancer testing «

    […] Barry (our sysadmin) has posted on the load balancer testing we’ve been doing to even out some bumps around here. […]

    Reply
  2. adam Avatar
    adam

    impressive, both for the drop, and for the rapidly increasing loads leading up to it.

    Reply
  3. Blade Avatar
    Blade

    It’s to bad other sites on the net will not do this from time to time.
    I like it when a site Admin takes great links to make shear that his or her site is running in top shape.
    I glade I’m on WordPress.

    Reply
  4. blaze Avatar
    blaze

    I really couldn’t work that stuff out if i tried. This is why I am glad I leave it to the experts 🙂

    Reply
  5. Vinny Avatar
    Vinny

    Any reason you’re not considering using hardware load balancers like F5 BIG-IP’s, besides the cost I guess? BIG-IP offer awesome load balancing and failover support in addition to data-center and geographic redundancy when paired with their 3DNS product.

    PS: I’m just a happy customer 🙂

    Reply
  6. barry Avatar
    barry

    F5s are great – I have used many of them in the past. The approach here is a little different though and I prefer to use commodity hardware both for the flexibility and the cost savings. Purchasing 2 F5s in a failover pair for a single data center may be a “little” expensive, but purchasing 10 sets for 10 data centers becomes *very* expensive. As the site grows and we look at setting up more points of presence around the world this becomes a big issue. Also, if you use such expensive specialized hardware in your configuration, you tend to become tied to having a few, large points of presence rather than many, smaller ones.

    Reply
  7. robbwindow Avatar
    robbwindow

    I still don’t feel any the wiser, but I shall perservere searching you guy’s are great I have faith in your progression, cause I just don’t have time these days.

    Reply
  8. Alex Avatar
    Alex

    Wondering if you’d finished your testing and had an update yet on what software load balancers were good or not so good please?

    I’m looking at using one on Amazon’s EC2 service.

    Thanks,
    Alex

    Reply
  9. barry Avatar
    barry

    We are currently using pound. It had all the functionality we needed and performance is ok (not great). We needed something that handled SSL, so our options were somewhat limited by that. If you don’t need SSL, you may want to look at perlbal.

    Reply
  10. Danny Howard Avatar
    Danny Howard

    You guys are still deciding? I have a similar agenda . . . glad to see you’re digging Pound . . .

    -danny

    Reply
  11. Load Balancer Update « Barry on WordPress Avatar
    Load Balancer Update « Barry on WordPress

    […] « Amazon AWS Outage Load Balancer Update April 28, 2008 A while back, I posted about some testing we were doing of various software load balancers for WordPress.com.  We chose […]

    Reply

Leave a comment

Blog at WordPress.com.