New servers for WordPress.com

31 Jan

We are getting ready to place an order for an additional 37 servers in a new datacenter. This new point of presence will serve as the 3rd active node for WordPress.com. Over the past few weeks, I have been doing lots of testing and seemingly endless negotiation with various hosting companies.

Background

The model we have adopted is to use commodity hardware to serve all the functions of the site. We do not rely on SANs or super-expensive multi-processor systems. Our web servers are usually either single or dual processor machines with 1-2GB of RAM and a small, inexpensive hard drive. Our database servers are single or dual processor machines with 4-8GB of RAM and 2-4 fast SCSI drives in a RAID array using a hardware RAID controller. Because there is redundancy built into the architecture that several of these machines can fail at once at the site is unaffected, the individual machines do not need to be extremely robust. Historically, CPU time has been the most precious resource on the web servers and disk I/O on the DBs.

Requirements

  • Provide the hardware outlined above
  • Support Debian AMD64 or similar
  • Provide a Gigabit private backend network for inter-server communication
  • Ability to deploy additional servers quickly and painlessly
  • Sales and support team that is competent and easy to work with
  • US Datacenter and not Dallas or San Diego

Initial Impressions

Dedicated server providers seem to fall into 2 classes:

1) No-frills provider that offers server between $79 – $149/month. No phone support or advanced services. This would be fine, but they usually cannot provide the DB-class machines we need and do not provide Gigabit backend networks.

2) Full-service provider that offers servers beginning at $250/month and up. These providers are usually not the best fit for us because they justify their higher prices by saying they have superior support. We don’t really need “superior” support, just someone that can take care of hardware issues when needed. Seems like a waste to pay for something you are never going to use and don’t really need. Also, in my experience, the more expensive the hosting company, the more painful the sales process is. Sometimes I feel like I am buying a car….

The Finalists

Over the past month, we have narrowed the field from about 10 different possible providers down to the following 3. Each of these fit somewhere between no-frills and full-service as mentioned above, but I definitely get the feeling that they lean one way or the other.

  • ServePath
    Based in San Francisco, California (where Automattic is also based) they seem to lean towards the full-service side, offering 24×7 phone support, higher-end servers, load balancers, and firewalls. Their sales process has been pretty agonizing. It is now going on 45 days of back and forth, price changes, configuration changes, conference calls, and about 50 one-on-one calls with our sales guy. I had chance to visit their offices and tour their datacenter earlier this week so I got a feel of how things work there — it appears to be a well-run organization. One thing that struck me as a bit odd was that they do not deploy any rack-mounted servers in their datacenter. All of their servers, which are built in house, are in tower (white-box) chassis like you would see under an office desk. This is something I would expect in the lower-end market, but at $500+ per month and rather large RAID arrays (300GB SCSI x 6) I would expect to see more rack-mounted chassis to take advantage of the superior cooling.
  • Server Beach
    Based in San Antonio, Texas and now owned by Peer1, Server Beach is definitely more on the no-frills side. When we were looking at new datacenters about 6 months ago, Server Beach could not provide what we wanted — they did not offer SCSI RAID or Gigabit privatenet, but said it was coming. Well, those things are now available. Kudos to Server Beach on a super-simple and painless sales process. Once I contacted them with the configuration we wanted, they scheduled a conference call to discuss the details. They had members of their support, operations, and sales teams on the line. About 30 minutes later we were all done and a day later they had sent over a proposal. It was right the first time – had everything we asked for and at a very fair price.
  • Joyent/TextDrive
    Honestly, we probably will not go with TextDrive for this deployment, as it is a pretty radical departure from our current configuration, and they don’t technically meet the requirements laid out above. They are worth mentioning, however, because they are doing some pretty cool stuff with OpenSolaris, zones, and ZFS, and it sounds like it could be a good fit for our architecture model. Their storage is super-fast, the container model allows you to replicate existing containers with a single command — there is no need to provision and setup a new physical server. ZFS offers all sorts of cool stuff like snapshots, compression, and built-in data consistency checks. Utilizing this architecture, would require that we maintain 2 completely separate environments — Linux and Solaris — and there is a definite time investment in doing so. I think that we will probably look at moving some of our services to Solaris containers in the near future, but I am not sure it will be WordPress.com.

The Verdict

First, let me say that we still have yet to make the final decision, but hope to do so by the end of the week. Both ServePath and Server Beach seem like they will be great companies to work with. ServePath is local, so we can walk over to their offices and meet with them if needed – there is something to be said for working with local vendors. Server Beach has been a pleasure to work with thus far and I have some experience working with them in the past. Peer1’s VP of Marketing also blogs on WordPress.com. Pricing and server configurations are almost identical, so that really isn’t as much of a factor as one would think.

Anyone have experience working with either of these companies? Suggestions? Feedback?

25 Responses to “New servers for WordPress.com”

  1. James Byers January 31, 2007 at 7:21 pm #

    ServerBeach is a generally good provider of low-end boxes – maybe higher-end too, though I don’t have experience on that side. We ultimately left ServerBeach for SoftLayer (a fantastic company, unfortunately in Dallas) because of the lack of a backend network.

    I always felt like ServerBeach had our interests at heart when handling hardware and support issues. That’s a big plus. We did have a variety of hardware problems, but nothing unexpected, and they were handled quickly. No network problems to speak of.

    Aside from the private network issue, my only complaint with ServerBeach was the lack of management tools. At the time I used them, there were no bandwidth graphs, IPMI / KVM, etc., tools for tracking and managing the status of servers. They had recently added remote power, so that helped somewhat.

    The management side may not be too much of an issue, I’m sure you’ve got plenty of your own tools there. But the features of ServerBeach in that arena are bare in comparison to a company like SoftLayer, who has gigabit private networking, fantastic IMPI and remote console support, SSL and PPTP VPN access, bandwidth usage information, everything you can imagine.

    On the whitebox vs. rack-mounted server question, ServerBeach I believe had the same practice for low-end servers. I’d be curious to know if this is still the case.

  2. Angela Ramirez February 1, 2007 at 12:16 am #

    In regards to James Byers’ comments, ServerBeach does offer gigabit private net and private racks.
    It’s a tough decision to make Barry. When you think of some of the customers ServerBeach hosts such as tailrank.com, youtube.com, dogster.com, mybloglog.com, clearspring.com, among others, you can be sure that ServerBeach will be able grow with wordpress in terms of hardware and network scalability while providing a personal experience. This is what ServerBeach does.

  3. James Byers February 1, 2007 at 12:38 am #

    I should have noted that my comments reflect the state of ServerBeach around the fall of 2006, when private networking was not generally available.

    In looking today at a server hosted at ServerBeach, I think my comments about management tools still hold true.

    To Angela’s point about the personal experience, I had nothing but positive interactions with ServerBeach staff – sales, support, in the forums, etc. That’s huge.

  4. george donnelly February 1, 2007 at 2:33 am #

    Ack! Servepath is horrible mess. Stay away from them. I did business with them for 5 years, ramped up to 20 servers and quickly had to move that business elsewhere due to problems with power, network, really dumb sales and customer support – you name it.

    Servepath has all kinds of problems. Check out webhostingtalk.com for more information.

  5. dan February 1, 2007 at 10:02 pm #

    Judging from your observations of the three companies, I’d say Server Beach is the way to go. I don’t think the proximity issue is all that big of a deal. Using a company that’s far away but responsive is far better than one that’s close but takes a long time to get things right.

  6. Charlie February 2, 2007 at 12:32 am #

    Charlie from Clearspring here.. Angela (from previous comment) is our Server Beach account manager. She’s been very helpful in accommodating all of our special config requests and giving us fast turnaround on new server deployments. Def recommended.

  7. eas February 2, 2007 at 6:57 am #

    Used serverbeach to host Harvey Danger’s online album release a year an a quarter ago. I was happy with the experience. They did stick us with a 10Mbps connection at first, even though we’d paid for 100, but they were very response about fixing it so we could rise to the big traffic spike we were about to get (yay lighttpd). We just had one box though, but it worked nicely for us.

    One thing though, I think they too rely on a lot of desktop style boxes on wire shelves (or they did in fall of 2005). Probably a sensible low cost strategy. Generic cases are cheaper than rackmount cases, and the lower system density that results probably lets them use older (and presumably cheaper) datacenters that can’t meet the power and cooling density needed by modern blade servers and 1U boxes with multiple multi-core chips.

  8. evariste February 2, 2007 at 8:12 am #

    For the ignorant, what’s wrong with Dallas and San Diego?

  9. barry February 2, 2007 at 8:14 am #

    We already have servers in both of those locations.

  10. evariste February 2, 2007 at 9:24 am #

    Oh. Now I get it. Thanks!

  11. teo February 2, 2007 at 10:25 am #

    The big problem for replicated DC is on database replication.

    How do you handle that? AFAIR, MySQL was not the best player on bidirectional replication.

  12. barry February 3, 2007 at 2:48 am #

    We are currently using Master-Slave replication so it’s not that big of a deal.

  13. Kevin Burton February 3, 2007 at 7:02 am #

    ServerBeach has been doing a great job for Tailrank….

    We’re going to be migrating to a dedicated rack soon no gigabit and on Debian 64bit which will be a big win for us.

    I don’t really care where our machines are hosted. Putting them in San Francisco is sort of pointless. I don’t want to go to the data center anyway and we have enough machines that I can try to do custom compiles if I want.

    Of course I’d never be able to get into the bios but we dont want to run raid controllers anyway.

  14. Kevin Burton February 3, 2007 at 7:02 am #

    BTW…. ping me if you want $250 off ServerBeach :)

  15. Jason D- February 9, 2007 at 6:13 pm #

    Did you guys make a decision yet? Just curious since we’re moving our entire operation to ServerBeach in a few weeks to their new Private Net service in San Antonio. They gave us a nice price and I’ve had nothing but great experiences working with them in the past.

  16. barry February 9, 2007 at 9:54 pm #

    Jason,

    In the next few weeks we are going to be deploying a new Akismet cluster with Server Beach in San Antonio and a WordPress.com cluster in San Francisco with ServePath. Both companies seem like they are good matches for what we are looking for, and as that is somewhat rare in the hosting market, we went with both.

  17. Alex April 25, 2007 at 11:07 am #

    Thank You

Trackbacks/Pingbacks

  1. New Servers for WordPress.com « Barry on Wordpress « BIG marketing for small business - January 31, 2007

    [...] Servers for WordPress.com « Barry on Wordpress Jump to Comments I read Barry’s blog this morning with great pleasure to see that ServerBeach (who I work for, along with our parent [...]

  2. Blog » Server-Aufrüstung bei Wordpress.com » Frank Helmschrott - January 31, 2007

    [...] weist auf einen interessanten Artikel von Barry hin (beide Automattic, betreiber von WordPress.com). Es geht um die Anschaffung von 37 neuen [...]

  3. tecosystems » links for 2007-02-01 - February 1, 2007

    [...] New servers for WordPress.com « Barry on WordPress lots of interesting tidbits in here; good read on what scaling means from a hosting perspective (tags: hosting ServerBeach ServePath Joyent TextDrive Debian Solaris ZFS) [...]

  4. People Over Process » Blog Archive » links for 2007-02-02 - February 2, 2007

    [...] New servers for WordPress.com « Barry on WordPress (tags: via:sogrady wordpress hosted hosting saas punkit zfs solaris sf) [...]

  5. Dogster Inc. Company Blog » Archive » Must Read for Anyone Renting Servers - February 3, 2007

    [...] If any of this is vaguely interest to you, give it a read. [...]

  6. WordPress Wednesday: Custom Fields Contest and Lots of WordPress.com News at The Blog Herald - February 7, 2007

    [...] Servers for WordPress.com: Barry shares information on new servers for WordPress.com and how the servers are setup to handle all the WordPress.com [...]

  7. Know About Linux Dedicated Hosting 79? New servers…. » Host News . biz - August 18, 2007

    [...] servers for WordPress.com New servers for WordPress.com We are getting ready to place an order for an additional 37 servers in a new datacenter. This new [...]

  8. New Servers for WordPress.com « Barry on Wordpress | Big Marketing For Small Business - November 25, 2009

    [...] read Barry’s blog this morning with great pleasure to see that ServerBeach (who I work for, along with our parent [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 856 other followers

%d bloggers like this: