HTTP caching can go a long way to help scale a web application - instead of dynamically serving every single request, we can use memcached, Squid, Varnish, or a multitude of other tools to respond in just a few milliseconds. However, as developers at Yahoo have noted, there is a common failure scenario that is [...]

6 conversations  •  4 comments  •  0 bookmarks  •  0 diggs  •  0 tweets

Performance testing can be a tricky proposition, how many times have you deployed code that fell apart in production? (Hey, we've all done it!) Having a staging environment, which replicates the data and deployment environments are great first steps, but even that often falls short from producing reliable results. And that is where load testing [...]

15 comments  •  9 conversations  •  0 bookmarks  •  0 diggs  •  0 tweets

DNS is a great example of a service that couldn't possibly work on paper, but performs spectacularly in practice, even with a hodge-podge of implementations all over the Internet. First, the authoritative DNS servers responds with a Time To Live (TTL) timestamp for every record (which you should keep fairly low in a virtual deployment [...]

2 comments  •  1 conversation  •  0 bookmarks  •  0 diggs  •  0 tweets

These past few weeks have brought forward a few interesting developments in the Ruby database access layer. First, the guys at espace announced Neverblock: a Ruby 1.9 library which makes heavy use Fibers and non-blocking IO (RubyInside and InfoQ have the scoop). And shortly thereafter we saw the announcement of MySQLPlus (also a direct result [...]

9 comments  •  2 conversations  •  0 bookmarks  •  0 diggs  •  0 tweets

It's not the choice of language that determines the success of a project, but the people behind it. Can a Rails app serve 3 million dynamic pageviews a day? Absolutely, causes.com is doing it on a daily basis. After stumbling across several great open-source projects by Keith Rarick (beanstalkd, curl-multi, and others), I've reached out [...]

4 comments  •  1 conversation  •  0 bookmarks  •  0 diggs  •  0 tweets

You hit a web 2.0 nerve and that magical 'viral coefficient' is working with full force, except now you' have a problem: scaling the database. No problem, you say, we have a few tricks up our sleeve: faster disks, loads of memory, dataset sharding, load balancing, connection pooling, master-master and master-slave replication schemes, multiple caching [...]

12 comments  •  7 conversations  •  0 bookmarks  •  0 diggs  •  0 tweets

It's unit tested, the integration and regressions tests are all green, you even added a set of performance benchmarks, and yet after a couple of hours of production use the process is falling over - we've all been there before. Good instrumentation in these situations is worth its weight in gold. If you're lucky, you [...]

3 conversations  •  2 comments  •  0 bookmarks  •  0 diggs  •  0 tweets

That’s a nice one! Wonder if I should use Ruby, Python, or maybe even Perl to build it? As gearheads we tend to jump to the technical details with little afterthought, but should you even be building that feature in the first place? How do you make that call? Whether you're a one man team with [...]

12 comments  •  8 bookmarks  •  2 conversations  •  0 diggs  •  0 tweets

Every language has its idiosyncrasies and Ruby is no different, hence, below is a collection of handy optimizations to keep in mind next time you're working on a Ruby project. Majority of them appear to fall into the syntactic sugar category, but it's important to understand the motivation behind each, as there is usually [...]

119 bookmarks  •  26 comments  •  17 conversations  •  0 diggs  •  0 tweets

Managing log files is like herding cats, except its worse: a typical LAMP / Rails stack will easily generate a dozen logs in different locations. The fun part is, of course, the debugging and server administration tasks which involve the forensic task of tracking down each of these files and then cross-referencing timestamps to [...]

51 bookmarks  •  15 comments  •  8 conversations  •  0 diggs  •  0 tweets

Over the course of the past year or so, I got myself into a habit of using Tumblr for storing insightful snippets and quotes from my daily endeavours on the internet. Usually, this entails saving a paragraph (too long for tweet, and not enough for del.icio.us) so I can revisit it later. However, having Tumblr [...]

15 bookmarks  •  7 conversations  •  4 comments  •  0 diggs  •  0 tweets

Wow, this was a nice surprise last night - a Ruby Hero award at RailsConf '08! The group of guys I got to share the stage with was unbelievable: Ryan Bates, Evan Weaver, Yehuda Katz, plus two more winners who couldn't make it to RailsConf. It's amazing to see the community support, and [...]

3 comments  •  1 bookmark  •  0 diggs  •  0 conversations  •  0 tweets

Ruby EventMachine is a framework, which depending on who you talk to, either yields a lot of excitement (Evented Mongrel, Analogger, Evented Starling, etc.) or its fair share of criticism. In part, the FUD is due to the mismatch of the language used, and the underlying implementation - namely, the Reactor pattern. The reactor design [...]

137 bookmarks  •  14 comments  •  12 conversations  •  0 diggs  •  0 tweets

A brand new Rails/Merb app you put together over a weekend, a pack of Mongrels, a reverse proxy (like Nginx), and you're up and running. Well, almost, what about that one request that tends to run forever, often forcing the user to double check their internet connection? Response time is king, and you always want [...]

120 bookmarks  •  12 comments  •  6 conversations  •  0 diggs  •  0 tweets

Brian Aker and Alan Kasindorf gave a great talk on Memcached - every developer's favorite caching layer - at MySQL User Conference earlier last week. The slides are available online, and they're well worth a bookmark. Brian and Alan promised to continue updating the deck as they hone out their presentation at OSCON and other [...]

359 bookmarks  •  14 conversations  •  8 comments  •  1 diggs  •  0 tweets

I can't help myself, I seem to spend inordinate amounts of my time working inside a remote SSH tunnel on a daily basis. And nothing is worse than trying to decipher the default dark-blue/green/red colors against a black background in PuTTY. Coming from SecureCRT, which has much better defaults, I set out on a hunt [...]

94 bookmarks  •  7 comments  •  4 conversations  •  1 tweet  •  0 diggs

You've scored yourself an amazing team, you've all wholesale committed to some form of agile development model (scrum, crystal, fdd, etc.) and now you're in the thick of it - with a release coming up - and the question is: how do you test, what do you test, when do you stop? The underlying [...]

19 bookmarks  •  7 comments  •  5 conversations  •  0 diggs  •  0 tweets

Unit::Test, RSpec, and good old manual testing are all a must before every deployment. However, testing is a tricky trade as no matter what your code coverage is, lack of failures does not mean lack of bugs - that is, unless you're really into formal verification. Not to mention, unless you have a true [...]

114 bookmarks  •  10 comments  •  9 conversations  •  2 diggs  •  0 tweets

If web architectures, performance, or scalability are topics you would like to keep on top of (who doesn't!), then chances are, you've heard of Nginx ("engine x"). Originally developed by Igor Sysoev for rambler.ru (second largest Russian web-site), it is a high-performance HTTP server / reverse proxy known for its stability, performance, and ease of [...]

449 bookmarks  •  27 comments  •  8 diggs  •  8 conversations  •  0 tweets

The spec says 'transactions', we immediately think 'InnoDB'. But what if you're not using a database, working on a light-weight application, or simply want to ensure a graceful recovery process? In the past that usually meant layers of additional logic, variables to keep and capture intermediate state, and all the other associated headaches. That is, [...]

72 bookmarks  •  13 comments  •  5 conversations  •  1 diggs  •  0 tweets