(Image via Chris Carraso @ChrisMCarrasco)

After the Node Summit 2013 conference, “bedtime story” can be added to the list of leading indicators for open source projects. At the two-day Node.js focused event, Walmart Sr. Architect Eran Hammer first described a suspenseful tale of how his team successfully deployed Node.js into production. Only weeks ahead of their busiest customer season, Walmart was furiously stress-testing their infrastructure, counting on Node.js for fronting their mobile e-commerce site. Despite surviving their last scheduled stress-test, the Node.js servers, intermediating between Java servers and the wilds of mobile browsers, continued to exhibit a slow memory leak that was threatening to require the Node.js team to reboot each server twice a day under anticipated Black Friday load. With even the small cluster of servers, that cadence would mean a long few days for the Node.js team. Because the leak was so small and carefully isolated by Eran’s team, it took months of convincing others and increasingly microscopic test cases to even classify the behavior as a leak in Node.js Core. An earlier Node Summit talk by Joyent SVP Engineering Bryan Cantrill walked through some of the available tools used (more on that talk in a later blog post).

What was the most nail-biting was that just before the scheduled test, Joyent Engineer and Node.js Core contributor TJ Fontaine had identified and eradicated the 4-byte memory leak with a delightfully succinct one-line fix, but the fix hadn’t made it into the testing system before the test window was closed. Still, the triumph was practical and psychological to Eran. Unfortunately production would have to proceed without stress-testing the updated code. On stage, Eran humorously noted that he was stressed out and breaking out in the days before Black Friday.

Turns out Eran was bored that entire long weekend. Not a single server needed rebooting. Eran spent his now idle time documenting with graphs and commentary using the Twitter hashtag #nodebf for Node Black Friday. Eran flashed up before and after graphs of memory consumption to show the absolute flatness of the resident set size (a critical measure of memory consumption) after TJ’s fix.

About that bedtime story? As a thank you gesture to TJ, Eran invited TJ on stage and cryptically asked him to lie down on the white couch. TJ gamely reclined and Eran brought out a rather thick looking book. He read from it. Turns out it was Eran’s new illustrated book “The Leek Seed”, a beautiful parody of the children’s book celebrating persistence, “The Carrot Seed”. If not for the audience’s laughter, the gentle reading would have likely lulled TJ to sleep. It seems like Node.js can now live happily ever after.