Somewhere between a great idea and universal acceptance of a website, there’s a lot of scary ground to cover – and few things hurt worse than your service being dead-in-the-water. But for some reason, no one really cares when Twitter is out. Let me state upfront I’ve worked in some serious high-load web infrastructures (I was the UI and business lead for UPS Tracking, which averaged more than 22 million daily sessions in 2009), so I know running a high-volume site isn’t a walk in the park. But here’s my question: Why can’t Twitter seem to get its act together? Think of all the other bandwidth- and storage-intensive sites on the web (Facebook, Amazon, YouTube, UPS, Travelocity, etc.) and you never see them down. It all boils down to one simple answer: no one loses actual dollars when the Fail Whale rears its ugly head. The good news for Twitter is they’re not mission-critical – it’s a hobby for people. But that’s also bad news for their business. Eventually that’s going to change – and then the outages are going to have serious stings to them.
Let’s look at some rough stats. Depending who you believe, Twitter averages somewhere between 200 tweets per second (a Robert Scoble number from last month’s Twitter conference) and 750 tweets per second (the official Twitter number posted on their blog). I’d guess it’s about 500, which makes for a nice whale-less Twitter experience. But fast-forward to a huge special event, and that traffic absolutely explodes. According to Twitter, Game 7 of the NBA Finals when the Lakers beat the Celtics drove 3,085 tweets per second, and just days later the Japan-Denmark World Cup game drove 3,283 tweets per second. So clearly sporting events are a major traffic driver. But are we to believe that Twitter hasn’t done capacity planning for 5x daily traffic? Really? Anyone worth their salt has an instant-on capacity plan for at least 10x traffic.
And shouldn’t an outage matter more than it does? While Twitter is a hobby – and before they figure out a sustainable monetization strategy that no doubt is data-related – who cares if the Whale appears? I’ll give you a real-life example I’ve seen that disturbs me. I prefer SimplyTweet as my iPhone Twitter client for a couple very specific product feature reasons: @ and DM notifications, no gaps in my timeline and a super clean UI. What I hate about SimplyTweet is it works about 87.632% of the time (an approximation). The rest of the time it whines about bad gateways and API problems. But for some reason, I can jump over the official Twitter iPhone client and everything seems fine. I know managing APIs can be just as tricky (if not more) than managing site traffic, but seeing this scenario at least three times a week makes me think Twitter is throttling external apps and/or playing other kinds of API games with non-official tools.
So the real question is can Twitter survive as an independent company? Most onlookers would agree a huge part of the Google-YouTube deal was all about keeping YouTube from hitting a capacity-induced brick wall at 200 mph. And perhaps most smartly, the Google guys simply brought infrastructure muscle and left the YouTube brand to continue on its journey unscathed. With more than 24 hours of video uploaded every minute, the storage and bandwidth pressures were literally chewing up any shot at profitability. Is this the destiny of Twitter? Will it be so successful in the hearts and minds of consumers that it can’t exist by itself? Are they so good it’s going to kill them?
If there’s any chance of the answer being no, they better jump to it quickly. The deeper they get into the business aspects of the company, the more pain outages will create. Not that sponsored tweets seemed to matter to anyone, but you can bet paid advertisers aren’t going to have patience for dead impressions. And I’d contend there’s a point of consumer revolt on the horizon as well (I think of a couple journalist buddies who lose a major communications tool when Twitter goes flatline).
So here’s my theory on how Twitter can become the beast it needs to be: lock-up a strategic partnership of massive scale with Amazon before you have to approach them to buy you. Think of it this way: Amazon has a massive cloud-based infrastructure that could almost instantly alleviate Twitter’s capacity issues, and from Amazon’s perspective, they’d be one hell of a reference client for the CloudFront kids. In addition, the Zappos acquisition sets a clear precedence that Amazon can do a smart deal and leave the brand to operate independently – as Twitter would require to maintain its cult-brand status. There’s also a super-smart future play if you consider the impact Twitter could end up having on trend data for e-commerce SKUs. Imagine a world where Amazon could overlay brand mentions on Twitter with their own sales data – and be able to model future sales trajectories.
But even without an outside deal, Twitter has to make moves. Granted I have no idea how Twitter’s systems are architected (and frankly it’d be one of the best business stories ever if they built a fully scalable infrastructure from Day One), but I can’t imagine there aren’t 50 or 100 ninjas in the country that could drive their uptime percentage above 98%. My advice is stop Hollywood-ing – Time Magazine has already deemed you as cool. Focus all your money and talent on system uptime, or risk being the Milli Vanilli of the technology industry. Either give us a service that works, or that whale is going to be the death of you. Or you could just blame it on the rain…