30 Apr 2009

Growing Pains

You guys have helped IntenseDebate grow a huge amount over the past few months. Overall this is great! We love helping people take the conversation on their blog / website to the next level, and of course once you get IntenseDebate going you know you grow even faster.  It has put quite a strain on our systems though.  I’m sure you’re all aware of our outages over the last few weeks.

We’ve been working non-stop to address these issues. Our highest priority is improving the performance and long term scalability of IntenseDebate for you.

Now we could leave it at that, be completely opaque about what changes we’re doing and just hope that you’ll trust that we’re working hard on the problem, but you all deserve more and need to trust that our service can perform for you.  So here’s a behind-the-scenes peek for those of you interested in how we’re tackling this problem.

The first major change we’ve been working on is sharding some of our larger and most frequently used tables into a more efficient and scalable database schema.  For those of you not familiar with sharding it means storing all your data in several smaller tables instead of one large monolithic table.  This allows us to get your data much more quickly and makes for a much more horizontally scalable system. (We can continue to add more tables to keep them small as our total data storage increases.)  This is complete for our most heavily used data, comments, and will continue where appropriate.

The second change is creating more summary tables to simplify the retrieval of common data.  Even after sharding some of the data is still too slow to compute on-the-fly.  Summary tables cache frequently used computations which will allow us to get this information almost instantly.

Last, but not least, we’re optimizing our logging and background processing.  We do quite a bit of logging to ensure things are running  smoothly and to help debug quickly when things don’t.  In particular the syncing process to and from WordPress blogs has a lot of activity we store for troubleshooting.  We’ve done some work to make this logging faster and less system heavy, as well as some fine tuning of what and when we log in order to help ensure that these troubleshooting tools don’t have a negative impact on the performance of the core service.

I want to reiterate we’re acutely aware of performance over the last few weeks and there is no excuse for poor service or performance.  We’ve fixed the most pressing issues, but we’re taking a number of proactive steps to ensure that we can maintain the level of service that you deserve from us.  Thank you all for your patience and understanding.

Posted by Jon Fox in News


Missing out on the latest IntenseDebate developments? Enter your email below to receive future announcements direct to your inbox. An email confirmation will be sent before your subscription is activated - please check your spam folder if you don't receive this.

Join 8,670 other subscribers

13 Comments »

  1. Thanks, that was "Intense" 😀

    Comment by hadri — April 30, 2009 @ 7:27 am
  2. So, having checked this out, you will note I am of the younger mind, try new things, and somewhat giddy tech generation, it has been a whirlwind since the first Radio Shack Tandy Computer back in 1982! So I have survived pretty much by ability to read manuals in complete English language. A possible thumbs up for me would be have some posting, somewhere with a little more "detail" for the old girl, hey? Not a lot, a few succinct sentences with a graphic or two? So far I personally know only a few feathered friends that twitter-but understand the tech concept, etc.

    Comment by Nanticokesun — May 1, 2009 @ 7:37 pm
  3. On the contrary Fox man, i have actually noticed that performance has improved over the past month to the quantum effect. Or perhaps that's just my superman web optimization skills at play. Yes, doubtless the latter. Indeed Foxy, there have been times when Intense debate really slows down the turtle of my site to a hare of pubis but nonetheless it's a stellar little system that could. I'm loving the enhanced optimization and double helix cross hybridized horizontally scaling that will be occurring in the near future. Speed is good but blow is better. Yet you are yet to resolve the biggest quandary of all; that which I have been waiting for although I am pondering whether I need Big Brother any more…where are my technicolor astronaut diapers and where is Facebook Connect??? 'I've been waiting all my life' as the cheesy romantic love song goes. But I love you guys. Keep rolling out the plugins. 😉

    Comment by Alex Cocaine — May 3, 2009 @ 6:52 am
  4. I wrote some other "growing pains" that you have already probably noticed… with the piont system:

    http://churchcrunch.com/2009/05/18/what-happens-w

    Comment by human3rror — May 18, 2009 @ 1:49 pm
  5. seslisohbet seslichat sesli sohbet sesli chat

    Comment by sesli — July 10, 2009 @ 4:19 am
  6. http:///speedywap.com – Go on there for free downloads

    http:///speedywap.com – Go on there for free downloads

    Comment by vishvadeep — July 26, 2009 @ 2:47 pm
  7. Enter text right here!
    http:///speedywap.com – Go on there for free downloads

    http:///speedywap.com – Go on there for free downloads

    Comment by vishvadeep — July 26, 2009 @ 2:50 pm
  8. I’m very interested in your article, and I suggest you to browse some online stores to find

    something different. Such as: http://www.hijordan.us/">Michael jordan shoes http://louis-vuitton-replica-handbags.com/"&… vuitton handbags http://www.bestmbtshoes.us/">Mbt shoes http://www.cheap-ed-hardy.com/">cheap ed hardy http://www.newghdstraightener.com">ghd straightener http://www.bestghdstraighteners.com">ghd hair straightners http://www.edhardyshirtsclub.com">ed hardy t shirt http://www.nfl-footballapparel.com">nfl apparel

    Comment by Louis vuitton — June 2, 2010 @ 2:19 am

RSS feed for comments on this post. TrackBack URI

Leave a comment

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. | IntenseDebate – Official Blog
%d