This is probably going to make me look dumb, but that’s never stopped me from posting before. Was reading another article this morning about how Twitter’s been down *again*, which got me to thinking about other options, then to think more about the technical hurdles facing Twitter. One thing I’ve read a few places is that Twitter uses XMPP (Jabber’s protocol) for pushing data around. I’m over simplifying here, partially because that’s about as much as I know 
I was able to get Twitter to send me an XMPP subscription to their full feed, and I set up a jabber server to listen and collect the data for awhile. Interesting concept, and perhaps that’s why they did it that way – to allow for a truer ‘publishing’ model. However, it seems that the overhead of that process might be causing them problems.
Am I too naive in thinking that straight polling would be easier, more scalable and perhaps more efficient? Publishing involves having to keep track of listeners and sending information (writing) multiple times. A straight polling approach would convert the app to more of a ‘read’-heavy app – something which most databases are very good at. Why is XMPP in the mix at all at Twitter?
Again, yes, there’s probably a perfectly good technical rationale for this, but since the technical infrastructure has seemd to remain largely a secret (yes we know they use Ruby on Rails and that couldn’t possibly have any bearing on their scaling issues because they’ve solved that scaling issue over and over…) I’m just guessing here. Can someone more informed than me shed more light on this? Thanks.
I'm currently working on a book for web freelancers, covering everything you need to know to get started or just get better. Want to stay updated? Sign up for my mailing list to get updates when the book is ready to be released!
