Yahoo! Messenger Protocol Tutorial - Part 1

[tansqrx 0]

One of the security passions that I have maintained over the past few years is the one with Yahoo! Messenger. In recent months Yahoo! Messenger has seen a decline in users due to some new policies. Although not as strong as perhaps a year ago, it is still very important to keep a watch on Yahoo! Messenger from a security point of view. Messenger, just like may of the programs we use, open a door out to the Internet. With each new door comes a unique set of security concerns. Perhaps the biggest reason to keep an eye on Yahoo! Messenger is because the user base is so large. What if you can exploit Messenger, how many million users would be affected?

The upcoming tutorial has been a work of mine for several years. It covers the basics behind the Yahoo! Messenger protocol. The protocol is closed source so you will for the most part not find the details on the web and certainly not from Yahoo! If you ever wanted to know what happens when you login to a Yahoo! server, the password hashes used, or how an IM works then this is the place for you.

The tutorial is long, well over 60 pages in Word. I will try and keep it all in the same thread to prevent spamming by myself. I will break the tutorial into several parts that will cover several months. After the years of hard work I donât think I will only get five days worth of credit out of this one. If the moderators feel that it should be put into larger sections so be it. Also if the moderators wish for me to do anything different please feel free to let me know.

PART Ia - Introduction

This section of my Yahoo! tutorial introduces you to the reason why I am studying Yahoo! and why I know/donât know what I do. Hopefully this will give you a small insight into my world of Yahoo!

My love/hate affair with Yahoo! Messenger started only less than two years ago. By that standard I shouldnât know that much about Yahoo! Messenger, as many friends I know have been using Yahoo! Messenger for many times that many years and still donât know the first thing about Yahoo! Messenger. For some odd reason, Yahoo! Messenger took hold over me and I had to know more about it.

I have used online IM services since the time AIM opened up to non-members which was who knows how many years ago. After a few short weeks of playing around with AIM the fancy of instant messaging soon lost its luster. The fact was that everyone on my buddy list was a close personal friend and if I wanted to talk to them then I would just call them on the phone. A second reason AIM soon lost its intrigue was that only a select few of my friends owned a computer and none of us even dreamed of DSL or cable connections.

Now fast forward a few years. I have since expanded my interest in computers and have even made a living at knowing the ins and outs of computers. I have learned several computer languages and work with computers constantly. Three almost separate events brought Yahoo! Messenger into my life. The first was a friend that told me to contact him on Yahoo! Messenger to arrange a game of Starcraft one night. This of course caused me to download Messenger and start playing around with it. The second event involved a run-in with a booter in one of Yahoo!âs chat rooms. After being booted several times I became irritated to say the least. This behavior really didnât surprise me or cause much alarm as I have used IRC for several years and was already familiar with some of the nastiness that other users can throw my way. As a side note, I was actually fairly amazed at how benign Yahoo! attacks were. On IRC where a userâs IP address is freely available, much nastier attacks take place, usually within ten seconds of signing into a server. Finally the third factor in my Yahoo! odyssey involved my passion for computer security. Over the years I have always been fascinated with computer security. This aspect of my life really took off when I started taking a series of computer security courses.

In the fall of 2004 I enrolled in a class entitled âInformation Assurance âI.â Information Assurance has to be one of my all time favorite classes that I have ever taken. The class was pretty much a how-to hack course; the basis of course was that you had to know how the enemy works. This combined with two of the best teachers that I have ever had, created an educational experience that will truly affect the rest of my life. One of the requirements for this class was to do a research project on any security aspect that I chose. The combination of playing around with Yahoo! Messenger for the previous few months, being booted, and having to write a paper about a security issue was the perfect spark to start my voyage into Yahoo! Messenger.

Much the information contained within this tutorial is my findings from my security class. In the âInformation Assurance âIâ class I wrote a paper entitled âYahoo Booters.â I then took the second part of the class (âInformation Assurance âIIâ) in the spring of 2005 and continued my Yahoo! research with a second paper not surprisingly titled âYahoo! Booters Part II.â The first paper was a survey of Yahoo! Messenger, instant messaging history, a brief look at the Yahoo! Messenger protocol, and some of the common booters. The second paper went much deeper, investigating if remote arbitrary code execution was possible through existing booter methods. The second paper required me to make my own Yahoo! Messenger client and integrated booter. After creating a client, I spent countless hours debugging Yahoo! Messenger with OlyDbg after a boot packet was sent to it. All of this resulted in a mapping of most of the common Yahoo! protocol packets and much valuable information.

One last question remains. Why am I going through all the trouble of publishing the information that I worked so hard to obtain? The first answer is I just suppose that I am a nice guy. The fact is that as hard as I tried I could not find any good information about the Yahoo! Messenger protocol on the web. The closest I came was a site that had a few packet captures from the login for version 11 [http://forums.xisto.com/no_longer_exists/ ] (the current version is 12.) Besides that I had little to nothing to work with. Of course Yahoo! itself will not publish any protocol information because it is a closed standard and Yahoo! uses Messenger to make money. I had to resort to firing up Ethereal and packet sniffing each packet as it traveled on the wire. If I can help anyone else out in the same situation, I will gladly give what I have learned.

The second reason that I am going through all this trouble is I still need information myself. As with any good research, the more rocks you turn over, the more questions you are left with. I have so many Yahoo! questions that I could keep a team of programmers busy for two years to find the solutions. The whole point of this website is to create an open exchange of ideas. I canât do it alone. Maybe if I provide enough information, someone with more knowledge than I will feel sorry for me and answer some of my questions.

Enough about me, lets get into some internet and instant messaging history.

http://www.ycoderscookbook.com/

Edited May 11, 2006 by tansqrx (see edit history)