Yahoo! Messenger Challenge Response Algorithm

[tansqrx 0]

Here is a question that came into my forum and I thought it needed wider coverage.

 

Q: Can you explain the Yahoo! Messenger challenge response algorithm?

 

The Yahoo! Messenger challenge response sequence is quite complex and unique to Yahoo! The challenge comes from the server and is then run through an algorithm on the client. When looking at the challenge and response in ASCII view it almost looks like a mathematical equation but it is not.

 

This complex algorithm came from several years ago when the username and password was sent in plain text over the network and was eventually exploited. Basic encryption such as MD5 was then added. This is when things got interesting and politics stepped in. In 2004 Yahoo! was having a battle with several third-party applications such as Trillian as to if they could make their own client and join the Yahoo! Messenger network (http://forums.xisto.com/no_longer_exists/). Messenger has an ad driven revenue model so Yahoo! did not appreciate having an unofficial client not displaying ads. The solution from Yahoo! was to implement an outrageous and very complex authentication algorithm that the other companies could not reverse engineer. As anyone with a third person view could have predicted, the new monster algorithm did hold off Trillian for a while but was eventually cracked and the code was leaked to the Net.

 

Several years later the authentication code is not a huge secret but Yahoo! sill uses this beast to authenticate their users. I have never been able to find the original leaked code but it does live on in Pidgin which is an open source multi-platform client. To get a look at the code go to the Pidgin website (pidgin.im) and download the source for the latest build (https://sourceforge.net/projects/pidgin/files/Pidgin/). Pidgin is written in c/c++ so it can be hard to read for someone not familiar with c. The code is also very integrated with the Pidgin base so it is next to impossible separate it out without having to rewrite the entire code base.

 

I have looked at the code and studied it for many hours and have come to the conclusion that it is overly complex and a nightmare to decipher. If Yahoo! wanted to make an algorithm that is hard to reverse engineer then this is a successful effort on their part. The downside is that a person would have to spend an insane amount of time to write their own representation of the code. The algorithm is a custom hash that has no direct relation to any common hash or encryption function. Parts of the code resemble MD5 while other parts look like DES. The majority of the code is based in lookup tables which is a common encryption technique.

 

A few years ago I wanted to make my own implementation in .NET because the DLL that YCC Trainer uses has been marked as a âvirusâ by most of the antivirus companies. (The DLL is not a virus but it appears that is has been marked that way because it is commonly distributed with booters. Most booters are also not viruses but in the infinite wisdom of the antivirus companies, we should be protected from ourselves but this is a different article all together.) After spending about a week trying to get the basics to work I realized that I hadnât even scratched the surface and gave up. I still use the shady booter DLL that I found many years ago.

 

In the end I donât want to discourage you from looking at the code for yourself but this is one fight that I decided not to take. It is ugly, nasty, and complex to the point of being a coding nightmare. If you do decide to look at the code I would love for you to post your findings, especially if you make another implementation. For now I am happy with my shady virus laden DLL that I once found in the far corners or the Internet.

[turbopowerdmaxsteel 0]

I remember having the same problem with YMSG12Encrypt.dll which was being flagged as a Trojan by Antiviruses. It would be much better if you could include the algorithm's code directly into your program.

[boyboy400 0]

Hi tansqrx,Thanks for your complete description about yahoo challenge response algorithm. But what do you think about the sites that hack yahoo mail. I have some firm evidences that they do it really and it's not cheating. And I'm partially sure that they do something in packet layer and the crack the gained challenge response offline. Of course maybe I'm wrong but what's your opinion about them? Thanks

[tansqrx 0]

I have to admit that I am not very familiar with hacking Yahoo! mail but I will give this a shot. About two years ago Yahoo! relied on a JavaScript to generate a MD5 hash of your password and then the browser then sent that to the mail servers. In that case it may have been possible to reverse the hash using rainbow tables or similar techniques if you had the MD5 hash in question.Modern Yahoo! servers use SSL (https://) to login to the mail servers. This has drastically increased the security of the login process. When sending your password now it is written to the network stream in plain text compared to MD5 in previous years but before it goes onto the wire it is encrypted via SSL. If there is a weakness at the protocol level now then it will be with SSL and if that is the case we have much bigger problems than securing Yahoo! mail.One very recent example of breaking into Yahoo! mail happened this past week and involves one of the Vice President candidates in the United States. Governor Palin from the state of Alaska had her Yahoo! mail compromised and the resulting data was posted to the Internet. From several news stories (http://www.securityfocus.com/brief/824) it appears that the attacker guessed her “lost password” questions and then gained access. Using freely available information from the Internet the attacker guessed the questions such as “Where did you meet your spouse?”There may have been weaknesses in the Yahoo! system in the past but all of these have been fixed to my knowledge. The current weak link in the system appears to be password recovery mechanism used on almost every social system on the Internet. From the very beginning I recognized this to be a problem and I never enter guessable information into these forms (usually I just slam my fingers against the keyboard a few times). To me if I loose my password then there is no way for me to access the account again. If you are asking me to tell you how to hack into Yahoo! email then I can’t do that per rules of this forum (as you can see I don’t know anyway) but I do want to highlight the importance of not only the strength of your password but also the guessability of your security questions.

[tansqrx 0]

I also forgot to mention that Yahoo! Messenger and mail use two different means to authenticate users. Email uses SSL while Messenger uses the method described above (http://forums.xisto.com/no_longer_exists/). Once again I have never heard of any reliable method of cracking this authentication even off line. The protocol is much like Kerberos (https://en.wikipedia.org/wiki/Kerberos_(protocol)) which prevents against such things. A challenge is sent and the password is then mixed into the challenge using encryption. Even if you were able to see the response code, it is only good for that one challenge string and the challenge always changes.

[turbopowerdmaxsteel 0]

Using client side scripts for security or robot denial, greatly increases the chances of the security being compromised. An online voting mechanism for a reality show in India used javascript validation for the captchas and I remember exploiting it. Although, it wouldn't have worked as it required unique E-Mail addresses per vote, which wasn't mentioned until the end of the show. Add to the fact that, greater weightage must have been given to the SMS votes which the people where paying for.They had a form wherein, you had to select a contestant, provide your E-Mail address, enter the captcha and submit the form. There was no registration mechanism, or even to verify that the E-Mail address actually exists. The captcha code was passed onto the browser in the form of a cookie along with the response to the GET request on the captcha image. The validation of the captcha was done using javascript and it gave away the inherent weakness in the protection mechanism.

[tansqrx 0]

Yes like I said before the previous scheme may have been vulnerable but the current scheme looks to be safe.I have come to understand that being a Yahoo! programmer automatically puts you into some of the more shady areas of the Internet. Most of the “programmers” that experiment with Yahoo! are not what I would call upstanding citizens. Like it or not you have a lot of script kiddies and just plain down right liars. Some of these Yahoo! experts say that they have an email exploit just to get attention and then can never produce results. When someone contacts me about a new Yahoo! exploit or hack I always have to say prove it. If I never hear from them again or they try to side step the issue then I usually know that they are either exaggerating or lying. We actually have several good Yahoo! programmers here on Xisto that you can generally trust. I know that turbomax and a few others will not blow smoke up your tail. You may have to review some of your “firm evidences” to see if this is actually the truth.

[boyboy400 0]

Yes like I said before the previous scheme may have been vulnerable but the current scheme looks to be safe.
I have come to understand that being a Yahoo! programmer automatically puts you into some of the more shady areas of the Internet. Most of the âprogrammersâ that experiment with Yahoo! are not what I would call upstanding citizens. Like it or not you have a lot of script kiddies and just plain down right liars. Some of these Yahoo! experts say that they have an email exploit just to get attention and then can never produce results. When someone contacts me about a new Yahoo! exploit or hack I always have to say prove it. If I never hear from them again or they try to side step the issue then I usually know that they are either exaggerating or lying.

We actually have several good Yahoo! programmers here on Xisto that you can generally trust. I know that turbomax and a few others will not blow smoke up your tail. You may have to review some of your âfirm evidencesâ to see if this is actually the truth.

Thank you so much my friends for your so complete answers. I know no one will learn me hacking yahoo step by step but the only thing I want to is some keywords or professional insights about those who realy do it and you did your best to help me. As you have said, you want them to prove it to you. I did it and I'm definitely sure that the account was hacked. Do not ask me why but I'm sure and I don't know how. I know some of them might use things like tromessenger or things like this but some of them are realy experts........
You talked about some shady areas, I'm so interested in entering them
But my friend, you don't know your security question. Me as a newbie think that what if you got hacked like Mr. poor president? If you know your recovery formula, there will be at least a hope but if not......
Thanks again for answering me with such long messages.....

[tansqrx 0]

If you want to discuss more on the topic of Yahoo! then I invite you to come over to my website at ycoderscookbook.com and http://forums.xisto.com/no_longer_exists/. There you can ask any question you like and hopefully get the same full answer as here.

As for my security question answers they look similar to the following:

âafdkljadshflaksdjfhkdsaâ

They are long, full of garbage, and I donât think anyone will be guessing it, especially since I donât know what it is myself. I have the belief that if you forget your password then you donât deserve an account anyway.

[tansqrx 0]

Here is another question about the authentication process

I fear that because of the lack of development the auth will sooner or later be obsolete. (any idea what version of the protocol they are using?)Although it is working, in light of the above, I am still looking for alternatives. I have looked at Trainer but... dont understand that vb thing :>

What part of the âauthâ are you referring to? This could mean the entire authentication sequence to login or the specific act of getting the challenge response string. Either way I think that you are safe from a drastic change from Yahoo! in the near future. The authentication sequence that is similar to a TCP three way handshake has changed some over the years with the change and addition of data fields but the basic principal remains the same. The current protocol has for the most part remained unchanged since version 12-13 and we are now in I believe 16.

The hash string function has undergone no changes since it was introduced and you can still use the same function from four years ago today. It is possible for Yahoo! to change this but I donât see that anytime soon. They just released Messenger 9 out of Beta today and it still uses the same old hash function. If it does change then it could go one of two ways; it could get even more draconian and hard to reverse engineer or it will go to an open standard. My feeling is that it will go open because Yahoo! has been better to embrace open standards in the past few years. Similar to Microsoft, they have realized that the functions will eventually be broken so there is no need for all the extra work. If it goes the other way then it will be up to smarter people than I to reverse it. Once again there will be a few weeks where the Messenger network will be closed off to third party clients but after that the work around will be posted all over the Net.

Also realize that Messenger is ultimately the property of Yahoo! and Yahoo! can change it at anytime without consulting us. I experience this all the time with YCC Yahoo! Bot Maker where Yahoo! changes a minor part of their signup page and I have to make corrections and release a new version. I too am actively looking for the source for the hash algorithm but I am happy with what I have currently.

P.S. If you are looking to design an entire client around Messenger and not fool with .NET then let me know. I have been considering making a separate tutorial on how to make your own language independent client. This would involve programming paradigms to how to sniff and capture traffic. I donât know how much demand for a tutorial like this would be so I havenât worked on it.

[bousta 0]

ithink that yahoo is best then google