Unicode Encoded Site - Characters Not Displaying Properly?

Hey guys,

I've been working on a AJAX based CMS for a client for the last few days and am in the final stages of deployment. I've run into a slight problem here. The site is supposed to be in Norwegian. The back-end uses the tinyMCE editor for adding in content, which is stored in MySQL. Now this presents no problem at all and the content is displayed just fine in the browser..

Problem arises when it comes to the site menu. It is a drop-down menu script from DynamicDrive, which reads off the menu items the same way from MySQL as the content and builds up the menus dynamically.. this is where it's messing up. The Norwegian characters in the menus are not being displayed properly.. in most cases, you can see that ugly "?" which turn up when your browser / page-encoding doesn't cover the displayed character set.

Now it can't be a problem with MySQL coz all these fields are declare utf8_general_ci. Also when I send in the page headers, I made sure to send in..

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

.. so that the page encoding is set right.

What baffles me is WHY the content should appear fine and NOT the menu items !! You can take a look at the demo site here: http://forums.xisto.com/no_longer_exists/

Any ideas how to fix this?

Cheers,

m^e

If it's the same thing I had to deal with a few months ago, then it is a MySQL issue (but not a problem). You can read more about this here.

Anyway, one of the solutions to this problem is to run this query every time you open a MySQL connection:

SET NAMES 'utf8'

Ugh.. Actually.. If the content appears OK, but the rest doesn't.. I might be wrong, but do give it a shot.

Well changing the encoding to Western ISO shows it correct, which suggests that's the encoding being used for the menu. Maybe part of the site, grabs ISO encoded data, that needs to be converted to Unicode, or you need to make sure each page, including external scripts be UTF-8 encoded when saved. Most editors should be capable of doing that, if not, then I don't think you should use those editors.Javascript works on UTF-16, so it's output should be fine, it could screw up with it's content it grabs though.I can't really dive too much into all this, I'm tired as anything, just got home, so hopefully you can sort it.Cheers,MC

Was away for a coupla days.. just came backLet me work on the ideas you guys have suggested and see if I can spot the problem..Regards,m^e

Hey guys,

 

Problem arises when it comes to the site menu. It is a drop-down menu script from DynamicDrive, which reads off the menu items the same way from MySQL as the content and builds up the menus dynamically.. this is where it's messing up. The Norwegian characters in the menus are not being displayed properly.. in most cases, you can see that ugly "?" which turn up when your browser / page-encoding doesn't cover the displayed character set.

 

Cheers,

m^e

 

are those menus being pulled from the database by using an ajax call? if they do you need to tell ajax to return the data in utf8 or utf16 format, i go with utf8.. it seems that the encoding of the menu was western ISO as given by the post above this.

i use to have that problem too then i explicitly told ajax to parse the data as utf8 since i am storing them as utf8_general_ci... in cases i forget to tell the ajax script about the encoding, i get the swedish charset which is the default on my server and my special chars turn into chunks of ?..

did you check the colation also in the mySQL database that the columns that holds the menu data are indeed set to utf8, i made this mistakes and mySQL uses the default charset.