Why Turkish text turns into ü and ş
Mojibake is what you get when text encoded in UTF-8 is read back as if it were something else — usually Windows-1252 or Windows-1254. UTF-8 stores "ü" as two bytes, 0xC3 0xBC. Read one byte at a time as Windows-1252, those two bytes are the characters à and ¼, so a single letter becomes a pair. Every Turkish character with a diacritic breaks the same way: ç becomes ç, ş becomes ÅŸ, ğ becomes ÄŸ.
The damage almost always happens at a boundary: a database column declared with the wrong character set, an HTTP response missing charset=utf-8, an Excel export, or a legacy client that predates UTF-8. Because the bytes are still there — just reinterpreted — the text can usually be recovered exactly, by encoding it back to bytes with the wrong code page and decoding those bytes as UTF-8.
This tool does that, and it handles the harder second case too: text that was mangled twice, where the correct character survived with an orphan lead byte glued to it (Tüürkçe rather than Türkçe). It repairs the input in place and reports how many passes were needed, so you can tell a single corruption from a repeated one.
Is any information lost?
- Usually not. The bytes are intact and only misread, so the repair is exact. Information is lost only when the wrong code page had no character for a byte and replaced it with a question mark or U+FFFD — at that point the original is gone and no tool can recover it.
How do I stop it happening again?
- Fix the boundary, not the data. Declare charset=utf-8 on responses, use NVARCHAR2 or an AL32UTF8 database character set on Oracle, and set the client encoding explicitly rather than relying on the operating system default — which on a Turkish Windows install is Windows-1254, not UTF-8.