Unicode

I've never cared for Unicode. First off, if you're going to make a "universal" character encoding standard, then make one. They have UTF-1, obsolted by UTF-8. Then there's UTF-16, UTF-32, and another obsoleted UTF-7, not to mention others not even prefixed "UTF."

Since the introduction of Unicode, I've experienced plenty of problems, and very little benefit. Being able to display the entire world's characters is not something of importance to the vast majority of people. Most people only know one language. Those who know multiple languages often know different languages which still use the same or mostly the same characters. Rarely will you find, and it is most likely Asians, who can read multiple languages with drastically different characters. This is why different character encodings exist.

But Unicode doesn't describe how to display characters from different languages alongside each other. That's a hard problem which most software developers don't want to worry about, and could never gain enough understanding of all of the world's languages to know if they've even done it right.

No one is going to create fonts with over a million glyphs in it. So now you need a full complement of fonts just in case somewhere there's a glyph your main font doesn't render.

To this day I still see web sites and other UI with broken glyphs (�). Apostrophes get mangled. If I find myself on a web page with a bundh of foreign glyphs, I have likely gone somewhere that is not even relevant to me.

Security

Further, UTF has introduced security issues. Since the introduction of UTF in our DNS, and since many languages share the same glyphs, URLs which look the same can in fact be completely different. You could find your browsing hijacked. You could be entering your password on the wrong site, and now your credentials have been stolen.

From the other perspective, if you own a web site and you allow user input in Unicode and need todo any filtering, you may need to check characters of foreign languages in addition to your native toung. How do you even begin to know what the entire set of such characters are?

Other Concerns

Back on my desktop, if I wanted to go to a URL which has Unicode characters in it, I can't type it in. If i have a file name with the same, I can't type that into my terminal window. Normally, if I get something from a foreign place, it is translated into my language. In English we write "Japan," not . In Japan they write U.S.A as .¹ When translating things into local language, the character encoding can be changed at that time as well.

I have even seen programming languages support Unicode. That's great, but what on Earth do I do with that? Again, I can't type it. So either every software project needs an international team with capable keyboards, or we need to use cut-and-paste everywhere to edit our code. I am much more comfortable with escape sequences like \U1234 or Ӓ. At least I can type those for the occasion I need the odd glyph.

Previously, I thought that at least this is an okay use of Unicode. But even then, the end reader has to have a font file which describes those glyphs in order to see the correct shapes. If the Unicode group were to provide a single font with decent rendering for every code point defined in their standard, then at least you could reasonably expect something on the display. However, as you can see in the update below, even that is simply impossible, because they merged Chinese and Japanese characters into one, so there is no one absolute rendering for every code point.

Conclusion

The main point is that we have a lot of different written languages, with a lot of different ways to write them, and translating among the different languages is a hard problem. Each time this needs to be done, care needs to be taken. Unicode tries to be a one-size-fits all for written languages and falls flat in doing so.

Update

More fun with Unicode. As I was writing this, I found a couple of other issues people have.

Permanent fix for unicode characters not displaying correctly indicates that creating a file on the Windows desktop with Unicode characters in the name might allow Windows to keep displaying Unicode correctly. As described, it seems Windows should already have all of the fonts needed. This is old but still highlights the kind of crazyness we have had to endure.

Your Code Displays Japanese Wrong alerts me to the fact that Chinese and Japanese characters have been munged into one homogeneous whole, even when Kanji characters do not look exactly the same as either form of Chinese characters. So some Unicode glyphs are identical with differing code points, and some languages use the same code point for different glyphs! The worst of both worlds.

Footnotes