Zawgyi and Unicode in Myanmar

by Aung Kham (www.myanmar-entrepreneur.com)

August 8, 2014

What is Language?

Language (according to Wikipedia) is the human capacity for acquiring and using complex systems of communication, and a language is any specific example of such a system. The scientific study of language is called linguistics.

Machine language is a set of instructions executed directly by a computer's central processing unit (CPU)

What is Myanmar Language?

Myanmar is the official language name that people in Myanmar use to communicate to each other (verbally and virtually). There are many Myanmar languages (due to the diversity of Minorities) such as Burmese, Shan, Karen, Mon. All Myanmar language which use the same scripts must have a uni-script.

What are Myanmar people using to communicate via internet?

- Zawgyi

Because Zawgyi is the first language for people to get in touch with Myanmar fonts and keyboards, 100 percent of Myanmar internet users use Zawgyi for communication.^[a][b][c]

- English

50 percent of Myanmar people use both Zawgyi and English language (US) for communication.

- Myanglish (using English font which sound in Myanmar language when reading)

20 percent of Myanmar internet users use English keyboard to type which sound Myanmar language.

- Unicode

Unicode is being used only by few people. The percentage is too small. But Government websites in Myanmar are using Myanmar unicode for updating information.

What is Zawgyi?

Zawgyi is a set of keyboard and machine language that was created for burmese people communication. This was produced by Alpha Mandalay company in Myanmar, early 2004s. It became the first language to be used in communicating with each other via digital devices such as: computer, mobiles, etc. Virtually it shows the correct characters to readers. But technically it does not obey the rules and regulation as provided by http://www.unicode.org. (Unicode is a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems.)

Virtually people can read it, but not all computers are able to read it. This is due to the error in the encoding system. For example, Zawgyi does not have text support for Shan, Karen and other similar minority languages in Myanmar.

When two and more people communicate via email (digital way), they both have to install a script (fonts and keyboard produced by Alpha Zawgyi). When people communicate, they use Internet browsers. When they want to search something, they use ‘Search engines’.If the internet browser can process language, no one don’t need to install any fonts and keyboards. But the problem is no browser and any search engine don’t support Zawgyi because technically Zawgyi is not correct. It renders in correct way for readers, but don’t for machine processing.

So what are the disadvantages for using Zawgyi and other non-unicode languages?

The main problem in Zawgyi is about processing. No computer can process Zawgyi in the ideal way.

The following facts are from http://my.wikipedia.org/wiki/Wikipedia:Font

• Those who have Zawgyi or pseudo-Unicode fonts in their computers find their computers may display incorrect text and might affect other Unicode-supported applications.
• Zawgyi uses extra code points which are reserved for minority languages of Myanmar (Shan, Mon, Pwo Karen, S'gaw Karen, Geba Karen, Kayah and Rumai Palaung, as well as Sanskrit and Pali). This creates a conflict, because Unicode has reserved those points to support minority languages in Myanmar, not for Burmese characters that should be intelligently rendered by Unicode engines.
• Zawgyi uses reserved code points for stacking Burmese consonants and creating combinations of diacritics and medials, whereas Unicode smartly renders these, so fewer code points are required for one language.
• There are serious problems with encoding and typing in Zawgyi.

• When users type in Zawgyi, typing in different ordering will create different words. These different words, even though they may render the same, are not stored the same (so search is impossible if words are typed in different ordering). On the other hand, in Unicode, there is only one correct way to store text, and incorrect ordering in typing is visible as you type (this means that searching for text is more difficult with Zawgyi).
• Unicode stores text in only one order and render correctly:

• Zawgyi can store text in several ways but superficially appear correct: 
• Many Zawgyi users type the number "zero" when they intend to input the consonant "wa." This creates a problem, because converters cannot intelligently find the instances where "zero" is intended and "wa" is intended. When using Unicode, "wa" and "zero" are separate (intelligent rendering shows you if you have typed "zero" instead of "wa").
• Unicode can intelligently tell apart wa and zero:

• Zawgyi cannot intelligently render zero and wa separately:

• Unicode can intelligently render yayit according to letter width with just one symbol: 
• Whereas for Zawgyi there needs to be two different symbols (total 8 different symbols for short head and short leg forms) for yayit:

Example -

The result of finding လ+  ူ+ မ+ ွ+ု ေရး                         The result of finding လ+  ူ+မ+  ု +ွ ေရး

Zawgyi renders the same font (Zawgyi show the same font) to users. But to a device, they are unable to  recognize it as they are not the same font due to different series of writing. In this image above you will see Google revealing two different result. In unicode, order of writing is according to its regulation whereas Zawgyi has no regulation to it.

As machine have to be programmed and cannot intellectually process Zawgyi and un-unicode languages. It creates future problem and block further development such as:

• Voice command technology in Myanmar language.
• Search Engine support for Myanmar people to finding information in Myanmar.^[d][e]
• Softwares and application to translate Myanmar to others and others to Myanmar.

It also effect that people from other countries who may not know what languages we are using. All they see are square boxes. They may not have chance to learn about Myanmar language and even if they install Zawgyi, because of the computer and others using Unicode they may have trouble adjusting. They will want to use Google translators and perhaps other softwares to translate and read about Myanmar but it becomes impossible because  Zawgyi fonts and keyboards software don’t translate in any translation apps.

What is Myanmar unicode?

Myanmar unicode is an encoding for the Myanmar writing system being produced according to rules and regulations laid down by the global computer standard organization (www.unicode.org)

• It is the internationally accepted standard by the World Wide Web Consortium, the main international standards organization for the World Wide Web.
• The fonts necessary to view and edit are freely available.
• Search is seamless with Unicode.
• Unicode fonts support 11 languages that use the Myanmar script:        Burmese, 2 liturgical languages: Pali and Sanskrit,  8 minority languages: Mon, Shan, Kayah, four Karen languages and Rumai Palaun,

There are more than 14 Myanmar unicode fonts, but which are using the same machine language, standardized by Unicode.org. The most people use is ‘Myanmar 3’ To learn more about Unicode here is the link - http://www.myanmarlanguage.org. the website builder is Myanmar unicode supporter. There are many fonts and keyboard proclaiming that they are unicodes.

Problem with Myanmar Unicode

Windows OSs before 2008 didn’t support Myanmar Unicode in machine. But from 2008 Windows OS onwards, users don’t need to install Myanmar Unicode to be able to read it. Mac OS also is supporting Myanmar unicode. Firefox and Google services (including Google Chrome) have also moved from Zawgyi to begin supporting Myanmar unicode. For example, Google is now automatically converting Zawgyi to MM unicode^{[f][g][h][i][j][k][l]}. But as most myanmar people are still using Windows 7, and are still installing Zawgyi, they still have problems in reading Unicode. Changing language setting in computer will also not support Myanmar Unicode even in Windows 8 and Mac OSs. So we still have problems due to unchanging behavior of Myanmar internet users.

Why people are hard to change their behavior of using Zawgyi to Unicode?

Yes, it is too busy and boring to change. The problem is the way of typing is not the same. People are familiar with Zawgyi typing layout. Unfortunately typing layout for Unicode is little bit different from Zawgyi.^[m][n]

So what are the solutions at the moment?

We, in only one way, have to use Unicode in the future for developing Myanmar ICT technology. But as people are hard to change, the following are possible solutions.

Converters such as

• Burmese Encoding Converter Bot (online)
• Kanaung Converter Engine (program)
• DocChartConvert (program)

Unfortunately there are some who can not be bothered to use the program or know how to. Still there are other solutions such as Keyboard type writing layout auto correct softwares.^[o][p][q]

There are some softwares which let people to type in Zawgyi keyboard layout then it process and output as Myanmar Unicode. The problem to this also is that:^[r][s][t][u]

1. Most are premium software. (People have to buy)
2. People don't want to install software (thinking it takes time and processes)

These are all still short term solutions. The long term solution is encouraging people to use Myanmar unicode. That’s what Unicode supporters in Myanmar are doing. Not only are all government sites in Myanmar making the change but also there are more than 50 websites in Myanmar now using Unicode standard. But most people visiting websites such as myanmarmobileapp.com and many others are only running on Zawgyi font. They embedded the font, meaning it doesn’t matter if you have Zawgyi or Myanmar Unicode. The web page will show you in Zawgyi font because it’s embedded. You can embedded Zawgyi or Unicode. It’s your choice.

 There are also some websites who accommodate by using dual machine language method.

Can there be a Zawgyi unicode?

Some said that Zawgyi developers already developed Unicode version of Zawgyi fonts and keyboard. But it still requires a software installation^{[v][w][x][y][z]} to adjust. It would be of great assistance to Myanmar language if Zawgyi became part of Unicode.

How about embedding fonts?

Embedding fonts in websites is a technique to show the only preferred font (in this case Zawgyi or Unicode) when a user visits the website.This is created so that a user can overcome the issue of incompatibility between different encoding standard especially if the PC doesn’t actually have the fonts in its processor. PC will just render the fonts getting from website. But the problem is that it doesn’t work in a mobile browser. The technology still needs to be developed.

Even if the font is embedded the site owners still faces the dilemma of whether to use Unicode or Zawgyi as their font of choice. If Zawgyi is not optimized and unicode is chosen this may still be a better solution. There are some who say that font embedding would slow down site speed. In conclusion site embedding is also not a better solution but a quick short solution.

There are still many other related issues to talk about. For example: people know about Win Myanmar font. But Win (like Zawgyi) is not unicode. Win Myanmar Systems, better known as Win Myanmar Fonts, has been the nation's de facto standard for Myanmar language processing since 1992. It translates MS Windows and Excel menu's to Myanmar language. The Win Myanmar font system for Microsoft Windows is very common in Myanmar for digitally processing the Burmese script. Win Myanmar Systems enables you to type, design, or print in Myanmar language right on your computers. The software supports both non-Unicode ASCII formats and ISO-compliant Unicode formats. Often called Win Fonts, they are not just fonts, as they come along with a very user-friendly keyboard drivers for Windows (compatible with Windows 3.0 to Windows 8). However in this conversation about Zawgyi and Myanmar Unicode, Win Myanmar is a topic that needs to be known on the side but not involved.

References -

http://www.unicode.org/notes/tn11/myanmar_uni-v2.pdf

http://my.wikipedia.org/wiki/Wikipedia:Font

https://code.google.com/p/zawgyi/wiki/WhyUnicode

http://www.myanmarlanguage.org

Update (April, 2015)

Latest Android OS such as Lollipop automatically support Myanmar Unicode. ^{[aa][ab][ac][ad][ae][af][ag][ah][ai][aj]}Latest Mac OS also already have built-in Myanmar language which follow unicode standard). Windows OS also have Myanmar Unicode already. It means all global tech products already have built-in Myanmar Language for Myanmar people. Myanmar people cannot just start changing habit.

Update (December 2016)

I have heard people complaining that Unicode-compatible keyboards are not natural, requiring people to type in a very special way. This is not true, since a smart keyboard can rearrange the input characters automatically, which makes typing much easier.

Note that Android has updated its Myanmar keyboard (Unicode) with reordering rules that make it easier to type. Feedback would be great. And how can people be more informed on Unicode keyboards?