Notes from Việt Nam on law, technology, languages, history, culture and travel.

VIEETS, VIET61 or VIET^'T: How Vietnamese Learned to Live on an English Keyboard

I have typed Vietnamese for most of my life on keyboards that do not carry a single Vietnamese letter. So has nearly everyone else in the country. The word viết, “to write”, leaves my fingers as vieets. A friend from Sài Gòn types viet61. Someone who learned on a 1990s email list might still write vie^'t. Three sets of keystrokes, one word, and none of them looks like the word.

A few weeks ago I wrote about how Vietnamese came to be written in Latin letters at all. This post is about the next problem: how those letters, with their stacks of marks, get into a machine.

A keyboard nobody owns

The alphabet has 29 letters, but the tone marks multiply them. RFC 1456, the 1993 Internet document that defined the first widely used Vietnamese encodings, puts the requirement plainly: beyond plain ASCII, Vietnamese needs “an additional 134 combinations of a letter and diacritical symbols”. A typewriter can hold a few of them as dead keys. A keyboard cannot hold 134.

There is a national standard for a Vietnamese key layout, TCVN 6064:1995, and Windows ships a layout based on it. I have never seen anyone use it. What people use instead is an ordinary US keyboard plus a small program that watches the keystrokes and swaps in the right letter. Telex, VNI and VIQR are the three rule sets those programs follow.

Telex: a trick from the telegraph office

Telex is the oldest, and its logic is half a century older than the personal computer. In the 1920s and 1930s Vietnam’s telegraph lines could carry only plain Latin letters. A telegram reading vo de could mean vỡ đê, “the dyke has burst”, or vợ đẻ, “my wife is in labour”, two messages you would very much like to tell apart. The convention is usually credited to the journalist and translator Nguyễn Văn Vĩnh; Saigoneer traces a proposal he published in the newspaper Trung Bắc Tân Văn in 1929. The idea was to spell the marks with letters.

The computer version keeps that idea. Doubling a vowel adds the circumflex (aa → â, ee → ê, oo → ô). The letter w adds the horn or the breve (aw → ă, ow → ơ, uw or plain w → ư), and dd gives đ. Five letters that almost never close a Vietnamese syllable carry the tones: s for sắc (á), f for huyền (à), r for hỏi (ả), x for ngã (ã), j for nặng (ạ), while z removes a mark. So vieets is v-i-ê-t plus a rising tone.

The weak point is obvious to anyone who has left Telex on while typing English: tool becomes tôl and address becomes ađress. The strong point is that your fingers never leave the letter rows, which is why nearly everyone I know uses Telex on a phone. The old rules are not dead either: Wikipedia notes that Vietnamese Morse telegraphy still spells the marks the Telex way.

VNI: numbers instead of letters

VNI came from the other side of the Pacific. Hồ Thành Việt founded VNI Software in Westminster, California, in 1987, to make Vietnamese usable on the PCs the diaspora was buying. His first design put the tone marks on the function keys, F1, F2 and so on. When operating systems claimed those keys, the method moved to the digits along the top row, and the mapping is easy to remember: 1 to 5 are the five tones in textbook order (sắc, huyền, hỏi, ngã, nặng), 6 is the circumflex, 7 the horn, 8 the breve, 9 turns d into đ, and 0 removes a mark. Viết is viet61; thật is tha6t5.

Digits never collide with English words, but reaching for 9 every time you need đ is slow. VNI also has a legal footnote. Microsoft built the method into the Vietnamese edition of Windows 95; VNI sued over unauthorised use, Microsoft settled out of court and withdrew it. It took until Windows 10 version 1903, released in May 2019, for Windows to offer Vietnamese input natively, with both Telex and a “Vietnamese Number key-based” keyboard, which is VNI in all but name.

VIQR: Vietnamese for seven-bit email

The third method is really a convention for writing Vietnamese with no special characters at all. It grew up on the diaspora’s email lists and the Usenet group soc.culture.vietnamese, and was written down in RFC 1456 in May 1993 by a volunteer group, the Vietnamese Standardization Working Group, better known as Viet-Std. Each mark becomes the punctuation character that looks most like it: ( for the breve (a( → ă), ^ for the circumflex, + for the horn (o+ → ơ), ' for sắc, ` for huyền, ? for hỏi, ~ for ngã, . for nặng, and dd for đ. Viết is vie^'t.

The RFC admits the obvious hazard: without a guard rule, “How are you?” would be read as How are yoủ. VIQR is rare now, but you still meet it in old forum archives and in the fingers of people who learned to type Vietnamese abroad in the 1990s.

Forty-six ways to store the same letters

Typing was only half the problem. The text also had to be stored, and here Vietnamese went through a long mess. Wikipedia counts as many as 46 character encodings for the language. The ones you actually met were VISCII (from Viet-Std), VSCII or “TCVN3” (the national standard TCVN 5712:1993), VNI’s own encoding, VPS and Microsoft’s Windows-1258. Each solved the 134-combination problem differently. VISCII squeezed six rarely used capital letters into control-code positions. VNI spelled a vowel as two bytes, the second one carrying the marks. TCVN3 dominated in the North and VNI in the South, and a document opened with the wrong font turned into the porridge every Vietnamese office worker of the era recognised: §¹i häc for Đại học, Vieät Nam for Việt Nam.

Unicode ended that, but not without a fight. Its original plan was to encode only the combining marks and let software stack them. Viet-Std lobbied for ready-made letters, and in 1993 the consortium agreed to encode every Vietnamese letter as a precomposed character. They are now scattered across four blocks, most of them in Latin Extended Additional. The legacy encodings faded; Mozilla dropped support for all of them except Windows-1258 in 2014.

Two leftovers still bite programmers. First, both spellings of a letter remain legal Unicode: ề can be one code point or three (e, circumflex, grave). UniKey has a setting for exactly this, plain “Unicode” (precomposed, dựng sẵn) versus “Unicode tổ hợp” (combining), and I have debugged a search bug where a file name typed on a Mac, whose file system historically stored names in decomposed form, failed to match the same name typed into a browser. Normalise your strings. Second, Vietnamese has two conventions for where the tone mark sits in syllables like hòa and thủy. Most input methods default to the older placement (hòa, thủy) with an option for the newer one (hoà, thuỷ), which is what the Ministry of Education prescribed for school textbooks in 2018. To a reader they are the same word. To grep they are two different strings.

The 2 KB program inside your iPhone

The software that made all of this invisible for most Vietnamese users was written by one student. In the early 1990s Phạm Kim Long, then at Hanoi University of Science and Technology, won a bet with three classmates by writing the smallest Vietnamese input program for DOS: about two kilobytes of assembly. He released UniKey for Windows free in 1999, opened the engine as x-unikey for Linux in 2001, and in 2006 agreed to hand Apple the source under the MIT licence. From Mac OS X Leopard in 2007, and from iOS 4 in 2010, the built-in Vietnamese keyboard on every Mac and iPhone has run on his engine. The acknowledgement is still there under Settings → General → About → Legal.

The marks people drop

The last chapter has no software fix. When phones could not show Vietnamese, people typed without marks, and tiếng Việt không dấu became a register of its own. It works most of the time because context does the work, but the ambiguity is real. dan can be dân, dần, dặn, dẫn, đan, đàn, đản, đạn or đần. con dau is a daughter-in-law (con dâu), a rubber stamp (con dấu), a child in pain (con đau) or “where has it gone?” (còn đâu).

Putting the marks back automatically is a well-studied problem, and the numbers show how hard it is. A 2017 paper by Thai-Hoang Pham, Xuan-Khoai Pham and Phuong Le-Hong treated restoration as machine translation from unmarked to marked Vietnamese. The phrase-based system reached 97.32% accuracy and a neural system 96.15%, the latter about twice as fast. Those are good scores, and they still mean roughly three mistakes per hundred syllables, or a wrong word in nearly every long sentence. The errors land exactly where context is thin: names, a one-word reply, a number spelled out. Machines are far better at this than they were. They are still worse than a human who knows the two people texting.

What I take from this

  • Learners: pick Telex on day one. It is the default on phones, on Macs and in Windows, and the rule “double the vowel, tone letter at the end” takes an afternoon to learn. The marks are half the information, so type them.
  • Switch modes, do not fight them. If you type English and Vietnamese all day, bind a hotkey to toggle the input method. Every ađress you send is a reminder.
  • Developers: normalise to NFC on the way in, and treat hòa and hoà as the same word in search. Never store Vietnamese without marks and hope to recover it later; the best restorers still miss three in a hundred.
  • Learn to read không dấu anyway. You will get texts in it, and the skill of guessing the right word from context is the same skill that makes you a better listener.

The three keystroke spellings of viết at the top of this post are, in the end, three generations of the same decision: a nation of a hundred million people chose to keep the world’s keyboard and teach software to understand them. It worked well enough that most of them have never thought about it.

Sources

Posted on Categories Languages