Intl.Segmenter:

For Stage 3

Daniel Ehrenberg

Summary

• What?
• Locale-tailored grapheme, word and sentence breaks (UTS 29)
• Line breaks (UAX 14)
• Why?
• Complex in-browser rich text editors, design tools, rendering
• Want to deprecate Chrome’s Intl.v8BreakIterator
• Out of scope
• Hyphenation

API shape

// Create a segmenter in your locale�let segmenter = new Intl.Segmenter("fr", {type: "word"});

�// Get an iterator over a string�let segmentIterator = segmenter.segment(

"Ceci n'est pas une pipe");��// Iterate over it!�for (let {index, breakType} of segmentIterator) {� console.log(`index: ${index} breakType: ${breakType}`);� break;�}

// logs the following to the console:�// index: 3 breakType: letter

Short-cut accelerated API:

​

let segmentIterator = segmenter.segment(

"Ceci n'est pas une pipe");

​

// next() returning undefined

segmentIterator.advancefollowing()

​

// index of current break result

segmentIterator.index()

​

// breakType of current break result

segmentIterator.breakType()

Jump to a point; reverse iteration

let segmenter = new Intl.Segmenter("fr", {type: "word"});�let segmentIterator = segmenter.segment("Ceci n'est pas une pipe");

​

// find the last break before a code unit�segmentIterator.preceding(10);

console.log(segmentIterator.index); // prints 5, space before n’est

​

// find the first break after a code unit

segmentIterator.following(12);

console.log(segmentIterator.index); // prints 14, space after pas

​

breakType is specified (NEW: minimal categories)

Uses class best practices

MS’s “Standardize ICU” concern

• Sticking with approach of specifying goal, not data
• Cannot standardize on a database, as i18n data changes (accuracy, legal/geopol)
• All vendors working to gradually upstream/harmonize data
• Windows will ship ICU API, so no lack of richness

Reviewers

• What do the reviewers think of Stage 3?
• Caridy
• TBD from PayPal