Rewriting the multilingual sentence segmenter - sentencex - in Rust

In October 2023, we announced sentencex — a sentence segmentation library built for the wide language diversity of Wikipedia and Wikimedia projects. The original release comprised two separate libraries: one in Python (used by MinT , our machine translation service) and one in JavaScript (used by Content Translation ). Both were rule-based, practical, and got the job done. But maintaining two…