tc-lib-unicode

Overview

tc-lib-unicode provides Unicode conversion utilities and bidirectional algorithm support for multilingual text processing.

It is designed for text pipelines where normalization, code-point handling, and directionality directly affect rendering quality, especially with right-to-left and mixed-script content.

Repository and API Docs

• GitHub: https://github.com/tecnickcom/tc-lib-unicode
• API docs: https://tcpdf.org/docs/srcdoc/tc-lib-unicode
• Packagist: https://packagist.org/packages/tecnickcom/tc-lib-unicode

Project Metadata

Installation

composer require tecnickcom/tc-lib-unicode

Where It Fits

Integrate when multilingual content, right-to-left scripts, or normalization-sensitive text rendering is required.

Features

Unicode Utilities

• UTF-8 character and ordinal conversion helpers
• String/character array transformations
• Integration-ready conversion methods for document engines

Bidirectional Support

• Unicode Bidirectional Algorithm (UAX #9) implementation that passes the full official BidiCharacterTest.txt conformance suite
• Right-to-left and mixed-direction text processing
• Arabic shaping driven by the Joining_Type property

Character Substitution

• Context-sensitive codepoint-level substitution via Substitution::replaceChars(), a pure codepoint transform with no font or PDF dependency
• Devanagari (U+0900–U+097F): left-positional matras (U+093F, U+094E) are moved before their base consonant cluster, including conjuncts joined by Virama (U+094D)
• Hangul Jamo (U+1100–U+11FF, U+A960–U+A97F, U+D7B0–U+D7FF): composed into precomposed syllables (U+AC00–U+D7A3) per section 3.12 of the Unicode standard
• Thai (U+0E00–U+0E7F): returned unchanged, because Thai preposed vowels are already stored in visual order

Codepoints belonging to unsupported scripts are passed through unchanged.

Version 3.0 Changes

Release 3.0 is a breaking change driven by UAX #9 conformance work and the Unicode 17 data update:

• The bidi implementation now conforms to UAX #9 and is validated against the official conformance suite.
• Arabic shaping is driven by the Joining_Type property instead of a hand-maintained table.
• The package requires tc-lib-unicode-data ^3.0, whose tables are generated from UCD 17.0.0.
• UTF-8 codepoint and character conversions are vectorized through a single bulk mb_convert_encoding() call per invocation instead of per-character ord()/chr() loops.

Typed Enums

\Com\Tecnick\Unicode\TextDirection is a backed enum for the forced paragraph direction: Auto (''), Rtl ('R'), and Ltr ('L'). The Bidi constructor and the tc-lib-pdf text methods accept either the enum case or the plain string.

Known Limitations

• The paragraph separator is dropped during processing and appended again at the end of the paragraph output rather than being reset by rule L1 and reversed by L2. A strict UAX #9 implementation would place it at the visual left edge of a right-to-left paragraph; keeping it at the end preserves line splitting for consumers.
• Rule L3 (combining marks displayed in a different order) is not implemented.
• Shaping is Arabic only; the other cursive scripts (Syriac, N’Ko, Mandaic, Adlam) are returned unshaped.
• Bidi and Convert require valid UTF-8: malformed byte sequences raise an exception, and codepoints that cannot be encoded are replaced with ?.

Integration Notes

• Normalize input text early to avoid subtle glyph inconsistencies.
• Explicitly test mixed-direction paragraphs with punctuation and numerals.
• Track script metadata for predictable shaping behavior.

Requirements

• PHP 8.2 or later
• Extensions: ctype, mbstring, pcre
• Package dependency: tecnickcom/tc-lib-unicode-data ^3.0
• Composer

Example

<?php

require_once __DIR__ . '/vendor/autoload.php';

use Com\Tecnick\Unicode\Bidi;
use Com\Tecnick\Unicode\TextDirection;

$bidi = new Bidi('hello ', null, null, TextDirection::Rtl, false);
echo $bidi->getString();

// Script-specific codepoint substitution.
$sub = new \Com\Tecnick\Unicode\Substitution();

// Devanagari: the left matra is repositioned before its base consonant cluster.
$result = $sub->replaceChars([0x0915, 0x093F]);  // [0x093F, 0x0915]

// Hangul: Jamo composed into a precomposed syllable.
$result = $sub->replaceChars([0x1100, 0x1161, 0x11A8]);  // [0xAC01]

Development and Packaging

• QA and local checks: make deps, make help, make qa
• Local example server: make server (or make server PORT=8080)
• Packaging: make rpm, make deb

Support and Contribution

• Sponsor: https://github.com/sponsors/tecnickcom
• Contribution guide: https://github.com/tecnickcom/tc-lib-unicode/blob/main/CONTRIBUTING.md
• Security policy: https://github.com/tecnickcom/tc-lib-unicode/blob/main/SECURITY.md