It appears that some code point pairs are being incorrectly composed; a starter followed by a non starter seems to always produce a composition, even if the logical resulting character doesn't exist. For instance, the sequence U+72 U+307 U+323 represents a lowercase latin r with a dot below and a dot above. There does exist LATIN SMALL LETTER R WITH DOT BELOW, so the composition of the first pair results in a valid character. The composition of the resulting code point (U+1E5B) together with U+323, however, results in U+1E64 (LATIN CAPITAL LETTER S WITH ACUTE AND DOT ABOVE) because there is no LATIN SMALL LETTER R WITH DOT BELOW AND DOT ABOVE.
Many other sequences produce errors. All of the following compositions are incorrectly produced by utf8proc, but of course there are infinitely many more:
- U+61 + U+307 = U+227
- U+227 + U+323 = U+2E
- U+227 + U+307 = U+1FC
- U+1E5B + U+323 = U+1E64
The actual character that's produced in error, of course, is the result of a reasonable index into the data table, so I doubt any check on the value of composition could reliably fix this.
Here's a minimal program demonstrating the bug:
#include <stdio.h>
#include <utf8proc.h>
int main(void)
{
char *src = "\x72\xCC\x87\xCC\xA3";
char *norm = (char *)utf8proc_NFC((utf8proc_uint8_t*)src);
puts(src);
puts(norm);
return 0;
}