Use an Arena Allocator 🔗

If a custom memory management routine is used, then it’s recommended to implement it as an arena allocator. Unicorn never retains allocated memory, so once a function returns, the arena can be deallocated all at once.

To further improve performance, when a realloc request is issued, it’s recommended to check if the pointer being reallocated refers to the most recent arena allocation. If so, then the last allocated block can be resized in place. This is an especially practical optimization because most functions that allocate memory (e.g. uni_norm, uni_casefold) do so for a single intermediary buffer.

To even further improve performance, consider using a double-sided arena allocator. While most function use a single intermediary buffer, some use two, for example uni_normcmp and uni_casefoldcmp. It’s recommended to alternate which side of the arena blocks are allocated from. If a realloc request is issued, it’s recommended to check both front and back blocks to see which one needs to grow.

Normalize Your Strings 🔗

Strings can be compared for canonical equivalence with uni_normcmp. This function will incrementally perform Unicode normalization on the input strings and compare the resulting characters for equivalence. While the implementation of the algorithm is performant, it is even more performant to normalize strings once, up front with uni_norm and re-use the result. If both strings are known to be normalized, they can be compared with memcmp directly. If both strings are encoded as UTF-8 and null terminated, then strcmp can be used.

This recommendation also applies to case folding: Instead of comparing strings with uni_casefoldcmp, which always folds the input strings regardless of whether they need it not, prefer to fold them once with uni_casefold and cache the result for future comparisons.

Quick Check Before Normalizing 🔗

Before normalizing text, consider performing a quick check to determine if it needs normalization with uni_normqchk. This function returns yes, no, or maybe depending on if the input text is, is not, or might be normalized. If the function returns 'yes', then the text is normalized and you can skip normalizing it. If it returns 'no' or 'maybe', then you must normalize the string.

The implementation of uni_normqchk is highly performant, especially when compared to uni_normchk. While uni_normchk does return a definitive 'yes' or 'no' answer, it does so at a performance cost roughly equivalent to having normalized the text.

Cache Sort Keys 🔗

Strings can be compared for sorting with uni_collate. This function internally constructs a sort key which is a 16-bit integer array used for comparing strings. Constructing a sort key is expensive (relative to comparing them). If the same strings will be compared multiple times, then it’s recommended to construct a sort key once with uni_sortkeymk and cache its result for future comparisons with uni_sortkeycmp.

Trust Well-formed Strings 🔗

All Unicorn functions that process text are “safe” by default and can catch potentially malformed text. However, if the text is known to be well-formed then these safety checks incur an unnecessary performance penalty. To avoid the penalty, the UNI_TRUST flag can be used to instruct the implementation to skip validation checks, improving character throughput.

If your text doesn't change often, then you can validate it with uni_validate and cache the result. If it’s known to be well-formed, then you can use UNI_TRUST from then on.