Compression

Compress and decompress short strings.

Functions 🔗

unistat 

uni_compress(

const void *text, unisize text_len, uniattr text_attr, uint8_t *buffer, size_t *buffer_length)

Compress text.

unistat 

uni_decompress(

const uint8_t *buffer, size_t buffer_length, void *text, unisize *text_len, uniattr text_attr)

Decompress text.

Discussion 🔗

Lossless data compression algorithms, like LZW, perform compression by deduplicating repetitions within a data stream. These algorithms are general purpose and are designed to operate on “bags of bytes” without any prior knowledge or assumptions about the structure or semantics of the data.

Unicode Technical Report #34 defines two algorithms designed specifically for the compression Unicode encoded text. These algorithms are Standard Compression Scheme for Unicode (SCSU) and Binary Ordered Compression for Unicode (BOCU). These algorithm leverages knowledge of the Unicode character set and beat general purpose compression algorithms for shorter strings. Both algorithms are intended for short to medium length Unicode strings (about several hundred characters in length). Once strings becomes longer and include many repetitions, then a general purpose compressor is preferable.

Support for compression must be enabled in the JSON configuration file.

{
    "algorithms": {
        "compression": true
    }
}