DZCM: a context-mixing codec of our own

Later on June 4, DevZIP added DZCM, a new context-mixing backend. The commit and design notes describe an original implementation rather than another wrapper around an existing compression library.

The idea is to predict the next bit from several kinds of context: recent bytes, words, sparse patterns and matches with earlier input. A mixer combines those predictions, and an arithmetic coder turns them into the compressed stream. The decoder repeats the same updates to reconstruct the original bytes.

The first recorded backend-only results were mixed in a useful way. DZCM compressed the text corpus about 30 percent smaller than the LZMA2 backend and improved code by roughly seven percent, but it was still behind ZPAQ-5. It performed worse on the executable corpus. Those are comparisons within the same DevZIP pipeline, not a new aggregate win over every archiver.

The backend could be selected directly or included in a best-of-N pool. Choosing the smallest candidate per block can protect the size result when one codec isn't a good fit, although trying another codec still consumes time and memory. Keeping the winning codec identity is what makes the mixed archive decodable.

The record includes byte-exact verification and additional per-file hash comparison for the code corpus. The roadmap names better context-state modeling, executable structure and hints from the transform pipeline as areas to explore next.

What I like about this stage is that it leaves the weak result on the page. We have a working experimental codec and a clear reason to continue improving it. The next decision should come from another measured comparison and round trip, rather than assuming the new backend belongs in every default just because we wrote it.