entropack-alp

Lossless floating-point compression in pure Java

Byte-identical to the Rust and TypeScript references — a stream compressed in Rust, C++, or the browser decodes bit-exact on the JVM, and vice versa.

English | 简体中文

Pure-Java implementation of ALP — Adaptive Lossless floating-Point compression (SIGMOD 2024). Full codec (encode + decode) for f64 and f32, zero runtime dependencies, JDK 17+. Native long throughout — no BigInteger, no boxing on the hot path.

The paper

Independent implementation of the algorithm from:

Azim Afroozeh, Leonardo X. Kuffo, and Peter Boncz. ALP: Adaptive Lossless floating-Point Compression. Proc. ACM Manag. Data 1(4), Article 230, December 2023. DOI · author self-archive (CWI) · C++ reference

Not affiliated with CWI. Independent implementation of the published algorithm.

Cross-language, byte for byte

The whole point of this package. It reads and writes the ALPF v1 wire format, identical to the Rust and TypeScript implementations. Verified two ways:

• Golden vectors — the 11 shared vectors in alp-format/testdata/v1, generated by the Rust encoder, all decode bit-exact here and toBytes(compress(input)) reproduces each golden file byte for byte — all four schemes (ALP, ALP_rd, dictionary, RLE), zero fallbacks.
• Identical ratios — ./gradlew bench produces bits/value numbers that are bit-for-bit identical to the Rust and TypeScript implementations on the same datasets (see the table below), because the encoders emit the same bytes.

So: compress once on the server (Rust/C++), ship the bytes, decode on any JVM — Android, Spark, Flink, a data-warehouse UDF.

What is implemented

• ALP + ALP_rd, both f64 and f32 (compress/decompress, compressF32/decompressF32) — the paper's full §3 core plus §4.4 float32.
• DICT/RLE + ALP cascade (paper Table 4) for repeated / run-heavy columns.
• ALPF wire format (toBytes/fromBytes) with CRC-32C (JDK java.util.zip.CRC32C); fromBytes fully validates untrusted input (structural caps, allocation guards, exact consumption) and throws FormatError — never crashes, hangs, or over-allocates.
• Deterministic: identical input always produces identical bytes.

Losslessness contract: decompress(compress(data)) and decompress(fromBytes(toBytes(compress(data)))) reproduce data bit-exactly for every input. Enforced by jqwik property tests over the full 64-bit and 32-bit bit-pattern domains — including NaN payloads, signaling NaNs, signed zeros, infinities, and denormals — and hardened by four rounds of cross-language adversarial review.

On the JVM, bit-exactness needs care a naïve port would miss. Two examples the review caught: float→double widening quiets signaling NaNs, so exception and ALP_rd bit patterns are read straight from the input array with Float.floatToRawIntBits / Double.doubleToRawLongBits (the Raw variants), never through a double carrier; and because Java has no unsigned integers, every count and run-length parsed from untrusted bytes is range-checked with Long.compareUnsigned before use, matching Rust's u64 semantics exactly. Malformed streams are rejected at the same point Rust rejects them.

Compression ratios

Measured with ./gradlew bench (Apple M2, JDK 21), on the same deterministic (splitmix64) datasets as the Rust and TypeScript implementations — the numbers are bit-for-bit identical across all three:

f64 dataset	values	bits/value	vs 64-bit
two-decimal prices	250,000	24.12	2.65x
integers stored as doubles	10,000	20.57	3.11x
constant column	5,000	0.09	727x
random bit patterns (ALP_rd)	120,000	64.44	0.99x
half prices, half random	102,400	52.05	1.23x
100 distinct prices shuffled (DICT+ALP)	100,000	7.13	8.98x
long constant runs (RLE+ALP)	150,000	0.23	281x

f32 dataset	values	bits/value	vs 32-bit
two-decimal prices	250,000	20.10	1.59x
random bit patterns (ALP_rd)	120,000	32.22	0.99x
ML-weight-style (ALP_rd)	8,192	27.00	1.19x

Random bit patterns fall back to ALP_rd and land near the 64/32-bit baseline (0.99x — the correct, honest result for incompressible data); structured columns compress 2.6–730×.

Throughput

Single-threaded, one 102,400-value row group of two-decimal prices (Apple M2, JDK 21, warmed-up JIT, median of 15 rounds):

operation	ns/value	values/s
compress	16.13	~62 M
decompress	1.59	~629 M

Decompress runs at roughly 5 GB/s of doubles. This scalar (non-SIMD) JVM codec decodes within ~4–5× of the NEON-vectorized Rust reference (~0.36 ns/value) and is an order of magnitude faster than the pure-TypeScript port (~32 ns/value decode) — the JIT auto-vectorizes the frame-of-reference unpack well. A jdk.incubator.vector (Java Vector API) width-specialized unpack path is the natural next step if you need more.

Quick start

import io.github.entropack.alp.Alp;
import io.github.entropack.alp.Compressed;

double[] data = { 8.0605, 1.23, 4.56 };
Compressed c = Alp.compress(data);

// in-memory round-trip
double[] restored = Alp.decompress(c);

// serialize to the cross-language ALPF byte format
byte[] bytes = Alp.toBytes(c);                 // decodable by alp-rs / alp-ts
double[] back = Alp.decompress(Alp.fromBytes(bytes));

// float32 (paper §4.4)
float[] weights = { 0.1f, 0.2f, 0.3f };
float[] w = Alp.decompressF32(Alp.compressF32(weights));

// untrusted bytes? fromBytes fully validates and throws FormatError — never crashes
try {
    Compressed safe = Alp.fromBytes(untrusted);
} catch (io.github.entropack.alp.FormatError e) {
    // e.kind, e.offset describe the rejection
}

Install

Not yet published to Maven Central. Once released:

// Gradle
implementation 'io.github.entropack:alp:0.1.0'

<!-- Maven -->
<dependency>
  <groupId>io.github.entropack</groupId>
  <artifactId>alp</artifactId>
  <version>0.1.0</version>
</dependency>

Testing

./gradlew test     # 82 tests: unit + conformance + property + malformed hardening
./gradlew bench    # compression ratios + throughput

Coverage: per-module unit tests mirroring the Rust suite; golden-vector conformance (byte-exact both directions, all four schemes); jqwik property tests over the full u64/u32 bit domains including NaN payloads, signaling NaNs, signed zeros, infinities and denormals; and malformed-stream tests (truncations, corrupted headers, out-of-range fields, hostile counts including u64 values with the sign bit set, random bit flips) that must all throw FormatError without crashing, hanging, or over-allocating.

License

Apache-2.0.