compress

Compression and decompression streams for gzip, deflate, brotli, and zstd.

Install

npm install @datastream/compress

gzip

gzipCompressStream Transform

Option	Type	Default	Description
quality	number	-1	Compression level (-1 to 9). -1 = default, 0 = none, 9 = best
maxOutputSize	number	—	Maximum compressed output in bytes (web variant)

gzipDecompressStream Transform

Option	Type	Default	Description
maxOutputSize	number	—	Maximum decompressed output in bytes. Destroys the stream with an error when exceeded

Example

import { pipeline } from '@datastream/core'
import { fileReadStream, fileWriteStream } from '@datastream/file'
import { gzipCompressStream, gzipDecompressStream } from '@datastream/compress'

// Compress
await pipeline([
  fileReadStream({ path: './data.csv' }),
  gzipCompressStream({ quality: 9 }),
  fileWriteStream({ path: './data.csv.gz' }),
])

// Decompress
await pipeline([
  fileReadStream({ path: './data.csv.gz' }),
  gzipDecompressStream(),
  fileWriteStream({ path: './data.csv' }),
])

deflate

deflateCompressStream Transform

Option	Type	Default	Description
quality	number	-1	Compression level (-1 to 9)
maxOutputSize	number	—	Maximum compressed output in bytes (web variant)

deflateDecompressStream Transform

Option	Type	Default	Description
maxOutputSize	number	—	Maximum decompressed output in bytes. Destroys the stream with an error when exceeded

brotli

brotliCompressStream Transform

Option	Type	Default	Description
quality	number	11	Compression level (0 to 11)

brotliDecompressStream Transform

Option	Type	Default	Description
maxOutputSize	number	—	Maximum decompressed output in bytes. Destroys the stream with an error when exceeded

zstd Node.js only

Requires Node.js with zstd support.

zstdCompressStream Transform

Option	Type	Default	Description
quality	number	3	Compression level

zstdDecompressStream Transform

Option	Type	Default	Description
maxOutputSize	number	—	Maximum decompressed output in bytes. Destroys the stream with an error when exceeded

Output size protection

Decompression bombs

A malicious compressed payload known as a “decompression bomb” can be as small as a few kilobytes but expand to gigabytes when decompressed, exhausting memory and crashing the process. Setting maxOutputSize ensures decompression is aborted before memory is exhausted. Always set this when decompressing untrusted input.

import { gzipDecompressStream } from '@datastream/compress'

// Limit decompressed output to 100MB
gzipDecompressStream({ maxOutputSize: 100 * 1024 * 1024 })

Compression output limits

Compression streams also support maxOutputSize (web variant) to cap compressed output size. This can be useful to enforce storage limits.

import { gzipCompressStream } from '@datastream/compress'

// Limit compressed output to 50MB
gzipCompressStream({ maxOutputSize: 50 * 1024 * 1024 })

Platform support

Algorithm	Node.js	Browser
gzip	node:zlib	CompressionStream
deflate	node:zlib	CompressionStream
brotli	node:zlib	CompressionStream
zstd	node:zlib	Not supported