clickhouse-js-node-rowbinary

ClickHouse JS RowBinary Codec Generator for Node.js

This skill generates both directions of the wire format: readers (decode bytes → values) and writers (encode values → bytes, the mirror). A given task normally needs only one side. This file is the shared entry point — the format gate plus the principles common to both directions; the per-direction decisions, guidance, and the per-type reference tables live in two sibling files.

Pick your side — read only the one you need:

Decoding a RowBinary* response from ClickHouse into JS values → reader.md. Streaming vs whole-buffer, row-objects vs columnar, fixed vs runtime schema, and the per-type reader reference.
Encoding JS values into a RowBinary payload to send to ClickHouse → writer.md. The Sink/writeX building blocks, writeRows streaming, and the per-type writer reference.

The per-type code is real, split by direction under src/readers/ and src/writers/.

First: is RowBinary even the right format?

RowBinary exists for throughput, but it is not automatically the fastest path — match the format to the shape of the data before committing to a bespoke parser.

Prefer a JSON* format (e.g. JSONEachRow) when the result is mostly strings / JSON-like values that you consume wholesale — randomly accessing essentially every field, running string/regexp methods on them, treating values as text. V8's native JSON.parse is heavily optimized C++ and builds JS strings and objects faster than a JS-level RowBinary decoder can; pair it with HTTP response compression (gzip / zstd, which crushes JSON's repetitive keys) and the wire cost shrinks too.

RowBinary clearly wins when the result is dominated by:

Wide numerics — Int128/Int256/UInt128/UInt256, Decimal128/Decimal256.
Binary / fixed-width blobs — IPv4, IPv6, UUID, FixedString.
High-volume fixed-width numeric columns generally, where each value is a single DataView read.

Prefer the Native format when columnar load and client-side analytics are the main goal (fold/scan/filter columns, feed typed arrays to a Worker or WASM). Native is column-major, so it loads straight into one typed array per column with no transpose.

For help choosing and consuming a JSON* format (or CSV / TSV) instead, use the clickhouse-js-node-coding skill.

Core guidance (both directions)

These principles apply whether you are generating a reader or a writer; the side-specific operational guidance is in reader.md / writer.md.

Little-endian only. RowBinary is little-endian; target x86/ARM. Read and write every multi-byte number with DataView accessors passing a literal true for the littleEndian flag.
Correct first, then optimize. First emit a correct codec built from the plain per-type API. Only after it's correct (and tested) specialize it. Don't bake performance assumptions in before correctness.
Monomorphize generic/composite types. Emit specialized, inlined code per type combination instead of passing functions as arguments where the type is known ahead of time.
Inline the leaf ops. The per-type readX/writeX functions are the correct, composable reference; the generated codec should INLINE their bodies, not call them, so the row loop is straight-line with no per-field indirection (and so the fixed-width coalescing can fold the offset arithmetic together).
Annotate the type per column. Inlining erases the type structure, so put a short comment above each column's encode/decode block naming the ClickHouse type it handles.
Shared scratch is not reentrant. Some hot methods reuse a module-level scratch buffer as a write-then-read pair — correct only because the access is fully synchronous. An async/yield boundary between populating and reading it corrupts the value.
TypeScript by default. Generate TypeScript code and helpers unless the user explicitly asks for plain JavaScript.

Worked examples

Six end-to-end examples with real speedup are catalogued in EXAMPLES.md.

Out of scope

JSON / CSV / TSV / Parquet parsing → use clickhouse-js-node-coding.
Connection errors, hangs, type mismatches → use clickhouse-js-node-troubleshooting.
Browser / Web Worker / Edge → @clickhouse/client-web.

Still Stuck?

ClickHouse JS RowBinary Codec Generator for Node.js

Pick your side — read only the one you need:

Decoding a RowBinary* response from ClickHouse into JS values → reader.md. Streaming vs whole-buffer, row-objects vs columnar, fixed vs runtime schema, and the per-type reader reference.
Encoding JS values into a RowBinary payload to send to ClickHouse → writer.md. The Sink/writeX building blocks, writeRows streaming, and the per-type writer reference.

The per-type code is real, split by direction under src/readers/ and src/writers/.

First: is RowBinary even the right format?

RowBinary exists for throughput, but it is not automatically the fastest path — match the format to the shape of the data before committing to a bespoke parser.

RowBinary clearly wins when the result is dominated by:

Wide numerics — Int128/Int256/UInt128/UInt256, Decimal128/Decimal256.
Binary / fixed-width blobs — IPv4, IPv6, UUID, FixedString.
High-volume fixed-width numeric columns generally, where each value is a single DataView read.

For help choosing and consuming a JSON* format (or CSV / TSV) instead, use the clickhouse-js-node-coding skill.

Core guidance (both directions)

These principles apply whether you are generating a reader or a writer; the side-specific operational guidance is in reader.md / writer.md.

Little-endian only. RowBinary is little-endian; target x86/ARM. Read and write every multi-byte number with DataView accessors passing a literal true for the littleEndian flag.
Correct first, then optimize. First emit a correct codec built from the plain per-type API. Only after it's correct (and tested) specialize it. Don't bake performance assumptions in before correctness.
Monomorphize generic/composite types. Emit specialized, inlined code per type combination instead of passing functions as arguments where the type is known ahead of time.
Inline the leaf ops. The per-type readX/writeX functions are the correct, composable reference; the generated codec should INLINE their bodies, not call them, so the row loop is straight-line with no per-field indirection (and so the fixed-width coalescing can fold the offset arithmetic together).
Annotate the type per column. Inlining erases the type structure, so put a short comment above each column's encode/decode block naming the ClickHouse type it handles.
Shared scratch is not reentrant. Some hot methods reuse a module-level scratch buffer as a write-then-read pair — correct only because the access is fully synchronous. An async/yield boundary between populating and reading it corrupts the value.
TypeScript by default. Generate TypeScript code and helpers unless the user explicitly asks for plain JavaScript.

Worked examples

Six end-to-end examples with real speedup are catalogued in EXAMPLES.md.

Out of scope

JSON / CSV / TSV / Parquet parsing → use clickhouse-js-node-coding.
Connection errors, hangs, type mismatches → use clickhouse-js-node-troubleshooting.
Browser / Web Worker / Edge → @clickhouse/client-web.