Benchmarks

Measured, including the results that do not flatter us. Apple M3, one node in Docker.

betterequalworsebaseline

At a glance

Six numbers

3.8×

cqlc parse speed vs sqlparser

40/40

cqlc, Cassandra 5.0 schema

97 %

cqlx vs raw driver

+0.29

cql-ai F1 vs keyword rules

+8.2 %

gym design, real ScyllaDB

400×

cql-ai slower than cqlc

cqlc

Fast, and complete for Cassandra 5

Parse 22 statements

3.8× faster than sqlparser

sqlparser is a SQL parser: no CQL schema checks.

KillrVideo schemas accepted

Cassandra 5.0 fully parsed

v4: the 7 rejected use DEFAULT, which is not CQL.

Real query examples analysed

v4 and v5 sample queries

The rest are invalid CQL in the samples (wrong vector size, date - date).

Numbers

cqlc 1.1	Time
lexer, 36 statements	13.9 µs
parser	52 µs
full analysis (lex, parse, resolve, type-check)	103 µs

cqlx

Level with the raw driver; safe by construction

YCSB throughput, k ops/s

all within run-to-run noise

cqlxraw driver

cqlx as share of raw driver

95 to 97 %

Zipfian 0.99, 64 tasks, median of 3 runs.

Build one statement

cqlx pays 250 ns for compile-time checks

Hidden by a ~1 ms network round trip.

Not measured here

cqlx 1.0 partial updates, LWT, batches and streaming are covered by tests on ScyllaDB and Cassandra 5.0/4.1, not by this benchmark.

cql-ai

Predicts measured bottlenecks, on data it never saw

Macro-F1 on held-out data

blue wins on all three

keyword rulesstatement onlywith data stats

Training data: 5,284 measured statements

ScyllaDB 2,670 + Cassandra 5.0 2,614

NoSQLBench tabularNoSQLBench iotcassandra-stressKillrVideo commentsNoSQLBench queue (tombstones)TSBS cpuNoSQLBench keyvalueYCSB

Time to check 36 statements

cqlc is the faster, simpler tool; cql-ai answers a different question

42 ms is far inside a CI budget.

Numbers

Macro-F1	with data stats	statement only	keyword rules
unseen worlds	0.855	0.683	0.557
unseen families	0.889	0.674	0.624
other engine (Cassandra 5.0)	0.802	0.624	0.516

Trained on ScyllaDB only for this table. Without data statistics the model is only a little better than the rules: the gain comes from knowing your data size. Released weights are trained on both engines.

cqlz-gym

An exact ILP wins; the design pays off on a real database

Extra cost over the best design

80–150 queries, lower is better

The learned policy is a fast approximation, never better than the ILP.

Instances solved to optimality in 15 s

80–150 queries

ILP time: under 10 ms.

Real ScyllaDB: throughput vs one table per query

+8.2 % on sensors, equal on killrvideo

Noise is a few percent; the model predicted −23 % cost for sensors, measured −7.6 % latency.

Environment steps per second, M steps/s

91 % efficient on 4 threads; 8 threads use the efficiency cores

measuredideal

Table: cost relative to best, by workload size

Queries	greedy	B&B	ILP	learned
4–7	1.095	1.000	1.000	1.000
8–16	1.009	1.000	1.000	1.003
25–50	1.054	1.001	1.000	1.005
80–150	1.117	1.078	1.000	1.006

cqlint

Fast enough to run on every save

Throughput, MB of CQL per second

3,600 files, 23.3 MB, 79,200 statements

Apple M3, 8 cores. No comparable CQL linter or formatter exists to measure against: absolute numbers only.

Statements per second

lint 330k, format check 150k

Formatter safety

tokens and comments unchanged, idempotent

KillrVideo (v3 to v5) and cqlc example files; plus 190 benchmark statements.

cql-migrate, cqlprof

Correctness is the benchmark

Integration tests against real databases

all pass

cql-migrate: lifecycle, failure and resolve, lock, drift. cqlprof: tombstone detection, bound values, JSON.

cqlprof detects what it should

10 live rows + 2,990 deleted rows in one partition

Same result on ScyllaDB and Cassandra 5.0; the clean partition raises no finding.

Reproduce

Run it yourself

git clone https://github.com/Max-imalgutaussehend/cqlz && cd cqlz/benchmarks
cargo bench --bench frontend        # cqlc vs sqlparser
cargo bench --bench inference       # cql-ai vs cqlc
cargo run --release --bin realworld # KillrVideo coverage
cargo run --release --bin ycsb -- 50000 10   # cqlx vs raw driver (Docker)

git clone https://github.com/Max-imalgutaussehend/cqlz-lab && cd cqlz-lab
cargo run --release -p cqlz-gym -- eval --policy artifacts --instances 8 --exact-seconds 15
cargo run --release -p cqlz-gym -- bench

Data sources and what is not measured

KillrVideo schemas and queries (Apache-2.0), YCSB core workloads A/B/C, NoSQLBench (cql-keyvalue2, cql-tabular2, cql-iot, cql-timeseries2), cassandra-stress profiles, TSBS CPU table, DataStax data-modelling examples (shapes of the gym workloads). Not measured: cluster throughput, multi-node scaling, tail latency under overload, one machine only.