cqlz-gym

Give it entities and a weighted query workload, get the Cassandra tables. An integer program and a small learned policy search for the design, cqlc-core verifies every candidate, and a calibrated cost model scores it.

GitHub crates.io Policy

$ cargo install cqlz-gym

How it works

Data modelling as a verified search

Cassandra data modelling means choosing denormalised tables so every query is a cheap single-partition read. The standard automated approach, NoSE (Mior, Salem, Aboulnaga, Liu, IEEE TKDE 2017), solves an integer program. cqlz-gym keeps that formulation as the exact baseline and adds a policy that builds the design step by step. After each step cqlc-core decides whether a query is valid and warning-free against the proposed schema; invalid actions are masked, so every proposed design is valid by construction and the reward needs no labelled data.

Workloadentities + weighted queries (TOML)

→

SearchILP, branch-and-bound, learned policy

→

cqlc-coreexact validity + action mask

→

Cost modelcalibrated on measurements

→

Designschema.cql, queries.cql, entities.rs

# design tables, print verified CQL, write files for cqlc (schema.cql, queries.cql) and cqlx (entities.rs)
cqlz-gym solve shop.toml --policy hf --emit out/

# load naive, greedy and best design into a real ScyllaDB and measure the workload mix
cqlz-gym validate shop.toml --node 127.0.0.1:9042

Results

Against an exact integer program

80–150-query workloads: cost relative to the best design (lower is better)

greedy

1.117

exact B&B (15 s limit)

1.078

naive

1.044

learned policy

1.006

ILP (MILP)

1.000

Bars start at 1.000. The ILP solved all 8 instances to optimality in under 10 ms; the branch-and-bound solved none within 15 s; the policy needs 0.16 s per rollout.

Workload size	greedy	B&B	ILP	learned	B&B solved	ILP solved
4–7 queries	1.095	1.000	1.000	1.000	8/8	8/8
8–16 queries	1.009	1.000	1.000	1.003	8/8	8/8
25–50 queries	1.054	1.001	1.000	1.005	7/8	8/8
80–150 queries	1.117	1.078	1.000	1.006	0/8	8/8

The honest result: an integer-programming solver is the right tool at these sizes. It finds the optimum in milliseconds, so the learned policy is a fast approximation within 0.6 %, never better. An earlier version of this page showed the policy beating an exact solver; that exact solver was a weak branch-and-bound, and the comparison was unfair.

Real workloads

Designs for domain workloads

cqlz-gym validate loads the naive, greedy and best design into a real ScyllaDB (Docker, one node, 10,000 rows per entity, 32 clients, 6 s per design) and measures the workload mix. The three workloads are modelled on DataStax data-modelling examples (sensors, shopping) and KillrVideo.

Workload	Tables naive → best	Model-predicted cost	Measured ops/s	Weighted latency
sensors	6 → 5	−23 %	+8.2 %	726 → 671 µs
shop	7 → 6	−5 %	+1.4 %	572 → 564 µs
killrvideo	7 → 7	0	noise	473 vs 486 µs

The direction of the model is right and its gain is overstated (−23 % predicted, −7.6 % latency measured). Run-to-run noise is a few percent on a laptop with Docker, so only the sensors result is clearly outside it. Where the best design equals the naive one (killrvideo), the differences between rows are noise.

Environment speed

Why this can be a gym

Environment steps per second, M3 (4 performance + 4 efficiency cores)

1 thread

383 k

2 threads

738 k

4 threads

1.40 M

8 threads

2.04 M

One cqlc-core verification takes 3.1 µs, and the scaling efficiency is 96 % on two, 91 % on four and 67 % on eight threads (the last four cores of an M3 are efficiency cores). The environment is cheap enough that the search, not the checking, is the cost.

Limits

What to keep in mind

The model is simplified: one equality-restricted partition and at most one range column per query, independent entities, no secondary indexes, materialised views or LWT; at most 256 candidate tables.
The cost model is calibrated on measurements from the cql-ai dataset (seek, per-row and per-write cost), but it is still a model. A policy can exploit its errors, so check final designs with cqlz-gym validate on a real ScyllaDB.
Learned search matters at large workloads; at small sizes the exact solvers win or tie and the policy adds nothing.
Related work: neural combinatorial optimisation (Bello et al., 2016; Kool, van Hoof, Welling, ICLR 2019), Brax (Freeman et al., 2021) for making the environment the scalable part, and RL for NoSQL index selection and relational schemas. We found none for Cassandra schema design; that rests on a handful of searches, not a literature review.