Require Parquet input for ORM ingest so the fast path is unavoidable

[skearnes]

Follow-up to #877 (COPY loader) and #879 (row-group sharding).

Motivation

The bulk ingest optimizations only apply to Parquet inputs. Today load_datasets/ingest_dataset still accept non-Parquet files (.pb, .pb.gz, .pbtxt) and quietly route them through database.add_dataset — the ORM unit-of-work path that these PRs exist to get away from (~45 rxn/s, ~10 INSERTs/reaction). That silent fallback is exactly how someone ends up "accidentally slow": point the CLI at a directory of .pb.gz files and you get none of the COPY/sharding speedups, with no signal that anything is wrong.

Make the fast path unavoidable: the loader should accept only Parquet, and reject anything else with a clear error that points at the converter, rather than falling back to the slow path.

Proposed change

• ingest_dataset / dataset_id_for_file (ord_schema/orm/loading.py): drop the non-Parquet branches; raise a ValueError naming ord_schema/scripts/pb_to_parquet_dataset.py when a non-.parquet input is passed.
• Drop the now-unused load_message / md5 imports in loading.py, and the parquet_files/other_files split in load_datasets (added in Shard Parquet ingest by row group for intra-dataset parallelism #879 precisely so this is a clean deletion) — the ingest stage becomes Parquet-only.
• Convert the ~3 tests that ingest the .pbtxt fixture (test_load_datasets, test_stages_ingest_then_derived, and the ORM comparison in test_parquet_copy_ingest_matches_orm) to write a Parquet fixture first (the _write_parquet_dataset helper already exists).

What stays

database.add_dataset (and from_proto/to_proto and the mapper layer) are not removed: add_dataset is the in-memory-proto path used by ord-interface for single-dataset submission, and from_proto builds the reaction trees the COPY loader itself walks. This issue only removes the non-Parquet file ingest from the bulk loader; converting existing serialized datasets is a one-time pb_to_parquet_dataset.py run.

Scope

Small and self-contained — a couple of branches, some imports, and a few test fixtures; not a subsystem removal. The value is enforcement (no silent slow path), not lines deleted.

🤖 Generated with Claude Code