fix(web): repair the web interface — startup, results, progress and privacy

[42piratas]

allelio serve exits on import, and every request path behind it has a defect of its
own. I found these trying the web UI on a real 23andMe export; the CLI path works and
is untouched here.

This started as the startup crash and grew as each fix exposed the next one. It is
fixes only — no new features — one commit per theme, so it reviews in pieces.

Startup and wiring

Where	Problem
web/routes.py	from starlette.concurrency import run_in_executor — that name is not in starlette and is not in its history. The import aborted allelio serve before uvicorn bound a port. It was also unused: the code calls loop.run_in_executor.
web/routes.py read_root	TemplateResponse("index.html", {"request": request}) is the old positional form. On the starlette this resolves to today it raises TypeError: unhashable type: 'dict', so / returned 500.
web/routes.py ×2	AIEngine.check_connection is async and was called without await. /api/status returned 500 serialising a coroutine; the upload path read a truthy coroutine as a live connection.
web/routes.py get_status	db.get_statistics() — the method is get_stats. The AttributeError was swallowed by the surrounding except, so the UI reported 0 ClinVar and 0 GWAS entries against a full database.
web/routes.py analyze_file	get_variant_warnings(rsid, genotype) — it takes one argument, the VariantResult. Every upload failed here, after the analysis had already run.
database/store.py	The sqlite connection is opened on the event loop thread, then used from run_in_executor. sqlite refuses that by default.
templates/index.html	Reads database_ready / ollama_connected; /api/status returns db_ready / ollama_available. The status panel said "Database Not Set Up" against a full database.
pyproject.toml	fastapi>=0.104.0 resolves a starlette that predates the TemplateResponse(request, name) signature. On a clean install at the declared floor the index page 500s. Floor raised to 0.108.

The upload looked like it hung

It did not — a whole genome takes about twelve minutes. The progress bar was on a
timer, faked its way to 90% in fifteen seconds and stopped there, with no text. There
was nothing to distinguish that from a hang.

• /api/analyze now publishes its stage to /api/progress and the page polls it, so
  the bar reflects the run: parsing, matching, then n of 50 while the explanations
  are written.
• Explanations went through explain_variants_batch instead of a sequential loop
  (12 min → 9.5 min on this machine).
• explain_variants_batch reported progress twice per variant, once with a stale
  count, so the bar counted backwards. It reports once now.
• The AI request timeout was 60s, which is not enough for this project's own default
  model once a few explanations run at once — 28 of 50 came back as the timeout
  fallback. 300s: 0 timeouts on the same input. That alone would let 50 variants,
  three at a time, hold the upload open for over an hour, so the batch also gets a
  30-minute ceiling and keeps whatever finished — as the fallback text, not as an
  empty string, so a slow model never leaves the user with less than no model would.
• One variant failing outside explain_variant's own guard used to cost one
  explanation. Once the per-variant try/except moved out of the route it cost the
  whole upload, minutes after the analysis had finished.
• Ollama being unreachable used to fail the whole upload with a 503. The README
  promises the tool works without it, so it now completes and says the summary is
  unavailable.

The results list showed the wrong thing

• Cards were sent clinvar_data / gwas_data, fields that do not exist on
  VariantResult. Every card read "Gene: Unknown" and every variant — pathogenic
  ones included — was badged BENIGN. They now get gene, significance and
  pubmed_id off the entries that are actually there.
• "Conflicting classifications of pathogenicity" was badged PATHOGENIC on a
  substring match — it contains the word. It is 130,833 rsIDs in the dump allelio download-db fetches, and it sorts to the top of the report, so it was the first
  thing a user saw. Calling it BENIGN instead would be the same lie in green, so it
  has its own badge now. A trait does too, rather than borrowing benign's green.
• The tabs filtered on health_conditions while the analyser emits
  Health Conditions, so four of the five tabs said "No results found".
• Picking a tab looked the button up by its position in a second, hardcoded category
  list. Anything missing from that list indexed to -1 and threw before the results
  re-rendered.
• That list was missing two of the six categories the analyser emits, carrier status
  and uncategorized — 21% of the findings on a real genome, reachable only under All.
• The ClinVar source link keyed on a clinvar_id that is in no payload and on no
  entry, so it rendered for no variant. It points at the rsID now.
• The summary prompt was handed counts alone and asked to summarize findings it had
  never been shown, so the model answered by saying so. It gets the variants.
• _strong_gwas compared p-values by slicing the exponent out of the string form.
  p_value is a float column and 6,271 GWAS rows hold 0.0, which has no exponent
  at all, so the strongest associations in the catalogue were read as the weakest.
• The top-50 sort keyed on significance_score, an attribute no result object has.
  It was dead rather than wrong — analyze_variants already returns them sorted —
  so it is gone.
• A ClinVar benign call was unreachable for any variant that also had a GWAS row,
  because the GWAS check ran first. ClinVar has the last word now.
• A GWAS row on its own was badged RISK — 37,108 of the 62,057 findings on a real
  genome, every one of them under a tab the analyser itself labelled Traits. Calling
  them all a trait instead demotes type 2 diabetes and coronary artery disease, and
  the analyser cannot tell them apart either: its GWAS test is a substring search for
  "risk" or "association" in the trait string, which 2,068 of the 553,549 GWAS rsIDs
  pass. So the badge says ASSOCIATION, which is what a GWAS row is.
• "Uncertain significance" had no branch at all, so it came out as a trait. It is
  1,236,063 ClinVar rows — the largest class — and it passes the benign filter, so a
  variant nobody can interpret was being drawn as a harmless characteristic. It has
  its own badge now, and a protective call reads as benign.
• get_variant_warnings computes a genetic-counselling warning for BRCA1/2, TP53,
  Lynch and APOE. It was computed for all 62,057 results and rendered nowhere. It is
  on the card and in the exported report now — once, not twice: explain_variant
  already folds it into the explanation, but only on the path where the model
  answered, so the test is on the text rather than on whether an explanation exists.
  It sits under the card header rather than inside the collapsed body — a BRCA1 card
  otherwise looked like every other card until you clicked it.
• Explanations were gated on the model being reachable, so a machine without Ollama
  got empty cards — even though explain_variant already writes a ClinVar/GWAS
  fallback for exactly that case.
• The model writes paragraphs and HTML collapses newlines, so the summary and every
  explanation arrived as one run-on block. The exported report needed the same.
• The progress bar only moved while explanations were being written, so it read 10%
  through every other stage.

Security

• allow_origins=["*"] with allow_credentials=True. Any page the user visited
  could POST a genome to localhost and read the result back. The UI is same-origin
  with the API, so the middleware is gone rather than narrowed.
• Stored markup in the results list. Cards were built by string interpolation
  from the uploaded file. Nothing validates a genotype except a -- check, so a
  crafted file executed script in the page. Every field is escaped now, and the two
  values that land inside a URL are pattern-checked rather than escaped.
• The exported HTML report had the same hole, server side, in its rows and in
  its own header fields — and /api/export accepts an arbitrary dict.
• The uploaded file's name came from the multipart header, unsanitised.
  Path(tempfile.gettempdir()) / file.filename with ../../.. in it resolves out of
  the temp directory; the route writes the request body there and then unlinks it.
  multipart/form-data is CORS-safelisted, so this needed no preflight and dropping
  the CORS middleware does not close it. mkstemp picks the name now.
• That same file is the user's whole genome, and it was landing at 0644 in a
  shared directory. mkstemp gives it 0600.
• The exported report was left in the shared temp directory at 0644, forever,
  with the user's genotypes in it. Same treatment, plus it is unlinked once sent.
• rel="noopener noreferrer" on the outbound links.

Verified

Against a 630,774-variant 23andMe export (Personal Genome Project, openly consented),
in Chrome, with Ollama and llama3.1:8b:

• allelio serve starts, / renders, /api/status reports db_ready: true with
  2,645,423 ClinVar and 1,191,572 GWAS entries
• upload → 62,057 findings, all rendered, every category tab matching the counts the
  analyser produces
• 50/50 explanations, no timeouts, a summary that names the genes it found
• export downloads a report
• a genotype file whose genotype column carries an <img onerror> payload renders as
  inert text, with no script execution

tests/test_web.py is new and covers the routes, the CORS behaviour, the two
classifiers, the escaping on both sides, the p-value comparison, the upload path and
the batch deadline. Full suite: 107 passing.

Left alone deliberately

• _get_significance_rank in analysis/lookup.py substring-matches "pathogenic"
  inside "pathogenicity", so Conflicting classifications of pathogenicity scores
  the same rank as a true pathogenic call and files under Health Conditions. On a real
  genome that interleaves 5,048 conflicting variants with 7,959 genuine ones at the top
  of the report. This PR gives them an honest badge; the ranking is an analysis-module
  change that moves CLI output too, so it wants its own PR. (SIGNIFICANCE_RANKS
  already has a conflicting interpretations key at rank 6 — ClinVar retired that
  wording, the dump says classifications, so the key never matches.)
• _progress is a module global, so two concurrent runs clobber each other's
  bar. Needs a per-run token; out of scope here.
• AllelioDB.close() has no caller, so each request leaks a connection.
• The 50-variant cap on explanations is yours by design, so I left it. Same for
  the report's results[:100], though that means the export carries 79 of the 1,634
  counselling warnings a real genome produces.
• _strong_gwas's p < 1e-5 matches 1,186,123 of the 1,191,572 GWAS rows, so it
  filters nothing. The old code had the same effective threshold, so this is not a
  regression — but the number the summary prompt is given is inflated.
• 62,057 rows render at once. I timed it — a few seconds to render, a few to
  switch tabs — so it is not the freeze I first assumed, but paging would be kinder.

Happy to split any of this out or drop pieces you would rather not carry.

[42piratas]

Moving this to draft — I found two more defects of the same class in analyze_file after opening it, and want the branch to cover the whole serve path rather than land half of it:

• ai_engine.generate_explanation(rsid, chromosome, position, ...) — no such method. It is explain_variant(result), taking the VariantResult.
• ai_engine.generate_summary(total_variants=..., significant_variants=..., top_categories=...) — the signature is generate_summary(results).

Both were caught by except Exception and written into the response as the string "Explanation generation failed", so every explanation silently came back as that placeholder while the request still returned 200.

Will push both and mark ready once I have a full run through the web UI confirming the explanations actually render.