Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
version: 2

updates:
# Actions are the highest-value thing to keep current: a stale third-party
# action is a supply-chain risk in every workflow that uses it.
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
day: monday
commit-message:
prefix: "ci"
groups:
actions:
patterns: ["*"]

- package-ecosystem: pip
directory: "/"
schedule:
interval: weekly
day: monday
commit-message:
prefix: "deps"
# One PR for the dev toolchain, separate PRs for anything in the request
# path: a ruff bump needs a glance, a FastAPI or SQLAlchemy bump needs
# reading.
groups:
dev-tooling:
patterns: ["ruff", "mypy", "pytest*", "pre-commit", "build", "twine", "mkdocs*"]
dependency-type: development
ignore:
# Retrieval quality is pinned to specific model and runtime versions and
# is measured on the eval set, not on whatever pip resolves this week.
# These move when someone re-runs the golden set, not on a schedule.
- dependency-name: torch
- dependency-name: transformers
- dependency-name: sentence-transformers
- dependency-name: FlagEmbedding
open-pull-requests-limit: 5

- package-ecosystem: docker
directory: "/"
schedule:
interval: weekly
day: monday
commit-message:
prefix: "docker"
60 changes: 60 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Publish documentation

on:
push:
branches: [master]
paths:
- "docs/**"
- "mkdocs.yml"
- ".github/workflows/docs.yml"
pull_request:
paths:
- "docs/**"
- "mkdocs.yml"
- ".github/workflows/docs.yml"
workflow_dispatch:

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: pages
cancel-in-progress: false

jobs:
build:
name: Build the site
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- run: pip install mkdocs-material mkdocs-static-i18n

# --strict is the point of running this on pull requests too: it fails on
# a broken internal link or a page missing from the navigation, which is
# exactly the breakage nobody notices until a reader hits it.
- run: mkdocs build --strict

- uses: actions/configure-pages@v5
if: github.event_name != 'pull_request'
- uses: actions/upload-pages-artifact@v3
if: github.event_name != 'pull_request'
with:
path: site

deploy:
name: Deploy to Pages
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
156 changes: 156 additions & 0 deletions README.en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Praxis

[Русский](README.md) · **English** · [Documentation](https://drobyshevdev.github.io/praxis/)

A legal assistant for Russian law. It answers a question and cites the specific
provisions the answer rests on — each citation checked separately by a model. When the
law does not support a claim, it says so.

<img width="2048" height="1152" alt="Praxis answering a question with verified citations" src="https://github.com/user-attachments/assets/cd75be1d-3944-4d3f-bbbd-dba4c4d1c12f" />

## The problem

A lawyer has three options today, and each is awkward in its own way.

ChatGPT and other general-purpose models invent: they cite articles that do not exist,
confuse revisions, and are confidently wrong. That cannot be trusted in practice, where
the price of a mistake is a lost case or a financial loss.

KonsultantPlyus and Garant give you a document search, not an answer. Matching the
provisions and drawing the conclusion is still the lawyer's job, and the subscription is
expensive.

Search engines return forums, superseded revisions and SEO noise.

Nothing sits between these options. No tool answers a question at the speed of a model
but with references you can open and check. That is the space Praxis occupies.

## Who it is for

Lawyers and in-house counsel at small and medium businesses without access to expensive
reference systems. Accountants and HR staff with questions about the Tax, Labour and
Administrative Offences Codes. Sole traders and founders. Individuals with everyday
disputes.

The Russian market comes first because the data is available, the audience is legible and
the problems are concrete. The same architecture transfers to EU and US law, where open
data is even more plentiful (EUR-Lex, CourtListener, the Caselaw Access Project).

## How it works

The question goes through an agent that plans the search. Retrieval is hybrid over the
corpus of provisions (BM25 plus dense embeddings), a reranker selects the best, and
cross-referenced provisions are pulled in through an article-to-article graph. The
generator assembles an answer in which every claim is bound to a provision, and the
Citation Verifier checks every binding.

What separates Praxis from a wrapper around chat-with-PDF:

**Citation Verifier.** A separate NLI model checks each reference for entailment: does
the text of the provision support this specific claim? What is not confirmed is not
presented as fact.

**Agentic self-RAG.** The agent breaks a complex question into sub-queries, searches again
and reformulates until it has enough grounding. The chain of reasoning is visible in the
answer.

**GraphRAG.** Provisions reference each other ("in accordance with article 15"). That is a
ready-made graph: a question about damages surfaces article 393, and the graph pulls in
article 15 that it names.

**Extractive by default.** Without an LLM key Praxis does not compose text — it quotes the
applicable provisions verbatim with references, and such an answer cannot hallucinate.
Synthesis through Claude is enabled by a key and passes the same per-sentence citation
check.

**Measured quality.** recall@k, MRR and citation precision are computed on a golden set,
not judged by eye.

## Data

The statutory data is available. Codes and federal laws are published in machine-readable
form at pravo.gov.ru, and that is what the main path is built on: the repository contains
a parser for the official text (`statute_parser`) and a sample corpus of the Civil Code;
the full corpus is loaded the same way.

The full text of the Civil Code is already extracted into the repository
(`corpus/gk-rf.json` — 1,712 articles, 4,717 provisions, all four parts, sourced from
Wikisource) and is loaded through `PRAXIS_CORPUS_DIR`. The revision in force should be
verified against pravo.gov.ru.

Judicial practice is harder. There is no open structured corpus for Russia comparable to
the Caselaw Access Project, and kad.arbitr and the GAS "Pravosudie" system give up their
data reluctantly. That is the next stage, as a separate pipeline.

## Quality

A run of the eval harness over the golden set (12 questions, `praxis-eval`):

| Metric | Real models (RTX 4060) | Offline fallback |
|---|---|---|
| recall@5 | 1.00 | 1.00 |
| MRR | 1.00 | 0.90 |
| hit-rate | 1.00 | 1.00 |
| mean confidence | 0.88 | 0.63 |
| citation precision | 0.29 | 0.40 |

Real models: BGE-M3 for embeddings, bge-reranker-v2-m3 for reranking, rubert-NLI for
citation checking, all on GPU. The needed provision always reaches the top of the
results. Citation precision is understated because the golden set has one reference
article per question while the system also returns adjacent relevant provisions. That is
fixed by labelling several correct articles per question.

On the full Civil Code corpus (4,717 provisions, an 18-question golden set) the real
models hold recall@5 0.92, MRR 0.94, hit-rate 1.0 and confidence 0.80. The offline
fallback drops to recall 0.64 at that size — on real data, real models are not optional.

## Roadmap

- v0. Skeleton, domain models, legal-aware chunking, BM25 baseline. Done.
- v1. Hybrid retrieval and reranking, Citation Verifier, self-RAG, eval, FastAPI and a web
UI. Done. Every ML component has a real implementation on GPU or through Claude, and a
deterministic offline fallback.
- v2. Real ingestion of the official text, GraphRAG over cross-references, a GPU run. Done.
- v3. The full corpus of codes and federal laws from pravo.gov.ru, judicial practice and a
provision-to-case graph, span-level citation highlighting in the UI, an expanded golden
set.
- v4. The full Civil Code extracted into the repository, a public API (`/v1`) with CORS and
a Python client, competitive analysis. Desktop and mobile on the same API are next.

How to run it — [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).

## Stack and ecosystem

Python 3.12, FastAPI, Postgres with pgvector, Docker. Retrieval: BM25 and BGE-M3 dense
embeddings, bge-reranker-v2-m3 as the reranker. Citation checking: NLI on GPU. The LLM is
plugged in through a provider: Claude for synthesis, Russian providers (GigaChat,
YandexGPT) for scenarios with data-residency requirements.

The project uses two libraries from the same organisation:
[glia](https://github.com/DrobyshevDev/glia) for the agent loop in LLM mode (search is
exposed as a glia tool, and the trace comes from its trajectory), and
[mlango](https://github.com/DrobyshevDev/mlango) for a tracked golden eval through its
evals subsystem (`integrations/mlango_eval`, `manage.py evaluate`).

Architecture — [ARCHITECTURE.md](ARCHITECTURE.md).

## Openness and clients

Everything is open (Apache-2.0) and runs locally without third-party tokens. The core
functionality — search, citation checking, answers — needs neither keys nor paid services:
local models download from HuggingFace for free, and the default answer is extractive, the
verbatim text of the provisions. Claude is an optional path to synthesis, not a condition
of the system working. Paid features and a subscription are for later.

A single API core (`/v1`) serves every client: the web UI now, desktop and mobile as thin
clients on the same API later. For ML practitioners there is a public API with CORS, an
OpenAPI schema and a Python client — [docs/API.md](docs/API.md).

Competitive analysis and where the project wins —
[docs/COMPETITIVE.md](docs/COMPETITIVE.md).

---

**Not legal advice.** Praxis surfaces provisions and checks that a citation supports a
claim. It does not assess your situation, does not account for procedural context, and
does not replace a lawyer.
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Praxis

**Русский** · [English](README.en.md) · [Документация](https://drobyshevdev.github.io/praxis/)

Юридический ассистент по российскому праву. Отвечает на вопрос и приводит ссылки на конкретные
нормы, каждую из которых отдельно проверяет модель. Если подтверждения в законе нет,
то сообщает об этом.
Expand Down Expand Up @@ -114,9 +116,11 @@ BGE-M3, реранкер bge-reranker-v2-m3. Проверка цитат: NLI н
провайдера: Claude для синтеза, RU-провайдеры (GigaChat, YandexGPT) для сценариев с
требованием резидентности.

Проект использует библиотеки автора (DenisDrobyshev): glia — агентный цикл в LLM-режиме
(поиск оформлен как glia-инструмент, трейс из trajectory); mlango — трекаемый golden-eval
через его подсистему evals (`integrations/mlango_eval`, `manage.py evaluate`).
Проект использует две библиотеки той же организации:
[glia](https://github.com/DrobyshevDev/glia) — агентный цикл в LLM-режиме (поиск оформлен
как glia-инструмент, трейс из trajectory); [mlango](https://github.com/DrobyshevDev/mlango) —
трекаемый golden-eval через его подсистему evals (`integrations/mlango_eval`,
`manage.py evaluate`).

Архитектура — [ARCHITECTURE.md](ARCHITECTURE.md).

Expand Down
3 changes: 2 additions & 1 deletion docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ curl -s http://localhost:8077/v1/search \
-d '{"query":"толкование договора","top_k":5}'
```

Python-клиент (только стандартная библиотека, [clients/python/praxis_client.py](../clients/python/praxis_client.py)):
Python-клиент (только стандартная библиотека,
[clients/python/praxis_client.py](https://github.com/DrobyshevDev/praxis/blob/master/clients/python/praxis_client.py)):

```python
from praxis_client import PraxisClient
Expand Down
90 changes: 90 additions & 0 deletions docs/index.en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Praxis

**A legal assistant for Russian law.** It answers a question and cites the specific
provisions the answer rests on — each citation checked separately by a model. When the
law does not support a claim, it says so instead of producing plausible text.

```bash
docker compose up app # → http://localhost:8077, no API keys required
```

## Why

A lawyer working on Russian law has three options today, each awkward in its own way.
General-purpose models invent: they cite articles that do not exist, confuse revisions,
and are confidently wrong. Commercial reference systems search documents rather than
answer, and are expensive. Search engines return forums, superseded revisions and SEO
noise.

Nothing sits between them: no tool answers at the speed of a model with references you
can open and check. That is the gap Praxis fills.

## How it works

The question goes through an agent that plans the search. Retrieval is hybrid over the
corpus of provisions — BM25 plus dense embeddings — a reranker selects the best, and
provisions connected by cross-reference are pulled in through an article-to-article
graph. The generator assembles an answer in which every claim is bound to a provision,
and the Citation Verifier checks every binding.

**Citation Verifier.** A separate NLI model checks each reference for entailment: does
the text of the provision actually support this specific claim? What is not confirmed is
not presented as fact.

**Agentic self-RAG.** The agent decomposes a complex question into sub-queries, searches
again and reformulates until it has enough grounding. The chain of reasoning is visible
in the answer.

**GraphRAG.** Provisions reference each other ("in accordance with article 15"). That is
a ready-made graph: a question about damages surfaces article 393, and the graph pulls in
article 15 that it names.

**Extractive by default.** Without an LLM key Praxis does not compose text — it quotes
the applicable provisions verbatim with references, and an answer like that cannot
hallucinate. Synthesis through Claude is enabled by a key and passes the same
per-sentence citation check.

## Quality

The eval harness over the golden set (12 questions, `praxis-eval`):

| Metric | Real models (RTX 4060) | Offline fallback |
|---|---|---|
| recall@5 | 1.00 | 1.00 |
| MRR | 1.00 | 0.90 |
| hit-rate | 1.00 | 1.00 |
| mean confidence | 0.88 | 0.63 |
| citation precision | 0.29 | 0.40 |

Real models: BGE-M3 for embeddings, bge-reranker-v2-m3 for reranking, rubert-NLI for
citation checking, all on GPU. The needed provision always reaches the top of the
results. Citation precision is understated because the golden set has one reference
article per question while the system also returns adjacent relevant provisions.

On the full Civil Code corpus (4,717 provisions, an 18-question golden set) the real
models hold recall@5 0.92, MRR 0.94, hit-rate 1.0 and confidence 0.80. The offline
fallback drops to recall 0.64 at that size — on real data the real models are not
optional.

## Data

Codes and federal laws are published in machine-readable form at pravo.gov.ru, and the
repository contains a parser for the official text (`statute_parser`). The full text of
the Civil Code is already extracted (`corpus/gk-rf.json` — 1,712 articles, 4,717
provisions, all four parts) and is loaded through `PRAXIS_CORPUS_DIR`. The revision in
force should always be checked against the official source.

There is no open structured corpus of Russian judicial practice comparable to the
Caselaw Access Project; that is the next pipeline rather than something already shipped.

## Next

- [Public API](API.md) — the `/v1` endpoints, the response format, the Python client.
- [Development](DEVELOPMENT.md) — running it locally and what CI checks.
- [Comparison](COMPETITIVE.md) — where Praxis wins and where it does not.
- [Architecture](https://github.com/DrobyshevDev/praxis/blob/master/ARCHITECTURE.md) — layers and data flow.

!!! warning "Not legal advice"
Praxis surfaces provisions and checks that a citation supports a claim. It does not
assess your situation, does not account for procedural context, and does not replace
a lawyer. Always verify the revision in force against the official source.
Loading
Loading