An interactive Streamlit app that visualises token-level log probabilities from OpenAI, Azure OpenAI (Azure AI Foundry), or GitHub Models.
Built as an educational tool to help explain how Large Language Models generate text.
- Python 3.10+
- One of the following:
- An OpenAI API key, or
- An Azure AI Foundry / Azure OpenAI resource with a deployed chat model, or
- A GitHub Personal Access Token (PAT) for GitHub Models
- Explains context-window usage — shows how much of the selected model's context window your prompt consumes, plus remaining headroom for output.
- Tokenises your prompt locally with
tiktoken— supports OpenAI model tokenizers and shows either token text or token IDs. - Sends the prompt to an OpenAI, Azure OpenAI, or GitHub Models chat model with
logprobsenabled. - Displays the response token-by-token with:
- Colour-coded confidence badges (green → red)
- A sortable data table with log probs, percentages, and top alternatives
- An interactive Plotly bar chart of probabilities across the full response
- Per-token expanders showing what other tokens the model considered
pip install -r requirements.txtCreate (or edit) a .env file in the project root. Choose the section that matches your provider:
PROVIDER=openai
OPENAI_API_KEY=sk-...your-openai-api-key...
LOGPROB_MODEL=gpt-4o # any model that supports logprobsPROVIDER=azure
OPENAI_API_KEY=<your-azure-openai-key>
AZURE_OPENAI_ENDPOINT=https://<your-resource>.cognitiveservices.azure.com
API_VERSION=2025-04-01-preview
LOGPROB_MODEL=gpt-5.2 # must match your Azure deployment namePROVIDER=github
GITHUB_TOKEN=ghp_...your-github-pat...
LOGPROB_MODEL=gpt-4o # any model available on GitHub ModelsTo create a PAT, visit github.com/settings/tokens and generate a token. If GITHUB_TOKEN is not set, the app falls back to OPENAI_API_KEY.
Notes:
PROVIDERacceptsazure,openai, orgithub. If omitted, the legacyUSE_AZUREtoggle is used for backward compatibility.- For Azure, the endpoint should be the base URL of your Azure Cognitive Services resource — the SDK builds the full Chat Completions path automatically.
- For OpenAI / GitHub Models,
LOGPROB_MODELshould be a model ID (e.g.gpt-4o,gpt-4o-mini). For Azure, it should match the deployment name in your Azure resource.- GitHub Models uses the endpoint
https://models.inference.ai.azure.comwhich is handled automatically.
python -m streamlit run logprob_demo.pyThe app opens in your browser at http://localhost:8501.
The Tokenizer tab is focused on prompt size and context-window planning:
- The large context bar shows
prompt tokens / context windowfor the selected OpenAI model. - The vertical marker reserves the model's max-output budget.
- Free in context is total remaining room for prompt text, retrieved documents, tool calls, and generated output.
- Headroom for output is the remaining safe prompt budget after reserving max output.
- The token stream can display either readable token text or numeric token IDs. The full stream is optional for long prompts.
| Colour | Probability | Meaning |
|---|---|---|
| 🟩 Green | > 90 % | Model was very confident |
| 🟨 Yellow | 50 – 90 % | Fairly confident |
| 🟧 Orange | 10 – 50 % | Multiple plausible options |
| 🟥 Red | < 10 % | Low confidence / surprising choice |
A log probability is the natural logarithm of the model's predicted probability for a token:
probability = e^(logprob)
logprob = 0.00→ 100 % confidencelogprob = −0.69→ ~50 %logprob = −2.30→ ~10 %
The closer to zero, the more certain the model was.
llm-logprob-demo/
├── .env # API credentials (not committed)
├── logprob_demo.py # Streamlit application
├── requirements.txt # Python dependencies
└── README.md # This file
- Streamlit — Web UI
- OpenAI Python SDK — Supports
OpenAI,AzureOpenAI, and GitHub Models clients - tiktoken — BPE tokeniser for OpenAI prompt visualisation and token IDs
- Plotly — Interactive charts
- python-dotenv —
.envfile loading
