fix: handle non-ASCII owner_name and owner_email in Langflow ingestion metadata

[coderabbitai]

Summary

During review of PR #1857 (fix: fix ingestion without page number), it was identified that ingestion may fail when owner_name or owner_email contains non-ASCII characters (e.g., accented letters, CJK characters, Greek characters, etc.).

Problem

owner_name and owner_email are passed via X-Langflow-Global-Var-* HTTP headers in src/services/langflow_file_service.py. HTTP headers must comply with RFC 7230 (ISO-8859-1 / ASCII), so passing non-ASCII values directly without encoding will cause encoding errors or silently corrupt the data.

Example values that would currently break:

owner_name = "José Łukasz 山田"
owner_email = "δοκιμή@παράδειγμα.δοκιμή"

Expected Behaviour

Non-ASCII owner_name and owner_email values should be safely encoded (e.g., percent-encoded or base64-encoded) when passed through HTTP headers, and correctly decoded on the receiving end, so ingestion proceeds without errors and the metadata is preserved faithfully.

Suggested Fix

• Encode owner_name and owner_email using a UTF-8-safe mechanism (e.g., urllib.parse.quote or RFC 5987 encoding) when setting the X-Langflow-Global-Var-* headers in src/services/langflow_file_service.py.
• Add a corresponding test verifying ingestion succeeds and metadata is preserved for non-ASCII owner names and emails.

References

• PR fix: fix ingestion without page number #1857: fix/langflow_ingest_ascii → main
• Comment: fix: fix ingestion without page number #1857 (comment) (raised by @edwinjosechittilappilly)

[edwinjosechittilappilly]

@coderabbitai Check how this could be handled, Investigate scenarios that the current code will raise issue. Also Propose a fix suggestion also.

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch the issue details
gh issue view 1858 --repo langflow-ai/openrag --json title,body,comments,labels,assignees

Length of output: 2517

---

🏁 Script executed:

#!/bin/bash
# Find and read the relevant file
fd "langflow_file_service.py" --exec cat {}

echo "---SEPARATOR---"

# Also search for X-Langflow-Global-Var header usage
rg "X-Langflow-Global-Var" -n -C 5

echo "---SEPARATOR---"

# Search for owner_name and owner_email usage
rg "owner_name|owner_email" -n -C 5

echo "---SEPARATOR---"

# Look for any existing encoding/decoding of header values
rg "quote|encode|decode|base64|RFC" -n -C 3 --type py

Length of output: 268419

---

---

@edwinjosechittilappilly, here is a thorough investigation of the issue, all the scenarios where current code breaks, and a concrete fix proposal.

---

Root Cause

owner_name and owner_email (and a few other fields) are written verbatim into X-Langflow-Global-Var-* HTTP headers:

src/services/langflow_file_service.py — run_ingestion_flow() (Lines ~423–427):

headers = {
    ...
    "X-Langflow-Global-Var-OWNER_NAME": str(owner_name),   # ← raw, no encoding
    "X-Langflow-Global-Var-OWNER_EMAIL": str(owner_email), # ← raw, no encoding
    ...
}

run_url_ingestion_flow() (Lines ~598–601): Same pattern.

Per RFC 7230 §3.2.6, HTTP header field values are restricted to visible ASCII octets (0x21–0x7E) plus SP and HTAB. Any character with a Unicode codepoint > 127 that isn't representable in ISO-8859-1 (codepoint > 255) will raise an exception; characters in 128–255 may silently corrupt.

---

Scenarios That Break the Current Code

Scenario	Example Value	What Fails
Accented Latin (> 0xFF)	owner_name = "Łukasz Wójcik"	UnicodeEncodeError — Ł is U+0141, outside ISO-8859-1
CJK characters	owner_name = "山田 太郎"	UnicodeEncodeError — all CJK codepoints > 0xFF
Greek / Cyrillic	owner_email = "δοκιμή@παράδειγμα.δοκιμή"	UnicodeEncodeError
Latin extended (within 0x80–0xFF)	owner_name = "José" (é = U+00E9)	Sent as raw byte 0xE9, likely misread by Langflow as Latin-1 instead of UTF-8 → silent data corruption
Non-ASCII filename	filename = "résumé.pdf"	Same problem — X-Langflow-Global-Var-FILENAME is also unencoded
IDN / non-ASCII source URL	source_url = "https://例え.jp/doc"	X-Langflow-Global-Var-SOURCE_URL is also raw

httpx (used via clients.langflow_request) internally delegates header encoding to Python's http.client, which will raise UnicodeEncodeError for codepoints > 255, causing the entire ingestion request to fail with an unhandled exception — and the document will not be indexed.

---

Proposed Fix

Strategy: percent-encode on send, percent-decode on receive

Use urllib.parse.quote (UTF-8 percent-encoding) for all string header values that can contain user-supplied text. Langflow flows that read these globals must call urllib.parse.unquote before using them.

1 — Add a small helper in langflow_file_service.py

from urllib.parse import quote, unquote  # add to imports at top of file

def _encode_header_value(value: str | None) -> str:
    """Percent-encode a header value so it is safe for RFC 7230 transmission."""
    return quote(str(value) if value is not None else "", safe="")

2 — Apply encoding in run_ingestion_flow()

headers = {
    "X-Langflow-Global-Var-JWT": str(jwt_token or ""),
    "X-Langflow-Global-Var-OWNER": _encode_header_value(owner),
    "X-Langflow-Global-Var-OWNER_NAME": _encode_header_value(owner_name),
    "X-Langflow-Global-Var-OWNER_EMAIL": _encode_header_value(owner_email),
    "X-Langflow-Global-Var-CONNECTOR_TYPE": _encode_header_value(connector_type),
    "X-Langflow-Global-Var-FILENAME": _encode_header_value(filename),
    "X-Langflow-Global-Var-MIMETYPE": _encode_header_value(mimetype),
    "X-Langflow-Global-Var-FILESIZE": str(file_size_bytes),
    "X-Langflow-Global-Var-SELECTED_EMBEDDING_MODEL": _encode_header_value(embedding_model),
    "X-Langflow-Global-Var-DOCUMENT_ID": resolved_document_id,
    "X-Langflow-Global-Var-SOURCE_URL": _encode_header_value(source_url) if source_url else "",
    "X-Langflow-Global-Var-DOCLING_TASK_ID": _encode_header_value(docling_task_id) if docling_task_id else "",
    "X-Langflow-Global-Var-DOCLING_SERVE_VERIFY_SSL": str(DOCLING_SERVE_VERIFY_SSL).lower(),
}

3 — Apply encoding in run_url_ingestion_flow()

headers = {
    "X-Langflow-Global-Var-JWT": str(jwt_token or ""),
    "X-Langflow-Global-Var-OWNER": _encode_header_value(owner),
    "X-Langflow-Global-Var-OWNER_NAME": _encode_header_value(owner_name),
    "X-Langflow-Global-Var-OWNER_EMAIL": _encode_header_value(owner_email),
    "X-Langflow-Global-Var-CONNECTOR_TYPE": _encode_header_value(connector_type),
    "X-Langflow-Global-Var-SELECTED_EMBEDDING_MODEL": _encode_header_value(embedding_model),
    "X-Langflow-Global-Var-DOCUMENT_ID": resolved_document_id,
    "X-Langflow-Global-Var-SOURCE_URL": _encode_header_value(docs_url),
    "X-Langflow-Global-Var-FILENAME": _encode_header_value(docs_url),
    ...
}

4 — Decode in the Langflow flow component (flows/components/opensearch_multimodal.py)

Wherever the global variables OWNER_NAME, OWNER_EMAIL, FILENAME, etc. are consumed, add decoding:

from urllib.parse import unquote

owner_name = unquote(self.owner_name or "")
owner_email = unquote(self.owner_email or "")

---

Additional Recommendation: Add a Unit Test

`@pytest.mark.asyncio`
async def test_non_ascii_owner_name_and_email(monkeypatch, ...):
    """Ingestion must succeed and preserve metadata for non-ASCII owner info."""
    result = await service.run_ingestion_flow(
        file_paths=["/tmp/source.pdf"],
        file_tuples=[("résumé.pdf", b"content", "application/pdf")],
        owner_name="José Łukasz 山田",
        owner_email="δοκιμή@παράδειγμα.δοκιμή",
        ...
    )
    headers = captured["headers"]
    # All header values must be pure ASCII after encoding
    for k, v in headers.items():
        assert v.isascii(), f"Header {k} contains non-ASCII: {v!r}"
    # Decoding must round-trip faithfully
    from urllib.parse import unquote
    assert unquote(headers["X-Langflow-Global-Var-OWNER_NAME"]) == "José Łukasz 山田"
    assert unquote(headers["X-Langflow-Global-Var-OWNER_EMAIL"]) == "δοκιμή@παράδειγμα.δοκιμή"

---

Summary

Where	Problem	Fix
run_ingestion_flow() headers	Raw non-ASCII → UnicodeEncodeError or silent corruption	_encode_header_value() using urllib.parse.quote
run_url_ingestion_flow() headers	Same	Same
X-Langflow-Global-Var-FILENAME	Non-ASCII filenames (e.g. CJK, accented)	Same encoding
X-Langflow-Global-Var-SOURCE_URL	IDN / non-ASCII URLs	Same encoding
Flow consumer (opensearch_multimodal.py)	Receives percent-encoded strings	urllib.parse.unquote before use

[edwinjosechittilappilly]

@coderabbitai can we fix this at the JWT level in saas? also without any of this fix will this also effect the DLS in opensearch since DLS depends on Boolean checks on UserID

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "## Issue details"
gh issue view 1858 --repo langflow-ai/openrag --json title,body,comments --jq '{title, body, comments: [.comments[] | {author: .author.login, body: .body}]}'

echo
echo "## Files named like langflow_file_service.py"
fd "langflow_file_service.py" --exec sh -c 'echo "--- {}"; sed -n "1,220p" "{}"; echo; sed -n "360,680p" "{}"' \;

echo
echo "## X-Langflow global var headers/usages"
rg -n -C 4 'X-Langflow-Global-Var-|Global-Var|OWNER_NAME|OWNER_EMAIL|DOCUMENT_ID|SOURCE_URL|USER_ID|USERID|user_id|owner_name|owner_email' --type py

echo
echo "## OpenSearch / DLS / permissions / owner/user filters"
rg -n -C 5 'DLS|dls|OpenSearch|opensearch|owner|user_id|userId|userid|USER_ID|Boolean|bool|term|terms|must|filter' --type py

echo
echo "## JWT encode/decode/claims usage"
rg -n -C 5 'jwt|JWT|decode_token|encode|claims|sub|email|owner|user_id|userid|USER_ID' --type py

Length of output: 283896

---

---

@edwinjosechittilappilly short answer: not fully at the SaaS JWT level, unless we stop sending the raw metadata headers. And DLS should not be impacted by owner_name / owner_email specifically, because DLS is tied to user_id / principals, not display name/email metadata.

1. Can this be fixed at SaaS JWT level?

Partially, but not as the code is currently written.

The JWT itself is safe to put in an HTTP header because it is base64url-encoded ASCII. Even if the JWT payload contains:

{
  "name": "José Łukasz 山田",
  "email": "δοκιμή@παράδειγμα.δοκιμή"
}

the serialized JWT header value remains ASCII.

But the current failure is caused by separate raw Langflow global-var headers, for example:

"X-Langflow-Global-Var-OWNER_NAME": str(owner_name),
"X-Langflow-Global-Var-OWNER_EMAIL": str(owner_email),

So even if SaaS puts corrected/encoded values inside the JWT, the request can still fail before Langflow gets a chance to decode the JWT, because these raw headers are still sent.

JWT-level fix is viable only if we change the contract

A SaaS/JWT-level design can work if we do one of these:

Option A — derive owner metadata from JWT inside Langflow

Stop sending these raw headers:

X-Langflow-Global-Var-OWNER_NAME
X-Langflow-Global-Var-OWNER_EMAIL

Then have the Langflow/OpenSearch component derive owner_name and owner_email from the JWT claims.

That avoids the Unicode header issue for name/email, but it still does not solve non-ASCII filename, source_url, etc. Those are not naturally JWT-owned fields.

Option B — put ingestion metadata in the backend ingest token/context

This repo already has a backend-owned ingest callback path via OPENRAG_INGEST_TOKEN, and src/services/langflow_ingest_token_service.py stores metadata like:

"owner": context.owner,
"owner_name": context.owner_name,
"owner_email": context.owner_email,
"filename": context.filename,
"source_url": context.source_url,

That token is also header-safe because it is serialized as a token string. A cleaner architecture would be:

• send only ASCII-safe headers:
  • X-Langflow-Global-Var-JWT
  • X-Langflow-Global-Var-OPENRAG_INGEST_TOKEN
  • X-Langflow-Global-Var-OPENRAG_INGEST_URL
  • IDs / booleans / numeric fields
• avoid sending raw owner_name, owner_email, filename, source_url as separate headers
• have the backend callback/token context be the source of truth for ingestion metadata

This is better than relying on the SaaS user JWT because it also covers file-specific fields.

2. Will this affect OpenSearch DLS?

For the reported issue — non-ASCII owner_name or owner_email — DLS should not be directly affected.

From the code path:

• JWT claims include sub and user_id from user.user_id.
• Search uses session_manager.get_user_opensearch_client(user_id, jwt_token).
• DLS principal refresh stores/uses auth_user_id, user.user_id, and OpenSearch username/principals.
• ACL lists are serialized with json.dumps(...), which defaults to ASCII escaping, so those list headers are less likely to hit this Unicode header issue.

So if only owner_name = "José" or owner_email = "δοκιμή@...", the main risks are:

1. ingestion request fails before indexing, or
2. display/search metadata is corrupted if it somehow passes through.

It should not change DLS authorization logic, because DLS is not doing Boolean checks on owner_name or owner_email.

Important caveat: owner / user_id

If owner / user_id itself can contain non-ASCII and is sent here:

"X-Langflow-Global-Var-OWNER": str(owner),

then the same header issue can affect ingestion. That could lead to missing or wrong owner metadata on indexed docs.

However, the OpenSearch DLS transport path using JWT is still header-safe because the JWT string is ASCII. The concern would be consistency between:

• JWT sub / user_id
• indexed owner
• allowed_users
• allowed_principals
• DLS principal lookup rows

If user_id is ASCII/stable, DLS should be fine.

Recommendation

I would not rely only on a SaaS JWT fix. Best fix:

1. Keep JWT as the auth/DLS source of truth.
2. Stop putting user-visible/free-text metadata directly into X-Langflow-Global-Var-* headers.
3. Either:
   • percent/base64 encode all non-token metadata headers, or
   • better: move metadata into OPENRAG_INGEST_TOKEN / backend ingest context and make Langflow consume metadata from there.
4. Add tests for:
   • non-ASCII owner_name
   • non-ASCII owner_email
   • non-ASCII filename
   • non-ASCII source_url
   • DLS still filtering on user_id / principals, not display metadata.

So: JWT-level can be part of the solution, but the current raw metadata headers still need to be removed or encoded. DLS based on UserID should not be affected by non-ASCII name/email unless UserID/owner itself is also non-ASCII or inconsistently mapped.

[philnash]

This is an issue because Langflow requires global variables to be set in HTTP headers, and HTTP headers are not supposed to use anything outside of ASCII. You could percent encode the values, and decode them again, but that's not the ideal outcome here.

Instead, I raised this issue with Langflow to ask that global variables can be sent within the body of the request rather than as headers. This has been completed and merged, and looks to be part of Langflow 1.10.0.

So, the fix is to upgrade Langflow and use the new globals property in the request body for all global variables.