Skip to content

fix(ingestion): encode the pipeline FQN in log URLs - #37262

Merged
manerow merged 1 commit into
mainfrom
fix/logs-mixin-encode-fqn
Nov 6, 2026
Merged

manerow merged 1 commit into
mainfrom
fix/logs-mixin-encode-fqn

Conversation

@manerow

@manerow manerow commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #37265

Problem

Agents send their logs to /services/ingestionPipelines/logs/{fqn}/{runId}, and the SDK put the pipeline's fully qualified name into that URL as it was. A service or agent name may contain /, for example Snowflake : Product/Eng NA. That slash adds an extra path segment, so no route matches and the server answers 404.

Sending logs is best effort, so nothing reports the failure. The run finishes as a success and its logs are lost.

A ? or # in a name breaks the same way, because each one ends the URL path early.

What this PR changes

Every log call in the SDK now encodes the name with the existing quote helper, the same one the SDK already uses for entity lookups by name. The / is sent as %2F, and the server decodes it back to the real name.

Names with only letters, digits, dots and underscores produce the same URL as before.

What users will notice

Agents on a service whose name contains / now ship their logs, so the log viewer has something to show.

How it was tested

  • Unit tests cover all six log calls with a name containing /, ?, #, %, quotes and spaces. They fail without this change.
  • End to end with the real OpenMetadata client and StreamableLogHandler, in the same startup order as a workflow, on a real service named logslash : Product/Eng NA:
    • The old writer sent the raw slash and got a 404.
    • The new writer shipped its records with no failed posts, closed the run, and the lines showed up in the UI log viewer.
  • Against the server's own log endpoint: the raw slash finds no route (404), and the encoded name reaches the write handler. That check used a pipeline that does not exist, so it proves routing only. The successful write above went through a separate log backend behind a load balancer, not the server's own log storage.

Not in this PR

Agents already deployed keep the old behaviour until they run an SDK with this change.

@manerow
manerow requested a review from a team as a code owner November 6, 2026 12:28
@manerow
manerow requested a review from mohittilala November 6, 2026 12:28
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

@github-actions github-actions Bot added backend safe to test Add this label to run secure Github workflows on PRs labels Nov 6, 2026
@gitar-bot

gitar-bot Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Code Review ✅ Approved

🟡 Medium risk · Encoding pipeline FQNs across log endpoints changes request routing; incompatible decoding could cause log delivery or retrieval to fail.

Encodes the pipeline FQN with the existing quote helper in all six SDK log calls, so names containing /, ?, #, %, quotes, or spaces no longer break the /services/ingestionPipelines/logs/{fqn}/{runId} route. No issues found.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

@manerow
manerow merged commit cefbc5a into main Nov 6, 2026
64 of 66 checks passed
@manerow
manerow deleted the fix/logs-mixin-encode-fqn branch November 6, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingestion log URLs break when a pipeline FQN contains /

2 participants