Logging uplift for model training purposes (command injection model) [Small change] #6330

dorien-koelemeijer · 2026-01-05T05:38:57Z

Summary

Logging uplift for model training purposes (command injection model). These logs will end up in Snowflake where we'll be able to join on tool_request_id to use the data included in the flagged security finding log and user decision log to generate training data for our command injection model (initial model is created, aim is to use production data for training and fine tuning).

In addition, prefixed prompt injection detection logs for easier manual searching if necessary.

Type of Change

AI Assistance

This PR was created or reviewed with AI assistance

Testing

TBD - testing currently

Copilot

Pull request overview

This PR enhances logging for security findings to support command injection model training. The changes add structured fields and consistent prefixes to logs that will be sent to Snowflake for analysis.

Key Changes

Added threat_type and tool_call_json fields to security finding logs
Added tool_request_id to user decision logs for join capability
Standardized log messages with "Prompt injection detection:" prefix

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
crates/goose/src/security/mod.rs	Enhanced security finding logs with threat_type field and full tool_call JSON serialization; updated log message prefixes for consistency
crates/goose/src/agents/tool_execution.rs	Added tool_request_id to user decision logs to enable joining with security findings in Snowflake

Copilot · 2026-01-05T05:41:22Z

crates/goose/src/security/mod.rs

+                    let tool_call_json =
+                        serde_json::to_string(&tool_call).unwrap_or_else(|_| "{}".to_string());
+
                    tracing::warn!(
                        counter.goose.prompt_injection_finding = 1,
+                        threat_type = "command_injection",
                        above_threshold = above_threshold,
                        tool_name = %tool_call.name,
                        tool_request_id = %tool_request.id,
+                        tool_call_json = %tool_call_json,


Logging the complete tool_call JSON may expose sensitive data. Tool arguments can contain credentials, API keys, file paths, or other sensitive information that could be logged to Snowflake. Consider sanitizing or redacting sensitive fields from tool_call.arguments before serialization, or implement a privacy filter for the logged data.

Suggested change

let tool_call_json =

serde_json::to_string(&tool_call).unwrap_or_else(|_| "{}".to_string());

tracing::warn!(

counter.goose.prompt_injection_finding = 1,

threat_type = "command_injection",

above_threshold = above_threshold,

tool_name = %tool_call.name,

tool_request_id = %tool_request.id,

tool_call_json = %tool_call_json,

tracing::warn!(

counter.goose.prompt_injection_finding = 1,

threat_type = "command_injection",

above_threshold = above_threshold,

tool_name = %tool_call.name,

tool_request_id = %tool_request.id,

Check if any sanitisation happens (i.e. in vector fx), otherwise think of another way + check if these logs go over the public internet

yeah I think the robot makes a good point. can we log something here that allows us to get the actual LLM request back from databricks? that would make sure only people who have access to that can see these potentially PII data

yeah this is problematic, could possibly do name only I would think.

I wonder if this isn't the right level to log things at.

Have set up a sensitive data filter in the OTLP logs here: https://github.com/squareup/ccd-goose-otlp-proxy/pull/12

DOsinga

the robot makes a good point though

DOsinga · 2026-01-05T12:44:02Z

crates/goose/src/security/mod.rs

+                    let tool_call_json =
+                        serde_json::to_string(&tool_call).unwrap_or_else(|_| "{}".to_string());
+
                    tracing::warn!(
                        counter.goose.prompt_injection_finding = 1,
+                        threat_type = "command_injection",
                        above_threshold = above_threshold,
                        tool_name = %tool_call.name,
                        tool_request_id = %tool_request.id,
+                        tool_call_json = %tool_call_json,


yeah I think the robot makes a good point. can we log something here that allows us to get the actual LLM request back from databricks? that would make sure only people who have access to that can see these potentially PII data

zanesq · 2026-01-26T19:16:51Z

@dorien-koelemeijer checking if still in progress?

dorien-koelemeijer · 2026-01-27T03:46:45Z

@dorien-koelemeijer checking if still in progress?

Hi, yes, just about to merge this! Just needed to get some other stuff out, but this can move forward now as well. Thanks for checking!

…odel)

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

…upport * origin/main: (79 commits) fix[format/openai]: return error on empty msg. (#6511) Fix: ElevenLabs API Key Not Persisting (#6557) Logging uplift for model training purposes (command injection model) [Small change] (#6330) fix(goose): only send agent-session-id when a session exists (#6657) BERT-based command injection detection in tool calls (#6599) chore: [CONTRIBUTING.md] add Hermit to instructions (#6518) fix: update Gemini context limits (#6536) Document r slash command (#6724) Upgrade GitHub Actions to latest versions (#6700) fix: Manual compaction does not update context window. (#6682) Removed the Acceptable Usage Policy (#6204) Document spellcheck toggle (#6721) fix: docs workflow cleanup and prevent cancellations (#6713) Docs: file bug directly (#6718) fix: dispatch ADD_ACTIVE_SESSION event before navigating from "View All" (#6679) Speed up Databricks provider init by removing fetch of supported models (#6616) fix: correct typos in documentation and Justfile (#6686) docs: frameDomains and baseUriDomains for mcp apps (#6684) docs: add Remotion video creation tutorial (#6675) docs: export recipe and copy yaml (#6680) ... # Conflicts: # ui/desktop/src/hooks/useChatStream.ts

…ovider * 'main' of github.com:block/goose: fix slash and @ keyboard navigation popover background color (#6550) fix[format/openai]: return error on empty msg. (#6511) Fix: ElevenLabs API Key Not Persisting (#6557) Logging uplift for model training purposes (command injection model) [Small change] (#6330) fix(goose): only send agent-session-id when a session exists (#6657) BERT-based command injection detection in tool calls (#6599) chore: [CONTRIBUTING.md] add Hermit to instructions (#6518) fix: update Gemini context limits (#6536) Document r slash command (#6724) Upgrade GitHub Actions to latest versions (#6700)

* 'main' of github.com:block/goose: Create default gooseignore file when missing (#6498) fix slash and @ keyboard navigation popover background color (#6550) fix[format/openai]: return error on empty msg. (#6511) Fix: ElevenLabs API Key Not Persisting (#6557) Logging uplift for model training purposes (command injection model) [Small change] (#6330) fix(goose): only send agent-session-id when a session exists (#6657) BERT-based command injection detection in tool calls (#6599) chore: [CONTRIBUTING.md] add Hermit to instructions (#6518) fix: update Gemini context limits (#6536) Document r slash command (#6724) Upgrade GitHub Actions to latest versions (#6700) fix: Manual compaction does not update context window. (#6682) Removed the Acceptable Usage Policy (#6204) Document spellcheck toggle (#6721) fix: docs workflow cleanup and prevent cancellations (#6713) Docs: file bug directly (#6718)

Copilot AI review requested due to automatic review settings January 5, 2026 05:38

Copilot started reviewing on behalf of dorien-koelemeijer January 5, 2026 05:39 View session

Copilot AI reviewed Jan 5, 2026

View reviewed changes

dorien-koelemeijer marked this pull request as draft January 5, 2026 06:33

DOsinga approved these changes Jan 5, 2026

View reviewed changes

dorien-koelemeijer marked this pull request as ready for review January 8, 2026 21:33

dorien-koelemeijer force-pushed the feat/uplift-application-logs-for-model-training branch from 51c2927 to f5e17be Compare January 8, 2026 22:06

Small logging uplift for model training purposes (command injection m…

0fa544d

…odel)

Copilot AI review requested due to automatic review settings January 27, 2026 04:06

dorien-koelemeijer force-pushed the feat/uplift-application-logs-for-model-training branch from f5e17be to 0fa544d Compare January 27, 2026 04:06

Copilot started reviewing on behalf of dorien-koelemeijer January 27, 2026 04:07 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

dorien-koelemeijer merged commit e9024b7 into main Jan 27, 2026
23 checks passed

dorien-koelemeijer deleted the feat/uplift-application-logs-for-model-training branch January 27, 2026 05:25

This was referenced Jan 29, 2026

chore(release): release version 1.22.0 (minor) #6812

Closed

chore(release): release version 1.22.0 (minor) #6813

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Logging uplift for model training purposes (command injection model) [Small change] #6330

Logging uplift for model training purposes (command injection model) [Small change] #6330

Uh oh!

dorien-koelemeijer commented Jan 5, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 5, 2026

Uh oh!

dorien-koelemeijer Jan 5, 2026 •

edited

Loading

Uh oh!

DOsinga Jan 5, 2026

Uh oh!

michaelneale Jan 5, 2026

Uh oh!

dorien-koelemeijer Jan 8, 2026

Uh oh!

DOsinga left a comment

Uh oh!

DOsinga Jan 5, 2026

Uh oh!

zanesq commented Jan 26, 2026

Uh oh!

dorien-koelemeijer commented Jan 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Logging uplift for model training purposes (command injection model) [Small change] #6330

Logging uplift for model training purposes (command injection model) [Small change] #6330

Uh oh!

Conversation

dorien-koelemeijer commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

AI Assistance

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

dorien-koelemeijer Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DOsinga Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

michaelneale Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

dorien-koelemeijer Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

DOsinga left a comment

Choose a reason for hiding this comment

Uh oh!

DOsinga Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

zanesq commented Jan 26, 2026

Uh oh!

dorien-koelemeijer commented Jan 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dorien-koelemeijer commented Jan 5, 2026 •

edited

Loading

dorien-koelemeijer Jan 5, 2026 •

edited

Loading