Skip to content

feat(workflows): comprehensive training data, tests, and Ollama synthesis#441

Merged
JerrettDavis merged 1 commit intomainfrom
feat/iss-404-ml-v2
Mar 30, 2026
Merged

feat(workflows): comprehensive training data, tests, and Ollama synthesis#441
JerrettDavis merged 1 commit intomainfrom
feat/iss-404-ml-v2

Conversation

@JerrettDavis
Copy link
Copy Markdown
Owner

Issue #404 follow-up — Training data quality, tests, and AI-assisted synthesis

After merging #440, this PR adds the quality tooling and coverage that was still needed.

What's included

AI-powered training data ( ools/JD.AI.Workflows.Training/)

  • OllamaClient.cs — HTTP client for Ollama REST API (qwen3.5:9b)
  • AiTrainingDataSynthesizer.cs — generates diverse labeled prompts using AI,
    validates existing data by cross-checking against qwen3.5
  • --ollama-generate — generate N new AI-synthesized examples
  • --ollama-validate — audit existing data for AI/label disagreements

Training data improvements

  • 100 new diverse examples generated via qwen3.5 (cross-domain: git, docker, k8s, database, CI/CD, etc.)
  • Merged with existing corpus: 418 total prompts
  • Trained model accuracy: 97.59%

Unit tests ( ests/JD.AI.Tests/Workflows/MlNetIntentClassifierTests.cs)

  • 10 focused tests covering: workflow multi-step, single-action, conversation questions, casual chat, opinion/brainstorm, empty/whitespace/null input, confidence range, hot-swap reload, disposal, sub-millisecond performance

Test results

  • All 5745 tests pass
  • All 10 MlNetIntentClassifier unit tests pass

Closes #404 (follow-up quality work)

…esis

Training tooling:
- OllamaClient: HTTP client for Ollama REST API (qwen3.5:9b)
- AiTrainingDataSynthesizer: generates + validates training data using AI
  (100 new diverse examples generated via Ollama, 97.59% accuracy)
- TrainingDataGenerator: added real JD session examples and expanded seed corpus
- Program.cs: --ollama-generate and --ollama-validate commands

Unit tests:
- MlNetIntentClassifierTests: 10 tests covering workflow/conversation edge cases,
  empty/whitespace/null input, hot-swap reload, performance, disposal
- All 5745 tests pass
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Deprecation Warning: The deny-licenses option is deprecated for possible removal in the next major release. For more information, see issue 997.

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 210a190.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

@JerrettDavis JerrettDavis merged commit f011fc4 into main Mar 30, 2026
7 checks passed
JerrettDavis added a commit that referenced this pull request Mar 31, 2026
feat(workflows): comprehensive training data, tests, and Ollama synthesis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] ML.NET workflow intent classifier — trained model upgrade path

1 participant