proxy,proxy/config: add global TTL feature

[mostlygeek]

Add a new configuration parameter globalTTL that all models will
inherit. The default value is 0 which matches the currently
functionality to never automatically unload a model.

The model.ttl's default has changed to -1, which means use the global
TTL value. Any model.ttl >=0 is now value with 0 meaning never unload.
This allows a model to override a globalTTL > 0 and be configured to
never unload.

Fixes #459
Closes #512

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0d4349 and 72d91fe.

📒 Files selected for processing (2)

• config.example.yaml
• proxy/config/config.go

🚧 Files skipped from review as they are similar to previous changes (1)

• config.example.yaml

---

Walkthrough

Adds a top-level globalTTL config and changes per-model ttl semantics so -1 means "inherit globalTTL", 0 disables unloading, and positive values set per-model TTL. Validation and tests updated; a package constant for the default model TTL was introduced.

Changes

Cohort / File(s)	Summary
Schema & Example config-schema.json, config.example.yaml	Added top-level globalTTL (integer, >=0, default 0). Updated model ttl to allow -1 (inherit global), 0 (disable unloading), and positive values; adjusted defaults and descriptions.
Config Struct & Loader proxy/config/config.go	Added GlobalTTL int \yaml:"globalTTL"`to Config, initialize default, validateGlobalTTL >= 0`, apply global TTL to models whose TTL equals the model default sentinel, and reject negative per-model TTLs.
Model Config Defaults proxy/config/model_config.go	Added MODEL_CONFIG_DEFAULT_TTL = -1 constant and updated ModelConfig YAML unmarshalling to use this sentinel default.
Config Tests proxy/config/config_test.go	Added tests for globalTTL inheritance, model override behaviors, defaulting to 0 when absent, and rejection of negative globalTTL.
Process & ProxyManager Tests proxy/process_test.go, proxy/proxymanager_test.go	Updated tests to reference MODEL_CONFIG_DEFAULT_TTL instead of hardcoded 0 and adjusted expectations where default TTL semantics changed to -1.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding a global TTL feature to the proxy and config packages.
Description check	✅ Passed	The description is directly related to the changeset, explaining the new globalTTL parameter and its interaction with model-level TTL values.
Linked Issues check	✅ Passed	The code changes fully implement the requirements from both linked issues: adding global TTL configuration, allowing per-model overrides, rejecting negative values, and updating schema/tests.
Out of Scope Changes check	✅ Passed	All changes are directly related to the global TTL feature implementation across schema, config, and tests; no out-of-scope modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch global-ttl-459

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@config.example.yaml`:
- Around line 188-193: Update the TTL documentation comments for the "ttl" field
so they are consistent: state that -1 => inherit globalTTL (use the global TTL
value), 0 => never unload, and >0 => unload after N seconds; remove the
contradictory "default: -1 (never unload)" and the incorrect "ttl must be >= 0"
line and ensure the remaining bullets reference the "ttl" key and "globalTTL"
clearly (e.g., "ttl: -1 inherits globalTTL; ttl: 0 never unloads; ttl: N (>0)
unloads after N seconds").

In `@proxy/config/config.go`:
- Around line 264-267: LoadConfigFromReader currently normalizes
MODEL_CONFIG_DEFAULT_TTL but allows invalid negative TTLs like -2; add
validation in LoadConfigFromReader after unmarshalling to reject any
modelConfig.UnloadAfter < -1 (return an error) and keep the existing
normalization that maps MODEL_CONFIG_DEFAULT_TTL to config.GlobalTTL; update the
code paths that construct/validate model configs (look for LoadConfigFromReader,
MODEL_CONFIG_DEFAULT_TTL, modelConfig.UnloadAfter, and config.GlobalTTL) so
invalid TTLs produce a clear error instead of being accepted.

---

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 390a35b and d0d4349.

📒 Files selected for processing (7)

• config-schema.json
• config.example.yaml
• proxy/config/config.go
• proxy/config/config_test.go
• proxy/config/model_config.go
• proxy/process_test.go
• proxy/proxymanager_test.go

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Reject invalid model ttl values below -1.

LoadConfigFromReader currently normalizes only -1, but accepts values like ttl: -2 without error. That violates the intended contract and can leak invalid runtime behavior.

💡 Suggested fix

 		// set model TTL to globalTTL it is the default value
+		if modelConfig.UnloadAfter < MODEL_CONFIG_DEFAULT_TTL {
+			return Config{}, fmt.Errorf("model %s: ttl must be >= -1", modelId)
+		}
 		if modelConfig.UnloadAfter == MODEL_CONFIG_DEFAULT_TTL {
 			modelConfig.UnloadAfter = config.GlobalTTL
 		}

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	// set model TTL to globalTTL it is the default value
	if modelConfig.UnloadAfter == MODEL_CONFIG_DEFAULT_TTL {
	modelConfig.UnloadAfter = config.GlobalTTL
	}
	// set model TTL to globalTTL it is the default value
	if modelConfig.UnloadAfter < MODEL_CONFIG_DEFAULT_TTL {
	return Config{}, fmt.Errorf("model %s: ttl must be >= -1", modelId)
	}
	if modelConfig.UnloadAfter == MODEL_CONFIG_DEFAULT_TTL {
	modelConfig.UnloadAfter = config.GlobalTTL
	}

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@proxy/config/config.go` around lines 264 - 267, LoadConfigFromReader
currently normalizes MODEL_CONFIG_DEFAULT_TTL but allows invalid negative TTLs
like -2; add validation in LoadConfigFromReader after unmarshalling to reject
any modelConfig.UnloadAfter < -1 (return an error) and keep the existing
normalization that maps MODEL_CONFIG_DEFAULT_TTL to config.GlobalTTL; update the
code paths that construct/validate model configs (look for LoadConfigFromReader,
MODEL_CONFIG_DEFAULT_TTL, modelConfig.UnloadAfter, and config.GlobalTTL) so
invalid TTLs produce a clear error instead of being accepted.