You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/src/content/docs/llm/application.mdx
+21-5Lines changed: 21 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -62,7 +62,7 @@ export namespace llm {
62
62
63
63
LLM function calling application schema from a native TypeScript class or interface type.
64
64
65
-
`typia.llm.application<App>()` is a function composing LLM (Large Language Model) calling application schema from a native TypeScript class or interface type. The function returns an `ILlmApplication` instance, which is a data structure representing a collection of LLM function calling schemas.
65
+
`typia.llm.application<App>()` is a function composing LLM (Large Language Model) calling application schema from a native TypeScript class or interface type. The function returns an `ILlmApplication` instance, which is a data structure representing a collection of LLM function calling schemas — each with built-in `parse()`, `coerce()`, and `validate()` methods.
66
66
67
67
If you put LLM function schema instances registered in the `ILlmApplication.functions` to the LLM provider like `OpenAI ChatGPT`, the LLM will select a proper function to call with parameter values of the target function in the conversations with the user. This is the "LLM Function Calling".
68
68
@@ -207,7 +207,17 @@ registerMcpControllers({
207
207
</Tabs.Tab>
208
208
</Tabs>
209
209
210
-
## Lenient JSON Parsing
210
+
## The Function Calling Harness
211
+
212
+
The **function calling harness** is typia's three-layer pipeline that turns unreliable LLM output into 100% correct structured data:
3.**Validation Feedback** — pinpoints remaining value errors with inline `// ❌` annotations so the LLM can self-correct and retry
217
+
218
+
Each layer catches what the previous one didn't. Together they form a deterministic correction loop around the probabilistic LLM.
219
+
220
+
### Lenient JSON Parsing & Type Coercion
211
221
212
222
<Tabsitems={[
213
223
"Parsing Example",
@@ -266,7 +276,7 @@ Some LLM SDKs (Anthropic, Vercel AI, LangChain, MCP) parse JSON internally and r
266
276
For more details, see [JSON Utilities](./json).
267
277
</Callout>
268
278
269
-
## Validation Feedback
279
+
###Validation Feedback
270
280
271
281
<LocalSource
272
282
path="examples/src/llm/application-validate.ts"
@@ -296,15 +306,21 @@ For more details, see [JSON Utilities](./json).
296
306
}
297
307
```
298
308
299
-
The LLM reads this feedback and self-corrects on the next turn.
309
+
The LLM reads this feedback and self-corrects on the next turn. Together with lenient parsing and type coercion above, this parse → coerce → validate → feedback → retry cycle completes the harness.
300
310
301
-
In the [AutoBe](https://github.com/wrtnlabs/autobe) project (AI-powered backend code generator), `qwen3-coder-next` showed only 6.75% raw function calling success rate on compiler AST types. However, with validation feedback, it reached 100%.
311
+
<Callouttype="info">
312
+
**In Production**
313
+
314
+
In the [AutoBe](https://github.com/wrtnlabs/autobe) project (AI-powered backend code generator by [Wrtn Technologies](https://wrtn.io)), `qwen3-coder-next` showed only **6.75%** raw function calling success rate on compiler AST types. With the complete harness, it reached **100%** — across all four tested Qwen models.
315
+
316
+
AutoBe once shipped a build with the system prompt completely missing. Nobody noticed — output quality was identical. The types were the best prompt; the harness was the best orchestration.
302
317
303
318
Working on compiler AST means working on any type and any use case.
The answer is not, and LLM (Large Language Model) vendors like OpenAI take a lot of type level mistakes when composing the arguments of the target function to call. Even though an LLM function calling schema has defined an `Array<string>` type, LLM often fills it just by a `string` typed value.
230
+
The answer is no, and LLM (Large Language Model) vendors like OpenAI take a lot of type level mistakes when composing the arguments of the target function to call. Even though an LLM function calling schema has defined an `Array<string>` type, LLM often fills it just by a `string` typed value. This is where the **function calling harness** comes in — a deterministic correction loop of schema generation, lenient parsing, type coercion, and validation feedback that turns unreliable LLM output into 100% correct structured data.
231
231
232
232
Therefore, when developing an LLM function calling agent, the validation feedback process is essentially required. If LLM takes a type level mistake on arguments composition, the agent must feedback the most detailed validation errors, and let the LLM to retry the function calling referencing the validation errors.
233
233
234
234
About the validation feedback, `@agentica/core` is utilizing [`typia.validate<T>()`](https://typia.io/docs/validators/validate) and [`typia.llm.application<Class>()`](https://typia.io/docs/llm/application/#application) functions. They construct validation logic by analyzing TypeScript source codes and types in the compilation level, so that detailed and accurate than any other validators like below.
235
235
236
-
Such validation feedback strategy and combination with `typia` runtime validator, `@agentica/core` has achieved the most ideal LLM function calling. In my experience, when using OpenAI's `gpt-4o-mini` model, it tends to construct invalid function calling arguments at the first trial about 50% of the time. By the way, if correct it through validation feedback with `typia`, success rate soars to 99%. And I've never had a failure when trying validation feedback twice.
236
+
Such validation feedback strategy and combination with `typia` runtime validator, `@agentica/core` has achieved the most ideal LLM function calling through the **function calling harness** pattern. In my experience, when using OpenAI's `gpt-4o-mini` model, it tends to construct invalid function calling arguments at the first trial about 50% of the time. By the way, if you correct it through validation feedback with `typia`, success rate soars to 99%. And I've never had a failure when trying validation feedback twice.
237
237
238
238
For reference, the embedded [`typia.validate<T>()`](/docs/validators/validate) function creates validation logic by analyzing TypeScript source codes and types in the compilation level. Therefore, it is accurate and detailed than any other validator libraries. This is exactly what is needed for function calling, and I can confidentelly say that `typia` is the best library for LLM function calling.
Copy file name to clipboardExpand all lines: website/src/content/docs/llm/http.mdx
+14-4Lines changed: 14 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -60,7 +60,7 @@ export namespace HttpLlm {
60
60
61
61
LLM function calling from OpenAPI documents.
62
62
63
-
`HttpLlm` is a utility module from `@typia/utils` that converts OpenAPI (Swagger) documents into LLM function calling schemas. While [`typia.llm.application<Class>()`](./application) generates schemas from TypeScript class types at compile time, `HttpLlm` generates them from OpenAPI documents at runtime — making any REST API instantly callable by LLMs.
63
+
`HttpLlm` is a utility module from `@typia/utils` that converts OpenAPI (Swagger) documents into LLM function calling schemas. While [`typia.llm.application<Class>()`](./application) generates schemas from TypeScript class types at compile time, `HttpLlm` generates them from OpenAPI documents at runtime — making any REST API instantly callable by LLMs. Every generated tool includes lenient parsing, type coercion, and validation feedback.
64
64
65
65
It supports all OpenAPI versions: Swagger v2.0, OpenAPI v3.0, v3.1, and v3.2.
66
66
@@ -215,7 +215,17 @@ registerMcpControllers({
215
215
</Tabs.Tab>
216
216
</Tabs>
217
217
218
-
## Validation Feedback
218
+
## The Function Calling Harness
219
+
220
+
The **function calling harness** is typia's three-layer pipeline that turns unreliable LLM output into 100% correct structured data:
3.**Validation Feedback** — pinpoints remaining value errors with inline `// ❌` annotations so the LLM can self-correct and retry
225
+
226
+
Each layer catches what the previous one didn't. Together they form a deterministic correction loop around the probabilistic LLM.
227
+
228
+
### Validation Feedback
219
229
220
230
When used through [MCP](/docs/llm/mcp), [Vercel AI SDK](/docs/llm/vercel), or [Agentica](/docs/llm/chat), `HttpLlm.controller()` embeds [`typia.validate<T>()`](/docs/validators/validate) in every tool for automatic argument validation. When validation fails, the error is returned as text content with inline `// ❌` comments at each invalid property:
221
231
@@ -230,7 +240,7 @@ When used through [MCP](/docs/llm/mcp), [Vercel AI SDK](/docs/llm/vercel), or [A
230
240
231
241
The LLM reads this feedback and self-corrects on the next turn.
232
242
233
-
In the [AutoBe](https://github.com/wrtnlabs/autobe) project (AI-powered backend code generator), `qwen3-coder-next` showed only 6.75% raw function calling success rate on compiler AST types. However, with validation feedback, it reached 100%.
243
+
In the [AutoBe](https://github.com/wrtnlabs/autobe) project (AI-powered backend code generator by [Wrtn Technologies](https://wrtn.io)), `qwen3-coder-next` showed only **6.75%** raw function calling success rate on compiler AST types. However, with the complete harness, it reached **100%** — across all four tested Qwen models.
234
244
235
245
Working on compiler AST means working on any type and any use case.
Copy file name to clipboardExpand all lines: website/src/content/docs/llm/json.mdx
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,7 +51,7 @@ export namespace LlmJson {
51
51
52
52
JSON utilities for LLM function calling.
53
53
54
-
`LlmJson` is a utility module from `@typia/utils` package, specifically designed for LLM (Large Language Model) function calling scenarios. It handles the common issues that arise when working with LLM responses:
54
+
`LlmJson` is a utility module from `@typia/utils` package, specifically designed for LLM (Large Language Model) function calling scenarios. Together, these utilities form the **function calling harness** — handling every common failure mode of LLM responses:
55
55
56
56
1.**Validation Feedback**: Format validation errors for LLM auto-correction
57
57
2.**Lenient JSON Parsing**: LLMs often produce incomplete, malformed, or non-standard JSON
@@ -121,7 +121,7 @@ This format is designed for LLM auto-correction. The LLM reads this feedback and
121
121
filename="examples/src/llm/application-parse.ts"
122
122
showLineNumbers />
123
123
124
-
`LlmJson.parse()` is a lenient JSON parser specifically designed for LLM outputs. It combines two capabilities:
124
+
`LlmJson.parse()` is a lenient JSON parser specifically designed for LLM outputs. It combines two capabilities in a single call:
125
125
126
126
1.**Lenient JSON parsing**: Handles malformed/incomplete JSON that would fail with `JSON.parse()`
127
127
2.**Type coercion**: Fixes double-stringified values based on the expected schema
@@ -190,7 +190,7 @@ If you omit the `parameters` argument, `LlmJson.parse()` still performs lenient
190
190
filename="examples/src/llm/application-coerce.ts"
191
191
showLineNumbers />
192
192
193
-
`LlmJson.coerce()` performs type coercion on already-parsed objects. This is the coercion logic from `parse()` extracted for use when you already have a JavaScript object (not a JSON string).
193
+
`LlmJson.coerce()` performs type coercion on already-parsed objects. Use it when an SDK has already parsed the JSON — this is the coercion logic from `parse()` extracted for use when you already have a JavaScript object (not a JSON string).
If you have TypeScript types available at compile time, prefer using `typia.validate<T>()` directly. It's faster (AOT-compiled) and provides better error messages. Use `LlmJson.validate()` only when you need runtime schema-based validation.
263
263
</Callout>
264
264
265
-
## Validation Feedback Loop
265
+
## Validation Feedback Loop (The Complete Harness)
266
266
267
267
The real power of these utilities is enabling automatic error correction by LLMs:
This pattern enables LLMs to automatically correct their mistakes by:
358
+
This is the **function calling harness** in action — the same pattern that powers [AutoBe](https://github.com/wrtnlabs/autobe)'s 100% compilation success across all tested LLM models. It enables LLMs to automatically correct their mistakes:
359
359
360
360
1. Parse LLM response with `func.parse()` (handles malformed JSON + type coercion)
Copy file name to clipboardExpand all lines: website/src/content/docs/llm/parameters.mdx
+13-3Lines changed: 13 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -157,7 +157,17 @@ You can utilize the `typia.llm.parameters<Parameters>()` function to generate st
157
157
158
158
Just configure output mode as JSON schema, and deliver the `typia.llm.parameters<Parameters>()`functionreturned value to the LLM provider like OpenAI (ChatGPT). Then, the LLM provider will automatically transform the output conversation into a structured data format of the `Parameters` type.
159
159
160
-
## Lenient JSON Parsing
160
+
## The Function Calling Harness
161
+
162
+
The **function calling harness** is typia's three-layer pipeline that turns unreliable LLM output into 100% correct structured data:
3. **Validation Feedback** — pinpoints remaining value errors with inline `// ❌` annotations so the LLM can self-correct and retry
167
+
168
+
Each layer catches what the previous one didn't. Together they form a deterministic correction loop around the probabilistic LLM.
169
+
170
+
### Lenient JSON Parsing & Type Coercion
161
171
162
172
<Tabs items={[
163
173
"Parsing Example",
@@ -216,7 +226,7 @@ Some LLM SDKs (Anthropic, Vercel AI, LangChain, MCP) parse JSON internally and r
216
226
For more details, see [JSON Utilities](./json).
217
227
</Callout>
218
228
219
-
## Validation Feedback
229
+
### Validation Feedback
220
230
221
231
<LocalSource
222
232
path="examples/src/llm/parameters-validate.ts"
@@ -241,7 +251,7 @@ Use [`typia.validate<T>()`](/docs/validators/validate) for validation feedback o
241
251
242
252
The LLM reads this feedback and self-corrects on the next turn.
243
253
244
-
In the [AutoBe](https://github.com/wrtnlabs/autobe) project (AI-powered backend code generator), `qwen3-coder-next` showed only 6.75% raw functioncalling success rate on compiler AST types. However, with validation feedback, it reached 100%.
254
+
In the [AutoBe](https://github.com/wrtnlabs/autobe) project (AI-powered backend code generator by [Wrtn Technologies](https://wrtn.io)), `qwen3-coder-next` showed only **6.75%** raw functioncalling success rate on compiler AST types. However, with the complete harness, it reached **100%** — across all four tested Qwen models.
245
255
246
256
Working on compiler AST means working on any type and any use case.
0 commit comments