Skip to content

Commit 0d3cf88

Browse files
tiansivivetiansivive
authored
Merge branch 'main' into siem/ea/watchlists/entity-source-api
2 parents d879c98 + 8b9828d commit 0d3cf88

8,199 files changed

Lines changed: 298860 additions & 204299 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/api-authz/SKILL.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
---
2+
name: api-authz
3+
description: Kibana API route authorization patterns. Use when configuring route security, working with requiredPrivileges, using authzResult for privilege-based branching, opting out of authorization, or naming custom privileges.
4+
---
5+
6+
# API Authorization
7+
8+
> All API routes in Kibana must have authorization checks. Authorization is not optional, even for `internal` routes.
9+
10+
## Route Security Configuration
11+
12+
Routes declare authorization via the `security` option in `KibanaRouteOptions`:
13+
14+
```ts
15+
router.get({
16+
path: '/api/path',
17+
security: {
18+
authz: {
19+
requiredPrivileges: ['<privilege_1>', '<privilege_2>'],
20+
},
21+
},
22+
...
23+
}, handler);
24+
```
25+
26+
## Privilege Naming
27+
28+
Privilege names follow the `<operation>_<subject>` convention using underscores only.
29+
30+
| Incorrect | Why | Correct |
31+
|-----------|-----|---------|
32+
| `read-entity-a` | Uses `-` instead of `_` | `read_entity_a` |
33+
| `delete_entity-a` | Mixes `_` and `-` | `delete_entity_a` |
34+
| `entity_manage` | Subject before operation | `manage_entity` |
35+
36+
## Privilege-Based Branching with `authzResult`
37+
38+
When a route handler branches logic based on user privileges (returns different data, enables different features), it **must** use `request.authzResult`. Do not use `capabilities.resolveCapabilities()` or other authorization checks for branching — `authzResult` is the single source of truth.
39+
40+
**Look for:** routes with `anyRequired` (OR logic), handlers that conditionally expose data based on permissions, or functions that check capabilities and return booleans for branching.
41+
42+
**Correct — use `authzResult`:**
43+
```ts
44+
router.get({
45+
path: '/api/path',
46+
security: {
47+
authz: {
48+
requiredPrivileges: ['privilege_3', { anyRequired: ['privilege_1', 'privilege_2'] }],
49+
},
50+
},
51+
...
52+
}, (context, request, response) => {
53+
const authzResult = request.authzResult;
54+
// { "privilege_3": true, "privilege_1": true, "privilege_2": false }
55+
56+
if (authzResult.privilege_1) {
57+
return response.ok({ body: ... });
58+
} else if (authzResult.privilege_2) {
59+
return response.ok({ body: ... });
60+
}
61+
62+
return response.ok({ body: { data: ... } });
63+
});
64+
```
65+
66+
**Wrong — using capabilities for authorization branching:**
67+
```ts
68+
const canReadDecryptedParams = async (routeContext: RouteContext) => {
69+
const { request, server } = routeContext;
70+
const capabilities = await server.coreStart.capabilities.resolveCapabilities(request, {
71+
capabilityPath: 'my_capability.*',
72+
});
73+
return capabilities.my_capability?.canReadParams ?? false;
74+
};
75+
76+
if (await canReadDecryptedParams(routeContext)) {
77+
return getDecryptedParams(routeContext, paramId);
78+
} else {
79+
return getBasicParams(routeContext, paramId);
80+
}
81+
```
82+
83+
**Fix:** declare both privileges in the route config with `anyRequired` and branch on `request.authzResult`:
84+
```ts
85+
router.get({
86+
path: '/api/params',
87+
security: {
88+
authz: {
89+
requiredPrivileges: [{ anyRequired: ['read_params_decrypted', 'read_params'] }],
90+
},
91+
},
92+
}, (context, request, response) => {
93+
if (request.authzResult.read_params_decrypted) {
94+
return getDecryptedParams(routeContext, paramId);
95+
} else {
96+
return getBasicParams(routeContext, paramId);
97+
}
98+
});
99+
```
100+
101+
## Opting Out of Authorization
102+
103+
When a route must opt out, use the predefined `AuthzOptOutReason` enum or `AuthzDisabled` helpers from `@kbn/core-security-server`:
104+
105+
```ts
106+
import { AuthzDisabled, AuthzOptOutReason } from '@kbn/core-security-server';
107+
108+
// Predefined helper
109+
router.get({
110+
path: '/api/path',
111+
security: { authz: AuthzDisabled.delegateToSOClient },
112+
...
113+
}, handler);
114+
115+
// Predefined enum
116+
router.get({
117+
path: '/api/path',
118+
security: {
119+
authz: { enabled: false, reason: AuthzOptOutReason.DelegateToSOClient },
120+
},
121+
...
122+
}, handler);
123+
124+
// Custom reason — only when no predefined reason applies
125+
router.get({
126+
path: '/api/health',
127+
security: {
128+
authz: {
129+
enabled: false,
130+
reason: 'This route is a health check endpoint that returns no sensitive information',
131+
},
132+
},
133+
...
134+
}, handler);
135+
```
136+
137+
**Invalid opt-out reasons — flag these:**
138+
- `"Opt out from authorization"` — too generic, no context
139+
- `"This route does not need authorization"` — no explanation why
140+
- `"Authorization not required"` — no context provided
141+
- `"Authorization is delegated to SO Client"` — use `AuthzOptOutReason.DelegateToSOClient` instead
142+
143+
## References
144+
145+
- [Kibana API Authorization Documentation](dev_docs/key_concepts/api_authorization.mdx)
146+
- [Kibana HTTP API Design Guidelines](dev_docs/contributing/kibana_http_api_design_guidelines.mdx)

‎.agents/skills/codeql/SKILL.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
name: codeql
3+
description: Work with CodeQL in Kibana — write, test, and debug custom queries locally, fetch scan results from GitHub, and validate inline suppression comments. Use when writing or debugging CodeQL queries, running CodeQL unit tests, analyzing SARIF results, fetching scan results, or checking codeql suppression justifications.
4+
disable-model-invocation: true
5+
---
6+
7+
# CodeQL
8+
9+
## Project Layout
10+
11+
```
12+
.github/codeql/
13+
├── codeql-config.yml # Main config (paths-ignore, packs, query-filters)
14+
├── custom-queries/
15+
│ ├── qlpack.yml # QL pack definition (name: kibana-custom-queries)
16+
│ ├── codeql-pack.lock.yml
17+
│ ├── suppression/ # Alert suppression logic
18+
│ │ ├── AlertSuppression.ql
19+
│ │ └── AlertSuppression.qll
20+
│ └── <category>/ # e.g. dos/, xss/
21+
│ ├── <RuleName>.ql # Query file
22+
│ ├── <RuleName>.qhelp # Help docs (XML)
23+
│ ├── <RuleName>.md # Human-readable docs
24+
│ ├── <category>-security.qls # Query suite
25+
│ └── <RuleName>/ # Unit test directory
26+
│ ├── <RuleName>.qlref # Points to the .ql file (relative to qlpack root)
27+
│ ├── <RuleName>.expected # Expected test output
28+
│ └── test.js # Test source code
29+
scripts/codeql/
30+
├── quick_check.sh # Local analysis via Docker
31+
└── codeql.dockerfile # Docker image (ubuntu + CodeQL CLI)
32+
```
33+
34+
## Running Queries Locally (Full Analysis)
35+
36+
Uses Docker to create a CodeQL database and run queries against real source code.
37+
38+
```bash
39+
# Analyze a source directory with custom queries
40+
bash scripts/codeql/quick_check.sh -s <source_dir> -q .github/codeql/custom-queries
41+
42+
# Analyze with a single query file
43+
bash scripts/codeql/quick_check.sh -s <source_dir> -q .github/codeql/custom-queries/dos/UnboundedArrayInRoute.ql
44+
45+
# Custom results directory
46+
bash scripts/codeql/quick_check.sh -s <source_dir> -r .codeql-results -q .github/codeql/custom-queries
47+
```
48+
49+
**Options:**
50+
- `-s <source_dir>` (required for analysis): directory to scan
51+
- `-q <query_dir|query_file>`: custom queries directory or single `.ql` file
52+
- `-r <results_dir>`: where to store DB and SARIF (default: `.codeql/`)
53+
- `-t`: run unit tests instead of analysis (use with `-q`, no `-s` needed)
54+
55+
**Output:** SARIF file at `<results_dir>/database/results.sarif`. If `jq` is installed, a colored summary prints automatically.
56+
57+
**First run builds a Docker image** (`codeql-env`) from `scripts/codeql/codeql.dockerfile`. On Apple Silicon, it runs with `--platform linux/amd64` (emulation).
58+
59+
## Running CodeQL Unit Tests
60+
61+
Unit tests validate that a query flags the correct lines. Each test lives in a subdirectory named after the query.
62+
63+
### Test structure
64+
65+
```
66+
<category>/<RuleName>/
67+
├── <RuleName>.qlref # Reference: "category/RuleName.ql"
68+
├── test.js # Source code with `// $ Alert` annotations
69+
└── <RuleName>.expected # Expected output (auto-generated or hand-written)
70+
```
71+
72+
- `// $ Alert` on a line means the query **should** flag that line
73+
- Lines without `// $ Alert` should **not** be flagged
74+
- `.expected` file has pipe-delimited format: `| <location> | <message> |`
75+
76+
### Running tests via Docker
77+
78+
Uses the same `codeql-env` Docker image built by `quick_check.sh` (built automatically on first run).
79+
80+
```bash
81+
# Run a specific test directory
82+
bash scripts/codeql/quick_check.sh -t -q .github/codeql/custom-queries/dos/UnboundedArrayInRoute
83+
84+
# Run all tests in the qlpack
85+
bash scripts/codeql/quick_check.sh -t -q .github/codeql/custom-queries
86+
```
87+
88+
### CI workflow
89+
90+
The `codeql-pr.yml` workflow automatically runs unit tests on PR. It finds all `*.qlref` directories and runs `codeql test run` against them.
91+
92+
## Fetching Remote SARIF / Scan Results
93+
94+
The `scripts/fetch_sarif.mjs` script (relative to this skill directory) fetches CodeQL SARIF results and alerts from GitHub for a PR or branch.
95+
96+
```bash
97+
# By PR number
98+
GITHUB_TOKEN=ghp_xxx node .agents/skills/codeql/scripts/fetch_sarif.mjs 252121
99+
100+
# By full ref
101+
GITHUB_TOKEN=ghp_xxx node .agents/skills/codeql/scripts/fetch_sarif.mjs refs/heads/main
102+
```
103+
104+
**Requires:** `GITHUB_TOKEN` env var with `security_events` scope. Depends on `@octokit/rest` (already in Kibana deps).
105+
106+
**What it does:**
107+
1. Lists recent CodeQL analyses for the ref
108+
2. Fetches full SARIF JSON (with rule severity cross-referencing)
109+
3. Prints formatted results (rule, severity, message, file:line)
110+
4. Fetches code scanning alerts for the same ref
111+
112+
## Writing a New Query
113+
114+
1. **Create the `.ql` file** in `.github/codeql/custom-queries/<category>/`:
115+
- Use `@id js/kibana/<descriptive-id>` (must be unique)
116+
- Include `@kind problem` (or `path-problem` for taint tracking)
117+
- Set `@problem.severity` and `@security-severity`
118+
- Import `javascript` module
119+
- Refer to existing queries like `UnboundedArrayInRoute.ql` for patterns
120+
121+
2. **Create a unit test directory** `<category>/<RuleName>/`:
122+
- `<RuleName>.qlref` containing `<category>/<RuleName>.ql`
123+
- `test.js` with annotated test cases (`// $ Alert` for expected hits)
124+
- Run tests to generate `.expected` — verify it matches expectations
125+
126+
3. **Add a `.qhelp`** (XML) and/or **`.md`** for documentation
127+
128+
4. **Optionally add a `.qls` query suite** if grouping multiple queries
129+
130+
5. **Test locally** with `quick_check.sh` against real Kibana source code
131+
132+
## Inline Suppressions
133+
134+
Suppressions use the format `// codeql[rule-id] justification text`. Every suppression **must** include a specific justification explaining why it is safe.
135+
136+
**Valid:**
137+
```ts
138+
// codeql[js/path-injection] User input is validated against an allowlist before use
139+
return fs.readFileSync(`/etc/${validatedPath}`, 'utf8');
140+
```
141+
142+
**Invalid — flag these:**
143+
- **Missing justification:** `// codeql[js/path-injection]` with no explanation
144+
- **Generic justification:** `"false positive"`, `"safe"`, `"not a vulnerability"` — says nothing about the actual mitigation
145+
- **Incomplete justification:** `"sanitized"` — does not explain how or by what mechanism
146+
147+
**Good justifications** describe the concrete security mechanism: allowlist validation, DOMPurify escaping, shell-quote library, test-only code, etc.
148+
149+
## Troubleshooting
150+
151+
| Issue | Fix |
152+
|-------|-----|
153+
| Docker build fails on ARM | Ensure `--platform linux/amd64` is set (script handles automatically) |
154+
| `qlpack.yml` not found | The script walks up from the `.ql` file to find it — ensure `qlpack.yml` exists at `custom-queries/` root |
155+
| Test produces `.actual` file | Diff `.actual` vs `.expected` — `.actual` files are gitignored |
156+
| Query finds nothing | Check `codeql-config.yml` `paths-ignore` — test/mock dirs are excluded |
157+
| `jq` not found for summary | Install jq: `brew install jq` |
158+
159+
## References
160+
161+
- [Writing CodeQL queries](https://codeql.github.com/docs/writing-codeql-queries/) — official guide covering query structure, QL tutorials, and running queries

0 commit comments

Comments
 (0)