Skip to content

Enhance dque fs sync #397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nickytd merged 11 commits into from
Nov 24, 2025
Merged

Enhance dque fs sync #397

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b2112fc
client: fix typo in dque client
nickytd Nov 21, 2025
e7b72c3
client: add dque size metric
nickytd Nov 21, 2025
4eeed0d
example: enhance fluent-bit performance dashboard
nickytd Nov 24, 2025
0289326
example: support single namespace performance test load
nickytd Nov 24, 2025
5228761
Merge branch 'master' into share-dque-with-controller
nickytd Nov 24, 2025
db765c5
Default value should not be quoted
nickytd Nov 24, 2025
c7abdc1
Update example/performance-test/down.sh
nickytd Nov 24, 2025
60e302e
Update example/performance-test/run.sh
nickytd Nov 24, 2025
48efb63
Update example/performance-test/down.sh
nickytd Nov 24, 2025
9ba1408
Update example/performance-test/down.sh
nickytd Nov 24, 2025
4a902f7
Update example/performance-test/run.sh
nickytd Nov 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions example/performance-test/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ JOBS ?= 10 # Number of parallel jobs per cluster
LOGS ?= 1000 # Number of log lines per job
LOGS_DELAY ?= 25s # Delay between log messages

SCENARIO ?= shoot # Test scenario: "shoot" or "seed"

LOGCLI_BIN := $(DIR)/bin/logcli

# Default target
Expand Down Expand Up @@ -55,7 +57,7 @@ setup: logcli check-kubeconfig
# Create test clusters and start log generation
run: check-kubeconfig setup
@echo "Starting performance test with $(CLUSTERS) clusters, $(JOBS) jobs per cluster, $(LOGS) logs per job..."
@export CLUSTERS=$(CLUSTERS) JOBS=$(JOBS) LOGS=$(LOGS) LOGS_DELAY=$(LOGS_DELAY) && $(DIR)/up.sh
@export CLUSTERS=$(CLUSTERS) JOBS=$(JOBS) LOGS=$(LOGS) LOGS_DELAY=$(LOGS_DELAY) && $(DIR)/run.sh $(SCENARIO)

# Check Current logs in Vali
check: check-kubeconfig
Expand All @@ -70,7 +72,7 @@ fetch: check-kubeconfig
# Clean up all test resources
down: check-kubeconfig
@echo "Removing performance test workloads resources..."
@export CLUSTERS=$(CLUSTERS) && $(DIR)/down.sh
@export CLUSTERS=$(CLUSTERS) && $(DIR)/down.sh $(SCENARIO)

# Alias for down
clean: check-kubeconfig down
Expand Down
4 changes: 2 additions & 2 deletions example/performance-test/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ sum(count_over_time({container_name=\"shoot--logging--dev-${i}\"}[24h]))
```

The script polls aggregated logs count per namespace. (CLUSTERS)
Expected count count ≈ `JOBS * LOGS` (after all pods finish and ingestion catches up).
Expected count ≈ `JOBS * LOGS` (after all pods finish and ingestion catches up).

---

Expand Down Expand Up @@ -75,7 +75,7 @@ Deploys test components with the helm chart in `charts/fluent-bit-plugin`.
* Installs prometheus and plutono with fluent-bit dashboard for monitoring.
* Installs logging backend Vali simulating a seed cluster and shared Vali for all simulated shoots.

### make up
### make run

Creates the test run workloads.

Expand Down
Loading
Loading