help request: Unclear if memory leak or something else in luajit process

[sebastian-popa]

Description

Running APISIX in Kubernetes, have been noticing memory going up constantly, and when it gets to 70-75%, Kubernetes kills the pods. In next image, first sawtooth at 70% is Kube killing the pods, the other ones are me restarting the pods.

Tried to reproduce locally, ran the same version with same configurations in my Docker, locally, and created a (K6) script to exercise one of the routes which I think might be related to the issue. Same request (route, payload), over and over, for 10 min, simulating 20 users. Exec-ed in the docker container, ran top command, noticed a raise in memory after each run.

First run of 10min

...
Second run of 10min

...
Fourth run of 10min

...
Looks like luajit went from 0.8% to 4%, which COULD explain the memory going up constantly. Digging around, found 2 links which might pertain to this issue:

1. help request: memory leakage problem in Apisix GW #11680 - this suggests setting a "memory limit" ("It does not appear to be a leak but rather the allocator behavior. Once a memory limit was set, this behavior was not observed again in 3.13"), any idea which memory limit for allocator it's talking about?
2. https://blog.openresty.com/en/luajit-plus/ - this one seems to be a plug for some "LuaJIT-plus", unclear if useful

Any help on dealing with this?
Thanks!

Environment

• APISIX version (run apisix version): 3.13
• Operating system (run uname -a): Linux 0abad5fe5166 6.10.14-linuxkit change: added doc of how to load plugin. #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64 GNU/Linux
• OpenResty / Nginx version (run openresty -V or nginx -V): nginx version: openresty/1.27.1.2
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): N/A
• APISIX Dashboard version, if relevant: N/A
• Plugin runner version, for issues related to plugin runners: N/A
• LuaRocks version, for installation issues (run luarocks --version): N/A

[sebastian-popa]

Can anyone help with this? Not sure if it's a bug and don't want to create one if is not.. Thanks :)

[Baoyuantop]

Hi @sebastian-popa, sorry for the delayed response. Could you share your local Docker configuration files for testing? This should include both the APISIX configuration and routing configuration. If you could also include a Docker Compose file, that would be even better. I'll use these configurations to reproduce your issue and then analyze it.

[shihuncl]

Hi @sebastian-popa, sorry for the delayed response. Could you share your local Docker configuration files for testing? This should include both the APISIX configuration and routing configuration. If you could also include a Docker Compose file, that would be even better. I'll use these configurations to reproduce your issue and then analyze it.

I've also encountered this issue. Now that the call volume has increased, the memory usage of the pod is getting larger and larger. However, looking at the memory metrics collected by Prometheus, only the prometheus metric is abnormal, while the others are normal.

values.yaml

[juzhiyuan]

I'm not sure if it's the same issue. However, as a reference, one Slack user asked about OOM-Killing.

The whole context is here: https://the-asf.slack.com/archives/CUC5MN17A/p1768992672387949