Leaking array buffers

[ronag]

Not sure if this is a user, undici, node or V8 bug but I have case where I have lot's of ArrayBuffers allocated and never freed in our Node based proxy:

{rss: 4519784448, heapTotal: 25522176, heapUsed: 20927688, external: 722839852, arrayBuffers: 721332770}

@mcollina @addaleax any hints on how to debug this?

[mcollina]

Have you got a repro? You can possibly find them using heapdump.
If you need something programmatic to run on servers, use https://github.com/nearform/heap-profiler.

[szmarczak]

@ronag I remember there was a bug in Node.js 12 related to TLS sockets. What Node.js version are you running?

[ronag]

14.8, HTTP

Happens at multiple locations with different node versions.

I see that V8 recently did something with GC of ArrayBuffers. Maybe coincidence...

[ronag]

I’ll have to continue digging on Monday. Anyway, a FYI in case it affects you.

[ronag]

Have you got a repro? You can possibly find them using heapdump.
If you need something programmatic to run on servers, use https://github.com/nearform/heap-profiler.

Does that work with cluster?

[mcollina]

It might work given some adaptation.

[ronag]

Seem to have the same problem on a totally different service which recently started to use undici

[ronag]

Been trying to create a repro without success 😞

[szmarczak]

@ronag How does the undici call look like? What function is used?

[ronag]

`Client.request``

I think it might be broken back pressure.

[ronag]

My current guess it that it has something to do with pause/resume multiple times. This seems to be growing arrayBuffer usage infinitiely:

'use strict'

const { Client } = require('..')
const { createServer } = require('http')
const { Readable } = require('stream')

const requests = 128

const server = createServer()

const limit = 1e9

server.on('request', (req, res) => {
  let bytesWritten = 0

  const interval = setInterval(() => {
    console.log(process.memoryUsage().arrayBuffers / 1e6, bytesWritten, bytesWritten / limit)
  }, 1e3)

  const buf = Buffer.allocUnsafe(16384)
  new Readable({
    read () {
      bytesWritten += buf.length
      this.push(buf)
      if (bytesWritten >= limit) {
        clearInterval(interval)
        this.push(null)
      }
    }
  }).pipe(res)
})

server.listen(0, () => {
  const client = new Client(`http://localhost:${server.address().port}`, {
    pipelining: 1
  })

  for (let n = 0; n < requests; ++n) {
    client.request({ path: '/', method: 'GET', opaque: 'asd' }, (err, data) => {
      if (err) {
        console.error(err)
      }
      data.body
        .resume()
        .on('data', () => {
          data.body.pause()
          setTimeout(() => data.body.resume(), 1e3)
        })
    })
  }
})

[ronag]

Could it be that socket.pause() does not work in consume() mode?

[ronag]

Yep, that's the problem.