Skip to content

in_tcp: skip emitting an empty event stream when a read has no complete message #5602

Open

Description

@Watson1978

Is your feature request related to a problem? Please describe.

Without <extract>, in_tcp calls router.emit_stream(@tag, es) after every read. It does this even when the read did not complete any message and es is empty. A long message arrives in many reads, so the router, the filters and the output are called many times with an empty stream. The output's emit_count counts these calls too.

With the configuration below, I sent one 1 MiB line in 64 writes of 16 KiB:

ruby -rsocket -e 's = TCPSocket.new("127.0.0.1", 5170); 64.times { s.write("x" * 16384); sleep 0.02 }; s.write("\n"); sleep 1'

monitor_agent then shows:

{"plugin_id":"in_tcp","emit_records":1}
{"plugin_id":"out_null","emit_records":1,"emit_count":66}
<source>
  @type monitor_agent
  bind 127.0.0.1
  port 24220
</source>

<source>
  @type tcp
  @id in_tcp
  tag test
  port 5170
  bind 127.0.0.1
  <parse>
    @type none
  </parse>
</source>

<match test>
  @type null
  @id out_null
</match>

Describe the solution you'd like

Call router.emit_stream only when es is not empty. The <extract> code path already emits nothing for such a read.

Describe alternatives you've considered

Keep the current behavior. The empty streams do not create buffer chunks, because Buffer#write_once returns early for empty data. The cost is the extra calls and the larger emit_count.

Additional context

Tested with Fluentd 1.20.1 (master at 1d6b112) and Ruby 4.0.9 on Linux.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementFeature request or improve operations

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions