Updates Concepts section of docs for new release

[eschabell]

key-concepts: add processor concept and update event types

  - Add Processor to the intro topic list
  - Add Processor section describing how processors differ from filters
  - Update Events/Records description to include traces and profiles
  - Fix grammar: "associated timestamps" → "associated timestamp"

data-pipeline: add processors to pipeline diagram and sections

  - Add Processors node to Mermaid pipeline diagram between Input and Parser
  - Add Processors section describing plugin-attached, tag-free, YAML-only
  processing
  - Update diagram accessible description to include processors

Fixes #2391

Summary by CodeRabbit

• Documentation
  • Updated data pipeline documentation with a clearer flow diagram showing the Processors stage.
  • Added comprehensive Processor concept documentation detailing configuration and plugin integration.
  • Expanded Events and Records definitions to include logs, metrics, traces, and profiles.

[coderabbitai]

📝 Walkthrough

Walkthrough

Documentation updates to Fluent Bit concepts section introduce a new Processors stage in the data pipeline diagram, restructuring the flow from Input → Processors → Parser → Filter → Buffer → Routing → Outputs. Added comprehensive Processor concept documentation covering attachment to plugins, threading model, and YAML-based configuration.

Changes

Cohort / File(s)	Summary
Concept Documentation Updates concepts/data-pipeline.md, concepts/key-concepts.md	Added Processors stage to pipeline diagram and restructured routing flow with multiple outputs. Introduced new Processor concept section defining processor attachment to plugins, thread locality, YAML-only configuration, and scope across logs, metrics, traces, and profiles. Updated Events/Records definition and refined wording for consistency.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• consolidate YAML / processor info  #2183: Modifies processor documentation with YAML pipeline content and processor section additions — directly aligned with processor documentation updates in this PR.
• Processor plugin docs updates #2369: Updates individual processor plugin documentation and configuration parameters — complements the processor concept introduction in this change.

Suggested reviewers

• cosmo0920
• patrick-stephens

Poem

🐰 A processor hops into the pipeline's lane,
Between Input and Parser, it dances with data's reign,
Threading through plugins with YAML so neat,
The concepts now flow in a structured beat! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately reflects the main changes: updates to the Concepts section documentation for the new 5.0 release.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #2391: added Processor concept documentation, updated the data pipeline diagram, and updated Events/Records descriptions.
Out of Scope Changes check	✅ Passed	All changes are within scope: both files updated (data-pipeline.md, key-concepts.md) directly address the Processor introduction and documentation alignment required by issue #2391.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

concepts/data-pipeline.md (1)

25-29: Consider varying sentence structure for readability.

Three consecutive sentences begin with "Processors," which creates repetitive flow. Consider rephrasing for smoother reading.

📝 Suggested refinement

-[Processors](../pipeline/processors.md) are components that modify, transform, or enhance data as it flows through the pipeline. Processors are attached directly to individual input or output plugins rather than defined globally, and they don't use tag matching.
+[Processors](../pipeline/processors.md) are components that modify, transform, or enhance data as it flows through the pipeline. These components are attached directly to individual input or output plugins rather than defined globally, and they don't use tag matching.

-Because processors run in the same thread as their associated plugin, they can reduce performance overhead compared to filters—especially when [multithreading](../administration/multithreading.md) is enabled.
+Because they run in the same thread as their associated plugin, processors can reduce performance overhead compared to filters—especially when [multithreading](../administration/multithreading.md) is enabled.

-Processors are configured in [YAML configuration files](../administration/configuring-fluent-bit/yaml.md) only.
+Configuration is supported in [YAML configuration files](../administration/configuring-fluent-bit/yaml.md) only.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@concepts/data-pipeline.md` around lines 25 - 29, Reword the three consecutive
sentences that start with "Processors" in concepts/data-pipeline.md to avoid
repetition: keep the same facts (they modify/transform/enhance data, are
attached to individual input/output plugins not globally, run in the plugin
thread and can be faster than filters with multithreading, and are configured
via YAML) but vary sentence openings and flow—for example, start one sentence
with an action (e.g., "They modify..."), another with a contrast or placement
(e.g., "Rather than being defined globally..."), and the last with configuration
detail (e.g., "Configure processors in YAML files...") so the paragraph reads
smoother while preserving content.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@concepts/data-pipeline.md`:
- Around line 25-29: Reword the three consecutive sentences that start with
"Processors" in concepts/data-pipeline.md to avoid repetition: keep the same
facts (they modify/transform/enhance data, are attached to individual
input/output plugins not globally, run in the plugin thread and can be faster
than filters with multithreading, and are configured via YAML) but vary sentence
openings and flow—for example, start one sentence with an action (e.g., "They
modify..."), another with a contrast or placement (e.g., "Rather than being
defined globally..."), and the last with configuration detail (e.g., "Configure
processors in YAML files...") so the paragraph reads smoother while preserving
content.

---

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dc38f3c and 5d1cc4f.

📒 Files selected for processing (2)

• concepts/data-pipeline.md
• concepts/key-concepts.md

[eschabell]

@cosmo0920 and @patrick-stephens ready for review, bit of an update for 5.0 release here!