Deploy YARA rules remotely and privately

[zayhanlon]

User story
As a detection & response engineers,
I want to deploy YARA rules to agents remotely and privately from a server I host myself (separate from Fleet)
so don't have to write rules to disk (too large of scope and too slow) or host rules on a non-private webserver (what osquery supports today).

Context

• Product designer: @zwass

Why doesn't current YARA rule deployment with osquery work?

• Too large of scope: If they wrote the rules to disk, any engineering (including outside detection & response team) could read these rules because they'd live in the organization's monorepo. If an engineer's account gets compromised then they can read these rules.
• Too slow: It takes months to get these rules reviewed.
• Privately: Viewing rules must require authentication. Currently, osquery doesn't provide a way to authenticate YARA rules.

Changes

Product

• UI changes: No changes
• CLI (fleetctl) usage changes: No changes
• YAML changes: TODO: Specify changes in the YAML files doc page as a PR to the reference docs release branch. Put "No changes" if there are no changes necessary.
• REST API changes: TODO: Specify changes in the the REST API doc page as a PR to reference docs release branch. Put "No changes" if there are no changes necessary. Move this item to the engineering list below if engineering will design the API changes.
• Fleet's agent (fleetd) changes: PR to allow osquery to authenticate YARA requests: Add --yara_sigurl_authenticate flag osquery/osquery#8437. With the yara_sigurl_authenticate flag enabled, osquery will send the node key when retrieving YARA rules, which will allow the Fleet server to authenticate the request before responding.
• Activity changes: No changes
• Permissions changes: No changes.
• Changes to paid features or tiers: TODO: Specify changes in pricing-features-table.yml as a PR to reference docs release branch. Specify "Fleet Free" and/or "Fleet Premium" if there are no changes to the pricing page necessary.
• Once shipped, requester has been notified
• Once shipped, dogfooding issue has been filed

Engineering

• Feature guide changes: Missing guide: deploy YARA rules remotely and privately #24703
• Database schema migrations: TODO
• Load testing: TODO

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Requires load testing: TODO
• Risk level: Low / High TODO
• Risk description: TODO

Manual testing steps

1. Step 1
2. Step 2
3. Step 3

Testing notes

Confirmation

1. Engineer (@____): Added comment to user story confirming successful completion of QA.
2. QA (@____): Added comment to user story confirming successful completion of QA.

[noahtalerman]

Feature fest: We have a yara table in Fleet: https://fleetdm.com/tables/yara

Does that address the need?

[zayhanlon]

Related to an ask from customer-domon re: #19553

[JoStableford]

Related to a Slack conversation

[nonpunctual]

The yara & yara_events tables do not have the capabilities to access sigrules from a remote server securely.