Skip to content

dashboards/vmcluster: vmauth reported as vmselect and vminsert #1709

@rtm0

Description

@rtm0

For some reason VictoriaMetrics - cluster dashboard treats vmauth as both vminsert and vmselect instances. This manifests in all charts that include vmselect and/or vminsert. Some examples where it is noticed right away:

  1. Stats section at the very top. The version and uptime charts show additional vmselects and vminserts:
Image Image
  1. Drilldown section at the very bottom. These additional vmselects and vminserts are included into the CPU Usage chart. This is where you can see that it is actually a vmauth instance - both vmselect and vminsert have the same IP and 8427 port.
Image

To Reproduce use our sandbox dashboard and choose lts deployment.


If vmauth consumes little resources, then its presence may be not noticed on other charts. But if it has significant resource utilization this will show as if vmselect or vminsert consume more resources that they actually do.

Additionally, even if vmauth consumes little resources, but has some utilization spikes (i.e. its max cpu is 0.5, but it suddenly uses 0.45 cpu) it will significantly distort vmselect and vminsert utilization charts.

An example from my personal cluster:

Here is the Resource Usage / CPU (All) chart. Notice the vmselect spikes (yellow line). The vminsert has exactly same spikes (the green line, it is hidden behind yellow line because it is exactly the same):

Image

If you look at the same chart but unaggregated, we will see that vmselect and vminsert actually have no spikes. Instead these are vmauth spikes.

Image

Because this chart does not use absolute numbers and instead uses the relative ones (current utilization-to-max), vmauth utilization wins (because the query also has max function).

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions