Doesn't visually scale to large numbers of labels in the reference, forcing us to cap it at max.labels=. We're not even talking about the number of labels that were assigned, just the number of labels available in the reference. Our cap could artificially remove useful diagnostics about assignment ambiguity by removing closely-related labels that just missed out on assignment.
Also, it's kinda hard to compare colors accurately, to determine whether the delta is large enough to consider the predicted assignment as unambiguous. The normalization doesn't help either.
The whole point of this plot is to check whether there is ambiguity in the assignments, by looking for other labels with scores close to the assigned label. So we might as well make a plot explicitly for that purpose.
Maybe for each label, we create a series of violin plots for the deltas of all runner-up labels. This would reveal whether any worrying ambiguity exists (if the delta is close to zero) and if said ambiguity refers to specific competing label.
Doesn't visually scale to large numbers of labels in the reference, forcing us to cap it at
max.labels=. We're not even talking about the number of labels that were assigned, just the number of labels available in the reference. Our cap could artificially remove useful diagnostics about assignment ambiguity by removing closely-related labels that just missed out on assignment.Also, it's kinda hard to compare colors accurately, to determine whether the delta is large enough to consider the predicted assignment as unambiguous. The normalization doesn't help either.
The whole point of this plot is to check whether there is ambiguity in the assignments, by looking for other labels with scores close to the assigned label. So we might as well make a plot explicitly for that purpose.
Maybe for each label, we create a series of violin plots for the deltas of all runner-up labels. This would reveal whether any worrying ambiguity exists (if the delta is close to zero) and if said ambiguity refers to specific competing label.