Add conventional tracking into ai-assisted tracking as complementary

[tongtongcao]

See details for update and validation in the attached document.
AI-assisted Tracking + Conventional Tracking as Complementary.pdf
With the update, as well as previous updates for forward tracking, slope for forward tracking reaches the goal: -0.001/nA, as test by RGA and RGD datasets.
This PR is for both dc2 and dc1. So this branch should also be merged into the dc1 branch.

[zieglerv]

A study I did on events with multiple AI candidates for a given track shows that a substantial fraction of AI failures comes from choosing the seed with maximum AI probability where AI gets the wrong set of segments. In such cases, some segments are real segments on track and some do not belong to the track. The real segments are near the track trajectory. Your algorithm could be improved by doing the following:

• Read all hits from the Hits bank
• Remake the clusters from these hits
• Look for segments not suggested by AI that are near the AI seed segment list (define a cut based on doca to the AI segments and local angle)
• Using this group of segments find the subgroup of segments yielding the best track
• After this search for tracks using the segments that are not on-track to find possible seeds that have no segments suggested by AI.

[tongtongcao]

The update actually covers the issue that you mentioned.
In the attached document, slide 7 summarize that all situations are covered by the update.
I provide four examples in slides 14 - 17 for cases with wrong AI prediction.
For these examples, cluster combs are predicted by AI, and fails in HB tracking.
After update, tracks are restored.
Among these examples, the first two examples in slides 14 and 15 are exactly cases that you found. (two clusters are close, and AI selects wrong one with higher probability).
For the last two examples, AI makes completely wrong prediction.

[tongtongcao]

The update is suggested by Raffaella. Add one more bit (bit 13) into status of tracks. It points out if a track is from AI-assisted tracking or conventional tracking. 0 means conventional tracking, and 1 means AI-assisted tracking.

[baltzell]

ready?

[zieglerv]

Looking at the code the list of clusters considered are the clusters which are not on tracks:
segmentsConv = segFinder.get_Segments(clustersConv,
event,
Constants.getInstance().dcDetector, false);
The track finding is redone using segmentsConv, which does not contain the segments flagged by AI. So there is no retry of the track AI suggestion using segments not suggested by AI along with the ones suggested by AI.
This will however give improvements for set of segments in R1,2,3 completely missed by AI.
My suggestion was to take it a step further and revisit the AI seeds by checking for segments in the proximity of the AI seed. I do not see this in the code, unless I am totally missing it. Anyway, like I said, the current algorithm does lead to improvements in efficiency and should be merged.

[zieglerv]

See my comment in the conversation section.

[tongtongcao]

Exactly to say, segmentsConv contains segments not on tracks from AI-assisted tracking.
If a cluster combo is predicted by AI, but fails in HB tracking, all clusters in the combos also will enter next step for conventional tracking.
There is only one case, which is missed by the PR:
A cluster combo is predicted by AI, and also passed in HB tracking. But the combo is actually wrong. One (most likely, maybe 2 or even more) cluster should be replaced by another cluster, which is close to it.
But situation could be inverse. AI gives correct prediction for a cluster combo, but there is another cluster close to one cluster in the combo.
For both situations, we can get two track candidates. Then we need to choose one from the two candidates.
From view of AI, choose one with higher probability.
From view of tracking, choose one with better tracking quality based on chi2/ndf.
Since HB tracking is kind of rough, my previous study told thaht selection by chi2/ndf was not better than by probability from AI.
So before further update to consider clusters (not in AI prediction) close to clusters in combos predicted by AI, we need to build a new routine, combining with chi2/ndf and probability. I also considered to include vz. But I have not got a way to build the new routine so far. It needs very detailed study.
Besides, current AI model just uses average wires as feature, but ignores slope of clusters.
It is the main reason which causes the issue you concerns about.
Gagik plans to build a new AI model with features including slope information.
I think that the new AI model is a better solution.