Vectorize Question Answering Prediction Head

[brandenchan]

This PR implements a more efficient way of disqualifying invalid start-end spans. Invalid spans are now assigned very low logit scores early on in the modelling pipeline. This is done through vector operations. This will be an improvement over the older method whereby all candidate spans are sorted by their scores and only later ruled out if they are invalid (e.g. end comes before start, either start or end points to padding).

This should also fix #572 where modelling times could vary wildly depending on whether the question is relevant or irrelevant.

• Show improvement in per component benchmark
• Make sure performance has not significantly changed
• Where to save per component benchmark
• Update Haystack benchmark

[brandenchan]

Master (passages per second)

deepset/bert-base-cased-squad2 - irrelevant - 19.260504316273845
deepset/bert-base-cased-squad2 -  relevant  - 87.872978351196
deepset/minilm-uncased-squad2  - irrelevant - 26.81841741284399
deepset/minilm-uncased-squad2  -  relevant  - 120.737699681037

This branch (passages per second)

deepset/bert-base-cased-squad2 - irrelevant - 87.9419574492753
deepset/bert-base-cased-squad2 -  relevant  - 88.23084025443052
deepset/minilm-uncased-squad2  - irrelevant - 121.63358417290499
deepset/minilm-uncased-squad2  -  relevant  - 123.58026447916457

[brandenchan]

Current squad dev performance with deepset/roberta-base-squad2 (evaluated with official script)

  "exact": 76.69502231954856,
  "f1": 80.03626318194439,
  "total": 11873,
  "HasAns_exact": 67.57759784075573,
  "HasAns_f1": 74.26966139663054,
  "HasAns_total": 5928,
  "NoAns_exact": 85.78637510513036,
  "NoAns_f1": 85.78637510513036,
  "NoAns_total": 5945

cf. numbers reported in model card

"exact": 78.49743114629833,
"f1": 81.73092721240889

The performance is somewhat worse than it should be but could be related to #552 and #602

[brandenchan]

Ran test/benchmark/question_answering_benchmarks.py. There is no signficant difference in performance or speed between this branch and master

[Timoeller]

Beautiful vectorization. Ready to merge from my side.