This is because tests is in project.ignore.
There's golden tests in there which can't be run outside FB, so they are ~guaranteed to fail for external contributions.
At least one proximate cause is #1026, which is that the pytest macros being used by the buck2 test suite are not available externally.