Make invocation of test method customizable

[binaryfire]

Hi Sebastian

Swoole-based framworks use coroutines, which require special handling when running tests. In Hyperf, this is how it's handled: https://github.com/hyperf/hyperf/blob/e901f78e453fb9c1af70bac534087f7a07202cfe/src/testing/src/Concerns/RunTestsInCoroutine.php#L24

Unfortunately changing runTest from protected to private in this PR means this is no longer possible with PHPUnit versions after 10.5.45.

Would you please consider making runTest protected and an officially supported extension point? Or if not, is there another way to achieve what we're doing in the linked trait?

I know you mentioned in #6172 that this is beyond the scope of the project, however that means there will be entire PHP frameworks and communities who will no longer be able to use PHPUnit. There are already discussions about exploring workarounds using the unfinalize package. It'd be a shame if that kind of thing was necessary to get it working.

[sebastianbergmann]

I have no experience with coroutines or frameworks such as Swoole. I have not found these technologies to be relevant (for me) to date, as I have not encountered them at conferences or meetups, nor in my dealings with customers. I am not trying to excuse my ignorance or dismiss these technologies as uninteresting or irrelevant. I simply want to explain why I have not dealt with them yet and why I have not run into the problem you are trying to solve.

Could you please explain why PHP code that uses coroutines and/or PHP code written for a framework such as Swoole has different requirements for executing PHPUnit tests? Why does a test method need to be run in a coroutine? Could you also explain why the desired outcome can only be achieved by changing parts of the TestCase::runTest() implementation?

What would your implementation of TestCase::runTest() look like? Would you need to replace the whole method with a custom implementation, or only parts of it? If the latter, which parts?

Would "run test in coroutine" be a generic enough option to include in PHPUnit, or must it be specific to a framework such as Swoole?

[binaryfire]

@sebastianbergmann Thanks for the quick reply!

Swoole is a PHP extension that adds coroutine-based async programming to PHP. It was developed primarily by Chinese developers and is used by companies like Alibaba and Baidu in China and PicPay in Brazil. It's very popular in Asia and South America but less known in Europe and North America, which probably explains why you haven't come across it at conferences. Several frameworks are built on top of it, including Hyperf (one of the most popular PHP frameworks in China) and Hypervel.

Swoole provides coroutine-aware async versions of common I/O operations - database connections (MySQL, PostgreSQL), Redis, HTTP clients, file operations, etc. These only work when called from inside a coroutine context created by Swoole\Coroutine\run(). Calling them outside a coroutine context results in:

Fatal Error: must be called in the coroutine

PHPUnit runs test methods in regular PHP execution with no coroutine scheduler active. So any test that uses Swoole's coroutine features fails immediately. The test method itself must run inside a coroutine container. We're not trying to replace runTest()'s logic - we need to wrap the test method invocation. All of PHPUnit's test execution (setup/teardown hooks, assertions, data providers, etc.) works exactly as normal. We just need the final call to the test method to happen inside Swoole\Coroutine\run().
The Hyperf RunTestsInCoroutine trait does this by overriding runTest() to swap the test method name to a proxy method that wraps the real test in a coroutine container:

final protected function runTest(): mixed
{
    if (extension_loaded('swoole') && Coroutine::getCid() === -1 && $this->enableCoroutine) {
        $this->realTestName = $this->name();
        parent::setName('runTestsInCoroutine');
    }
    return parent::runTest();
}

Ideally, we'd have a hook that fires around the test method invocation, so frameworks can provide their own wrapper without modifying PHPUnit internals. PHPUnit wouldn't need to know about Swoole specifically. A hook or callback that lets frameworks wrap the test method invocation would be generic enough for any coroutine runtime to use.

Tagging @albertcht who maintains a new Swoole-based framework Hypervel. He has deeper knowledge of the Swoole ecosystem and can provide more detail if needed.

[sebastianbergmann]

Ideally, we'd have a hook that fires around the test method invocation, so frameworks can provide their own wrapper without modifying PHPUnit internals.

Is this what you need?

diff --git a/src/Framework/TestCase.php b/src/Framework/TestCase.php
index 7d3a04b7f8..0ef21b1259 100644
--- a/src/Framework/TestCase.php
+++ b/src/Framework/TestCase.php
@@ -1282,6 +1282,15 @@ protected function onNotSuccessfulTest(Throwable $t): never
         throw $t;
     }
 
+    /**
+     * @param array<mixed> $testArguments
+     */
+    protected function runTestMethod(array $testArguments): mixed
+    {
+        /** @phpstan-ignore method.dynamicName */
+        return $this->{$this->methodName}(...$testArguments);
+    }
+
     /**
      * Returns the data set as a string compatible with the --filter CLI option.
      *
@@ -1313,8 +1322,7 @@ private function runTest(): mixed
         $this->startErrorLogCapture();
 
         try {
-            /** @phpstan-ignore method.dynamicName */
-            $testResult = $this->{$this->methodName}(...$testArguments);
+            $testResult = $this->runTestMethod($testArguments);
 
             $this->verifyErrorLogExpectation();
         } catch (Throwable $exception) {

[binaryfire]

@sebastianbergmann Thank you, I think that's exactly what we need! But please wait for @albertcht to confirm as he has more experience with the Swoole internals and can verify.

[sebastianbergmann]

I was pointed to https://github.com/WyriHaximus/reactphp-phpunit-run-tests-in-fiber/blob/main/src/RunTestsInFibersTrait.php#L69-L90 which was implemented by @WyriHaximus for what appears to me the same purpose, but in the context of ReactPHP.

[albertcht]

@sebastianbergmann Thank you for considering this! I can confirm that making runTestMethod() protected and overridable would solve our problem perfectly.

I'd like to emphasize that this benefit extends beyond just Swoole. As you mentioned, @WyriHaximus implemented a similar solution for ReactPHP. This pattern is equally applicable to:

• Swoole (coroutine-based async runtime)
• ReactPHP (event loop with Fibers)
• AMPHP (another async framework)
• Custom coroutine schedulers that developers may implement

The common requirement across all these scenarios is: each test method needs to run inside its own coroutine/fiber scheduler context. The runTestMethod() hook you've proposed is the perfect extension point for this - it's precisely where we need to wrap the test method invocation with scheduler initialization.

Without this extension point being officially supported, developers are forced to resort to workarounds like patching vendor files at runtime. For example, Hyperf community has created packages like https://github.com/friendsofhyperf/co-phpunit/blob/main/phpunit-patch.php that programmatically modify PHPUnit's internals just to make tests work.

Making runTestMethod() a protected, documented extension point would eliminate the need for these fragile runtime patches while providing a clean, maintainable solution for all async PHP frameworks. The approach is framework-agnostic since PHPUnit doesn't need to know about specific implementations, and it wouldn't affect standard PHPUnit usage at all as it's purely opt-in for those who need it.

This would be tremendously valuable for the entire async PHP ecosystem. Thank you for being open to this solution!

[WyriHaximus]

Would it be possible to add a configuration or extension hook to provide a test runner wrapper/decorator so? Having the runTestMethod method protected and overridable is great, but it still seems somewhat fragile, unless that is the designated hook and documented as such?

[sebastianbergmann]

unless that is the designated hook and documented as such?

The patch in #6450 (comment) would make the runTestMethod() overwritable as it is declared protected. Also note that it would not be marked @internal. Therefore it would become part of the public API of TestCase.