8376286: VM asserts with "assert(false) failed: Possible safepoint reached by thread that does not allow it" when running with -XX:+ExitOnFullCodeCache

[0zzy0sbourne]

With -XX:+ExitOnFullCodeCache, handle_full_code_cache() calls before_exit() directly from a compiler thread. This triggers an assertion failure because compiler threads are not allowed to reach a safepoint, and before_exit() acquires the BeforeExit_lock which may safepoint.

Replace the manual before_exit() + exit_globals() + vm_direct_exit() sequence with vm_exit(), which transitions the thread state to _thread_in_vm before initiating shutdown, making safepoint-reaching operations legal.

Reproducible with

java -XX:+ExitOnFullCodeCache -XX:NonNMethodCodeHeapSize=500M -XX:ReservedCodeCacheSize=512008K -version

Testing

Built hotspot (linux-x86_64-server-release) successfully.

---

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

• JDK-8376286: VM asserts with "assert(false) failed: Possible safepoint reached by thread that does not allow it" when running with -XX:+ExitOnFullCodeCache (Bug - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/31667/head:pull/31667
$ git checkout pull/31667

Update a local copy of the PR:
$ git checkout pull/31667
$ git pull https://git.openjdk.org/jdk.git pull/31667/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 31667

View PR using the GUI difftool:
$ git pr show -t 31667

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/31667.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back 0zzy0sbourne! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@0zzy0sbourne The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[openjdk]

The total number of required reviews for this PR has been set to 2 based on the presence of this label: hotspot-compiler. This can be overridden with the /reviewers command.

[mlbridge]

Webrevs

• 00: Full (d83fcb27)