Experiment: PGO for desktop application

[deepak1556]

From Profile Guided Optimization

Profile information enables better optimization. For example, knowing that a branch is taken very frequently helps the compiler make better decisions when ordering basic blocks. Knowing that a function foo is called more frequently than another function bar helps the inliner. Optimization levels -O2 and above are recommended for use of profile guided optimization.

Clang supports profile guided optimization with two different kinds of profiling. A sampling profiler can generate a profile with very low runtime overhead, or you can build an instrumented version of the code that collects more detailed profile information. Both kinds of profiles can provide execution counts for instructions in the code and information on branches taken and function invocation.

Regardless of which kind of profiling you use, be careful to collect profiles by running your code with inputs that are representative of the typical behavior. Code that is not exercised in the profile will be optimized as if it is unimportant, and the compiler may make poor optimization choices for code that is disproportionately used while profiling.

Chromium build configuration has opt-in support for PGO https://chromium.googlesource.com/chromium/src.git/+/refs/heads/main/docs/pgo.md. The issue is to track the effort of optimizing the Electron runtime with PGO for VSCode. This will be a multi-step process and initially some manual steps are also involved, we will investigate automating the whole flow if the results are satisfying.

For this experiment, we are using the instrumented version to collect the profile. Steps involved are as follows,

Instrumentation Phase

Building

• 🏃 Create MS Electron builds with following additional build flags

chrome_pgo_phase = 1
enable_resource_allowlist_generation = false

• Create nightly exploration builds based on the above custom runtime build

Running

• Get builds from https://builds.code.visualstudio.com/builds/exploration corresponding to the branch robo/explore_pgo

NB: Builds from branch robo/explore_pgo will always reflect latest changes from main, only difference is the instrumented runtime build, so it will not affect your everyday use.

• Selfhost on the build and perform your everyday tasks, at the end of every run files with extension *.profraw will be generated in the user data directory under PGO_Profiles folder.

• 🏃 At EOD, upload the generated *.profraw files to #proj-pgo channel

NB: There is no limit on the number of days to selfhost on this build during the experiment. Scheduled nightly updates will be available.

Optimization Phase

Building

• Combine the .profraw files into .profdata using llvm-profdata script
• Build MS Electron builds with generated .profdata using the following build flags

chrome_pgo_phase = 2
pgo_data_path = {path-to-the-profdata}

• Test startup numbers and feed this build into the instrumentation phase until a stable range is obtained.
• Ship optimized build to Insiders
• Ship optimized build to Stable

[deepak1556]

Couple of gotchas:

1. Prebuilt profiling runtime binaries (libclang_rt.profile-<arch>.a) are not available for Linux arm64, Linux armhf and Windows arm64 from the chromium source tree. Although it is possible to build them from source, it will take considerable changes to the MS Electron build pipeline which is outside the scope of current phase for this experiment. We will skip instrumenting the corresponding VSCode builds for sake of simplicity.

2. Profiles are stored for each process from a file pool (which is currently set to 4 for child processes, ref: https://bugs.chromium.org/p/chromium/issues/detail?id=1059335) and one separate file is used for the main process. Also, the location of these files unless modified will write to the current working directory as seen in https://source.chromium.org/chromium/chromium/src/+/main:content/common/profiling_utils.cc;l=35-61. This is an issue when running product builds of VSCode which will try to write in the installation folder and causes the sandboxed child process to crash as seen below,

[57408:0112/184938.708273:ERROR:network_service_instance_impl.cc(470)] Network service crashed, restarting service.
[57408:0112/184938.709155:ERROR:profiling_utils.cc(87)] Opening file: /child_pool-0.profraw failed with -5
[57666:0112/184938.764001:ERROR:validation_errors.cc(106)] Invalid message: VALIDATION_ERROR_UNEXPECTED_NULL_POINTER (null field 1)

However, we can mitigate this by controlling the path via LLVM_PROFILE_FILE environment variable. I have decided to store the files in the user data directory, an example folder structure is as follows.

[deepak1556]

• Backport latest changes based on https://bugs.chromium.org/p/chromium/issues/detail?id=1071664 and https://bugs.chromium.org/p/chromium/issues/detail?id=1242138 into Electron shutdown code path to properly dump profile data on shutdown.