Skip to content

Cross module r2r rebased#70581

Closed
davidwrighton wants to merge 9 commits intodotnet:mainfrom
davidwrighton:cross_module_r2r_rebased
Closed

Cross module r2r rebased#70581
davidwrighton wants to merge 9 commits intodotnet:mainfrom
davidwrighton:cross_module_r2r_rebased

Conversation

@davidwrighton
Copy link
Copy Markdown
Member

No description provided.

…ation to Crossgen2

- Refactor Module into ModuleBase and Module
  - The goal is to have allow a subset version of Module which can only hold refs, this is to be used by the manifest module in an R2R image to allow for version resilient cross module references.
  - Update handling of ModuleBase so that its used everywhere that tokens are parsed from R2R
  - Remove ENCODE_MODULE_ID_FOR_STATICS and ENCODE_ACTIVE_DEPENDENCY
    - These were only used for NGEN, and conflict with easy impelmentation for the ModuleBase concept
  - Remove locking in ENCODE_STRING_HANDLE processing, and tweak comments. Comments applied to the removed ngen based code, and the lock was only necessary for the old ngen thing.
  - Adjust ComputeLoaderModuleWorker for locating loader module
    - Follow comment more accurately, to avoid putting every generic into its definition module. This will make R2R function lookup able to find compiled instantiations in some cases. This may be what we want long term, it may not.
  - Remove MemberRefToDesc map and replace with LookupMap like the other token types. We no longer make use of the hot table, so this is more efficient
    - Also reduces complexity of implementation of ModuleBase

- Build fixup to describe a single method as a standalone blob of data
  - There are parallel implementations in Crossgen2 and in the runtime
  - They produce binary identical output
  - Basic R2RDump support for new fixup

- Adjust module indices used within the R2R format to support a module index which refers to the R2R manifest metadata. This requires bumping the R2R version to 6.2
  - Add a module index between the set of assembly refs in the index 0 module and the set of assembly refs in the R2R manifest metadata

- Adjust compilation dependency rules to include a few critical AsyncStateMachineBox methods

- Remove PEImage handling of native metadata which was duplicative

- Do not enable any more devirtualization than was already in use, even in the cross module compilation scenario. In particular, do not enable devirtualization of methods where the decl method isn't within the version bubble, even if the decl method could be represented with a cross-module reference token. (This could be fixed, but is out of scope for this initial investigation)

Make the compilation deterministic in this new model, even though we are generating new tokens on demand
  - Implement this by detecting when we need new tokens during a compile, and recompiling with new tokens when necessary
  - This may result in compiling the same code as much as twice

Compile the right set of methods with cross module inlining enabled
- Add support for compiling the called virtual methods on generic types
  - This catches the List<T> and Dictionary<TKey,TValue> scenarios

- Add command line switches to enable/disable the new behavior

NOTE: Module::IsInSameVersionBubble doesn't actually behave as expected. Currently we're generating a native manifest metadata in all circumstances, which will cause all modules to be treated as having InputBubble semantics. However, fortunately, we don't actually do anything like that at runtime. The only use of IsInSameVersionBubble is used to control R2R function lookup, which will not cause a problem.
- We don't model the details necessary to prevent this from possibly being a problem at this time
@davidwrighton davidwrighton force-pushed the cross_module_r2r_rebased branch from 99a801e to 308c5c1 Compare June 10, 2022 23:35
@davidwrighton
Copy link
Copy Markdown
Member Author

/azp help

@azure-pipelines
Copy link
Copy Markdown

Supported commands
  • help:
    • Get descriptions, examples and documentation about supported commands
    • Example: help "command_name"
  • list:
    • List all pipelines for this repository using a comment.
    • Example: "list"
  • run:
    • Run all pipelines or specific pipelines for this repository using a comment. Use this command by itself to trigger all related pipelines, or specify specific pipelines to run.
    • Example: "run" or "run pipeline_name, pipeline_name, pipeline_name"
  • where:
    • Report back the Azure DevOps orgs that are related to this repository and org
    • Example: "where"

See additional documentation.

@davidwrighton
Copy link
Copy Markdown
Member Author

/azp list

@azure-pipelines
Copy link
Copy Markdown

@davidwrighton
Copy link
Copy Markdown
Member Author

/azp run runtime-coreclr crossgen2

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@davidwrighton
Copy link
Copy Markdown
Member Author

/azp run runtime-coreclr crossgen2

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@ghost ghost locked as resolved and limited conversation to collaborators Jul 20, 2022
@davidwrighton davidwrighton deleted the cross_module_r2r_rebased branch April 13, 2023 18:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant