Fix type mismatches in VN

[SingleAccretion]

Value numbering can see through some forms of aliasing, but does not always account for type mismatches properly. One example of an incorrect behavior can be seen here: #58309 (comment), there may be others.

Problematic places need to be investigated and fixed. A short-term solution could be to give up and give a "new, unique" VN upon seeing such casts, long-term we should consider using GT_BITCAST as a VNFunc for them, with support for folding (and delete assertion propagation code that expects these type mismatches), since, for better or worse, we have a strongly typed model in VN.

category:correctness
theme:value-numbering
skill-level:expert
cost:small
impact:small

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Value numbering can see through some forms of aliasing, but does not always account for type mismatches properly. One example of an incorrect behavior can be seen here: #58309 (comment), there may be others.

Problematic places need to be investigated and fixed. A short-term solution could be to give up and give a "new, unique" VN upon seeing such casts, long-term we should consider using GT_BITCAST as a VNFunc for them, with support for folding (and delete assertion propagation code that expects these type mismatches), since, for better or worse, we have a strongly typed model in VN.

Author:	SingleAccretion
Assignees:	-
Labels:	area-CodeGen-coreclr, untriaged
Milestone:	-

[SingleAccretion]

So, one would think this is a rather simple fix: just update VNApplySelectorsTypeCheck and VNApplySelectorsAssignTypeCoerce to cover for the type mismatches.

However, things get complicated by the fact that the VNF_MapSelect and VNF_MapStore machinery can give struct and ref types to VNs that actually represent proper field values (i. e. not just to "field map" VNs, though those also tend to appear in unexpected places and we end up with Cast(ref -> int)). Need to spend some time to review the code and decide what should be done about this (noting as well that this is small CQ issue too, as the additional casts block the recursive traversal inside VNApplySelectors).

More type mismatches: VNApplySelectors normalized SIMD types, VNApplySelectorsAssign does not. Edit: fixed in #61370.

And more:

***** BB01, STMT00000(before)
N006 (  6,  6) [000005] -A-XG-------              *  ASG       float 
N004 (  4,  4) [000004] *--XG--N----              +--*  IND       float 
N003 (  2,  2) [000018] -------N----              |  \--*  ADD       byref 
N001 (  1,  1) [000000] ------------              |     +--*  LCL_VAR   ref    V01 arg1         u:1
N002 (  1,  1) [000017] ------------              |     \--*  CNS_INT   long   24 field offset Fseq[SimdField, X]
N005 (  1,  1) [000003] ------------              \--*  CNS_DBL   float  1.0000000000000000

N001 [000000]   LCL_VAR   V01 arg1         u:1 => $80 {InitVal($40)}
N002 [000017]   CNS_INT   24 field offset Fseq[SimdField, X] => $100 {LngCns:  24}
N003 [000018]   ADD       => $140 {ADD($80, $100)}
N005 [000003]   CNS_DBL   1.0000000000000000 => $180 {FltCns[1.000000]}
  Known type Vector4
  VNApplySelectors:
    VNForHandle(SimdField) is $1c0, fieldType is simd16, size = 16
    VNForMapSelect($81, $1c0):simd16 returns $200 {$81[$1c0]}
    VNForMapSelect($200, $80):simd16 returns $201 {$200[$80]}
  VNApplySelectorsAssign:
    VNForHandle(X) is $1c1, fieldType is float
    VNForMapStore($201, $1c1, $180):float in BB01 returns $240 {$201[$1c1 := $180]}
    VNForMapStore($200, $80, $240):simd16 in BB01 returns $280 {$200[$80 := $240]}
  VNApplySelectorsAssign:
    VNForHandle(SimdField) is $1c0, fieldType is struct
    VNForCastOper(float) is $43
    VNForCast($280, $43) returns $2c0 {Cast($280, $43)}
    Cast to float inserted in VNApplySelectorsAssignTypeCoerce (elemTyp is simd16)
    VNForMapStore($81, $1c0, $2c0):struct in BB01 returns $300 {$81[$1c0 := $2c0]}
  fgCurMemoryVN[GcHeap] assigned for StoreField at [000005] to VN: $300.
N006 [000005]   ASG       => $VN.Void

The above example is quite puzzling in general.

One more example, this time things are, arguably, actually "broken":

private static int Problem(ClassWithTwoFields cwtf)
{
    cwtf.Second = 2;

    IL.Emit.Ldarg(0);
    IL.Emit.Ldflda(new(typeof(ClassWithTwoFields), "First"));
    IL.Emit.Ldc_I8(1);
    IL.Emit.Stind_I8();

    return cwtf.Second;
}

class ClassWithTwoFields
{
    public int First;
    public int Second;
}

***** BB01, STMT00002(before)
N005 (  5,  5) [000012] ---XG-------              *  RETURN    int   
N004 (  4,  4) [000011] ---XG-------              \--*  IND       int   
N003 (  2,  2) [000018] -------N----                 \--*  ADD       byref 
N001 (  1,  1) [000010] ------------                    +--*  LCL_VAR   ref    V02 arg2         u:1 (last use)
N002 (  1,  1) [000017] ------------                    \--*  CNS_INT   long   12 field offset Fseq[Second]

N001 [000010]   LCL_VAR   V02 arg2         u:1 (last use) => $80 {InitVal($40)}
N002 [000017]   CNS_INT   12 field offset Fseq[Second] => $100 {LngCns:  12}
N003 [000018]   ADD       => $140 {ADD($80, $100)}
  VNApplySelectors:
    VNForHandle(Second) is $180, fieldType is int
      AX2: $180 != $181 ==> select([$203]store($201, $181, $240), $180) ==> select($201, $180) remaining budget is 99.
      AX1: select([$81]store($201, $180, $200), $180) ==> $200.
    VNForMapSelect($203, $180):int returns $200 {$1c0[$80 := $40]}
      AX1: select([$1c0]store($200, $80, $40), $80) ==> $40.
    VNForMapSelect($200, $80):int returns $40 {IntCns 2}
N004 [000011]   IND       => <l:$1c5 {norm=$40 {IntCns 2}, exc=$280 {NullPtrExc($80)}}, c:$1c4 {norm=$c1 {MemOpaque:NotInLoop}, exc=$280 {NullPtrExc($80)}}>
N005 [000012]   RETURN    => $c2 {MemOpaque:NotInLoop}

When we "give up" in VNApplySelectorsAssign, that is only for the field being stored, but in all likelihood we should propagate the "impropriety" of the store up the VN tree and end up with an opaque VN for the heap as a whole.

[SingleAccretion]

Another case of type mismatch, the infamous "promoted struct with one field":

N011 ( 22, 16) [000008] -ACXG---R---              *  ASG       struct (copy) $VN.Void
N010 (  3,  2) [000006] D------N----              +--*  LCL_VAR   struct<System.Collections.Immutable.ImmutableArray`1[[Microsoft.CodeAnalysis.CSharp.Symbol, Microsoft.CodeAnalysis.CSharp, Version=2.10.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]], 8>(P) V08 loc0         d:2
                                                  +--*    ref    V08.array (offs=0x00) -> V37 tmp25
N009 ( 18, 13) [000004] --CXG-------              \--*  CALL      struct Microsoft.CodeAnalysis.CSharp.Binder.GetCandidateMembers $1c0
N005 (  1,  1) [000000] ------------ arg0 in rcx     +--*  LCL_VAR   ref    V01 arg1         u:1 (last use) $81
N006 (  1,  1) [000001] ------------ arg1 in rdx     +--*  LCL_VAR   ref    V02 arg2         u:1 (last use) $82
N007 (  1,  1) [000002] ------------ arg2 in r8      +--*  LCL_VAR   int    V04 arg4         u:1 $c1
N008 (  1,  1) [000003] ------------ arg3 in r9      \--*  LCL_VAR   ref    V05 arg5         u:1 $83

Here the field, ref V08, gets a VN of TYP_STRUCT...

Edit: fixed in #64898.