You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Program that should not compile, using statically resolved type parameters, gets runtime exception instead #4924
Using statically resolved type parameters, we ran into a problem where we wrote some code that was wrong according to what the types said, and yet compiled (and so we didn't realize it was wrong). While writing unit tests against it, we got an InvalidCastException, which was exceedingly puzzling, because nowhere in our code did we use casting. I have boiled this down to a minimal working example.
Repro steps
Provide the steps required to reproduce the problem
Read the code and comments in Program.fs to get an idea of what's going on hear. SRTP.fs contains the backing statically resolved type parameter code etc.
Compile.
Run the resulting .exe and observe the output.
Expected behavior
Program should not typecheck.
Actual behavior
Depends on what you do with the values that do not match their declared types:
If you just ignore those values or generally print them to the screen, their declared type is ignored and they just behave the way you would expect given their real type.
If you try to use them in a more sophisticated way such as pattern matching on them, which is of course according to their declared type in the code, you will get an InvalidCastException at runtime explaining that the real type can't be cast to the supposed type.
Known workarounds
I can correct my code and get it to work but my only workaround for regaining confidence that code that compiles is correct is to stop using statically resolved type parameters.
Related information
Provide any related information
Windows 7
F# 4.1
.NET >= 4.5 (so you can use statically resolved type parameters)
Visual Studio 2017
From my perspective, this is a severe bug, as it really destroys my confidence in the F# compiler's ability to typecheck my code.
Please let me know if more detailed information is needed. This is a high priority issue for me as it seriously compromises the integrity of my F# code.
If it is useful, I now have a branch where I got the compile to fail by using more member constraints to fake out type-level guarantees on the conceptual higher-kinded types I was trying to model. This is a verbose workaround but one that bolsters the overall solidity of my code and in my mind the fact that it worked probably does reduce the severity of this bug.
I'm seriously impressed by this code, I've never used SRTP to such complexity. I'm curious, as I see quite some syntax that doesn't compile with F# 4.0, and I wasn't aware of syntax changes or significant added features in F# 4.1.
Which makes you wonder whether this is caused by the new feature's implementation or that this has always been a bug (provided it is one, I cannot be the judge of that). Not sure if this can be rewritten with F# 4.0 syntax though and if so, whether that would yield the same results.
I've managed before to create compilable code with SRTP, that threw hard-to-diagnose errors. In my case that was caused by the compiler allowing the order to be random on static inline class members, but that in reality they need to be in the correct order, as if they are in a module, otherwise inexplicable compile or runtime errors can occur. Since you use a lot of static members, I would wonder whether this could in any way be related to an issue with the order of declaration.
Just my 2p, perhaps not very useful, but your code intrigued me ;).
PS: it may be useful to add what would be the correct way of calling your code, to see it run without that error. You write "This type clearly does not match the implementation", and I'm sure for some people it is "clear", but it wasn't for me, and I find SRTP quite hard to read and diagnose whenever it gets beyond a few lines.
@abelbraaksma Thanks for your intriguing comments. I will certainly own up to the fact that this is insanely complex SRTP code. Polite of you to say you are impressed; you should rather say disgusted! But sadly, it's the part of my codebase keeping me sane (there is much much more code written in this style in my actual code base: I'm a bit of a crackpot I suppose). Basically I have found that if you write code with Either<_,_> (or Result<_,_>), list<_> and _*_ much, you quickly need traverse; but in a rather monomorphic language you end up with mxn implementations, which is not fun. Anyway that is my mad scientist explanation for this exotic specimen.
Yes, this code is enabled by F# 4.1 and was not possible in F# 4.0 AFAIK. I looked into it in F# 4.0 and couldn't see a way to make it work anyhow. My coworker resurrected the question after they added the new tweaks in F# 4.1 and as odd as the little addition was it allowed us to make it all work (or so we thought, haha!).
Good thought about ordering. I don't pretend to understand what's happening well enough to know whether that is the issue at stake but it's good to know that someone else has hit something like this.
You are right in saying I was a bit hasty about saying it was "clear" that the type didn't match the implementation; should have explained a bit. The intend of traverseBB is to be traverse from Haskell, which has the type signature:
traverse :: (Traversable t, Applicative f) => (a -> f b) -> t a -> f (t b)
with the specific conditions that both t and f are bifunctors, but we are only caring about their second type parameter for the sake of this traverse (i.e. in my actual codebase I have traverseAB, traverseBA...). So for example, since my first function is maybeZero :: Int -> Maybe Int (using Haskell syntax for parity with the above, and because the lack of higher-kinded types means it would only be pseudo-syntax if I mimicked F# here), traverseBB maybeZero should have the type t Int -> Maybe (t Int). The t Int in question is (List String, Int). Thus the type should be (List String, Int) -> Maybe (List String, Int), or reverting to F# syntax now that everything is plugged in, the type should be list<string>*int -> Maybe<list<string>*int>, which it should be safe to say at this point clearly does not match list<string>*int -> Either<string,list<string>> (Maybe<_> is a type alias for Either<unit,_>).
I recognize that in some sense I am abusing F# in all this. Perhaps the users who abuse the program are the ones who tend to find the bugs.
I'm hesitant to say that your example is written incorrectly, because nobody's ever issued a guide on the "correct" way to write SRTP, but congratulations (?) on your journey along the unmarked path to, uh, SRTP enlightenment, or something. You're correct that some of the constructs you're using weren't possible until F# 4.1; in fact, I'm guilty of having made it easier when I reported one of the things you're now using in #392 (wow, that was a long time ago). The reason you're getting the InvalidCastException is because the constraint solver likes to try to use obj if it can't infer anything better in certain cases, and you've found one of them: specifically, use of explicit type parameters to inline functions (i.e. f<Multifunctor, _> x y z). This can result in code that typechecks, but fails at runtime in strange ways such as this. On principle it's easy to suggest that any situation where typechecking should intuitively fail but does not constitutes a bug or soundness issue, but SRTP rubs up against a lot of corner cases: consider the following:
let inlinef(x :^a)=(^a:(memberToString: unit -> string) x)// which isval inlinef:x:^a ->string when ^a :(member ToString :^a->string)let ()=printfn "%s" (f 1)(*System.NullReferenceException:Object reference not set to an instance of an object.at <StartupCode$FSI_0003>.$FSI_0003.main@()Stopped due to error*)
Yet, this works fine:
let inlinef x =(^a:(memberToString: unit -> string)(x :> obj))(* harmless warning about ^a being solved to obj *)val inlinef:x:'a ->string
...and of course, constraints not involving bowels of .NET tend to be far better behaved already:
let inlineg x =(^a:(memberHoopty: unit -> string) x)// which isval inlineg:x :^a ->string when ^a :(member Hoopty :^a->string)let _ =g 1(*g 1;;--^stdin(16,4):error FS0001:The type 'int' does not support the operator 'Hoopty'*)
But I digress.
Indeed, SRTP "abuse" such as this is a part of the language few users explore, and despite its utility (nobody is arguing that it does not enable a number of things otherwise not possible) there does seem to be some sentiment that its use is a bridge too far. That said, the code you've written goes against one of the (unwritten) teachings of the SRTP Elders: instead of using explicit type parameters in order to add to solver's overload candidate set, you need to instead--unfortunately--pass a dummy value (i.e. term, not type) that will appropriately influence the solver. I'm sure you've probably noticed that most ways of trying to avoid the dummy value result in strange errors from the compiler involving internally-generated type parameters with names like ^?123. Ideally this wouldn't be necessary (and the fact that you have to do this is not, as far as I know, by design), but that's what you have to do today, and while I'm not speaking for anyone but myself here, I imagine these issues won't get looked at any sooner than F# 5.0, which is when other SRTP rollup issues are slated for review.
So, let's apply this one weird trick discovered by a local F# wonk (compiler authors hate him!):
diff --git a/SRTP/SRTP.fs b/SRTP/SRTP.fs
index d23fd18..8b6a499 100644
--- a/SRTP/SRTP.fs+++ b/SRTP/SRTP.fs@@ -28,37 +28,41 @@ type Multifunctor() =
static member inline mapB (f : ^b -> ^b0, x : Either< ^a,^b>) : Either< ^a,^b0> = Either.rmap f x
static member inline mapB (f : ^b -> ^b0, (a : ^a, b : ^b, c : ^c)) : ^a * ^b0 * ^c = (a, f b, c)
-let inline mapB_< ^mf, ^f, ^a, ^b when (^mf or ^a) : (static member mapB : ^f * ^a -> ^b)> (f : ^f, x : ^a) : ^b =- ((^mf or ^a) : (static member mapB : ^f * ^a -> ^b) (f, x))+ static member inline Invoke (f : ^f, x : ^a) =+ let inline call (d : ^mf) =+ ((^mf or ^b) : (static member mapB : ^f * ^a -> ^b) (f, x)) in+ call Unchecked.defaultof<Multifunctor>++let inline mapB_ (f : ^f, x : ^a) : ^b = Multifunctor.Invoke (f, x)
let inline mapB (f : ^f) (x : ^a) : ^b when (Multifunctor or ^a) : (static member mapB : ^f * ^a -> ^b) =
- mapB_<Multifunctor,^f,^a,^b> (f, x)+ mapB_ (f, x)
type Multiapplicative() =
static member inline wrapB (_ : Either< ^a,^b>, x0 : ^b) : Either< ^a,^b> = Right x0
static member inline wrapB (_ : Result< ^a,^b>, x0 : ^b) : Result< ^a,^b> = Error x0
-let inline wrapB_< ^mf, ^a, ^b when (^mf or ^b) : (static member wrapB : ^b * ^a -> ^b)> (instance : ^b, x : ^a) : ^b =+let inline wrapB_ (instance : ^b, x : ^a) : ^b =
((^mf or ^b) : (static member wrapB : ^b * ^a -> ^b) (instance, x))
let inline wrapB (x : ^a) : ^b when (Multiapplicative or ^b) : (static member wrapB : ^b * ^a -> ^b) =
- wrapB_<Multiapplicative,^a,^b> (Unchecked.defaultof< ^b>, x)+ wrapB_ (Unchecked.defaultof< ^b>, x)
type Multimonad() =
static member inline bindB (f : ^b -> Either< ^a,^b0>, x : Either< ^a,^b>) : Either< ^a,^b0> = Either.rbind f x
static member inline bindB (f : ^b -> Result< ^a,^b0>, x : Result< ^a,^b>) : Result< ^a,^b0> = failwith "TODO: why u no haz Result.errorBind"
-let inline bindB_< ^mf, ^f, ^a, ^b when (^mf or ^a) : (static member bindB : ^f * ^a -> ^b)> (f : ^f, x : ^a) : ^b =+let inline bindB_ (f : ^f, x : ^a) : ^b =
((^mf or ^a) : (static member bindB : ^f * ^a -> ^b) (f, x))
let inline bindB (f : ^f) (x : ^a) : ^b when (Multimonad or ^a) : (static member bindB : ^f * ^a -> ^b) =
- bindB_<Multimonad,^f,^a,^b> (f, x)+ bindB_ (f, x)
type Multitraverse() =
static member inline traverseBB (f : ^b -> ^b0, (a : ^a, b : ^b)) : ^b1
when (Multifunctor or ^b0) : (static member mapB : _ * ^b0 -> ^b1) =
- mapB_<Multifunctor,_,_,_> (addfst a, (f b))+ mapB_ (addfst a, (f b))
static member inline traverseBB (f : ^a -> ^a0, (_ : ^b, b : ^a, _ : ^c) as t) : ^a1
when (Multifunctor or ^a0) : (static member mapB : _ * ^a0 -> ^a1) =
- mapB_<Multifunctor,_,_,_> ((fun x -> mapB (k x) t), f b)+ mapB_ ((fun x -> mapB (k x) t), f b)
static member inline traverseBB (f : ^b -> ^b0, x : Either< ^a,^b>) : ^b1
when (^b0 or Multiapplicative) : (static member wrapB : _ -> ^b0)
@@ -67,12 +71,12 @@ type Multitraverse() =
and (^b1 or Multifunctor) : (static member mapB : _*_ -> _ ) =
Either.match_ (Left >> wrapB) (mapB Right << f) x
-let inline traverseBB_< ^mf, ^a, ^b, ^c, ^d when (^mf or ^a) : (static member traverseBB : (^a -> ^b) * ^c -> ^d)> (f : ^a -> ^b, x : ^c) : ^d =+let inline traverseBB_ (f : ^a -> ^b, x : ^c) : ^d =
((^mf or ^a) : (static member traverseBB : (^a -> ^b) * ^c -> ^d) (f, x))
let inline traverseBB (f : ^a -> ^b) (x : ^c) : ^d
when (Multitraverse or ^c) : (static member traverseBB : (^a -> ^b) * ^c -> ^d)
and (Multimonad or ^b) : (static member bindB : (_ -> ^b) * ^b -> ^b) =
// The intent of requiring monad is just trying to be lawful
- traverseBB_<Multitraverse,^a,^b,^c,^d> (f, x)+ traverseBB_ (f, x)
Now our program fails to typecheck as hoped:
SRTP\Program.fs(11,5): error FS0001: Type mismatch. Expecting a 'Either<string,string list>' but given a 'Either<string,(string list * 'a)>' The type 'string list' does not match the type 'string list * 'a'
SRTP\Program.fs(11,16): error FS0001: Type mismatch. Expecting a 'int -> Either<string,'a>' but given a 'int -> Maybe<int>' The type 'string' does not match the type 'unit'
SRTP\Program.fs(20,5): error FS0001: Type mismatch. Expecting a 'Either<(Maybe<int> * Maybe<bool>),string list>' but given a 'Either<(Maybe<int> * Maybe<bool>),(string list * 'a)>' The type 'string list' does not match the type 'string list * 'a'
SRTP\Program.fs(20,16): error FS0001: Type mismatch. Expecting a 'Maybe<int> -> Either<(Maybe<int> * Maybe<bool>),'a>' but given a 'Maybe<int> -> Either<string,Maybe<int>>' The type 'Maybe<int> * Maybe<bool>' does not match the type 'string'
Hope this helps in the meantime. I've submitted a talk to Open F# this year about this very kind of thing, so if it gets in then it should be right up your alley! Also, I suspect you'll find FSharpPlus to be a very useful library in this same vein.
@cartermp I'll let you know where to send the coffee. ;)
Reacted by Jared Hester, Keith Pinson and Abel Braaksma
@drvink Thanks for your guidance on SRTP. I will update my code accordingly. Indeed, it is not well documented; haha! I actually fixed an error in the docs while trying to learn about it, lol...
@Kazark We will need a much smaller minimal repro of the underlying problem - that's a whole lot of SRTP in that repro. It's a good stress test I suppose, but it's not the kind of use of SRTP I'd personally want to encounter often :)
typeFoo<'a,'b>= Bar of'a*'btypeMultifunctor()=static member inlinemapB(f :^b ->^b0,(a :^a,b :^b)):^a * ^b0 =(a, f b)static member inlinemapB(f :^b ->^b0,x :Foo<^a,^b>)= f
let inlinemapB_<^mf,^f,^a,^bwhen(^mf or ^a):(static member mapB :^f*^a->^b)>(f :^f,x :^a):^b =((^mfor^a):(static membermapB:^f*^a->^b)(f, x))typeMultitraverse()=static member inlinetraverseBB(f :^b1 ->^b2,x :^a1 * ^c2 * ^d3)=fun a -> printfn "hi"; a
static member inlinetraverseBB(f :^b ->^b0,(a :^a,b :^b)):^b1when (Multifunctor or ^b0):(static member mapB :_ *^b0->^b1)=
mapB_<Multifunctor,_,_,_>((fun x -> x),(f b))let inlinetraverseBB_<^mf,^a,^b,^c,^dwhen(^mf or ^a):(static member traverseBB :(^a->^b)*^c->^d)>(f :^a ->^b,x :^c):^d =((^mfor^a):(static membertraverseBB:(^a->^b)*^c->^d)(f, x))let inlinetraverseBB(f :^a ->^b)(x :^c):^dwhen (Multitraverse or ^c):(static member traverseBB :(^a->^b)*^c->^d)=// The intent of requiring monad is just trying to be lawful
traverseBB_<Multitraverse,^a,^b,^c,^d>(f, x)letprivatemaybeZero:int ->Foo<'a,'b>=fun x -> Bar(0,2)// This type clearly does not match the implementation, and yet it type checksletprivatewhatInTheWorld:list<string>*int -> System.Random =
traverseBB maybeZero
smaller again.
Using DU's with less than 2 generics will result in typecheck errors again.
now it's very obvious that the return type is super wrong.
Reacted by Keith Pinson, Abel Braaksma and Chris Mills
Right now it's so amusing to see what the compiler outputs if at particular points you introduce a new static typvar vs. using existing typvars from parameters 😂
oh please compare the error output between these 3 codes ahahahaha 😂 It's too funny!
Example 1:
typeFoo<'a,'b>= Bar of'a*'btypeMultifunctor()=static member inlinemapB(f :^b ->^b0,(a :^a,b :^b)):^a * ^b0 =(a, f b)static member inlinemapB(f :^b ->^b0,x :Foo<^a,^b0>)= f
let inlinemapB_<^mf,^f,^a,^bwhen(^mf or ^a):(static member mapB :^f*^a->^b)>(f :^f,x :^a):^b =((^mfor^a):(static membermapB:^f*^a->^b)(f, x))typeMultitraverse()=static member inlinetraverseBB(f :^b1 ->^b2,x :^a1 * ^c2 * ^d3)=fun a -> printfn "hi"; a
static member inlinetraverseBB(f :^b ->^b0,(a :^a,b :^b)):^b1when (Multifunctor or ^b0):(static member mapB :_ *^b0->^b1)=
mapB_<Multifunctor,_,_,_>((fun x -> a,x),(f b))let inlinetraverseBB_<^mf,^a,^b,^c,^dwhen(^mf or ^a):(static member traverseBB :(^a->^b)*^c->^d)>(f :^a ->^b,x :^c):^d =((^mfor^a):(static membertraverseBB:(^a->^b)*^c->^d)(f, x))let inlinetraverseBB(f :^a ->^b)(x :^c):^dwhen (Multitraverse or ^c):(static member traverseBB :(^a->^b)*^c->^d)=// The intent of requiring monad is just trying to be lawful
traverseBB_<Multitraverse,^a,^b,^c,^d>(f, x)letprivatemaybeZero:int ->Foo<int,int>=fun x -> Bar(0,2)// This type clearly does not match the implementation, and yet it type checksletprivatewhatInTheWorld:list<string>*int -> System.Random =
traverseBB maybeZero
Compiler like => "Ah you want to confuse me? ** you, I give up (Internal error etc...)"
I changed from (fun x -> x) to (fun x -> a,x) in traverseBB with the SRTP constraint.
Example 2:
typeFoo<'a,'b>= Bar of'a*'btypeMultifunctor()=static member inlinemapB(f :^b ->^b0,(a :^a,b :^b)):^a * ^b0 =(a, f b)static member inlinemapB(f :^b ->^b0,x :Foo<^a,^b0>)= f
let inlinemapB_<^mf,^f,^a,^bwhen(^mf or ^a):(static member mapB :^f*^a->^b)>(f :^f,x :^a):^b =((^mfor^a):(static membermapB:^f*^a->^b)(f, x))typeMultitraverse()=static member inlinetraverseBB(f :^b1 ->^b2,x :^a1 * ^c2 * ^d3)=fun a -> printfn "hi"; a
static member inlinetraverseBB(f :^b ->^b0,(a :^a,b :^b)):^b1when (Multifunctor or ^b0):(static member mapB :_ *^b0->^b1)=
mapB_<Multifunctor,_,_,_>((fun x -> a),(f b))let inlinetraverseBB_<^mf,^a,^b,^c,^dwhen(^mf or ^a):(static member traverseBB :(^a->^b)*^c->^d)>(f :^a ->^b,x :^c):^d =((^mfor^a):(static membertraverseBB:(^a->^b)*^c->^d)(f, x))let inlinetraverseBB(f :^a ->^b)(x :^c):^dwhen (Multitraverse or ^c):(static member traverseBB :(^a->^b)*^c->^d)=// The intent of requiring monad is just trying to be lawful
traverseBB_<Multitraverse,^a,^b,^c,^d>(f, x)letprivatemaybeZero:int ->Foo<int,int>=fun x -> Bar(0,2)// This type clearly does not match the implementation, and yet it type checksletprivatewhatInTheWorld:list<string>*int -> System.Random =
traverseBB maybeZero
Compiler like => "You poke me again?! I'll give you an overload error although actually the return type is super wrong 😜
I changed from (fun x -> x,a) to (fun x -> a) in traverseBB with the SRTP constraint.
Example 3:
typeFoo<'a,'b>= Bar of'a*'btypeMultifunctor()=static member inlinemapB(f :^b ->^b0,(a :^a,b :^b)):^a * ^b0 =(a, f b)static member inlinemapB(f :^b ->^b0,x :Foo<^a,^b0>)= f
let inlinemapB_<^mf,^f,^a,^bwhen(^mf or ^a):(static member mapB :^f*^a->^b)>(f :^f,x :^a):^b =((^mfor^a):(static membermapB:^f*^a->^b)(f, x))typeMultitraverse()=static member inlinetraverseBB(f :^b1 ->^b2,x :^a1 * ^c2 * ^d3)=fun a -> printfn "hi"; a
static member inlinetraverseBB(f :^b ->^b0,(a :^a,b :^b)):^b1when (Multifunctor or ^b0):(static member mapB :_ *^b0->^b1)=
mapB_<Multifunctor,_,_,_>((fun x -> x),(f b))let inlinetraverseBB_<^mf,^a,^b,^c,^dwhen(^mf or ^a):(static member traverseBB :(^a->^b)*^c->^d)>(f :^a ->^b,x :^c):^d =((^mfor^a):(static membertraverseBB:(^a->^b)*^c->^d)(f, x))let inlinetraverseBB(f :^a ->^b)(x :^c):^dwhen (Multitraverse or ^c):(static member traverseBB :(^a->^b)*^c->^d)=// The intent of requiring monad is just trying to be lawful
traverseBB_<Multitraverse,^a,^b,^c,^d>(f, x)letprivatemaybeZero:int ->Foo<int,int>=fun x -> Bar(0,2)// This type clearly does not match the implementation, and yet it type checksletprivatewhatInTheWorld:list<string>*int -> System.Random =
traverseBB maybeZero
Compiler like => "Oh yeah this is fine, just go on. I don't see this insane typing :)"
Original example with (fun x -> x)traverseBB with the SRTP constraint.
typeFoo<'a,'b>= Bar of'a*'btypeMultifunctor()=static member inlinemapB(f :^b ->^b0,(a :^a,b :^b)):^a * ^b0 =(a, f b)static member inlinemapB(f :^b ->^b0,x :Foo<^a,^b0>)= f
let inlinemapB_<^mf,^f,^a,^bwhen(^mf or ^a):(static member mapB :^f*^a->^b)>(f :^f,x :^a):^b =((^mfor^a):(static membermapB:^f*^a->^b)(f, x))typeMultitraverse()=static member inlinetraverseBB(f :^b1 ->^b2,x :^a1 * ^c2 * ^d3)=fun a -> printfn "hi"; a
static member inlinetraverseBB(f :^b ->^b0,(a :^a,b :^b)):^b1when (Multifunctor or ^b0):(static member mapB :_ *^b0->^b1)=
mapB_<Multifunctor,_,_,_>((fun x ->""),(f b))let inlinetraverseBB_<^mf,^a,^b,^c,^dwhen(^mf or ^a):(static member traverseBB :(^a->^b)*^c->^d)>(f :^a ->^b,x :^c):^d =((^mfor^a):(static membertraverseBB:(^a->^b)*^c->^d)(f, x))let inlinetraverseBB(f :^a ->^b)(x :^c):^dwhen (Multitraverse or ^c):(static member traverseBB :(^a->^b)*^c->^d)=// The intent of requiring monad is just trying to be lawful
traverseBB_<Multitraverse,^a,^b,^c,^d>(f, x)letprivatemaybeZero:int ->Foo<int,int>=fun x -> Bar(0,2)// This type clearly does not match the implementation, and yet it type checksletprivatewhatInTheWorld:list<string>*int -> System.Random =
traverseBB maybeZero
replace x with "" and we get this very interesting output (not good...):
error FS0073: internal error: Undefined or unsolved type variable: '_?1254408
error FS0041: No overloads match for method 'mapB'. The available overloads are shown below.
error FS0193: Possible overload: 'static member _.Multifunctor.mapB : f:( ^b -> ^b0) * x:_.Foo< ^a, ^b0> -> ( ^b -> ^b0)'. Type constraint mismatch. The type
'_.Foo<Microsoft.FSharp.Core.int,Microsoft.FSharp.Core.int>'
is not compatible with type
'_.Foo<'a,'b>'
.
error FS0193: Possible overload: 'static member _.Multifunctor.mapB : f:( ^b -> ^b0) * ( ^a * ^b) -> ^a * ^b0'. Type constraint mismatch. The type
'_.Foo<Microsoft.FSharp.Core.int,Microsoft.FSharp.Core.int>'
is not compatible with type
''a * 'b'
.
Interesting code 2:
typeFoo<'a,'b>= Bar of'a*'btypeMultifunctor()=static member inlinemapB(f :^b ->^b0,(a :^a,b :^b)):^a * ^b0 =(a, f b)static member inlinemapB(f :^b ->^b0,x :Foo<^a,^b0>)= f
let inlinemapB_<^mf,^f,^a,^bwhen(^mf or ^a):(static member mapB :^f*^a->^b)>(f :^f,x :^a):^b =((^mfor^a):(static membermapB:^f*^a->^b)(f, x))typeMultitraverse()=static member inlinetraverseBB(f :^b1 ->^b2,x :^a1 * ^c2 * ^d3)=fun a -> printfn "hi"; a
static member inlinetraverseBB(f :^b ->^b0,(a :^a,b :^b)):^b1when (Multifunctor or ^b0):(static member mapB :_ *^b0->^b1)=
mapB_<Multifunctor,_,_,_>((fun x ->1),(f b))let inlinetraverseBB_<^mf,^a,^b,^c,^dwhen(^mf or ^a):(static member traverseBB :(^a->^b)*^c->^d)>(f :^a ->^b,x :^c):^d =((^mfor^a):(static membertraverseBB:(^a->^b)*^c->^d)(f, x))let inlinetraverseBB(f :^a ->^b)(x :^c):^dwhen (Multitraverse or ^c):(static member traverseBB :(^a->^b)*^c->^d)=// The intent of requiring monad is just trying to be lawful
traverseBB_<Multitraverse,^a,^b,^c,^d>(f, x)letprivatemaybeZero:int ->Foo<int,int>=fun x -> Bar(0,2)// This type clearly does not match the implementation, and yet it type checksletprivatewhatInTheWorld:list<string>*int -> System.Random =
traverseBB maybeZero
or replace it with an constant int value and we get a correct compiler error:
error FS0001: This expression was expected to have type
'System.Random'
but here has type
''a -> int'
Activity
Kazark commentedon May 17, 2018
If it is useful, I now have a branch where I got the compile to fail by using more member constraints to fake out type-level guarantees on the conceptual higher-kinded types I was trying to model. This is a verbose workaround but one that bolsters the overall solidity of my code and in my mind the fact that it worked probably does reduce the severity of this bug.
abelbraaksma commentedon May 19, 2018
I'm seriously impressed by this code, I've never used SRTP to such complexity. I'm curious, as I see quite some syntax that doesn't compile with F# 4.0, and I wasn't aware of syntax changes or significant added features in F# 4.1.
Which makes you wonder whether this is caused by the new feature's implementation or that this has always been a bug (provided it is one, I cannot be the judge of that). Not sure if this can be rewritten with F# 4.0 syntax though and if so, whether that would yield the same results.
I've managed before to create compilable code with SRTP, that threw hard-to-diagnose errors. In my case that was caused by the compiler allowing the order to be random on static inline class members, but that in reality they need to be in the correct order, as if they are in a module, otherwise inexplicable compile or runtime errors can occur. Since you use a lot of static members, I would wonder whether this could in any way be related to an issue with the order of declaration.
Just my 2p, perhaps not very useful, but your code intrigued me ;).
PS: it may be useful to add what would be the correct way of calling your code, to see it run without that error. You write "This type clearly does not match the implementation", and I'm sure for some people it is "clear", but it wasn't for me, and I find SRTP quite hard to read and diagnose whenever it gets beyond a few lines.
Kazark commentedon May 19, 2018
@abelbraaksma Thanks for your intriguing comments. I will certainly own up to the fact that this is insanely complex SRTP code. Polite of you to say you are impressed; you should rather say disgusted! But sadly, it's the part of my codebase keeping me sane (there is much much more code written in this style in my actual code base: I'm a bit of a crackpot I suppose). Basically I have found that if you write code with
Either<_,_>(orResult<_,_>),list<_>and_*_much, you quickly need traverse; but in a rather monomorphic language you end up with mxn implementations, which is not fun. Anyway that is my mad scientist explanation for this exotic specimen.Yes, this code is enabled by F# 4.1 and was not possible in F# 4.0 AFAIK. I looked into it in F# 4.0 and couldn't see a way to make it work anyhow. My coworker resurrected the question after they added the new tweaks in F# 4.1 and as odd as the little addition was it allowed us to make it all work (or so we thought, haha!).
Good thought about ordering. I don't pretend to understand what's happening well enough to know whether that is the issue at stake but it's good to know that someone else has hit something like this.
You are right in saying I was a bit hasty about saying it was "clear" that the type didn't match the implementation; should have explained a bit. The intend of
traverseBBis to betraversefrom Haskell, which has the type signature:with the specific conditions that both
tandfare bifunctors, but we are only caring about their second type parameter for the sake of this traverse (i.e. in my actual codebase I havetraverseAB,traverseBA...). So for example, since my first function ismaybeZero :: Int -> Maybe Int(using Haskell syntax for parity with the above, and because the lack of higher-kinded types means it would only be pseudo-syntax if I mimicked F# here),traverseBB maybeZeroshould have the typet Int -> Maybe (t Int). Thet Intin question is(List String, Int). Thus the type should be(List String, Int) -> Maybe (List String, Int), or reverting to F# syntax now that everything is plugged in, the type should belist<string>*int -> Maybe<list<string>*int>, which it should be safe to say at this point clearly does not matchlist<string>*int -> Either<string,list<string>>(Maybe<_>is a type alias forEither<unit,_>).I recognize that in some sense I am abusing F# in all this. Perhaps the users who abuse the program are the ones who tend to find the bugs.
drvink commentedon May 19, 2018
Hi @Kazark,
I'm hesitant to say that your example is written incorrectly, because nobody's ever issued a guide on the "correct" way to write SRTP, but congratulations (?) on your journey along the unmarked path to, uh, SRTP enlightenment, or something. You're correct that some of the constructs you're using weren't possible until F# 4.1; in fact, I'm guilty of having made it easier when I reported one of the things you're now using in #392 (wow, that was a long time ago). The reason you're getting the
InvalidCastExceptionis because the constraint solver likes to try to useobjif it can't infer anything better in certain cases, and you've found one of them: specifically, use of explicit type parameters toinlinefunctions (i.e.f<Multifunctor, _> x y z). This can result in code that typechecks, but fails at runtime in strange ways such as this. On principle it's easy to suggest that any situation where typechecking should intuitively fail but does not constitutes a bug or soundness issue, but SRTP rubs up against a lot of corner cases: consider the following:Yet, this works fine:
...and of course, constraints not involving bowels of .NET tend to be far better behaved already:
But I digress.
Indeed, SRTP "abuse" such as this is a part of the language few users explore, and despite its utility (nobody is arguing that it does not enable a number of things otherwise not possible) there does seem to be some sentiment that its use is a bridge too far. That said, the code you've written goes against one of the (unwritten) teachings of the SRTP Elders: instead of using explicit type parameters in order to add to solver's overload candidate set, you need to instead--unfortunately--pass a dummy value (i.e. term, not type) that will appropriately influence the solver. I'm sure you've probably noticed that most ways of trying to avoid the dummy value result in strange errors from the compiler involving internally-generated type parameters with names like
^?123. Ideally this wouldn't be necessary (and the fact that you have to do this is not, as far as I know, by design), but that's what you have to do today, and while I'm not speaking for anyone but myself here, I imagine these issues won't get looked at any sooner than F# 5.0, which is when other SRTP rollup issues are slated for review.So, let's apply this one weird trick discovered by a local F# wonk (compiler authors hate him!):
Now our program fails to typecheck as hoped:
Hope this helps in the meantime. I've submitted a talk to Open F# this year about this very kind of thing, so if it gets in then it should be right up your alley! Also, I suspect you'll find FSharpPlus to be a very useful library in this same vein.
@cartermp I'll let you know where to send the coffee. ;)
Kazark commentedon May 21, 2018
@drvink Wow, thanks for the detailed response. I may respond more after digesting it.
Kazark commentedon May 21, 2018
@drvink Thanks for your guidance on SRTP. I will update my code accordingly. Indeed, it is not well documented; haha! I actually fixed an error in the docs while trying to learn about it, lol...
drvink commentedon May 21, 2018
@Kazark no problem--and yes, more documentation would be great!
dsyme commentedon May 30, 2018
@Kazark We will need a much smaller minimal repro of the underlying problem - that's a whole lot of SRTP in that repro. It's a good stress test I suppose, but it's not the kind of use of SRTP I'd personally want to encounter often :)
realvictorprm commentedon Nov 22, 2018
OK I need to update my example further. Appears for me only in the example repository.
realvictorprm commentedon Nov 22, 2018
So one thing to note here is that replacing (^mf or ^a) with (^mf or ^b) as support at https://github.com/Kazark/fsharp-srtp-broken/blob/master/SRTP/SRTP.fs#L32 reports back the expected type error.
realvictorprm commentedon Dec 9, 2018
@dsyme #5973 is to my knowledge the same bug (with the small difference that during IL generation the compiler notices that a variable is unresolved).
5 remaining items
abelbraaksma commentedon Dec 14, 2018
Thanks, clearer now (still odd it crashed vs, but perhaps that had an unrelated cause).
realvictorprm commentedon Dec 14, 2018
smaller again.
Using DU's with less than 2 generics will result in typecheck errors again.
now it's very obvious that the return type is super wrong.
realvictorprm commentedon Dec 14, 2018
Right now it's so amusing to see what the compiler outputs if at particular points you introduce a new static typvar vs. using existing typvars from parameters 😂
realvictorprm commentedon Dec 14, 2018
oh please compare the error output between these 3 codes ahahahaha 😂 It's too funny!
Example 1:
Compiler like => "Ah you want to confuse me? ** you, I give up (Internal error etc...)"
I changed from
(fun x -> x)to(fun x -> a,x)intraverseBBwith the SRTP constraint.Example 2:
Compiler like => "You poke me again?! I'll give you an overload error although actually the return type is super wrong 😜
I changed from
(fun x -> x,a)to(fun x -> a)intraverseBBwith the SRTP constraint.Example 3:
Compiler like => "Oh yeah this is fine, just go on. I don't see this insane typing :)"
Original example with
(fun x -> x)traverseBBwith the SRTP constraint.realvictorprm commentedon Jul 12, 2019
Interesting code 1:
replace
xwith""and we get this very interesting output (not good...):Interesting code 2:
or replace it with an constant int value and we get a correct compiler error: