redefining struct

[tpapp]

Since structs cannot be redefined, attempting to do so currently gives a failure to evaluate changes warning.

I wonder if this can be worked around by triggering a reload of the entire module in this case.

[timholy]

The problem is that then you have two notions of MyType, MyModuleNew.MyType and MyModuleOld.MyType. You don't see the Old/New when displayed (they both print as MyModule) but in fact they both exist. Any objects::MyModuleOld.MyType that you've created are still around, and you can't use them in methods that dispatch on MyModuleNew.MyType. But since they don't print differently the error messages are extremely confusing, e.g.,

ERROR: MethodError: no method matching foo(::MyModule.MyType)
Closest candidates are:
  foo(::MyModule.MyType) at REPL[2]:1

(The first is really MyModuleOld.MyType and the second is MyModuleNew.MyType.) I don't relish trying to explain that to anyone.

[timholy]

Here's an actual example:

julia> module MyModule
       struct MyType end
       foo(x::MyType) = 1
       end
MyModule

julia> x = MyModule.MyType()
MyModule.MyType()

julia> MyModule.foo(x)
1

julia> module MyModule
       struct MyType end
       foo(x::MyType) = 1
       end
WARNING: replacing module MyModule
MyModule

julia> MyModule.foo(x)
ERROR: MethodError: no method matching foo(::MyModule.MyType)
Closest candidates are:
  foo(::MyModule.MyType) at REPL[4]:3

[tpapp]

I am aware of the issue, but still consider reloading the whole module the best solution. It is recommended in the manual, and is still preferable to restarting Julia, which is the only other workaround I am aware of, and is usually much more time-consuming (especially if one uses Plots.jl etc) than regenerating some values.

I understand your reluctance to deal with this in Revise.jl, as it would be confusing. Perhaps a warning issued in the REPL, or an option that tells Revise.jl to attempt this, would work. But I agree that these are kludges.

[timholy]

I think I wrote that documentation, but turnabout is fair play 😄.

As you probably know, it gets complicated really fast. If you reload A, but have module B that depends on A, you also need to reload B. And (as shown in that manpage) all your variables need to be redefined. Perhaps we could store the entire REPL history for this session and re-execute it? What about statements that, e.g., chop the last 10 bytes off a file, did you really mean to do that twice?

The problems seem quite daunting to me. The "reload all dependent modules" problem may be fairly easily solvable in cases of precompilation because the cache file holds a reliable record of dependencies. (For non-precompiled modules we'd have to rely on a parsing strategy, which might work most of the time but would be fundamentally less reliable.)

The really scary problem is what to do with old variables. I'm not against the idea of trying to do something, but I don't have any good ideas for how to solve it.

[timholy]

OK, I confess you got me thinking: JuliaLang/julia#22721

[vtjnash]

I was wondering: instead of renaming the old type, would it be possible to rename the new one instead? It won't get every case, but perhaps just manually calling eval(rename!(expand())) on everything Revise.jl-aware would cover enough usages to be useful?

[timholy]

That might be work. We'd still have to use methodswith and ask for the callers of the old-type constructor. If the backedges are going away, is that something we can do?

[vtjnash]

It could, but you might also just be able to just look through your parse tree for that symbol. The parse tree will perhaps also be more accurate at finding the actual reference to the type, which doesn't necessarily need to be the same function as the inference tree leading up to an allocation of that type.

[timholy]

Yuck.

julia> struct Foo end

julia> function fakefoo()
           Foo = sqrt
           return Foo(7)
       end
fakefoo (generic function with 1 method)

julia> fakefoo()
2.6457513110645907

Another thing I haven't mentioned yet is that I was planning on trying to scan the eval_user_input args for old types by searching for #RV#, and then warn("I see you have some old types there. It'd be a shame to not use the shiny new ones."). But I could keep a list of specific types to watch out for.

[c42f]

@cstjean just came up with a surprisingly simple but clever idea which could be an effective workaround: https://discourse.julialang.org/t/advice-for-dealing-with-struct-during-development/21732/6

The only obvious downsides being that you'd have to annotate the structs which you anticipate changing before loading the module, and that using such structs would be slow. That seems acceptable to me though.

@timholy, would you consider this (or something similar) for inclusion in Revise? It seems like a developer tool which would be somewhat specific to a Revise-based workflow.

[mauro3]

One thing which that trick doesn't allow is type parameters, or at least no changing type parameters.

[timholy]

I'm interested in exploring this. No bandwidth now, though, not until #243 merges and the associated debugger stack is out there. (And also not a great time to make lots of changes in the Revise codebase, I recommend waiting a week or so.)

[cstjean]

It seems like a developer tool which would be somewhat specific to a Revise-based workflow.

It could be useful for those who include files, too. My inclination would be to make it a separate package, unless there was a specific reason to hook it into Revise.jl. That said, I don't have time to work on it either, so that decision is not up to me.