MutableTangent

[oxinabox]

Mutable primal types need mutable differentials.

This is one of the things needed to support mutability in forwards mode.
(For reverse mode this is not enough).

We should have a MutableComposite type for them.

Whether it is backed by a Dict; or backed by a NamedTuple that gets swapped out everytime setproperty is called, idk.

[nickrobinson251]

What's the difference to Composite exactly?
How's this different from elementwise_add and _update! (which needs to be changed to work with Composite)

[oxinabox]

What's the difference to Composite exactly?

Primary difference is setproperty! will work.

(also accumulate! will work (both directly, and with InplaceThunks))

[oxinabox]

Might also be used for Dict
though slightly different concern to structs since might have hundreds of elements.

[oxinabox]

This would now be called MutableTangent{T} so I have updated the title

[ToucheSir]

With JuliaDiff/Diffractor.jl#189 in mind, how do you think type stability should be addressed? Currently field values could be any of AbstractZero, structural tangents or natural tangents (including natural zeros).