Backwards compatibility hazard with `Box` + `&mut` behaviour that cannot be replicated by a pure-library type

[huonw]

With Box built deeply into the compiler, one can currently write

let mut y = 1;
let x: Box<&mut int> = box &mut y;
**x = 2;

That is, one can mutate through a &mut stored in the Box, even if the Box itself is not in a mutable slot. This is (presumably) because the compiler currently understands a lot about Boxs behaviour, in particular, that it can act like an immutable unaliased pointer (&uniq).

There's currently no way to get this behaviour for library types, e.g. Vec<&mut int> is theoretically identical to Box in terms of ownership/aliasability of its contents, but, as it is a library type, the following cannot be made to work:

let mut y = 1;
let x: Vec<&mut int> = vec![&mut y];
// error: can't mutate a `& &mut int`
**x.get(0) = 2;
// error: can't borrow x as mutable
**x.get_mut(0) = 2;

We would need an immutable &uniq pointer for this behaviour to be possible for general simply-owned types like an in-library Box, Vec, HashMap etc.

Having this behaviour for Box represents a possible backwards compatibility hazard if we ever decide to move Box more into libraries. Specifically, moving Box into a library and preserving its current behaviour perfectly would require (at least) adding this new &uniq pointer type.

(Thanks to /u/SteveMcQwark on reddit for prompting this.)

(Nominating.)

[alexcrichton]

Learn something new every day!

[pcwalton]

I never considered it a priority to be able to move Box fully into the library for 1.0, and it's clear that more machinery will be necessary. So I'm not sure this is a hazard.

[pnkfelix]

Assigning P-low. And maybe we should actually close this.

cc: @nikomatsakis

[nikomatsakis]

The real question is whether we will ever support other "unique" pointer types in the library. It is not entirely clear to me what the best resolution is here.

[lilyball]

The issue with Vec<&mut int> is one I hadn't considered before, and is rather unfortunate. &uniq could be leveraged in some fashion to try and fix this, but of course we were hoping to remove &uniq with by-value closures.

Theoretically, borrowck could try and track whether a & reference is actually non-aliased, where a non-aliased & reference is one derived from something else known to be non-aliased (such as a non-borrowed local variable), or derived from an existing non-aliased & reference (including being returned from a function that declares it as having the same lifetime as the non-aliased & parameter). It would have to be able to "downgrade" a non-aliased & to a regular & as well, if the local variable or any "parent" reference is accessed while the non-aliased & is still alive. This would allow *v.get(0) = 2 to work without any changes to the libraries or user code, and similarly should allow Box<&mut T> to work, but is rather complex.

A more limited suggestion would be to have borrowck merely consider the results of Deref<T> to be non-aliased if the dereferenced value is known to be non-aliased. deref() returns a &-reference but that reference is hidden from the client code. A separate attribute can be used to declare that a smart pointer is considered unique, and the two combined would allow Box<&mut T> to be implemented fully in the library. But this approach doesn't fix Vec<&mut T>.

[lilyball]

Actually, thinking about it some more, let x: Box<&mut int> = ..; let y = *x; needs to move the &mut int out of the box and into y, and then the Box destructor needs to know that its value is moved (which right now can be done with zeroing, but AIUI we're eventually going to move to precise tracking). I don't know how a library-implemented Box would know about the movement.

[huonw]

Theoretically it can be reborrowed, i.e. it could be equivalent to let y = &mut **x;. (Also, &mut doesn't have a destructor and so doesn't need zeroing: the relationship between moves and zeroing is just to stop spurious/incorrect destruction of invalid values.)

(Frankly I don't think we need to preserve this behaviour if we do decide to make compiler types and library types equal in this respect, it is almost certainly a very rare edge case and can easily be fixed by adding a mut.)

[lilyball]

@huonw mut doesn't replicate current behavior:

fn main() {
    let mut a = 3i;
    let b = box &mut a;
    let c = &b;
    **b = 4i;
    println!("{}", c);
}

Note that I can still assign to **b despite having an outstanding borrow on b.

More importantly, you can't say let m = &mut **b; due to the outstanding immutable borrow of b, which means that taking a &mut reference is not equivalent to the ability to modify the box.

The question is, is the current behavior worth keeping, or can all clients of the current behavior be made to work with mut and &mut just as well?

---

Also I'm mildly concerned about the fact that I can mutate the boxed &mut T despite having an outstanding immutable reference to the box. Is there any aliasing issues here? I'm guessing no, because the reference I have points to the unique pointer, and the unique pointer is what I mutated through, so reading the value through the reference does ultimately read through the same pointer used to mutate, but I don't have a complete understanding of LLVM's data dependency aliasing rules.

[huonw]

That just seems like a bug (filed as #14498 ):

fn main() {
    let mut a = 3i;
    let b = box &mut a;
    let c = &b;
    let d = &***c;
    **b = 4i;
    println!("{} {}", c, d);
}

[lilyball]

Ooh, you can get a &int and still mutate through the box? That's no good. Note that the simpler form is correctly identified by borrowck as an error:

fn main() {
    let mut a = 3i;
    let b = box &mut a;
    let c = &**b;
    **b = 4i;
    println!("{}", c);
}