name collision for "take"

[StefanKarpinski]

This is a rather unfortunate name collision:

• julia/base/multi.jl

  Lines 726 to 736 in 493f9c0

  	function take(rv::RemoteValue)
  	wait_full(rv)
  	val = rv.result
  	rv.done = false
  	rv.result = nothing
  	notify_empty(rv)
  	val
  	end
  	take_ref(rid) = take(lookup_ref(rid))
  	take(rr::RemoteRef) = call_on_owner(take_ref, rr)

• https://github.com/JuliaLang/Iterators.jl/blob/master/src/Iterators.jl#L42-L50

[JeffBezanson]

We should probably rename the RemoteRef take to take! and put to put!. That's probably enough to fix this too since the take in Iterators is inherently a pure function.

[StefanKarpinski]

That seems fine to me, although thus far we've only used the ! convention for mutation of arrays, not other things with state (e.g. I/O objects).

[JeffBezanson]

A RemoteRef acts like a synchronized queue of length 1, so it is just as much a data structure as an I/O thing.

[StefanKarpinski]

Fair enough. Just thought I'd point out that it's stretching the convention further than we have so far.

[JeffBezanson]

Or, on the theory that this thing is a queue and might someday allow more than one element, we should use some combination of pop!, push!, shift!, unshift!, enqueue!, and dequeue!, though I'm not really sure which.

[StefanKarpinski]

Yikes. The only real reason to have a collection-like API for such a thing, even if it is actually a queue, is if it might make sense to pass RemoteRefQueue into some generic code that works on a Queue or plain old Array. Does that actually make any sense? I'm not sure, but I feel like that's the fundamental question that needs to be asked.

[JeffBezanson]

As soon as the put and take functions were added, RemoteRef became genuinely equivalent to a "channel" or "synchronized queue". As such its name is no longer very accurate, except in that it can still hold only one value. I think at this point I'd be fine with renaming it to Channel or something else. If you have a function that consumes things from a queue, passing it one of these would be totally legitimate as far as I can tell.

[StefanKarpinski]

Ok, in that case, maybe we should go ahead with the rename and make sure that you can exchange a normal Queue object with a Channel object. Using push! and shift! for channels does seem a bit odd, but sure.

[JeffBezanson]

I don't insist on rocking the boat so much here; I was just really excited to remove two exports. I see the point that channels don't quite fit either queues or I/O devices. Do you have any ideas for names?

[StefanKarpinski]

I think put! and take! are ok. Go calls these operations "put" and "take" [1] even though the they use special <- syntax for them. Rich Hickey also uses the terms "put" and "take" [2], although core.async calls these operations >! and <!. So I think calling them put! and take! is pretty reasonable. One option is to call the operations that and then make normal collections support the put! and take! operations where appropriate. There are two directions of interchangeability to consider:

1. Using a Channel in generic code for a normal Queue object.
2. Using a normal Queue object in generic code for a Channel.

For the former, you want Channel to support Queue operations, whereas for the latter, you want Queue to support Channel operations. It might also be reasonable to use the put! and take! operations for queues and stacks one can imagine swapping one for the other to change behavior from LIFO to FIFO or vice versa – or using some kind of priority queue for even different behavior.

[1] http://golangtutorials.blogspot.com/2011/06/channels-in-go.html
[2] http://clojure.com/blog/2013/06/28/clojure-core-async-channels.html

[JeffBezanson]

I'm fine with put! and take!; those are some pretty solid precedents. But if we are actually going to add put! and take! for other data structures, and/or queue operations for channels, then I think we should just consider channels a special kind of queue and use the queue names everywhere. They are just queues that happen to block when empty or full.

[jakebolewski]

This will mesh really nicely with using accelerators such as GPU's. I like the fact that at a high level you will be able to put! or take! an Array to/from a GPU as easily as a remote processor. One thing to think about from an API perspective is distinguishing between blocking/nonblocking calls. It will be nice to have a consistent API for this.

[amitmurthy]

Sort of related to the discussion above.

If RemoteRefis going to replaced by something like a Channel, which is basically a Q used for communication between processes, then I feel that the implementation/behavior of produce / consume should also be changed to use Channels too. We can thus have the same pattern for inter-task messaging - whether between tasks on the same process or between tasks on two different processes.

I suggest,

• Every Task has two implicit Channels. An inbox and and outbox. These should actually be instantiated only when they are accessed for the first time.
• recv() waits on the self input channel for a Task
• recv(t) waits on the output channel for a Task t
• consume(t, v) will be deprecated. It becomes equivalent to m=recv(t); send(t, v); m
• send(v) appends to the output channel for self. Returns immediately - does not wait for it
  to be consumed. This helps the producer and consumer run on separate processes without any synchronization.
• send(t, v) appends to the output channel for t. Returns immediately.

Independent Channels and their corresponding take! and put! will not be required in the normal course of development - they are low level APIs. To be used only when you want more complex messaging patterns like a single task waiting on multiple channels or similar.

remotecall* family of functions will be used for executing code on different processes, while messaging between tasks is via send/recv calls, with Channel, put! and take! being lower level functions.