[TOPI] FIFO buffer op, to accelerate sequence modeling with dilated convolutions

[hcho3]

Motivation. Dilated convolutions have appeared as an effective alternative to recurrent units in modeling sequences. For example, WaveNet [1] uses a stack of dilated convolutional layers to generate raw audio waveforms from text. Snips [2] modifies the WaveNet architecture to detect a keyword in an audio stream.

In order to capture temporal context, the WaveNet architecture feeds a sliding window over the input sequence into the first convolutional layer. As noted in [2] and [3], computing convolution over the sliding window results in redundant computation:

This pull request implements a FIFO buffer operator where intermediate outputs are cached from each convolutional layer, so as to eliminate redundant computation. This is like [4], except that here the re-use is explicit and inherent in the model. Note that caching is only applicable in inference time (so not applicable to training).

Semantics. The FIFO buffer op should behave like

concat(buffer, data, axis=axis)
.slice_axis(axis=axis, begin=data.shape[axis], end=data.shape[axis]+buffer.shape[axis])

Usage. See topi/tests/python/test_fifo_buffer.py

Limitation. Currently, the buffer op exists only in TOPI. To make it useful, we want to merge it into MXNet and other frameworks. Alternatively, we could conceivably implement a custom pass in Relay so that the user can annotate a stack of convolutional layers.

References
[1] "WaveNet: A Generative Model for Raw Audio." https://arxiv.org/abs/1609.03499
[2] "Efficient keyword spotting using dilated convolutions and gating" https://arxiv.org/abs/1811.07684
[3] "Fast Wavenet Generation Algorithm" https://arxiv.org/abs/1611.09482
[4] "Deep reuse: streamline CNN inference on the fly via coarse-grained computation reuse" https://dl.acm.org/citation.cfm?id=3330384

Special thanks to Thibaud Senechal (Amazon) for initially suggesting the concept of FIFO buffer.

cc @yongwww @wweic @zhiics @kevinthesun @anijain2305

[hcho3]

TODO.

• Create an end to end example.
• Send a pull request to MXNet.

[tqchen]

cc @vinx13 @merrymercy would be great if you can help comment and review

[anijain2305]

Thanks for the contribution. I will have to look into the details to understand the compute, but overall looks good to me. Will do one more round by tomorrow.

[zhiics]

Thanks for the contribution. I left some minor reviews. Otherwise, looks good to me.

[yongwww]

LGTM

[zhiics]

LGTM. @vinx13 Can you take another look?