make dtype-next available via Java API?

[subes]

As discussed here: cnuernber/libjulia-clj#4 (comment)

Though I guess dtype-next is a low level API and higher level functionality is available when using neanderthal, tech.ml.dataset or scicloj on top of or with it.

But having a java api for dtype-next would still allow to create abstractions to transport data from java into scala efficiently to then work with other scala libraries without copying data back and forth. I would integrate that here: https://github.com/invesdwin/invesdwin-context/tree/master/invesdwin-context-parent/invesdwin-context-clojure

The arrow integration of tech.ml.dataset could also be used to transport datasets between the jvm and languages like r/python/julia. Though dunno if for that case tech.ml.dataset should also be made available as a java api later or if arrow integration should be implement on the outside for a java abstraction that uses dtype-next.

Same applies to a possible abstraction to other interfaces like nd4j, tablesaw, smile, ojalgo, ...

[subes]

Regarding libjulia-clj and libpython-clj I guess giving it a dtype-next object will already do an efficient transfer into julia arrays or numpy arrays since that is anyway part of the clojure bindings?

[cnuernber]

Semi efficient meaning it will still allocate the native numpy/julia type but it will only need to do a memcpy as opposed to now where it is constructing a java tensor, then allocating the numpy/julia type, then doing the memcpy - assuming there is no transpose performed on either side. If there is a transpose then it has to go in a parallelized index-by-index transformation.

Dataset makes more sense as the arrow integration does allow very fast transfer - serializing to arrow is quite quick and loading is pretty much instantaneous.

[cnuernber]

But to use dataset you need parts of dtype-next so it would start here. I am currently rebuilding the arrow bindings for dataset to support JDK-17 and will loop back to this soon.

[subes]

I have now made some significant progress with the clojure integration here: https://github.com/invesdwin/invesdwin-context/blob/master/invesdwin-context-parent/invesdwin-context-clojure/src/main/java/de/invesdwin/context/clojure/pool/

It seems single threaded tests run fine now, but when using more than 1 thread the bindings don't seem to work. From what I read clojure should use threadlocals for namespace variables. But maybe I am missing some system property to isolate everything properly? (similar to jruby's org.jruby.embed.localcontext.scope=threadsafe)

Am I doing something wrong or is clojure not capable to be called from multiple threads in parallel without namespace conflicts?
It seems clojure binds itself to the first thread that initialized it and any other thread runs into binding errors:

=> Upstream ticket here: https://ask.clojure.org/index.php/11482/unable-resolve-symbol-after-pushthreadbindings-second-thread

[cnuernber]

I honestly have no idea with the above issue. The Clojure compiler/runtime system itself isn't something I have spent any more time than necessary getting into and especially the interaction between defining vars in namespaces and threads is totally opaque to me.

I think, for example, that ns-unmap is a global function so if a var is unmapped in one thread it is unmapped in all threads.

I would, in a threadsafe way, define the var if undefined and set dynamic like you did. Then when you assign a value that is when you pushThread. I would never undefine it but you can pop the var's thread stack which should reset it's value to null (or nil in clojure land - LISP heritage is nil, not null).

[subes]

Thanks, the hint about ns-unmap working on globals is indeed a problem. I have now worked around it by checking if the variable really is in the threads local binding. The "binding not available" issue was that the bindings leaked to a different thread. I made it purely thread local now so that the bindings instance really is invoking the constructor in a given thread. So the test cases now work in a second thread.

Though there are still issues with global variables in the scripts:
A similar problem to ns-unmap happens when one uses def for global variables in the scripts. I tried using set! (https://clojure.org/reference/vars#set) instead to bind it thread local as well in the script, but it seems the "var" has to be declared in some other way before set! works, because it throws an error. Similar problems will occur when defining functions with the same name in different threads. Since that did not work, I looked for a different solution.

---

The correct solution is to not use one "user" namespace for all threads and (in-ns switches to it) but instead a unique namespace per thread. In that regard the bindings can be applied globally but separated. Both def and ns-unmap work properly that way. This way my testcases are green now.

So thanks a lot for the hint, this made me understand the problem a lot better. Clojure just does not have something like "threadsafe" isolation availabe that other languages provides, but one can build it via namespaces.

[cnuernber]

Got it - yep. I am working on providing a java layer for dtype-next right now.

[subes]

I also got a cache working that reuses clojure compilations between invocations.

Existing scripting integrations for clojure are a bit lacking:

• alternatives are listed here: implement scripting for clojure invesdwin/invesdwin-context#14
• no isolation per thread
• some are slow because they reapply bindings on each eval
• no api to remove/clear bindings
• no reuse of compilation

Here a benchmark for the reuse of compilation (100 iterations of the testcases in 1 thread with disabled logging):

• always parsing: PT13.233.921.654S
• always reusing: PT12.676.846.056S

So it might make a bigger difference for tight loops.

I have also added a JSR223 wrapper for the improved binding: https://github.com/invesdwin/invesdwin-context/tree/master/invesdwin-context-parent/invesdwin-context-clojure/src/main/java/de/invesdwin/context/clojure/jsr223

Docs and a full list of supported JVM languages (each having JSR-223 support in addition to the invesdwin scripting interfaces): https://github.com/invesdwin/invesdwin-context#scripting-modules-for-jvm-languages

[cnuernber]

Java API for dtype-next '8.057' is up - https://cnuernber.github.io/dtype-next/javadoc/tech/v3/package-summary.html.

Note that neither libpthon-clj nor libjulia-clj have been updated so you will need to override this in the dependencies for now.

QuickNDirty test - https://github.com/cnuernber/dtype-next/blob/master/java_test/java/jtest/Main.java

[cnuernber]

And just like that, the test discovered that I hadn't included nio buffer support.
'8.058' includes nio buffer support by default.

[subes]

I looked at the docs a bit, makes a very good impression!

Regarding Readers, maybe we should also support the other less common types?

//DTye.bool => BooleanReader exists with variants
  DType.int8 => ByteReader not available
  DType.uint8 => not needed or ShortReader? or can we treat a ByteReader that way?
  DType.int16 => ShortReader not available
  DType.uint16 => not needed or IntegerReader? or can we treat a ShortReader that way?
  DType.int32 => IntegerReader not available
  DType.uint32 => not needed or LongReader? or can we treat a IntegerReader that way?
//DType.int64 => LongReader exists with variants
  DType.uint64 => not needed or DoubleReader? or can we treat a LongReader that way?
  DType.float32 => FloatReader not available
//DType.float64 => DoubleReader exists with variants

[subes]

I also tried using the new types with libjulia-clj:

for (final int count : new int[] { 10, 100, 250, 500, 1000, 10000 }) {
    System.out.println("\nArray Size: " + count);
    final double[] array = new double[count];
    for (int i = 0; i < count; i++) {
        array[i] = i / 10000D;
    }
    final int iterations = 100;
    final LibjuliacljScriptTaskEngineJulia engine = new LibjuliacljScriptTaskEngineJulia(this);
    for (int t = 0; t < 2; t++) {
        Instant start = new Instant();
        for (int i = 0; i < iterations; i++) {
            putDoubleVector("asdf", array);
            final double[] out = getDoubleVector("asdf");
            Assertions.checkEquals(array, out);
        }
        if (t == 1) {
            System.out.println("createArray: " + start);
        }
        //not convertible
        start = new Instant();
        for (int i = 0; i < iterations; i++) {
            putDoubleVectorBuffer("asdf", array);
            final double[] out = getDoubleVector("asdf");
            Assertions.checkEquals(array, out);
        }
        if (t == 1) {
            System.out.println("buffer: " + start);
        }
        //Cannot serialize type Ptr{Nothing}
        start = new Instant();
        for (int i = 0; i < iterations; i++) {
            putDoubleVectorBufferReader("asdf", array);
            final double[] out = getDoubleVector("asdf");
            Assertions.checkEquals(array, out);
        }
        if (t == 1) {
            System.out.println("bufferReader: " + start);
        }
        //put works, get returns "nothing"
        start = new Instant();
        for (int i = 0; i < iterations; i++) {
            putDoubleVectorTensor("asdf", array);
            final double[] out = getDoubleVector("asdf");
            Assertions.checkEquals(array, out);
        }
        if (t == 1) {
            System.out.println("tensor: " + start);
        }
    }
}

@Override
public void putDoubleVector(final String variable, final double[] vector) {
    final Object array = libjulia_clj.java_api.createArray("float64", new int[] { 1, vector.length }, vector);
    putGlobalFunction.invoke(variable, array);
}

public void putDoubleVectorTensor(final String variable, final double[] vector) {
    final NDBuffer tensor = Tensor.asTensor(vector);
    putGlobalFunction.invoke(variable, tensor);
}

public void putDoubleVectorBuffer(final String variable, final double[] vector) {
    final DoubleBuffer tensor = DoubleBuffer.wrap(vector);
    putGlobalFunction.invoke(variable, tensor);
}

public void putDoubleVectorBufferReader(final String variable, final double[] vector) {
    final DoubleReader tensor = new DoubleReader() {
        @Override
        public long lsize() {
            return vector.length;
        }

        @Override
        public double readDouble(final long idx) {
            return vector[(int) idx];
        }
    };
    putGlobalFunction.invoke(variable, tensor);
}

I guess we don't yet have the types considered in the put pathway for buffers and buffer readers (though dunno if it makes sense to support these types). Though I guess only the tensor variant will be supported (not yet complete). Or will we need the abstractions of tech.ml.dataset to support this, as you suggested previously?

[cnuernber]

Glad you got into it so quickly! That is encouraging. A few things I see -

Wrapping a double array in a reader like above means we disable the high performance copy pathway. When dtype wraps double arrays it does so in a way that allows the system to get back to the original buffer for high perf loops and such:

    double[] testData = toDoubleArray(range(100));
    Buffer wrappedData = toBuffer(testData);
    ArrayBufferData origData = asArrayBuffer(wrappedData);

    System.out.println(String.valueOf(System.identityHashCode(testData))
		       + " " + String.valueOf(System.identityHashCode(origData.arrayData)));

// Outputs:
// 748921347 748921347

There are DoubleWriters and such but one thing to note is that 'toBuffer' creates a buffer implementation that will allow writing.

For efficiently putting data into existing containers, there is DType.copy which falls back to the lower level arrayCopy or copyMemory fastpaths.

One thing you can do now is after creating the Julia array you can get a tensor from it with Tensor.asTensor. So you can can manipulate the data in the tensor from the Java side for the case where you are doing something other than just bulk uploads. Or you can copy new data in with DType.copy. You can also transpose it so you get the same representation on the Java side as Julia is seeing.

I didn't create the other integer types of reader base interfaces or float interfaces as if, for instance, you create a LongReader and override elemwiseDatatype to return int16 the system will interpret that as a short buffer. You need to make sure you return longs in the range of shorts but casting from a long to a short and back I think is nearly instantaneous and that is true for all integer types. I am not sure about casting from a double to float in terms of time but because the virtual call times are much longer than the cast times I didn't create the various subtype overloads. I think they do help in code clarity if you are dealing with signed integer types but as I said I don't believe there is an efficiency argument there.

The one perf test I think would be the time difference of Julia createArray between a double[][] and a single double[] reshaped to be a tensor of shape [1000 10].

[subes]

The reader was an idea to maybe transparently transpose during the copy into julia. But yes, it could be more efficient to just transpose on the java side, then copy to julia. Or copy to julia, then transpose there.

To test the double[][] I guess first this pathway needs to work:

 //put works, get returns "nothing"
start = new Instant();
for (int i = 0; i < iterations; i++) {
    putDoubleVectorTensor("asdf", array);
    final double[] out = getDoubleVector("asdf");
    Assertions.checkEquals(array, out);
}
if (t == 1) {
    System.out.println("tensor: " + start);
}

I checked it in more detail:

final double[] array = new double[10];
final NDBuffer tensor = Tensor.asTensor(array);
putGlobalFunction.invoke("asdf", tensor);
libjulia_clj.java_api.runString("println(asdf)");
final Object arrayOut = libjulia_clj.java_api.runString("asdf");
System.out.println(arrayOut);
final Map<?, ?> map = libjulia_clj.java_api.arrayToJVM(array);

The output is:

nothing
nothing
2022-01-18 16:21:34.821 [ |7-1:InitializingJul] ERROR de.invesdwin.ERROR.process                                   - processing #00000002
de.invesdwin.context.log.error.LoggedRuntimeException: #00000002 java.lang.Exception: nil value passed into ->array
        ... 13 omitted, see following cause or error.log
Caused by - java.lang.Exception: nil value passed into ->array
        at tech.v3.datatype.copy_make_container$__GT_array.invokeStatic(copy_make_container.clj:160)
        at tech.v3.datatype.copy_make_container$__GT_array.invoke(copy_make_container.clj:153)
        at tech.v3.datatype.copy_make_container$__GT_int_array.invokeStatic(copy_make_container.clj:195)
        at tech.v3.datatype.copy_make_container$__GT_int_array.invoke(copy_make_container.clj:192)
        at tech.v3.datatype$__GT_int_array.invokeStatic(datatype.clj:89)
        at tech.v3.datatype$__GT_int_array.invoke(datatype.clj:86)
        at clojure.lang.Var.invoke(Var.java:384)
        at libjulia_clj.java_api$_arrayToJVM.invokeStatic(java_api.clj:126)
        at libjulia_clj.java_api$_arrayToJVM.invoke(java_api.clj:120)
        at libjulia_clj.java_api.arrayToJVM(Unknown Source)
      * at de.invesdwin.context.julia.runtime.libjuliaclj.internal.UncheckedJuliaEngineWrapper.init(UncheckedJuliaEngineWrapper.java:82) *

Thus the putGlobalFunction.invoke(...) fails to put the value. Instead "nothing" arrives in julia.

[cnuernber]

The most efficient transpose is definitely going to be on the Julia side.

What I meant was something more like:

Object juliaAry = createArray(data, "float64", [10]);
NDBuffer tens = asTensor(juliaAry);

Then you can do whatever you like to it in Java land as well as in Julia land. Reshape is the best way to take a flat buffer of data and end up with a tensor.