arm64: extremely low performance in `reverse` mode

[iyesin]

Hello,

I wanted to use gocryptfs in reverse mode to sync my existing local backup folder with cloud storage.
Before doing this, I performed some tests and, it seems, my tiny "NAS" would not be able to perform this task.

The device itself is basically a Radxa Zero 3E (Rockchip RK3566 cpu).
The self-identification of this CPU in Linux looks like:

# grep Features /proc/cpuinfo | head -1
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp

As you can see aes extension is present.

Nevertheless, even gocryptfs -speed shows miserable values:

# gocryptfs -speed
gocryptfs v2.6.1 without_openssl; go-fuse v2.8.0; 2025-08-10 go1.24.6 linux/arm64
cpu: unknown; with AES-GCM acceleration
AES-GCM-256-OpenSSL                 N/A
AES-GCM-256-Go                   155.97 MB/s    (selected in auto mode)
AES-SIV-512-Go                    22.33 MB/s
XChaCha20-Poly1305-OpenSSL          N/A
XChaCha20-Poly1305-Go             35.55 MB/s    (selected in auto mode)

These numbers looks ok to me. So I decided to make and experiment with reverse mode.

### Note, /run is big enough tmpfs
# mkdir -p /run/test/{plain,cipher}
# cd /run/test/
# gocryptfs -reverse -init plain
# cd plain
# dd if=/dev/urandom bs=100k count=1 status=progress | split --suffix-length=10 -b 1 - tmp. # pollute plain directory with 100k one-byte files
# cd ..
# gocryptfs -reverse plain cipher
# time tar -cf /dev/null plain

real    0m4.418s
user    0m1.648s
sys     0m2.772s
# time tar -cf /dev/null cipher

real    2m16.581s
user    0m4.740s
sys     0m21.396s

That's insanely slow. Inappropriately slow.

Here are results from my laptop:

$ gocryptfs -speed
gocryptfs v2.6.1 without_openssl; go-fuse v2.8.0; 2025-08-10 go1.24.6 linux/amd64
cpu: Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz; with AES-GCM acceleration
AES-GCM-256-OpenSSL                 N/A
AES-GCM-256-Go                  3553.45 MB/s    (selected in auto mode)
AES-SIV-512-Go                   538.82 MB/s
XChaCha20-Poly1305-OpenSSL          N/A
XChaCha20-Poly1305-Go           2155.61 MB/s    (selected in auto mode)
user@raspberrypi:~/Photos/2025$ 

and the same benchmark on it:

$ cd /run/user/1000
$ mkdir -p test/{plain,cipher}
$ cd test/
$ gocryptfs -reverse -init plain
$ cd plain/
$ dd if=/dev/urandom bs=100k count=1 status=progress | split --suffix-length=10 -b 1 - tmp.
$ cd ..
$ gocryptfs -reverse plain cipher/
$ time tar -cf /dev/null plain

real    0m0,176s
user    0m0,060s
sys     0m0,105s
$ time tar -cf /dev/null cipher

real    0m1,132s
user    0m0,069s
sys     0m0,207s

Okay. Okay. I agree, it is 10x difference. Yet, for Arm SBC, it is 34x slower.

For both tests github released binaries were used. v2.6.1
My question is: is there a way to speed-up this path somehow?

[rfjakob]

The AES-SIV speed is similar to the Raspberry Pi 4 acc. to https://github.com/rfjakob/gocryptfs/wiki/CPU-Benchmarks#raspberry-pi-4-model-b-bcm2835-4-x-cortex-a72--15ghz .

But for tiny 1-byte files, it likely that encryption is not the bottleneck. I will see if I find time to profile this.

[rfjakob]

I wrote a script to reproduce this, https://github.com/rfjakob/gocryptfs/blob/master/profiling/tinyfiles.bash

Seems fast here:

jakob@pi4:~/code/gocryptfs/profiling $ ./tinyfiles.bash 
Creating 100k 1-byte files
done, 4 seconds
Running tar under profiler...
tar: Removing leading `/' from member names
done, 25 seconds

Can you check with latest master? It's possible that 700432e fixed it.

[iyesin]

Thank you, @rfjakob . I will try master build this week. Unfortunately, I don't have enough free time this month.