You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
QEMU based VMs with Ubuntu 16.04.4 LTS and development tools.
iperf3.6 compiled from source
Test procedure
The server is called with iperf3 -s -1 and the client with iperf3 -c <ip_server> -n <size> --reverse -J -logfile receiver.json
When examining the capture files, much more traffic is sent than expected (~3.6M). When digging in the resulting statistics file, the receiver stats show it received 100K bytes more than the specified 1.8M.
This can be critical in some applications, where we use iperf3 to simulate more complex applications...
I have observed the same anomalous behaviour as @paaguti running iperf 3.5 (cJSON 1.5.2) on Darwin 17.6.0.
I have attached a plot that shows the distribution of the observed transfers (in green) around the expected value (in red-ish), in my case 1Mbyte. See in particular the odd spike around 1.25M.
1.8MB is a pretty small transfer compared to what iperf3 was designed for. At that size you'll run into some effects where the default sending length (-l flag) will be significant, because iperf3 only does writes (at the socket layer) of that particular length...it's 128K by default for TCP transfers.
Still I see some non-intuitive behavior even on my local macOS laptop sending to itself, and it feels like there's something not quite right here. I'm particularly puzzled by the fact that the number of bytes sent is different on different runs...if we ran for a certain length of time (which is the default mode of operation) I'd expect that, but not with -n specified.
This line (around line 2205 in src/iperf_api.c) appears to be the culprit:
testp->multisend = 10; /* arbitrary */
It basically allows the iperf3 sender to blow through the bytes limit by up to 10 writes (defined by whatever the -l parameter is). I am not sure what's the best way to fix it....changing 10 to 1 seems to solve the nondeterminism problem but might have some other effects we don't want.
From a naive user’s point of view:
If I use TCP and I set my transfer size, I’m expecting transactions of that size to occur. If I wouldn’t care about size, I wouldn’t use the -n parameter or use UDP.
Best, /PA
Enviado desde mi iPad
El 27 jul 2018, a las 23:18, Bruce A. Mah ***@***.***> escribió:
This line (around line 2205 in src/iperf_api.c) appears to be the culprit:
testp->multisend = 10; /* arbitrary */
It basically allows the iperf3 sender to blow through the bytes limit by up to 10 writes (defined by whatever the -l parameter is). I am not sure what's the best way to fix it....changing 10 to 1 seems to solve the nondeterminism problem but might have some other effects we don't want.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
From the JSON statictics at the server and client for a run with
iperf3 --json -c 127.0.0.1 -n 1.8M
Repeated 100 times in two separate runs (one getting server and one getting client stats). So now, the transfer size is roughly 100K over the 1.8M, but at least, this behaviour is consistent.
@paaguti: I'm not disputing that there's a bug here, just that I wasn't sure what was the best way to solve it. The patch you provided, or something similar, might be the best way forward. It might also work for -k, which I imagine might have a similar problem.
You understand right that TCP does not guarantee the size of packets on the wire. It's a bytestream oriented protocol, so it can consolidate small sends into larger packets, up to the MSS. Similarly, it can take huge sends from the socket layer but break them down into MSS-sized segments to go into MTU-sized IP packets.
Is this still a problem? In particular since multithreading was added?
From tests I run it seems that the issue was indeed resolved for both -n and -k in the multi-threading version by the following code in iperf_send_mt():
if (multisend>1&&test->settings->bytes!=0&&test->bytes_sent >= test->settings->bytes)
break;
if (multisend>1&&test->settings->blocks!=0&&test->blocks_sent >= test->settings->blocks)
break;
Actually, it seems that the related help messages are now wrong, and:
-n, --bytes #[KMG] transmit until the end of the interval when the client sent or received
(per direction) at least this number of bytes (instead of -t or -k)
-k, --blockcount #[KMG] transmit until the end of the interval when the client sent or received
(per direction) at least this number of blocks (instead of -t or -n)
Should be reverted back to:
-n, --bytes #[KMG] number of bytes to transmit (instead of -t or -k)
-k, --blockcount #[KMG] number of blocks (packets) to transmit (instead of -t or -n)
Activity
paaguti commentedon Jul 7, 2018
Confirming for 3.6
Environment
QEMU based VMs with Ubuntu 16.04.4 LTS and development tools.
iperf3.6 compiled from source
Test procedure
The server is called with
iperf3 -s -1and the client withiperf3 -c <ip_server> -n <size> --reverse -J -logfile receiver.jsonWhen examining the capture files, much more traffic is sent than expected (~3.6M). When digging in the resulting statistics file, the receiver stats show it received 100K bytes more than the specified 1.8M.
This can be critical in some applications, where we use iperf3 to simulate more complex applications...
receiver.json.gz
thomas-fossati commentedon Jul 15, 2018
I have observed the same anomalous behaviour as @paaguti running
iperf 3.5 (cJSON 1.5.2)onDarwin 17.6.0.I have attached a plot that shows the distribution of the observed transfers (in green) around the expected value (in red-ish), in my case 1Mbyte. See in particular the odd spike around 1.25M.
bmah888 commentedon Jul 27, 2018
1.8MB is a pretty small transfer compared to what iperf3 was designed for. At that size you'll run into some effects where the default sending length (
-lflag) will be significant, because iperf3 only does writes (at the socket layer) of that particular length...it's 128K by default for TCP transfers.Still I see some non-intuitive behavior even on my local macOS laptop sending to itself, and it feels like there's something not quite right here. I'm particularly puzzled by the fact that the number of bytes sent is different on different runs...if we ran for a certain length of time (which is the default mode of operation) I'd expect that, but not with
-nspecified.bmah888 commentedon Jul 27, 2018
This line (around line 2205 in
src/iperf_api.c) appears to be the culprit:It basically allows the iperf3 sender to blow through the bytes limit by up to 10 writes (defined by whatever the
-lparameter is). I am not sure what's the best way to fix it....changing10to1seems to solve the nondeterminism problem but might have some other effects we don't want.paaguti commentedon Jul 28, 2018
paaguti commentedon Jul 31, 2018
On a Ubuntu 18.04 system:
Linux pavilion 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/LinuxI modified the command line argument handling:
And now things seem to improve for an intuitive point of view. Looking at the stats as
{ bytes transfered: times}:Gotten from extracting:
From the JSON statictics at the server and client for a run with
Repeated 100 times in two separate runs (one getting server and one getting client stats). So now, the transfer size is roughly 100K over the 1.8M, but at least, this behaviour is consistent.
bmah888 commentedon Aug 3, 2018
@paaguti: I'm not disputing that there's a bug here, just that I wasn't sure what was the best way to solve it. The patch you provided, or something similar, might be the best way forward. It might also work for
-k, which I imagine might have a similar problem.You understand right that TCP does not guarantee the size of packets on the wire. It's a bytestream oriented protocol, so it can consolidate small sends into larger packets, up to the MSS. Similarly, it can take huge sends from the socket layer but break them down into MSS-sized segments to go into MTU-sized IP packets.
CLI backward compatible fix for issue esnet#768
bmah888 commentedon Jun 4, 2026
Is this still a problem? In particular since multithreading was added?
davidBar-On commentedon Jun 5, 2026
From tests I run it seems that the issue was indeed resolved for both
-nand-kin the multi-threading version by the following code iniperf_send_mt():iperf/src/iperf_api.c
Lines 2229 to 2232 in 2b9cfa6
Actually, it seems that the related help messages are now wrong, and:
Should be reverted back to: