Skip to content

iperf3 3.5 TCP option -n not working #768

Open

Description

@paaguti

Context

  • Version of iperf3:
    iperf3.5

  • Hardware:
    amd64 laptop with Intel i5 CPU,

  • Operating system (and distribution, if any):
    Running on QEMU based VMs
    Host OS: Ubuntu 16.04.4LTS
    Guest OS: Alpine Linux 3.7+testing

  • Other relevant information (for example, non-default compilers,
    libraries, cross-compiling, etc.):

Please fill out one of the "Bug Report" or "Enhancement Request"
sections, as appropriate.

Bug Report

  • Expected Behavior
    iperf3 -c -n 1.8M should produce a data transaction with 1.8M bytes

  • Actual Behavior
    Dumping from the PCAP file and filtering the FIN/ACK packet:

       time       sport      dport          seq              ack      flags
     0.19117      56562       5201               38          2904690  FA

Expecting something around 1887436 and not 2904690 in the ack number

  • Steps to Reproduce
    VM1 as server, iperf3 -s ,capture packets with tshark
    VM2 as client, iperf3 -c <IP_VM1> -n 1.8M
    Analyse TCP SEQ and ACK numbers

  • Possible Solution
    Yet N/A

Activity

  1. paaguti commented on Jul 7, 2018

    @paaguti
    Author

    Confirming for 3.6

    Environment

    QEMU based VMs with Ubuntu 16.04.4 LTS and development tools.
    iperf3.6 compiled from source

    Test procedure

    The server is called with iperf3 -s -1 and the client with iperf3 -c <ip_server> -n <size> --reverse -J -logfile receiver.json
    When examining the capture files, much more traffic is sent than expected (~3.6M). When digging in the resulting statistics file, the receiver stats show it received 100K bytes more than the specified 1.8M.

    This can be critical in some applications, where we use iperf3 to simulate more complex applications...

    receiver.json.gz

  2. thomas-fossati commented on Jul 15, 2018

    @thomas-fossati

    I have observed the same anomalous behaviour as @paaguti running iperf 3.5 (cJSON 1.5.2) on Darwin 17.6.0.

    I have attached a plot that shows the distribution of the observed transfers (in green) around the expected value (in red-ish), in my case 1Mbyte. See in particular the odd spike around 1.25M.

    iperf-bytes-anomaly

  3. bmah888 commented on Jul 27, 2018

    @bmah888
    Contributor

    1.8MB is a pretty small transfer compared to what iperf3 was designed for. At that size you'll run into some effects where the default sending length (-l flag) will be significant, because iperf3 only does writes (at the socket layer) of that particular length...it's 128K by default for TCP transfers.

    Still I see some non-intuitive behavior even on my local macOS laptop sending to itself, and it feels like there's something not quite right here. I'm particularly puzzled by the fact that the number of bytes sent is different on different runs...if we ran for a certain length of time (which is the default mode of operation) I'd expect that, but not with -n specified.

  4. self-assigned this
    on Jul 27, 2018
  5. bmah888 commented on Jul 27, 2018

    @bmah888
    Contributor

    This line (around line 2205 in src/iperf_api.c) appears to be the culprit:

        testp->multisend = 10;	/* arbitrary */
    

    It basically allows the iperf3 sender to blow through the bytes limit by up to 10 writes (defined by whatever the -l parameter is). I am not sure what's the best way to fix it....changing 10 to 1 seems to solve the nondeterminism problem but might have some other effects we don't want.

  6. paaguti commented on Jul 28, 2018

    @paaguti
    Author
  7. paaguti commented on Jul 31, 2018

    @paaguti
    Author

    On a Ubuntu 18.04 system:

    Linux pavilion 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

    I modified the command line argument handling:

    		case 'n':
    			test->settings->bytes = unit_atoi(optarg);
    /*paag: if bytes are set, make it deterministic*/
    			test->multisend = 1;
    			client_flag = 1;
    			break;
    

    And now things seem to improve for an intuitive point of view. Looking at the stats as
    { bytes transfered: times} :

    paag@pavilion:~/Devel/iperf$ ./kk.py client.json
    {1966080: 100}
    paag@pavilion:~/Devel/iperf$ ./kk.py server.json 
    {1966080: 98, 1835008: 1, 1769419: 1}
    

    Gotten from extracting:

            bytes = elem["intervals"][0]["sum"]["bytes"]
    

    From the JSON statictics at the server and client for a run with

    iperf3 --json -c 127.0.0.1 -n 1.8M
    

    Repeated 100 times in two separate runs (one getting server and one getting client stats). So now, the transfer size is roughly 100K over the 1.8M, but at least, this behaviour is consistent.

  8. bmah888 commented on Aug 3, 2018

    @bmah888
    Contributor

    @paaguti: I'm not disputing that there's a bug here, just that I wasn't sure what was the best way to solve it. The patch you provided, or something similar, might be the best way forward. It might also work for -k, which I imagine might have a similar problem.

    You understand right that TCP does not guarantee the size of packets on the wire. It's a bytestream oriented protocol, so it can consolidate small sends into larger packets, up to the MSS. Similarly, it can take huge sends from the socket layer but break them down into MSS-sized segments to go into MTU-sized IP packets.

  9. added
    bug:test-dataBugs related to the amount of data sent during a test
    on Jul 13, 2020
  10. linked a pull request that will close this issueCLI backward compatible fix for issue #768 #797on Sep 17, 2020
  11. bmah888 commented on Jun 4, 2026

    @bmah888
    Contributor

    Is this still a problem? In particular since multithreading was added?

  12. davidBar-On commented on Jun 5, 2026

    @davidBar-On
    Contributor

    Is this still a problem? In particular since multithreading was added?

    From tests I run it seems that the issue was indeed resolved for both -n and -k in the multi-threading version by the following code in iperf_send_mt():

    iperf/src/iperf_api.c

    Lines 2229 to 2232 in 2b9cfa6

    if (multisend > 1 && test->settings->bytes != 0 && test->bytes_sent >= test->settings->bytes)
    break;
    if (multisend > 1 && test->settings->blocks != 0 && test->blocks_sent >= test->settings->blocks)
    break;

    Actually, it seems that the related help messages are now wrong, and:

     -n, --bytes     #[KMG]    transmit until the end of the interval when the client sent or received
                                (per direction) at least this number of bytes (instead of -t or -k)
     -k, --blockcount #[KMG]   transmit until the end of the interval when the client sent or received
                                (per direction) at least this number of blocks (instead of -t or -n)
    

    Should be reverted back to:

     -n, --bytes     #[KMG]    number of bytes to transmit (instead of -t or -k)
     -k, --blockcount #[KMG]   number of blocks (packets) to transmit (instead of -t or -n)
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugbug:test-dataBugs related to the amount of data sent during a test

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions