Skip to content

CVPN-2515/CVPN-2512: Send/receive multiple packets to TUN at once#406

Open
kp-thomas-yau wants to merge 12 commits intomainfrom
send-multiple-tun
Open

CVPN-2515/CVPN-2512: Send/receive multiple packets to TUN at once#406
kp-thomas-yau wants to merge 12 commits intomainfrom
send-multiple-tun

Conversation

@kp-thomas-yau
Copy link
Copy Markdown
Contributor

@kp-thomas-yau kp-thomas-yau commented Apr 28, 2026

Description

If offload config is on, enable TUN offload so that we can send/receive multiple packets at the same time. Both receive and sending side must use its send/receive_multiple function variants, as regular send/receive function from tun-rs is now sending raw packets.

Motivation and Context

Now that there's an option to batch receive packets, we can receive/send packets into TUN all at once.

How Has This Been Tested?

Passed e2e CI tests

Test reference, waiting for iPerf3 test
When tested with our regular testing server, speed mostly are quite similar, but upload speed got a nice speed bump of 1898 -> 2262 Mbps (with speedtest CLI)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • The correct base branch is being used, if not main

@kp-thomas-yau kp-thomas-yau force-pushed the send-multiple-tun branch 2 times, most recently from 424e1ea to dfa9243 Compare April 28, 2026 10:07
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 28, 2026

Code coverage summary for a3aa60a:

Filename                                                          Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
lightway-app-utils/src/args/cipher.rs                                   5                 5     0.00%           1                 1     0.00%           5                 5     0.00%           0                 0         -
lightway-app-utils/src/args/connection_type.rs                          8                 8     0.00%           2                 2     0.00%           8                 8     0.00%           0                 0         -
lightway-app-utils/src/args/duration.rs                               121                22    81.82%          11                 5    54.55%          64                15    76.56%           0                 0         -
lightway-app-utils/src/args/ip_map.rs                                  27                27     0.00%           5                 5     0.00%          19                19     0.00%           0                 0         -
lightway-app-utils/src/args/keyshare.rs                                 5                 5     0.00%           1                 1     0.00%           5                 5     0.00%           0                 0         -
lightway-app-utils/src/args/logging.rs                                 37                37     0.00%           3                 3     0.00%          31                31     0.00%           0                 0         -
lightway-app-utils/src/args/nonzero_duration.rs                       128                24    81.25%          10                 4    60.00%          68                16    76.47%           0                 0         -
lightway-app-utils/src/connection_ticker.rs                           229                17    92.58%          28                 4    85.71%         126                15    88.10%           0                 0         -
lightway-app-utils/src/dplpmtud_timer.rs                              209                13    93.78%          22                 4    81.82%         117                11    90.60%           0                 0         -
lightway-app-utils/src/event_stream.rs                                 19                 0   100.00%           3                 0   100.00%          11                 0   100.00%           0                 0         -
lightway-app-utils/src/packet_codec.rs                                  2                 2     0.00%           1                 1     0.00%           1                 1     0.00%           0                 0         -
lightway-app-utils/src/sockopt/ip_mtu_discover.rs                      81                81     0.00%           6                 6     0.00%          72                72     0.00%           0                 0         -
lightway-app-utils/src/sockopt/ip_pktinfo.rs                           14                14     0.00%           1                 1     0.00%          16                16     0.00%           0                 0         -
lightway-app-utils/src/tun.rs                                         393               389     1.02%          42                41     2.38%         251               245     2.39%           0                 0         -
lightway-app-utils/src/utils.rs                                        21                21     0.00%           1                 1     0.00%          11                11     0.00%           0                 0         -
lightway-client/src/config.rs                                         234                39    83.33%          12                 5    58.33%         172                34    80.23%           0                 0         -
lightway-client/src/dns_manager.rs                                     18                18     0.00%           4                 4     0.00%          16                16     0.00%           0                 0         -
lightway-client/src/io/inside.rs                                      267                 6    97.75%          29                 0   100.00%         181                 2    98.90%           0                 0         -
lightway-client/src/io/inside/tun.rs                                  115               115     0.00%          16                16     0.00%          85                85     0.00%           0                 0         -
lightway-client/src/io/outside.rs                                      14                14     0.00%           2                 2     0.00%          12                12     0.00%           0                 0         -
lightway-client/src/io/outside/tcp.rs                                  85                85     0.00%          12                12     0.00%          52                52     0.00%           0                 0         -
lightway-client/src/io/outside/udp.rs                                 163               163     0.00%          17                17     0.00%         104               104     0.00%           0                 0         -
lightway-client/src/io/outside/udp_batch_receiver.rs                   61                11    81.97%           6                 1    83.33%          56                12    78.57%           0                 0         -
lightway-client/src/keepalive.rs                                      615                44    92.85%          55                 6    89.09%         330                25    92.42%           0                 0         -
lightway-client/src/lib.rs                                           1253               677    45.97%         103                54    47.57%         845               499    40.95%           0                 0         -
lightway-client/src/main.rs                                           367               367     0.00%          20                20     0.00%         236               236     0.00%           0                 0         -
lightway-client/src/platform/linux/dns_manager.rs                     118                91    22.88%          14                 9    35.71%          87                62    28.74%           0                 0         -
lightway-client/src/platform/linux/dns_manager/direct_file.rs         106                42    60.38%          12                 6    50.00%          65                29    55.38%           0                 0         -
lightway-client/src/platform/linux/dns_manager/resolvconf.rs           61                61     0.00%           8                 8     0.00%          52                52     0.00%           0                 0         -
lightway-client/src/platform/linux/dns_manager/resolvectl.rs           41                41     0.00%           5                 5     0.00%          34                34     0.00%           0                 0         -
lightway-client/src/route_manager.rs                                 1190               212    82.18%          77                 8    89.61%         663               117    82.35%           0                 0         -
lightway-core/src/borrowed_bytesmut.rs                                373                 0   100.00%          24                 0   100.00%         185                 0   100.00%           0                 0         -
lightway-core/src/builder_predicates.rs                                24                12    50.00%           4                 2    50.00%          24                12    50.00%           0                 0         -
lightway-core/src/cipher.rs                                            13                 0   100.00%           2                 0   100.00%          10                 0   100.00%           0                 0         -
lightway-core/src/connection.rs                                      1722               875    49.19%          75                29    61.33%        1211               579    52.19%           0                 0         -
lightway-core/src/connection/builders.rs                              246                49    80.08%          22                 9    59.09%         258                59    77.13%           0                 0         -
lightway-core/src/connection/dplpmtud.rs                             1741                81    95.35%          63                 0   100.00%         830                 7    99.16%           0                 0         -
lightway-core/src/connection/expresslane.rs                            26                11    57.69%           4                 1    75.00%          34                 7    79.41%           0                 0         -
lightway-core/src/connection/fragment_map.rs                          366                 6    98.36%          25                 0   100.00%         254                 3    98.82%           0                 0         -
lightway-core/src/connection/io_adapter.rs                            526                23    95.63%          34                 5    85.29%         276                21    92.39%           0                 0         -
lightway-core/src/connection/key_update.rs                             34                13    61.76%           5                 0   100.00%          38                19    50.00%           0                 0         -
lightway-core/src/context.rs                                          212                51    75.94%          28                11    60.71%         229                64    72.05%           0                 0         -
lightway-core/src/context/ip_pool.rs                                    8                 3    62.50%           1                 0   100.00%           5                 0   100.00%           0                 0         -
lightway-core/src/context/server_auth.rs                               32                24    25.00%           4                 3    25.00%          24                20    16.67%           0                 0         -
lightway-core/src/encoding_request_states.rs                            3                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
lightway-core/src/io.rs                                                30                30     0.00%           5                 5     0.00%          25                25     0.00%           0                 0         -
lightway-core/src/keyshare.rs                                           5                 0   100.00%           1                 0   100.00%           5                 0   100.00%           0                 0         -
lightway-core/src/lib.rs                                                9                 0   100.00%           3                 0   100.00%           9                 0   100.00%           0                 0         -
lightway-core/src/metrics.rs                                           51                41    19.61%          21                17    19.05%          48                38    20.83%           0                 0         -
lightway-core/src/packet.rs                                            38                10    73.68%           4                 1    75.00%          30                 6    80.00%           0                 0         -
lightway-core/src/plugin.rs                                           303                13    95.71%          21                 3    85.71%         145                 7    95.17%           0                 0         -
lightway-core/src/utils.rs                                            334                26    92.22%          22                 2    90.91%         165                17    89.70%           0                 0         -
lightway-core/src/version.rs                                           93                 0   100.00%          17                 0   100.00%          82                 0   100.00%           0                 0         -
lightway-core/src/wire.rs                                             438                42    90.41%          29                 1    96.55%         235                12    94.89%           0                 0         -
lightway-core/src/wire/auth_failure.rs                                 27                 1    96.30%           3                 0   100.00%          17                 0   100.00%           0                 0         -
lightway-core/src/wire/auth_request.rs                                472                12    97.46%          26                 0   100.00%         241                 0   100.00%           0                 0         -
lightway-core/src/wire/auth_success_with_config_ipv4.rs               222                 3    98.65%          11                 0   100.00%         124                 0   100.00%           0                 0         -
lightway-core/src/wire/data.rs                                         51                 1    98.04%           5                 0   100.00%          33                 0   100.00%           0                 0         -
lightway-core/src/wire/data_frag.rs                                   127                 1    99.21%          14                 0   100.00%          80                 0   100.00%           0                 0         -
lightway-core/src/wire/encoding_request.rs                             82                 2    97.56%           6                 0   100.00%          42                 1    97.62%           0                 0         -
lightway-core/src/wire/encoding_response.rs                            82                 2    97.56%           6                 0   100.00%          42                 1    97.62%           0                 0         -
lightway-core/src/wire/expresslane_config.rs                          166                 1    99.40%           8                 0   100.00%          92                 0   100.00%           0                 0         -
lightway-core/src/wire/expresslane_data.rs                           1038                61    94.12%          41                 5    87.80%         528                36    93.18%           0                 0         -
lightway-core/src/wire/ping.rs                                         95                 2    97.89%           7                 0   100.00%          59                 0   100.00%           0                 0         -
lightway-core/src/wire/pong.rs                                        109                 2    98.17%           8                 0   100.00%          72                 0   100.00%           0                 0         -
lightway-core/src/wire/server_config.rs                                69                 2    97.10%           4                 0   100.00%          37                 0   100.00%           0                 0         -
lightway-server/src/auth.rs                                           273                49    82.05%          22                 6    72.73%         171                28    83.63%           0                 0         -
lightway-server/src/config.rs                                          23                23     0.00%           1                 1     0.00%          35                35     0.00%           0                 0         -
lightway-server/src/connection.rs                                     137               137     0.00%          11                11     0.00%         110               110     0.00%           0                 0         -
lightway-server/src/connection_manager.rs                             335               335     0.00%          39                39     0.00%         288               288     0.00%           0                 0         -
lightway-server/src/connection_manager/connection_map.rs              407                16    96.07%          21                 1    95.24%         228                 7    96.93%           0                 0         -
lightway-server/src/io/inside/tun.rs                                   44                44     0.00%           9                 9     0.00%          30                30     0.00%           0                 0         -
lightway-server/src/io/outside/tcp.rs                                  90                90     0.00%           9                 9     0.00%          65                65     0.00%           0                 0         -
lightway-server/src/io/outside/udp.rs                                 320               320     0.00%          16                16     0.00%         206               206     0.00%           0                 0         -
lightway-server/src/io/outside/udp/cmsg.rs                            197                53    73.10%          14                 5    64.29%         158                43    72.78%           0                 0         -
lightway-server/src/ip_manager.rs                                     542                50    90.77%          22                 4    81.82%         242                23    90.50%           0                 0         -
lightway-server/src/ip_manager/ip_pool.rs                             540                 0   100.00%          27                 0   100.00%         252                 0   100.00%           0                 0         -
lightway-server/src/lib.rs                                            287               287     0.00%          15                15     0.00%         185               185     0.00%           0                 0         -
lightway-server/src/main.rs                                           231               231     0.00%          10                10     0.00%         122               122     0.00%           0                 0         -
lightway-server/src/metrics.rs                                        254               250     1.57%          85                83     2.35%         215               211     1.86%           0                 0         -
lightway-server/src/statistics.rs                                     132                59    55.30%           8                 4    50.00%          86                35    59.30%           0                 0         -
uniffi-bindgen/src/uniffi-bindgen.rs                                    3                 3     0.00%           1                 1     0.00%           3                 3     0.00%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                               18989              5998    68.41%        1393               560    59.80%       11683              4166    64.34%           0                 0         -

✅ Region coverage 68% passes
✅ Line coverage 64% passes

@kp-thomas-yau kp-thomas-yau changed the base branch from main to inside-recv-buf-takes-bytesmut May 6, 2026 04:22
@kp-thomas-yau kp-thomas-yau force-pushed the send-multiple-tun branch 3 times, most recently from 0eb651a to 175d1d3 Compare May 6, 2026 04:50
@kp-thomas-yau kp-thomas-yau changed the title CVPN-2515: Send multiple packets to TUN at once CVPN-2515: Send/receive multiple packets to TUN at once May 6, 2026
@kp-thomas-yau kp-thomas-yau changed the title CVPN-2515: Send/receive multiple packets to TUN at once CVPN-2515/CVPN-2512: Send/receive multiple packets to TUN at once May 6, 2026
Base automatically changed from inside-recv-buf-takes-bytesmut to main May 6, 2026 06:51
@kp-thomas-yau kp-thomas-yau force-pushed the send-multiple-tun branch 6 times, most recently from 01ef021 to 8f1cfb6 Compare May 6, 2026 11:41
It will also enable batch receive if TUN offloading is ON as well.
Implement the trait and also add GROTable to TunDirect object so that we can reuse this across different calls.

The implementation now is not exactly perfect as well, as we're now doing an extra layer of copying to create an offset in BytesMut to pre-allocate space for VIRTIO_NET_HDR_LEN
Add this variable in to let us determine whether we need to batch the packets here or not
Add a queue for the packets that are going to the TUN device if `inside_batch_enabled` is on. Also add a new method to flush all the queued packets to be sent all at once via send_multiple function.
Move this value to this constant for better clarity
This struct will be used later on so that the caller of TUN's receive method can pass in a mutable reference of a buffer for recv_multiple to use directly. This fixes the issue of requiring a mutex lock needed for every buffer and easier to tell who is the owner of the buffer space.
Add a new `recv_multiple_buf` to `InsideIORecv` trait, which allows user to receive multiple packets from the TUN device directly (via TUN offloading).
This wrapper makes thing easier by abstracting the logic of between receiving multiple and single buffer from the socket, so that the main inside_io_task loop won't be too messy and hard test.

Added some unit tests thanks to mockall.
@kp-thomas-yau kp-thomas-yau marked this pull request as ready for review May 7, 2026 04:21
@kp-thomas-yau kp-thomas-yau requested a review from a team as a code owner May 7, 2026 04:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant