Skip to content

runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390

Open
achille-roussel wants to merge 3 commits into
tinygo-org:devfrom
achille-roussel:wasip2-net
Open

runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390
achille-roussel wants to merge 3 commits into
tinygo-org:devfrom
achille-roussel:wasip2-net

Conversation

@achille-roussel
Copy link
Copy Markdown
Contributor

Stacked on #5386. Mirrors the wasip1 work for wasip2. The cooperative scheduler's idle path now calls wasi:io/poll.Poll over a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a single monotonic-clock subscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines.

End to end:

$ tinygo build -target=wasip2 -o tcpecho_wasip2.wasm ./tcpecho_wasip2.go
$ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm &
listening on 127.0.0.1:9999
tick 1
tick 2
tick 3

$ echo hello | nc 127.0.0.1 9999
hello                                    # echoed by the wasm

Two concurrent nc clients both echo while the ticker keeps progressing — the cooperative scheduler keeps running goroutines parked on sock_recv via the new pollable registry.

How it works

Wasip2's polling primitive is wasi:io/poll.Poll(list<own<pollable>>) -> list<u32> taking a list of pollable resource handles. Each blocking wasi operation has a subscribe() that yields a pollable.

TCPConn.Read / Write   ─would-block─►  internal/poll registry  ─►  task.Pause()
                                                │
                                                ▼
                       scheduler idle  ──►  pollIO(timeoutNs)
                                                │
                                                ├─ build pollables: [clock?, p1, p2, …]
                                                ├─ wasi:io/poll.Poll(...)
                                                └─ wake matched tasks → run queue

The wasip2 path mirrors the wasip1 PR's structure file-for-file:

wasip1 wasip2
runtime/netpoll_wasip1.go (FD-keyed poll_oneoff registry) runtime/netpoll_wasip2.go (pollable-keyed wasi:io/poll.Poll registry)
runtime/scheduler_idle_wasip1.go (cooperative sleepTicks routes via pollIO) runtime/scheduler_idle_wasip2.go (cooperative sleepTicks routes via pollIO)
runtime/scheduler_idle_wasip1_none.go (non-coop fallback) runtime/scheduler_idle_wasip2_none.go (non-coop fallback)
internal/poll/fd_wasip1.go (FD type) internal/poll/fd_wasip2.go (WasipNFD over (TcpSocket, InputStream, OutputStream))
syscall/syscall_libc_wasip1.go park-on-EAGAIN (not needed — wasip2 wasi calls return would-block directly into internal/poll)

What's added

  • src/runtime/netpoll_wasip2.go — pollable-keyed pollDesc registry, pollIO(timeoutNs) building one combined wasi:io/poll.Poll call (clock + active pollables), linkname-exposed wake helpers (runtime_netpoll_addpollable_wasip2 / done / pdfired / wake). Three timing cases handled like wasip1's pollIO.
  • src/runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go — cooperative-variant sleepTicks / waitForEvents that route through pollIO when pollables are registered; non-coop fallback uses monotonicclock.Block. Matches the wasip1 file split.
  • src/runtime/runtime_wasip2.gosleepTicks moved out (now per-config in scheduler_idle_wasip2*.go).
  • src/runtime/wait_other.go — build tag tightened to exclude wasip2.
  • src/internal/poll/fd_wasip2.goWasipNFD wraps (TcpSocket, InputStream, OutputStream). DialTCPWasip2, ListenTCPWasip2, Accept, Read, Write, Close, SetDeadline*. Each blocking op tries the wasi call, on would-block subscribes, parks via runtime_netpoll_addpollable_wasip2 + task.Pause, on resume drops pollable + retries. Deadline-aware variants use parkUntil + time.AfterFunc + runtime_netpoll_wake_wasip2. Linkname-friendly Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers for test / future net callers.
  • src/internal/poll/errors_wasip.go — error sentinels (ErrFileClosing, ErrNetClosing, ErrDeadlineExceeded, ErrNoDeadline) extracted from fd_wasip1.go to a wasip1 || wasip2 shared file.
  • src/os/poll_link_wasip2.go — pulls internal/poll into the wasip2 build (no-op blank import with justifying comment for the lint check).
  • loader/goroot.golistGorootMergeLinks now filters TinyGo files by //go:build constraints (via go/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides for follow-up work without disturbing wasip1.

Non-goals (deferred)

  • net.Listen("tcp", ...) / net.Dial("tcp", ...) on wasip2 via the standard net package. Upstream Go's net doesn't currently build for wasip2 because its cgo_linux.go reaches for netdb.h even though TinyGo doesn't enable cgo. Bringing up upstream net on wasip2 (either by filtering its cgo files from the merge or providing a TinyGo-native net override) is its own piece of work — the linkname-friendly TCP helpers in this PR are the foundation. The synthetic test program calls them directly.
  • UDP / PacketConn and DNS resolution. Same shape will work (wasi:sockets/udp, wasi:sockets/ip-name-lookup) but not in scope here.

Wasip1 regression sweep: tcpecho.wasm from #5386 still passes (echo hi | nc 127.0.0.1 9998 round-trip + concurrent clients); time.Sleep / parkfile / parksynth unchanged.

achille-roussel and others added 3 commits May 11, 2026 16:08
…ation + net.FileListener

On wasip1 today every syscall.Read/Write blocks the entire wasm module — the
cooperative scheduler invokes poll_oneoff only for sleep/timer wakeups, and
there's no path from the net package to a working TCP server. This change
fixes both: it threads poll_oneoff through the scheduler's idle path so a
goroutine doing FD I/O parks instead of blocking the module, and it provides
enough internal/poll / os / syscall surface that upstream Go's
net.FileListener / net.FileConn works on a host-pre-opened TCP socket.

End to end:

  $ tinygo build -target=wasip1 -o tcpecho.wasm ./tcpecho.go
  $ wasmtime run -Spreview2=false -Stcplisten=127.0.0.1:9999 ./tcpecho.wasm &
  listening on FD 3

  $ echo hello | nc 127.0.0.1 9999
  hello                                  # echoed by the wasm

Concurrent connections work too — multiple nc clients hit the server back-to-
back; both got accepted and echoed while the cooperative scheduler kept
running goroutines parked on sock_recv. The tcpecho.go source is the minimal
upstream-Go-idiomatic TCP echo server, no TinyGo net override required:

  f := os.NewFile(3, "tcplisten")
  ln, _ := net.FileListener(f)
  for {
      c, _ := ln.Accept()
      go func(c net.Conn) { defer c.Close(); io.Copy(c, c) }(c)
  }

Architecture:

The cooperative scheduler's idle path now calls poll_oneoff with one combined
subscription array: a clock subscription for the next timer/sleep deadline,
plus one FD subscription per goroutine that's parked waiting on I/O.

  syscall.Read/Write ─EAGAIN─► internal/poll registry ─► task.Pause()
                                       │
                                       ▼
                  scheduler idle  ──►  pollIO(timeoutNs)
                                       │
                                       ├─ build subs: [clock, fd1, fd2, …]
                                       ├─ poll_oneoff(...)
                                       └─ wake matched tasks → run queue

Upstream net's net.FileListener(f) flow plumbs through these layers:

  1. (*os.File).PollFD() returns a cached *poll.FD stored in a new
     pfd pollFD field on the shared file struct. pollFD is a per-target
     alias — *poll.FD on wasip1, literal struct{} (zero bytes, non-
     trailing) on every other target.
  2. (*poll.FD).Copy() increments a SysFile refcount so f.Close() and
     ln.Close() cooperatively release the syscall FD.
  3. Upstream net/file_wasip1.go calls fd_fdstat_get_type (linknamed into
     our syscall) to detect FILETYPE_SOCKET_STREAM.
  4. Listener.Accept → (*poll.FD).Accept → syscall.Accept → sock_accept
     wasmimport. EAGAIN parks the goroutine via the runtime netpoll
     registry.
  5. Conn.Read/Write see isSocket() == true (cached SysFile.Filetype) and
     dispatch to sock_recv / sock_send direct wasmimports with park-on-
     EAGAIN + deadline support.
  6. Conn.Close triggers Shutdown (→ sock_shutdown) and refcount-aware
     Close.

Non-goals (deferred):

- net.Dial("tcp", ...), net.Listen("tcp", ...) — wasip1 has no
  sock_connect / bind / listen (only sock_accept / recv / send /
  shutdown). Outbound TCP requires the wasi-sockets proposal (preview2+)
  or a runtime-specific extension. The relevant stubs return ENOSYS so
  callers see a clean error.
- UDP / PacketConn — upstream's filePacketConn returns ENOPROTOOPT on
  wasip1.
- DNS resolution — FileListener / FileConn paths bypass the resolver
  entirely.
- wasip2 — uses pollable resources, structurally different. Future PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TestBinarySize/hifive1b/examples/echo regressed by 32 bytes after the
previous commit routed the scheduler's idle wait through a
schedulerIdleWait helper. The extra call frame + branch landed on every
non-wasip1 cooperative target, where the original direct sleepTicks /
waitForEvents calls compile to a single inlined call.

Push the wasip1-specific FD-polling logic down into the wasip1 sleepTicks
/ waitForEvents definitions themselves and revert scheduler_cooperative.go
to its pre-PR shape. Non-wasip1 targets now produce the exact same code
they did before this PR. On wasip1, sleepTicks routes through pollIO
when FD waiters are registered (unchanged behavior, just relocated).

- scheduler_cooperative.go: back to sleepTicks / waitForEvents.
- scheduler_idle_wasip1.go: defines the integrated sleepTicks +
  waitForEvents for wasip1 cooperative builds.
- scheduler_idle_wasip1_none.go: fallback definitions for wasip1 builds
  that don't use the cooperative scheduler (-scheduler=none/threads).
- runtime_wasip1.go: drop the now-relocated sleepTicks.
- wait_other.go: exclude wasip1 (covered by the two files above).
- scheduler_idle_other.go: removed; no longer needed.
Mirrors PR tinygo-org#5386's wasip1 work for wasip2. The cooperative scheduler's
idle path now calls wasi:io/poll.Poll over a combined list of (clock
pollable, registered pollables) instead of blocking the wasm module on
a single monotonic-clock subscription, so goroutines doing TCP I/O can
park while the scheduler runs other goroutines.

Plumbing components:

- runtime/netpoll_wasip2.go: pollable-keyed pollDesc registry; pollIO
  builds one combined wasi:io/poll.Poll call (clock pollable + active
  pollables). Linkname-exposed runtime_netpoll_addpollable_wasip2 /
  done / pdfired / wake for internal/poll and future net.
- runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go:
  cooperative-variant sleepTicks / waitForEvents that route through
  pollIO; non-coop fallback uses monotonicclock.Block. Mirrors the
  wasip1 structure introduced in 7000e7b.
- runtime/runtime_wasip2.go: sleepTicks moved out to the
  scheduler_idle_wasip2*.go files.
- runtime/wait_other.go: build tag tightened to exclude wasip2.

internal/poll surface:

- internal/poll/fd_wasip2.go: WasipNFD wraps a (TcpSocket, InputStream,
  OutputStream) triple. DialTCPWasip2, ListenTCPWasip2, Accept, Read,
  Write, Close, SetDeadline*. Each blocking op tries the wasi call,
  on would-block subscribes, parks, retries — same pattern as the
  wasip1 internal/poll.FD but pollable-keyed. Linkname-friendly
  Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers
  for test / future net callers.
- internal/poll/errors_wasip.go: ErrFileClosing / ErrNetClosing /
  ErrDeadlineExceeded / ErrNoDeadline extracted from fd_wasip1.go to
  a wasip1||wasip2 shared file.

Loader change:

- loader/goroot.go: listGorootMergeLinks now filters TinyGo files by
  //go:build constraints (via go/build.Context.MatchFile) before
  deciding "TinyGo owns this directory". Files that don't match the
  current target no longer cause upstream Go files at the same level
  to be dropped. Unblocks per-target overrides in directories like
  src/net/ for future net.wasip2 work without disturbing wasip1.

End-to-end verification:

  $ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm &
  listening on 127.0.0.1:9999
  tick 1
  tick 2
  tick 3
  $ echo hello | nc 127.0.0.1 9999
  hello                                    # echoed by the wasm
  $ # two concurrent clients echo cleanly while ticker keeps ticking

The test program (not shipped) uses //go:linkname to drive the
internal/poll TCP helpers directly, since TinyGo doesn't yet have a
net.Listen / net.Dial path on wasip2 (upstream Go's net doesn't build
for wasip2 due to cgo_linux.go reaching for Linux headers). The
src/net/ wasip2 wrappers are out of scope for this PR and tracked as
follow-up — once they land, callers will use net.Listen / Dial
directly and the linkname wrappers can drop.

Wasip1 regression sweep: tcpecho.wasm still passes; time.Sleep / parkfile
/ parksynth unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant