The visible goal was a Web app: type a URL, see the page, fetched over the
TCP stack SUB-OS built. Getting there surfaced -- and forced fixes to --
two deeper problems.
Async HTTP client (net/http_client.c)
tcp_connect/recv all block, so fetching from the GUI would freeze the
compositor. Fetches now run on one long-lived kernel worker thread that
services a single-slot mailbox; the requester posts a job and polls its
state. Ownership follows the state, and the two hand-off points (worker
completion, requester release) run under interrupts-off, so a window can
close mid-fetch with neither leak nor use-after-free. do_fetch returns
its terminal state rather than publishing it, closing a window where the
requester could free the job before finalize read it.
Cooperative scheduling for the idle thread (the core fix)
The login prompt, the shell and the GUI compositor all run on the pid-0
idle thread, which the scheduler by design never preempts (sched_tick
skips pid 0) and never requeues on a yield (weave_on clear). So it could
neither be preempted to let the worker run, nor cleanly yield -- the
worker was simply starved, and an early attempt to yield from it froze
the shell outright. The idle thread now joins the weave rotation once
hardware interrupts are live, and the network wait loops (tcp, dns, ping,
the worker's idle) yield through a new net_wait() instead of halting the
CPU. The compositor spends its frame-budget slack yielding too. Result:
a background fetch progresses while the desktop holds 54 FPS.
Web app (gui/gui_app_web.c)
A text browser: URL bar, Go, and a Raw/Reader toggle. Raw shows the
response verbatim; Reader strips markup (dropping <script>/<style>
bodies), decodes the common entities, and word-wraps the body. The line
index is cached against content/view/width so scrolling and idle
repaints are free. It only ever polls the worker, so it never stalls.
TCP throughput (the limit flagged last time)
The 32-slot table with a 2 s TIME_WAIT linger capped sustained serving
at ~16 req/s. The table is now 64 slots, a connection this end closed
after serving a request lingers 250 ms rather than 2 s, and expired
TIME_WAIT entries are reaped from the count path so a quiet table still
drains. netstat/netmon gained served-since-boot and peak-slots counters.
Also: a `webget` CLI (the async client from a shell), listen/accept
introspection already present, and a globe icon for the app.
Verified on x86_64: `webget` fetched an 812-byte page (HTTP 200, 10 ms) on
the worker thread; the GUI Web app rendered the same page in Reader mode
while the desktop held 54 FPS; an 80-request burst through the 64-slot
table returned 80/80 (was 39/40 before) and netstat reported "80 served,
peak 33 of 64 slots" -- a peak that would have overflowed the old table.
Builds clean on x86_64, aarch64 and armv8i.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Fh5z6V7nGPbHvqVG5p4CXX
Kernel: the passive-open half of TCP. tcp_listen claims a port, inbound
SYNs to it are completed by the receive path, and the finished
connections queue in a backlog until tcp_accept collects them. Accepted
connections run the same stream state machine as active opens, so
tcp_send/tcp_recv/tcp_close work on them unchanged. sys_listen and
sys_accept expose this through the BSD socket API, with sys_accept
returning a fresh descriptor and leaving the listener available for the
next caller. The built-in sshd and httpd predate this and are still
answered inline from the receive path; the role they use is now named
TCP_ROLE_SERVICE to keep that distinction explicit.
GUI: a Network Monitor app. Interface configuration and link state, a
rolling dual-line chart of receive and transmit rates sampled from the
NIC counters on a fixed 2 Hz cadence (so it reads in packets per half
second no matter how often the compositor repaints), byte and packet
totals, and the TCP engine's connection table with each connection's real
state and direction, scrollable when it outgrows the window.
Two bugs the monitor made visible, both fixed:
- Served HTTP connections were never closed. The response advertises
"Connection: close" but the engine left the connection ESTABLISHED
forever, so the 32-entry table filled up and the server stopped
answering after 32 requests. The service path now sends its FIN after
the response and lingers in TIME_WAIT until the reaper takes it. 40
sequential requests through a 32-slot table now yield 39 successes
where the 33rd onwards previously failed permanently.
- tcp_close gave up in FIN_WAIT2 after 500 ms and freed the slot, so a
peer whose own FIN arrived later got an RST instead of an ACK. It now
falls into TIME_WAIT instead, and TIME_WAIT acknowledges a first FIN
rather than only a retransmitted one. Expired TIME_WAIT entries are
also reaped from tcp_get_connections_count, so a table that sees no
further connections still empties out.
Verified on x86_64 against real peers, checked packet by packet in a
capture: a host client connected to `tcpserve 9000` in the guest and got
its reply -- SYN/SYN-ACK/ACK with a random ISN, 30-byte request, 43-byte
reply, and a clean four-way close whose final ACK is correct even when
the peer's FIN arrives 1.2 s late. Zero bad checksums throughout. The
monitor was screenshotted under live load showing connections moving
through TIME_WAIT and being reclaimed. Builds clean on x86_64, aarch64
and armv8i.
Known limit: 32 connection slots with a 2 s TIME_WAIT linger means
sustained rates much above ~16 requests/second can transiently exhaust
the table; it recovers on its own as entries expire.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Fh5z6V7nGPbHvqVG5p4CXX
