The visible goal was a Web app: type a URL, see the page, fetched over the
TCP stack SUB-OS built. Getting there surfaced -- and forced fixes to --
two deeper problems.
Async HTTP client (net/http_client.c)
tcp_connect/recv all block, so fetching from the GUI would freeze the
compositor. Fetches now run on one long-lived kernel worker thread that
services a single-slot mailbox; the requester posts a job and polls its
state. Ownership follows the state, and the two hand-off points (worker
completion, requester release) run under interrupts-off, so a window can
close mid-fetch with neither leak nor use-after-free. do_fetch returns
its terminal state rather than publishing it, closing a window where the
requester could free the job before finalize read it.
Cooperative scheduling for the idle thread (the core fix)
The login prompt, the shell and the GUI compositor all run on the pid-0
idle thread, which the scheduler by design never preempts (sched_tick
skips pid 0) and never requeues on a yield (weave_on clear). So it could
neither be preempted to let the worker run, nor cleanly yield -- the
worker was simply starved, and an early attempt to yield from it froze
the shell outright. The idle thread now joins the weave rotation once
hardware interrupts are live, and the network wait loops (tcp, dns, ping,
the worker's idle) yield through a new net_wait() instead of halting the
CPU. The compositor spends its frame-budget slack yielding too. Result:
a background fetch progresses while the desktop holds 54 FPS.
Web app (gui/gui_app_web.c)
A text browser: URL bar, Go, and a Raw/Reader toggle. Raw shows the
response verbatim; Reader strips markup (dropping <script>/<style>
bodies), decodes the common entities, and word-wraps the body. The line
index is cached against content/view/width so scrolling and idle
repaints are free. It only ever polls the worker, so it never stalls.
TCP throughput (the limit flagged last time)
The 32-slot table with a 2 s TIME_WAIT linger capped sustained serving
at ~16 req/s. The table is now 64 slots, a connection this end closed
after serving a request lingers 250 ms rather than 2 s, and expired
TIME_WAIT entries are reaped from the count path so a quiet table still
drains. netstat/netmon gained served-since-boot and peak-slots counters.
Also: a `webget` CLI (the async client from a shell), listen/accept
introspection already present, and a globe icon for the app.
Verified on x86_64: `webget` fetched an 812-byte page (HTTP 200, 10 ms) on
the worker thread; the GUI Web app rendered the same page in Reader mode
while the desktop held 54 FPS; an 80-request burst through the 64-slot
table returned 80/80 (was 39/40 before) and netstat reported "80 served,
peak 33 of 64 slots" -- a peak that would have overflowed the old table.
Builds clean on x86_64, aarch64 and armv8i.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Fh5z6V7nGPbHvqVG5p4CXX
Adds the client half of TCP -- tcp_connect/tcp_send/tcp_recv/tcp_close --
and wires SOCK_STREAM sockets to it, so the kernel can originate an
outbound stream rather than only answer inbound ones. Inbound segments
are now matched against the connection table before being offered to the
listeners; previously anything not addressed to a listening port was
answered with RST, which would have rejected the SYN-ACK completing our
own handshake.
Transmission is stop-and-wait: one segment outstanding, retransmitted
until acknowledged. Every blocking call is bounded against the timer tick
and halts the CPU between checks, so a silent peer costs a timeout rather
than a hang. Connections linger in TIME_WAIT so a late duplicate cannot
be mistaken for traffic on a new incarnation.
Verifying it against a real peer uncovered two pre-existing bugs that had
made TCP unusable in both directions. Neither was visible from inside the
guest, which is why the servers had looked fine:
1. The TCP and UDP checksums were computed by reading the header a
uint16_t at a time, immediately after the caller stored zero into that
same header's checksum field. Nothing stops the compiler reordering
those word reads around that store, and at -O2 it did: the stale
checksum field was folded into its own result. Every TCP segment the
kernel had ever sent carried a wrong checksum and was silently dropped
by the peer -- no client could complete a handshake with sshd or
httpd, and it is the likely reason DHCP and DNS replies never arrived.
The checksum is now summed a byte at a time through unsigned char
pointers, which may alias any object and so cannot be reordered, and
which also drops the alignment assumption that reading a packed header
as uint16_t made -- a fault rather than a slowdown on ARM. The helper
is shared by IP, TCP and UDP instead of being open-coded four times.
2. net_receive derived the datagram length from the Ethernet frame rather
than from the IP header's total_length, so the padding on any frame
shorter than 60 bytes was delivered as protocol payload. A bare ACK or
FIN appeared to carry six bytes, which pushed the acknowledgement
number out of step with the peer and turned every close into a FIN
retransmission storm answered with RST.
Also adds `tcpconnect` and `httpget` applets, and gives netstat the real
per-connection state and direction from the engine's table.
Verified on x86_64 against a real TCP stack, checked packet by packet in
a capture: `httpget` fetched a 349-byte page from a server on the host --
handshake, 81-byte request, response, orderly four-way close, every
sequence number exact, zero bad checksums, no retransmissions. In the
other direction curl on the host now fetches a live page from the guest's
httpd, which had never worked. Builds clean on x86_64, aarch64 and armv8i.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Fh5z6V7nGPbHvqVG5p4CXX
