feat(net,gui,sched): async HTTP client + Web browser app, and the scheduler fix that makes background work possible

The visible goal was a Web app: type a URL, see the page, fetched over the
TCP stack SUB-OS built. Getting there surfaced -- and forced fixes to --
two deeper problems.

Async HTTP client (net/http_client.c)
  tcp_connect/recv all block, so fetching from the GUI would freeze the
  compositor. Fetches now run on one long-lived kernel worker thread that
  services a single-slot mailbox; the requester posts a job and polls its
  state. Ownership follows the state, and the two hand-off points (worker
  completion, requester release) run under interrupts-off, so a window can
  close mid-fetch with neither leak nor use-after-free. do_fetch returns
  its terminal state rather than publishing it, closing a window where the
  requester could free the job before finalize read it.

Cooperative scheduling for the idle thread (the core fix)
  The login prompt, the shell and the GUI compositor all run on the pid-0
  idle thread, which the scheduler by design never preempts (sched_tick
  skips pid 0) and never requeues on a yield (weave_on clear). So it could
  neither be preempted to let the worker run, nor cleanly yield -- the
  worker was simply starved, and an early attempt to yield from it froze
  the shell outright. The idle thread now joins the weave rotation once
  hardware interrupts are live, and the network wait loops (tcp, dns, ping,
  the worker's idle) yield through a new net_wait() instead of halting the
  CPU. The compositor spends its frame-budget slack yielding too. Result:
  a background fetch progresses while the desktop holds 54 FPS.

Web app (gui/gui_app_web.c)
  A text browser: URL bar, Go, and a Raw/Reader toggle. Raw shows the
  response verbatim; Reader strips markup (dropping <script>/<style>
  bodies), decodes the common entities, and word-wraps the body. The line
  index is cached against content/view/width so scrolling and idle
  repaints are free. It only ever polls the worker, so it never stalls.

TCP throughput (the limit flagged last time)
  The 32-slot table with a 2 s TIME_WAIT linger capped sustained serving
  at ~16 req/s. The table is now 64 slots, a connection this end closed
  after serving a request lingers 250 ms rather than 2 s, and expired
  TIME_WAIT entries are reaped from the count path so a quiet table still
  drains. netstat/netmon gained served-since-boot and peak-slots counters.

Also: a `webget` CLI (the async client from a shell), listen/accept
introspection already present, and a globe icon for the app.

Verified on x86_64: `webget` fetched an 812-byte page (HTTP 200, 10 ms) on
the worker thread; the GUI Web app rendered the same page in Reader mode
while the desktop held 54 FPS; an 80-request burst through the 64-slot
table returned 80/80 (was 39/40 before) and netstat reported "80 served,
peak 33 of 64 slots" -- a peak that would have overflowed the old table.
Builds clean on x86_64, aarch64 and armv8i.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Fh5z6V7nGPbHvqVG5p4CXX
This commit is contained in:
subhobhai943andClaude Opus 4.8 committed 2026-08-27 02:21:48 +05:30
1 parent eb92502deb
commit dd4922767a
16 files changed
+898 -22

No files matched your search

+56 -2
View File
@@ -7,6 +7,7 @@
#include <net/dhcp.h>
#include <net/udp.h>
#include <net/tcp.h>
#include <net/http_client.h>
#include <net/socket.h>
#include <drivers/e1000.h>
#include <drivers/ata.h>
@@ -2226,7 +2227,7 @@ static int applet_netstat(int argc, char** argv) {
// Live connections from the TCP engine's own table: inbound ones accepted
// by a listener, outbound ones opened by tcp_connect.
for (size_t i = 0; i < 32; i++) {
for (int i = 0; i < tcp_conn_table_size(); i++) {
const tcp_conn_t* c = tcp_get_connection(i);
if (!c) continue;
@@ -2256,7 +2257,10 @@ static int applet_netstat(int argc, char** argv) {
udp_get_stats(&urx, &utx, &udrop);
printk(ANSI_YELLOW "\nTransport statistics:\n" ANSI_RESET);
printk(" udp: %llu datagrams received, %llu sent, %llu dropped\n", urx, utx, udrop);
printk(" tcp: %u active connection(s)\n", (unsigned)tcp_get_connections_count());
printk(" tcp: %u active, %llu served since boot, peak %d of %d slots\n",
(unsigned)tcp_get_connections_count(),
(unsigned long long)tcp_get_total_opened(),
tcp_get_peak_connections(), tcp_conn_table_size());
return 0;
}
@@ -2435,6 +2439,55 @@ static int applet_tcpserve(int argc, char** argv) {
return 0;
}
// Fetch a URL through the ASYNC client: the request runs on the kernel HTTP
// worker thread while this applet just polls, exactly as the GUI Web app does.
// It is the command-line counterpart to that app and the way to exercise the
// worker path from a serial console.
static int applet_webget(int argc, char** argv) {
if (argc < 2) {
printk("usage: webget <url> (e.g. webget 10.0.2.2:8000/)\n");
return 1;
}
http_fetch_t* f = http_fetch_start(argv[1], 16384);
if (!f) {
printk(ANSI_BRIGHT_RED "webget: could not start (bad URL, or a fetch is "
"already running)\n" ANSI_RESET);
return 1;
}
printk("Fetching %s on the HTTP worker thread ...\n", argv[1]);
// Poll while the worker runs. Yielding hands it the CPU; a wall-clock guard
// stops us waiting forever if something wedges.
uint64_t t0 = pit_get_ticks();
while (f->state == HTTP_STATE_RUNNING && (pit_get_ticks() - t0) < 1500) {
sched_yield();
}
if (f->state == HTTP_STATE_RUNNING) {
printk(ANSI_YELLOW "webget: still running after 15s; giving up on the "
"poll (worker will clean up)\n" ANSI_RESET);
http_fetch_release(f);
return 1;
}
if (f->state == HTTP_STATE_ERROR) {
printk(ANSI_BRIGHT_RED "webget: %s\n" ANSI_RESET, f->err);
http_fetch_release(f);
return 1;
}
char ips[16];
ip_to_str(f->ip, ips);
printk(ANSI_BRIGHT_GREEN "HTTP %d" ANSI_RESET " %d bytes %u ms from %s:%u (%s)\n",
f->status_code, f->len, f->elapsed_ms, f->host, f->port, ips);
printk("---\n%s\n---\n", f->buf);
http_fetch_release(f);
return 0;
}
// A minimal HTTP/1.0 client over the active-open path.
static int applet_httpget(int argc, char** argv) {
if (argc < 2) {
@@ -2806,6 +2859,7 @@ static const lazybox_applet_t applets[] = {
{"sockstat", applet_sockstat, "sockstat", "Socket table & UDP stats", "Network"},
{"tcpconnect", applet_tcpconnect, "tcpconnect <host> <port> [text]", "Open an outbound TCP stream", "Network"},
{"httpget", applet_httpget, "httpget <host> [path] [port]", "Fetch a URL over TCP", "Network"},
{"webget", applet_webget, "webget <url>", "Fetch a URL via the async worker", "Network"},
{"tcpserve", applet_tcpserve, "tcpserve <port>", "Accept one inbound TCP stream", "Network"},
// Storage & Devices