lib: implement callout_when so TCP timers can fire

callout_when() existed only as an empty body in ff_stub_14_extra.c, a file of
link-only stubs for FreeBSD 14 symbols whose defining sources this library does
not compile. kern_timeout.c is one of those: it is replaced by
ff_kern_timeout.c, which reimplements the callwheel but never carried
callout_when across.

That silently disabled every TCP timer. tcp_timer_activate() computes a
deadline into &tp->t_timers[which] by calling this function, then asks
tcp_timer_next() for the earliest pending one. With nothing ever written, the
entries kept their initial SBT_MAX, tcp_timer_next() reported that no timer was
pending, and the arming path fell through to callout_stop(). Every request to
start a retransmit, persist, delayed-ack or keepalive timer stopped it instead.
Because FreeBSD 14 drives all five from one callout per connection, a single
missing function disabled all of them at once.

Implement it in ff_kern_timeout.c, next to the callwheel it feeds, and drop the
stub. The stub file states that its contents are link-only, that reaching one
at runtime means an unsupported path, and that bodies must not be hand-edited
because the file is generated -- so a working implementation cannot live there.
A stub for callout_when must not be regenerated into it.

The implementation follows callout_when() in sys/kern/kern_timeout.c, less two
parts that depend on machinery this library does not build:

 - Upstream anchors a hardclock-driven callout to the last hardclock edge, read
   from per-CPU state maintained by kern_clocksource.c. That file is not
   compiled here, so there is nothing to read and sbinuptime() is used
   throughout. The deadline can therefore be up to one tick later than upstream
   would compute, and callouts armed within the same tick are not batched.
 - Upstream derives a precision floor from C_PRELGET(flags) so the scheduler
   can coalesce callouts with overlapping tolerance. This callwheel is
   tick-granular with no sub-tick slack to trade, and the only caller passes
   precision 0, so the caller's value is passed through unchanged.

A second fix is required with it, because the first one exposes it.
callout_reset_sbt_on() divided its sbintime by tick_sbt to index the
tick-based callwheel. That is correct for a duration and wrong for the absolute
deadline tcp_timer_next() passes with C_ABSOLUTE: dividing a deadline yields
uptime-in-ticks, scheduling the callout an uptime into the future, and past
roughly 24 days of uptime at hz=1000 the tick count exceeds INT_MAX and wraps
negative. ff_callout_delay_ticks() subtracts the current uptime when the
deadline is absolute, saturates SBT_MAX to INT_MAX, treats an already-past
deadline as one tick, and rounds up so a callout cannot fire early. Relative
callers keep the previous arithmetic exactly.

Impact before the fix: a connection died on its first lost segment. Nothing
retransmitted it, so snd_una never advanced, the congestion window stayed full
of unacknowledged data, tcp_output() computed len=0 indefinitely, and the send
buffer could never drain -- writes returned EAGAIN for as long as the process
lived. Small responses never exposed this, because a few hundred bytes never
put enough in flight to lose any.

Reproducer, using F-Stack's own bundled nginx:

 1. Build lib/ and app/nginx as usual.

 2. Serve a 1 MB file over plain HTTP with one worker:

      worker_processes 1;
      events { worker_connections 1024; use kqueue; }
      http {
          sendfile off;
          server { listen 80; root <directory containing a 1 MB file>; }
      }

 3. From a peer, drop a small fraction of what the server sends. Any loss
    injector works; this is the narrowest one, scoped to the test port:

      iptables -I INPUT -s <fstack-ip> -p tcp --sport 80 \
          -m statistic --mode random --probability 0.03 -j DROP

 4. curl http://<fstack-ip>/big

How to A/B it. The baseline is simply this branch without the patch, but the
two builds must be kept genuinely separate, which takes more than `make clean`:

 a. Between the two builds, force a full rebuild of the library:

      rm -f lib/*.o lib/libfstack.a lib/libfstack.ro
      make -C lib

    `make clean` is not sufficient -- it left 248 stale objects here -- and
    freebsd/sys/callout.h is not a tracked prerequisite, so changing it does
    not trigger a recompile of tcp_timer.c and friends on its own. A partial
    rebuild either fails to link on ff_callout_delay_ticks or, worse, links a
    mixture and silently measures the wrong thing.

 b. Relink nginx too, since libfstack.a is not one of its declared
    prerequisites:

      rm -f app/nginx-*/objs/nginx && make -C app/nginx-*

 c. Confirm which build you actually have before trusting a run:

      nm lib/libfstack.a | grep ff_callout_delay_ticks

    Baseline: no output, and no undefined reference to it either. Patched: one
    entry (lowercase `t` -- the library localises symbols outside its ff_api
    set after `ld -r`, which is expected and does not affect linking).

 d. Run step 3-4 above against each build, several times: with 3% loss the
    outcome is probabilistic, and a single unlucky-free run completes even on
    the baseline.

Loss is essential to the reproducer. Without it the transfer completes either
way, because no segment is ever lost and no retransmit timer is needed. With
3% loss, five consecutive attempts:

  before this patch:  one completed, then the rest hung indefinitely and were
                      killed at a 40 s timeout. The connection is not reset and
                      no error is reported -- the transfer simply stops, with
                      snd_una frozen and the send buffer unable to drain.

  after this patch:   5/5 completed, 1048576 bytes each, in 0.004 s to 0.50 s.
                      The spread is recovery: the fast ones lost nothing, the
                      slow ones lost segments and retransmitted them.

Also verified that retransmit, delayed-ack and keepalive callouts are entered
into the callwheel and fire; before the patch the callwheel stayed empty.

Environment: DPDK 24.11.6, vmxnet3 under VMware bound with uio_pci_generic, one
lcore. Testing was only possible on vmxnet3 in a virtual machine. The change is
in generic timer code rather than anything driver specific, but it has not been
exercised on a physical NIC.

Signed-off-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>

Author: lijunwangs

freebsd/sys/callout.h

@@ -93,8 +93,10 @@ void	_callout_init_lock(struct callout *, struct lock_object *, int);
 #define	callout_pending(c)	((c)->c_iflags & CALLOUT_PENDING)
 int callout_reset_tick_on(struct callout *, int, void (*)(void *),
 	void *, int, int);
+int ff_callout_delay_ticks(sbintime_t, int);
 #define callout_reset_sbt_on(c, sbt, pr, fn, args, cpu, flags) \
-    callout_reset_tick_on((c), (sbt)/tick_sbt, (fn), (args), (cpu), (flags))
+    callout_reset_tick_on((c), ff_callout_delay_ticks((sbt), (flags)),	\
+        (fn), (args), (cpu), (flags))
 #define	callout_reset_sbt(c, sbt, pr, fn, arg, flags)			\
     callout_reset_sbt_on((c), (sbt), (pr), (fn), (arg), -1, (flags))
 #define	callout_reset_sbt_curcpu(c, sbt, pr, fn, arg, flags)		\

lib/ff_kern_timeout.c

@@ -56,6 +56,8 @@ __FBSDID("$FreeBSD$");
 #include <sys/systm.h>
 #include <sys/bus.h>
 #include <sys/callout.h>
+#include <sys/limits.h>	/* INT_MAX, for the saturated tick delay */
+#include <sys/time.h>	/* tick_sbt, sbinuptime() */
 
 /*
  * F-Stack: 14.0+ removed CALLOUT_LOCAL_ALLOC and CS_EXECUTING.
@@ -326,6 +328,63 @@ callout_get_bucket(int to_ticks)
     return (to_ticks & callwheelmask);
 }
 
+/*
+ * Compute the absolute deadline a callout should fire at.
+ *
+ * Follows callout_when() in sys/kern/kern_timeout.c, without its hardclock-edge
+ * anchoring (kern_clocksource.c is not built here) or its precision floor (this
+ * callwheel is tick-granular, so there is no sub-tick slack to trade).
+ */
+void
+callout_when(sbintime_t sbt, sbintime_t precision, int flags,
+    sbintime_t *sbt_out, sbintime_t *precision_out)
+{
+    sbintime_t to_sbt;
+
+    if ((flags & (C_ABSOLUTE | C_PRECALC)) != 0) {
+        *sbt_out = sbt;
+        *precision_out = precision;
+        return;
+    }
+    /* A hardclock-based timer cannot resolve finer than one tick. */
+    if ((flags & C_HARDCLOCK) != 0 && sbt < tick_sbt)
+        sbt = tick_sbt;
+
+    to_sbt = sbinuptime();
+    /* Saturate: a wrapped deadline is negative, so it would fire at once. */
+    if (SBT_MAX - to_sbt < sbt)
+        to_sbt = SBT_MAX;
+    else
+        to_sbt += sbt;
+
+    *sbt_out = to_sbt;
+    *precision_out = precision;
+}
+
+/*
+ * Delay in ticks for this callwheel, which callout_reset_sbt_on() indexes by.
+ *
+ * An absolute deadline needs the current uptime subtracted first: dividing one
+ * by tick_sbt as though it were a duration schedules the callout an uptime into
+ * the future.
+ */
+int
+ff_callout_delay_ticks(sbintime_t sbt, int flags)
+{
+    sbintime_t now;
+
+    if ((flags & C_ABSOLUTE) == 0)
+        return ((int)(sbt / tick_sbt));
+
+    if (sbt == SBT_MAX)
+        return (INT_MAX);       /* never */
+    now = sbinuptime();
+    if (sbt <= now)
+        return (1);             /* already due */
+    /* Round up, so the callout cannot fire before its deadline. */
+    return ((int)((sbt - now) / tick_sbt) + 1);
+}
+
 void
 callout_tick(void)
 {

lib/ff_stub_14_extra.c

@@ -146,12 +146,6 @@ void buf_ring_free(struct buf_ring *br, struct malloc_type *type)
 
 }
 
-void callout_when(sbintime_t sbt, sbintime_t precision, int flags, sbintime_t *sbt_out, sbintime_t *precision_out);
-void callout_when(sbintime_t sbt, sbintime_t precision, int flags, sbintime_t *sbt_out, sbintime_t *precision_out)
-{
-    
-}
-
 vm_paddr_t dump_avail[16] = {0};
 
 void fdescfree_adapt_use(struct proc *p);