Skip to content

Commit 6b1d91a

Browse files
committed
Add English translation of MTU change spec documents
Translate all 16 Chinese spec documents (00-15) under docs/mtu_change_spec/zh_cn/ to English, placed alongside zh_cn/ in docs/mtu_change_spec/. Cross-references updated to English filenames.
1 parent 1336077 commit 6b1d91a

16 files changed

Lines changed: 1920 additions & 0 deletions
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# F-Stack MTU Modification After DPDK NIC Takeover — Investigation Overview
2+
3+
> Conclusion-first document. All conclusions are cross-validated against **actual code** (`lib/` + `freebsd/`) + **runtime testing** + **external authoritative sources**; conflicts resolved in favor of code and runtime results.
4+
> Investigation target: `/data/workspace/f-stack/`, DPDK 24.11.6, runtime environment is the local DPDK-exclusive NIC (virtio, `0000:00:09.0`, IP `9.134.214.176`).
5+
6+
## One-Line Conclusion
7+
8+
**Partial support**: After f-stack's DPDK takes over the NIC, **decreasing MTU (≤1500) works**; **increasing MTU (>1500, i.e., jumbo frame) is unsupported**`ff_ioctl(SIOCSIFMTU)` returns `EINVAL(22)` directly for >1500.
9+
10+
## Per-Scenario Conclusions
11+
12+
| Scenario | Supported | Direct Evidence |
13+
|---|---|---|
14+
| **Decrease MTU** (e.g., 1500→1400) |**Supported** | Runtime test `mtu 1400` succeeded (exit=0); code-wise `ether_ioctl` writes `ifp->if_mtu` directly for values ≤ETHERMTU (`if_ethersubr.c:1181`) |
15+
| **Increase MTU to jumbo** (e.g., 9000) |**Unsupported** | Runtime test returns `ioctl SIOCSIFMTU (set mtu): Invalid argument`; code-wise `ether_ioctl` returns `EINVAL` for `ifr_mtu > ETHERMTU` (`if_ethersubr.c:1178-1179`) |
16+
| **Increase MTU to 2000** (slightly over standard frame) |**Unsupported** | Same as above, same EINVAL; upper bound locked at `ETHERMTU=1500` |
17+
18+
## Root Cause (Software/Hardware Gap)
19+
20+
1. **Protocol-stack layer hardcodes 1500 upper bound**: f-stack's `ff_veth_ioctl` (`lib/ff_veth.c:235`) delegates `SIOCSIFMTU` to FreeBSD `ether_ioctl` (L248). `ether_ioctl` (`freebsd/net/if_ethersubr.c:1174-1181`) hardcodes `EINVAL` for `ifr_mtu > ETHERMTU(1500)`. Also `ether_ifattach` (`if_ethersubr.c:985`) statically sets `if_mtu` to `ETHERMTU=1500` on NIC attach.
21+
2. **DPDK hardware layer has no MTU wiring at all**: `lib/ff_dpdk_if.c` port init/config flow has **no** `rte_eth_dev_set_mtu` call; `rxmode` has no `mtu`/`max_rx_pkt_len`/jumbo settings; mbuf pool created with `RTE_MBUF_DEFAULT_BUF_SIZE`(=2048), not enlarged by large MTU. Even bypassing the protocol-stack 1500 limit, DPDK hardware cannot send/receive jumbo frames.
22+
3. **No software/hardware linkage**: Even if `if_mtu` (software value) is changed, it is never propagated to the DPDK hardware port. For ≤1500 this is harmless (hardware default 2048 buf suffices for standard frames); for jumbo it guarantees failure.
23+
24+
## Three-Source Evidence Consistency
25+
26+
| Evidence Source | Conclusion |
27+
|---|---|
28+
| **Code** (`ff_veth.c`/`if_ethersubr.c`/`ff_dpdk_if.c`) | ether_ioctl hardcodes 1500 upper bound; DPDK has no set_mtu/jumbo/large mbuf |
29+
| **Runtime test** (`tools/sbin/ifconfig`) | 1400 succeeds; 9000/2000 both EINVAL; upper bound 1500 |
30+
| **External** (F-Stack official issues #239 / #720) | Maintainer confirms "mtu cannot exceed 1500"; jumbo support issue remains OPEN (enhancement, unimplemented) |
31+
32+
## Modification Points for Full Jumbo Support (Increasing MTU)
33+
34+
See `06-solution-and-conclusion.md` for details. Core four points (all required):
35+
1. `ff_dpdk_if.c`: port config calls `rte_eth_dev_set_mtu` + enables jumbo `rxmode` (`RTE_ETH_RX_OFFLOAD_SCATTER` or enlarged `mtu`).
36+
2. `ff_dpdk_if.c`: mbuf pool `data_room_size` enlarged by max MTU (or multi-pool/scatter receive).
37+
3. `ff_veth.c` / `ether_ifattach` path: make `if_mtu` upper bound configurable (bypass `ether_ioctl`'s ETHERMTU hard check, typically by intercepting `SIOCSIFMTU` in `ff_veth_ioctl` to handle it and propagate to DPDK).
38+
4. `config.ini`: add `mtu` config item for `ff_veth_setup_interface` to read and initialize.
39+
40+
## Document Index
41+
42+
- `01-protocol-stack-analysis.md` — SIOCSIFMTU ioctl conversion and ether_ioctl 1500 upper-bound chain
43+
- `02-dpdk-hardware-analysis.md` — port/mbuf/rxmode config gaps
44+
- `03-software-hardware-gap-analysis.md` — different consequences of decreasing vs. increasing MTU
45+
- `04-external-research.md` — F-Stack official issues, DPDK docs, tech blogs
46+
- `05-runtime-test-report.md` — runtime evidence chain (tools/sbin/ifconfig)
47+
- `06-solution-and-conclusion.md` — full jumbo modification plan + final conclusion
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Protocol-Stack Layer Analysis: SIOCSIFMTU Full Path
2+
3+
> This document analyzes f-stack's user-space FreeBSD protocol stack handling of MTU modification (`SIOCSIFMTU`/`SIOCGIFMTU`). All references are actual code `file:line`.
4+
5+
## 1. Application Layer → f-stack ioctl Conversion
6+
7+
f-stack exposes Linux-semantics `ff_ioctl`, which must convert Linux `SIOCSIFMTU`/`SIOCGIFMTU` to FreeBSD equivalents.
8+
9+
`lib/ff_syscall_wrapper.c`:
10+
```c
11+
177: #define LINUX_SIOCGIFMTU 0x8921
12+
178: #define LINUX_SIOCSIFMTU 0x8922
13+
...
14+
518: case LINUX_SIOCGIFMTU:
15+
519: return SIOCGIFMTU;
16+
520: case LINUX_SIOCSIFMTU:
17+
521: return SIOCSIFMTU;
18+
```
19+
20+
- Read MTU: `LINUX_SIOCGIFMTU(0x8921)` → FreeBSD `SIOCGIFMTU`
21+
- Set MTU: `LINUX_SIOCSIFMTU(0x8922)` → FreeBSD `SIOCSIFMTU`
22+
23+
The protocol-stack layer **does have** the conversion path to receive MTU modification requests; `tools/sbin/ifconfig` also changes MTU through this path.
24+
25+
## 2. Generic ifioctl Layer (if.c)
26+
27+
The converted `SIOCSIFMTU` enters FreeBSD's generic interface ioctl dispatch.
28+
29+
`freebsd/net/if.c`:
30+
```c
31+
2729: case SIOCSIFMTU:
32+
2733: error = priv_check(td, PRIV_NET_SETIFMTU); // permission check
33+
2736: if (ifr->ifr_mtu < IF_MINMTU || ifr->ifr_mtu > IF_MAXMTU) // broad bounds
34+
... (out-of-range EINVAL)
35+
// call driver's if_ioctl callback
36+
2755: if_notifymtu(ifp); // notify MTU change on success and change
37+
```
38+
Read MTU:
39+
```c
40+
2515: ifr->ifr_mtu = ifp->if_mtu; // SIOCGIFMTU returns software value
41+
```
42+
43+
- The `if.c` layer only does `priv_check` and **broad** bounds checking (`IF_MINMTU`~`IF_MAXMTU`, `IF_MAXMTU` up to 65535) — not the source of the 1500 limit.
44+
- After passing `priv_check`/bounds, it calls the driver's `if_ioctl` callback (i.e., `ff_veth_ioctl`).
45+
46+
## 3. Driver Layer ff_veth_ioctl (Key Delegation Point)
47+
48+
`lib/ff_veth.c`:
49+
```c
50+
234: static int
51+
235: ff_veth_ioctl(if_t ifp, u_long cmd, caddr_t data)
52+
236: {
53+
238: struct ff_veth_softc *sc = if_getsoftc(ifp);
54+
240: switch (cmd) {
55+
241: case SIOCSIFFLAGS: // only explicitly handles UP/DOWN
56+
242: if (if_getflags(ifp) & IFF_UP) { ff_veth_init(sc); }
57+
244: else if (...) ff_veth_stop(sc);
58+
246: break;
59+
247: default:
60+
248: error = ether_ioctl(ifp, cmd, data); // SIOCSIFMTU falls here
61+
249: break;
62+
250: }
63+
252: return (error);
64+
253: }
65+
```
66+
67+
Callback registration (`ff_veth_setup_interface`):
68+
```c
69+
923: if_setioctlfn(ifp, ff_veth_ioctl);
70+
```
71+
72+
- `ff_veth_ioctl` **does not handle `SIOCSIFMTU` separately**; it delegates entirely to FreeBSD's generic Ethernet handler `ether_ioctl`.
73+
- This means: **f-stack has no DPDK hardware propagation hook for MTU modification at the driver layer** (no `rte_eth_dev_set_mtu` call).
74+
75+
## 4. ether_ioctl's 1500 Hard Upper Bound (True Source of EINVAL)
76+
77+
`freebsd/net/if_ethersubr.c`:
78+
```c
79+
1174: case SIOCSIFMTU:
80+
1178: if (ifr->ifr_mtu > ETHERMTU) {
81+
1179: error = EINVAL; // >1500 rejected directly
82+
1181: } else ifp->if_mtu = ifr->ifr_mtu; // ≤1500 writes software value
83+
```
84+
85+
- `ETHERMTU = 1500`. This is the **true upper bound source for MTU modification**:
86+
- `ifr_mtu ≤ 1500`: accepted, writes `ifp->if_mtu` (software value).
87+
- `ifr_mtu > 1500`: returns `EINVAL`, `if_mtu` unchanged.
88+
89+
## 5. MTU Initial Value
90+
91+
`ether_ifattach` statically initializes MTU to 1500 on NIC attach:
92+
```c
93+
985: ifp->if_mtu = ETHERMTU;
94+
```
95+
`ff_veth_setup_interface` (`ff_veth.c:927`) calls `ether_ifattach(ifp, sc->mac)`, so the f-stack NIC defaults to MTU=1500.
96+
97+
## Summary (Protocol-Stack Layer)
98+
99+
- **Can change**: The protocol-stack layer supports modifying `if_mtu` software value via `ff_ioctl(SIOCSIFMTU)`, provided it is **≤1500**.
100+
- **Cannot increase**: `ether_ioctl` hardcodes `> ETHERMTU(1500)``EINVAL`; f-stack does not intercept/override in `ff_veth_ioctl`, so jumbo is rejected at the protocol-stack layer.
101+
- **No hardware propagation**: Regardless of success, the `if_mtu` software value is never propagated to the DPDK port via `rte_eth_dev_set_mtu` (see `02-dpdk-hardware-analysis.md`).
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# DPDK Hardware Layer Analysis: Port / mbuf / rxmode Config Gaps
2+
3+
> This document analyzes f-stack's current MTU/jumbo-related implementation at the DPDK port init/config stage. All references are actual code `file:line` (`lib/ff_dpdk_if.c`, DPDK 24.11.6).
4+
5+
## 1. No rte_eth_dev_set_mtu Call
6+
7+
Full-text search of `lib/ff_dpdk_if.c` for `mtu` / `rte_eth_dev_set_mtu` / `max_rx_pkt_len` / `jumbo` / `RTE_ETHER_MTU`: **0 matches** (only matches `rte_pktmbuf_pool_create`, `rte_eth_dev_configure`, `rxmode.offloads` and other MTU-unrelated items).
8+
9+
Conclusion: **f-stack never calls `rte_eth_dev_set_mtu`**; port MTU is always the PMD default (standard Ethernet 1500), and the protocol-stack `if_mtu` software value has **no linkage** to the hardware MTU.
10+
11+
## 2. mbuf Pool Created at Standard Frame Size
12+
13+
`lib/ff_dpdk_if.c`:
14+
```c
15+
425: if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
16+
426: snprintf(s, sizeof(s), "mbuf_pool_%d", socketid);
17+
427: pktmbuf_pool[socketid] =
18+
428: rte_pktmbuf_pool_create(s, nb_mbuf,
19+
429: MEMPOOL_CACHE_SIZE, 0,
20+
430: RTE_MBUF_DEFAULT_BUF_SIZE, socketid); // 2048B per mbuf data area
21+
431: } else {
22+
432: ... rte_mempool_lookup(s); // secondary reuses
23+
433: }
24+
```
25+
26+
- `RTE_MBUF_DEFAULT_BUF_SIZE = 2048` (DPDK definition = `RTE_PKTMBUF_HEADROOM(128) + 1920`, actual usable data area ~1920B).
27+
- Sufficient for standard Ethernet frames (1500 + 14 header + FCS/overhead), but **far insufficient for jumbo frames (9000+)**.
28+
- No `data_room_size` enlargement by large MTU, no scatter receive enabled, so a single mbuf cannot hold a jumbo frame.
29+
30+
## 3. Port rxmode Has No jumbo / max_rx_pkt_len Config
31+
32+
`lib/ff_dpdk_if.c` port config section (around L700-923), `port_conf.rxmode` settings:
33+
```c
34+
733: port_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_RSS; // RSS
35+
781-782: ... RTE_ETH_RX_OFFLOAD_VLAN_STRIP // VLAN strip
36+
787: port_conf.rxmode.offloads &= ~RTE_ETH_RX_OFFLOAD_KEEP_CRC;
37+
791-793: ... DEV_RX_OFFLOAD_TCP_LRO // LRO (conditional)
38+
799-803: ... RTE_ETH_RX_OFFLOAD_CHECKSUM // checksum
39+
807-809: ... RTE_ETH_RX_OFFLOAD_TIMESTAMP // timestamp
40+
```
41+
```c
42+
856: ret = rte_eth_dev_configure(port_id, nb_queues, nb_queues, &port_conf);
43+
...
44+
885: rxq_conf.offloads = port_conf.rxmode.offloads;
45+
886: ret = rte_eth_rx_queue_setup(port_id, q, nb_rxd, ...);
46+
```
47+
48+
- `rxmode` has **no** `mtu` field set (in DPDK 24.x `rte_eth_dev_configure` uses `port_conf.rxmode.mtu`; f-stack leaves it unset → defaults to `RTE_ETHER_MTU=1500`).
49+
- **No** jumbo-related offload (e.g., `RTE_ETH_RX_OFFLOAD_SCATTER`) for receiving large frame segments.
50+
- Therefore the DPDK port initializes with standard 1500 MTU; the PMD's jumbo send/receive capability is never enabled.
51+
52+
> Note: The local DPDK NIC is virtio; its jumbo/scatter capability is limited by the PMD. Even if f-stack code adds the above config, the PMD and underlying NIC/vSwitch must jointly support jumbo to truly enable it (see `04-external-research.md`).
53+
54+
## 4. External API Has No MTU Item
55+
56+
- `lib/ff_api.h`: search `mtu` = 0 matches, **no external MTU get/set interface**.
57+
- `config.ini`: search `mtu`/`jumbo` = 0 matches, **no MTU config item**.
58+
59+
f-stack neither reads any MTU config at init nor exposes a programming interface for upper layers to set hardware MTU.
60+
61+
## Summary (DPDK Hardware Layer)
62+
63+
| Check Item | Status | Impact |
64+
|---|---|---|
65+
| `rte_eth_dev_set_mtu` | Never called | Software `if_mtu` not propagated to hardware |
66+
| mbuf `data_room_size` | Fixed 2048 (`RTE_MBUF_DEFAULT_BUF_SIZE`) | Cannot hold >~1920B jumbo frames |
67+
| `rxmode.mtu` / jumbo offload | Unset (defaults to 1500) | Port initializes at standard frame |
68+
| scatter (segmented receive) | Not enabled | Single mbuf cannot assemble jumbo frame |
69+
| `config.ini` MTU item / `ff_api` MTU interface | None | No config/programming entry point |
70+
71+
**Conclusion**: The DPDK hardware layer has no MTU wiring or jumbo support whatsoever. Combined with the protocol-stack 1500 upper bound, this forms the "decrease MTU works, increase MTU unsupported" software/hardware gap (see `03-software-hardware-gap-analysis.md`).
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Software/Hardware Gap and Difference Analysis
2+
3+
> Synthesizing `01-protocol-stack-analysis.md` (software layer) and `02-dpdk-hardware-analysis.md` (hardware layer), this document analyzes the different consequences and the nature of the gap for "decreasing MTU" vs. "increasing MTU".
4+
5+
## 1. Nature of the Gap
6+
7+
f-stack's MTU handling has two independent stages, neither of which is wired through to hardware:
8+
9+
```
10+
Application ff_ioctl(SIOCSIFMTU)
11+
│ ff_syscall_wrapper.c:520 (LINUX_SIOCSIFMTU→SIOCSIFMTU)
12+
13+
if.c:2729 (priv_check + IF_MINMTU/IF_MAXMTU broad check)
14+
15+
16+
ff_veth_ioctl (ff_veth.c:248 default branch)
17+
18+
19+
ether_ioctl (if_ethersubr.c:1178)
20+
├── ifr_mtu ≤ 1500 → write ifp->if_mtu (software value) ✅
21+
└── ifr_mtu > 1500 → EINVAL ❌ (hard upper bound)
22+
23+
✗ Gap: no path calls rte_eth_dev_set_mtu
24+
25+
DPDK port hardware MTU stays at PMD default (1500), mbuf buf=2048
26+
```
27+
28+
**Gap point**: The `if_mtu` modified at the protocol-stack layer is a **pure software value**, never propagated to the DPDK hardware port. f-stack lacks the bridge from "software MTU change → `rte_eth_dev_set_mtu` hardware propagation".
29+
30+
## 2. Decreasing MTU (≤1500): Works
31+
32+
**Protocol-stack layer**: `ether_ioctl` accepts, writes `if_mtu` (`if_ethersubr.c:1181`). The protocol stack then uses the smaller `if_mtu` for TCP MSS negotiation and IP fragmentation decisions — i.e., outbound data is organized by the new smaller MTU.
33+
34+
**Hardware layer**: DPDK port MTU remains 1500, mbuf buf=2048. Since the new MTU (e.g., 1400) is **smaller** than hardware capacity, send/receive is unaffected:
35+
- Send: protocol stack already organizes packets ≤1400; frames are smaller; hardware sends normally.
36+
- Receive: hardware still receives ≤1500 frames; protocol stack processes per `if_mtu`.
37+
38+
**Conclusion**: Decreasing MTU **works**. The software/hardware gap is harmless in the "decrease" direction — because hardware capacity (1500/2048) is already ≥ the new software MTU. Runtime test confirmed 1400 succeeds with normal connectivity (see `05-runtime-test-report.md`).
39+
40+
> Note: Decreasing MTU only changes the local protocol stack's fragmentation/MSS behavior, not hardware; this is distinct from "physical link MTU". No side effects for the decrease scenario.
41+
42+
## 3. Increasing MTU (>1500, jumbo): Unsupported
43+
44+
**Protocol-stack layer**: `ether_ioctl` returns `EINVAL` for `ifr_mtu > ETHERMTU(1500)` (`if_ethersubr.c:1178-1179`). The request is **rejected at the protocol-stack layer**; `if_mtu` unchanged. Runtime test confirmed 9000 and 2000 both return `Invalid argument`.
45+
46+
**Even bypassing the protocol-stack upper bound** (e.g., directly changing `ether_ifattach`'s `if_mtu`, as in issue #720's hacky approach), the hardware layer still fails:
47+
- mbuf `data_room_size` is fixed 2048, cannot hold a 9000B frame (`ff_dpdk_if.c:430`).
48+
- Port `rxmode` has no jumbo/scatter config, no `rte_eth_dev_set_mtu` call; PMD initializes at 1500.
49+
- RX: >1500 frames are dropped or truncated by the PMD; TX: single mbuf cannot hold jumbo and scatter is not enabled.
50+
51+
**Conclusion**: Increasing MTU is **unsupported**, and it is a dual block of "protocol-stack upper bound + hardware gap"; both must be modified to support it (see `06-solution-and-conclusion.md`).
52+
53+
## 4. Consistency Check with Three-Layer Architecture Docs / Knowledge Graph
54+
55+
- The three-layer architecture docs (`docs/03-LAYER3-FUNCTIONS.md` etc.) describe `ff_veth_ioctl`/`ff_syscall_wrapper`'s ioctl conversion responsibilities, consistent with this investigation; the docs do not list MTU capability separately — this investigation supplements the explicit conclusion "MTU modification is software-layer only, ≤1500".
56+
- The knowledge graph (`KNOWLEDGE_GRAPH_WIKI.md`) does not record MTU/jumbo-related symbols; this investigation's conclusions can serve as incremental supplements (if needed later).
57+
- **Consistency principle**: where docs and code differ, the actual code `file:line` referenced in this investigation prevails.
58+
59+
## 5. Difference Summary Table
60+
61+
| Dimension | Decrease MTU (≤1500) | Increase MTU (>1500 jumbo) |
62+
|---|---|---|
63+
| Protocol-stack `ether_ioctl` | Accepts, writes `if_mtu` | `EINVAL` rejection |
64+
| Propagated to DPDK hardware | No (but harmless) | No (and harmful) |
65+
| mbuf 2048 can hold | Yes (new MTU smaller) | No |
66+
| PMD port default 1500 | Compatible | Incompatible |
67+
| Runtime test | Success | Failure (EINVAL) |
68+
| Overall conclusion | ✅ Supported | ❌ Unsupported |

0 commit comments

Comments
 (0)