`blockchain.transaction.broadcast` returns txid but transaction never reaches the network — no mempool to serve `getdata`

[steepdawn974]

Summary

blockchain.transaction.broadcast accepts a transaction, validates it, returns a txid to the Electrum client, and announces it to P2P peers via inv. But when peers request the transaction data via getdata, libbitcoin-server cannot provide it — the tx is not in the block database (it's unconfirmed) and there is no mempool to store it in. Instead of responding with a notfound message, libbitcoin disconnects the peer. The transaction never reaches any node on the network.

Environment

• libbitcoin-server: v4.0.0 (commit 7179371)
• libbitcoin-node: v4.0.0 (same commit)
• Network: Bitcoin mainnet, fully synced at block ~960,139
• P2P: Active outbound connections, inbound accepted on 8333
• Electrum protocol: 1.4.2
• Client: Sparrow Wallet 2.5.2

Reproduction

1. Start bs with a fully synced mainnet store and active P2P connections
2. Connect via Electrum protocol (port 50001 or 50002)
3. Submit a valid signed transaction via blockchain.transaction.broadcast
4. Observe that a txid is returned (success)
5. Check any public mempool explorer (e.g. mempool.space) — the transaction does not appear
6. The Sparrow client reports "Timeout searching for broadcasted transaction"

Root cause — source code trace

Step 1: Electrum broadcast → validation + BROADCAST

libbitcoin-server/src/protocols/electrum/protocol_electrum_transactions.cpp:

code protocol_electrum::broadcast_tx(const chain::transaction::cptr& tx) NOEXCEPT
{
    if (const auto ec = validate_tx(*tx))
        return ec;

    BROADCAST(peer::transaction, to_shared<peer::transaction>(tx));
    return {};
}

The tx is validated and BROADCAST is called. The handler returns success and the Electrum layer sends the txid to the client. The tx is held alive only by the shared_ptr during the broadcast — once the handler returns, the pointer is destroyed. There is no mempool to store it in.

Step 2: P2P inv announcement

libbitcoin-node/src/protocols/protocol_transaction_out_106.cpp:

bool protocol_transaction_out_106::handle_broadcast_transaction(const code& ec,
    const transaction::cptr& message, uint64_t sender) NOEXCEPT
{
    if (stopped(ec))
        return false;

    if (sender == identifier())
        return true;

    return announce(message->transaction_ptr->hash(false));
}

announce() sends an inv message with type_id::transaction to the peer. This is correct.

Step 3: Peer sends getdata — libbitcoin cannot serve it

void protocol_transaction_out_106::send_transaction(const code& ec,
    size_t index, const get_data::cptr& message) NOEXCEPT
{
    // ...
    const auto& query = archive();
    const auto ptr = query.get_transaction(query.to_tx(item.hash), witness);
    if (!ptr)
    {
        LOGR("Requested tx " << encode_hash(item.hash)
            << " from [" << opposite() << "] not found.");
        stop(system::error::not_found);  // disconnects the peer
        return;
    }
    // ...
}

archive() is the block database — it only contains confirmed transactions. The unconfirmed tx is not there, so get_transaction() returns null. Instead of sending a notfound message (per Bitcoin P2P protocol), libbitcoin calls stop(system::error::not_found), which disconnects the peer.

Step 4: Tx data is gone

The to_shared<peer::transaction>(tx) was a temporary shared_ptr. After the BROADCAST handler completes, the reference count drops to zero and the tx data is freed. By the time getdata arrives from the peer (a network round-trip later), the data no longer exists in memory.

The full failure sequence

1. Client sends blockchain.transaction.broadcast → libbitcoin validates, BROADCASTs, returns txid
2. inv sent to all P2P peers
3. Peer sends getdata requesting the tx
4. libbitcoin looks up tx in archive() (block database) → not found
5. libbitcoin disconnects the peer (stop(system::error::not_found))
6. Tx data was a temporary shared_ptr, already destroyed
7. No other node ever receives the transaction
8. Client polls for the tx → never appears → timeout

The txid returned to the Electrum client is meaningless — the transaction never reaches any mempool on the network.

Additional: inbound transaction fetching is a stub

libbitcoin-node/src/protocols/protocol_transaction_in_106.cpp:

bool protocol_transaction_in_106::handle_receive_inventory(const code& ec,
    const inventory::cptr&) NOEXCEPT
{
    BC_ASSERT(stranded());

    if (stopped(ec))
        return false;

    // bip144: get_data uses witness type_id but inv does not.

    // TODO: get and handle tx as required.
    ////const auto tx_count = message->count(type_id::transaction);
    ////set_announced(hash);
    return true;
}

The node receives inv messages from peers but never sends getdata to fetch the actual transactions. The logic is commented out as a TODO. Even if a mempool existed, it would never be populated from the network.

Additional: mempool.get_fee_histogram is hardcoded empty

libbitcoin-server/src/protocols/electrum/protocol_electrum_mempool.cpp:

// TODO: could be simulated with block fees.
send_result(array_t{}, 42);

Always returns [] regardless of configuration.

Impact

• Transaction broadcast is broken: blockchain.transaction.broadcast returns a txid but the tx never reaches the network
• Peer connections degraded: every peer that requests the announced tx gets disconnected, potentially reducing the node's connectivity over time
• Electrum wallet clients incompatible: any client relying on transaction broadcast (Sparrow, Electrum, etc.) cannot use libbitcoin-server as their backend

Suggested minimal fix

1. Store broadcasted transactions in a local in-memory mempool (txid → tx data, with expiry)
2. Check the mempool in send_transaction() before falling back to archive()
3. Send a notfound message instead of disconnecting the peer when a tx is not available

Related

• #817 (comment) — Original investigation into estimatefee returning -1; maintainer confirmed "There is no mempool in libbitcoin-server v4"
• sparrowwallet/sparrow#2023 — Sparrow "Timeout searching for broadcasted transaction" (client-side symptom of this issue)

[evoskuil]

Store broadcasted transactions in a local in-memory mempool

This is not going to happen in v4 (tx pool is the central item that is specifically out of scope), and won't happen in this manner in v5 (libbitcoin won't ever store a pool of txs in RAM - no "mempool"). However the behavior will be comparable - just not the implementation.

Send a notfound message instead of disconnecting the peer when a tx is not available

Possibly, I think the disconnect was based on the presumption that the peer would have had to receive an announcement in order to be legitimately request the tx. But this should probably be relaxed even in v4.

This is a legitimate v4 issue, where we are hitting up against the design scope in this case where we are blocks-only but do relay as long as the tx can validate against the confirmed chain. The broadcast can propagate the actual tx in this case (owned by a shared ptr) after it's been validated - never actually storing it. However it won't be the case that a peer query will be able to obtain the tx from the node. The relay will work, but the node is blocks only.