The semantic of components is guessed

[umbobabo]

To create good semantic HTML via the Content Tree we may need to make assumption based on the component usage.
Let me try to explain with an example. This is currently the edited HTML for a Big Number:

<div className="...">
      <div className="...">{number}</div>
      <div className="...">
        {description}
      </div>
    </div>

It's was likely done before the introduction on HTML5 semantic tags. Ideally, today, should be something more like...

<aside>
  <dl>
      <dt>111</dt>
      <dd>Venenatis urna massa</dd>
  </dl>
</aside>

Semantically, this read something like This box contains an aside information for this article, it's a definition list with the term 111 and related description.
However, the Content tree doesn't have anything semantic information to pass to the frontend, in this case we can just assume that because it's a BigNumber, it will be an aside information.

Some components like Timeline could be potentially used as aside or relevant part of the reading flow, as such, make assumptions may lead to wrong semantic tags.

Should be the Content Tree Nodes semantic?

[umbobabo]

Note from last meeting with @epavlova @todor-videv1 and @lastHokage (FYI @adgad) .

@epavlova expressed the interest in having a more semantic Content Tree. Mainly, being capable of understand which nodes are critical to readability of the article and which ones are aside or related information, may help intelligent manipulation of the content tree. For instance, some machine consumers may not need aside or related content (e.g. reccommended block).

Assuming semantic based to StoryBlock e.g. Info is an aside, may potentially lead to errors. That is because we don't know how Editorial will use a particular block.

Info, for instance has 2 layouts, Full-width and half-width:

While half-width will likely to be used by Editorial for an aside information, the full-width could be essential part of the reading flow.

In this particular case, we could transition to Editorial not selecting a Presentational information like layout (Full-width or half-width) but selecting something like Section and Aside. Those more semantic choices will then need to be mapped into a layout.

From

to

Some blocks, like Recomended, may perhaps always be considered as asides information.

My suggestion is to try to audit all the StoryBlocks in Spark working with designers and see if there is a way forward that however may change the approach to in Spark, transitioning some components from a presentational language to a more semantic one.

[epavlova]

In addition to what is outlined, I can offer a few examples that highlight why it’s important to distinguish between text that forms the core semantic flow of an article and text that represents auxiliary or non-editorial information.

Incorrect content on stream/home pages on ft.com and the app

The article TikTok layoff videos pose risks for posters and employers from Dec 1 2024 is not about the topic European business schools. Reading the article, it is about TikTok, Media Industry, Work & Careers, etc. However this article appears on our European business schools stream page.

The reason is that the article contains an experimental tag (data-layout-name=card) which promotes the European Business Schools Ranking content piece:

The automated algorithm we use to extract topics from an article has no ability to distinguish between text that belongs to the core editorial narrative and text that should be treated differently. In this case, the system identifies topics such as European business schools and Business education, and these end up in the content annotations - even though they are not relevant to the article itself - unless they are manually revised.

Incorrect topics recommendations on ft.com

Reusing the same article as an example, the algorithm on ft.com for recommending topics also fails relying on the wrong about annotations.

Large volume mistakes and failing machine learning algorithms

This issue affects not only our internal data science models but also the algorithms used by consumers such as traders and analysts who consume FT content on bulk via B2B licence. These systems typically depend on large volumes of high-quality text to generate accurate insights, so in theory a small amount (as in the first example) of noise should not be problematic. In practice, however, the impact is substantial because the added promotional texts follows consistent, repeated patterns that introduce systematic errors.

Most of these additions are not related to experimental tagging, they are standard paragraphs placed at the beginning or end of articles. Machine learning models mistakenly interpret this auxiliary text as part of the editorial narrative. This has led to distortions such as Instagram and Twitter appearing among the most frequently mentioned organisations in the FT content.

For example, under the topic Gardens, we published A heady herbalist’s dream in Sansepolcro on December 1, 2024. The recurring end-of-article promotional content caused the system to tag irrelevant entities such as Instagram:

Over time, these repeated mentions create a heavily skewed view of FT coverage and reduce the accuracy of any insight-extraction algorithm built on top of our content feeds.

Reduced quality in search and recommendation algorithms

Our search systems (the ft.com hybrid search and Ask FT) as well as our recommendation algorithms rely heavily on vectorised representations of content (content embeddings). These embeddings are generated from the full text extracted from the bodyXML (content tree in the future).

When auxiliary text is inadvertently included in the embedding input, the resulting vectors no longer reflect the true semantic meaning of the article. This degrades the quality of search results, weakens relevance ranking, and reduces the accuracy of our recommendation models.

As an example, let's search for the well-known The AI Shift and one of its authors John Burn-Murdoch on ft.com search page. The results include a mixture of relevant articles and completely unrelated pieces. This happens because many content items contain recurring promotional paragraphs that mention the newsletter, even though the article itself has no connection to AI or to the author’s work. These promotional additions introduce false signals into our search index and directly undermine result relevance.
First page results are:

Those articles contain paragraphs such as:

Alternative solution: reuse the blocks with added semantic labelling

An alternative approach would be to keep the existing content tree nodes and therefore preserve the current Spark building blocks but enhance the editorial workflow by giving editors explicit control over whether a block of text should be treated as part of the core article. This could be implemented through an UI option that allows Editorial to assign a purpose to each block, such as promotional, additional information, contextual sidebar, etc.

This solution is an extension of the current content model: we won't change the blocks editors are used to have, but we would add an orthogonal layer that classifies the intent of the text. As a result, most of the content tree nodes/Spark blocks could be paired with a purpose label, allowing for combinations that preserve the editorial experience while enabling downstream systems to distinguish between core content and auxiliary material.

Depending on the outcome of the analysis we suggest, I expect that the solution will involve a combination of both: additional labelling for existing blocks and new block types for the cases where the text clearly represents a separate semantic and we need separate visual element.

[umbobabo]

to assign a purpose to each block, such as promotional, additional information, contextual sidebar, etc.

It makes senses. There will be IMO at least two level of semantics.

1. Node level
Purpose embedded in the node.
This is the case where just using a node, will tell us the semantic we need.
There will be however nodes that are merely used for composition (Building Blocks for other nodes).

If we think about the existing Info that achieved many purpose you may have an Info that contains Content related information and a Promo that contains some promo to external entity, both may relying on a shared building block Card.

Card will never be semantic, Editorial cannot act on it, it's just for composition. We will need to avoid that those components become too polymorphic (They do too much, they should stay as atomic as possible)

** Not a proposal, just an example **

interface Card extends Parent {
      type: "card"
      _meta: {
            purpose: "layout"
       }
      /** The title of this information */
      title: string
      children: [ImageSet, Paragraph]
}
/**
* An info describes a subject via a single card
*/
interface Info extends Parent {
	type: "info"
        _meta: {
            purpose: "content",            
       }
       /** The layout width supported by this node */
       layoutWidth: InfoLayoutWidth
       children: [Card]
}

/**
* A Promo promotes some external content
*/
interface Info extends Parent {
	type: "promo"
        _meta: {
            purpose: "promotional",            
       }
       children: [Card]
}

2. Variation of semantic inside a node
There are cases, like the Info box, when the semantic may be affected by the way the component is used by Editorial.

One example is the InfoBox with Half Width (Aside) or Full-width (Section).
We could potentially evaluate if we should avoid this case just splitting components, however the usage of more semantic field is doable.

From non semantic

interface Info extends Parent {
	type: "info"
       /** The layout width supported by this node */
       layoutWidth: "full-width" | "inset-left"
}

to semantic

interface Info extends Parent {
	type: "info"
       NameToBeDefined: "aside" | "section" << Coincidentally, but not intentionally, they match the HTML semantic tags
}

[adgad]

Proposal here