refactor(markdown-magic): Replatform onto remark/mdast

[Joshua Smithrud (Josmithr)]

This pattern unlocks a lot more flexibility in how we do generation / content copying. New functionality included in this PR:

• Support for .mdx files (like we use in the website)
• Automatic heading level detection when creating contents that include headings.

Future improvements that will be possible:

• Automatic file path link re-mapping (so you can safely embed contents that include links).

Also updates the code to use TypeScript.

[github-actions]

Hi! Thank you for opening this PR. Want me to review it?

Based on the diff (21305 lines, 263 files), I've queued these reviewers:

• Correctness — logic errors, race conditions, lifecycle issues
• Security — vulnerabilities, secret exposure, injection
• API Compatibility — breaking changes, release tags, type design
• Performance — algorithmic regressions, memory leaks
• Testing — coverage gaps, hollow tests

How this works

• Adjust the reviewer set by ticking/unticking boxes above. Reviewer toggles alone don't trigger anything.

• Tick Start review below to dispatch the review fleet.

• After review finishes, tick Start review again to request another run — it auto-resets after each dispatch.

• This comment updates as new commits land; your reviewer selections are preserved.

• Start review

[Joshua Smithrud (Josmithr)]

TODO: remove this before we merge the PR

[Joshua Smithrud (Josmithr)]

Note for reviewers: the syntax for our transforms has changed. A key benefit to the new syntax is that it supports JSON for the options.

[Joshua Smithrud (Josmithr)]

Note for reviewers: the Markdown formatting used by ReMark is different from what we were previously hard-coding. There should be no end-reader-visible changes made here, but the PR does include a lot of formatting changes. There wasn't a practical way to prevent this :/

[Joshua Smithrud (Josmithr)]

Note for reviewers: there wasn't a great way to implement this package scripts functionality in the new model without needing to declare all of the relevant scripts up-front. That said, the existing behavior was VERY error-prone, and the new behavior is much more robust. In particular, if a referenced script is removed from the package.json, the docs build will error if that script is still mentioned here, which is quite useful.

But all-in-all, I'm on the fence about this. It's strictly opt-in, so READMEs don't need to do this. But I also question how valuable this content really is.

I'm interested in others' opinions :)

[Joshua Smithrud (Josmithr)]

Note for reviewers: I guess the changes here were substantial enough that git doesn't see this as a rename. But this file wasn't removed. It's just now a .mdx file.

[Alex Villarreal (alexvy86)]

A few comments below. Otherwise overall lgtm. I did not look in detail at all the src/test files, and I only skimmed the docs/transforms/ files, but the idea looks correct and the implementation seems to be working as intended. Since stakes are relatively low, I think less nitpicky reviews and quick iteration is where this code wants to be, so approving anyway.

[Alex Villarreal (alexvy86)]

If we're making markdown-magic an ESM module, can we bump the chalk version now?

[Joshua Smithrud (Josmithr)]

Good catch! I think so!

[Joshua Smithrud (Josmithr)]

It looks like at least one other package in the client workspace is using the older version too. I'll do this update as a separate PR.

[Alex Villarreal (alexvy86)]

[Agent-generated]

Summary of a deep review of commit 954cce44497ac96e7a131f18c69b60d63dbb244a, scoped to markdown-magic, its tests/configuration, and representative generated docs and MDX integration.

Recommendation: address these three reproduced correctness issues before merging. These are generator edge cases, not failures observed in the checked-in website.

1. Included generation markers break subsequent runs (processing.ts:119-120). Including a source containing a valid help region succeeds, but writes nested markers into the destination. The next run throws Generated regions must not nest. Validate generated-region structure in the complete proposed output before writing, preserving the original file on failure.

2. Reference definitions can leak beyond the selected line range (include.ts:268-275). When start: 0, end: 2 selects an opening code fence and its first content line, appended reference definitions are absorbed into the open code block and emitted, even though they are outside the requested range. Keep reference-definition context separate from constructs in the selected text and add a regression for this case.

3. Duplicate reference definitions resolve to the wrong URL (include.ts:250-254). A source containing [x][r] followed by definitions for https://first.example and https://second.example generates a link to the second URL. new Map(...) makes the last definition win, contrary to CommonMark's first-definition-wins rule. Retain the first definition for each normalized identifier and cover duplicate labels in tests.

The remaining repository-wide README churn and lockfile contents were not reviewed line by line.

[github-actions]

Bundle size comparison

Base commit: 2918565f70f4e3b8477ee2f14055c44325670869
Head commit: bb80e9181fe967eadab71c3df1a9f19daf9c8693

Pending — Build - client packages is running. Results will appear here when the build completes.

[github-actions]

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output

$ start-server-and-test "npm run serve -- --host 127.0.0.1 --no-open" http://127.0.0.1:3000 check-links
1: starting server using command "npm run serve -- --host 127.0.0.1 --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-website@0.0.0 serve
> docusaurus serve --host 127.0.0.1 --no-open

[SUCCESS] Serving "build" directory at: http://127.0.0.1:3000/

> fluid-framework-website@0.0.0 check-links
> linkcheck http://127.0.0.1:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  351523 links
    2064 destination URLs
    2323 URLs ignored
       0 warnings
       0 errors