docs(crawlee): warn against router: as top-level CheerioCrawler option#2735
Draft
DaveHanns wants to merge 1 commit into
Draft
docs(crawlee): warn against router: as top-level CheerioCrawler option#2735DaveHanns wants to merge 1 commit into
router: as top-level CheerioCrawler option#2735DaveHanns wants to merge 1 commit into
Conversation
Reviewing AI-generated Crawlee code, one recurring mistake is passing the
router under a `router` option (or in addition to `requestHandler`):
new CheerioCrawler({ router, requestHandler: async (ctx) => { ... } })
Crawlee has no top-level `router` option. At runtime this fails ow validation
with `ArgumentError: Did not expect property 'router' to exist ... at new
HttpCrawler`. In TypeScript projects it fails at compile time with
`TS2353: 'router' does not exist in type 'CheerioCrawlerOptions'`.
The scraping-basics-legacy "Initialization and setting up" lesson already
shows the correct pattern (`requestHandler: router`) — this commit adds a
short caution admonition right below that snippet that spells out the
anti-pattern, quotes both error messages verbatim, and calls out that this
is a common suggestion from AI chat tools and agentic coding assistants.
No code paths or examples change; the addition is a single admonition block.
Surfaced during an evaluation of Apify surfaces for agent-driven Actor development.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When reviewing AI-generated Crawlee code (from ChatGPT, Cursor, Claude Code, etc.), one very common mistake is passing the router under a
routeroption — or in addition torequestHandler:Crawlee has no top-level
routeroption. Theinitializing_and_setting_up.mdlesson inscraping_basics_legacy/challenge/already shows the correct pattern (requestHandler: router), so this PR adds a short:::cautionadmonition immediately after that code sample which:requestHandler, there is norouteroption).owerror + TS2353), so search hits from those exact strings land here.No existing code paths or examples change; the addition is a single admonition block.
Evidence
Both error messages are reproducible with any recent Crawlee release.
Runtime (
owpredicate rejection at construction):TypeScript compile-time:
Minimal reproducer:
Test plan
:::cautionblock correctly in light + dark themes.textcode blocks inside the admonition render without syntax-highlighter warnings.Surfaced during an evaluation of Apify surfaces for agent-driven Actor development.