Handle Cloudflare challenge on eprint.iacr.org PDF downloads

[rex1fernando]

Summary

eprint.iacr.org now serves .pdf paths behind a Cloudflare managed JS challenge (cf-mitigated: challenge, "Just a moment…"), which 403s urllib — and also curl and even browser-TLS-impersonating clients (tested with curl_cffi), so there is no clean programmatic bypass. Abstract pages are not challenged, so BibTeX scraping still works.

iacreprint_handler changes:

• Scrapes the BibTeX first, from the already-fetched abstract page.
• On a 403 for the PDF, falls back to opening the PDF URL in the user's browser (same pattern as dlacm_handler) and returns no PDF data.

Auto-filing the browser download: so that ck add stays hands-off beyond solving the challenge, the manual "save the PDF and press a key" step is automated. ck add now watches the browser's download directory and, when the PDF the user downloaded appears, files it under the citation key after a single confirm (default Yes). Details:

• New optional DownloadsDir config key (defaults to ~/Downloads), documented in ck-sample.config.
• Partial downloads are never picked up: browsers write to .crdownload/.part/.download and atomically rename to the final .pdf on completion.
• Saving directly to the destination path still works (watcher detects it), and the old manual prompt remains as a fallback on timeout (5 min), Ctrl-C, or declining the confirm.
• The dl.acm.org handler benefits too, since it shares this code path.

The IACR PDF integration test now monkeypatches click.launch (no browser windows during test runs) and accepts both outcomes: a direct download if Cloudflare ever unblocks, or the browser fallback.

Test plan

• 7 new unit tests for the watcher (wait_for_browser_pdf): fresh vs. old files, partial/zero-byte files, direct-to-destination saves, newest-wins, timeouts
• Full suite: 115 passed, 2 xfailed (pre-existing SIAM/IEEE)
• End-to-end against the live site: ck add https://eprint.iacr.org/2018/721 with a scratch config hit the real Cloudflare 403, opened the browser, detected a PDF dropped into a fake Downloads dir mid-wait, and filed it as TBP+18e.pdf (moved, not copied) alongside the auto-downloaded .bib

🤖 Generated with Claude Code