Intent-based stack

CLI stack for collect web sources.

firecrawl is a strong fit for research. firecrawl responds locally; authenticate before real work. Use this page as a permalink for humans, agents, and runbooks.

Primary recommendation

Firecrawl

48K npm weekly downloads · Agent readiness: Usable with guardrails (50/100)

Agent pack

firecrawl fits research well, especially for web extraction, crawling, and llm-ready markdown from the terminal.

48,018 npm weekly downloads.

Easy to automate.

Install

npm i -g firecrawl

Verify

firecrawl --help

Recommended stack

1. Collect web sources

Firecrawl

Verify: firecrawl --help

Pack

2. Extract PDFs

Poppler Utils

Verify: pdftotext -v

Pack

3. Search the corpus

ripgrep

Verify: rg --version

Pack

4. Write the report

Pandoc

Verify: pandoc --version

Pack

Alternatives

GNU Wget

Choose this if you care more about download files, mirror websites, and research collection from the terminal.

HTTPie

Choose this if structured output matters more.

Posting

Choose this if keeping more of the workflow local matters.

yt-dlp

Choose this if you care more about video downloads, audio extraction, and archiving from the terminal.

OCRmyPDF

Choose this if keeping more of the workflow local matters.

Poppler Utils

Choose this if keeping more of the workflow local matters.