EngineVersion 1
crawl
The Crawl Matrix provides the semantic model for web research actions: search, site map discovery, and page scraping. Actions invoke the Platform web integratio
URI: https://poliglot.io/kleo/spec/crawl#
Imports: act, http, matrix, os, platform, svc, workspace
Classes
- crawl:Page: A scraped web page with markdown content and metadata.
- crawl:PageMetadata: Metadata extracted from a scraped web page.
- crawl:SearchResult: A single web search result returned by Brave Search.
- crawl:SiteLink: A discovered link from a site map crawl via Firecrawl.
Actions
- crawl:Map: Discover links on a website via the platform proxy (Firecrawl). Returns a list of discovered site links.
- crawl:Scrape: Scrape a web page via the platform proxy (Firecrawl). Returns the page content as markdown with metadata.
- crawl:Search: Execute a web search via the platform proxy (Brave Search). Returns a list of search results with URL, title, and descri...