The Scan Pipeline

klyrek-cli's run_scan() is the one function that ties all fourteen packages together. Here's the real sequence, in order:

def run_scan(target_url, options=None, transport=None) -> ScanResult:
    scope = AuthorizationScope(authorized_hosts=[hostname, f"*.{hostname}", *options.extra_hosts])
    target = Target(base_url=target_url, hosts=[hostname])

    with KlyrekHTTPClient(scope, config=..., transport=transport) as client:
        result = Crawler(client, max_pages=..., max_depth=...).crawl(target)
        _enrich_pages(client, result, options)               # tech + headers + session detection
        _enrich_scripts(client, result, options)              # klyrek-js: secrets + endpoints
        _enrich_forms(result, options)                        # klyrek-auth: form classification
        _enrich_api(client, result, options)                  # klyrek-api: spec/root/graphql discovery
        _enrich_assets(client, result, options)                # klyrek-assets: sensitive file probing
        _enrich_discovered_endpoints(client, result, options)  # tech + headers for JS/API-discovered URLs

    return result

Stage by stage

1. Crawl (klyrek-crawler) — a breadth-first crawl within the declared scope builds the initial page graph, recording crawler (pages), crawler-script (<script src> references), and crawler-form (form actions + field names) endpoints.
2. Enrich pages — every crawler-sourced endpoint is re-fetched and passed through klyrek-tech's fingerprinting, klyrek-headers's security header checks, and klyrek-auth's session-mechanism detection.
3. Enrich scripts — every crawler-script endpoint's JS body is scanned by klyrek-js for hardcoded secrets and API-shaped endpoint literals, which get added to the result as new js-endpoint records.
4. Enrich forms — klyrek-auth's find_auth_forms() classifies every crawler-form endpoint as login/register/password-reset/logout based on URL and field-name heuristics.
5. Enrich API — klyrek-api's build_api_inventory() probes common OpenAPI/Swagger/GraphQL paths against the scan's base URL, adding any discovered routes.
6. Enrich assets — klyrek-assets probes a curated list of commonly-exposed file paths (.env, .git/config, backup.sql, ...) against the base URL.
7. Enrich discovered endpoints — anything found in steps 3, 5, or 6 (js-endpoint, api-discovery, api-spec, api-graphql) gets the same tech-fingerprint and header-check treatment as step 2. This exists specifically because a JS-only-referenced API (a fetch() call with no <a> link — the common single-page-app pattern) would otherwise never get analyzed at all, despite being a real, in-scope, reachable endpoint.

The shared result object

Every stage writes into the same ScanResult, so you can stop after any step and inspect exactly what's been found so far — result.endpoints, result.technologies, result.findings all accumulate as the pipeline runs. klyrek-report and klyrek-monitor both operate purely on this shared object, which is what lets you render a report, snapshot a scan, or diff two scans without any of them needing to know how the result was produced.