================================================================================
ERROR_HANDLING_AND_LOGGING_SPEC_FINAL_5_12_26.txt
CollabORhythm / Collabtunes — Engineering Hardening Phase
Generated: 5.12.26 | Black Claude — Final Engineering Hardening
PURPOSE: Every error condition, its severity, its handling, and its log format.
         Mixed Claude implements exactly this — no improvised error handling.
STATUS: ENGINEERING PREP — authoritative error spec
================================================================================

================================================================================
SECTION 1 — ERROR SEVERITY HIERARCHY
================================================================================

Five severity levels. Mixed Claude uses exactly these strings in all log calls.

SEVERITY    | MEANING                                       | PROGRAM CONTINUES?
------------|-----------------------------------------------|--------------------
DEBUG       | Internal state trace — dev/diagnostic only    | YES
INFO        | Normal operation milestone                    | YES
WARNING     | Recoverable anomaly — flagged, not fatal      | YES
ERROR       | Non-fatal failure — item skipped, logged      | YES (skip item)
CRITICAL    | Serious failure — major output affected       | YES (flag + continue)
ABORT       | Unrecoverable — program exits immediately     | NO (SystemExit)

ABORT conditions are rare and reserved for safety violations only.
Everything else continues, logs the problem, and carries on.

================================================================================
SECTION 2 — ERROR CODE REGISTRY
================================================================================

Every error has an exact error_code string. Mixed Claude uses these strings
verbatim in logger.log_error() calls. No improvised codes.

ERROR_CODE               | SEVERITY  | DESCRIPTION
-------------------------|-----------|------------------------------------------
MISSING_DEPENDENCY       | ABORT     | Required input file not found at startup
NETWORK_UNREACHABLE      | ABORT     | Cannot connect to collabtunes.com
WRITE_OUTSIDE_PATH       | ABORT     | Code attempted write outside allowed dirs
SENSITIVE_OPENED         | ABORT     | Sensitive file was read (security violation)
SITE_WRITE_DETECTED      | ABORT     | Non-GET/HEAD request to live site detected
REPO_INTEGRITY_ERROR     | ABORT     | Source file mtime changed during run
WRITE_ERROR              | CRITICAL  | Output file is 0 bytes after write
CORRUPT_ZIP              | ERROR     | zipfile.BadZipFile raised on open
ENCODING_ERROR           | ERROR     | UnicodeDecodeError reading a TXT file
MAX_PAGES_REACHED        | WARNING   | Crawl hit --max-pages limit
MAX_DEPTH_REACHED        | WARNING   | Nested ZIP recursion hit --max-zip-depth
RATE_LIMIT_HIT           | WARNING   | HTTP 429 or server throttle detected
REQUEST_TIMEOUT          | WARNING   | HTTP request timed out (after 1 retry)
MANIFEST_MISMATCH        | WARNING   | ZIP manifest list ≠ actual ZIP contents
MISSING_MANIFEST         | WARNING   | ZIP has no manifest/README member
STATUS_MISMATCH          | WARNING   | HTTP status ≠ expected from seed file
CHECKPOINT_WRITE_FAIL    | WARNING   | Checkpoint file could not be written
JS_RENDERED_PAGE         | INFO      | Page body detected as JS-rendered
NAMING_VIOLATION         | INFO      | File does not comply with naming standard
MISPLACED_FILE           | INFO      | File found in wrong folder for its category
ORPHAN_FILE              | INFO      | File has no manifest, chain, or group
ORPHAN_URL               | INFO      | Live URL found not in any nav source
CHAPTER_DRIFT_DETECTED   | INFO      | Nav label number ≠ URL slug number
URL_CONFLICT             | INFO      | Duplicate or colliding URL detected
AMBIGUOUS_VERSION        | INFO      | Two files same date, same base, different content
SENSITIVE_CATALOGED      | INFO      | Sensitive file inventoried (not opened)

================================================================================
SECTION 3 — ABORT CONDITIONS (program exits)
================================================================================

These six conditions always abort immediately. No recovery. No retry.
Each calls logger.log_abort(reason) which writes the abort entry and raises SystemExit.

ABORT 1 — MISSING_DEPENDENCY
  Trigger: A required input file (seed file, nav file, repo root) is not found
           at startup during Phase 0 pre-flight check.
  Action:
    logger.log_error("PHASE_0", "MISSING_DEPENDENCY", f"Required file not found: {filepath}")
    logger.log_abort(f"Cannot proceed without {filepath}. Provide the file and retry.")
  Message format: "ABORT: MISSING_DEPENDENCY — {filepath} not found at startup"
  Why abort: Running without required inputs produces garbage output.

ABORT 2 — NETWORK_UNREACHABLE (SGC-1 only)
  Trigger: HEAD request to https://collabtunes.com/sitemap.xml (probe URL)
           fails during Phase 0 network validation check.
  Action: log_abort("Cannot reach collabtunes.com. Check network and retry.")
  Message: "ABORT: NETWORK_UNREACHABLE — collabtunes.com not responding"
  Why abort: SGC-1 has nothing to crawl.

ABORT 3 — WRITE_OUTSIDE_PATH
  Trigger: validate_output_path() assertion fails — write attempted outside
           /outputs/ or /logs/.
  Action:
    logger.log_error("OUTPUT", "WRITE_OUTSIDE_PATH", f"Attempted: {filepath}")
    logger.log_abort("CRITICAL: Write attempted outside allowed paths. Code bug.")
  Message: "ABORT: WRITE_OUTSIDE_PATH — {filepath}"
  Why abort: This is a code bug. Cannot trust anything else this run produced.

ABORT 4 — SENSITIVE_OPENED
  Trigger: verify_sensitive_not_opened() finds any sensitive filepath in
           opened_files_log at end of sgc2 run.
  Action: log_abort("CRITICAL: Sensitive file was read. Contents may be in memory.")
  Message: "ABORT: SENSITIVE_OPENED — {filepath}"
  Why abort: Privacy/legal violation. Must alert immediately.
  NOTE: This abort fires at END of run during Phase 9, not mid-run.
        The run completes but outputs are marked COMPROMISED in their manifest.
        Then SystemExit fires. Tom reviews before using any output from this run.

ABORT 5 — SITE_WRITE_DETECTED (SGC-1 only)
  Trigger: sgc1_http_requester detects a request method other than GET or HEAD
           is about to be sent to collabtunes.com.
  Action: log_abort("CRITICAL: Non-read HTTP method detected. Code bug.")
  Message: "ABORT: SITE_WRITE_DETECTED — method={method} url={url}"
  Why abort: Would modify the live site. Must never happen.

ABORT 6 — REPO_INTEGRITY_ERROR (SGC-2 only)
  Trigger: verify_mtimes_unchanged() finds any source file mtime changed
           during the run (compared against baseline from Phase 1).
  Action: log_abort("CRITICAL: Source file was modified during run.")
  Message: "ABORT: REPO_INTEGRITY_ERROR — {filepath} mtime changed"
  Why abort: An external process (or bug) modified project files during scan.
             Output integrity cannot be guaranteed.

================================================================================
SECTION 4 — RECOVERABLE ERRORS (program continues, item skipped)
================================================================================

CORRUPT_ZIP
  Trigger: zipfile.BadZipFile exception raised when opening a ZIP.
  Action:
    Skip this ZIP entirely.
    Add to flags: {flag_type: "CORRUPT_ZIP", item: filepath, severity: "ERROR"}
    logger.log_error("PHASE_3", "CORRUPT_ZIP", f"Cannot open {filepath}")
  Output effect: ZIP appears in output with status = "CORRUPT" and member_count = 0.

ENCODING_ERROR
  Trigger: UnicodeDecodeError when reading a TXT file.
  Action:
    Skip body scan for this file. Catalog filename metadata only.
    Add to flags: {flag_type: "ENCODING_ERROR", item: filepath, severity: "ERROR"}
    logger.log_error("PHASE_4", "ENCODING_ERROR", f"Cannot read {filepath}")
  Output effect: File appears in output with txt_metadata = null.

================================================================================
SECTION 5 — WARNING CONDITIONS (logged, run continues normally)
================================================================================

MAX_PAGES_REACHED
  Trigger: crawl_queue processing reaches args.max_pages limit.
  Action:
    Stop adding to crawl_queue. Finish current item. Proceed to Phase 3.
    logger.log("PHASE_2", f"MAX_PAGES_REACHED: {max_pages} URLs crawled",
               severity="WARNING")
    Add to run_manifest: "NOTE: Crawl stopped at max-pages limit. Use --max-pages to increase."

RATE_LIMIT_HIT
  Trigger: HTTP response status 429 received.
  Action:
    Sleep 10 seconds (double the normal rate limit).
    Retry once.
    If 429 again: mark URL as RATE_LIMITED, continue to next.
    logger.log("PHASE_2", f"RATE_LIMIT_HIT on {url}", severity="WARNING")

REQUEST_TIMEOUT
  Trigger: requests.Timeout exception on HEAD or GET request.
  Action:
    Sleep 2 seconds. Retry once.
    If timeout again: mark URL as TIMEOUT_ERROR in results. Continue.
    logger.log("PHASE_2", f"REQUEST_TIMEOUT on {url}", severity="WARNING")

MANIFEST_MISMATCH
  Trigger: crossref_manifest_vs_members() finds a discrepancy.
  Action:
    Add conflict object: {conflict_type: "MANIFEST_MISMATCH", zip: filepath, ...}
    logger.log("PHASE_3", f"MANIFEST_MISMATCH in {filepath}", severity="WARNING")

MISSING_MANIFEST
  Trigger: ZIP has no member matching manifest/README pattern.
  Action:
    Add flag: {flag_type: "MISSING_MANIFEST", item: filepath, severity: "WARNING"}
    logger.log("PHASE_3", f"MISSING_MANIFEST: {filepath}", severity="WARNING")

CHECKPOINT_WRITE_FAIL
  Trigger: Exception during write_checkpoint() call.
  Action:
    logger.log_error("CHECKPOINT", "CHECKPOINT_WRITE_FAIL",
                     f"Could not write checkpoint for phase {phase}")
    Continue run without checkpoint. Note in manifest.
  Rationale: A missing checkpoint is not fatal. Run can still produce valid output.

STATUS_MISMATCH
  Trigger: HTTP status from live crawl ≠ expected_status from seed file.
  Action:
    Add conflict object with conflict_type = "STATUS_MISMATCH".
    logger.log("PHASE_2", f"STATUS_MISMATCH: {url} expected={expected} got={actual}",
               severity="WARNING")

================================================================================
SECTION 6 — LOG FORMAT SPECIFICATION
================================================================================

Every log entry is a dict with these exact keys:

{
  "timestamp": "ISO 8601 with milliseconds",   # datetime.now().isoformat()
  "step": "PHASE_0 | PHASE_1 | ... | SETUP | OUTPUT | CHECKPOINT | INTEGRITY",
  "message": "human-readable description",
  "severity": "DEBUG | INFO | WARNING | ERROR | CRITICAL | ABORT",
  "error_code": null | "ERROR_CODE_STRING"      # null if severity < ERROR
}

STEP VALUES (use exactly these strings):
  SETUP        — pre-flight, directory creation, arg parse
  PHASE_0      — pre-flight checks
  PHASE_1      — seed load / directory walk
  PHASE_2      — HTTP requests / file classification
  PHASE_3      — nav crossref / ZIP inspection
  PHASE_4      — body parsing / TXT scanning
  PHASE_5      — folder validation / routing classification
  PHASE_6      — conflict detection / dependency mapping
  PHASE_7      — deduplication
  PHASE_8      — export
  PHASE_8R     — rollback verification (SGC-1) / PHASE_9 for SGC-2
  CHECKPOINT   — checkpoint write operations
  OUTPUT       — file write operations
  INTEGRITY    — mtime checks, sensitive file verification

EXAMPLE ENTRIES:

INFO:
  {"timestamp": "2026-05-12T14:30:01.123", "step": "PHASE_0",
   "message": "Seed file loaded: 127 URLs found", "severity": "INFO",
   "error_code": null}

WARNING:
  {"timestamp": "2026-05-12T14:31:45.456", "step": "PHASE_2",
   "message": "REQUEST_TIMEOUT on https://collabtunes.com/song-list-5/",
   "severity": "WARNING", "error_code": "REQUEST_TIMEOUT"}

ABORT:
  {"timestamp": "2026-05-12T14:29:12.001", "step": "PHASE_0",
   "message": "ABORT: MISSING_DEPENDENCY — MASTER_URL_AUTHORITY_REGISTRY not found at ./data/",
   "severity": "ABORT", "error_code": "MISSING_DEPENDENCY"}

================================================================================
SECTION 7 — CONFIRMATION GATE LOG FORMAT (LIVE_RUN only)
================================================================================

Before Phase 2 begins in LIVE_RUN mode, the code prints to stdout (not log):

  ============================================================
  SGC-{N} LIVE RUN — CONFIRMATION REQUIRED
  ============================================================
  Run ID:        {run_id}
  Seed URLs:     {N} loaded
  Network:       https://collabtunes.com       [SGC-1 only]
  Root Path:     {root}                         [SGC-2 only]
  Output Dir:    {output_dir}
  Mode:          LIVE_RUN
  Max Pages:     {max_pages}                    [SGC-1 only]
  Max Zip Depth: {max_zip_depth}                [SGC-2 only]
  Estimated Time: ~{estimate} minutes
  ============================================================
  Type 'yes' to proceed or anything else to cancel: 

If input ≠ "yes" (case-insensitive):
  print("Run cancelled. No files written.")
  sys.exit(0)   # clean exit — not an abort

If input == "yes":
  logger.log("SETUP", "LIVE_RUN confirmed by operator", severity="INFO")
  Proceed to Phase 2.

This gate fires ONCE per run. It is NOT fired in DRY_RUN mode.
It is NOT fired on --resume (resume continues the original confirmed run).

================================================================================
SECTION 8 — DRY_RUN OUTPUT FORMAT
================================================================================

In DRY_RUN mode, the code writes one file to /logs/ only:

  SGC{N}_DRY_RUN_{run_id}.txt

Contents (plain text, human-readable):

  ============================================================
  DRY RUN REPORT — SGC-{N} — {run_id}
  ============================================================
  Mode:       DRY_RUN
  Date:       {timestamp}
  
  WHAT WOULD HAVE HAPPENED:
  ─────────────────────────────────────────────────────────────
  Phase 0:  Pre-flight checks
    [loaded] MASTER_URL_AUTHORITY_REGISTRY — 127 URLs
    [loaded] FINAL_NAVIGATION_AUTHORITY_MAP
    [would create] /outputs/ if absent
    [would create] /logs/ if absent
  
  Phase 1:  URL normalization + crawl queue build
    [would process] 127 URLs → crawl queue
    [would skip] 22 placeholder URLs (not in crawl scope)
    [would skip] 6 dev pages (not in crawl scope)
  
  Phase 2:  HTTP HEAD requests (WOULD MAKE LIVE NETWORK CALLS)
    [would send] 99 HEAD requests to collabtunes.com
    [rate limit] 1.5 seconds between requests
    [estimated time] ~3 minutes
  
  ... (etc. for all phases)
  
  FILES THAT WOULD BE WRITTEN TO /outputs/:
    SGC1_LIVE_SITE_SNAPSHOT_{run_id}.json          (~400 KB estimated)
    SGC1_LIVE_SITE_SNAPSHOT_{run_id}_SUMMARY.txt   (~80 KB estimated)
    SGC1_RUN_MANIFEST_{run_id}.txt                 (~5 KB estimated)
  
  FILES THAT WOULD BE WRITTEN TO /logs/:
    SGC1_CHECKPOINT_{run_id}_START.json
    SGC1_CHECKPOINT_{run_id}_PHASE2_COMPLETE.json
    SGC1_CHECKPOINT_{run_id}_PHASE4_COMPLETE.json
    SGC1_CHECKPOINT_{run_id}_COMPLETE.json
  
  TO EXECUTE: python -m sgc1.sgc1_main --mode LIVE_RUN [other args]
  ============================================================

The dry run report itself is the only output. Nothing else is written.
DRY_RUN is always safe — no network calls, no file writes beyond this report.

================================================================================
SECTION 9 — INTEGRITY VERIFICATION LOG ENTRIES
================================================================================

These three entries MUST appear in run_log for every successful LIVE_RUN.
Mixed Claude treats their absence as an implicit WARNING.

SGC-1 integrity entry:
  {"step": "PHASE_8R", "severity": "INFO",
   "message": "SITE_INTEGRITY_VERIFIED: read-only crawl confirmed. 0 non-GET/HEAD requests made."}

SGC-2 integrity entries:
  {"step": "PHASE_9", "severity": "INFO",
   "message": "REPO_INTEGRITY_VERIFIED: 0 source files modified during run."}
  {"step": "PHASE_9", "severity": "INFO",
   "message": "SENSITIVE_FILES_CLEAN: 0 sensitive files opened during run."}

If any of these entries is missing from the run_log after a LIVE_RUN,
the run should be treated as UNVERIFIED until investigated.

================================================================================
END ERROR_HANDLING_AND_LOGGING_SPEC_FINAL_5_12_26.txt
================================================================================
