| 1 | # Epistemic methods and calibration v1 |
| 2 | |
| 3 | Тела секций epistemic/calibration/hpmor из `agent-notes.md`. L0 `epistemic-default-distrust-v1` — выше `public-cut`. Extended: `knowledge/epistemic-default-distrust-extended-v1.md`. |
| 4 | |
| 5 | --- |
| 6 | <!-- section:calibration-playbook-14d-v1 --> |
| 7 | ## Calibration Playbook 14D (под нас) v1 |
| 8 | |
| 9 | Статус: active |
| 10 | Период: 14 дней |
| 11 | Режим: lightweight, без бюрократии. |
| 12 | |
| 13 | ### Day 0: Baseline |
| 14 | 1) Зафиксировать стартовые значения: |
| 15 | - TTUC |
| 16 | - decision latency |
| 17 | - execution start lag |
| 18 | - recovery SLA |
| 19 | - rework rate |
| 20 | - agency gain (0-10) |
| 21 | |
| 22 | 2) Определить 3 типовых сценария: |
| 23 | - рабочий стрессовый, |
| 24 | - творческий/исследовательский, |
| 25 | - low-resource день. |
| 26 | |
| 27 | ### Day 1-7: Instrument only |
| 28 | - Ничего радикально не менять. |
| 29 | - Просто собирать метрики и наблюдать паттерны. |
| 30 | - Ловить top-3 повторяющихся friction points. |
| 31 | |
| 32 | Daily log (2-3 минуты): |
| 33 | - today_top_friction: |
| 34 | - action_now: |
| 35 | - TTUC: |
| 36 | - recovery_time: |
| 37 | - agency_0_10: |
| 38 | |
| 39 | ### Day 8-14: Targeted intervention |
| 40 | - Взять top-3 friction points. |
| 41 | - На каждый — 1 минимальный фикс (не big-bang). |
| 42 | - Для сложных задач обязательно representative-model-scaling. |
| 43 | - Для причин — N-Why с dual stop. |
| 44 | |
| 45 | ### Review (end of day 14) |
| 46 | 1) Что ускорилось устойчиво? |
| 47 | 2) Где был ложный прогресс? |
| 48 | 3) Какие ритуалы реально работают? |
| 49 | 4) Что закрепляем как постоянный стандарт? |
| 50 | |
| 51 | ### Exit criteria |
| 52 | - >=20% улучшение минимум по 2 core metrics ИЛИ |
| 53 | - заметное снижение recovery SLA + rework rate при сохранении safety. |
| 54 | |
| 55 | ### If no improvement |
| 56 | - Пересобрать модель (не винить исполнителя). |
| 57 | - Проверить level collapse и boundary blindness. |
| 58 | - Запустить ещё один 7-дневный микроцикл с другим primary-подмиром. |
| 59 | <!-- /section:calibration-playbook-14d-v1 --> |
| 60 | |
| 61 | <!-- section:epistemic-cognitive-failure-modes-v1 --> |
| 62 | ## Epistemic Cognitive Failure Modes v1 |
| 63 | - Failure mode: motivated reasoning. |
| 64 | - Early signal: evidence selection favors preferred outcome. |
| 65 | - Countermeasure: require strongest counter-argument before final decision. |
| 66 | - Failure mode: planning fallacy. |
| 67 | - Early signal: repeated optimistic deadlines with low variance. |
| 68 | - Countermeasure: reference-class forecasting from past similar tasks. |
| 69 | - Failure mode: confirmation bias. |
| 70 | - Early signal: searches/tests only support current hypothesis. |
| 71 | - Countermeasure: mandatory disconfirming test. |
| 72 | - Failure mode: halo effect in tool/vendor choice. |
| 73 | - Early signal: global positive impression replaces requirement fit. |
| 74 | - Countermeasure: score options by explicit constraints/SLO/cost. |
| 75 | - Failure mode: narrative overfitting. |
| 76 | - Early signal: "beautiful" explanation with weak predictive power. |
| 77 | - Countermeasure: prefer simpler model with better falsifiability. |
| 78 | - Failure mode: ambiguity denial. |
| 79 | - Early signal: binary answers for uncertain state. |
| 80 | - Countermeasure: use confidence intervals and reversible decisions where possible. |
| 81 | - Failure mode: status-quo inertia. |
| 82 | - Early signal: known-bad path persists due to migration fear. |
| 83 | - Countermeasure: bounded experiment with rollback safety. |
| 84 | <!-- /section:epistemic-cognitive-failure-modes-v1 --> |
| 85 | |
| 86 | <!-- section:epistemic-daily-checklist-v1 --> |
| 87 | ## Epistemic Daily Checklist v1 |
| 88 | - What do we believe right now, and with what confidence? |
| 89 | - What is our prior before seeing new evidence? |
| 90 | - What would change our mind today (explicit disconfirming evidence)? |
| 91 | - Did we write observations separately from interpretation? |
| 92 | - Did we test mechanism-level causality (not just narrative fit)? |
| 93 | - What is the global objective and success metric (before tactics)? |
| 94 | - Are we using proxy metrics without divergence monitoring? |
| 95 | - Are we stuck in a fake dichotomy (is there a third option)? |
| 96 | - Did we check base rates before trusting anecdotes? |
| 97 | - Which assumption is most likely wrong and most expensive if wrong? |
| 98 | - What single observation can most reduce uncertainty today? |
| 99 | - Are we calibrated (confidence aligned with recent hit rate)? |
| 100 | - Is there a reversible experiment before irreversible commitment? |
| 101 | - Did we run a pre-mortem for high-impact decision? |
| 102 | - Did we model incentives, not just stated intentions? |
| 103 | - Did we account for second-order effects? |
| 104 | - Did we red-team critical plan assumptions? |
| 105 | - Is decision traceable to evidence and constraints? |
| 106 | - Is rollback path defined if assumption breaks? |
| 107 | - Did we complete full validation chain before closure? |
| 108 | - Did KPI improvements preserve guard metrics (quality/cost/reliability)? |
| 109 | - Is process quality sound independent from outcome luck? |
| 110 | - Is this learning transferable and reproducible by another reviewer? |
| 111 | - Did we update memory with distilled learning after outcome? |
| 112 | <!-- /section:epistemic-daily-checklist-v1 --> |
| 113 | |
| 114 | <!-- section:epistemic-hpmor-heuristics-v1 --> |
| 115 | ## Epistemic HPMOR Heuristics v1 |
| 116 | - Heuristic: Belief as probability, not identity. |
| 117 | - Prevents: dogmatic lock-in and defensive reasoning. |
| 118 | - Operational check: explicitly state confidence range before decision. |
| 119 | - Revisit trigger: new evidence with high likelihood ratio appears. |
| 120 | - Heuristic: Confusion is a signal, not a failure. |
| 121 | - Prevents: fake certainty and narrative patching. |
| 122 | - Operational check: if model feels inconsistent, pause and list unknowns. |
| 123 | - Revisit trigger: mismatch between prediction and observed outcome. |
| 124 | - Heuristic: Make predictions that can fail. |
| 125 | - Prevents: unfalsifiable explanations. |
| 126 | - Operational check: attach at least one measurable expected outcome. |
| 127 | - Revisit trigger: outcome not observed in expected window. |
| 128 | - Heuristic: Avoid affect heuristic in technical decisions. |
| 129 | - Prevents: preference-driven architecture choices. |
| 130 | - Operational check: separate "I like it" from "it meets constraints". |
| 131 | - Revisit trigger: tradeoff analysis cannot be reproduced by another reviewer. |
| 132 | - Heuristic: Scope uncertainty explicitly. |
| 133 | - Prevents: hidden assumptions and accidental overreach. |
| 134 | - Operational check: mark assumptions and their confidence. |
| 135 | - Revisit trigger: assumption invalidated by runtime/production evidence. |
| 136 | - Heuristic: Distinguish map from territory. |
| 137 | - Prevents: treating documentation/model as reality. |
| 138 | - Operational check: verify with instrumentation and direct signals. |
| 139 | - Revisit trigger: docs/model disagree with live behavior. |
| 140 | - Heuristic: Update fast, not defensively. |
| 141 | - Prevents: sunk-cost continuation of bad decisions. |
| 142 | - Operational check: define in advance what evidence will trigger change. |
| 143 | - Revisit trigger: trigger condition met. |
| 144 | <!-- /section:epistemic-hpmor-heuristics-v1 --> |
| 145 | |
| 146 | <!-- section:epistemic-methodology-v1 --> |
| 147 | ## Epistemic Methodology v1 |
| 148 | - Scope: general cognition methodology (outside pure IT stack), used to improve reasoning quality in any domain. |
| 149 | - Current source anchor: HPMOR (in progress, targeted for completion as active track). |
| 150 | - Working principles: |
| 151 | - separate observation, inference, and hypothesis confidence; |
| 152 | - prefer falsifiable claims over rhetorical certainty; |
| 153 | - track update triggers: what evidence should change current belief; |
| 154 | - minimize identity lock-in to previous conclusions; |
| 155 | - optimize for truth-seeking and decision quality, not verbal elegance. |
| 156 | - Integration rule: epistemic principles shape decision process globally, while domain sections (IT/Portal/etc.) store concrete technical playbooks. |
| 157 | - HPMOR completion pipeline: |
| 158 | 1) finish remaining chapters, |
| 159 | 2) extract explicit reasoning heuristics, |
| 160 | 3) map each heuristic to decision/diagnostic behavior, |
| 161 | 4) attach source anchors to original works where referenced, |
| 162 | 5) publish concise daily checklist for applied reasoning. |
| 163 | - Output format: |
| 164 | - heuristic, |
| 165 | - failure mode it prevents, |
| 166 | - operational check, |
| 167 | - revisit trigger. |
| 168 | <!-- /section:epistemic-methodology-v1 --> |
| 169 | |
| 170 | <!-- section:epistemic-to-domain-bridge-v1 --> |
| 171 | ## Epistemic to Domain Bridge v1 (general) |
| 172 | - Scope: general methodological layer for any domain (IT, science, strategy, education, product, research, communication). |
| 173 | - Core mapping: |
| 174 | - probabilistic belief updates -> confidence-aware decisions, |
| 175 | - falsifiability -> measurable success/failure criteria, |
| 176 | - map vs territory -> prioritize direct evidence over narrative assumptions, |
| 177 | - anti-confirmation-bias -> require at least one disconfirming check, |
| 178 | - planning-fallacy control -> reference-class forecasting, |
| 179 | - ambiguity handling -> reversible decisions under uncertainty, |
| 180 | - fast Bayesian update -> rapid pivot/rollback when strong evidence appears. |
| 181 | - Operational rule: |
| 182 | - no major decision without explicit confidence statement and falsification trigger. |
| 183 | - Domain adaptation: |
| 184 | - each domain keeps its own metrics and diagnostics, but epistemic checks are shared. |
| 185 | - Memory policy: |
| 186 | - epistemic base remains global; |
| 187 | - domain sections consume it as applied playbooks. |
| 188 | <!-- /section:epistemic-to-domain-bridge-v1 --> |
| 189 | |
| 190 | <!-- section:epistemic-to-it-bridge-v1 --> |
| 191 | ## Epistemic to IT Bridge v1 |
| 192 | - Heuristic class -> IT application pattern: |
| 193 | - probabilistic belief updates -> confidence-aware architecture decisions, |
| 194 | - falsifiability -> measurable success/failure criteria in rollouts, |
| 195 | - map vs territory -> prefer instrumentation over assumption, |
| 196 | - anti-confirmation-bias -> mandatory disconfirming test before merge, |
| 197 | - planning-fallacy control -> reference-class estimates for delivery timelines, |
| 198 | - ambiguity handling -> reversible decisions under uncertainty, |
| 199 | - fast Bayesian update -> quicker rollback or strategy pivot on strong evidence. |
| 200 | - Operational rule: |
| 201 | - no major technical decision without explicit confidence statement and falsification trigger. |
| 202 | - Diagnostics tie-in: |
| 203 | - epistemic checks run alongside latency/error/saturation metrics in incident and architecture reviews. |
| 204 | <!-- /section:epistemic-to-it-bridge-v1 --> |
| 205 | |
| 206 | <!-- section:hpmor-l1-pool-v1 --> |
| 207 | ## HPMOR (L1) |
| 208 | Эпистемические эвристики из ГПиМРМ вынесены в L1. По запросу: route_context('HPMOR'|'epistemic heuristics') или knowledge/agent-notes-l1-pool.md. |
| 209 | <!-- /section:hpmor-l1-pool-v1 --> |
| 210 | |
| 211 | |
| 212 | |