| 1 | # Energy-First Training One-Pager |
| 2 | |
| 3 | ## 1) Problem |
| 4 | |
| 5 | Current training often pays high compute cost too early: |
| 6 | |
| 7 | - large raw data ingestion, |
| 8 | - weak stage separation, |
| 9 | - expensive re-learning of already seen patterns. |
| 10 | |
| 11 | Goal: keep quality, reduce energy and compute. |
| 12 | |
| 13 | ## 2) Core Hypothesis |
| 14 | |
| 15 | A staged protocol inspired by human development (listening -> imitation -> independent action) can reach comparable or better quality with lower total compute. |
| 16 | |
| 17 | ## 3) Protocol (v0.1) |
| 18 | |
| 19 | ### Phase A: Passive Incubation |
| 20 | |
| 21 | Model mostly observes and listens without strict output pressure. |
| 22 | |
| 23 | Expected effect: |
| 24 | |
| 25 | - builds broad priors, |
| 26 | - captures environment rhythm/context, |
| 27 | - lowers noise before active training. |
| 28 | |
| 29 | ### Phase B: Imitation Micro-Actions |
| 30 | |
| 31 | Short supervised attempts by pattern: |
| 32 | |
| 33 | - repeat, |
| 34 | - complete, |
| 35 | - paraphrase, |
| 36 | - transform. |
| 37 | |
| 38 | Fast and soft correction after each micro-action. |
| 39 | |
| 40 | Expected effect: |
| 41 | |
| 42 | - faster stabilization of internal representations, |
| 43 | - fewer costly mistakes in later stages. |
| 44 | |
| 45 | ### Phase C: Independent Target Tasks |
| 46 | |
| 47 | Focused task training with transfer from A+B. |
| 48 | |
| 49 | Expected effect: |
| 50 | |
| 51 | - less fine-tuning data needed, |
| 52 | - lower expensive compute per useful capability. |
| 53 | |
| 54 | ## 4) Experimental Design |
| 55 | |
| 56 | ### Groups |
| 57 | |
| 58 | - Group A (baseline): direct training on raw data. |
| 59 | - Group B: three-phase protocol (A -> B -> C). |
| 60 | - Group C: A -> C (no imitation phase). |
| 61 | |
| 62 | ### Main Metrics |
| 63 | |
| 64 | - quality per joule (primary), |
| 65 | - steps-to-threshold quality, |
| 66 | - wall-clock to threshold, |
| 67 | - robustness under context shift, |
| 68 | - transfer to unseen tasks. |
| 69 | |
| 70 | ### Success Criteria |
| 71 | |
| 72 | - Group B quality >= Group A quality, |
| 73 | - Group B uses meaningfully less compute/energy than Group A, |
| 74 | - Group B outperforms Group C, confirming the value of imitation phase. |
| 75 | |
| 76 | ## 5) Why Human Analogy Matters |
| 77 | |
| 78 | Humans learn motor and language interfaces through exposure + imitation + practice. |
| 79 | The child hears speech before speaking, then makes uncertain attempts, then forms stable language use "on the fly". |
| 80 | |
| 81 | This suggests that "pre-structuring" interaction before full autonomous tasks may be compute-optimal for model learning too. |
| 82 | |
| 83 | ## 6) Minimal v0 Execution Plan |
| 84 | |
| 85 | 1. Select one multimodal toy environment and one language-like benchmark. |
| 86 | 2. Keep architecture fixed across groups. |
| 87 | 3. Equalize total token/frame budget where possible. |
| 88 | 4. Run 3 seeds per group. |
| 89 | 5. Report not only quality, but quality-per-joule and error bars. |
| 90 | |
| 91 | ## 7) Risks |
| 92 | |
| 93 | - Phase boundaries may be set poorly. |
| 94 | - Benefits may depend on domain/task family. |
| 95 | - Short runs can hide long-term gains or losses. |
| 96 | |
| 97 | Mitigation: ablations on phase duration and curriculum shape. |
| 98 | |
| 99 | --- |
| 100 | |
| 101 | ## RU short summary |
| 102 | |
| 103 | Идея: обучать по этапам, как у людей: |
| 104 | |
| 105 | 1) сначала пассивное восприятие, |
| 106 | 2) потом короткие попытки по подражанию с мягкой коррекцией, |
| 107 | 3) и только затем целевые самостоятельные задачи. |
| 108 | |
| 109 | Проверка: сравнить с прямым обучением "в лоб" и вариантом без фазы подражания. |
| 110 | Ключевая метрика: качество на джоуль. |
| 111 | |