Spaces:

firobeid
/

RoPEHeatmapMatrix

Running

App Files Files Community

firobeid commited on 15 days ago

Commit

b22e88e

verified ·

1 Parent(s): 79030e9

Upload index.html

Browse files

Files changed (1) hide show

index.html +405 -18

index.html CHANGED Viewed

@@ -1,19 +1,406 @@
-<!doctype html>
-<html>
-	<head>
-		<meta charset="utf-8" />
-		<meta name="viewport" content="width=device-width" />
-		<title>My static Space</title>
-		<link rel="stylesheet" href="style.css" />
-	</head>
-	<body>
-		<div class="card">
-			<h1>Welcome to your static Space!</h1>
-			<p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
-			<p>
-				Also don't forget to check the
-				<a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
-			</p>
-		</div>
-	</body>
 </html>

+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>RoPE — Full Rotation Matrix</title>
+<style>
+  :root {
+    --color-background-primary: #ffffff;
+    --color-background-secondary: #f5f4f0;
+    --color-background-tertiary: #eeecea;
+    --color-background-info: #e6f1fb;
+    --color-text-primary: #1a1a18;
+    --color-text-secondary: #73726c;
+    --color-text-info: #185fa5;
+    --color-border-tertiary: rgba(0,0,0,0.12);
+    --color-border-secondary: rgba(0,0,0,0.22);
+    --color-border-info: #185fa5;
+    --font-sans: "Anthropic Sans", system-ui, sans-serif;
+  }
+  @media (prefers-color-scheme: dark) {
+    :root {
+      --color-background-primary: #1e1e1c;
+      --color-background-secondary: #2a2a27;
+      --color-background-tertiary: #222220;
+      --color-background-info: #0c2a42;
+      --color-text-primary: #e8e6de;
+      --color-text-secondary: #9c9a92;
+      --color-text-info: #85b7eb;
+      --color-border-tertiary: rgba(255,255,255,0.12);
+      --color-border-secondary: rgba(255,255,255,0.22);
+      --color-border-info: #378add;
+    }
+  }
+  * { box-sizing: border-box; margin: 0; padding: 0; }
+  body { font-family: var(--font-sans); background: var(--color-background-primary); color: var(--color-text-primary); padding: 1.5rem; }
+  .pe-wrap { padding: 0.5rem 0 1rem; }
+  .pe-title { font-size: 15px; font-weight: 500; color: var(--color-text-primary); margin-bottom: 4px; }
+  .pe-sub { font-size: 12px; color: var(--color-text-secondary); margin-bottom: 14px; line-height: 1.6; }
+  .pe-axis-label { font-size: 12px; color: var(--color-text-secondary); text-align: center; }
+  .pe-y-label { writing-mode: vertical-rl; transform: rotate(180deg); font-size: 12px; color: var(--color-text-secondary); display: flex; align-items: center; justify-content: center; min-width: 20px; }
+  .pe-row { display: flex; align-items: stretch; gap: 8px; }
+  .pe-legend { display: flex; align-items: center; gap: 8px; margin-top: 8px; font-size: 11px; color: var(--color-text-secondary); }
+  .annotation-box { background: var(--color-background-secondary); border: 0.5px solid var(--color-border-tertiary); border-radius: 8px; padding: 10px 14px; margin-top: 12px; font-size: 12px; color: var(--color-text-secondary); line-height: 1.6; }
+  .ann-row { display: flex; gap: 16px; }
+  .ann-col { flex: 1; }
+  .ann-head { font-weight: 500; font-size: 12px; color: var(--color-text-primary); margin-bottom: 3px; }
+  .controls-row { display: flex; gap: 8px; flex-wrap: wrap; margin-bottom: 8px; align-items: center; }
+  .pe-btn { background: var(--color-background-secondary); border: 0.5px solid var(--color-border-secondary); border-radius: 6px; padding: 4px 11px; font-size: 12px; cursor: pointer; color: var(--color-text-primary); }
+  .pe-btn.active { background: var(--color-background-info); color: var(--color-text-info); border-color: var(--color-border-info); }
+  .tooltip { position: absolute; background: var(--color-background-primary); border: 0.5px solid var(--color-border-secondary); border-radius: 6px; padding: 6px 10px; font-size: 11px; color: var(--color-text-primary); pointer-events: none; opacity: 0; transition: opacity 0.15s; white-space: nowrap; z-index: 10; line-height: 1.6; }
+  .badge { display: inline-block; font-size: 10px; padding: 2px 6px; border-radius: 4px; background: var(--color-background-info); color: var(--color-text-info); border: 0.5px solid var(--color-border-info); margin-left: 6px; }
+  .divider { border: none; border-top: 0.5px solid var(--color-border-tertiary); margin: 8px 0; }
+  .dual-wrap { display: flex; gap: 6px; flex: 1; }
+  .matrix-panel { flex: 1; position: relative; }
+</style>
+</head>
+<body>
+<div class="pe-wrap">
+  <div class="pe-title">RoPE — Full Rotation Matrix <span class="badge">Rotary Position Encoding</span></div>
+  <div class="pe-sub">
+    RoPE rotates each dimension pair (q<sub>2i</sub>, q<sub>2i+1</sub>) by angle <strong>pos · θ<sub>i</sub></strong> using a 2×2 rotation matrix — requiring both <strong style="color:#0F6E56">cos</strong> and <strong style="color:#BA7517">sin</strong>. Toggle between three views of the same underlying data.
+  </div>
+  <div class="controls-row">
+    <span style="font-size:12px;color:var(--color-text-secondary)">View:</span>
+    <button class="pe-btn active" id="btn-view-sidebyside" onclick="setView('sidebyside')">cos | sin side by side</button>
+    <button class="pe-btn" id="btn-view-interleaved" onclick="setView('interleaved')">interleaved (as in code)</button>
+    <button class="pe-btn" id="btn-view-angle" onclick="setView('angle')">rotation angle φ = pos·θ</button>
+  </div>
+  <div class="controls-row" style="margin-bottom:10px">
+    <span style="font-size:12px;color:var(--color-text-secondary)">Hover mode:</span>
+    <button class="pe-btn active" id="btn-cell" onclick="setMode('cell')">Cell value</button>
+    <button class="pe-btn" id="btn-col" onclick="setMode('col')">Column (dim pair)</button>
+    <button class="pe-btn" id="btn-row" onclick="setMode('row')">Row (position)</button>
+  </div>
+  <!-- Side-by-side view -->
+  <div id="view-sidebyside">
+    <div style="display:flex;gap:6px;margin-left:28px;margin-bottom:3px">
+      <div style="flex:1;text-align:center;font-size:11px;font-weight:500;color:#0F6E56;background:rgba(29,158,117,0.1);border-radius:4px;padding:2px 0">cos(pos · θ<sub>i</sub>)</div>
+      <div style="flex:1;text-align:center;font-size:11px;font-weight:500;color:#854F0B;background:rgba(186,117,23,0.1);border-radius:4px;padding:2px 0">sin(pos · θ<sub>i</sub>)</div>
+    </div>
+    <div class="pe-row">
+      <div class="pe-y-label">Sequence position <em>(pos ↓)</em></div>
+      <div class="dual-wrap">
+        <div class="matrix-panel">
+          <canvas id="pe-canvas-cos" style="width:100%;display:block;cursor:crosshair;"></canvas>
+          <div class="tooltip" id="tt-cos"></div>
+        </div>
+        <div style="width:1px;background:var(--color-border-tertiary);margin:0 1px;flex-shrink:0"></div>
+        <div class="matrix-panel">
+          <canvas id="pe-canvas-sin" style="width:100%;display:block;cursor:crosshair;"></canvas>
+          <div class="tooltip" id="tt-sin"></div>
+        </div>
+      </div>
+    </div>
+    <div style="margin-left:28px">
+      <div class="pe-axis-label">Dimension pair index <em>(i →)</em></div>
+      <div style="display:flex;gap:6px;margin-top:2px">
+        <div style="flex:1;display:flex;justify-content:space-between;font-size:10px;color:var(--color-text-secondary)"><span>0 (fast)</span><span>64</span><span>127 (slow)</span></div>
+        <div style="width:3px;flex-shrink:0"></div>
+        <div style="flex:1;display:flex;justify-content:space-between;font-size:10px;color:var(--color-text-secondary)"><span>0 (fast)</span><span>64</span><span>127 (slow)</span></div>
+      </div>
+    </div>
+    <div class="pe-legend" style="margin-left:28px">
+      <span>−1</span>
+      <canvas id="legend-bar-sbs" height="12" style="flex:1;border-radius:3px;"></canvas>
+      <span>+1</span>
+    </div>
+  </div>
+  <!-- Interleaved view -->
+  <div id="view-interleaved" style="display:none">
+    <div style="margin-left:28px;margin-bottom:3px;font-size:11px;color:var(--color-text-secondary)">
+      Columns alternate <strong style="color:#0F6E56">cos</strong>, <strong style="color:#854F0B">sin</strong>, <strong style="color:#0F6E56">cos</strong>, <strong style="color:#854F0B">sin</strong>… matching how <code>cos_pos</code> and <code>sin_pos</code> are broadcast in the code via <code>repeat(..., rep=2)</code>.
+    </div>
+    <div class="pe-row">
+      <div class="pe-y-label">Sequence position <em>(pos ↓)</em></div>
+      <div style="flex:1;position:relative;">
+        <canvas id="pe-canvas-interleaved" style="width:100%;display:block;cursor:crosshair;"></canvas>
+        <div class="tooltip" id="tt-il"></div>
+      </div>
+    </div>
+    <div style="margin-left:28px">
+      <div class="pe-axis-label">Embedding dimension d (pairs of columns = one rotation)</div>
+      <div style="display:flex;justify-content:space-between;font-size:10px;color:var(--color-text-secondary);margin-top:2px"><span>d=0 cos</span><span>d=128</span><span>d=255 sin</span></div>
+    </div>
+    <div class="pe-legend" style="margin-left:28px">
+      <span>−1</span>
+      <canvas id="legend-bar-il" height="12" style="flex:1;border-radius:3px;"></canvas>
+      <span>+1</span>
+    </div>
+  </div>
+  <!-- Angle view -->
+  <div id="view-angle" style="display:none">
+    <div style="margin-left:28px;margin-bottom:3px;font-size:11px;color:var(--color-text-secondary)">
+      The raw rotation angle φ = pos · θ<sub>i</sub> in radians. Both cos and sin are derived from this single scalar. Colour = angle magnitude (dark = 0, bright = max).
+    </div>
+    <div class="pe-row">
+      <div class="pe-y-label">Sequence position <em>(pos ↓)</em></div>
+      <div style="flex:1;position:relative;">
+        <canvas id="pe-canvas-angle" style="width:100%;display:block;cursor:crosshair;"></canvas>
+        <div class="tooltip" id="tt-ang"></div>
+      </div>
+    </div>
+    <div style="margin-left:28px">
+      <div class="pe-axis-label">Dimension pair index <em>(i →)</em></div>
+      <div style="display:flex;justify-content:space-between;font-size:10px;color:var(--color-text-secondary);margin-top:2px"><span>0 (fast, large θ)</span><span>64</span><span>127 (slow, tiny θ)</span></div>
+    </div>
+    <div class="pe-legend" style="margin-left:28px">
+      <span>0 rad</span>
+      <canvas id="legend-bar-ang" height="12" style="flex:1;border-radius:3px;"></canvas>
+      <span id="ang-max-label">max rad</span>
+    </div>
+  </div>
+  <div class="annotation-box" id="ann-box"></div>
+</div>
+<script>
+const ROWS = 50, COLS = 128, D_MODEL = 256;
+const isDark = matchMedia('(prefers-color-scheme: dark)').matches;
+function theta(i) { return Math.pow(10000, -(2 * i) / D_MODEL); }
+const cosMatrix = [], sinMatrix = [], ilMatrix = [], angMatrix = [];
+const maxAngle = (ROWS - 1) * theta(0);
+for (let r = 0; r < ROWS; r++) {
+  cosMatrix[r] = []; sinMatrix[r] = []; ilMatrix[r] = []; angMatrix[r] = [];
+  for (let c = 0; c < COLS; c++) {
+    const phi = r * theta(c);
+    cosMatrix[r][c] = Math.cos(phi);
+    sinMatrix[r][c] = Math.sin(phi);
+    angMatrix[r][c] = phi;
+  }
+  for (let d = 0; d < 256; d++) {
+    const pair = Math.floor(d / 2), phi = r * theta(pair);
+    ilMatrix[r][d] = (d % 2 === 0) ? Math.cos(phi) : Math.sin(phi);
+  }
+}
+// Diverging color for trig values [-1, 1]: green for positive, purple for negative
+function trig2color(v, alpha) {
+  const t = (v + 1) / 2;
+  if (isDark) {
+    if (t > 0.5) {
+      const s = (t - 0.5) * 2;
+      return `rgba(${Math.round(20+s*10)},${Math.round(60+s*150)},${Math.round(70+s*60)},${alpha})`;
+    } else {
+      const s = (0.5 - t) * 2;
+      return `rgba(${Math.round(40+s*120)},${Math.round(20+s*20)},${Math.round(70+s*150)},${alpha})`;
+    }
+  } else {
+    if (t > 0.5) {
+      const s = (t - 0.5) * 2;
+      return `rgba(${Math.round(245-s*200)},${Math.round(252-s*70)},${Math.round(240-s*170)},${alpha})`;
+    } else {
+      const s = (0.5 - t) * 2;
+      return `rgba(${Math.round(235-s*50)},${Math.round(225-s*170)},${Math.round(252-s*30)},${alpha})`;
+    }
+  }
+}
+// Sequential amber for angle values [0, maxAngle]
+function angle2color(v, alpha) {
+  const t = Math.min(v / maxAngle, 1);
+  if (isDark) {
+    return `rgba(${Math.round(15+t*230)},${Math.round(15+t*150)},${Math.round(15+t*10)},${alpha})`;
+  } else {
+    return `rgba(${Math.round(252-t*30)},${Math.round(252-t*145)},${Math.round(252-t*235)},${alpha})`;
+  }
+}
+function drawCanvas(canvas, matrix, colorFn, cols, hc, hr) {
+  const w = canvas.offsetWidth;
+  if (!w) return;
+  const rows = matrix.length;
+  const h = Math.round(w * (rows / cols) * 1.4);
+  const dpr = window.devicePixelRatio || 1;
+  canvas.width = w * dpr; canvas.height = h * dpr;
+  canvas.style.height = h + 'px';
+  const ctx = canvas.getContext('2d');
+  ctx.scale(dpr, dpr);
+  const cw = w / cols, ch = h / rows;
+  const dim = hc >= 0 || hr >= 0;
+  for (let r = 0; r < rows; r++) {
+    for (let c = 0; c < cols; c++) {
+      const isHL = (hc >= 0 && c === hc) || (hr >= 0 && r === hr);
+      ctx.fillStyle = colorFn(matrix[r][c], dim && !isHL ? 0.28 : 1);
+      ctx.fillRect(c * cw, r * ch, cw + 0.5, ch + 0.5);
+    }
+  }
+  if (hc >= 0) { ctx.strokeStyle='#EF9F27'; ctx.lineWidth=2; ctx.strokeRect(hc*cw+1,1,cw-1,rows*ch-2); }
+  if (hr >= 0) { ctx.strokeStyle='#1D9E75'; ctx.lineWidth=2; ctx.strokeRect(1,hr*ch+1,cols*cw-2,ch-1); }
+}
+function drawLegend(id, colorFn, minV, maxV) {
+  const lb = document.getElementById(id); if (!lb) return;
+  const lctx = lb.getContext('2d');
+  const w = lb.offsetWidth || 300, dpr = window.devicePixelRatio || 1;
+  lb.width = w * dpr; lb.style.width = '100%';
+  for (let x = 0; x < w; x++) {
+    lctx.fillStyle = colorFn(minV + (x / w) * (maxV - minV), 1);
+    lctx.fillRect(x * dpr, 0, dpr, 12);
+  }
+}
+let currentView = 'sidebyside', mode = 'cell';
+let hlCol = -1, hlRow = -1;
+function redraw(hc, hr) {
+  if (hc !== undefined) { hlCol = hc; hlRow = hr; }
+  const hc_ = hlCol, hr_ = hlRow;
+  if (currentView === 'sidebyside') {
+    drawCanvas(document.getElementById('pe-canvas-cos'), cosMatrix, trig2color, COLS, hc_, hr_);
+    drawCanvas(document.getElementById('pe-canvas-sin'), sinMatrix, trig2color, COLS, hc_, hr_);
+    drawLegend('legend-bar-sbs', trig2color, -1, 1);
+  } else if (currentView === 'interleaved') {
+    const ilHc = hc_ >= 0 ? hc_ * 2 : -1;
+    drawCanvas(document.getElementById('pe-canvas-interleaved'), ilMatrix, trig2color, 256, ilHc, hr_);
+    drawLegend('legend-bar-il', trig2color, -1, 1);
+  } else {
+    drawCanvas(document.getElementById('pe-canvas-angle'), angMatrix, angle2color, COLS, hc_, hr_);
+    drawLegend('legend-bar-ang', angle2color, 0, maxAngle);
+    document.getElementById('ang-max-label').textContent = maxAngle.toFixed(1) + ' rad';
+  }
+}
+function clearHL() { hlCol = -1; hlRow = -1; redraw(); }
+function setView(v) {
+  currentView = v;
+  ['sidebyside','interleaved','angle'].forEach(k => {
+    document.getElementById('view-'+k).style.display = k===v ? '' : 'none';
+    document.getElementById('btn-view-'+k).classList.toggle('active', k===v);
+  });
+  clearHL(); updateAnnotation();
+}
+function setMode(m) {
+  mode = m;
+  ['cell','col','row'].forEach(k => document.getElementById('btn-'+k).classList.toggle('active', k===m));
+  clearHL();
+}
+function attachHover(canvasId, ttId, colsOverride, getInfo) {
+  const canvas = document.getElementById(canvasId);
+  const tt = document.getElementById(ttId);
+  canvas.addEventListener('mousemove', function(e) {
+    const rect = canvas.getBoundingClientRect();
+    const x = e.clientX - rect.left, y = e.clientY - rect.top;
+    const cols = colsOverride || COLS;
+    const col = Math.floor(x / (rect.width / cols));
+    const row = Math.floor(y / (rect.height / ROWS));
+    if (col < 0 || col >= cols || row < 0 || row >= ROWS) return;
+    const info = getInfo(row, col);
+    if (mode === 'cell')      { redraw(-1, -1); tt.innerHTML = info.cell; }
+    else if (mode === 'col')  { redraw(info.pairCol, -1); tt.innerHTML = info.colTip; }
+    else                      { redraw(-1, row); tt.innerHTML = info.rowTip; }
+    tt.style.left = Math.min(x+10, rect.width-250)+'px';
+    tt.style.top = Math.max(y-44,0)+'px';
+    tt.style.opacity = '1';
+  });
+  canvas.addEventListener('mouseleave', () => { tt.style.opacity='0'; clearHL(); });
+}
+function pairInfo(row, col) {
+  const phi = (row * theta(col)).toFixed(4);
+  const cv = Math.cos(row*theta(col)).toFixed(4);
+  const sv = Math.sin(row*theta(col)).toFixed(4);
+  const th = theta(col).toExponential(3);
+  const speed = col < 10 ? 'fast' : col < 60 ? 'moderate' : 'slow';
+  return { phi, cv, sv, th, speed };
+}
+attachHover('pe-canvas-cos', 'tt-cos', null, (row, col) => {
+  const { phi, cv, sv, th, speed } = pairInfo(row, col);
+  return {
+    pairCol: col,
+    cell:   `<b>pos=${row}, pair i=${col}</b><br>cos(φ) = ${cv}<br>sin(φ) = ${sv}<br>φ = ${phi} rad`,
+    colTip: `<b>Pair i=${col}</b> (${speed}) · θ = ${th}<br>cos col highlighted left, sin col right`,
+    rowTip: `<b>pos=${row}</b> · full rotation vector (all pairs)`
+  };
+});
+attachHover('pe-canvas-sin', 'tt-sin', null, (row, col) => {
+  const { phi, cv, sv, th, speed } = pairInfo(row, col);
+  return {
+    pairCol: col,
+    cell:   `<b>pos=${row}, pair i=${col}</b><br>sin(φ) = ${sv}<br>cos(φ) = ${cv}<br>φ = ${phi} rad`,
+    colTip: `<b>Pair i=${col}</b> (${speed}) · θ = ${th}`,
+    rowTip: `<b>pos=${row}</b> · full rotation vector (all pairs)`
+  };
+});
+attachHover('pe-canvas-interleaved', 'tt-il', 256, (row, d) => {
+  const pair = Math.floor(d / 2), isCos = d % 2 === 0;
+  const { phi, cv, sv, th, speed } = pairInfo(row, pair);
+  return {
+    pairCol: d,
+    cell:   `<b>pos=${row}, d=${d}</b> (pair ${pair}, ${isCos?'cos':'sin'})<br>value = ${isCos?cv:sv}<br>φ = ${phi} rad`,
+    colTip: `<b>d=${d}</b> · pair ${pair} · ${isCos?'<b style="color:#1D9E75">cos</b>':'<b style="color:#BA7517">sin</b>'} channel`,
+    rowTip: `<b>pos=${row}</b> · interleaved rotation vector (256 dims)`
+  };
+});
+attachHover('pe-canvas-angle', 'tt-ang', null, (row, col) => {
+  const { phi, cv, sv, th, speed } = pairInfo(row, col);
+  return {
+    pairCol: col,
+    cell:   `<b>pos=${row}, pair i=${col}</b><br>φ = ${phi} rad<br>→ cos = ${cv}, sin = ${sv}`,
+    colTip: `<b>Pair i=${col}</b> (${speed}) · θ = ${th}<br>angle grows by θ per position step`,
+    rowTip: `<b>pos=${row}</b> · angle = pos × θ<sub>i</sub> for each pair`
+  };
+});
+function updateAnnotation() {
+  const ann = document.getElementById('ann-box');
+  if (currentView === 'sidebyside') {
+    ann.innerHTML = `<div class="ann-row">
+      <div class="ann-col">
+        <div class="ann-head" style="color:#0F6E56">cos matrix (left) — the "stay" component</div>
+        cos(pos·θ<sub>i</sub>) scales how much of the original vector survives. At φ=0 it is 1 (identity). At φ=π it is −1 (full flip). Fast pairs (left columns) cycle rapidly; slow pairs (right) stay near 1 across all 50 positions.
+      </div>
+      <div class="ann-col">
+        <div class="ann-head" style="color:#854F0B">sin matrix (right) — the "spin" component</div>
+        sin(pos·θ<sub>i</sub>) scales the 90°-rotated copy <code>qw2 = [−q<sub>odd</sub>, q<sub>even</sub>]</code>. At φ=0 it is 0 (no spin); at φ=π/2 it is 1 (maximum rotation contribution). Together cos² + sin² = 1, preserving vector length exactly.
+      </div>
+    </div>
+    <hr class="divider">
+    <div style="font-size:11px"><b>Full rotation formula:</b> q<sub>new</sub> = q · cos_pos + [−q<sub>odd</sub>, q<sub>even</sub>] · sin_pos</div>`;
+  } else if (currentView === 'interleaved') {
+    ann.innerHTML = `<div class="ann-row">
+      <div class="ann-col">
+        <div class="ann-head">How the code broadcasts</div>
+        <code>cos_pos = repeat(sinusoidal[..., 1::2], rep=2)</code> duplicates each cos value twice, so both dims of a pair share the same scalar.<br>
+        <code>sin_pos = repeat(sinusoidal[..., ::2], rep=2)</code> does the same for sin.<br><br>
+        Each consecutive pair of columns (d, d+1) gets the same rotation angle — only the trig function differs.
+      </div>
+      <div class="ann-col">
+        <div class="ann-head">What to notice</div>
+        Adjacent columns are always identical in value (the repeat). Visually this makes each pair look like a 2-pixel-wide stripe. Fast pairs (left) stripe rapidly across positions; slow pairs (right) are nearly uniform. The pattern <em>is</em> the side-by-side view — just interleaved instead of separated.
+      </div>
+    </div>`;
+  } else {
+    ann.innerHTML = `<div class="ann-row">
+      <div class="ann-col">
+        <div class="ann-head">The underlying clock: φ = pos · θ<sub>i</sub></div>
+        Both cos and sin are simply read off the unit circle at this angle. The left column (pair 0) reaches ${maxAngle.toFixed(1)} rad by pos=49 — many full revolutions. The right column barely ticks above 0. This triangular ramp is the speed spectrum that gives RoPE its multi-scale position sensitivity.
+      </div>
+      <div class="ann-col">
+        <div class="ann-head">Why the gradient is triangular</div>
+        φ = pos × θ<sub>i</sub>, where θ<sub>i</sub> = 10000<sup>−2i/d</sup> decreases geometrically rightward and pos increases linearly downward. The brightest cell is bottom-left (pos=49, pair 0). Moving right, the brightness drops exponentially; moving up, it drops linearly. This is the "clock" RoPE winds at different speeds per pair.
+      </div>
+    </div>`;
+  }
+}
+setTimeout(() => { redraw(); updateAnnotation(); }, 60);
+window.addEventListener('resize', () => redraw());
+</script>
+</body>
 </html>