Blame for site/methodology.html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Methodology &amp; Mathematical Formulation - Q-TensorFormer</title>
  <meta name="description" content="Mathematical formulation of Q-TensorFormer: Lagrangian marginal-utility optimization, dual-subgradient PID control, and hysteresis switching deadbands.">
  <link rel="canonical" href="https://premchan369.github.io/Q-TensorFormer/methodology.html">
  
  <meta property="og:type" content="article">
  <meta property="og:site_name" content="Q-TensorFormer Research Project">
  <meta property="og:title" content="Methodology &amp; Mathematical Formulation - Q-TensorFormer">
  <meta property="og:description" content="Formal control-theoretic foundations for closed-loop Transformer inference.">
  <meta property="og:url" content="https://premchan369.github.io/Q-TensorFormer/methodology.html">

  <script type="application/ld+json">
  {
    "@context": "https://schema.org",
    "@type": "TechArticle",
    "headline": "Q-TensorFormer Mathematical Methodology and Optimization Framework",
    "description": "Formulation of marginal utility resource budgeting, PID subgradient updates, and chattering suppression proofs.",
    "url": "https://premchan369.github.io/Q-TensorFormer/methodology.html",
    "author": { "@type": "Person", "name": "Premchand Yadav" },
    "datePublished": "2026-10-13"
  }
  </script>
  <link rel="stylesheet" href="assets/style.css">
</head>
<body>
  <header>
    <div class="header-container">
      <div class="logo">
        <a href="index.html"><span>Q-TensorFormer</span> Research</a>
      </div>
      <nav>
        <a href="index.html">Overview</a>
        <a href="architecture.html">Architecture</a>
        <a href="methodology.html" class="active">Methodology</a>
        <a href="benchmarks.html">Benchmarks</a>
        <a href="edge-inference.html">Edge Inference</a>
        <a href="training.html">Training</a>
        <a href="documentation.html">Docs & API</a>
        <a href="research.html">Research</a>
        <a href="limitations.html">Limitations</a>
        <a href="contributing.html">Contributing</a>
      </nav>
    </div>
  </header>

  <main class="container">
    <h1>Methodology &amp; Mathematical Formulation</h1>
    <p class="lead">
      The central algorithmic challenge in adaptive neural inference is deciding when to allocate computation. Q-TensorFormer treats this as a dynamic constrained optimization problem governed by Lagrangian duality and PID control.
    </p>

    <section>
      <h2>1. The Constrained Allocation Objective</h2>
      <p>
        Let <code>a &isin; A</code> denote an allocation decision (comprising tensor rank <code>r</code>, layer depth mode, attention pathway, and KV precision). We formulate the inference objective as maximizing expected prediction quality subject to hardware budgets:
      </p>
      <div class="callout">
        <p><strong>Constrained Optimization Problem:</strong></p>
        <p><code>max_{a} &Delta;Q(a | z_t) &nbsp; s.t. &nbsp; C_k(a | z_t) &le; B_k, &nbsp; &forall; k &isin; {Latency, Memory, Energy, Bandwidth}</code></p>
      </div>
      <p>
        Forming the Lagrangian dual with shadow prices <code>&lambda; = (&lambda;_L, &lambda;_M, &lambda;_E, &lambda;_B) &ge; 0</code>:
      </p>
      <pre><code>L(a, &lambda; | z_t) = &Delta;Q(a | z_t) - sum_k &lambda;_k * [C_k(a | z_t) - B_k]</code></pre>
      <p>
        The unconstrained marginal utility of candidate action <code>a</code> is defined as:
      </p>
      <pre><code>U(a | z_t, &lambda;) = &Delta;Q_hat(a | z_t) - sum_k &lambda;_k * &Delta;C_k_hat(a | z_t)</code></pre>
    </section>

    <section>
      <h2>2. Hysteresis Switching Deadband (&tau;)</h2>
      <p>
        Without stabilization, noisy fluctuations in token entropy cause rapid switching between different matrix ranks on consecutive tokens: a phenomenon known as <strong>routing chattering</strong>. Chattering destroys hardware cache locality and incurs CPU instruction pipeline stalls.
      </p>
      <p>
        To prevent chattering, Q-TensorFormer implements a hysteresis deadband threshold <code>&tau; = 0.15</code>:
      </p>
      <pre><code>Let a* = argmax_{a} U(a | z_t, &lambda;)

Action Decision Rule:
a_t = a*       if  U(a* | z_t, &lambda;) - U(a_{t-1} | z_t, &lambda;) &ge; &tau;
a_t = a_{t-1}   otherwise (retain previous configuration)</code></pre>
      <div class="callout success">
        <p><strong>Empirical Impact:</strong> Over a sequence of 64 fluctuating tokens, introducing &tau; = 0.15 reduced rank-switching churn from <strong>25.40% down to 0.00%</strong>, completely eliminating instability without quality loss.</p>
      </div>
    </section>

    <section>
      <h2>3. Dual-Subgradient PID Controller</h2>
      <p>
        Because hardware execution conditions change dynamically (e.g. thermal throttling, OS background processes), static Lagrange multipliers fail over long horizons. Q-TensorFormer updates shadow prices using a Proportional-Integral-Derivative (PID) dual controller (<code>src/resource_allocator.py</code>):
      </p>
      <pre><code>e_k(t) = C_k_measured(t) - B_k_target
Integral term: I_k(t) = clamp(I_k(t-1) + K_i * e_k(t), -AntiWindup, +AntiWindup)
Derivative term: D_k(t) = K_d * (e_k(t) - e_k(t-1))

Multiplier Update:
&lambda;_k(t) = max(0, &lambda;_k(t-1) + K_p * e_k(t) + I_k(t) + D_k(t))</code></pre>
      <p>
        Anti-windup clamping prevents integral saturation during extended bursts of heavy inference, allowing immediate recovery when sequence demand relaxes.
      </p>
    </section>

    <section>
      <h2>4. Dynamic Information State Normalization</h2>
      <p>
        The raw 8D feature probe vector <code>z_tilde_t</code> contains signals spanning different physical units (e.g., entropy in nats, norms in Euclidean distance, latencies in milliseconds). Q-TensorFormer computes online Exponential Moving Average (EMA) statistics to normalize each dimension into a stationary <code>[0, 1]</code> range:
      </p>
      <pre><code>&mu;_t = &alpha; * &mu;_{t-1} + (1 - &alpha;) * z_tilde_t
&sigma;^2_t = &alpha; * &sigma;^2_{t-1} + (1 - &alpha;) * (z_tilde_t - &mu;_t)^2

Normalized State:
z_t = sigmoid( (z_tilde_t - &mu;_t) / sqrt(&sigma;^2_t + 1e-6) ) * m_ablate</code></pre>
    </section>
  </main>

  <footer>
    <div class="footer-links">
      <a href="index.html">Home</a>
      <a href="architecture.html">Architecture</a>
      <a href="methodology.html">Methodology</a>
      <a href="benchmarks.html">Benchmarks</a>
      <a href="documentation.html">Docs</a>
      <a href="limitations.html">Limitations</a>
      <a href="sitemap.xml">Sitemap</a>
      <a href="llms.txt">llms.txt</a>
      <a href="https://huggingface.co/Premchan369/Q-TensorFormer">Hugging Face</a>
    </div>
    <p>&copy; 2026 Premchand Yadav. Licensed under Apache-2.0.</p>
  </footer>
</body>
</html>