# 13 Euler-Maclaurinの和公式

黒木玄

2018-07-04～2019-04-03

* Copyright 2018 Gen Kuroki
* License: MIT https://opensource.org/licenses/MIT
* Repository: https://github.com/genkuroki/Calculus

このファイルは次の場所できれいに閲覧できる:

* http://nbviewer.jupyter.org/github/genkuroki/Calculus/blob/master/13%20Euler-Maclaurin%20summation%20formula.ipynb

* https://genkuroki.github.io/documents/Calculus/13%20Euler-Maclaurin%20summation%20formula.pdf

このファイルは Julia Box で利用できる.

自分のパソコンにJulia言語をインストールしたい場合には

* [WindowsへのJulia言語のインストール](http://nbviewer.jupyter.org/gist/genkuroki/81de23edcae631a995e19a2ecf946a4f)

* [Julia v1.1.0 の Windows 8.1 へのインストール](https://nbviewer.jupyter.org/github/genkuroki/msfd28/blob/master/install.ipynb)

を参照せよ. 前者は古く, 後者の方が新しい.

論理的に完璧な説明をするつもりはない. 細部のいい加減な部分は自分で訂正・修正せよ.

$
\newcommand\eps{\varepsilon}
\newcommand\ds{\displaystyle}
\newcommand\Z{{\mathbb Z}}
\newcommand\R{{\mathbb R}}
\newcommand\C{{\mathbb C}}
\newcommand\QED{\text{□}}
\newcommand\root{\sqrt}
\newcommand\bra{\langle}
\newcommand\ket{\rangle}
\newcommand\d{\partial}
\newcommand\sech{\operatorname{sech}}
\newcommand\cosec{\operatorname{cosec}}
\newcommand\sign{\operatorname{sign}}
\newcommand\real{\operatorname{Re}}
\newcommand\imag{\operatorname{Im}}
$

目次

1  Bernoulli多項式

1.1  Bernoulli多項式の定義

1.2  Bernoulli多項式の基本性質

1.3  べき乗和

1.4  Bernoulli数の計算法

1.5  周期的Bernoulli多項式のFourier級数展開

2  Euler-Maclaurinの和公式

2.1  Euler-Maclaurinの和公式の導出

2.2  Euler-Maclaurinの和公式の2つの解釈

2.3  Euler-Maclaurinの和公式の形式的導出

2.4  べき乗和(再)

2.5  Stirlingの近似公式

2.6  Poissonの和公式とEuler-Maclaurinの和公式の関係

2.7  台形公式とPoissonの和公式の関係

3  ゼータ函数への応用

3.1  解析接続

3.2  ζ(2)の近似計算

3.3  s = 1でのζ(s)の定数項がEuler定数になること

3.4  負の整数におけるゼータ函数の特殊値の計算

3.5  発散級数の有限部分と ζ(-r) の関係

```julia
using Base.MathConstants
using Base64
using Printf
using Statistics
const e = ℯ
endof(a) = lastindex(a)
linspace(start, stop, length) = range(start, stop, length=length)

using Plots
gr(); ENV["PLOTS_TEST"] = "true"
#clibrary(:colorcet)
clibrary(:misc)

function pngplot(P...; kwargs...)
    sleep(0.1)
    pngfile = tempname() * ".png"
    savefig(plot(P...; kwargs...), pngfile)
    showimg("image/png", pngfile)
end
pngplot(; kwargs...) = pngplot(plot!(; kwargs...))

showimg(mime, fn) = open(fn) do f
    base64 = base64encode(f)
    display("text/html", """""")
end

using SymPy
#sympy.init_printing(order="lex") # default
#sympy.init_printing(order="rev-lex")

using SpecialFunctions
using QuadGK
```

## Bernoulli多項式

### Bernoulli多項式の定義

**定義(Bernoulli多項式):** Bernoulli多項式** $B_n(x)$ ($n=0,1,2,\ldots$)を

$$
\frac{ze^{zx}}{e^z-1} = \sum_{n=0}^\infty \frac{B_n(x)}{n!}z^n
$$

によって定義する. $\QED$

### Bernoulli多項式の基本性質

**一般化Bernoulli多項式の基本性質:** Bernoulli多項式 $B_n(x)$ は以下の性質を満たしている:

(1) $B_0(x)=1$.

(2) $\ds\int_0^1 B_n(x)\,dx = \delta_{n,0}$.

(3) $\ds B_n(x+h) = \sum_{k=0}^n\binom{n}{k}B_{n-k}(x)h^k =
\sum_{k=0}^n \binom{n}{k} B_k(x) h^{n-k}$.

(4) $B_n'(x)=nB_{n-1}(x)$.

(5) $\ds B_n(x+1)=B_n(x)+nx^{n-1}$.

(6) $B_n(1-x)=(-1)^n B_n(x)$.

(7) $B_n(1)=B_n(0)+\delta_{n,1}$ となる.

(8) $B_n(0)=1$, $\ds B_n(0)=-\frac{1}{2}$ とな, $n$ が3以上の奇数ならば $B_n(0)=0$ となる.

**証明:** (1) $e^{zx}=1+O(z)$, $\ds\frac{e^z-1}{z}=1+O(z)$ より, $\ds\frac{ze^{zx}}{e^z-1}=1+O(z)$ なので $B_0(x) = 1$.

(2)を示そう.

$$
\begin{aligned}
&
\int_0^1 \frac{ze^{zx}}{e^z-1}\,dx = \frac{z}{e^z-1}\int_0^1 e^{zx}\,dx =
\frac{z}{e^z-1}\frac{e^z-1}{z} = 1,
\\ &
\int_0^1\frac{ze^{zx}}{e^z-1}\,dx = \sum_{n=0}^\infty\frac{z^n}{n!}\int_0^1 B_n(x)\,dx
\end{aligned}
$$

なので, これらを比較して $\ds\int_0^1 B_n(x)\,dx = \delta_{n,0}$.

(3) 二項定理より,

$$
\int_0^1 (x+y)^n\,dy =
\sum_{k=0}^n \binom{n}{k} x^{n-k} \int_0^1 y^k\,dy.
$$

ゆえに, $x$ の函数を $x$ の函数に移す線形写像(前方移動平均)

$$
f(x)\mapsto \int_0^1 f(x+y)\,dy
$$

は多項式を多項式に移し, 最高次の係数が1の多項式を最高次の係数が1の同次の多項式に移す. これより, 線形写像 $\ds f(x)\mapsto \int_0^1 f(x+y)\,dy$ は多項式どうしの一対一対応を与える線形写像になっていることがわかる. そして,

$$
\begin{aligned}
&
\int_0^1\frac{ze^{z(x+y)}}{e^z-1}\,dx =
\sum_{n=0}^\infty\frac{\int_0^1 B_n(x+y)\,dy}{n!}z^n,
\\ &
\int_0^1\frac{ze^{z(x+y)}}{e^z-1}\,dx =
\frac{ze^{zx}}{e^z-1}\int_0^1 e^{zy}\,dy =
\frac{ze^{zx}}{e^z-1}\frac{e^z-1}{z} =
e^{zx} =
\sum_{n=0}^\infty \frac{x^n}{n!}z^n
\end{aligned}
$$

なので, これらを比較して,

$$
\int_0^1 B_n(x+y)\,dy = x^n
$$

が成立することがわかる. ゆえに,

$$
\int_0^1 B_n(x+h+y)\,dy = (x+h)^n = \sum_{k=0}^n \binom{n}{k}x^{n-k}h^k =
\int_0^1 \sum_{k=0}^n \binom{n}{k}B_{n-k}(x+y)h^k \,dy
$$

より

$$
B_n(x+h) = \sum_{k=0}^n \binom{n}{k}B_{n-k}(x)h^k.
$$

(4) すぐ上の等式の右辺の $h$ の係数を見ることによって,

$$
B_n'(x) = n B_{n-1}(x).
$$

(5) Bernoulli多項式の母函数の $x$ に $x+1$ を代入すると,

$$
\frac{ze^{z(x+1)}}{e^z-1} = \frac{ze^z e^{zx}}{e^z-1} =
\frac{z(1+(e^z-1))e^{zx}}{e^z-1} = \frac{ze^{zx}}{e^z-1} + ze^{zx}
$$

なので両辺を $z$ について展開して比較すれば(5)が得られる.

(6) Bernoulli多項式の母函数の $x$ に $1-x$ を代入すると,

$$
\frac{ze^{z(1-x)}}{e^z-1} = \frac{ze^z e^{-zx}}{e^z-1} =
\frac{ze^{-zx}}{1-e^{-z}} = \frac{-ze^{-zx}}{e^{-z}-1}
$$

とBernoulli多項式の母函数の $z$ に $-z$ を代入したものになるので, 両辺を $z$ について展開して比較すれば(5)が得られる.

(7) 上の(2)と(4)より, $n$ が2以上のとき,

$$
B_n(1)-B_n(0) = \int_0^1 B_n'(x)\,dx = n\int_0^1 B_{n-1}(x)\,dx = n\delta_{n-1,0} = \delta_{n,1}
$$

ゆえに $n$ が2以上のとき $B_n(1)=B_n(0)+\delta_{n,1}$.

(8) 次の函数が $z$ の偶函数で $z\to 0$ で $1$ になることから, (6)が得られる:

$$
\frac{z}{e^z-1} + \frac{z}{2} = \frac{z}{2}\frac{e^{z/2}+e^{-z/2}}{e^{z/2}-e^{-z/2}}.
\qquad \QED
$$

**注意:** $B_n=B_n(0)$ は**Bernoulli数**と呼ばれている. (3)で $(x,h)$ を $(0,x)$ で置き換えると, Bernoulli多項式がBernoulli数で表わされることがわかる:

$$
B_n(x) = \sum_{k=0}^n \binom{n}{k}B_k x^{n-k}.
$$

上の定理の条件(1),(2),(4)によってBernoulli多項式 $B_n(x)$ が $n$ について帰納的に一意的に決まる. $\QED$

**例:**
$$
B_0 = 1, \quad B_1 = -\frac{1}{2}, \quad
B_2 = \frac{1}{6}, \quad B_3=0, \quad B_4 = -\frac{1}{30}
$$

なので

$$
\begin{aligned}
&
B_0(x)=1, \quad
B_1(x)=x-\frac{1}{2}, \quad
B_2(x)=x^2-x+\frac{1}{6},
\\ &
B_3(x)=x^3-\frac{3}{2}x^2+\frac{1}{2}x, \quad
B_4(x)=x^4-2x^3+x^2-\frac{1}{30}.
\qquad\QED
\end{aligned}
$$

```julia
BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
x = symbols("x", real=true)
[BernoulliPolynomial(n,x) for n in 0:10]
```

\[ \left[ \begin{array}{r}1\\x - \frac{1}{2}\\x^{2} - x + \frac{1}{6}\\x^{3} - \frac{3 x^{2}}{2} + \frac{x}{2}\\x^{4} - 2 x^{3} + x^{2} - \frac{1}{30}\\x^{5} - \frac{5 x^{4}}{2} + \frac{5 x^{3}}{3} - \frac{x}{6}\\x^{6} - 3 x^{5} + \frac{5 x^{4}}{2} - \frac{x^{2}}{2} + \frac{1}{42}\\x^{7} - \frac{7 x^{6}}{2} + \frac{7 x^{5}}{2} - \frac{7 x^{3}}{6} + \frac{x}{6}\\x^{8} - 4 x^{7} + \frac{14 x^{6}}{3} - \frac{7 x^{4}}{3} + \frac{2 x^{2}}{3} - \frac{1}{30}\\x^{9} - \frac{9 x^{8}}{2} + 6 x^{7} - \frac{21 x^{5}}{5} + 2 x^{3} - \frac{3 x}{10}\\x^{10} - 5 x^{9} + \frac{15 x^{8}}{2} - 7 x^{6} + 5 x^{4} - \frac{3 x^{2}}{2} + \frac{5}{66}\end{array} \right] \]

```julia
# (2) ∫_0^1 B_n(x) dx = δ_{n0}

BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
x = symbols("x", real=true)
[integrate(BernoulliPolynomial(n,x), (x,0,1)) for n = 0:10]'
```

\[\left[ \begin{array}{rrrrrrrrrrr}1&0&0&0&0&0&0&0&0&0&0\end{array}\right]\]

```julia
# (3) B_n(x+h) = Σ_{k=0}^n binom(n,k) B_{n-k}(x) h^k

BernoulliNumber(n) = sympy.bernoulli(n)
BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
BinomCoeff(n,k) = sympy.binomial_coefficients_list(n)[k+1]
x, h = symbols("x h", real=true)
[BernoulliPolynomial(n,x) == sum(k->BinomCoeff(n,k)*BernoulliNumber(k)*x^(n-k), 0:n) for n in 0:10]'
```

    1×11 LinearAlgebra.Adjoint{Bool,Array{Bool,1}}:
     true  true  true  true  true  true  true  true  true  true  true

```julia
# (4) B_n'(x) = n B_{n-1}(x)

BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
x = symbols("x", real=true)
[diff(BernoulliPolynomial(n,x), x) == n*BernoulliPolynomial(n-1,x) for n = 1:10]'
```

    1×10 LinearAlgebra.Adjoint{Bool,Array{Bool,1}}:
     true  true  true  true  true  true  true  true  true  true

```julia
# (5) B_n(x+1) = B_n(x) + n x^{n-1}

BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
x = symbols("x", real=true)
[simplify(BernoulliPolynomial(n,x+1) - BernoulliPolynomial(n,x)) for n in 0:10]
```

\[ \left[ \begin{array}{r}0\\1\\2 x\\3 x^{2}\\4 x^{3}\\5 x^{4}\\6 x^{5}\\7 x^{6}\\8 x^{7}\\9 x^{8}\\10 x^{9}\end{array} \right] \]

```julia
# (6) B_n(1-x) = (-1)^n B_n(x)

BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
x = symbols("x", real=true)
[expand(BernoulliPolynomial(n,1-x)) == (-1)^n*BernoulliPolynomial(n,x) for n in 0:10]'
```

    1×11 LinearAlgebra.Adjoint{Bool,Array{Bool,1}}:
     true  true  true  true  true  true  true  true  true  true  true

```julia
# (7) B_n(1) = B_n(0) + δ_{n1}

BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
x = symbols("x", real=true)
[expand(BernoulliPolynomial(n,1)) - BernoulliPolynomial(n,0) for n in 0:10]'
```

\[\left[ \begin{array}{rrrrrrrrrrr}0&1&0&0&0&0&0&0&0&0&0\end{array}\right]\]

```julia
# (8) B_n = B_n(0) は n が3以上の奇数ならば0になる.

BernoulliNumber(n) = sympy.bernoulli(n)
[(n, BernoulliNumber(n)) for n in 0:10]
```

    11-element Array{Tuple{Int64,Sym},1}:
     (0, 1)
     (1, -1/2)
     (2, 1/6)
     (3, 0)
     (4, -1/30)
     (5, 0)
     (6, 1/42)
     (7, 0)
     (8, -1/30)
     (9, 0)
     (10, 5/66)

### べき乗和

$m$ は正の整数であるする. Bernoulli多項式について,

$$
B_{m+1}(x+1)-B_{m+1}(x) = (m+1)x^m,
\quad\text{i.e.}\quad
x^m = \frac{B_{m+1}(x+1)-B_{m+1}(x)}{m+1}
$$

が成立しているので, これを $x=0,1,\ldots,n$ について足し上げると,

$$
\sum_{j=1}^n j^m = \frac{B_{m+1}(n+1)-B_{m+1}}{m+1}.
\qquad \QED
$$

```julia
PowerSum(m, n) = sum(j->j^m, 1:n)
BernoulliNumber(n) = sympy.bernoulli(n)
BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)
PowerSumFormula(m, n) = (BernoulliPolynomial(m+1,n+1)-BernoulliNumber(m+1))/(m+1)
[(m, PowerSum(m,10), PowerSumFormula(m, 10)) for m in 1:10]
```

    10-element Array{Tuple{Int64,Int64,Sym},1}:
     (1, 55, 55)
     (2, 385, 385)
     (3, 3025, 3025)
     (4, 25333, 25333)
     (5, 220825, 220825)
     (6, 1978405, 1978405)
     (7, 18080425, 18080425)
     (8, 167731333, 167731333)
     (9, 1574304985, 1574304985)
     (10, 14914341925, 14914341925)

### Bernoulli数の計算法

Bernoulli数 $B_n$ は

$$\displaystyle
\frac{z}{e^z-1}=\sum_{n=1}^\infty B_n\frac{z^n}{n!}
$$

で定義される. しかし, この展開を直接計算することによって Bernoulli 数を求めるのは効率が悪い.

まず, 左辺の $z\to 0$ の極限を取ることによって $B_0=1$ であることはすぐにわかる.

次に, $n$ が $3$ 以上の奇数のとき $B_n=0$ となることを(再び)示そう.

$$\displaystyle
\frac{z}{e^z-1} + \frac z2
=\frac z2\frac{e^z+1}{e^z-1}
=\frac z2\frac{e^{z/2}+e^{-z/2}}{e^{z/2}-e^{-z/2}}
$$

より, 左辺は偶函数になるので, その展開の奇数次の項は消える. このことから, $B_1=-1/2$ でかつ, $0=B_3=B_5=B_7=\cdots$ であることもわかる.

$$\displaystyle
\frac{ze^z}{e^z-1}
=\sum_{j,k=0}^\infty \frac{z^j}{j!}\frac{B_k z^k}{k!}
=\sum_{n=0}^\infty\left(\sum_{k=0}^n \binom{n}{k} B_k\right)\frac{z^n}{n!}
$$

でかつ

$$\displaystyle
\frac{ze^z}{e^z-1}
=\frac{z}{e^z-1}+z
=\sum_{n=0}^\infty(B_n+\delta_{n1})\frac{z^n}{n!}
$$

なので, これらを比較すると

$$\displaystyle
\sum_{k=0}^{n-1} \binom{n}{k} B_k = \delta_{n1}.
$$

ゆえに, $n$ を $n+1$ で置き換え, $n\geqq 1$ とし, $B_n$ を他で表わす式に書き直すと

$$\displaystyle
B_n = -\frac{1}{n+1}\sum_{k=0}^{n-1}\binom{n+1}{k}B_k
\qquad (n\geqq 1).
$$

これを使えば帰納的に $B_n$ を求めることができる. $B_0=1$, $B_1=-1/2$, $0=B_3=B_5=B_7=\cdots$ であることを使うと,

$$\displaystyle
B_{2m} = -\frac{1}{2m+1}\left(
1 -\frac{2m+1}{2}
+\sum_{k=1}^{m-1}\binom{2m+1}{2k}B_{2k}
\right).
$$

**問題:** 上の方ではSymPyにおけるBernoulli数の函数を利用した. Bernoulli数を計算するためのプログラムを自分で書け. $\QED$

**解答例:** 次のセルの通り. $\QED$

```julia
# binomial coefficient: binom(n,k) = n(n-1)・(n-k+1)/k!
#
mydiv(a, b) = a / b
mydiv(a::Integer, b::Integer) = a ÷ b
function binom(n, k)
    k < 0 && return zero(n)
    k == 0 && return one(n)
    b = one(n)
    for j in 1:k
        b = mydiv(b*(n-k+j), j)
    end
    b
end

@show binom(Rational(big"100")/3, 30)

# Bernoulli numbers: B(n) = Bernoulli[n+1] = B_n
#
struct Bernoulli{T}
    B::Array{T,1}
end
function Bernoulli(; maxn=200)
    B = zeros(Rational{BigInt},maxn+1)
    B[1] = 1      # B_0
    B[2] = -1//2  # B_1
    for n in big"2":2:maxn+1
        B[n+1] = -(1//(n+1))*sum(j->binom(n+1,j)*B[j+1], 0:n-1)
        # B_n = -(1/(n+1)) Σ_{j=0}^{n-1} binom(n+1,j)*B_j
    end
    Bernoulli(B)
end
(B::Bernoulli)(n) = B.B[n+1]

maxn = 200
@time B = Bernoulli(maxn=maxn) # B_n を B_{maxn} まで計算
BB(n) = float(B(n)) # B(n) = B_n である.  BB(n)はその浮動小数点版

# SymPyのBernoulli数と比較して正しく計算できているかどうかを確認
#
BernoulliNumber(n) = sympy.bernoulli(n)
@show B_eq_B = [B(n) == BernoulliNumber(n) for n in 0:maxn]
println()
@show all(B_eq_B)

maxnprint = 30
println()
for n in [0; 1; 2:2:maxnprint]
    println("B($n) = ", B(n))
end
println()
for n in [0; 1; 2:2:maxnprint]
    println("BB($n) = ", BB(n))
end
```

    binom(Rational(#= In[11]:15 =# @big_str("100")) / 3, 30) = 11240781188817808072725280//984770902183611232881
      1.913094 seconds (17.53 M allocations: 309.111 MiB, 19.93% gc time)
    B_eq_B = [B(n) == BernoulliNumber(n) for n = 0:maxn] = Bool[true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true]

    all(B_eq_B) = true

    B(0) = 1//1
    B(1) = -1//2
    B(2) = 1//6
    B(4) = -1//30
    B(6) = 1//42
    B(8) = -1//30
    B(10) = 5//66
    B(12) = -691//2730
    B(14) = 7//6
    B(16) = -3617//510
    B(18) = 43867//798
    B(20) = -174611//330
    B(22) = 854513//138
    B(24) = -236364091//2730
    B(26) = 8553103//6
    B(28) = -23749461029//870
    B(30) = 8615841276005//14322

    BB(0) = 1.0
    BB(1) = -0.50
    BB(2) = 0.1666666666666666666666666666666666666666666666666666666666666666666666666666674
    BB(4) = -0.03333333333333333333333333333333333333333333333333333333333333333333333333333359
    BB(6) = 0.02380952380952380952380952380952380952380952380952380952380952380952380952380947
    BB(8) = -0.03333333333333333333333333333333333333333333333333333333333333333333333333333359
    BB(10) = 0.0757575757575757575757575757575757575757575757575757575757575757575757575757578
    BB(12) = -0.2531135531135531135531135531135531135531135531135531135531135531135531135531131
    BB(14) = 1.166666666666666666666666666666666666666666666666666666666666666666666666666661
    BB(16) = -7.092156862745098039215686274509803921568627450980392156862745098039215686274513
    BB(18) = 54.97117794486215538847117794486215538847117794486215538847117794486215538847111
    BB(20) = -529.1242424242424242424242424242424242424242424242424242424242424242424242424247
    BB(22) = 6192.123188405797101449275362318840579710144927536231884057971014492753623188388
    BB(24) = -86580.25311355311355311355311355311355311355311355311355311355311355311355311313
    BB(26) = 1.425517166666666666666666666666666666666666666666666666666666666666666666666661e+06
    BB(28) = -2.729823106781609195402298850574712643678160919540229885057471264367816091954032e+07
    BB(30) = 6.015808739006423683843038681748359167714006423683843038681748359167714006423638e+08

### 周期的Bernoulli多項式のFourier級数展開

$\widetilde{B}_k(x) = B_k(x-\lfloor x\rfloor)$ を**周期的Bernoulli多項式**と呼ぶことにする. 周期的Bernoulli多項式は $\widetilde{B}_k(x+1)=\widetilde{B}_k(x)$ を満たしている.

周期的Bernoulli多項式の母函数 $\ds\frac{z e^{z(x-\lfloor x\rfloor)}}{e^z-1}$ の $x$ の函数としての Fourier係数 $a_n(z)$ は次のように求まる:

$$
\frac{e^z-1}{z}a_n(z) = \int_0^1 e^{zx}e^{-2\pi inx}\,dx =
\left[\frac{e^{(z-2\pi in)x}}{z-2\pi in}\right]_{x=0}^{x=1} =
\frac{e^z-1}{z-2\pi in},
\qquad
a_n(z) = \frac{z}{z-2\pi in}.
$$

ゆえに $a_0(z)=1$ であり, $n\ne 0$ のとき

$$
a_n(z) = -\sum_{k=1}^\infty \frac{z^k}{(2\pi in)^k}
$$

これより, $\widetilde{B}_k(x)$ のFourier係数 $a_{k,n}$ は, $a_{0,n}=\delta_{n,0}$, $a_{k,0}=\delta_{k,0}$ を満たし, $k\ne 0$, $n\geqq 1$ のとき

$$
a_{k,n} = -\frac{k!}{(2\pi in)^k}
$$

となることがわかる. したがって, Fourier級数論より, $k=1$ のときは整数ではない実数 $x$ について, $k\geqq 2$ の場合にはすべての実数 $x$ について次が成立することがわかる:

$$
\widetilde{B}_k(x) = B_k(x-\lfloor x\rfloor) =
-k!\sum_{n\ne 0} \frac{e^{2\pi inx}}{(2\pi in)^k}.
$$

すなわち, $k=1,2,3,\ldots$ について

$$
\widetilde{B}_{2k-1}(x) =
(-1)^k 2(2k-1)!\sum_{n=1}^\infty\frac{\sin(2\pi nx)}{(2\pi n)^{2k-1}},
\qquad
\widetilde{B}_{2k}(x) =
(-1)^{k-1} 2(2k)!\sum_{n=1}^\infty \frac{\cos(2\pi nx)}{(2\pi n)^{2k}}.
$$

このことから, $k$ が大きいとき(実際には $k=5,6$ 程度ですでに), 周期的Bernoulli多項式は $n=1$ の項だけで

$$
\widetilde{B}_{2k-1}(x) \approx
(-1)^k 2(2k-1)!\frac{\sin(2\pi x)}{(2\pi)^{2k-1}},
\qquad
\widetilde{B}_{2k}(x) \approx
(-1)^{k-1} 2(2k)!\frac{\cos(2\pi x)}{(2\pi)^{2k}}
$$

と近似できることがわかる. 適当にスケールすれば周期的Bernoulli多項式は $k\to\infty$ で三角函数に収束する.

```julia
BBB = Bernoulli(Float64.(B.B)) # Float64 Bernoulli numbers
BP(k,x) = sum(j->binom(k,j)*BBB(k-j)*x^j, 0:k) # Float64 Bernoulli polynomial
PBP(k,x) = BP(k, x - floor(x)) # periodic Bernoulli polynomial

# partial sum of Fourier series of periodic Bernoulli polynomial
function PSFS(k, N, x)
    k == 0 && return zero(x)
    if isodd(k)
        return (-1)^((k+1)÷2)*2*factorial(k)*sum(n->sin(2π*n*x)/(2π*n)^k, 1:N)
    else
        return (-1)^(k÷2-1)*2*factorial(k)*sum(n->cos(2π*n*x)/(2π*n)^k, 1:N)
    end
end

PP = []
x = -1.0:0.001:0.999
for (k,N) in [(1,20), (2,10), (3,3), (4,2), (5,1), (6,1)]
    y = PBP.(k,x)
    z = PSFS.(k, N, x)
    ymin = 1.2*minimum(y)
    ymax = 2.7*maximum(y)
    P = plot(legend=:topleft, size=(400, 250), ylim=(ymin, ymax))
    plot!(x, y, label="B_$k(x-[x])")
    plot!(x, z, label="partial sum of Fourier series (N=$N)")
    push!(PP, P)
end

plot(PP[1:2]..., size=(750, 280))
```

```julia
plot(PP[3:4]..., size=(750, 280))
```

```julia
plot(PP[5:6]..., size=(750, 280))
```

## Euler-Maclaurinの和公式

### Euler-Maclaurinの和公式の導出

Bernoulli多項式 $B_n(x)$ とBernoulli数 $B_n$ について

$$
\begin{aligned}
&
B_0(x) = 1, \quad \frac{d}{dx}\frac{B_n(x)}{n!} = \frac{B_{n-1}(x)}{(n-1)!},
\\ &
B_1(0)=-\frac{1}{2}, \quad B_1(1)=\frac{1}{2},
\\ &
B_n(1)=B_n(0)=B_n \quad (n=0,2,3,4,5,\ldots)
\\ &
B_{2j+1} = 0 \quad (j=1,2,3,\ldots)
\end{aligned}
$$

が成立している. 以下ではしばらくのあいだこれらの条件しか使わない.

部分積分を繰り返すことによって,

$$
\begin{aligned}
\int_0^1 f(x)\,dx &= \int_0^1 B_0(x)f(x)\,dx
\\ &=
[B_1(x)f(x)]_0^1 - \int_0^1 B_1(x)f'(x)\,dx
\\ &=
[B_1(x)f(x)]_0^1 - \frac{1}{2}[B_2(x)f'(x)]_0^1 + \int_0^1 \frac{B_2(x)}{2}f''(x)\,dx
\\ &=
[B_1(x)f(x)]_0^1 - \frac{1}{2}[B_2(x)f'(x)]_0^1 + \frac{1}{3!}[B_3(x)f''(x)]_0^1 - \int_0^1 \frac{B_3(x)}{3!}f'''(x)\,dx
\\ &=
\cdots\cdots\cdots\cdots\cdots
\\ &=
\sum_{k=1}^n \frac{(-1)^{k-1}}{k!}\left[B_k(x)f^{(k-1)}(x)\right]_0^1 +
(-1)^n\int_0^1 \frac{B_n(x)}{n!}f^{(n)}(x)\,dx
\\ &=
\frac{f(0)+f(1)}{2} + \sum_{k=2}^n(-1)^{k-1}\frac{B_k}{k!} (f^{(k-1)}(1)-f^{(k-1)}(0)) +
(-1)^n\int_0^1 \frac{B_n(x)}{n!}f^{(n)}(x)\,dx.
\end{aligned}
$$

実数 $x$ に対して, $x$ 以下の最大の整数を $\lfloor x\rfloor$ と書く. このとき, $x-\lfloor x\rfloor$ は $x$ の「小数部分」になる. このように記号を準備しておくと, 整数 $j$ に対して,

$$
\begin{aligned}
\int_j^{j+1} f(x)\,dx &= \int_0^1 f(x+j)\,dx
\\ &=
\frac{f(j)+f(j+1)}{2} + \sum_{k=2}^n (-1)^{k-1} \frac{B_k}{k!} (f^{(k-1)}(j+1)-f^{(k-1)}(j)) +
(-1)^n\int_0^1 \frac{B_n(x)}{n!}f^{(n)}(x+j)\,dx
\\ &=
\frac{f(j)+f(j+1)}{2} + \sum_{k=2}^n (-1)^{k-1}\frac{B_k}{k!} (f^{(k-1)}(j+1)-f^{(k-1)}(j)) +
(-1)^n\int_j^{j+1} \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx.
\end{aligned}
$$

$a<b$ を満たす整数 $a,b$ に対して, 上の式を $j=a$ から $j=b-1$ まで足し上げると,

$$
\begin{aligned}
\int_a^b f(x)\,dx &=
\sum_{j=a}^{b-1}\frac{f(j)+f(j+1)}{2} +
\sum_{k=2}^n (-1)^{k-1} \frac{B_k}{k!} (f^{(k-1)}(b)-f^{(k-1)}(a)) +
(-1)^n\int_a^b \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx
\\ &=
\frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j) +
\sum_{k=2}^n (-1)^{k-1}\frac{B_k}{k!} (f^{(k-1)}(b)-f^{(k-1)}(a)) - R_n.
\end{aligned}
$$

ここで

$$
R_n = (-1)^{n-1}\int_a^b \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx
$$

とおいた. さらい, $n$ が3以上の奇数のとき $B_n=0$ となることを使うと,

$$
\int_a^b f(x)\,dx =
\frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j) -
\sum_{1\leqq i\leqq n/2} \frac{B_{2i}}{(2i)!} (f^{(2i-1)}(b)-f^{(2i-1)}(a)) - R_n.
$$

この公式を**Euler-Maclaurinの和公式**と呼ぶ.  これは $n$ が3以上の奇数のとき $B_n=0$ となることを使うと次のように書き直される:

$$
\int_a^b f(x)\,dx =
\frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j) -
\sum_{k=2}^n \frac{B_k}{k!} (f^{(k-1)}(b)-f^{(k-1)}(a)) - R_n.
$$

**注意:** 上の公式は $n$ が2以上の整数の場合の結果である. $n=1$ の場合の結果は次のようにして得られる.

$$
\int_0^1 f(x)\,dx = \int_0^1 x' f(x)\,dx =
[xf(x)]_0^1 - \int_0^1 xf'(x)\,dx =
f(1) - \int_0^1 x f'(x)\,dx
$$

より, $a<b$ を満たす整数 $a,b$ に対して,

$$
\begin{aligned}
\int_a^b f(x)\,dx &=
\sum_{j=a}^{b-1}f(j+1) -
\sum_{j=a}^{b-1}\int_0^1 x f'(x+j)\,dx
\\ &=
\sum_{j=a}^b f(j) - f(a) - \int_a^b (x-\lfloor x\rfloor)f'(x)\,dx
\end{aligned}
$$

すなわち,

$$
\begin{aligned}
\sum_{j=a}^b f(j) &=
\int_a^b f(x)\,dx + f(a) + \int_a^b (x-\lfloor x\rfloor)f'(x)\,dx
\\ &=
\int_a^b f(x)\,dx + f(a) + \sum_{j=a}^{b-1}\int_0^1 x f'(x+j)\,dx
\end{aligned}
$$

これらのシンプルな公式も結構よく使われる. $\QED$

**注意:** Euler-Maclaurinの和公式の証明は部分積分の繰り返しであった. それはTaylorの定理の証明とほぼ同じような議論である. Taylorの定理も以下のような部分積分の繰り返しで証明される:

$$
p_n(t) = \frac{(x-t)^n}{n!}
$$

とおくと, $p_0(t)=1$, $-p_n'(t) = p_{n-1}(t)$, $\ds[g(t)(-p_n(t))]_a^x = g(a)\frac{(x-a)^n}{n!}$ なので

$$
\begin{aligned}
f(x) &= f(a) + \int_a^x f(t)\,dt =
f(a) + \int_a^x f(t)p_0(t)\,dt
\\ &=
f(a) + f'(a)(x-a) + \int_a^x f'(t)p_1(t)\,dt
\\ &=
f(a) + f'(a)(x-a) + f''(a)\frac{(x-a)^2}{2} + \int_a^x f''(t)p_2(t)\,dt
\\ &=
f(a) + f'(a)(x-a) + f''(a)\frac{(x-a)^2}{2} + f'''(a)\frac{(x-a)^3}{3!} +
\int_a^x f'''(t)p_3(t)\,dt
\\ &=
\cdots\cdots\cdots\cdots\cdots
\\ &=
\sum_{k=0}^n f^{(k)}(a)\frac{(x-a)^k}{k!} + R_n.
\end{aligned}
$$

ここで

$$
R_n = \int_a^x f^{(n)} p_n(t)\,dt =
\int_a^x f^{(n)} \frac{(x-t)^n}{n!}\,dt
$$

とおいた.  この意味でEuler-Maclaurinの和公式はTaylorの公式の「仲間」だと言える. $\QED$

### Euler-Maclaurinの和公式の2つの解釈

**Euler-Maclaurinの和公式の解釈1:**

$$
\sum_{j=a}^{b-1}\frac{f(j)+f(j+1)}{2} = \frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j)
$$

は積分 $\ds\int_a^b f(x)\,dx$ の近似計算に使われる台形公式である. ゆえに, Euler-Maclaurinの和公式

$$
\begin{aligned}
&
\int_a^b f(x)\,dx =
\frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j) -
\sum_{1\leqq i\leqq n/2} \frac{B_{2i}}{(2i)!} (f^{(2i-1)}(b)-f^{(2i-1)}(a)) - R_n,
\\&
R_n = (-1)^{n-1}\int_a^b \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx
\end{aligned}
$$

は台形公式による積分の近似の誤差が,

$$-
\sum_{1\leqq i\leqq n/2} \frac{B_{2i}}{(2i)!} (f^{(2i-1)}(b)-f^{(2i-1)}(a)) - R_n
$$

になっていることを意味している. 例えば, $n=1$ の場合には, $\ds B_1(x)=x-\frac{1}{2}$ なので,

$$
\int_a^b f(x)\,dx =
\frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j) -
\int_a^b\left(x-\lfloor x\rfloor-\frac{1}{2}\right)f'(x)\,dx.
$$

$n=2$ の場合には $\ds B_2(x)=x^2-x+\frac{1}{6}$, $\ds B_2=\frac{1}{6}$ であり,

$$
\int_a^b f(x)\,dx =
\frac{f(a)+f(b)}{2} + \sum_{j=a+1}^{b-1} f(j) -
\frac{f'(b)-f'(a)}{12} +
\int_a^b\frac{B_2(x-\lfloor x\rfloor)}{2}f''(x)\,dx.
$$

となる.  $\QED$

```julia
# すぐ上の公式の検証

BernoulliNumber(n) = sympy.bernoulli(n)
BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)

function EulerMaclaurinIntegral(f, a, b, n)
    x = symbols("x", real=true)
    (
        (f(a)+f(b))/Sym(2)
        + sum(j->f(j), a+1:b-1)
        - sum(k -> (
                BernoulliNumber(k)/factorial(Sym(k))
                * (diff(f(x), x, k-1)(x=>b) - diff(f(x), x, k-1)(x=>a))
            ), 2:n)
    )
end

function EulerMaclaurinRemainder(f, a, b, n)
    x = symbols("x", real=true)
    g = diff(f(x), x, n)
    (-1)^(n-1) * sum(k -> (
            integrate(BernoulliPolynomial(n,x)*g(x=>x+k), (x,0,1))
        ), a:b-1)/factorial(Sym(n))
end

x = symbols("x", real=true)

[integrate(x^m, (x, 0, 10)) for m in 7:15] |> display

[
    EulerMaclaurinIntegral(x->x^m, 0, 10, 5) - EulerMaclaurinRemainder(x->x^m, 0, 10, 5)
    for m in 7:15
] |> display
```

\[ \left[ \begin{array}{r}12500000\\\frac{1000000000}{9}\\1000000000\\\frac{100000000000}{11}\\\frac{250000000000}{3}\\\frac{10000000000000}{13}\\\frac{50000000000000}{7}\\\frac{200000000000000}{3}\\625000000000000\end{array} \right] \]

\[ \left[ \begin{array}{r}12500000\\\frac{1000000000}{9}\\1000000000\\\frac{100000000000}{11}\\\frac{250000000000}{3}\\\frac{10000000000000}{13}\\\frac{50000000000000}{7}\\\frac{200000000000000}{3}\\625000000000000\end{array} \right] \]

**Euler-Maclaurinの和公式の解釈2:** Euler-Maclaurinの和公式は次のように書き直される:

$$
\begin{aligned}
&
\sum_{j=a}^b f(j) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\sum_{1\leqq i\leqq n/2} \frac{B_{2i}}{(2i)!} (f^{(2i-1)}(b)-f^{(2i-1)}(a)) + R_n,
\\ &
R_n = (-1)^{n-1}\int_a^b \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx
\end{aligned}
$$

これは $n$ が3以上の奇数のとき $B_n=0$ となることを使うと次のように書き直される:

$$
\begin{aligned}
&
\sum_{j=a}^b f(j) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\sum_{k=2}^n \frac{B_k}{k!} (f^{(k-1)}(b)-f^{(k-1)}(a)) + R_n,
\\ &
R_n = (-1)^{n-1}\int_a^b \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx
\end{aligned}
$$

この等式は函数 $f$ の整数における値の和 $\ds\sum_{j=a}^b f(j)$ を積分 $\ds\int_a^b f(x)\,dx$ で近似したときの誤差が

$$
\frac{f(a)+f(b)}{2} +
\sum_{1\leqq i\leqq n/2} \frac{B_{2i}}{(2i)!} (f^{(2i-1)}(b)-f^{(2i-1)}(a)) + R_n
$$

になっていることを意味している. 例えば, $n=1$ の場合には, $\ds B_1(x)=x-\frac{1}{2}$ なので,

$$
\sum_{j=a}^b f(j) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\int_a^b\left(x-\lfloor x\rfloor-\frac{1}{2}\right)f'(x)\,dx.
$$

$n=2$ の場合には $\ds B_2(x)=x^2-x+\frac{1}{6}$, $\ds B_2=\frac{1}{6}$ であり,

$$
\sum_{j=a}^b f(j) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\frac{f'(b)-f'(a)}{12} -
\int_a^b\frac{B_2(x-\lfloor x\rfloor)}{2}f''(x)\,dx.
$$

となる.  $\QED$

```julia
# すぐ上の公式を検証

PowerSum(m, n) = sum(j->j^m, 1:n)
BernoulliNumber(n) = sympy.bernoulli(n)
BernoulliPolynomial(n,x) = sympy.bernoulli(n,x)

function EulerMaclaurinSum(f, a, b, n)
    x = symbols("x", real=true)
    (
        integrate(f(x), (x, a, b))
        + (f(a)+f(b))/Sym(2)
        + sum(k -> (
                BernoulliNumber(k)/factorial(Sym(k))
                * (diff(f(x), x, k-1)(x=>b) - diff(f(x), x, k-1)(x=>a))
            ), 2:n)
    )
end

function EulerMaclaurinRemainder(f, a, b, n)
    x = symbols("x", real=true)
    g = diff(f(x), x, n)
    (-1)^(n-1) * sum(k -> (
            integrate(BernoulliPolynomial(n,x)*g(x=>x+k), (x,0,1))
        ), a:b-1)/factorial(Sym(n))
end

[PowerSum(m, 10) for m in 1:10] |> display

[EulerMaclaurinSum(x->x^m, 1, 10, m+1) for m in 1:10] |> display

[
    EulerMaclaurinSum(x->x^m, 1, 10, m-1) + EulerMaclaurinRemainder(x->x^m, 1, 10, m-1)
    for m in 3:10
] |> display

[
    EulerMaclaurinSum(x->x^m, 1, 10, m-2) + EulerMaclaurinRemainder(x->x^m, 1, 10, m-2)
    for m in 4:10
] |> display

[
    EulerMaclaurinSum(x->x^m, 1, 10, m-3) + EulerMaclaurinRemainder(x->x^m, 1, 10, m-3)
    for m in 5:10
] |> display
```

    10-element Array{Int64,1}:
              55
             385
            3025
           25333
          220825
         1978405
        18080425
       167731333
      1574304985
     14914341925

\[ \left[ \begin{array}{r}55\\385\\3025\\25333\\220825\\1978405\\18080425\\167731333\\1574304985\\14914341925\end{array} \right] \]

\[ \left[ \begin{array}{r}3025\\25333\\220825\\1978405\\18080425\\167731333\\1574304985\\14914341925\end{array} \right] \]

\[ \left[ \begin{array}{r}25333\\220825\\1978405\\18080425\\167731333\\1574304985\\14914341925\end{array} \right] \]

\[ \left[ \begin{array}{r}220825\\1978405\\18080425\\167731333\\1574304985\\14914341925\end{array} \right] \]

### Euler-Maclaurinの和公式の形式的導出

函数 $f(x)$ に対して, ある函数 $F(x)$ で

$$
F(x+1) - F(x) = f(x+h)
$$

という条件を満たすものを求める問題を考える. そのとき, $\ds D=\frac{\d}{\d x}$ とおくと, 形式的にその条件は

$$
(e^D-1)F(x) = e^{hD}f(x) = De^{hD}\int f(x)\,dx
$$

と書き直される. これより, 形式的には

$$
F(x) = \frac{De^{hD}}{e^D-1}\int f(x)\,dx =
\sum_{k=0}^\infty \frac{B_k(h)}{k!}D^k \int f(x)\,dx =
\int f(x)\,dx + \sum_{k=1}^\infty \frac{B_k(h)}{k!}f^{(k-1)}(x).
$$

これより, 整数 $a<b$ について, 形式的には

$$
\sum_{j=a}^{b-1} f(j+h) = F(b)-F(b) =
\int_a^b f(x)\,dx + \sum_{k=1}^\infty \frac{B_k(h)}{k!}(f^{(k-1)}(b)-f^{(k-1)}(a)).
$$

これは $h=0$ とおけば形式的にEuler-Maclaurinの和公式

$$
\sum_{j=a}^{b-1} f(j) = F(b)-F(b) =
\int_a^b f(x)\,dx + \sum_{k=1}^\infty \frac{B_k}{k!}(f^{(k-1)}(b)-f^{(k-1)}(a)).
$$

に一致する.

### べき乗和(再)

$f^{(m+1)}(x)=0$ ならばEuler-Maclaurinの和公式における $R_m$ が消えて, $f^{(m)}(x)$ が定数函数になるので,

$$
\sum_{j=a}^b f(j) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\sum_{k=2}^{m+1} \frac{B_k}{k!} (f^{(k-1)}(b)-f^{(k-1)}(a))
$$

となる. $m$ を正の整数であるとし, $f(x)=x^m$, $a=0$, $b=n$ とすると,

$$
(x^m)^{(k-1)} = m(m-1)\cdots(m-k+2)x^{m-k+1} = \frac{k!}{m+1}\binom{m+1}{k}x^{m+1-k}
$$

なので, $B_0(1)=B_0=1$, $\ds B_1(1)=B_1+1=\frac{1}{2}$, $B_n(1)=B_n+\delta_{n,1}$ も使うと,

$$
\begin{aligned}
\sum_{j=1}^n j^m &=
\int_0^n x^m\,dx + \frac{n^m}{2} +
\sum_{k=2}^m \frac{B_k}{k!}\frac{k!}{m+1}\binom{m+1}{k}x^{m+1-k}
\\ &=
\frac{n^{m+1}}{m+1} + \frac{n^m}{2} +
\frac{1}{m+1}\sum_{k=2}^m \binom{m+1}{k} B_k n^{m+1-k}
\\ &=
\frac{1}{m+1}\sum_{k=0}^m \binom{m+1}{k} B_k(1) n^{m+1-k}
\\ &=
\frac{B_{m+1}(x)-B_{m+1}}{m+1}.
\end{aligned}
$$

最後の等号で

$$
B_{m+1}(x+h) = \sum_{k=0}^{m+1} \binom{m+1}{k}B_k(h)x^{m+1-k}
$$

を使った.  以上の結果は前節で得た結果と一致する. $\QED$

### Stirlingの近似公式

$\log n!=\log 1+\log 2+\cdots+\log n$ の $\log x$ の積分による近似をEuler-Maclaurinの和公式を使って精密化すると, Stirlingの近似公式

$$
\log n! = n\log n - n + \frac{1}{2}\log n + \log\sqrt{2\pi} + o(1)
$$

の精密化が得られることを説明しよう.

$f(x)=\log x$ にStirlingの和公式を適用すると, $k=1,2,3,\ldots$ に対して,

$$
f^{(k)}(x) = \frac{(-1)^{k-1}(k-1)!}{x^k}.
$$

$K$ は3以上の奇数とする. 3以上の奇数 $k$ について $B_k=0$ なので, $N>n$ のとき,

$$
\begin{aligned}
\log n! &= \log N! + \log n - \sum_{j=n}^N \log j
\\ &= \log N! + \log n -\left(
\int_n^N \log x\,dx + \frac{\log n+\log N}{2} +
\sum_{k=2}^{K-1}\frac{B_k}{k(k-1)} \left(\frac{1}{N^{k-1}} - \frac{1}{n^{k-1}}\right) +
R_{K,N}
\right)
\\ &=
\log N! - \left(N\log N - N + \frac{1}{2}\log N\right) -
\sum_{k=2}^{K-1} \frac{B_k}{k(k-1)} \frac{1}{N^{k-1}}
\\ &\,+
n\log n - n +\frac{1}{2}\log n +
\sum_{k=2}^{K-1}\frac{B_k}{k(k-1)} \frac{1}{n^{k-1}} + R_{K,N},
\\
R_{K,N} &= (-1)^{K-1}\int_n^N \frac{\tilde{B}_K(x)}{K}\frac{(-1)^{K-1}}{x^K}\,dx
\end{aligned}
$$

ただし, $\tilde{B}_n(x)=B_n(\lfloor x\rfloor)$ とおいた.

ここでは, $N\to\infty$ のとき

$$
\log N! - \left(N\log N - N + \frac{1}{2}\log N\right) \to \sqrt{2\pi}
$$

となることは既知であるものとする. 例えば, ノート「10 Gauss積分, ガンマ函数, ベータ函数」「12 Fourier解析」のStirlingの近似公式の節を参照して欲しい.  以下ではそれらのノートよりも精密な結果を得る.

このとき, 上の結果で $N\to\infty$ とすると,

$$
\begin{aligned}
&
\log n! =
n\log n - n +\frac{1}{2}\log n + \log\sqrt{2\pi} +
\sum_{k=2}^{K-1}\frac{B_k}{k(k-1)} \frac{1}{n^{k-1}} + R_K,
\\ &
R_K = (-1)^{K-1}\int_n^\infty \frac{\tilde{B}_K(x)}{K}\frac{(-1)^{K-1}}{x^K}\,dx =
O\left(\frac{1}{n^{K-1}}\right).
\end{aligned}
$$

$K=2L+1$ とおくことによって次が得られる: 正の整数 $L$ に対して,

$$
\log n! =
n\log n - n + \frac{1}{2}\log n + \log\sqrt{2\pi} +
\sum_{l=1}^L \frac{B_{2l}}{(2l)(2l-1)}\frac{1}{n^{2l-1}} + O\left(\frac{1}{n^{2L}}\right).
$$

これが求めていた結果である.

例えば, $L=2$ のとき, $\ds B_2=\frac{1}{6}$, $\ds B_4=-\frac{1}{30}$ なので,

$$
\log n! =
n\log n - n + \frac{1}{2}\log n + \log\sqrt{2\pi} +
\frac{1}{12n} - \frac{1}{360n^3} + O\left(\frac{1}{n^4}\right).
$$

これより,

$$
n! = n^n e^{-n}\sqrt{2\pi n}
\left(1+\frac{1}{12n} + \frac{1}{288n^2} - \frac{139}{51840n^3} +  O\left(\frac{1}{n^4}\right)\right).
$$

```julia
x = symbols("x")
series(exp(x/12-x^3/360), x, n=4)
```

\begin{equation*}1 + \frac{x}{12} + \frac{x^{2}}{288} - \frac{139 x^{3}}{51840} + O\left(x^{4}\right)\end{equation*}

### Poissonの和公式とEuler-Maclaurinの和公式の関係

Poissonの和公式とは, 急減少函数 $f(x)$ に対して,

$$
\sum_{m\in\Z} f(m) = \sum_{n\in\Z} \hat{f}(n), \qquad
\hat{f}(p) = \int_\R f(x)e^{2\pi i px}\,dx
$$

が成立するという結果であった. これの右辺は以下のように変形できる:

$$
\begin{aligned}
\sum_{n\in\Z} \hat{f}(n) &=
\sum_{n\in\Z} \int_\R f(x)e^{2\pi i nx}\,dx =
\int_\R f(x)\,dx + 2\sum_{n=1}^\infty\int_\R f(x)\cos(2\pi nx)\,dx
\\ &=
\int_\R f(x)\,dx - \sum_{n=1}^\infty\int_\R f'(x)\frac{\sin(2\pi nx)}{\pi n}\,dx
\\ &=
\int_\R f(x)\,dx + \int_\R \left(-\sum_{n=1}^\infty\frac{\sin(2\pi nx)}{\pi n}\right)f'(x)\,dx
\end{aligned}
$$

2つ目の等号では $e^{2\pi inx}+e^{-2\pi inx}=2\cos(2\pi nx)$ を用い, 3つ目の等号では部分積分を実行し, 4つ目の等号では無限和と積分の順序を交換した. それらの操作は $f(x)$ が急減少函数であれば容易に正当化される.

一方, Euler-Maclaurinの和公式の

$$
B_1(x-\lfloor x\rfloor) = x - \lfloor x\rfloor - \frac{1}{2}
$$

を使う場合から,

$$
\sum_{m\in\Z} f(m) =
\int_\R f(x)\,dx + \int_\R \left(x - \lfloor x\rfloor - \frac{1}{2}\right) f'(x)\,dx
$$

が導かれる. これは部分積分によって得られる次の公式からただちに導かれる易しい公式であることにも注意せよ:

$$
\begin{aligned}
\int_n^{n+1} \left(x - n - \frac{1}{2}\right) f'(x)\,dx &=
\left[\left(x - n - \frac{1}{2}\right)f(x)\right]_n^{n+1} - \int_n^{n+1}f(x)\,dx - n
\\ &=
\frac{f(n+1)-f(n)}{2} - \int_n^{n+1}f(x)\,dx.
\end{aligned}
$$

以上の2つの結果を比較すると, Poissonの和公式とEuler-Maclaurinの和公式の $B_1(x-\lfloor x\rfloor)$ を使った場合は,

$$
x - \lfloor x\rfloor - \frac{1}{2} =
-\sum_{n=1}^\infty\frac{\sin(2\pi nx)}{\pi n}
\tag{$*$}
$$

という公式で結び付いていることがわかる. この公式を認めれば, Euler-Maclaurinの和公式の $B_1(x-\lfloor x\rfloor)$ を使った場合からPoissonの和公式が導かれる.

公式($*$)の左辺はいわゆる**のこぎり波**であり, 右辺はそのFourier級数である. 公式($*$)はFourier級数論における非常に有名な公式であり, 本質的にそれと同じ公式はFourier級数論について書かれた文献には例として必ず載っていると言ってよいくらいである. (Fourier級数論より, 公式($*$)は $x$ が整数でないときには実際に成立していることがわかる.)

このように, のこぎり波のFourier級数展開という非常に特殊な公式はPoissonの和公式という一般的な公式を導くだけの力を持っているのである.

**まとめ:** のこぎり波のFourier級数展開は部分積分を通してPoissonの和公式と本質的に同値である! $\QED$

この節で解説したことは次の文献で指摘されている:

* Tim Jameson, An elementary derivation of the Poisson summation formula

```julia
B_1(x) = x - 1/2
b(x) = B_1(x - floor(x))
S(N,x) = -sum(n->sin(2π*n*x)/(π*n), 1:N)
x = -2:0.001:1.999
N = 10
plot(size=(400,200), ylim=(-0.6,1.2), legend=:top)
plot!(x, b.(x), label="B_1(x-[x]) = x - [x] -1/2")
plot!(x, S.(N,x), label="partial sum of Fourier series (N=$N)")
```

**補足:** このノートの上の方の周期的Bernoulli多項式 $B_k(x-\lfloor x\rfloor)$ のFourier級数展開の節を見ればわかるように,

$$
\sum_{n=1}^\infty \frac{\cos(2\pi nx)}{n^k}, \quad
\sum_{n=1}^\infty \frac{\sin(2\pi nx)}{n^k}
$$

の型のFourier級数の収束先は平行移動と定数倍の違いを除いて周期的Bernoulli多項式になる. $\QED$

### 台形公式とPoissonの和公式の関係

簡単のため $f(x)$ は $\R$ 上の急減少函数であるとし, $a,b\in\Z$ かつ $a<b$ であると仮定する. このとき,

$$
\left(\frac{\sin(2\pi nx)}{\pi n}\right)' = 2\cos(2\pi nx)
$$

を使った部分積分とノコギリ波のFourier級数展開

$$
-\sum_{n=1}^\infty \frac{\sin(2\pi nx)}{\pi n} = x-\lfloor x\rfloor - \frac{1}{2} =B_1(x-\lfloor x\rfloor)
$$

より,

$$
\begin{aligned}
\sum_{n\in\Z}\int_a^b f(x) e^{-2\pi inx} \,dx &=
\int_a^b f(x)\,dx + \sum_{n=1}^\infty 2\int_a^b f(x)\cos(2\pi nx)\,dx
\\ &=
\int_a^b f(x)\,dx - \sum_{n=1}^\infty \int_a^b f'(x)\frac{\sin(2\pi nx)}{\pi n}\,dx
\\ &=
\int_a^b f(x)\,dx + \int_a^b f'(x) \left(x-\lfloor x\rfloor - \frac{1}{2}\right)\,dx
\\ &=
\int_a^b f(x)\,dx + \sum_{m=a}^{b-1} \int_m^{m+1} f'(x) \left(x-\lfloor x\rfloor - \frac{1}{2}\right)\,dx
\\ &=
\int_a^b f(x)\,dx + \sum_{m=a}^{b-1} \int_0^1 f'(t+m) \left(t - \frac{1}{2}\right)\,dt
\\ &=
\int_a^b f(x)\,dx +
\sum_{m=a}^{b-1}\left(\left[f(t+m)\left(t - \frac{1}{2}\right)\right]_0^1 - \int_0^1 f(t+m)\,dt \right)
\\ &=
\sum_{m=a}^{b-1}\frac{f(m)+f(m+1)}{2}
\\ &=
\sum_{m=a}^b f(m) - \frac{f(a)+f(b)}{2}.
\end{aligned}
$$

要するに, ノコギリ波 $B_1(x-\lfloor x\rfloor)$ のFourier級数展開の公式を認めて使えば,

$$
\sum_{n\in\Z}\int_a^b f(x) e^{-2\pi inx} \,dx =
\sum_{m=a}^{b-1}\frac{f(m)+f(m+1)}{2} =
\sum_{m=a}^b f(m) - \frac{f(a)+f(b)}{2}
$$

という公式が得られる. この公式の右辺は積分の近似計算で使われる台形公式に一致しており, 途中の計算では $B_1(x-\lfloor x\rfloor)$ を使う場合の Euler-Maclaurinの公式を使っているとみなされる. (上の計算ではEuler-Maclaurinの公式を使わずに直接的に計算した.) Euler-Maclaurinの公式が積分を台形公式で近似するときの誤差項を与えていたことも思い出そう. さらにこの公式で $a\to-\infty$, $b\to\infty$ の極限を取れば, Poissonの和公式

$$
\sum_{n\in\Z}\int_{-\infty}^\infty f(x) e^{-2\pi inx} \,dx = \sum_{m=-\infty}^\infty f(m)
$$

が得られる($f(x)$ は急減少函数だと仮定してあった).

以上のように, **台形公式, Euler-Maclaurinの公式, Poissonの和公式は同一の数学的現象を異なる視点から眺めたものに過ぎないことがわかる!**

積分の簡易的な近似計算で使われる台形公式は, 純粋数学的に価値の高いEuler-Maclaurinの公式とPoissonの公式に直接的に繋がる公式だったのである. 台形公式の誤差を部分積分の繰り返しによって評価すればEuler-Maclaurinの和公式が得られ, 台形公式をFourier解析の立場から見直せばPoissonの和公式が得られる.

## ゼータ函数への応用

$s>1$ のとき(より一般には $\real s>1$ のとき),

$$
\zeta(s) = \sum_{n=1}^\infty \frac{1}{n^s}
$$

は絶対収束しているのであった. これにEuler-Maclaurinの和公式

$$
\begin{aligned}
&
\sum_{j=a}^b f(j) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\sum_{k=2}^n \frac{B_k}{k!} (f^{(k-1)}(b)-f^{(k-1)}(a)) + R_n,
\\ &
R_n = (-1)^{n-1}\int_a^b \frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x)\,dx
\end{aligned}
$$

を適用してみよう.

### 解析接続

$\real s > 1$ であるとし, $f(x)=x^{-s}$ とおく. このとき,

$$
\begin{aligned}
&
\int_a^\infty f(x)\,dx = \int_1^\infty x^{-s}\,dx =
\left[\frac{x^{-s+1}}{-s+1}\right]_1^\infty = \frac{a^{-(s-1)}}{s-1}, \qquad
f(b)=b^{-s}\to 0 \quad(b\to\infty).
\\ &
\frac{B_k}{k!}f^{(k-1)}(x) =
\frac{B_k}{k}\binom{-s}{k-1} x^{-s-k+1}, \quad
\frac{B_n(x-\lfloor x\rfloor)}{n!}f^{(n)}(x) =
\binom{-s}{n}B_n(x-\lfloor x\rfloor)x^{-s-n}
\end{aligned}
$$

なので, 2以上の整数 $n$ について,

$$
\begin{aligned}
&
\zeta(s) = \frac{1}{s-1} + \frac{1}{2} -
\sum_{k=2}^n \frac{B_k}{k}\binom{-s}{k-1} + R_n,
\\ &
R_n = (-1)^{n-1}\binom{-s}{n}\int_1^\infty B_n(x-\lfloor x\rfloor)x^{-s-n}\,dx.
\end{aligned}
$$

積分 $R_n$ は $\real s+n>1$ ならば絶対収束している.  ゆえに, 複素平面全体に $\zeta(s)$ を自然に拡張する方法(解析接続する方法)が得られた.

$\ds \sum_{k=1}^\infty \frac{1}{n^s}$ そのものではなく, $n=a$ から始まる無限和 $\ds \sum_{k=a}^\infty \frac{1}{n^s}=\zeta(s)-\sum_{n=1}^{a-1}\frac{1}{n^s}$ にEuler-Maclaurinの和公式を適用すると,

$$
\begin{aligned}
&
\zeta(s) = \sum_{n=1}^{a-1} \frac{1}{n^s} - \frac{a^{1-s}}{1-s} +
\frac{1}{2a^s} - \sum_{k=2}^n \frac{B_k}{k a^{s+k-1}}\binom{-s}{k-1} + R_{n,a},
\\ &
R_{n,a} = (-1)^{n-1}\binom{-s}{n}\int_a^\infty B_n(x-\lfloor x\rfloor)x^{-s-n}\,dx.
\end{aligned}
$$

```julia
# 上の公式における ζ(s) - R_{n,a} の函数化

# ζ(s) - R_{n,a} = Σ_{m=1}^{a-1} m^{-s} - a^{1-s}/(1-s) + 1/(2a^s)
#                - Σ_{k=2}^n B_k/(k a^{s+k-1}) binom(-s,k-1)    (k is even)
#
function ApproxZeta(a, n, s)
    ss = float(big(s))
    z = zero(ss)
    z += (a ≤ 1 ? zero(ss) : sum(m->m^(-ss), 1:a-1)) # Σ_{m=1}^{a-1} m^{-s}
    z += -a^(1-ss)/(1-ss)                            # -a^{1-s}/(1-s)
    n == 0 && return z
    z += 1/(2*a^ss)                                  # 1/(2a^s)
    n == 1 && return z
    z -= sum(k -> BB(k)/(k*a^(ss+k-1))*binom(-ss,k-1), 2:2:n)
            #   -Σ_{k=2}^n B_k/(k a^{s+k-1}) binom(-s,k-1)    (k is even)
end

A = ApproxZeta(40, 80, big"0.5")
Z = zeta(big"0.5")
@show A
@show Z;
```

    A = -1.460354508809586812889499152515298012467229331012581490542886087825530529474572
    Z = -1.460354508809586812889499152515298012467229331012581490542886087825530529474503

$\real s > 0$ のとき,

$$
\frac{1}{2a^s} - \sum_{k=2}^n \frac{B_k}{k a^{s+k-1}}\binom{-s}{k-1} + R_{n,a}
$$

は $a\to\infty$ で $0$ に収束するので,

$$
\zeta(s) = \lim_{a\to\infty}\left(\sum_{n=1}^{a-1} \frac{1}{n^s} - \frac{a^{1-s}}{1-s}\right)
\quad (\real s > 0)
$$

が成立することがわかる. これは, Dirichlet級数の部分和 $\ds\sum_{n=1}^{a-1}\frac{1}{n^s}$ から補正項

$$
\frac{a^{1-s}}{1-s}
$$

を引き去ってから, Dirichlet級数の総和を取れば, $0 < \real s < 1$ でも収束して, $\zeta(s)$ の正確な値が得られることを意味している.

```julia
# 上の結果のプロット

ApproxZeta0(a, s) = sum(n->n^(-s), 1:a-1) - a^(1-s)/(1-s)
a = 100
s = 0.05:0.01:0.95
@time z = zeta.(s)
@time w = ApproxZeta0.(a, s)
plot(size=(400, 250), legend=:bottomleft, xlabel="s")
plot!(s, z, label="zeta(s)", lw=2)
plot!(s, w, label="Euler-Maclaurin sum for n=0, a=$a", lw=2, ls=:dash)
```

      0.235033 seconds (507.61 k allocations: 26.628 MiB, 13.12% gc time)
      0.108976 seconds (319.80 k allocations: 15.620 MiB)

```julia
# さらに項の数を1つ増やした場合のプロット

# ζ(s) - R_{1,a} = Σ_{n=1}^{a-1} n^{-s}  - a^{1-s}/(1-s) + 1/(2a^s)
#
ApproxZeta1(a, s) = sum(n->n^(-s), 1:a-1) - a^(1-s)/(1-s) + 1/(2*a^s)

s = -0.95:0.01:0.5
a = 10^3
@time z = zeta.(s)
@time w = ApproxZeta1.(a,s)
plot(size=(400, 250), legend=:bottomleft, xlabel="s")
plot!(s, z, label="zeta(s)", lw=2)
plot!(s, w, label="Euler-Maclaurin sum for n=1, a=$a", lw=2, ls=:dash)
```

      0.000172 seconds (8 allocations: 1.563 KiB)
      0.117856 seconds (313.70 k allocations: 15.673 MiB)

```julia
# さらに一般の場合のプロット
#
# Euler-Maclaurinの和公式で ζ(s) の負の s での値をぴったり近似できていることがわかる.

[(-m, zeta(-m), Float64(ApproxZeta(2, 17, -m))) for m = 0:12] |> display

n = 10
s = -1.5:0.05:0.5
a = 10
@time z = zeta.(s)
@time w = ApproxZeta.(a, n, s)
P1 = plot(size=(400, 250), legend=:bottomleft, xlabel="s")
plot!(s, z, label="zeta(s)", lw=2)
plot!(s, w, label="Euler-Maclaurin sum for a=$a, n=$n", lw=2, ls=:dash)

n = 17
s = -16:0.05:-2.0
a = 2
@time z = zeta.(s)
@time w = ApproxZeta.(a, n, s)
P2 = plot(size=(400, 250), legend=:topright, xlabel="s")
plot!(s, z, label="zeta(s)", lw=2)
plot!(s, w, label="Euler-Maclaurin sum for a=$a, n=$n", lw=2, ls=:dash)
```

    13-element Array{Tuple{Int64,Float64,Float64},1}:
     (0, -0.5, -0.5)
     (-1, -0.08333333333333338, -0.08333333333333333)
     (-2, -0.0, -1.2954252832641667e-77)
     (-3, 0.008333333333333345, 0.008333333333333333)
     (-4, -0.0, -3.454467422037778e-77)
     (-5, -0.0039682539682539715, -0.003968253968253968)
     (-6, -0.0, 0.0)
     (-7, 0.004166666666666668, 0.004166666666666667)
     (-8, -0.0, 0.0)
     (-9, -0.007575757575757582, -0.007575757575757576)
     (-10, -0.0, -4.421718300208356e-75)
     (-11, 0.0210927960927961, 0.021092796092796094)
     (-12, -0.0, 0.0)

      0.000048 seconds (8 allocations: 800 bytes)
      0.144020 seconds (387.92 k allocations: 19.316 MiB)
      0.000267 seconds (8 allocations: 2.719 KiB)
      0.069766 seconds (555.19 k allocations: 20.968 MiB, 17.44% gc time)

```julia
display(P1)
```

上と下のグラフを見ればわかるように, Euler-Maclaurinの和公式によって負の実数での $\zeta$ 函数の値を非常によく近似できている. 実は $\zeta(s)$ を実部が負の複素数まで拡張してもこの近似はうまく行っている.

```julia
display(P2)
```

### ζ(2)の近似計算

$\ds\zeta(2)=\sum_{n=1}^\infty \frac{1}{n^2}$ を計算せよという問題は**Basel問題**と呼ばれているらしい. Basel問題はEulerによって1743年ころに解かれたらしい. Eulerがどのように考えたかについては次の文献を参照せよ.

* 杉本敏夫, バーゼル問題とオイラー, 2007年8月23日, 数理解析研究所講究録, 第1583巻, 2008年, pp.159-167

Eulerは $\zeta(2)$ の近似値を自ら開発したEuler-Maclaurinの和公式を使って精密に計算したらしい.

近似式

$$
\zeta(s) \approx
\sum_{n=1}^{a-1} \frac{1}{n^s} - \frac{a^{1-s}}{1-s} +
\frac{1}{2a^s} - \sum_{k=2}^n \frac{B_k}{k a^{s+k-1}}\binom{-s}{k-1}
$$

を用いて, $\zeta(2)$ を計算してみよう. 3以上の奇数 $n$ について $B_n=0$ となるので, $n=2m$ のとき, 右辺の項数は $a+m+1$ になる.

例えば, $a=10$, $m=9$ とし, 20項の和を取ると,

$$
\zeta(2) \approx 1.64493\;40668\;4749\cdots
$$

となり, 正確な値 $\ds\frac{\pi^2}{6}=1.64493\;40668\;4822\cdots$ と小数点以下第11桁まで一致している.

Eulerは後に $\ds\zeta(2)=\frac{\pi^2}{6}$ を得る. Eulerは競争相手に議論に厳密性に欠けるとして様々な批判を受けたのだが, 以上のような数値計算の結果を知っていたので, 正解を得たという確信は微塵も揺らがなかっただろうと思われる.

**注意:** 論理的に厳密な証明の方法が発達した現代においても, 人間は常に証明を間違う可能性がある. 人間が行った証明は絶対的には信用できない. だから, たとえ証明が完成したと思っていたとしても, 可能ならば数値計算によって論理的に厳密な証明以外の証拠を作っていた方が安全だと思われる. $\QED$

**注意:** 数学のノートを作りながら, 気軽に数値的証拠も同時に得るための道具として, 筆者がこのノート作成のために用いているJulia言語とJupyterとNbextensionsのLive Markdown Previewはこれを書いている時点で相当に優秀な道具であるように思われる. $\QED$

```julia
# 20項の和

N = 20
[(m, N-m-1, 2m, ApproxZeta(N-m-1, 2m, 2) - big(π)^2/6) for m in 2:N÷2-1] |> display

m = 9
a = N-m-1
Z = big(π)^2/6
A = ApproxZeta(a, m, 2)
@show a,m
@show Z
@show A;
```

    8-element Array{Tuple{Int64,Int64,Int64,BigFloat},1}:
     (2, 17, 4, -5.77451793863474833797788940478358503699585407578399357681001619323664145261758e-11)
     (3, 16, 6, 4.808127352395625095013460112150325878389866054958153137408430487774776509324829e-13)
     (4, 15, 8, -8.630887513943044224615236465465206970650911046136527708026292186723865816966492e-15)
     (5, 14, 10, 3.116217978527385328054235573023871173466586797186396436897662720414552852297451e-16)
     (6, 13, 12, -2.200847274100542514575619216657515798396053843860275532661691594239630209744406e-17)
     (7, 12, 14, 3.035248943857815147777677383711316694019656935103319432181355248871820406062584e-18)
     (8, 11, 16, -8.335321043122531064769674746337938450627967961329547742403411230422546148897753e-19)
     (9, 10, 18, 4.746601814392005312714027578027970306539540935051342164737224161514796063021067e-19)

    (a, m) = (10, 9)
    Z = 1.644934066848226436472415166646025189218949901206798437735558229370007470403185
    A = 1.644934066847493071302595112118921642731166540690350214159737969261778785588307

###  s = 1でのζ(s)の定数項がEuler定数になること

$\zeta(s)=\ds\sum_{n=1}^\infty \frac{1}{n^s}$ にEuler-Maclaurinの和公式を使って, 2以上の $n$ について次の公式が得られるのであった:

$$
\begin{aligned}
&
\zeta(s) = \frac{1}{s-1} + \frac{1}{2} -
\sum_{k=2}^n \frac{B_k}{k}\binom{-s}{k-1} + R_n,
\\ &
R_n = (-1)^{n-1}\binom{-s}{n}\int_1^\infty B_n(x-\lfloor x\rfloor)x^{-s-n}\,dx.
\end{aligned}
$$

$n=1$ の場合には

\begin{aligned}
\sum_{j=a}^b f(j) &=
\int_a^b f(x)\,dx + f(a) + \int_a^b (x-\lfloor x\rfloor)f'(x)\,dx
\\ &=
\int_a^b f(x)\,dx + f(a) + \sum_{j=a}^{b-1}\int_0^1 x f'(x+j)\,dx
\end{aligned}

を $f(x)=x^{-s}$, $f'(x)=-sx^{-s-1}$, $a=1$, $b=\infty$ の場合に適用して,

$$
\zeta(s) =
\frac{1}{s-1} + 1 - s\sum_{j=1}^\infty\int_0^1 \frac{x}{(x+j)^{s+1}}\,dx
$$

を得る. したがって,

$$
\lim_{s\to 1}\left(\zeta(s)-\frac{1}{s-1}\right) =
1 - \sum_{j=1}^\infty\int_0^1 \frac{x}{(x+j)^2}\,dx.
$$

そして, $x=t-j$ と置換すると,

$$
\begin{align}
-\int_0^1\frac{x}{(x+j)^2}\,dx &=
-\int_j^{j+1}\frac{-(t-j)}{t^2}\,dt =
-\left[\log t + \frac{j}{t}\right]_j^{j+1}
\\ &=
-\log(j+1)+\log j -\frac{j}{j+1}+1 =
\frac{1}{j+1} + \log j - \log(j+1)
\end{align}
$$

なので, これを $j=1$ から $j=N-1$ まで足し上げることによって,

$$
1 - \sum_{j=1}^{n-1}\int_0^1\frac{x}{(x+j)^2}\,dx =
\sum_{j=1}^N\frac{1}{j} - \log N.
$$

これの $N\to\infty$ での極限はEuler定数 $\gamma=0.5772\cdots$ の定義であった. 以上によって次が示された:

$$
\lim_{s\to 1}\left(\zeta(s)-\frac{1}{s-1}\right) = \gamma = 0.5772\cdots.
$$

### 負の整数におけるゼータ函数の特殊値の計算

Euler-Maclaurinの和公式: $3$ 以上の整数 $k$ について $B_k=0$ なので, 以下の公式で $k$ は偶数のみを動くとしてよい:

$$
\begin{aligned}
&
\sum_{n=a}^b f(n) =
\int_a^b f(x)\,dx + \frac{f(a)+f(b)}{2} +
\sum_{k=2}^m \frac{B_k}{k!}(f^{(k-1)}(b) - f^{(k-1)}(a)) + R_m,
\\ &
R_n = (-1)^{m-1}\int_a^b \frac{\tilde{B}_m(x)}{m!} f^{(m)}(x)\,dx.
\end{aligned}
$$

ここで $\tilde{B}_m(x)=B_m(x-\lfloor x\rfloor)$ とおいた.

Euler-Maclaurinの和公式を $f(x)=n^{-s}$, $a=1$, $b=\infty$ の場合に適用することによって $\zeta(s)$ は次の形で $\Re s > 1-m$ まで自然に延長(解析接続)されるのであった:

$$
\zeta(s) =
\frac{1}{s-1} + \frac{1}{2} -
\frac{1}{1-s}\sum_{k=2}^m \binom{1-s}{k} B_k +
(-1)^{m-1}\int_a^b \binom{-s}{m} \tilde{B}_m(x) x^{-s-m}\,dx.
$$

この公式と $k\geqq 2$ のとき $\ds\binom{1}{k}=0$ となることより,

$$
\zeta(0) = \frac{1}{0-1} + \frac{1}{2} = -\frac{1}{2}.
$$

$r$ は正の整数であるとする. このとき, $m>r$ とすると $\ds\binom{r}{m}=0$ となるので, $B_0=1$, $B_1=-1/2$ なので,

$$
\begin{aligned}
\zeta(-r) &=
-\frac{1}{r+1} + \frac{1}{2} -
\frac{1}{r+1}\sum_{k=2}^{r+1} \binom{m+1}{k} B_k
\\ =&
-\frac{1}{r+1}\sum_{k=0}^{r+1} \binom{m+1}{k} B_k =
-\frac{B_{r+1}}{r+1}.
\end{aligned}
$$

最後の等号で, Bernoulli数を帰納的に計算するために使える公式 $\ds\sum_{k=0}^r \binom{r+1}{k}B_k=0$ を用いた. 例えば, $r=1$ のとき $B_0+2B_1=1+2(-1/2)=0$ となり, $r=2$ のとき, $B_0+3B_1+3B_2=1+3(-1/2)+3(1/6)=0$ となる.

以上によって次が証明された:

$$
\zeta(0)=-\frac{1}{2}, \quad
\zeta(-r) = -\frac{B_{r+1}}{r+1} \quad (r=1,2,3,\ldots).
$$

これらの公式は $B_n(1)=B_n+\delta_{n,1}$, $B_1=-1/2$ を使うと,

$$
\zeta(-r) = -\frac{B_{r+1}(1)}{r+1} \quad (r=0,1,2,\ldots)
$$

の形にまとめられる.

### 発散級数の有限部分と ζ(-r) の関係

前節の結果 $\ds\zeta(-r)=-\frac{B_{r+1}(1)}{r+1}$ ($r=0,1,2,\ldots$) は

$$
\begin{aligned}
&
1+1+1+1+\cdots = -\frac{1}{2},
\\ &
1+2+3+4+\cdots = -\frac{1}{12}
\end{aligned}
$$

のような印象的な形式で書かれることもある. ただし, その場合には左辺が通常の無限和ではなく, ゼータ函数 $\zeta(s)$ の解析接続の意味であることを了解しておかなければいけない.

実はさらに解析接続として理解するだけではなく, 「左辺の発散する無限和から適切に無限大を引き去れば右辺に等しくなる」というようなタイプの命題をうまく作ることもできる. 以下ではそのことを解説しよう.

以下, $\eta$ は非負の実数に値を持つ $\R$ 上の**急減少函数**であると仮定する. ($\R$ 上の急減少函数とは $\R$ 上の $C^\infty$ 函数でそれ自身およびそのすべての階数の導函数に任意の多項式函数をかけたものが $|x|\to\infty$ で $0$ に収束するもののことである.) さらに,

$$
\eta(0)=1, \quad \eta'(0)=0
$$

と仮定する. 例えば $\eta(x)=e^{-x^2}$ はそのような函数の例になっている.

このとき, $\eta(x)$ が急減少函数であることより, $N>0$ のとき, 級数

$$
\sum_{n=1}^\infty n^r \eta(n/N) = 1^r\eta(1/N) + 2^r\eta(2/N) + 3^r\eta(3/N) + \cdots
$$

は常に絶対収束する. $r$ が非負の整数のとき, $N\to\infty$ とすると, この級数は発散級数 $1^r+2^r+3^r+\cdots$ になってしまう.  以下の目標は, Euler-Maclaurinの和公式を使うと, その $N\to\infty$ での発散部分が $CN^{r+1}$ ($C$ は $\eta$ と $r$ で具体的に決まる定数) の形にまとまることを示すことである. そして, 残った有限部分は**常に** $\zeta(-r)$ に収束することも示される.

$\tilde{B}_n(x)=B_n(x-\lfloor x\rfloor)$ と書くことにする.

このとき, $f(x)=\eta(x/N)$ にEuler-Maclaurinの和公式を適用すると, $f(0)=1$, $f'(0)=f(\infty)=f'(\infty)=0$ より,
$$
\begin{aligned}
1+\sum_{n=1}^\infty\eta(x/N) &=
\sum_{n=0}^\infty\eta(x/N)
\\ &=
\int_0^\infty\eta(x/N)\,dx + \frac{1}{2} +B_2(f'(\infty)-f'(0)) -
\int_0^\infty\frac{\tilde{B}_2(x)}{2!}\frac{1}{N^2}\eta''(x/N)\,dx
\\ &=
N\int_0^\infty\eta(y)\,dy + \frac{1}{2} -
\frac{1}{N}\int_0^\infty\frac{\tilde{B}_2(Ny)}{2!}\eta''(y)\,dy.
\end{aligned}
$$

ゆえに, $\zeta(0)=-1/2$ を使うと,

$$
\sum_{n=1}^\infty\eta(x/N) - N\int_0^\infty\eta(y)\,dy =
\zeta(0) + O(1/N).
$$

これは $N\to\infty$ で発散級数 $1+1+1+1+\cdots$ になる無限和 $\ds \sum_{n=1}^\infty\eta(x/N)$ から, その発散部分 $\ds N\int_0^\infty\eta(y)\,dy$ を引き去って, $N\to\infty$ の極限を取ると, $\zeta(0)$ に収束することを意味している. これが欲しい結果の1つ目である.

$r$ は正の整数であるとし, $f(x)=x^r\eta(x/N)$ とおく. そのとき, Leibnitz則

$$
(\varphi(x) \psi(x))^{(m)} = \sum_{i=0}^r \binom{m}{i}\varphi^{(i)}(x)\psi^{(m-i)}(x)
\\
$$

を使うと,

$$
f^{(r+2)}(x) = \frac{1}{N^2}F(x/N), \quad
F(y) = \binom{r+2}{0}y^r\eta^{(r+2)}(y) + \cdots + \binom{r+2}{r}r!\eta(y)
$$

その $f(x)$ にEuler-Maclaurinの和公式を適用すると, $f^{(k)}(\infty)=f^{(k)}(\infty)=0$ および,

$$
f(0) = f'(0) = \cdots = f^{(r-1)}(0) = f^{(r+1)}(0) = 0, \quad
f^{(r)}(0) = r!
$$

より,

$$
\begin{aligned}
\sum_{n=1}^\infty n^r\eta(n/N) &=
\sum_{n=0}^\infty f(n) =
\int_0^\infty f(x)\,dx - \frac{B_{r+1}}{(r+1)!}r! - \frac{B_{r+2}}{(r+2)!}0 +
(-1)^{r+1}\int_0^\infty \frac{\tilde{B}_{r+2}(x)}{(r+2)!} f^{(r+2)}(x)\,dx
\\ &=
N^{r+1}\int_0^\infty y^r\eta(y)\,dy - \frac{B_{r+1}}{r+1} +
(-1)^{r+1}\frac{1}{N}\int_0^\infty \frac{\tilde{B}_{r+2}(Ny)}{(r+2)!} F(y)\,dy
\\ &=
N^{r+1}\int_0^\infty y^r\eta(y)\,dy - \frac{B_{r+1}}{r+1} + O(1/N).
\end{aligned}
$$

ゆえに, $\ds\zeta(-r)=-\frac{B_{r+1}}{r+1}$ を使うと,

$$
\sum_{n=1}^\infty n^r\eta(n/N) - N^{r+1}\int_0^\infty y^r\eta(y)\,dy =
\zeta(-r) + O(1/N).
$$

これは $N\to\infty$ で発散級数 $1^r+2^r+3^r+4^r+\cdots$ になる無限和 $\ds \sum_{n=1}^\infty n^r\eta(x/N)$ から, その発散部分 $\ds N^{r+1}\int_0^\infty y^r\eta(y)\,dy$ を引き去って, $N\to\infty$ の極限を取ると, $\zeta(-r)$ に収束することを意味している. これが欲しい結果である.

**注意:** 以上の計算のポイントは, 非負の急減少函数 $\eta(x)$ で $\eta(0)=1$, $\eta'(0)=0$ を満たすもので発散級数を正則化して得られる級数の場合には, Euler-Maclaurinの和公式の「途中の項」がほとんど消えてしまうことである. $C N^{r+1}$ 型の発散項と定数項と $O(1/N)$ の部分の3つの項しか生き残らない. $\QED$

**注意:** 以上の結果に関するより進んだ解説については次のリンク先を参照せよ:

* Terence Tao, The Euler-Maclaurin formula, Bernoulli numbers, the zeta function, and real-variable analytic continuation, Blog: What's new, 10 April, 2010.

このブログ記事はかなり読み易い. $\QED$

**問題:** 以上の結果を数値計算でも確認してみよ. $\QED$

**ヒント:** $\eta(x)=e^{-x^2}$ の場合を試してみよ. そのとき,

$$
\int_0^\infty y^r\eta(y)\,dy =
\int_0^\infty y^r e^{-y^2}\,dy =
\frac{1}{2}\Gamma\left(\frac{r+1}{2}\right)
$$

となっている. $\QED$

**解答例:** 次のリンク先のノートを見よ.

* 黒木玄, ζ(s) の Re s ＜ 1 での様子 $\QED$

```julia
y = symbols("y", real=true)
r = symbols("r", positive=true)
integrate(y^r*exp(-y^2), (y, 0, oo))
```

\begin{equation*}\frac{\Gamma\left(\frac{r}{2} + \frac{1}{2}\right)}{2}\end{equation*}

```julia

```

13 Euler-Maclaurin summation formula.ipynb


Esté um notebook Colab contendo exercícios de programação em python, numpy e pytorch.

## Coloque seu nome

```python
print('Meu nome é: Fernanda Caldas')
```

    Meu nome é: Fernanda Caldas

# Parte 1:

## Exercícios de Processamento de Dados

Nesta parte pode-se usar as bibliotecas nativas do python como a `collections`, `re` e `random`. Também pode-se usar o NumPy.

## Exercício 1.1
Crie um dicionário com os `k` itens mais frequentes de uma lista.

Por exemplo, dada a lista de itens `L=['a', 'a', 'd', 'b', 'd', 'c', 'e', 'a', 'b', 'e', 'e', 'a']` e `k=2`, o resultado deve ser um dicionário cuja chave é o item e o valor é a sua frequência: {'a': 4, 'e': 3}

```python
from collections import Counter

def top_k(L, k):
    return Counter(L).most_common(k)
```

Mostre que sua implementação está correta usando uma entrada com poucos itens:

```python
L = ['f', 'a', 'a', 'd', 'b', 'd', 'c', 'e', 'a', 'b', 'e', 'e', 'a', 'd']
k = 3
resultado = top_k(L=L, k=k)
print(f'resultado: {resultado}')
```

    resultado: [('a', 4), ('d', 3), ('e', 3)]

Mostre que sua implementação é eficiente usando uma entrada com 10M de itens:

```python
import random
L = random.choices('abcdefghijklmnopqrstuvwxyz', k=10_000_000)
k = 10000
```

```python
%%timeit
resultado = top_k(L=L, k=k)
```

    1 loop, best of 5: 538 ms per loop

## Exercício 1.2

Em processamento de linguagem natural, é comum convertemos as palavras de um texto para uma lista de identificadores dessas palavras. Dado o dicionário `V` abaixo onde as chaves são palavras e os valores são seus respectivos identificadores, converta o texto `D` para uma lista de identificadores.

Palavras que não existem no dicionário deverão ser convertidas para o identificador do token `unknown`.

O código deve ser insensível a maiúsculas (case-insensitive).

Se atente que pontuações (vírgulas, ponto final, etc) também são consideradas palavras.

```python
"""
Eu tinha conseguido fazer até a parte abaixo.
Não estava encontrando uma função para substituir o vetor K pelos identificadores, nem tornar case-insensitive.
"""

import re
D = 'Eu gosto de comer pizza.'
K = re.findall(r"[\w']+|[.,!?;]", D)
print(K)
"""
No código do aluno Andersson Andreé Romero Deza, aprendi essa função "get()" que substitui os tokens pelos identificadores.

O segundo parâmetro (value) retorna o valor atribuído a uma palavra que não esteja no dicionário. [https://www.w3schools.com/python/ref_dictionary_get.asp]

No mesmo código, também encontrei a opção "text.lower()" que torna o código case-insensitive ao deixar tudo minúsculo.

Agradeço ao Andersson pela ajuda neste exercício.
"""

def tokens_to_ids(text, vocabulary):
    import re

    K = re.findall(r"[\w']+|[.,!?;]", text.lower())
    ids = []

    for k in K:
        ids.append(vocabulary.get(k, vocabulary['unknown']))

    return ids
```

    ['Eu', 'gosto', 'de', 'comer', 'pizza', '.']

Mostre que sua implementação esta correta com um exemplo pequeno:

---

```python
V = {'eu': 1, 'de': 2, 'gosto': 3, 'comer': 4, '.': 5, 'unknown': -1}
D = 'Eu gosto de comer pizza.'

print(tokens_to_ids(D, V))
```

    [1, 3, 2, 4, -1, 5]

Mostre que sua implementação é eficiente com um exemplo grande:

```python
V = {'eu': 1, 'de': 2, 'gosto': 3, 'comer': 4, '.': 5, 'unknown': -1}
D = ' '.join(1_000_000 * ['Eu gosto de comer pizza.'])
```

```python
%%timeit
resultado = tokens_to_ids(D, V)
```

    1 loop, best of 5: 2.62 s per loop

## Exercício 1.3

Em aprendizado profundo é comum termos que lidar com arquivos muito grandes.

Dado um arquivo de texto onde cada item é separado por `\n`, escreva um programa que amostre `k` itens desse arquivo aleatoriamente.

Nota 1: Assuma amostragem de uma distribuição uniforme, ou seja, todos os itens tem a mesma probablidade de amostragem.

Nota 2: Assuma que o arquivo não cabe em memória.

Nota 3: Utilize apenas bibliotecas nativas do python.

```python
def sample(path: str, k: int):
    import random

    with open(path) as f:
        D = [line.rstrip('\n') for line in f] #Fonte: https://stackoverflow.com/a/17570045

    return random.choices(D, k = k)
```

Mostre que sua implementação está correta com um exemplo pequeno:

```python
filename = 'small.txt'
total_size = 100
n_samples = 10

with open(filename, 'w') as fout:
    fout.write('\n'.join(f'line {i}' for i in range(total_size)))

samples = sample(path=filename, k=n_samples)
print(samples)
print(len(samples) == n_samples)
```

    ['line 99', 'line 94', 'line 3', 'line 56', 'line 98', 'line 17', 'line 96', 'line 41', 'line 58', 'line 86']
    True

Mostre que sua implementação é eficiente com um exemplo grande:

```python
filename = 'large.txt'
total_size = 1_000_000
n_samples = 10000

with open(filename, 'w') as fout:
    fout.write('\n'.join(f'line {i}' for i in range(total_size)))
```

```python
%%timeit
samples = sample(path=filename, k=n_samples)
assert len(samples) == n_samples
```

    1 loop, best of 5: 256 ms per loop

# Parte 2:

## Exercícios de Numpy

Nesta parte deve-se usar apenas a biblioteca NumPy. Aqui não se pode usar o PyTorch.

## Exercício 2.1

Quantos operações de ponto flutuante (flops) de soma e de multiplicação tem a multiplicação matricial $AB$, sendo que a matriz $A$ tem tamanho $m \times n$ e a matriz $B$ tem tamanho $n \times p$?

Resposta:
- número de somas: $m\cdot p\cdot n$
- número de multiplicações: $m\cdot p\cdot (n-1)$

## Exercício 2.2

Em programação matricial, não se faz o loop em cada elemento da matriz,
mas sim, utiliza-se operações matriciais.

Dada a matriz `A` abaixo, calcule a média dos valores de cada linha sem utilizar laços explícitos.

Utilize apenas a biblioteca numpy.

```python
import numpy as np
np.set_printoptions(edgeitems=10, linewidth=180)
```

```python
A = np.arange(24).reshape(4, 6)
print(A)
```

    [[ 0  1  2  3  4  5]
     [ 6  7  8  9 10 11]
     [12 13 14 15 16 17]
     [18 19 20 21 22 23]]

```python
np.sum(A, axis=1)/(A.shape[1])
```

    array([ 2.5,  8.5, 14.5, 20.5])

## Exercício 2.3

Seja a matriz $C$ que é a normalização da matriz $A$:
$$ C(i,j) = \frac{A(i,j) - A_{min}}{A_{max} - A_{min}} $$

Normalizar a matriz `A` do exercício acima de forma que seus valores fiquem entre 0 e 1.

```python
from numpy import array

C = (array(A) - np.amin(A))/(np.amax(A) - np.amin(A))
C
```

    array([[0.        , 0.04347826, 0.08695652, 0.13043478, 0.17391304, 0.2173913 ],
           [0.26086957, 0.30434783, 0.34782609, 0.39130435, 0.43478261, 0.47826087],
           [0.52173913, 0.56521739, 0.60869565, 0.65217391, 0.69565217, 0.73913043],
           [0.7826087 , 0.82608696, 0.86956522, 0.91304348, 0.95652174, 1.        ]])

## Exercício 2.4

Modificar o exercício anterior de forma que os valores de cada *coluna* da matriz `A` sejam normalizados entre 0 e 1 independentemente dos valores das outras colunas.

```python
C = (array(A) - array(A.min(axis=0)))/(array(A.max(axis=0)) - array(A.min(axis=0)))
C
```

    array([[0.        , 0.        , 0.        , 0.        , 0.        , 0.        ],
           [0.33333333, 0.33333333, 0.33333333, 0.33333333, 0.33333333, 0.33333333],
           [0.66666667, 0.66666667, 0.66666667, 0.66666667, 0.66666667, 0.66666667],
           [1.        , 1.        , 1.        , 1.        , 1.        , 1.        ]])

## Exercício 2.5

Modificar o exercício anterior de forma que os valores de cada *linha* da matriz `A` sejam normalizados entre 0 e 1 independentemente dos valores das outras linhas.

```python
C = ((array(A.T) - array((A.T).min(axis=0)))/(array((A.T).max(axis=0)) - array((A.T).min(axis=0)))).T
C
```

    array([[0. , 0.2, 0.4, 0.6, 0.8, 1. ],
           [0. , 0.2, 0.4, 0.6, 0.8, 1. ],
           [0. , 0.2, 0.4, 0.6, 0.8, 1. ],
           [0. , 0.2, 0.4, 0.6, 0.8, 1. ]])

## Exercício 2.6

A [função softmax](https://en.wikipedia.org/wiki/Softmax_function) é bastante usada em apredizado de máquina para converter uma lista de números para uma distribuição de probabilidade, isto é, os números ficarão normalizados entre zero e um e sua soma será igual à um.

Implemente a função softmax com suporte para batches, ou seja, o softmax deve ser aplicado a cada linha da matriz. Deve-se usar apenas a biblioteca numpy. Se atente que a exponenciação gera estouro de representação quando os números da entrada são muito grandes. Tente corrigir isto.

#### Resposta:

Em [Stack Overflow](https://stackoverflow.com/a/34969389), é sugerida a seguinte adaptação para números muito grandes:

\begin{equation}
\sigma(\mathbf{z})_i = \frac{e^{z_i - z_{max}}}{\sum_{j=1}^K e^{z_j - z_{max}}}
\end{equation}
pois teríamos

\begin{equation}
\sigma(\mathbf{z})_i = \frac{e^{z_i}}{e^{z_{max}}\sum_{j=1}^K e^{z_j} e^{-z_{max}}} = \frac{e^{z_i}}{\sum_{j=1}^K e^{z_j}}
\end{equation}

```python
import numpy as np
from numpy import array

def softmax(A):
    '''
    Aplica a função de softmax à matriz `A`.

    Entrada:
      `A` é uma matriz M x N, onde M é o número de exemplos a serem processados
      independentemente e N é o tamanho de cada exemplo.

    Saída:
      Uma matriz M x N, onde a soma de cada linha é igual a um.
    '''
    aux = np.zeros(A.shape)
    mx = A.max(axis=1)
    den = np.sum(np.exp(array(A.T) - (A.T).max(axis=0)), axis=0)
    for i in range(A.shape[0]):
        aux[i] = np.exp(array(A[i,:]) - array(mx[i]))/array(den[i])

    return aux
```

Mostre que sua implementação está correta usando uma matriz pequena como entrada:

```python
A = np.array([[0.5, -1, 1000],
              [-2,   0, 0.5]])
softmax(A)
```

    array([[0.        , 0.        , 1.        ],
           [0.04861082, 0.35918811, 0.59220107]])

O código a seguir verifica se sua implementação do softmax está correta.
- A soma de cada linha de A deve ser 1;
- Os valores devem estar entre 0 e 1

```python
np.allclose(softmax(A).sum(axis=1), 1) and softmax(A).min() >= 0 and softmax(A).max() <= 1
```

    True

Mostre que sua implementação é eficiente usando uma matriz grande como entrada:

```python
A = np.random.uniform(low=-10, high=10, size=(128, 100_000))
```

```python
%%timeit
softmax(A)
```

    1 loop, best of 5: 593 ms per loop

```python
SM = softmax(A)
np.allclose(SM.sum(axis=1), 1) and SM.min() >= 0 and SM.max() <= 1
```

    True

## Exercício 2.7

A codificação one-hot é usada para codificar entradas categóricas. É uma codificação onde apenas um bit é 1 e os demais são zero, conforme a tabela a seguir.

| Decimal | Binary | One-hot
| ------- | ------ | -------
| 0 | 000    | 1 0 0 0 0 0 0 0
| 1 | 001    | 0 1 0 0 0 0 0 0
| 2 | 010    | 0 0 1 0 0 0 0 0
| 3 | 011    | 0 0 0 1 0 0 0 0
| 4 | 100    | 0 0 0 0 1 0 0 0
| 5 | 101    | 0 0 0 0 0 1 0 0
| 6 | 110    | 0 0 0 0 0 0 1 0
| 7 | 111    | 0 0 0 0 0 0 0 1

Implemente a função one_hot(y, n_classes) que codifique o vetor de inteiros y que possuem valores entre 0 e n_classes-1.

```python
import sys
np.set_printoptions(suppress=True)

def one_hot(y, n_classes):
    A = ((10**(n_classes - y - 1)).astype(int)).astype(str)

    return np.char.zfill(A, n_classes)
```

```python
N_CLASSES = 9
N_SAMPLES = 10
y = (np.random.rand((N_SAMPLES)) * N_CLASSES).astype(int)
print(y)
print(one_hot(y, N_CLASSES))
```

    [3 8 5 5 6 7 6 7 2 4]
    ['000100000' '000000001' '000001000' '000001000' '000000100' '000000010' '000000100' '000000010' '001000000' '000010000']

Mostre que sua implementação é eficiente usando uma matriz grande como entrada:

```python
N_SAMPLES = 100_000
N_CLASSES = 1_000
y = (np.random.rand((N_SAMPLES)) * N_CLASSES).astype(int)
```

```python
%%timeit
one_hot(y, N_CLASSES)
```

    1 loop, best of 5: 221 ms per loop

## Exercício 2.8

Implemente uma classe que normalize um array de pontos flutuantes `array_a` para a mesma média e desvio padrão de um outro array `array_b`, conforme exemplo abaixo:
```
array_a = np.array([-1, 1.5, 0])
array_b = np.array([1.4, 0.8, 0.3, 2.5])
normalize = Normalizer(array_b)
normalized_array = normalize(array_a)
print(normalized_array)  # Deve imprimir [0.3187798  2.31425165 1.11696854]
```

Mostre que seu código está correto com o exemplo abaixo:

```python
array_a = [-1, 1.5, 0]
array_b = [1.4, 0.8, 0.3, 2.5]
normalize = Normalizer(array_b)
normalized_array = normalize(array_a)
print(normalized_array)
```

# Parte 3:

## Exercícios Pytorch: Grafo Computacional e Gradientes

Nesta parte pode-se usar quaisquer bibliotecas.

Um dos principais fundamentos para que o PyTorch seja adequado para deep learning é a sua habilidade de calcular o gradiente automaticamente a partir da expressões definidas. Essa facilidade é implementada através do cálculo automático do gradiente e construção dinâmica do grafo computacional.

## Grafo computacional

Seja um exemplo simples de uma função de perda J dada pela Soma dos Erros ao Quadrado (SEQ - Sum of Squared Errors):
$$ J = \sum_i (x_i w - y_i)^2 $$
que pode ser reescrita como:
$$ \hat{y_i} = x_i w $$
$$ e_i = \hat{y_i} - y_i $$
$$ e2_i = e_i^2 $$
$$ J = \sum_i e2_i $$

As redes neurais são treinadas através da minimização de uma função de perda usando o método do gradiente descendente. Para ajustar o parâmetro $w$ precisamos calcular o gradiente $  \frac{ \partial J}{\partial w} $. Usando a
regra da cadeia podemos escrever:
$$ \frac{ \partial J}{\partial w} = \frac{ \partial J}{\partial e2_i} \frac{ \partial e2_i}{\partial e_i} \frac{ \partial e_i}{\partial \hat{y_i} } \frac{ \partial \hat{y_i}}{\partial w}$$

```
    y_pred = x * w
    e = y_pred - y
    e2 = e**2
    J = e2.sum()
```

As quatro expressões acima, para o cálculo do J podem ser representadas pelo grafo computacional visualizado a seguir: os círculos são as variáveis (tensores), os quadrados são as operações, os números em preto são os cálculos durante a execução das quatro expressões para calcular o J (forward, predict). O cálculo do gradiente, mostrado em vermelho, é calculado pela regra da cadeia, de trás para frente (backward).

Para entender melhor o funcionamento do grafo computacional com os tensores, recomenda-se leitura em:

https://pytorch.org/docs/stable/notes/autograd.html

```python
import torch
```

```python
torch.__version__
```

    '1.10.0+cu111'

**Tensor com atributo .requires_grad=True**

Quando um tensor possui o atributo `requires_grad` como verdadeiro, qualquer expressão que utilizar esse tensor irá construir um grafo computacional para permitir posteriormente, após calcular a função a ser derivada, poder usar a regra da cadeia e calcular o gradiente da função em termos dos tensores que possuem o atributo `requires_grad`.

```python
y = torch.arange(0, 8, 2).float()
y
```

    tensor([0., 2., 4., 6.])

```python
x = torch.arange(0, 4).float()
x
```

    tensor([0., 1., 2., 3.])

```python
w = torch.ones(1, requires_grad=True)
w
```

    tensor([1.], requires_grad=True)

## Cálculo automático do gradiente da função perda J

Seja a expressão: $$ J = \sum_i ((x_i  w) - y_i)^2 $$

Queremos calcular a derivada de $J$ em relação a $w$.

## Forward pass

Durante a execução da expressão, o grafo computacional é criado. Compare os valores de cada parcela calculada com os valores em preto da figura ilustrativa do grafo computacional.

```python
# predict (forward)
y_pred = x * w; print('y_pred =', y_pred)

# cálculo da perda J: loss
e = y_pred - y; print('e =',e)
e2 = e.pow(2) ; print('e2 =', e2)
J = e2.sum()  ; print('J =', J)
```

    y_pred = tensor([0., 1., 2., 3.], grad_fn=<MulBackward0>)
    e = tensor([ 0., -1., -2., -3.], grad_fn=<SubBackward0>)
    e2 = tensor([0., 1., 4., 9.], grad_fn=<PowBackward0>)
    J = tensor(14., grad_fn=<SumBackward0>)

## Backward pass

O `backward()` varre o grafo computacional a partir da variável a ele associada (raiz) e calcula o gradiente para todos os tensores que possuem o atributo `requires_grad` como verdadeiro.
Observe que os tensores que tiverem o atributo `requires_grad` serão sempre folhas no grafo computacional.
O `backward()` destroi o grafo após sua execução. Esse comportamento é padrão no PyTorch.

A título ilustrativo, se quisermos depurar os gradientes dos nós que não são folhas no grafo computacional, precisamos primeiro invocar `retain_grad()` em cada um desses nós, como a seguir. Entretanto nos exemplos reais não há necessidade de verificar o gradiente desses nós.

```python
e2.retain_grad()
e.retain_grad()
y_pred.retain_grad()
```

E agora calculamos os gradientes com o `backward()`.

w.grad é o gradiente de J em relação a w.

```python
if w.grad: w.grad.zero_()
J.backward()
print(w.grad)
```

    tensor([-28.])

Mostramos agora os gradientes que estão grafados em vermelho no grafo computacional:

```python
print(e2.grad)
print(e.grad)
print(y_pred.grad)
```

    tensor([1., 1., 1., 1.])
    tensor([ 0., -2., -4., -6.])
    tensor([ 0., -2., -4., -6.])

## Exercício 3.1
Calcule o mesmo gradiente ilustrado no exemplo anterior usando a regra das diferenças finitas, de acordo com a equação a seguir, utilizando um valor de $\Delta w$ bem pequeno.

$$ \frac{\partial J}{\partial w} = \frac{J(w + \Delta w) - J(w - \Delta w)}{2 \Delta w} $$

```python
def J_func(w, x, y):
    J = torch.sum((x*w - y)**2)

    return J

def grad_J(w, dw, x, y):
    grad = (J_func(w + dw, x, y) - J_func(w - dw, x, y))/(2*dw)

    return grad

# Calcule o gradiente usando a regra diferenças finitas
# Confira com o valor já calculado anteriormente
x = torch.arange(0, 4).float()
y = torch.arange(0, 8, 2).float()
w = torch.ones(1)
dw = 0.01*torch.ones(1)
grad = grad_J(w, dw, x, y)
print('grad=', grad)
```

    grad= tensor([-28.0000])

```python
w
```

    tensor([1.])

```python
x
```

    tensor([0., 1., 2., 3.])

## Exercício 3.2

Minimizando $J$ pelo gradiente descendente

$$ w_{k+1} = w_k - \lambda \frac {\partial J}{\partial w} $$

Supondo que valor inicial ($k=0$) $w_0 = 1$, use learning rate $\lambda = 0.01$ para calcular o valor do novo $w_{20}$, ou seja, fazendo 20 atualizações de gradientes. Deve-se usar a função `J_func` criada no exercício anterior.

Confira se o valor do primeiro gradiente está de acordo com os valores já calculado acima

```python
learning_rate = 0.01
iteracoes = 20

x = torch.arange(0, 4).float()
y = torch.arange(0, 8, 2).float()
w = torch.ones(1)
J = torch.ones(iteracoes)
dw = 0.05*w

for i in range(iteracoes):
    print('i =', i)
    J[i] = J_func(w, x, y)
    print('J=', J[i])
    grad = grad_J(w, dw, x, y)
    print('grad =',grad)
    w = w - learning_rate*grad
    print('w =', w)

import matplotlib.pyplot as plt
# Plote o gráfico da loss J pela iteração i
plt.plot(J)
```

## Exercício 3.3

Repita o exercício 2 mas usando agora o calculando o gradiente usando o método backward() do pytorch. Confira se o primeiro valor do gradiente está de acordo com os valores anteriores. Execute essa próxima célula duas vezes. Os valores devem ser iguais.

```python
learning_rate = 0.01
iteracoes = 20

x = torch.arange(0, 4).float()
y = torch.arange(0, 8, 2).float()
w = torch.ones(1, requires_grad=True)

for i in range(iteracoes):
    print('i =', i)
    J = J_func(w, x, y)
    print('J=', J)
    grad = ?
    print('grad =',grad)
    w = ?
    print('w =', w)

# Plote aqui a loss pela iteração
```

##Exercício 3.4

Quais são as restrições na escolha dos valores de $\Delta w$ no cálculo do gradiente por diferenças finitas?

Resposta:

##Exercício 3.5

Até agora trabalhamos com $w$ contendo apenas um parâmetro. Suponha agora que $w$ seja uma matriz com $N$ parâmetros e que o custo para executar $(x_i w - y_i)^2$ seja $O(N)$.
> a) Qual é o custo computacional para fazer uma única atualização (um passo de gradiente) dos parâmetros de $w$ usando o método das diferencas finitas?
>
> b) Qual é o custo computacional para fazer uma única atualização (um passo de gradiente) dos parâmetros de $w$ usando o método do backpropagation?

Resposta (justifique):

a)

b)

##Exercício 3.6

Qual o custo (entropia cruzada) esperado para um exemplo (uma amostra) no começo do treinamento de um classificador inicializado aleatoriamente?

A equação da entropia cruzada é:
$$L = - \sum_{j=0}^{K-1} y_j \log p_j, $$
Onde:

- K é o número de classes;

- $y_j=1$ se $j$ é a classe do exemplo (ground-truth), 0 caso contrário. Ou seja, $y$ é um vetor one-hot;

- $p_j$ é a probabilidade predita pelo modelo para a classe $j$.

A resposta tem que ser em função de uma ou mais das seguintes variáveis:

- K = número de classes

- B = batch size

- D = dimensão de qualquer vetor do modelo

- LR = learning rate

Resposta:

Fim do notebook.


```python
from IPython.core.display import HTML
HTML("<style>.container { width:95% !important; }</style>")
```

<style>.container { width:95% !important; }</style>

# Lecture 8, Optimality conditions

We are still studying the full problem

$$
\begin{align} \
\min \quad &f(x)\\
\text{s.t.} \quad & g_j(x) \geq 0\text{ for all }j=1,\ldots,J\\
& h_k(x) = 0\text{ for all }k=1,\ldots,K\\
&x\in \mathbb R^n.
\end{align}
$$

## What aspects are important for optimality conditions?
* Think about this for a while
* You can first think about the case without constraints and, then, what should be added there

## Optimality conditions for **unconstrained** optimization

* (**Necessary condition**) Let $f$ be twice differentiable at $x^*\in\mathbb R^n$. If $x^*$ is a local minimizer, then $\nabla f(x^*)=0$ and the Hessian matrix $H(x^*)$ is positively semidefinite.
* (**Sufficient condition**) Let $f$ be twice continuously differentiable at $x^*\in\mathbb R^n$. If $\nabla f(x^*)=0$ and $H(x^*)$ is positively definite, then $x^*$ is a strict local minimizer.

In order to identify which points are optimal, we want to define similar conditions as there are for unconstrained problems through the gradient:

>If $x$ is a  local optimum to function $f$, then $\nabla f(x)=0$.

## Karush-Kuhn-Tucker (KKT) conditions

**Theorem (First order Karush-Kuhn-Tucker (KKT) Necessary Conditions)**

Let $x^*$ be a local minimum for problem
$$
$$
\begin{align} \
\min \quad &f(x)\\
\text{s.t.} \quad & g_j(x) \geq 0\text{ for all }j=1,\ldots,J\\
& h_k(x) = 0\text{ for all }k=1,\ldots,K\\
&x\in \mathbb R^n.
\end{align}
$$
$$

Let us assume that objective and constraint functions are continuosly differentiable at a point $x^*$ and assume that $x^*$ satisfies some regularity conditions (see e.g., https://en.wikipedia.org/wiki/Karush%E2%80%93Kuhn%E2%80%93Tucker_conditions#Regularity_conditions_.28or_constraint_qualifications.29 ). Then there exists unique Lagrance multiplier vectors $\mu^*=(\mu_1^*,\ldots,\mu_J^*)$ and $\lambda^* = (\lambda^*_1,\ldots,\lambda_K^*)$ such that

$$
\begin{align}
&\nabla_xL(x^*,\mu^*,\lambda^*) = 0\\
&\mu_j^*\geq0,\text{ for all }j=1,\ldots,J \text{  (also known as **Dual feasibility**)}\\
&\mu_j^*g_j(x^*)=0,\text{for all }j=1,\ldots,J,
\end{align}
$$

where $L$ is the *Lagrangian function* $$L(x,\mu,\lambda) = f(x)- \sum_{j=1}^J\mu_jg_j(x) -\sum_{k=1}^K\lambda_kh_k(x)$$.

* Lagrangian Function can be viewed as a function aggregated the original objective function plus the **penalized terms on constraint violations**.

## An example of constraint qualifications for inequality constraint problems

**Definition (regular point)**

A point $x^*\in S$ is *regular* if the set of gradients of the active inequality constraints

$$
\{\nabla g_j(x^*) | \text{ constraint } i \text{ is active}\}
$$

is linearly independent. This means that none of them can be expressed as a linear combination of the others. (*In a simple language one might say that they point to different directions; as an example you can think of the basis vectors of $\mathbb R^n$*.)

KKT conditions were developed independently by
* William Karush:"Minima of Functions of Several Variables with Inequalities as Side Constraints". *M.Sc. Dissertation*, Dept. of Mathematics, Univ. of Chicago, 1939
* Harold W. Kuhn &  Albert W. Tucker: "Nonlinear programming", In: *Proceedings of 2nd Berkeley Symposium*, pp. 481–492, 1951

The coefficients $\mu$ and $\lambda$ are called the *KKT multipliers*.

The first equality

$$
\nabla_xL(x,\mu,\lambda) = 0
$$

is called the stationary rule and the requirement

$$
\mu_j^*g_j(x)=0,\text{for all }j=1,\ldots,J
$$

is called the complementarity rule.

### Note:

* In some cases, the necessary conditions are also sufficient for optimality.

* For example, the necessary conditions mentioned above are sufficient for optimality if $f$, $g_j$ and $h_k (\forall j, k)$ are convex (in a minimization problem).

## Example

Consider the optimization problem

$$
\begin{align}
\min &\qquad (x_1^2+x^2_2+x^2_3)\\
\text{s.t}&\qquad x_1+x_2+x_3-3\geq 0.
\end{align}
$$

Let us verify the KKT necessary conditions for the local optimum $x^*=(1,1,1)$.

We can see that

$$
L(x,\mu,\lambda) = (x_1^2+x_2^2+x_3^2)-\mu_1(x_1+x_2+x_3-3)
$$

and thus

$$
\nabla_x L(x,\mu,\lambda) = (2x_1-\mu_1,2x_2-\mu_1,2x_3-\mu_1)
$$

and if $\nabla_x L([1,1,1],\mu,\lambda)=0$, then

$$
2-\mu_1=0 $$
which holds when $$
\mu_1=2.
$$

In addition to this, we can see that $x^*_1+x^*_2+x^*_3-3= 0$. Thus, the completementarity rule holds even though $\mu_1\neq 0$.

## Example 2

Let us check the KKT conditions for a solution that is not a local optimum. Let us have $x^*=(0,1,1)$.

$$
\nabla_x L(x,\mu,\lambda) = (2x_1-\mu_1,2x_2-\mu_1,2x_3-\mu_1)
$$

We can easily see that in this case, the conditions are

$$\left\{
\begin{array}{c}
-\mu_1 = 0\\
2-\mu_1=0
\end{array}
\right.
$$

Clearly, there does not exist a $\mu_1\in \mathbb R$ such that these equalities would hold.

## Example 3

Let us check the KKT conditions for another solution that is not a local optimum. Let us have $x^*=(2,2,2)$.

$$
\nabla_x L(x,\mu,\lambda) = (2x_1-\mu_1,2x_2-\mu_1,2x_3-\mu_1)
$$

We can easily see that in this case, the conditions are

$$
4-\mu_1 = 0
$$

Now, $\mu_1=4$ satisfies this equation. However, now

$$
\mu_1(x^*_1+x^*_2+x^*_3-3)=4(6-3) = 12 \neq 0.
$$

Thus, the completementarity rule fails and the KKT conditions are not true.

### Another example

Formulate the KKT conditions for the following example:
$$
min 𝑓(\mathbf{x}) = (𝑥_1 − 3)^2 + (𝑥_2 − 2)^2\\
s.t. \\
𝑥_1^2 + 𝑥_2^2 ≤ 5,\\
𝑥_1 + 2𝑥_2 = 4,\\
𝑥_1, 𝑥_2 ≥ 0
$$

Check them for $x^* = (2,1)$

* This part will be completed during the lecture by students (10 min).
* You need to use both conditions to find the KKT multipliers. There are three inequality constraints (j=1,2,3) and one equality constraint.

### A reminder of KKT necessary conditions:
$$
\begin{align}
&\nabla_xL(x^*,\mu^*,\lambda^*) = 0\\
&\mu_j^*\geq0,\text{ for all }j=1,\ldots,J\\
&\mu_j^*g_j(x^*)=0,\text{for all }j=1,\ldots,J,
\end{align}
$$

where $L$ is the *Lagrangian function* $$L(x,\mu,\lambda) = f(x)- \sum_{j=1}^J\mu_jg_j(x) -\sum_{k=1}^K\lambda_kh_k(x)$$

```python

```

## Geometric interpretation of the KKT conditions

## Stationary rule

Consider the *Lagrangian function* L as:  $$L(x,\mu,\lambda) = f(x)- \sum_{j=1}^J\mu_jg_j(x) -\sum_{k=1}^K\lambda_kh_k(x)$$.

The stationary rule is:
$$
\nabla_xL(x,\mu,\lambda) = 0
$$

The stationary rule can be written as: There exist $\mu,\lambda'$ so that

$$
-\nabla f(x) = -\sum_{j=1}^K\mu_j\nabla g_j(x) + \sum_{k=1}^K\lambda'_k\nabla h_k(x).
$$

Notice that we have slightly different $\lambda'$.

Now, remember that the $-\nabla v(x)$ gives us the direction of reduction for a function $v$.

Thus, the above equation means that the direction of reduction of the function $-\nabla f(x)$ is countered by the direction of the reduction of the inequality constraints $-\nabla g_j(x)$ and the directions of either growth (or reduction, since $\lambda'$ can be negative) of the equality constraints $\nabla h_k(x)$.

**This means that the function cannot get reduced without reducing the inequality constraints (making the solution infeasible, if already at the bound), or increasing or decreasing the equality constraints (making, thus, the solution again infeasible).**

#### With just one inequality constraint this means that the negative gradients of $f$ and $g$ must point to the same direction.

#### With equality constraints this means that the negative gradient of the objective function and the gradient of the equality constraint must either point to the same or opposite directions

## Complementarity conditions
Another way of expressing complementarity condition

$$
\mu_jg_j(x) = 0 \text{ for all } j=1,\ldots,J
$$

is to say that both $\mu_j$ and $g_j(x)$ cannot be positive at the same time. Especially, if $\mu_j>0$, then $g_j(x)=0$.

**This means that if we want to use the gradient of a constraint for countering the reduction of the function, then the constraint must be at the boundary.**

### Sufficient conditions:

* The necessary conditions are sufficient for optimality if $f$, $g_j$ and $h_k (\forall j, k)$ are convex (in a minimization problem).

* In general, the necessary conditions are not sufficient for optimality and additional information is required, e.g., the Second Order Sufficient Conditions for smooth functions.

### Second-order sufficient conditions (Projected Hessian is positive definite)

For a smooth, non-linear optimization problem, a second order sufficient condition is given as follows:

If $(x^*, \mu^*, \lambda^*$ be a constrained local minimum for the Lagrangian function

$$L(x,\mu,\lambda) = f(x)- \sum_{j=1}^J\mu_jg_j(x) -\sum_{k=1}^K\lambda_kh_k(x)$$

Then,

$$ d^T \nabla _{\mathbf{xx}}^2L(x^*,\mu^*,\lambda^*) d > 0  \text {         (Hessian is positive definite) }$$

But in constrained optimization we are **not interested in all d**.

Instead, we are looking for the $d$ vectors that lies on the tangent space (active constraints).

* i.e., $$ \forall d \neq 0 \text{,   } [\nabla _{x}g_{j}(x^{*}),\nabla _{x}h_{k}(x^{*})]^Td = 0 \text{;   } \forall j, k.$$


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 2.2em;
    line-height: 100%;
    color: rgb(0, 80, 120);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 400;
    font-size: 1.9em;
    line-height: 100%;
    color: rgb(245,179,64);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(94,127,192);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 100%;
}

</style>

```python
from sympy import init_printing, Matrix, symbols
from IPython.display import Image
from warnings import filterwarnings
```

```python
init_printing(use_latex = 'mathjax')
filterwarnings('ignore')
```

# Projection matrices and least squares

## Least squares

* Consider from the previous lecture the three data point in the plane
$$ ({t}_{i},{y}_{i}) =(1,1), (2,2),(3,2) $$
* From this we need to construct a straight line
* This could be helpful say in, statistics (remember, though in statistics we might have to get rid of statistical outliers)
* Nonetheless (view image above) we note that we have a straight line in slope-intercept form
$$ {y}={C}+{Dt} $$
* On the line at *t* values of 1, 2, and 3 we will have
$$ {y}_{1}={C}+{D}=1 \\ {y}_{2}={C}+{2D}=2 \\ {y}_{3}={C}+{3D}=2 $$
* The actual *y* values at these *t* values are 1, 2, and 2, though
* We are thus including an error of
$$ \delta{y} \\ { \left( { e }_{ 1 } \right)  }^{ 2 }={ \left[ \left( C+D \right) -1 \right]  }^{ 2 }\\ { \left( { e }_{ 2 } \right)  }^{ 2 }={ \left[ \left( C+2D \right) -2 \right]  }^{ 2 }\\ { \left( { e }_{ 3 } \right)  }^{ 2 }={ \left[ \left( C+3D \right) -2 \right]  }^{ 2 } $$
* Since some are positive and some are negative (actual values below or above the line), we simply determine the square (which will always be positive)
* Adding the (three in our example here) squares we have the sum total of the error (which is actuall just the sqautre of the distance between the line and actual *y* values)
* The line will be the best fit when this error sum is at a minimum (hence *least squares*)
* We can do this with calculus or with linear algebra
* For calculus we take the partial derivatives of both unknowns and set to zero
* For linear algebra we project orthogonally onto the columnspace (hence minimizing the error)
    * Note that the solution **b** does not exist in the columnspace (it is not a linear combination of the columns)

### Calculus method

* We'll create a function *f*(C,D) and successively take the partial derivatives of both variables and set it to zero
* We fill then have two equation with two unknowns to solve (which is easy enough to do manually or by simple linear algebra and row reduction)

```python
C, D = symbols('C D')
```

```python
e1_squared = ((C + D) - 1) ** 2
e2_squared = ((C + 2 * D) - 2) ** 2
e3_squared = ((C + 3 * D) - 2) ** 2
f = e1_squared + e2_squared + e3_squared
f
```

$$\left(C + D - 1\right)^{2} + \left(C + 2 D - 2\right)^{2} + \left(C + 3 D - 2\right)^{2}$$

```python
f.expand() # Expanding the expression
```

$$3 C^{2} + 12 C D - 10 C + 14 D^{2} - 22 D + 9$$

* Doing the partial derivatives will be
$$ f\left( C,D \right) =3{ C }^{ 2 }+12CD-10C+14{ D }^{ 2 }-22D+9\\ \frac { \partial f }{ \partial C } =6C+12D-10=0\\ \frac { \partial f }{ \partial D } =12C+28D-22=0 $$

```python
f.diff(C) # Taking the partial derivative with respect to C
```

$$6 C + 12 D - 10$$

```python
f.diff(D) # Taking the partial derivative with respect to D
```

$$12 C + 28 D - 22$$

* Setting both equal to zero (and creating a simple augmented matrix) we get
$$ 6C+12D-10=0\\ 12C+28D-22=0\\ \therefore \quad 6C+12D=10\\ \therefore \quad 12C+28D=22 $$

```python
A_augm = Matrix([[6, 12, 10], [12, 28, 22]])
A_augm
```

$$\left[\begin{matrix}6 & 12 & 10\\12 & 28 & 22\end{matrix}\right]$$

```python
A_augm.rref() # Doing a Gauss-Jordan elimination to reduced row echelon form
```

$$\begin{pmatrix}\left[\begin{matrix}1 & 0 & \frac{2}{3}\\0 & 1 & \frac{1}{2}\end{matrix}\right], & \begin{bmatrix}0, & 1\end{bmatrix}\end{pmatrix}$$

* We now have a solution
$$ {y}=\frac{2}{3} + \frac{1}{2}{t}$$

### Linear algebra

* We note that we can construct the following
$$ {C}+{1D}={1} \\ {C}+{2D}={2} \\ {C}+{3D}={2} \\ {C}\begin{bmatrix} 1 \\ 1\\ 1 \end{bmatrix}+{D}\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}=\begin{bmatrix} 1 \\ 2 \\ 2 \end{bmatrix} \\ A\underline { x } =\underline { b } \\ \begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}\begin{bmatrix} C \\ D \end{bmatrix}=\begin{bmatrix} 1 \\ 2 \\ 2 \end{bmatrix} $$
* **b** is not in the columnspace of A and we have to do orthogonal projection
$$ { A }^{ T }A\hat { x } ={ A }^{ T }\underline { b } \\ \hat { x } ={ \left( { A }^{ T }A \right)  }^{ -1 }{ A }^{ T }\underline { b }  $$

```python
A = Matrix([[1, 1], [1, 2], [1, 3]])
b = Matrix([1, 2, 2])
A, b # Showing the two matrices
```

$$\begin{pmatrix}\left[\begin{matrix}1 & 1\\1 & 2\\1 & 3\end{matrix}\right], & \left[\begin{matrix}1\\2\\2\end{matrix}\right]\end{pmatrix}$$

```python
x_hat = (A.transpose() * A).inv() * A.transpose() * b
x_hat
```

$$\left[\begin{matrix}\frac{2}{3}\\\frac{1}{2}\end{matrix}\right]$$

* Again, we get the same values for C and D

* Remember the following
$$ \underline{b} = \underline{p}+\underline{e} $$
* **p** and **e** are perpendicular
* Indeed **p** is in the columnspace of A and **e** is perpendicular to the columspace (or any vector in the columnspace)

## Example problem

### Example problem 1

* Find the quadratic (second order polynomial) equation through the origin, with the following data points: (1,1), (2,5) and (-1,-2)

#### Solution

* Let's just think about a quadratic equation in *y* and *t*
$$ {y}={c}_{1} +{C}{t}+{D}{t}^{2} $$
* Through the origin (0,0) means *y* = 0 and *t* = 0, thus we have
$$ {0}={c}_{1} +{C}{0}+{D}{0}^{2} \\ {c}_{1}=0 \\ {y}={C}{t}+{D}{t}^{2} $$

* This gives us three equation for our three data points
$$ C\left( 1 \right) +D{ \left( 1 \right)  }^{ 2 }=1\\ C\left( 2 \right) +D{ \left( 2 \right)  }^{ 2 }=5\\ C\left( -1 \right) +D{ \left( -1 \right)  }^{ 2 }=-2\\ C\begin{bmatrix} 1 \\ 2 \\ -1 \end{bmatrix}+D\begin{bmatrix} 1 \\ 4 \\ 1 \end{bmatrix}=\begin{bmatrix} 1 \\ 5 \\ -2 \end{bmatrix}\\ A=\begin{bmatrix} 1 & 1 \\ 2 & 4 \\ -1 & 1 \end{bmatrix}\\ \underline { x } =\begin{bmatrix} C \\ D \end{bmatrix}\\ \underline { b } =\begin{bmatrix} 1 \\ 5 \\ -2 \end{bmatrix} $$

* Clearly **b** is not in the columnspace of A and we have to project orthogonally onto the columnspace using
$$ \hat { x } ={ \left( { A }^{ T }A \right)  }^{ -1 }{ A }^{ T }\underline { b } $$

```python
A = Matrix([[1, 1], [2, 4], [-1, 1]])
b = Matrix([1, 5, -2])
x_hat = (A.transpose() * A).inv() * A.transpose() * b
x_hat
```

$$\left[\begin{matrix}\frac{41}{22}\\\frac{5}{22}\end{matrix}\right]$$

* Here's a simple plot of the equation

```python
import matplotlib.pyplot as plt # The graph plotting module
import numpy as np # The numerical mathematics module
%matplotlib inline
```

```python
x = np.linspace(-2, 3, 100) # Creating 100 x-values
y = (41 / 22) * x + (5 / 22) * x ** 2 # From the equation above
plt.figure(figsize = (8, 6)) # Creating a plot of the indicated size
plt.plot(x, y, 'b-') # Plot the equation above , in essence 100 little plots using small segmnets of blue lines
plt.plot(1, 1, 'ro') # Plot the point in a red dot
plt.plot(2, 5, 'ro')
plt.plot(-1, -2, 'ro')
plt.plot(0, 0, 'gs') # Plot the origin as a green square
plt.show(); # Create the plot
```

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_16_Projection_matrices_and_least_squares.ipynb