```python
import sympy as sy
import control as pc
```

Part of the big question is whether we are simulating the dynamic response, or the steady-state mathematics iterations. Here below is the steady state reponse mathamtical system design, rather than a dynamic diffferential changes with time. Hence, Python's control toolbox cannot solve a steady-state response of a already stable system. However, attempting to perform a fully stable model means that if we want to perform feedback, there's a factor of recalculating stability from that until differential changes become 0 at different time-steps. So for example, if you were to perform feedback after an initial state, you can get a steady state reponse, but this is not the dynamic response, this is solving for the right answer after a number of iterations, and all the initial steady-state answers are wrong for a feedback system. So this gives a steady state response. If we change an input signal also at another point of time, then there is a new steady state response, but the question is whether you want to account for the time dynamics. Because currently, the system is accounting for the digital computing dynamics rather than the actual transient model, so you can't account for how the system dynamically responds and results in a real-time steady-state. So if an input changes in time, you can actually model the time differential change. If we begin considering delays and inputs into the rest of the system, we need to model time as time and computations as computations - and hence we need a time variable input.

There's also the aspect that optics is fundamentally linear and 3D, so how do we account for differential changes in it? What is its dynamic response? There are only dynamic changes on dynamic inputs. What are time changes in optics? You could argue it is near instantaneous. So there's group delay as the time it takes for light to travel through the waveguide, specially if it has been affected in any way. So there is a need to model transient and steady-state responses, but each of these models should be kept separate.

Now, getting a time domain model of a thermo-optic phase shifter response is non-trivial nor necessarily accurate. So, time is a completely new thing in the optical domain for these type of modelled systems. Yes, there is propagation time effects.

Let's consider an MZI, in one arm there's light going over a certain length which incurrs a certain propagation time. Assuming we're only in steady-state mode, when does change occur? You can differtiate phase shift against length for example, but you would be discretizing space-wise. So imagine Pac-Mans moving in a grid of time, according to their position, their properties change, such as phase and this is also dependent in temperature at a given position, but then there must be a differential in temperature and length to calculate a steady state response. So there's this big aspect of time differentials. Could we work based purely on steady-state time differentials? Actually, that could work. There's also the factor of where to add delays, but if we consider every wavelength arm as a system, then we can simply follow the basic principles of control theory.

Hence, the time discretization must occur at a fraction of the time of propagation of the shortest path in the model.

So our inputs into the control system are the electromagnetic signals, that are time-domain constrained

#### Beamsplitter
\begin{equation}
\begin{bmatrix} E_{out0}(t) \\ E_{out1}(t) \end{bmatrix} =
\begin{bmatrix} r_{ac}& t_{bc} \\  t_{ad}& r_{bd} \end{bmatrix}
\begin{bmatrix} E_{in0}(t) \\ E_{in1}(t) \end{bmatrix} =
\begin{bmatrix} \sqrt{r} &  i\sqrt{1-r} \\  i \sqrt{1-r} & \sqrt{r} \end{bmatrix}
\begin{bmatrix} E_{in0}(t) \\ E_{in1}(t) \end{bmatrix}  =
\textbf{B}\textbf{E(t)}
\end{equation}

#### Phase Shifter on one arm

Assuming a phase shifter on top arm
\begin{equation}
\textbf{ $\Phi$ } = \begin{bmatrix} e^{i\phi} & 0 \\ 0 & 1\end{bmatrix}
\end{equation}

Note we're not yet taking into account Linear and LinearGroup delay, or the quantum probability variable within the design of the device.

Hence the MZI configuration is:

\begin{equation}
M\textbf{E(t)}  = \textbf{B} \textbf{ $\Phi$ } \textbf{B}\textbf{E(t)}
\end{equation}

\begin{equation}
M\textbf{E(t)}  = \begin{bmatrix} \sqrt{r} &  i\sqrt{1-r} \\  i \sqrt{1-r} & \sqrt{r} \end{bmatrix} \begin{bmatrix} e^{i\phi} & 0 \\ 0 & 1\end{bmatrix} \begin{bmatrix} \sqrt{r} &  i\sqrt{1-r} \\  i \sqrt{1-r} & \sqrt{r} \end{bmatrix}\begin{bmatrix} E_{in0}(t) \\ E_{in1}(t) \end{bmatrix} = \begin{bmatrix} E_{out0}(t) \\ E_{out1}(t) \end{bmatrix}
\end{equation}

Now we could just crunch some matrices together and input our time signals, but what if we wanted to leverage classical control system theory with regards to MIMO inputs and outputs, so that it is easier to design this type of systems togther and include feedback and nonlinearity back into the classical control models?

The standard way to start is to create the differential equations for this type of system and we can use the fast linear properties of SymPy to create a single MZI component, and the quantum probability variable could be added into this later. Right now let´s consider that we've only got differential equations, with equal reflectivity on both sides of the matrix:

```python
t = sy.Symbol("t")
E0 = sy.Function("E_0")(t)
E0_s = sy.Symbol("E_0")
dE0_s = sy.Symbol("dE_0")
E1 = sy.Function("E_1")(t)
E1_s = sy.Symbol("E_1")
dE1_s = sy.Symbol("dE_1")
phi = sy.Function("\phi")(t)
r_0, r_1 = sy.symbols("r_0 r_1")
B_0  = sy.Matrix([[sy.sqrt(r_0), sy.I*sy.sqrt(1 - r_0)],
				[sy.I*sy.sqrt(1 - r_0), sy.sqrt(r_0)]])
B_1  = sy.Matrix([[sy.sqrt(r_1), sy.I*sy.sqrt(1 - r_1)],
				[sy.I*sy.sqrt(1 - r_1), sy.sqrt(r_1)]])
Phi = sy.Matrix([[sy.exp(sy.I * phi), 0],
				[0, 1]])
E = sy.Matrix([[E0], [E1]])

out = B_0 * Phi * B_1 * E
out
```

$\displaystyle \left[\begin{matrix}\left(\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right) \operatorname{E_{0}}{\left(t \right)} + \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}}\right) \operatorname{E_{1}}{\left(t \right)}\\\left(\sqrt{r_{0}} \sqrt{r_{1}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}}\right) \operatorname{E_{1}}{\left(t \right)} + \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}} e^{i \phi{\left(t \right)}}\right) \operatorname{E_{0}}{\left(t \right)}\end{matrix}\right]$

```python
differential_time_response = out.diff(t)
differential_time_response = differential_time_response.replace(E0.diff(t), dE0_s)
differential_time_response = differential_time_response.replace(E1.diff(t), dE1_s)
differential_time_response
```

$\displaystyle \left[\begin{matrix}dE_{0} \left(\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right) + dE_{1} \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}}\right) + i \sqrt{r_{0}} \sqrt{r_{1}} \operatorname{E_{0}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - \sqrt{r_{0}} \sqrt{1 - r_{1}} \operatorname{E_{1}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}\\dE_{0} \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}} e^{i \phi{\left(t \right)}}\right) + dE_{1} \left(\sqrt{r_{0}} \sqrt{r_{1}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}}\right) - \sqrt{r_{1}} \sqrt{1 - r_{0}} \operatorname{E_{0}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - i \sqrt{1 - r_{0}} \sqrt{1 - r_{1}} \operatorname{E_{1}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}\end{matrix}\right]$

The problem becomes an equilibrium condition to solve. One could argue that when the output is in equilibrium or in steady state:

\begin{equation}
\begin{bmatrix} \frac{dE_{out0}}{dt}  \\ \frac{dE_{out1}}{dt} \end{bmatrix}  = 0
\end{equation}

```python
# Note we're only expecting one solution
dE0_dt = sy.solve(differential_time_response[0], dE0_s)[0]
dE0_dt = dE0_dt.replace(E0, E0_s)
dE0_dt = dE0_dt.replace(E1, E1_s)
dE0_dt
```

$\displaystyle \frac{- i E_{0} \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} + E_{1} \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - i dE_{1} \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} - i dE_{1} \sqrt{r_{1}} \sqrt{1 - r_{0}}}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}}$

```python
# Note we're only expecting one solution
dE1_dt = sy.solve(differential_time_response[0], dE1_s)[0]
dE1_dt = dE1_dt.replace(E0, E0_s)
dE1_dt = dE1_dt.replace(E1, E1_s)
# dE1_dt = dE1_dt.replace(dE0_s, dE0_dt) # Cannot be solved canonically as it is a complementary system.
dE1_dt
```

$\displaystyle \frac{i \left(i E_{0} \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - E_{1} \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} + dE_{0} \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - dE_{0} \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right)}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}}$

Note that this is a complemenetary function with regards to the other state variable.

Hence, we can write in the nonlinear state-space equations:

\begin{equation}
\dot{\textbf{x}} = \textbf{A} \textbf{x} +  \textbf{B} \textbf{u}
\end{equation}

\begin{equation}
\textbf{y} = \textbf{C} \textbf{x} + \textbf{D} \textbf{u}
\end{equation}

```python
A, B = sy.linear_eq_to_matrix([dE0_dt, dE1_dt], [E0_s, E1_s])
B = sy.simplify(B)
A = sy.simplify(A)
```

```python
A
```

$\displaystyle \left[\begin{matrix}- \frac{i \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}} & \frac{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}}\\- \frac{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}} & - \frac{i \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}}\end{matrix}\right]$

```python
B
```

$\displaystyle \left[\begin{matrix}\frac{i dE_{1} \left(\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}\right)}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}}\\\frac{i dE_{0} \left(- \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right)}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}}\end{matrix}\right]$

Note that the differential state inputs are coupled into the state space system for the opposite original state variables, hence are coupled together and this is the nonlinear represenatation. We can no longer set the differntials inputs to 0, since they are our effective state changes. However, we know have a nonlinear system we have to deal with. It is both nonlinear in $\phi$ and in $\frac{dE_0}{dt} * e^{\phi}$. Note that this system should always be oscillating, that is the nature of light. So let us redefine what a steady state of a system is, because there will be none using standard control theory in optics. But is there a steady state in an MZI? That is a different question. But there is a steady state in an MZI for a given wavelength and is fully dependent on the phase input.

We're also dealing with complementary systems.

### Thermo-optic Phase Shifter Model

```python

```

```python

```


# メモ

手を動かしてまなぶ 線形代数 藤岡敦

を読む。

http://www.shokabo.co.jp/

に詳細な解答があるって。

# 行列

1. 行列とは数を長方形状に並べたものである。
1. すべての成分が 0 の行列を零行列という。
1. 行と列の個数が等しい行列を正方行列という。
1. 特別な正方行列として、単位行列、対角行列、スカラー行列、上三角行列、下三角行列があげられる。
1. 単位行列はクロネッカーのデルタを用いて、簡単に表すことができる。
1. 転置をとっても変わらない正方行列を対称行列という。
1. 行ベクトル、列ベクトルをあわせて数ベクトルという。
1. すべての成分が 0 の数ベクトルを零ベクトルという。

自然数 $i=1,2,\cdots,m$ および $j=1,2,\cdots,n$ に対して、数 $a_{ij}$ が対応しているとする。

このとき、$mn$ 個の数 $a_{11},a_{12},\cdots,a_{mn}$ を丸括弧  ( ) や角括弧 [ ] を用いて長方形状に並べたものを $m \times n$ 行列または$m$行$n$列の行列という。

$a_{ij}$ の $i$ や $j$ を添字(そえじ)という。

```latex
%%latex
\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix}, \quad
\begin{bmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{bmatrix}
```

\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix}, \quad
\begin{bmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{bmatrix}

例

次の行列は $2 \times 3$ 行列である。

```latex
%%latex
\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}

```

\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}

```
from sympy import *
A=Matrix([[1,2,3],[4,5,6]])
display(A)
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\end{matrix}\right]$

行列 $A$ について
$A = (a_{ij})$
と書く。

$\quad a_{ij},\quad (a_{i1} a_{i2} \cdots a_{in}), \quad

\begin{pmatrix}
    a_{1j} \\
    a_{2j} \\
    a_{2j} \\
    a_{mj}
\end{pmatrix}$

を$A$の$(i,j)$成分、第$i$行、第$j$列という。

```latex
%%latex
\begin{pmatrix}
    a_{1j} \\
    a_{2j} \\
    a_{2j} \\
    a_{mj}
\end{pmatrix}
```

\displaystyle
\begin{pmatrix}
    a_{1j} \\
    a_{2j} \\
    a_{2j} \\
    a_{mj}
\end{pmatrix}

 $ A=(a_{ij})_{m \times n} $ を $ m \times n $ 行列、 $ B=(b_{kl})_{p \times q} $ を $ p \times q $ 行列とする。

$ A $ と $ B $ が同じ型で、対応する成分が等しいとき、すなわち、 $ m=p $ かつ $ n=q $ で、
任意の $ i=1,2,\cdots,m $ および $ j=1,2,\cdots,n $ に対して $ a_{ij}=b_{ij} $ が成り立つとき

$$ A = B$$

と記し、A $ と $ B $ は等しい、という。A = B $ でないときは

$$ A \neq B $$

$$
\begin{array}{rcl}
\begin{pmatrix}
    a & b & c \\
    d & e & f
\end{pmatrix}
&=&
\begin{pmatrix}
    p & q & r \\
    s & t & u
\end{pmatrix} \\[0.8em]

&\Updownarrow& \\[0.8em]

a = p,\quad  b = q,\quad  c = r&,& d = s,\quad  e = t,\quad  f = u

\end{array}
$$

```latex
%%latex
\begin{array}{rcl}
\begin{pmatrix}
    a & b & c \\
    d & e & f
\end{pmatrix} &=&
\begin{pmatrix}
    p & q & r \\
    s & t & u
\end{pmatrix} \\[0.8em]

&\Updownarrow& \\[0.8em]

a = p,\quad  b = q,\quad  c = r&,& d = s,\quad  e = t,\quad  f = u
\end{array} \\

```

\begin{array}{rcl}
\begin{pmatrix}
    a & b & c \\
    d & e & f
\end{pmatrix} &=&
\begin{pmatrix}
    p & q & r \\
    s & t & u
\end{pmatrix} \\[0.8em]

&\Updownarrow& \\[0.8em]

a = p,\quad  b = q,\quad  c = r&,& d = s,\quad  e = t,\quad  f = u
\end{array} \\

```latex
%%latex
すべての成分が~0~の~m \times n~行列を~O_{m,n}~または~O~と書き、零行列という。
\\[0.8em]
\quad \quad \quad \quad
O_{3,2} =
\begin{pmatrix}
    0 & 0 \\
    0 & 0 \\
    0 & 0
\end{pmatrix}
\\[0.8em]
2~列、~3~行
```

すべての成分が~0~の~m \times n~行列を~O_{m,n}~または~O~と書き、零行列という。
\\[0.8em]
\quad \quad \quad \quad
O_{3,2} =
\begin{pmatrix}
    0 & 0 \\
    0 & 0 \\
    0 & 0
\end{pmatrix}
\\[0.8em]
2~列、~3~行

```latex
%%latex
一般に、2~つの命題~P,~Q~に対して、P~ならば~Q~であることを~P \Longrightarrow Q~と表し、P~と~Q~が同値であることを
~P \Longleftrightarrow Q~と表す。
```

一般に、2~つの命題~P,~Q~に対して、P~ならば~Q~であることを~P \Longrightarrow Q~と表し、P~と~Q~が同値であることを
~P \Longleftrightarrow Q~と表す。

```latex
%%latex
A = (a_{ij})_{n \times n}~を~n \times n~ 行列とするとき、A~を~n~次の正方行列という。\\
成分~a_{11}, a_{22}, \cdots a_{nn}~を~A~の対角成分という。\\[0.8em]

正方行列 ~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j) \\[0.8em]

を満たすとき、A~を対角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j), \quad a_{11} = a_{22} = \cdots = a_{nn} \\[0.8em]

を満たすとき、A~をスカラー行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i > j) \\[0.8em]

を満たすとき、A~を上三角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i < j) \\[0.8em]

を満たすとき、A~を下三角行列という。
```

A = (a_{ij})_{n \times n}~を~n \times n~ 行列とするとき、A~を~n~次の正方行列という。\\
成分~a_{11}, a_{22}, \cdots a_{nn}~を~A~の対角成分という。\\[0.8em]

正方行列 ~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j) \\[0.8em]

を満たすとき、A~を対角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j), \quad a_{11} = a_{22} = \cdots = a_{nn} \\[0.8em]

を満たすとき、A~をスカラー行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i > j) \\[0.8em]

を満たすとき、A~を上三角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i < j) \\[0.8em]

を満たすとき、A~を下三角行列という。

```latex
%%latex
2~次の正方行列
~A~ =
\begin{pmatrix}
    a & b \\
    c & d
\end{pmatrix}
~とする。\\[0.8em]

A~の対角成分は~
\begin{pmatrix}
    a & 0 \\
    0 & d
\end{pmatrix}\\[0.8em]

A~が対角行列、上三角行列、下三角行列となるときそれぞれ\\[0.8em]

\begin{pmatrix}
    a & 0 \\
    0 & a
\end{pmatrix}, \quad

\begin{pmatrix}
    a & b \\
    0 & d
\end{pmatrix}, \quad

\begin{pmatrix}
    a & 0 \\
    c & d
\end{pmatrix}
```

2~次の正方行列
~A~ =
\begin{pmatrix}
    a & b \\
    c & d
\end{pmatrix}
~とする。\\[0.8em]

A~の対角成分は~
\begin{pmatrix}
    a & 0 \\
    0 & d
\end{pmatrix}\\[0.8em]

A~が対角行列、上三角行列、下三角行列となるときそれぞれ\\[0.8em]

\begin{pmatrix}
    a & 0 \\
    0 & a
\end{pmatrix}, \quad

\begin{pmatrix}
    a & b \\
    0 & d
\end{pmatrix}, \quad

\begin{pmatrix}
    a & 0 \\
    c & d
\end{pmatrix}

```latex
%%latex
対角成分がすべて~1~の~n~次スカラー行列を~E_n~または~E~と書き、~n~次単位行列という。 ~I_n~や~I~と書くこともある。\\[0.8em]

E_1 = (1) = 1, \quad E_2 =
\begin{pmatrix}
    1 & 0 \\
    0 & 1
\end{pmatrix}, \quad E_3 =
\begin{pmatrix}
    1 & 0 & 0 \\
    0 & 1 & 0 \\
    0 & 0 & 1
\end{pmatrix}
```

対角成分がすべて~1~の~n~次スカラー行列を~E_n~または~E~と書き、~n~次単位行列という。 ~I_n~や~I~と書くこともある。\\[0.8em]

E_1 = (1) = 1, \quad E_2 =
\begin{pmatrix}
    1 & 0 \\
    0 & 1
\end{pmatrix}, \quad E_3 =
\begin{pmatrix}
    1 & 0 & 0 \\
    0 & 1 & 0 \\
    0 & 0 & 1
\end{pmatrix}

```
# クロネッカーのデルタ
%%latex
クロネッカーのデルタ\\[0.8em]

i,j = 1,2, \cdots, n~ に対して\\[0.8em]

\quad \quad \quad \quad \delta =

\left \{
\begin{array}{cl}
    1 & (i = j) \\
    0 & (i \neq j)
\end{array}
\right .
\\[0.8em]
により、~0~または~1~をとる記号~\delta_{ij}~を定める。 \\
~\delta_{ij}~をクロネッカーのデルタという。
```

クロネッカーのデルタ\\[0.8em]

i,j = 1,2, \cdots, n~ に対して\\[0.8em]

\quad \quad \quad \quad \delta =

\left \{
\begin{array}{cl}
    1 & (i = j) \\
    0 & (i \neq j)
\end{array}
\right .
\\[0.8em]
により、~0~または~1~をとる記号~\delta_{ij}~を定める。 \\
~\delta_{ij}~をクロネッカーのデルタという。

```latex
%%latex
i,j = 1,2 のとき、クロネッカーのデルタ~\delta_{ij}~の値を求める。\\[0.8em]

\quad \quad \quad \quad \delta_{11} = \delta_{22} = 1, \quad \delta_{12} = \delta_{21} = 0
```

i,j = 1,2 のとき、クロネッカーのデルタ~\delta_{ij}~の値を求める。\\[0.8em]

\quad \quad \quad \quad \delta_{11} = \delta_{22} = 1, \quad \delta_{12} = \delta_{21} = 0

```latex
%%latex
クロネッカーのデルタを用いると、~n~次単位行列~E_n~は~E_n = (\delta_{ij})_{n \times n}~ と表すことができる。\\
例えば、~2~次単位行列は\\[0.8em]

\quad \quad \quad \quad E_2 =

\begin{pmatrix}
  \delta_{11} & \delta_{12} \\
  \delta_{21} & \delta_{22}
\end{pmatrix} =

\begin{pmatrix}
  1 & 0 \\
  0 & 1
\end{pmatrix} \\[0.8em]

となる。
```

クロネッカーのデルタを用いると、~n~次単位行列~E_n~は~E_n = (\delta_{ij})_{n \times n}~ と表すことができる。\\
例えば、~2~次単位行列は\\[0.8em]

\quad \quad \quad \quad E_2 =

\begin{pmatrix}
  \delta_{11} & \delta_{12} \\
  \delta_{21} & \delta_{22}
\end{pmatrix} =

\begin{pmatrix}
  1 & 0 \\
  0 & 1
\end{pmatrix} \\[0.8em]

となる。

```latex
%%latex
転置行列\\[0.8em]

m \times n~ 行列~A~の行と列を入れ替えて得られる~n \times m~ 行列を~{}^t A, A^t~ または~{}^T A~ などと書き、\\
~A~の転置行列という。 すなわち、\\[0.8em]
\quad \quad \quad \quad
A =
\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

のとき、\\[0.8em]

\quad \quad \quad \quad
{}^t A =
\begin{pmatrix}
    a_{11} & a_{21} & \ldots & a_{m1} \\
    a_{12} & a_{22} & \ldots & a_{m2} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{1n} & a_{2m} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

である。定義より、\\[0.8em]

\quad \quad \quad \quad
{}^t({}^t A) = A \\[0.8em]

が成り立つ。 なお転置行列を作ることを、転置を取る、ともいう。
```

転置行列\\[0.8em]

m \times n~ 行列~A~の行と列を入れ替えて得られる~n \times m~ 行列を~{}^t A, A^t~ または~{}^T A~ などと書き、\\
~A~の転置行列という。 すなわち、\\[0.8em]
\quad \quad \quad \quad
A =
\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

のとき、\\[0.8em]

\quad \quad \quad \quad
{}^t A =
\begin{pmatrix}
    a_{11} & a_{21} & \ldots & a_{m1} \\
    a_{12} & a_{22} & \ldots & a_{m2} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{1n} & a_{2m} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

である。定義より、\\[0.8em]

\quad \quad \quad \quad
{}^t({}^t A) = A \\[0.8em]

が成り立つ。 なお転置行列を作ることを、転置を取る、ともいう。

```latex
%%latex
例題 \\[0.8em]

2 \times 3~ 行列~
\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}~
の転置行列を求める。 \\[0.8em]
```

例題 \\[0.8em]

2 \times 3~ 行列~
\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}~
の転置行列を求める。

```
from sympy import *
init_printing()
A = Matrix([[1,2,3],[4,5,6]])
display(A)
print()
display (A.T)
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}1 & 4\\2 & 5\\3 & 6\end{matrix}\right]$

```latex
%%latex
対称行列\\[0.8em]

~n~次の正方行列の転置行列は再び~n~次の正方行列となる。よって、次のような正方行列を考えることができる。\\[0.8em]

定義 \quad {}^t A = A が成り立つ正方行列~A~を対称行列という。\\[0.8em]

例 \\[0.8em]

\quad \quad \quad \quad
(1), \quad
\begin{pmatrix}
  1 & 2 \\
  2 & 3
\end{pmatrix} , \quad
\begin{pmatrix}
  1 & 2 & 3\\
  2 & 4 & 5\\
  3 & 5 & 6
\end{pmatrix}
```

対称行列\\[0.8em]

~n~次の正方行列の転置行列は再び~n~次の正方行列となる。よって、次のような正方行列を考えることができる。\\[0.8em]

定義 \quad {}^t A = A が成り立つ正方行列~A~を対称行列という。\\[0.8em]

例 \\[0.8em]

\quad \quad \quad \quad
(1), \quad
\begin{pmatrix}
  1 & 2 \\
  2 & 3
\end{pmatrix} , \quad
\begin{pmatrix}
  1 & 2 & 3\\
  2 & 4 & 5\\
  3 & 5 & 6
\end{pmatrix}

```latex
%%latex
定義より、対称行列について、次の定理が成り立つ。\\[0.8em]

A=(a_{ij})~を~n~次の正方行列とする。 ~A~が対称行列であるための必要十分条件は\\[0.8em]

\quad \quad \quad \quad
a_{ij} = a_{ji} \quad (i,j = 1,2,\cdots, n)\\[0.8em]

である。
```

定義より、対称行列について、次の定理が成り立つ。\\[0.8em]

A=(a_{ij})~を~n~次の正方行列とする。 ~A~が対称行列であるための必要十分条件は\\[0.8em]

\quad \quad \quad \quad
a_{ij} = a_{ji} \quad (i,j = 1,2,\cdots, n)\\[0.8em]

である。

```latex
%%latex
ベクトルと名のつくものを簡単に書くときは太文字、ボールド体を用いる。\\[0.8em]

\quad \quad \quad \quad \mathbf{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

手書きするときは、文字の左側を~2~重にする。\\[0.8em]

\quad \quad \quad \quad \mathbb{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

大文字しか~2~重にならない。
```

ベクトルと名のつくものを簡単に書くときは太文字、ボールド体を用いる。\\[0.8em]

\quad \quad \quad \quad \mathbf{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

手書きするときは、文字の左側を~2~重にする。\\[0.8em]

\quad \quad \quad \quad \mathbb{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

大文字しか~2~重にならない。 => いまの考えテキストが~2~重文字のとき、テキスト重視の考え方から\\
~\mathbb~

```latex
%%latex
問題\\[0.8em]

~3~次の正方行列~A~を~A =
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    a_{21} & a_{22} & a_{23} \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}~と表す。

1.~A~の対角成分を答える。
2.~A~が対角行列、スカラー行列、上三角行列、下三角行列になるとき、~A~をそれぞれ具体的に示す。
```

問題\\[0.8em]

~3~次の正方行列~A~を~A =
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    a_{21} & a_{22} & a_{23} \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}

```latex
%%latex
1.~a_{11},a_{12},a_{13} \\[0.8em]
2.~
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{22} & 0 \\
    0 & 0 & a_{33}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{11} & 0 \\
    0 & 0 & a_{11}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    0 & a_{22} & a_{23} \\
    0 & 0 & a_{33}
\end{pmatrix}~, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    a_{21} & a_{22} & 0 \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}
```

1.~a_{11},a_{12},a_{13} \\[0.8em]
2.~
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{22} & 0 \\
    0 & 0 & a_{33}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{11} & 0 \\
    0 & 0 & a_{11}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    0 & a_{22} & a_{23} \\
    0 & 0 & a_{33}
\end{pmatrix}~, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    a_{21} & a_{22} & 0 \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}

```latex
%%latex
問題 \\[0.8em]

クロネッカーのデルタ~\delta_{ij} \quad (i,j = 1,2, \cdots, n)~ について、~i,j=1,2,3~のとき、~\delta_{ij}~の値を求める。 \\[0.8em]

解答 \\[0.8em]

\delta_{11}=\delta_{22}=\delta_{33}=1, \quad

\delta_{12}=\delta_{13}=\delta_{21}=\delta_{23}=\delta_{31}=\delta_{32}=0

```

問題 \\[0.8em]

クロネッカーのデルタ~\delta_{ij} \quad (i,j = 1,2, \cdots, n)~ について、~i,j=1,2,3~のとき、~\delta_{ij}~の値を求める。 \\[0.8em]

解答 \\[0.8em]

\delta_{11}=\delta_{22}=\delta_{33}=1, \quad

\delta_{12}=\delta_{13}=\delta_{21}=\delta_{23}=\delta_{31}=\delta_{32}=0

```latex
%%latex
問題 \\[0.8em]

~3 \times 2~行列~
\begin{pmatrix}
    5 & 4 \\
    3 & 2 \\
    1 & 0
\end{pmatrix}~の転置行列を求める。
```

問題 \\[0.8em]

~3 \times 2~行列~
\begin{pmatrix}
    5 & 4 \\
    3 & 2 \\
    1 & 0
\end{pmatrix}

```
from sympy import *
from IPython.display import Markdown
init_printing()
display(Markdown("解答"))
print()
display(Matrix([[5,4],[3,2],[1,0]]))
print()
display(Matrix([[5,4],[3,2],[1,0]]).T)
```

解答

$\displaystyle \left[\begin{matrix}5 & 4\\3 & 2\\1 & 0\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}5 & 3 & 1\\4 & 2 & 0\end{matrix}\right]$

```latex
%%latex
問題 \\[0.8em]
\begin{pmatrix}
    a^2+b^2 & ab + bc \\
    ab + bc & b^2 + c^2
\end{pmatrix}
=
\begin{pmatrix}
    1 & 0 \\
    0 &  4
\end{pmatrix}
```

問題 \\[0.8em]
\begin{pmatrix}
    a^2+b^2 & ab + bc \\
    ab + bc & b^2 + c^2
\end{pmatrix}
=
\begin{pmatrix}
    1 & 0 \\
    0 &  4
\end{pmatrix}

```
from sympy import *
from IPython.display import Markdown
init_printing()
display(Markdown("問題"))
a,b,c = symbols('a,b,c')
A = Matrix([[a**2+b**2, a*b+b*c],[a*b+b*c, b**2+c**2]])
display(A)
B = Matrix([[1,0],[0,4]])
display(B)
display(Eq(A,B))
display(Markdown("解答"))
display(solve((A-B),a,b,c))
display(solve((A-B)))
```

```
from sympy import *
from IPython.display import Markdown
init_printing()
display(Markdown("問題"))
a,b,c = symbols('a,b,c')
A = Matrix([[a**2+b**2, a*b+b*c],[a*b+b*c, b**2+c**2],[]])
display(A)
```

問題

$\displaystyle \left[\begin{matrix}a^{2} + b^{2} & a b + b c\\a b + b c & b^{2} + c^{2}\end{matrix}\right]$

# いまここ p.10


```python
from sympy import *
from math import *
import sys

from lib_rungekutta import *
```

# Phương pháp Runge - Kutta hiện giải bài toán Cauchy cho phương trình vi phân thường

## 1. Nhập dữ liệu

Nhập biểu thức $ y' = f(x, y)$. Lưu ý PT phải đặt trong dấu nháy kép

```python
expr = "x"
```

Nhập giá trị ban đầu $x_{0}$ và $y(x_{0})$

```python
x_0 = 0
y_0 = 0
```

Nhập khoảng cách giữa 2 điểm liên tiếp $h$

```python
h = 0.1
```

Nhập số mốc của lưới điểm $n$

```python
n = 150
```

Nhập số nấc của Runge-Kutta. Các giá trị bao gồm 1, 2, 3, 4 hoặc kiểu hệ số của Heun

```python
s = 4
```

## 2. Giải PTVP và in ra kết quả
Cấp chính xác:

```python
precision_x   = 3;
precision_y   = 7;
precision_eps = 12;
```

Giải và in kết quả:

```python
s  = s if(s == "heun" or s == "Heun") else int(s);
uu = rungekutta_oop(expr, x_0, y_0, h, n, s);
g  = (uu.Solve());
```

```python
if(s == "heun" or s == "Heun"):
#{
    print(f"Phương pháp Runge-Kutta 3 nấc với cấu hình hệ số kiểu Heun hoàn tất với lưới điểm sau, sai số toàn cục O(h^3) = {round(h**3, precision_eps)}:");
#}
else:
#{
    print(f"Phương pháp Runge-Kutta {s} nấc với hoàn tất với lưới điểm sau, sai số toàn cục O(h^{s}) = {round(h**s, precision_eps)}:");
#}
for x in g: print(round(x[0], precision_x), round(x[1], precision_y));
```

    Phương pháp Runge-Kutta 4 nấc với hoàn tất với lưới điểm sau, sai số toàn cục O(h^4) = 0.0001:
    0 1
    0.1 1.005
    0.2 1.02
    0.3 1.045
    0.4 1.08
    0.5 1.125
    0.6 1.18
    0.7 1.245
    0.8 1.32
    0.9 1.405
    1.0 1.5
    1.1 1.605
    1.2 1.72
    1.3 1.845
    1.4 1.98
    1.5 2.125
    1.6 2.28
    1.7 2.445
    1.8 2.62
    1.9 2.805
    2.0 3.0
    2.1 3.205
    2.2 3.42
    2.3 3.645
    2.4 3.88
    2.5 4.125
    2.6 4.38
    2.7 4.645
    2.8 4.92
    2.9 5.205
    3.0 5.5
    3.1 5.805
    3.2 6.12
    3.3 6.445
    3.4 6.78
    3.5 7.125
    3.6 7.48
    3.7 7.845
    3.8 8.22
    3.9 8.605
    4.0 9.0
    4.1 9.405
    4.2 9.82
    4.3 10.245
    4.4 10.68
    4.5 11.125
    4.6 11.58
    4.7 12.045
    4.8 12.52
    4.9 13.005
    5.0 13.5
    5.1 14.005
    5.2 14.52
    5.3 15.045
    5.4 15.58
    5.5 16.125
    5.6 16.68
    5.7 17.245
    5.8 17.82
    5.9 18.405
    6.0 19.0
    6.1 19.605
    6.2 20.22
    6.3 20.845
    6.4 21.48
    6.5 22.125
    6.6 22.78
    6.7 23.445
    6.8 24.12
    6.9 24.805
    7.0 25.5
    7.1 26.205
    7.2 26.92
    7.3 27.645
    7.4 28.38
    7.5 29.125
    7.6 29.88
    7.7 30.645
    7.8 31.42
    7.9 32.205
    8.0 33.0
    8.1 33.805
    8.2 34.62
    8.3 35.445
    8.4 36.28
    8.5 37.125
    8.6 37.98
    8.7 38.845
    8.8 39.72
    8.9 40.605
    9.0 41.5
    9.1 42.405
    9.2 43.32
    9.3 44.245
    9.4 45.18
    9.5 46.125
    9.6 47.08
    9.7 48.045
    9.8 49.02
    9.9 50.005
    10.0 51.0
    10.1 52.005
    10.2 53.02
    10.3 54.045
    10.4 55.08
    10.5 56.125
    10.6 57.18
    10.7 58.245
    10.8 59.32
    10.9 60.405
    11.0 61.5
    11.1 62.605
    11.2 63.72
    11.3 64.845
    11.4 65.98
    11.5 67.125
    11.6 68.28
    11.7 69.445
    11.8 70.62
    11.9 71.805
    12.0 73.0
    12.1 74.205
    12.2 75.42
    12.3 76.645
    12.4 77.88
    12.5 79.125
    12.6 80.38
    12.7 81.645
    12.8 82.92
    12.9 84.205
    13.0 85.5
    13.1 86.805
    13.2 88.12
    13.3 89.445
    13.4 90.78
    13.5 92.125
    13.6 93.48
    13.7 94.845
    13.8 96.22
    13.9 97.605
    14.0 99.0
    14.1 100.405
    14.2 101.82
    14.3 103.245
    14.4 104.68
    14.5 106.125
    14.6 107.58
    14.7 109.045
    14.8 110.52
    14.9 112.005
    15.0 113.5

Đồ thị:

```python
uu.getPlot()
```

## 3. Nghiệm kiểm chứng
Nhập PT nghiệm kiểm chứng hoặc "NONE" nếu không có

```python
sol_expr = "NONE"
```

```python
if(sol_expr != "NONE"):
#{
    sol = lambdify(symbols("x"), sympify(sol_expr), "math");
    print("So sánh với giá trị đúng của nghiệm:");
    for x in g: print(f"y*({round(x[0], precision_x)}) = {round(x[1], precision_y)}, so sánh với giá trị chuẩn y({round(x[0], precision_x)}) = {round(sol(x[0]), precision_y)}, sai số {round(abs(sol(x[0]) - x[1]), precision_eps)}");
#}
```

Topic 5 - Solving Differential Equations/28.Runge_Kutta/main_rungekutta.ipynb


```python
import torch
import torch.nn as nn
import torch.distributions as td
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
```

## Environment
Here, we are going to use a simple environment to test our model. Suppose we control a particle (represented by white rectangular) that moves in a 2D plane. We can directly control the velocity of the particle along $x$ and $y$ axis. The task is to control the particle moving towards a goal position (represented by red rectangular) at the center of the plane (see video below).

Now, assume we can not directly observe the position of the particle and the goal, but we can access to some visual observation that is represented as images (see figure below).

Now we are going to learn a state space model of this environment. Given the learnt model, we will be able to forward simulate trajectoris in latent space and plan some actions that minimize the distance to the goal position.

```python
class ParticleEnv:
    def __init__(self):
        self.state = np.array([0, 0])
        self.goal_state = np.array([0.5, 0.5])

        self.action_scale = 0.025
        self.step_count = 0

    def reset(self):
        self.step_count = 0
        self.state = 0.8 * (np.random.rand(2) - 0.5) + 0.5

        reward = -np.sqrt(np.square(self.state - self.goal_state).sum())
        state_image = self.generate_image()

        # potential_goal_states = [[0.0, 0.0], [0.9, 0.9], [0.9, 0.0], [0.0, 0.9]]
        # self.goal_state = np.array(potential_goal_states[random.choice([0, 1, 2, 3])])
        return self.state.copy(), state_image, reward

    def step(self, action):
        self.state += action * self.action_scale
        self.state = np.clip(self.state, a_min=0.0, a_max=1.0)

        reward = -np.sqrt(np.square(self.state - self.goal_state).sum())

        if np.sqrt(np.square(self.state - self.goal_state).sum()) < 0.01:
            done = True
        else:
            done = False

        state_image = self.generate_image()
        self.step_count += 1

        if self.step_count > 10:
            done = True
        return self.state.copy(), state_image, reward, done

    def generate_image(self):
        resolution = 32
        radius = 3
        image_canvas = np.zeros(shape=[3, resolution, resolution])

        pixel_x = int(self.state[0].item() * (resolution - 1))
        pixel_y = int(self.state[1].item() * (resolution - 1))

        for i in range(radius):
            for j in range(radius):
                image_canvas[:, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0
                image_canvas[:, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[:, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[:, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0

        pixel_x = int(self.goal_state[0].item() * (resolution - 1))
        pixel_y = int(self.goal_state[1].item() * (resolution - 1))

        for i in range(radius):
            for j in range(radius):
                image_canvas[0, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0
                image_canvas[0, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[0, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[0, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0

        return image_canvas
```

Here, we define a ReplayBuffer to collect data when interacting with the environment. The ReplayBuffer will record the visual observations (images), actions, reward (negative distance to the goal) and terminal flag at each time step. We also define some neural network template that will be used later.

```python
class ReplayBuffer(object):
    """Buffer to store and replay environment transitions."""

    def __init__(self, obs_shape, action_shape, reward_shape, capacity, batch_size, length, device='cpu'):
        self.capacity = capacity
        self.batch_size = batch_size
        self.length = length
        self.device = device
        # Initialize all the buffers
        self.obs_buffer = np.empty(shape=(capacity, *obs_shape), dtype=np.float32)
        self.action_buffer = np.empty(shape=(capacity, *action_shape), dtype=np.float32)
        self.reward_buffer = np.empty(shape=(capacity, *reward_shape), dtype=np.float32)
        self.done_buffer = np.empty(shape=(capacity, *reward_shape), dtype=np.float32)
        self.idx = 0

    def add(self, obs, action, reward, done):
        if self.idx < self.capacity:
            self.obs_buffer[self.idx] = obs
            self.action_buffer[self.idx] = action
            self.reward_buffer[self.idx] = reward
            self.done_buffer[self.idx] = done
            self.idx += 1
        else:
            self.obs_buffer = self.obs_buffer[1:]
            self.obs_buffer = np.append(self.obs_buffer,
                                        obs.reshape((1, obs.shape[0], obs.shape[1], obs.shape[2])),
                                        axis=0)
            self.action_buffer = self.action_buffer[1:]
            self.action_buffer = np.append(self.action_buffer,
                                           action.reshape((1, action.shape[0])),
                                           axis=0)
            self.reward_buffer = self.reward_buffer[1:]
            self.reward_buffer = np.append(self.reward_buffer,
                                           reward.reshape((1, 1)),
                                           axis=0)
            self.done_buffer = self.done_buffer[1:]
            self.done_buffer = np.append(self.done_buffer,
                                         done.reshape((1, done.shape[0])),
                                         axis=0)

    def sample(self):
        idxs = np.random.randint(
            0, self.capacity - self.length + 1 if self.idx == self.capacity else self.idx - self.length + 1,
            size=self.batch_size)
        obses = torch.as_tensor(self.obs_buffer[idxs], device=self.device).unsqueeze(1).float()
        actions = torch.as_tensor(self.action_buffer[idxs], device=self.device).unsqueeze(1).float()
        rewards = torch.as_tensor(self.reward_buffer[idxs], device=self.device).unsqueeze(1).float()
        dones = torch.as_tensor(self.done_buffer[idxs], device=self.device).unsqueeze(1).float()

        for i in range(1, self.length):
            next_obses = torch.as_tensor(self.obs_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            next_actions = torch.as_tensor(self.action_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            next_rewards = torch.as_tensor(self.reward_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            next_dones = torch.as_tensor(self.done_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            obses = torch.cat((obses, next_obses), 1)
            actions = torch.cat((actions, next_actions), 1)
            rewards = torch.cat((rewards, next_rewards), 1)
            dones = torch.cat((dones, next_dones), 1)

        return obses, actions, rewards, dones

class CNNDenseModel(nn.Module):
    def __init__(self, embed_dim: int, layers: int, h_dim: int,
                 activation=nn.ReLU, min=1e-4, max=10.0):
        super().__init__()
        self._embed_size = embed_dim
        self._layers = layers
        self._hidden_size = h_dim
        self.activation = activation
        self.model = self.build_model()
        self.soft_plus = nn.Softplus()
        self._min = min
        self._max = max
        self.conv_channels = 4

        self.conv_in = nn.Sequential(torch.nn.Conv2d(in_channels=3,
                                                     out_channels=self.conv_channels,
                                                     kernel_size=3,
                                                     stride=3))
        self.fc_out = self.build_model()

    def build_model(self):
        model = [nn.Linear(400, self._hidden_size)]
        model += [self.activation()]
        for i in range(self._layers - 1):
            model += [nn.Linear(self._hidden_size, self._hidden_size)]
            model += [self.activation()]
        model += [nn.Linear(self._hidden_size, self._embed_size)]
        return nn.Sequential(*model)

    def forward(self, obs_visual):
        x_visual = self.conv_in(obs_visual)
        x_visual = x_visual.contiguous()
        x_visual = x_visual.view(-1, self.conv_channels * 10 * 10)
        x = self.fc_out(x_visual)
        return x

class CNNDecoder(torch.nn.Module):
    def __init__(self, z_dim=10, h_dim=64):
        super().__init__()
        self.conv_channels = 4

        self.fc = nn.Sequential(torch.nn.Linear(z_dim, h_dim),
                                torch.nn.ReLU(),
                                torch.nn.Linear(h_dim, self.conv_channels * 10 * 10))
        self.deconv = nn.Sequential(
            torch.nn.ConvTranspose2d(in_channels=self.conv_channels, out_channels=3, kernel_size=5, stride=3))

    def forward(self, x):
        x = self.fc(x)
        h = x.view(-1, self.conv_channels, 10, 10)
        h = self.deconv(h)  # , output_size=(x.size(0), 3, 28, 28))
        return h

class DenseModelNormal(nn.Module):
    def __init__(self, feature_dim: int, output_shape: tuple, layers: int, h_dim: int, activation=nn.ELU,
                 min=1e-4, max=10.0):
        super().__init__()
        self._output_shape = output_shape
        self._layers = layers
        self._hidden_size = h_dim
        self.activation = activation
        # For adjusting pytorch to tensorflow
        self._feature_size = feature_dim
        # Defining the structure of the NN
        self.model = self.build_model()
        self.soft_plus = nn.Softplus()

        self._min = min
        self._max = max

    def build_model(self):
        model = [nn.Linear(self._feature_size, self._hidden_size)]
        model += [self.activation()]
        for i in range(self._layers - 1):
            model += [nn.Linear(self._hidden_size, self._hidden_size)]
            model += [self.activation()]
        model += [nn.Linear(self._hidden_size, 2 * int(np.prod(self._output_shape)))]
        return nn.Sequential(*model)

    def forward(self, features):
        dist_inputs = self.model(features)
        reshaped_inputs_mean = torch.reshape(dist_inputs[..., :np.prod(self._output_shape)],
                                             features.shape[:-1] + self._output_shape)
        reshaped_inputs_std = torch.reshape(dist_inputs[..., np.prod(self._output_shape):],
                                            features.shape[:-1] + self._output_shape)

        reshaped_inputs_std = torch.clamp(self.soft_plus(reshaped_inputs_std), min=self._min, max=self._max)
        return td.independent.Independent(td.Normal(reshaped_inputs_mean, reshaped_inputs_std), len(self._output_shape))
```

## State Space Model
Here, we define our State Space Model (SSM). Intuitively, the SSM models an agent that is sequentially taking actions in a world and receiving rewards and visual observations. The observation $x_t$ at time $t$ visual observations are generated from the latent state $z_t$. The model assumes Markovian transitions where the next state is conditioned upon the current state and the action $a_t$ taken by the agent. Upon taking an action, the agent receives reward $r_t$. Given the graphical structure in the figure below, the SSM's joint distribution factorizes as:

\begin{align}
    &p_\theta(x_{1:T},r_{1:T},z_{0:T},a_{1:T-1}) = \prod_{t=1}^{T} p_\theta(x_{t}|z_{t})p_\theta(r_{t}|z_{t})p_\theta(z_{t}|z_{t-1},a_{t-1})p_\psi(a_{t-1}|z_{t-1})p(z_0)
\end{align}

Each of the factorized distributions are modelled using nonlinear functions:

* Transitions: $p_\theta(z_{t}|z_{t-1},a_{t-1}) = p(z_{t}| f_\theta(z_{t-1},a_{t-1}))$
* Observations: $p_\theta(x_{t}|z_{t}) = p(x_{t}| d_\theta(z_{t})) $
* Rewards: $p_\theta(r_{t}|z_{t}) = p(r_{t}| r_\theta(z_{t})) $

where $f_\theta$, $d^m_\theta$, $r_\theta$, and $\pi_\psi$ are neural networks parameterized by $\theta$. Given nonlinearity of these factorized distributions, the posterior distribution $p(z_{1:T}|x_{1:T},a_{1:T-1})$ is intractable. Thus, we approximate it by

$$p(z_{1:T}|x_{1:T},a_{1:T-1})\approx\hat{q}_{\phi}(z_{1:T}|x_{1:T},a_{1:T-1})=\prod_{t=2}^T q_\phi(z_{t}|g_\phi(x_{t}, z_{t-1}, a_{t-1}))p(z_1)$$

where $q_\phi$ is modeled as a Gaussian distribution and $g_\theta(x_{t}, z_{t-1}, a_{t-1})$ is a neural network parameterized by $\phi$, which is typically called inference network.

Given all these distributions and trajectories of the form $\tau = \left\{(x_{t}, a_{t}, r_{t})\right\}_{t=1}^{T}$ that samples from the ReplayBuffer, we seek to learn the parameters $\theta$ and $\phi$. Because maximum likelihood estimation is intractable in this setting, we optimize the evidence lower bound (ELBO) under the data distribution $p_d$ using a variational distribution $q$ over the latent state variables $z_t$

\begin{align}
\mathbb{E}_{p_d}[\mathrm{ELBO}] \leq \mathbb{E}_{p_d}[\log p_\theta(x_{1:T},r_{1:T}|a_{1:T-1})]
\end{align}

where
\begin{align}
     \mathrm{ELBO} = & \sum_{t=1}^{T}\Big(\displaystyle  \mathop{\mathbb{E}}_{\hat{q}_{\phi}(z_{t})}\left[\log p_{\theta}(x_{t}|z_{t})\right] +\displaystyle\mathop{\mathbb{E}}_{\hat{q}_{\phi}(z_{t})}\left[\log p_{\theta}(r_{t}|z_{t})\right]  \nonumber\\
    &- \displaystyle\mathop{\mathbb{E}}_{\hat{q}_{\phi}(z_{t-1})}\left[\mathrm{KL}\left[\hat{q}_{\phi}(z_{t}) \| p_{\theta}(z_{t}|z_{t-1},a_{t-1})\right]\right]\Big)
\end{align}

```python
class SequentialVAE(nn.Module):
    def __init__(self, z_dim=8, action_dim=2, embed_dim=32):
        super().__init__()
        self.image_embed_net = CNNDenseModel(embed_dim=embed_dim, h_dim=64, layers=2)
        self.inference_net = DenseModelNormal(feature_dim=embed_dim + z_dim + action_dim,
                                              output_shape=(z_dim,),
                                              layers=2,
                                              h_dim=32)

        self.transition_net = DenseModelNormal(feature_dim=z_dim + action_dim,
                                               output_shape=(z_dim,),
                                               layers=2,
                                               h_dim=32)

        self.emission_net = CNNDecoder(z_dim=z_dim, h_dim=32)
        self.reward_net = DenseModelNormal(feature_dim=z_dim,
                                           output_shape=(1,),
                                           layers=2,
                                           h_dim=32)

        self.z_dim = z_dim
        self.action_dim = action_dim

        self.optimizer = torch.optim.Adam(params=self.parameters(), lr=0.002)

        self.anneal_factor = 2.0

    def get_z_post_dist(self, obs, z=None, action=None):

        obs_embed = self.image_embed_net(obs)

        if len(obs.shape) == 3:
            obs_embed = obs_embed[0]

        if z is None:
            z = torch.zeros(obs_embed.shape[:-1] + (self.z_dim,))

        if action is None:
            action = torch.zeros(obs_embed.shape[:-1] + (self.action_dim,))
        z_post_dist = self.inference_net(torch.cat([obs_embed, z, action], dim=-1))
        return z_post_dist

    def cross_entropy_planning(self, z):
        horizon = 4
        sample_size = 100
        discount_factor = 0.8

        action_samples = torch.randint(low=-1, high=2, size=(sample_size, horizon, self.action_dim))

        accumulated_reward = torch.zeros((1, 1))
        z_temp = z.clone().unsqueeze(0).repeat(sample_size, 1)
        for t in range(horizon):
            z_temp_dist = self.transition_net(torch.cat([z_temp, action_samples[:, t]], dim=-1))
            z_temp = z_temp_dist.mean

            reward = self.reward_net(z_temp).mean
            accumulated_reward = accumulated_reward + reward * (discount_factor ** t)

        opti_id = torch.argmax(accumulated_reward, dim=0)
        opti_action_sample = action_samples[opti_id.item()]
        return opti_action_sample[0]

    def learn(self, replay_buffer, episode_count):
        max_epoch = 200
        for i in range(max_epoch):
            obs, action, reward, done = replay_buffer.sample()

            batch_t = obs.shape[1]

            # get latent_state
            z_post_rsample = [[]] * batch_t
            z_post_mean = [[]] * batch_t
            z_post_std = [[]] * batch_t

            for t in range(batch_t):
                if t == 0:
                    z_post_dist = self.get_z_post_dist(obs[:, t])
                else:
                    z_post_dist = self.get_z_post_dist(obs[:, t], z_post_rsample[t - 1], action[:, t - 1])
                z_post_rsample[t] = z_post_dist.rsample()
                z_post_mean[t] = z_post_dist.mean
                z_post_std[t] = z_post_dist.stddev
            z_post_rsample = torch.stack(z_post_rsample, dim=1)
            z_post_mean = torch.stack(z_post_mean, dim=1)
            z_post_std = torch.stack(z_post_std, dim=1)

            z_trans_mean = [[]] * batch_t
            z_trans_std = [[]] * batch_t
            for t in range(batch_t):
                if t == 0:
                    z_trans_mean[t] = z_post_mean[:, t]
                    z_trans_std[t] = z_post_std[:, t]
                else:
                    z_trans_dist = self.transition_net(torch.cat([z_post_rsample[:, t - 1], action[:, t]], dim=-1))
                    z_trans_mean[t] = z_trans_dist.mean
                    z_trans_std[t] = z_trans_dist.stddev
            z_trans_mean = torch.stack(z_trans_mean, dim=1)
            z_trans_std = torch.stack(z_trans_std, dim=1)

            obs_rec = self.emission_net(z_post_rsample)
            obs_rec_loss = 100 * torch.square(obs_rec - obs.view(-1, 3, 32, 32)).mean()

            reward_dist = self.reward_net(z_post_mean.detach())
            reward_rec_loss = -reward_dist.log_prob(reward).mean()

            kl_loss = (z_trans_std.log() - z_post_std.log()
                       + (z_post_std.pow(2) + (z_trans_mean - z_post_mean).pow(2))
                       / (2 * z_trans_std.pow(2) + 1e-5) - 0.5).mean()

            self.optimizer.zero_grad()
            loss = obs_rec_loss + reward_rec_loss + 1.0 * torch.max(kl_loss, torch.ones(1) * self.anneal_factor)
            loss.backward()
            self.optimizer.step()

        self.anneal_factor = np.clip(self.anneal_factor * 0.9, 0.1, 10.0)

        print(
            f'{episode_count}: obs_loss:{obs_rec_loss.item()}, reward_loss:{reward_rec_loss.item()}, kl_loss:{kl_loss.item()}')
```

## Main Loop
Without loss of generality, we begin at time-step $t-1$ where we have a sample of $z_{t-1}$ and taken a action $a_{t-1}$. At time step $t$, we first recieve an visual observation $x_t$ and reward $r_t$. Then,we pass $z_{t-1}$, $a_{t-1}$ and $x_t$ into the variational distribution $q_\phi(z_t|z_{t-1}, a_{t-1}, x_t)$ and sample a $z_t$ from $q_\phi$. Given the sample $z_t$, we optimize the following objective:

\begin{align}
\mathrm{argmax}_{a_{t:t+H-1}} \ \ J = \mathrm{E}_{p(z_{t+1:t+H}\ \ \ \ |a_{t:t+H-1}\ \ \ \ ,z_t)}\left[\sum_{k=t+1}^{t+H} {\gamma^tr(z_t)} \right]
\end{align}

where $p(z_{t+1:t+H}|a_{t:t+H-1},z_t)=\prod_{k=t+1}^{t+H}p_\theta(z_{k}|f_\theta(z_{k-1},a_{k-1}))$ We choose action $a_t$ to execute and replan at each time step. After that,environment fowards to the $t+1$.

```python
def main(test=False):
    env = ParticleEnv()
    replay_buffer = ReplayBuffer(obs_shape=(3, 32, 32),
                                 action_shape=(2,),
                                 reward_shape=(1,),
                                 capacity=1000,
                                 batch_size=50,
                                 length=10)

    model = SequentialVAE(z_dim=5, action_dim=2)
    if test:
        model.load_state_dict(torch.load('./model.pt'))

    model.eval()
    _, obs, reward = env.reset()

    z_post_mean = None
    action = None

    max_episode = 100
    episode_count = 0
    episode_data_size = 200

    visualize_freq = 5

    ims = []
    fig, ax = plt.subplots()
    while True:
        avg_reward = 0.0
        for i in range(episode_data_size):
            z_post_mean = model.get_z_post_dist(torch.as_tensor(obs).float(), z_post_mean, action).mean

            action = model.cross_entropy_planning(z_post_mean)
            _, obs, reward, done = env.step(action.detach().numpy())

            avg_reward += reward
            visualize_image = obs.transpose((1, 2, 0))
            if episode_count % visualize_freq == 0:

                im = ax.imshow(visualize_image, animated=True)
                ims.append([im])

            replay_buffer.add(obs, action, reward, np.array([done]))

            if done:
                _, obs, reward = env.reset()
                z_post_mean = None
                action = None

        avg_reward /= episode_data_size
        print(f'avg_reward:{avg_reward}')

        if episode_count % visualize_freq == 0:
            ani = animation.ArtistAnimation(fig, ims, interval=50, blit=True,
                                            repeat_delay=1000)
            writergif = animation.PillowWriter(fps=30)

            if test:
                file_name = f'./test_episode_{episode_count}.gif'
            else:
                file_name = f'./episode_{episode_count}.gif'
            ani.save(file_name, writer=writergif)
            ims.clear()

        if not test:
            # train model using collected data
            model.train()
            model.learn(replay_buffer, episode_count)
            model.eval()
            torch.save(model.state_dict(), './model.pt')

        episode_count += 1
        if episode_count > max_episode:
            break
```

```python
main(False)
```

```python

```

```python

```


## Forecasting

```python
# Libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline

# Databank
import mysql.connector
from sqlalchemy import create_engine, exc

import itertools ### Modul, welches bestimmte Schleifen ermöglicht
import warnings  ###Modul, welches uns erlaubt, bestimmte Warnungen ein- und auszublenden

#Sonstige Module und Funktionen
from time import gmtime, strftime #Für die aktuelle Zeit

```

```python
#Einstellen der Darstellungsoptionen

#Pandas - Mehr Zeilen und Spalten anzeigen
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)

#Seaborn
sns.set_style("darkgrid")
plt.matplotlib.style.use('default')

my_colors = ["windows blue", "saffron", "hot pink", "algae green", "dusty purple", "greyish", "petrol", "denim blue", "lime"]
sns.set_palette(sns.xkcd_palette(my_colors))
colors = sns.xkcd_palette(my_colors)

#Warnings
warnings.filterwarnings("ignore")

#Connection-String für die Datenbank
str_db_typ = 'mysql+mysqlconnector://'
str_db_user = 'root'
str_db_password = ''
str_db_adr = '@localhost'
str_db_schema = ''
```

```python
### Sammlung aller selbstgeschriebenen Funktionen

#Mein Connection-String
def my_con_str(db_typ='mysql+mysqlconnector', db_user='root', db_passwort='', db_adr = 'localhost', db_schema = ''):
    '''Funktion, die einen DB-Connection-String zusammensetzt'''
    try:
        if len(db_schema) == 0:
            schema_sep = ''
            print('Achtung: Es wurde keine Datenbank angeben.')
        else:
            schema_sep ='/'
        con_str = '{0}://{1}{2}@{3}{4}{5}'.format(db_typ, db_user, db_passwort, db_adr, schema_sep, db_schema)
        print('Connection: {}'.format(con_str))
        return con_str
    except:
        print('Etwas ist schief gelaufen. Bitte alle Parameter überprüfen.')
        con_str = ''
        return con_str

#Anlegen einer neuen Datenbank, falls diese noch nicht da ist.
def my_create_schema(dbname, con):
    engine = create_engine(con)
    engine.execute('CREATE DATABASE IF NOT EXISTS {}'.format(dbname))
    return

#Meine Datenzusammenfassung
def my_df_summary(data):
    '''Eigene Funktion für die Summary'''
    try:
        dat = data.copy()
        df = pd.DataFrame([dat.min(), dat.max(), dat.mean(), dat.std(), dat.isna().sum(), dat.nunique(), dat.dtypes],
                     index=['Minimum', 'Maximum', 'Mittelwert', 'Stand. Abw.','#NA', '#Uniques', 'dtypes'])
        return df
    except:
        print('Es konnte keine Summary erstellt werden.')
        return data

```

### Datenbank einrichten

```python
con = my_con_str()
```

    Achtung: Es wurde keine Datenbank angeben.
    Connection: mysql+mysqlconnector://root@localhost

```python
my_create_schema('timeseries',con)
```

```python
con = my_con_str(db_schema='timeseries')
```

    Connection: mysql+mysqlconnector://root@localhost/timeseries

### Datensatz

```python
#Einlesen und anschauen des zu bearbeitenden Datensatzes
df = pd.read_excel(r'NYSE_sample.xlsx')
```

```python
df.head()
```

      Date
      AABA
      AAPL
      AMZN
      AXP
      BA
      CAT
      CSCO
      CVX
      DIS
      GE
      GOOGL
      GS
      HD
      IBM
      INTC
      JNJ
      JPM
      KO
      MCD
      MMM
      MRK
      MSFT
      NKE
      PFE
      PG
      TRV
      UNH
      UTX
      VZ
      WMT
      XOM

      0
      2006-01-03
      40.91
      10.68
      47.58
      52.58
      70.44
      57.80
      17.45
      59.08
      24.40
      35.37
      217.83
      128.87
      41.24
      82.06
      25.57
      61.63
      40.19
      20.45
      33.52
      79.11
      32.75
      26.84
      10.74
      23.78
      58.78
      45.99
      61.73
      56.53
      30.38
      46.23
      58.47

      1
      2006-01-04
      40.97
      10.71
      47.25
      51.95
      71.17
      59.27
      17.85
      58.91
      23.99
      35.32
      222.84
      127.09
      40.48
      81.95
      25.91
      62.58
      39.62
      20.41
      33.82
      78.71
      33.13
      26.97
      10.69
      24.55
      58.89
      46.50
      61.88
      56.19
      31.27
      46.32
      58.57

      2
      2006-01-05
      41.53
      10.63
      47.65
      52.50
      70.33
      59.27
      18.35
      58.19
      24.41
      35.23
      225.85
      127.04
      40.08
      82.50
      26.27
      62.32
      39.74
      20.51
      33.86
      77.99
      33.18
      26.99
      10.76
      24.58
      58.70
      46.95
      61.69
      55.98
      31.63
      45.69
      58.28

      3
      2006-01-06
      43.21
      10.90
      47.87
      52.68
      69.35
      60.45
      18.77
      59.25
      24.74
      35.47
      233.06
      128.84
      40.38
      84.95
      26.31
      62.60
      40.02
      20.70
      34.06
      78.63
      33.12
      26.91
      10.72
      24.85
      58.64
      47.21
      62.90
      56.16
      31.35
      45.88
      59.43

      4
      2006-01-09
      43.42
      10.86
      47.08
      53.99
      68.77
      61.55
      19.06
      58.95
      25.00
      35.38
      233.68
      130.39
      40.82
      83.73
      26.47
      62.99
      40.67
      20.80
      34.71
      79.02
      33.66
      26.86
      10.88
      24.85
      59.08
      47.23
      61.40
      56.80
      31.48
      45.71
      59.40

```python
df = df.iloc[:,0:6]
```

```python
df.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 3020 entries, 0 to 3019
    Data columns (total 6 columns):
     #   Column  Non-Null Count  Dtype
    ---  ------  --------------  -----
     0   Date    3020 non-null   object
     1   AABA    3019 non-null   float64
     2   AAPL    3019 non-null   float64
     3   AMZN    3019 non-null   float64
     4   AXP     3020 non-null   float64
     5   BA      3020 non-null   float64
    dtypes: float64(5), object(1)
    memory usage: 141.7+ KB

```python
df.describe()
```

      AABA
      AAPL
      AMZN
      AXP
      BA

      count
      3019.000000
      3019.000000
      3019.000000
      3020.000000
      3020.000000

      mean
      28.412726
      64.662892
      299.376231
      59.385315
      101.515868

      std
      13.258163
      44.536448
      279.980161
      19.371901
      47.480903

      min
      8.950000
      7.240000
      26.070000
      10.260000
      29.360000

      25%
      16.130000
      23.420000
      81.090000
      45.715000
      70.600000

      50%
      27.100000
      57.920000
      205.440000
      57.815000
      85.490000

      75%
      36.635000
      99.480000
      375.140000
      75.892500
      130.615000

      max
      72.930000
      176.420000
      1195.830000
      99.700000
      297.900000

```python
my_df_summary(df)
```

      Date
      AABA
      AAPL
      AMZN
      AXP
      BA

      Minimum
      2006-01-03
      8.95
      7.24
      26.07
      10.26
      29.36

      Maximum
      2017-12-29
      72.93
      176.42
      1195.83
      99.7
      297.9

      Mittelwert
      NaN
      28.4127
      64.6629
      299.376
      59.3853
      101.516

      Stand. Abw.
      NaN
      13.2582
      44.5364
      279.98
      19.3719
      47.4809

      #NA
      0
      1
      1
      1
      0
      0

      #Uniques
      3020
      1960
      2579
      2901
      2385
      2556

      dtypes
      object
      float64
      float64
      float64
      float64
      float64

```python
df.plot()
```

```python
#Oder grafisch
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]
fig = df.plot(kind='line')
plt.title('Plot aller numerischen Werte', size=14)
plt.ylabel('Preis in USD')
plt.legend(fontsize=12)
plt.show()
```

### 1. Schritt: In welcher Frequenz liegen meine Zeitreihendaten vor?

```python
df_ = df.copy()

s_Dates = pd.to_datetime(df_.iloc[:,0], format='%Y-%m-%d', errors='ignore')

#Mit "Nummer" des Wochentages
df_NuDay = pd.DataFrame(s_Dates.dt.dayofweek.value_counts(dropna=False))
df_NuDay = df_NuDay.reset_index()
df_NuDay.columns=['#Tag', 'Anzahl']

#Mit "Namen" des Wochentages
df_NaDay = pd.DataFrame(s_Dates.dt.day_name().value_counts(dropna=False))
df_NaDay = df_NaDay.reset_index()
df_NaDay.columns=['Tag', 'Anzahl']
```

```python
df_NaDay
```

      Tag
      Anzahl

      0
      Wednesday
      622

      1
      Tuesday
      618

      2
      Thursday
      609

      3
      Friday
      605

      4
      Monday
      566

```python
#Oder grafisch
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 4]
fig=sns.barplot(x='Tag', y='Anzahl', data=df_NaDay)
plt.title("Wie häufig sind die einzelnen Wochentage enthalten?", size=14)
plt.xlabel("")
plt.ylabel("Häufigkeiten")
plt.show()
```

### Aufbereiten der fehlenden Werte

```python
#Das muss Zeitreihe für Zeitreihe individuell gemacht werden!
#Schritt 0: Auswahl einer Zeitreihe
df_ = df.iloc[:,0:2].copy() #Hier wähle ich den Index (Spalte 0) und die erste Datenspalte (Spalte 1) aus.

# df.iloc[:,[0,4]].copy() #So würde ich bspw. die 4. Datenspalte (Spalte 5) auswählen.
```

```python
df_.head()
```

      Date
      AABA

      0
      2006-01-03
      40.91

      1
      2006-01-04
      40.97

      2
      2006-01-05
      41.53

      3
      2006-01-06
      43.21

      4
      2006-01-09
      43.42

```python
#Cleansing
#Schritt 1:
l_colnames = df_.columns.to_list()
l_colnames[0] = 'Date'
df_.columns = l_colnames

df_['Date'] = pd.to_datetime(df_['Date'], format='%Y-%m-%d', errors='ignore')
daterange = pd.date_range(start=min(df_['Date']), end=max(df_['Date']), freq='B')
df_ts = pd.DataFrame(daterange)
df_ts.columns = ['Date']
df_ts = df_ts.merge(df_, how='left', on='Date')

#Schritt 2
print('{} fehlende Werte werden durch den zuletzt gültigen Wert ersetzt.'.format(df_ts.iloc[:,1].isna().sum()))
df_ts = df_ts.fillna(method='ffill')
df_ts = df_ts.set_index('Date', drop=True)
```

    110 fehlende Werte werden durch den zuletzt gültigen Wert ersetzt.

```python
#Plotten der vollständigen Daten
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]
fig = df_ts.plot(kind='line')
plt.title('Vollständige Zeitreihe', size=18)
plt.legend(fontsize=12)
plt.ylabel('Preis in USD', size=12)
plt.show()
```

### Ausreißer erkennen

```python
#Ausreißer Anzeigen - Für Zeitreihen NICHT entfernen.
#Ausreißer erkennt man in Zeitreihen an dem Verhalten der prozentualen Veränderungen zum jeweils vorherigen Wert.
%matplotlib inline
df_ = df.iloc[:,0:2].copy()
df_.iloc[:,1] = np.log(df_.iloc[:,1]) #Log-Differenzen sind die prozentualen Veränderungen
df_diff = df_.iloc[:,1].diff()

plt.rcParams['figure.figsize'] = [15, 2]

fig = sns.boxplot(data=df_diff, orient='h')

plt.title('Häufigkeiten der prozentualen Wertveränderungen', size=14)
plt.xlabel('')
plt.show()
```

```python
i_iqr_faktor = 2
df_ = df.iloc[:,0:2].copy()
df_['Date'] = pd.to_datetime(df_['Date'], format='%Y-%m-%d', errors='ignore')
df_.iloc[:,1:] = np.log(df_.iloc[:,1:])
df_diff = df_.diff()

q25 = df_diff.iloc[:,1].quantile(0.25)
q75 = df_diff.iloc[:,1].quantile(0.75)

iqr = q75-q25

grenze_unten = q25 - (i_iqr_faktor*iqr)
grenze_oben = q75 + (i_iqr_faktor*iqr)
df_[((df_diff.iloc[:,1] < grenze_unten) | (df_diff.iloc[:,1] > grenze_oben))]

df_dates = df_[((df_diff.iloc[:,1] < grenze_unten) | (df_diff.iloc[:,1] > grenze_oben))]
df_dates = df_dates.reset_index()
df_dates = df_dates.iloc[:,0:2]

print('Bei den eingegebenen Daten und des IQR-Faktors sind Tage, an denen die Wertveränderung < {0:.2f} oder > {1:.2f} war, auffällig.'.format(grenze_unten, grenze_oben))
print('Dies tritt an {} Tagen auf: '.format(len(df_dates)))
df_dates
```

    Bei den eingegebenen Daten und des IQR-Faktors sind Tage, an denen die Wertveränderung < -0.05 oder > 0.06 war, auffällig.
    Dies tritt an 89 Tagen auf:

      index
      Date

      0
      10
      2006-01-18

      1
      73
      2006-04-19

      2
      136
      2006-07-19

      3
      179
      2006-09-19

      4
      265
      2007-01-24

      5
      323
      2007-04-18

      6
      335
      2007-05-04

      7
      450
      2007-10-17

      8
      457
      2007-10-26

      9
      458
      2007-10-29

      10
      465
      2007-11-07

      11
      508
      2008-01-10

      12
      517
      2008-01-24

      13
      521
      2008-01-30

      14
      523
      2008-02-01

      15
      554
      2008-03-18

      16
      586
      2008-05-02

      17
      587
      2008-05-05

      18
      614
      2008-06-12

      19
      630
      2008-07-07

      20
      672
      2008-09-04

      21
      682
      2008-09-18

      22
      684
      2008-09-22

      23
      689
      2008-09-29

      24
      692
      2008-10-02

      25
      696
      2008-10-08

      26
      697
      2008-10-09

      27
      699
      2008-10-13

      28
      700
      2008-10-14

      29
      701
      2008-10-15

      30
      702
      2008-10-16

      31
      705
      2008-10-21

      32
      710
      2008-10-28

      33
      712
      2008-10-30

      34
      718
      2008-11-07

      35
      721
      2008-11-12

      36
      722
      2008-11-13

      37
      725
      2008-11-18

      38
      726
      2008-11-19

      39
      729
      2008-11-24

      40
      732
      2008-11-28

      41
      733
      2008-12-01

      42
      734
      2008-12-02

      43
      740
      2008-12-10

      44
      761
      2009-01-12

      45
      764
      2009-01-15

      46
      772
      2009-01-28

      47
      781
      2009-02-10

      48
      785
      2009-02-17

      49
      790
      2009-02-24

      50
      805
      2009-03-17

      51
      823
      2009-04-13

      52
      833
      2009-04-27

      53
      872
      2009-06-22

      54
      898
      2009-07-29

      55
      1094
      2010-05-10

      56
      1144
      2010-07-21

      57
      1203
      2010-10-13

      58
      1308
      2011-03-15

      59
      1348
      2011-05-11

      60
      1396
      2011-07-20

      61
      1407
      2011-08-04

      62
      1409
      2011-08-08

      63
      1410
      2011-08-09

      64
      1412
      2011-08-11

      65
      1424
      2011-08-29

      66
      1431
      2011-09-08

      67
      1446
      2011-09-29

      68
      1449
      2011-10-04

      69
      1450
      2011-10-05

      70
      1468
      2011-10-31

      71
      1504
      2011-12-21

      72
      1664
      2012-08-10

      73
      1715
      2012-10-23

      74
      1896
      2013-07-17

      75
      2031
      2014-01-29

      76
      2085
      2014-04-16

      77
      2099
      2014-05-07

      78
      2126
      2014-06-16

      79
      2194
      2014-09-22

      80
      2283
      2015-01-29

      81
      2359
      2015-05-19

      82
      2454
      2015-10-02

      83
      2496
      2015-12-02

      84
      2517
      2016-01-04

      85
      2520
      2016-01-07

      86
      2546
      2016-02-16

      87
      2758
      2016-12-15

      88
      2877
      2017-06-08

```python
#Sind einzelne Jahre (Monate) besonders auffällig?
%matplotlib inline
df_dates['Jahr'] = df_dates['Date'].map(lambda x: x.strftime('%Y'))
plt.rcParams['figure.figsize'] = [15, 4]
df_dates.groupby('Jahr').size().plot(kind = 'bar')
plt.xlabel('Periode')
plt.ylabel('Anzahl')
plt.show()
```

```python
#Zu untersuchende Daten auswählen
df_ts_clean = df_ts[df_ts.index > '2013-12-31']
print('Insgesamt liegen {} zusammenhängende ähnliche Beobachtungen vor.'.format(len(df_ts_clean)))
```

    Insgesamt liegen 1043 zusammenhängende ähnliche Beobachtungen vor.

### Erzeugen eines Trainings- und eines Testdatensatzes

```python
#Zerteilung in Trainings- und Testdaten - Im Normalfall beginnt man mit dem Verhältnis 80:20
#Weil wir Tagesdaten haben und mit "klassischen" Zeitreihenanalyseverfahren schwerlich mehr als 60 Perioden mit hoher
#Genauigkeit schätzen kann, teilen wir 95:5.

i_split = int(0.95*len(df_ts_clean))

df_train, df_test = df_ts_clean.iloc[:i_split,:], df_ts_clean.iloc[i_split:,:]

print('Train und Test sind zusammen {} Einträge lang.'.format(len(df_train)+len(df_test)))
print('D.h., alle Forecasts müssen {} Perioden lang sein.'.format(len(df_test)))
```

    Train und Test sind zusammen 1043 Einträge lang.
    D.h., alle Forecasts müssen 53 Perioden lang sein.

```python
#Plot

plt.rcParams['figure.figsize'] = [15, 6]

plt.plot(df_train.index, df_train.values, label='Trainingsdaten')
plt.plot(df_test.index, df_test.values, label='Testdaten', color=colors[1])

plt.axvline(x = df_ts_clean.index[i_split], linewidth=2, color='grey', ls='--')
plt.legend(loc=2, fontsize=10)
plt.title('Aktienkurs (Close) von {} an der NYSE'.format(df_train.columns[0]), fontsize=14)
plt.xlabel('Zeit', fontsize=10)
plt.ylabel('Preis in USD', fontsize=10)
plt.show()
```

```python

```

## Prognose-Verfahren

## Naiver Forecast - Einfaches Fortschreiben des letzten Wertes

\begin{align}
  \hat{y}_{t+1} &= y_{t}
\end{align}

```python
vals = np.asarray(df_train.values)
y_hat = df_test.copy()
y_hat['naiv'] = vals[-1][0]
```

```python
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]

plt.plot(df_train.index, df_train.values, label='Traininsdaten')
plt.plot(df_test.index, df_test.values, label='Testdaten')
plt.plot(y_hat.index, y_hat['naiv'], label='Naiver Forecast')
plt.legend(loc='best')
plt.title("Naiver Forecast")
plt.show()
```

### Wie lässt sich die Güte eines Forecasts bewerten?

### Root Mean Squared Error - RMSE

Tipp: https://en.wikipedia.org/wiki/Root-mean-square_deviation

\begin{align}
\text{RMSE}   \; &=  \sqrt{\frac{\sum_{t=1}^T (\hat{y}_t - y_t)^2}{T}}
\end{align}

Seltener schaut man auch einfach auch den durchschnittlichen Fehler.

\begin{align*}
\text{ME}   \; &=  \frac{\sum_{t=1}^T (\hat{y}_t - y_t)}{T}
\end{align*}

```python
#Importieren der Fehler-Schätzstatistiken aus sklearn
from sklearn.metrics import mean_squared_error
from math import sqrt #Importieren einer Wurzel-Funktion aus math
```

```python
rmse = sqrt(mean_squared_error(df_test, y_hat.naiv))
me = (df_test.iloc[:,0] - y_hat['naiv']).sum() / len(df_test)
print('Für den naiven Forecast ergeben sich ein ME: {0:.4f} und ein RMSE: {1:.4f}.'.format(me,rmse))
```

    Für den naiven Forecast ergeben sich ein ME: 3.2947 und ein RMSE: 3.7385.

Auch kann man die Güte es Forecasts an bestimmten Eigenschaften der Residuen (Schätzfehler) ablesen.

\begin{align*}
\text{Residuen}   \; &=  \hat{y}_t - y_t \quad = \epsilon_t
\end{align*}

```python
residuen = (df_test.iloc[:,0] - y_hat['naiv'])
print('Die Residuen haben für diesen Forecast folgene Standardabweichung: {0:.4f}.'.format(residuen.std()))
stdres = residuen.std()
```

    Die Residuen haben für diesen Forecast folgene Standardabweichung: 1.7835.

```python
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 12]
plt.subplot(2,1,1)
residuen.hist(bins=50, density=True)
plt.subplot(2,1,2)
plt.plot(df_test.index, residuen.values, label='Residuen', linewidth=2)
plt.legend(loc=2)
plt.title("Residuen")
plt.show()
```

### Wie kann man Verfahren denn Vergleichen?

```python
#Anlegen einer Tabelle, um später die Güte verschiedener Verfahren miteinander vergleichen zu können.
#ACHTUNG: Mit dieser Zeile wird ein leerer DataFrame erzeugt.
df_Fehler = pd.DataFrame(columns=['Methode', 'ME', 'RMSE', 'StdRes'])
```

```python
#Einfügen der Güte-Maße
df_Fehler = df_Fehler.append({'Methode': 'Naives Fortschreiben', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

```python
df_Fehler
```

      Methode
      ME
      RMSE
      StdRes

      0
      Naives Fortschreiben
      3.294717
      3.73847
      1.78354

## Moving Average (Gleitender Mittelwert)

```python
#Mit rolling und mean() kann man den gleitenden Mittelwert ganz einfach erzeugen.
#Mit einer Schleife und append schreibt man die Werte fort.

n = 60 #Bspw. den gleitenden Durchschnitt über alle Handelstage des letzten Quartals.
df_train.rolling(n).mean().iloc[-1][0]
```

    63.95716666666658

```python
n = 60
df_mav = df_train.copy()
for i in range(len(df_test)):
    df_mav = df_mav.append({df_mav.columns[0] : df_mav.rolling(n).mean().iloc[-1][0]}, ignore_index=True)
```

```python
y_hat = df_test.copy()
y_hat_mav = df_mav.iloc[-len(df_test):].copy()
y_hat['mav'] = y_hat_mav.values
```

```python
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]

plt.plot(df_train.index, df_train.values, label='Trainingsdaten')
plt.plot(df_test.index, df_test.values, label='Testdaten')
plt.plot(df_test.index, y_hat['mav'].values, label='Moving Average Forecast')

plt.title('Gleitender Durchschnitt')
plt.legend(loc='best')
plt.show()
```

```python
rmse = sqrt(mean_squared_error(df_test, y_hat.mav))
me = (df_test.iloc[:,0] - y_hat['mav']).sum() / len(df_test)
print('Für den naiven Forecast ergeben sich ein ME: {0:.4f} und ein RMSE: {1:.4f}.'.format(me,rmse))

residuen = (df_test.iloc[:,0] - y_hat['mav'])
print('Die Residuen haben für diesen Forecast folgene Standardabweichung: {0:.4f}.'.format(residuen.std()))
stdres = residuen.std()
```

    Für den naiven Forecast ergeben sich ein ME: 4.6000 und ein RMSE: 4.8247.
    Die Residuen haben für diesen Forecast folgene Standardabweichung: 1.4691.

```python
df_Fehler = df_Fehler.append({'Methode': 'Moving Average', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

```python
df_Fehler
```

      Methode
      ME
      RMSE
      StdRes

      0
      Naives Fortschreiben
      3.294717
      3.738470
      1.783540

      1
      Moving Average
      4.600020
      4.824714
      1.469149

```python
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 12]
plt.suptitle('Moving Average')
plt.subplot(2,1,1)
residuen.hist(bins=50, density=True)
plt.subplot(2,1,2)
plt.plot(df_test.index, residuen.values, label='Residuen', linewidth=2)
plt.legend(loc=2)
plt.title("Residuen - Moving Average")
plt.show()
```

Tipp: Wer sich intensiver mit der Mathematik hinter den meisten bekannten und etablierten Verfahren zur Zeitreihenen-Analyse beschäftigen möchte, kann sich u.a. hier einlesen:

https://www.stat.berkeley.edu/~arturof/Teaching/STAT248/lab10_part1.html

### Exponentielle Glättung

Einfache (naive) exponentielle Glättung

Für die einfache exponentielle Glättung gilt:

\begin{align}
  \hat{y}_{t+1} &= \hat{y}_{t} + \alpha ( y_t - \hat{y}_{t})
\end{align}

Weil $\alpha$ zwischen 0 und 1 liegen muss, kann man die Gleichung umschreiben:

\begin{align}
  \hat{y}_{t+1} &=  \alpha y_t + (1-\alpha) \hat{y}_{t}
\end{align}

Und das lässt sich schreiben als:

\begin{align}
  \hat{y}_{t+1} &= \alpha y_{t} + \alpha(1-\alpha) y_{t-1} + \alpha(1-\alpha)^2 y_{t-2} + \cdots + \alpha(1-\alpha)^{t-1} y_{1} + \alpha(1-\alpha)^{t} \hat{y}_{1}
\end{align}

```python
import statsmodels.api as sm
from statsmodels.tsa.api import ExponentialSmoothing, SimpleExpSmoothing, Holt
```

```python
#Forecasts mit verschiedenen Glättungsparametern erzeugen
fit1 = SimpleExpSmoothing(df_train).fit(smoothing_level=0.25,optimized=False)
fcast1 = fit1.forecast(len(df_test)).rename(r'$\alpha=0.25$')

fit2 = SimpleExpSmoothing(df_train).fit(smoothing_level=0.50,optimized=False)
fcast2 = fit2.forecast(len(df_test)).rename(r'$\alpha=0.50$')

fit3 = SimpleExpSmoothing(df_train).fit()
fcast3 = fit3.forecast(len(df_test)).rename(r'$\alpha=%s$'%round(fit3.model.params['smoothing_level'],1))
```

```python
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]

fcast1.plot(legend=True, color=colors[0], ls='--')
fit1.fittedvalues.plot(color=colors[0])

fcast2.plot(legend=True, color=colors[1], ls='--')
fit2.fittedvalues.plot(color=colors[1])

fcast3.plot(legend=True, color=colors[2], ls='--')
fit3.fittedvalues.plot(color=colors[2])

plt.title('Anpassung der einfachen exponentiellen Glättung auf {}'.format(df_train.columns[0]), fontsize=14)
plt.xlabel('Zeit', fontsize=10)
plt.ylabel('Preis in USD', fontsize=10)

plt.show()
```

```python
fit3.params
```

    {'smoothing_level': 0.9779823062246565,
     'smoothing_slope': nan,
     'smoothing_seasonal': nan,
     'damping_slope': nan,
     'initial_level': 40.42145456370888,
     'initial_slope': nan,
     'initial_seasons': array([], dtype=float64),
     'use_boxcox': False,
     'lamda': None,
     'remove_bias': False}

```python
rmse = sqrt(mean_squared_error(df_test, fcast1.values))
me = (df_test.iloc[:,0] - fcast1.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast1.values)
stdres = residuen.std()
print('Forecast 1   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))
```

    Forecast 1   -   RMSE: 2.9550, ME: 2.3688, StdRes: 1.7835

```python
df_Fehler = df_Fehler.append({'Methode': 'EinfExpGlätt 0.25', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

```python
rmse = sqrt(mean_squared_error(df_test, fcast2.values))
me = (df_test.iloc[:,0] - fcast2.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast2.values)
stdres = residuen.std()
print('Forecast 2   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))
```

    Forecast 2   -   RMSE: 3.2135, ME: 2.6843, StdRes: 1.7835

```python
df_Fehler = df_Fehler.append({'Methode': 'EinfExpGlätt 0.5', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

```python
rmse = sqrt(mean_squared_error(df_test, fcast3.values))
me = (df_test.iloc[:,0] - fcast3.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast3.values)
stdres = residuen.std()
print('Forecast 3   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))
```

    Forecast 3   -   RMSE: 3.7143, ME: 3.2672, StdRes: 1.7835

```python
df_Fehler = df_Fehler.append({'Methode': 'EinfExpGlätt 1', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

```python
df_Fehler
```

      Methode
      ME
      RMSE
      StdRes

      0
      Naives Fortschreiben
      3.294717
      3.738470
      1.783540

      1
      Moving Average
      4.600020
      4.824714
      1.469149

      2
      EinfExpGlätt 0.25
      2.368758
      2.954997
      1.783540

      3
      EinfExpGlätt 0.5
      2.684340
      3.213515
      1.783540

      4
      EinfExpGlätt 1
      3.267222
      3.714261
      1.783540

Exponentielle Glättung mit 'Level' und 'Trend' (Holt-Verfahren)

Das Holt-Verfahren nennt man auch zweifache exponentielle Glättung:

\begin{align*}
\text{Level: }   \; \quad \ell_t &=  \alpha y_t + (1-\alpha) (\ell_{t-1} + b_{t-1}) \\ \\
\text{Growth: }   \; \quad b_t &= \beta^* (\ell_t - \ell_{t-1}) + (1-\beta^*) b_{t-1} \\ \\
\text{Forecast: } \; \hat{y}_{t+h|t} &= \ell_t + b_t h \\ \\
\end{align*}

Manchmal wird der Trend auch gedämpft.

\begin{align*}
\text{Level: }   \; \quad \ell_t &=  \alpha y_t + (1-\alpha) (\ell_{t-1} + \phi b_{t-1}) \\ \\
\text{Growth: }   \; \quad b_t &= \beta^* (\ell_t - \ell_{t-1}) + (1-\beta^*) b_{t-1} \\ \\
\text{Forecast: } \; \hat{y}_{t+h|t} &= \ell_t + (\phi + \phi^2 + \cdots + \phi^h) b_t h \\ \\
\end{align*}

```python
#Schätzen der zweifachen exponentiellen Glättung
fit4 = Holt(df_train.iloc[:,0]).fit(smoothing_level=0.8, smoothing_slope=0.2, optimized=False)
fcast4 = fit4.forecast(len(df_test)).rename("Holt's linear trend")

fit5 = Holt(df_train.iloc[:,0], exponential=True).fit(smoothing_level=0.8, smoothing_slope=0.2, optimized=False)
fcast5 = fit5.forecast(len(df_test)).rename("Exponential trend")

fit6 = Holt(df_train.iloc[:,0], damped=True).fit()
fcast6 = fit6.forecast(len(df_test)).rename("Additive damped trend")
```

```python
fit6.params
```

    {'smoothing_level': 0.9044915581422307,
     'smoothing_slope': 1.0,
     'smoothing_seasonal': nan,
     'damping_slope': 0.04365336398134816,
     'initial_level': 40.46649733865921,
     'initial_slope': 0.0,
     'initial_seasons': array([], dtype=float64),
     'use_boxcox': False,
     'lamda': None,
     'remove_bias': False}

```python
#Plotten der Ergebnisse

%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]

fcast4.plot(legend=True, color=colors[3], ls='--')
fit4.fittedvalues.plot(color=colors[3])

fcast5.plot(legend=True, color=colors[4], ls='--')
fit5.fittedvalues.plot(color=colors[4])

fcast6.plot(legend=True, color=colors[5], ls='--')
fit6.fittedvalues.plot(color=colors[5])

plt.title('Anpassung der zweifachen exponentiellen Glättung auf {}'.format(df_train.columns[0]), fontsize=14)
plt.xlabel('Zeit', fontsize=10)
plt.ylabel('Preis in USD', fontsize=10)

plt.show()
```

```python
rmse = sqrt(mean_squared_error(df_test, fcast4.values))
me = (df_test.iloc[:,0] - fcast4.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast4.values)
stdres = residuen.std()
print('Forecast 4   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))

df_Fehler = df_Fehler.append({'Methode': 'Holt LT', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)

rmse = sqrt(mean_squared_error(df_test, fcast5.values))
me = (df_test.iloc[:,0] - fcast5.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast5.values)
stdres = residuen.std()
print('Forecast 5   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))

df_Fehler = df_Fehler.append({'Methode': 'Holt ET', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)

rmse = sqrt(mean_squared_error(df_test, fcast6.values))
me = (df_test.iloc[:,0] - fcast6.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast6.values)
stdres = residuen.std()
print('Forecast 6   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))

df_Fehler = df_Fehler.append({'Methode': 'Holt add, damped trend', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

    Forecast 4   -   RMSE: 9.0572, ME: 8.1390, StdRes: 4.0116
    Forecast 5   -   RMSE: 8.6140, ME: 7.7632, StdRes: 3.7684
    Forecast 6   -   RMSE: 3.6790, ME: 3.2271, StdRes: 1.7836

```python
df_Fehler
```

      Methode
      ME
      RMSE
      StdRes

      0
      Naives Fortschreiben
      3.294717
      3.738470
      1.783540

      1
      Moving Average
      4.600020
      4.824714
      1.469149

      2
      EinfExpGlätt 0.25
      2.368758
      2.954997
      1.783540

      3
      EinfExpGlätt 0.5
      2.684340
      3.213515
      1.783540

      4
      EinfExpGlätt 1
      3.267222
      3.714261
      1.783540

      5
      Holt LT
      8.138998
      9.057173
      4.011575

      6
      Holt ET
      7.763224
      8.613972
      3.768397

      7
      Holt add, damped trend
      3.227057
      3.679006
      1.783596

Exponentielle Glättung mit 'Level', 'Trend' und 'Saisoneffekten' (Holt-Winters)

Das Holt-Winters-Verfahren nennt man (spezielle) dreifache exponentielle Glättung:

\begin{align*}
\text{Level: }   \; \quad \ell_t  &=  \alpha \frac{y_t}{s_{t-m}} + (1-\alpha) (\ell_{t-1} + b_{t-1}) \\\\
\text{Growth: }   \; \quad b_t  &= \beta^* (\ell_t - \ell_{t-1}) + (1-\beta^*) b_{t-1} \\\\
\text{Seasonal: } \; \quad  s_t &= \gamma  \frac{y_t}{\ell_{t-1} + b_{t-1}} + (1-\gamma) s_{t-m} \\\\
\text{Forecast: }\;  \hat{y}_{t+h|t}  &= (\ell_t + b_t h ) s_{t-m+h_m^+} \\
\end{align*}

```python
#Schätzen von Holt-Winters
fit7 = ExponentialSmoothing(df_train.iloc[:,0], seasonal_periods=5, trend='add', seasonal='add').fit()
fcast7 = fit7.forecast(len(df_test)).rename("Holt-Winters Additive")

fit8 = ExponentialSmoothing(df_train.iloc[:,0], seasonal_periods=5, trend='add', seasonal='mul').fit()
fcast8 = fit8.forecast(len(df_test)).rename("Holt-Winters Multiplikativ")

```

```python
#Plotten der Ergebnisse

%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]

fcast7.plot(legend=True, color=colors[1], ls='--')
fit7.fittedvalues.plot(color=colors[2])

fcast8.plot(legend=True, color=colors[3], ls='--')
fit8.fittedvalues.plot(color=colors[4])

plt.title('Anpassung der dreifachen exponentiellen Glättung auf {}'.format(df_train.columns[0]), fontsize=14)
plt.xlabel('Zeit', fontsize=10)
plt.ylabel('Preis in USD', fontsize=10)

plt.show()
```

```python
rmse = sqrt(mean_squared_error(df_test, fcast7.values))
me = (df_test.iloc[:,0] - fcast7.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast7.values)
stdres = residuen.std()
print('Forecast 7   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))

df_Fehler = df_Fehler.append({'Methode': 'Holt Winters add', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)

rmse = sqrt(mean_squared_error(df_test, fcast8.values))
me = (df_test.iloc[:,0] - fcast8.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast8.values)
stdres = residuen.std()
print('Forecast 8   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))

df_Fehler = df_Fehler.append({'Methode': 'Holt Winters mult', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)

```

    Forecast 7   -   RMSE: 2.3152, ME: 1.6727, StdRes: 1.6160
    Forecast 8   -   RMSE: 2.3310, ME: 1.6947, StdRes: 1.6158

```python
df_Fehler
```

      Methode
      ME
      RMSE
      StdRes

      0
      Naives Fortschreiben
      3.294717
      3.738470
      1.783540

      1
      Moving Average
      4.600020
      4.824714
      1.469149

      2
      EinfExpGlätt 0.25
      2.368758
      2.954997
      1.783540

      3
      EinfExpGlätt 0.5
      2.684340
      3.213515
      1.783540

      4
      EinfExpGlätt 1
      3.267222
      3.714261
      1.783540

      5
      Holt LT
      8.138998
      9.057173
      4.011575

      6
      Holt ET
      7.763224
      8.613972
      3.768397

      7
      Holt add, damped trend
      3.227057
      3.679006
      1.783596

      8
      Holt Winters add
      1.672665
      2.315189
      1.616035

      9
      Holt Winters mult
      1.694724
      2.331032
      1.615823

## Faktor-Dekomposition
### Wie bekomme ich heraus, ob und welche Saisonalität in den Daten vorliegt?

```python
%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 8]
sm.tsa.seasonal_decompose(df_train).plot()
plt.show()
```

```python

```

## Autoregressive Prozesse AR, ARMA und ARIMA

Tipp: Eine Einführung in ARIMA-Modelle findet sich hier:

https://towardsdatascience.com/unboxing-arima-models-1dc09d2746f8

Autoregressive Prognose-Modelle

Bei ARIMA-Modellen wird eine Zeitreihe durch einen verzögerten Term der zu erklärenden Variable (AR) und einen Moving-Average-Term (MA) erklärt. Unter bestimmten Umständen muss das Modell insgesamt als integrierte Gleichung (I; ein- oder mehrfach differenziert) optimiert werden.

Dafür gilt es, die optimale Anzahl an Verzögerungen (p), das optimale Momentum (q) und, ggf., das optimale Differenz-Niveau (d) gefunden werden.

\begin{align*}
\text{Forecast:} \; \quad \hat{y}_{t} &=  \mu + \phi_{1}y{t-1}+\cdots+\phi_{p}y_{t-p}-\theta_{1}\epsilon_{t-1}-\cdots-\theta_{q}\epsilon_{t-q} \\\\
\text{Für d = 0:} \; \quad y_t &= Y_t \\\\
\text{Für d = 1:} \; \quad y_t &= Y_t - Y_{t-1} \\\\
\text{Für d = 2:} \; \quad y_t &= (Y_t - Y_{t-1})-(Y_{t-1} - Y_{t-2})
\end{align*}

```python
from statsmodels.tsa.arima_model import ARIMA
warnings.filterwarnings("ignore") # specify to ignore warning messages
```

```python
#Eigene Auto-Arima-Funktion
#Erzeugen einer Liste von allen Parametern, die getestet werden sollen.
p = range(0, 6)
d = range(0, 3)
q = range(0, 5)
pdq = list(itertools.product(p, d, q))
pdq
```

    [(0, 0, 0),
     (0, 0, 1),
     (0, 0, 2),
     (0, 0, 3),
     (0, 0, 4),
     (0, 1, 0),
     (0, 1, 1),
     (0, 1, 2),
     (0, 1, 3),
     (0, 1, 4),
     (0, 2, 0),
     (0, 2, 1),
     (0, 2, 2),
     (0, 2, 3),
     (0, 2, 4),
     (1, 0, 0),
     (1, 0, 1),
     (1, 0, 2),
     (1, 0, 3),
     (1, 0, 4),
     (1, 1, 0),
     (1, 1, 1),
     (1, 1, 2),
     (1, 1, 3),
     (1, 1, 4),
     (1, 2, 0),
     (1, 2, 1),
     (1, 2, 2),
     (1, 2, 3),
     (1, 2, 4),
     (2, 0, 0),
     (2, 0, 1),
     (2, 0, 2),
     (2, 0, 3),
     (2, 0, 4),
     (2, 1, 0),
     (2, 1, 1),
     (2, 1, 2),
     (2, 1, 3),
     (2, 1, 4),
     (2, 2, 0),
     (2, 2, 1),
     (2, 2, 2),
     (2, 2, 3),
     (2, 2, 4),
     (3, 0, 0),
     (3, 0, 1),
     (3, 0, 2),
     (3, 0, 3),
     (3, 0, 4),
     (3, 1, 0),
     (3, 1, 1),
     (3, 1, 2),
     (3, 1, 3),
     (3, 1, 4),
     (3, 2, 0),
     (3, 2, 1),
     (3, 2, 2),
     (3, 2, 3),
     (3, 2, 4),
     (4, 0, 0),
     (4, 0, 1),
     (4, 0, 2),
     (4, 0, 3),
     (4, 0, 4),
     (4, 1, 0),
     (4, 1, 1),
     (4, 1, 2),
     (4, 1, 3),
     (4, 1, 4),
     (4, 2, 0),
     (4, 2, 1),
     (4, 2, 2),
     (4, 2, 3),
     (4, 2, 4),
     (5, 0, 0),
     (5, 0, 1),
     (5, 0, 2),
     (5, 0, 3),
     (5, 0, 4),
     (5, 1, 0),
     (5, 1, 1),
     (5, 1, 2),
     (5, 1, 3),
     (5, 1, 4),
     (5, 2, 0),
     (5, 2, 1),
     (5, 2, 2),
     (5, 2, 3),
     (5, 2, 4)]

```python
ts = df_train.copy()
```

```python
AIC = []
ARIMA_model = []
i = 0

for param in pdq:
    try:
        mod = ARIMA(ts, order=param)
        results = mod.fit()
        print()
        print(i, ": ",'ARIMA{} - AIC:{}'.format(param, results.aic), end='\r')
        AIC.append(results.aic)
        ARIMA_model.append([param])
    except:
        continue
    i = i + 1
```

    0 :  ARIMA(0, 0, 0) - AIC:7013.454578932967
    1 :  ARIMA(0, 0, 1) - AIC:5771.249334363882
    2 :  ARIMA(0, 1, 0) - AIC:2151.2132474557666
    3 :  ARIMA(0, 1, 1) - AIC:2152.659153522236
    4 :  ARIMA(0, 1, 2) - AIC:2154.6282701671657
    5 :  ARIMA(0, 1, 3) - AIC:2154.2940952408876
    6 :  ARIMA(0, 1, 4) - AIC:2156.2916346441207
    7 :  ARIMA(0, 2, 0) - AIC:2855.776649047264
    8 :  ARIMA(0, 2, 1) - AIC:2155.557809569962
    9 :  ARIMA(0, 2, 2) - AIC:2156.873683836948
    10 :  ARIMA(0, 2, 3) - AIC:2158.8664849005554
    11 :  ARIMA(0, 2, 4) - AIC:2158.185413456307
    12 :  ARIMA(1, 0, 0) - AIC:2161.715398545889
    13 :  ARIMA(1, 1, 0) - AIC:2152.652069831722
    14 :  ARIMA(1, 1, 1) - AIC:2154.05849222667
    15 :  ARIMA(1, 1, 2) - AIC:2156.00943161475
    16 :  ARIMA(1, 1, 3) - AIC:2156.283781964605
    17 :  ARIMA(1, 1, 4) - AIC:2158.1436481289193
    18 :  ARIMA(1, 2, 0) - AIC:2554.3582284510103
    19 :  ARIMA(1, 2, 1) - AIC:2156.868439790889
    20 :  ARIMA(1, 2, 2) - AIC:2158.318046812955
    21 :  ARIMA(1, 2, 4) - AIC:2159.039955302013
    22 :  ARIMA(2, 0, 0) - AIC:2163.274614068158
    23 :  ARIMA(2, 1, 0) - AIC:2154.6011925086386
    24 :  ARIMA(2, 1, 1) - AIC:2156.015511628101
    25 :  ARIMA(2, 1, 2) - AIC:2156.9449701306753
    26 :  ARIMA(2, 1, 3) - AIC:2157.324687034196
    27 :  ARIMA(2, 1, 4) - AIC:2159.308318725509
    28 :  ARIMA(2, 2, 0) - AIC:2460.2116081875092
    29 :  ARIMA(2, 2, 1) - AIC:2158.8482822691212
    30 :  ARIMA(2, 2, 4) - AIC:2156.52216639959
    31 :  ARIMA(3, 0, 0) - AIC:2165.1799595355556
    32 :  ARIMA(3, 1, 0) - AIC:2154.555067840616
    33 :  ARIMA(3, 1, 1) - AIC:2156.5490428831517
    34 :  ARIMA(3, 1, 2) - AIC:2157.348646689716
    35 :  ARIMA(3, 1, 3) - AIC:2153.1014330217354
    36 :  ARIMA(3, 1, 4) - AIC:2154.1306360821527
    37 :  ARIMA(3, 2, 0) - AIC:2387.4182292152723
    38 :  ARIMA(3, 2, 1) - AIC:2158.5533428297467
    39 :  ARIMA(3, 2, 2) - AIC:2158.862946838637
    40 :  ARIMA(3, 2, 3) - AIC:2159.023606585547
    41 :  ARIMA(3, 2, 4) - AIC:2163.844070173267
    42 :  ARIMA(4, 0, 0) - AIC:2165.367570202199
    43 :  ARIMA(4, 1, 0) - AIC:2156.5530637506677
    44 :  ARIMA(4, 1, 1) - AIC:2157.5206766635756
    45 :  ARIMA(4, 1, 2) - AIC:2159.348635885285
    46 :  ARIMA(4, 1, 3) - AIC:2154.070742005638
    47 :  ARIMA(4, 1, 4) - AIC:2147.873205912918
    48 :  ARIMA(4, 2, 0) - AIC:2351.2852861599386
    49 :  ARIMA(4, 2, 1) - AIC:2160.5327629534695
    50 :  ARIMA(4, 2, 2) - AIC:2160.8313532391257
    51 :  ARIMA(4, 2, 3) - AIC:2162.708666584628
    52 :  ARIMA(4, 2, 4) - AIC:2157.925120733335
    53 :  ARIMA(5, 0, 0) - AIC:2167.3500849145366
    54 :  ARIMA(5, 1, 0) - AIC:2158.134017272393
    55 :  ARIMA(5, 1, 1) - AIC:2158.0045903186533
    56 :  ARIMA(5, 1, 2) - AIC:2157.36639751832
    57 :  ARIMA(5, 1, 3) - AIC:2155.7195524462245
    58 :  ARIMA(5, 1, 4) - AIC:2152.439435093944
    59 :  ARIMA(5, 2, 0) - AIC:2337.485641686427
    60 :  ARIMA(5, 2, 1) - AIC:2161.9573835202627

```python
print('Das AIC nimmt mit {} für das Modell ARIMA{} den kleinsten Wert an.'.format(min(AIC), ARIMA_model[AIC.index(min(AIC))][0]))
```

    Das AIC nimmt mit 2147.873205912918 für das Modell ARIMA(4, 1, 4) den kleinsten Wert an.

```python
mod = ARIMA(ts,order=ARIMA_model[AIC.index(min(AIC))][0])
fit9 = mod.fit()
```

```python
#Achtung, wenn d != 0, dann werden Differenzen vorhergesagt.
fcast9 = fit9.predict(start=len(ts), end=len(ts)+len(df_test)-1, dynamic=False)
fcast9.head()
```

    2017-10-18    0.112075
    2017-10-19    0.178367
    2017-10-20    0.254370
    2017-10-23    0.075667
    2017-10-24    0.274277
    Freq: B, dtype: float64

```python
y_hat_ = fcast9.copy()
y_hat_[0] = y_hat_[0] + ts.iloc[-1][0]
y_hat_ = np.cumsum(y_hat_.values)
fcast9[:] = y_hat_
```

```python
#Plotten der Ergebnisse

%matplotlib inline
plt.rcParams['figure.figsize'] = [15, 6]

plt.plot(df_train.index, df_train.values, label='Trainingsdaten')
plt.plot(fcast9.index, fcast9.values, label='ARIMA', color=colors[2])

plt.legend(loc=2)
plt.title('Anpassung von autoregressiven Modellen auf {}'.format(df_train.columns[0]), fontsize=14)
plt.xlabel('Zeit', fontsize=10)
plt.ylabel('Preis in USD', fontsize=10)

plt.show()
```

```python
rmse = sqrt(mean_squared_error(df_test, fcast9.values))
me = (df_test.iloc[:,0] - fcast9.values).sum() / len(df_test)
residuen = (df_test.iloc[:,0] - fcast9.values)
stdres = residuen.std()
print('Forecast 7   -   RMSE: {0:.4f}, ME: {1:.4f}, StdRes: {2:.4f}'.format(rmse, me, stdres))

df_Fehler = df_Fehler.append({'Methode': 'ARIMA(4,1,4)', 'ME': me, 'RMSE': rmse, 'StdRes': stdres},
                             ignore_index=True)
```

    Forecast 7   -   RMSE: 2.6475, ME: 2.0076, StdRes: 1.7424

```python
df_Fehler
```

      Methode
      ME
      RMSE
      StdRes

      0
      Naives Fortschreiben
      3.294717
      3.738470
      1.783540

      1
      Moving Average
      4.600020
      4.824714
      1.469149

      2
      EinfExpGlätt 0.25
      2.368758
      2.954997
      1.783540

      3
      EinfExpGlätt 0.5
      2.684340
      3.213515
      1.783540

      4
      EinfExpGlätt 1
      3.267222
      3.714261
      1.783540

      5
      Holt LT
      8.138998
      9.057173
      4.011575

      6
      Holt ET
      7.763224
      8.613972
      3.768397

      7
      Holt add, damped trend
      3.227057
      3.679006
      1.783596

      8
      Holt Winters add
      1.672665
      2.315189
      1.616035

      9
      Holt Winters mult
      1.694724
      2.331032
      1.615823

      10
      ARIMA(0,1,0)
      2.007619
      2.647510
      1.742428

      11
      ARIMA(4,1,4)
      2.007619
      2.647510
      1.742428

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```