```python
# %load /Users/facai/Study/book_notes/preconfig.py
%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
#sns.set(font='SimHei')
plt.rcParams['axes.grid'] = False

#from IPython.display import SVG
def show_image(filename, figsize=None, res_dir=True):
    if figsize:
        plt.figure(figsize=figsize)

    if res_dir:
        filename = './res/{}'.format(filename)

    plt.imshow(plt.imread(filename))
```

Chapter 7 Regularization for Deep Learning
==========================================

the best fitting model is a large model that has been regularized appropriately.

### 7.1 Parameter Norm Penalties

\begin{equation}
    \tilde{J}(\theta; X, y) = J(\theta; X, y) + \alpha \Omega(\theta)
\end{equation}

where $\Omega(\theta)$ is a paramter norm penalty.

typically, penalizes **only the weights** of the affine transformation at each layer and leaves the biases unregularized.

#### 7.1.1 $L^2$ Parameter Regularization

#### 7.1.2 $L^1$ Regularization

The sparsity property induced by $L^1$ regularization => feature selection

### 7.2 Norm Penalties as Constrained Optimization

constrain $\Omega(\theta)$ to be less than some constant $k$:

\begin{equation}
    \mathcal{L}(\theta, \alpha; X, y) = J(\theta; X, y) + \alpha(\Omega(\theta) - k)
\end{equation}

In practice, column norm limitation is always implemented as an explicit constraint with reprojection.

### 7.3 Regularization and Under-Constrained Problems

regularized matrix is guarantedd to be invertible.

### 7.4 Dataset Augmentation

create fake data:

+ transform
+ inject noise

### 7.5 Noise Robustness

+ add noise to data
+ add noise to weight (Bayesian: variable distributaion):
  is equivalent with an additional regularization term.
+ add noise to output target: label smooothing

### 7.6 Semi-Supervised Learning

Goal: learn a representation so that example from the same class have similar representations.

### 7.7 Multi-Task Learning

1. Task-specific paramters
2. Generic parameters

```python
show_image("fig7_2.png")
```

### 7.8 Early Stopping

run it until the ValidationSetError has not imporved for some amount of time.

Use the parameters of the lowest ValidationSetError during the whole train.

```python
show_image("fig7_3.png", figsize=[10, 8])
```

### 7.9 Parameter Tying adn Parameter Sharing

+ regularized the paramters of one model (supervised) to be close to model (unsupervised)
+ to force sets of parameters to be equal: parameter sharing => convolutional neural networks.

### 7.10 Sparse Representations

place a penalty on the activations of the units in a neural network, encouraging their activations to be sparse.

### 7.11 Bagging and Other Ensemble Methods

### 7.12 Dropout

increase the size of the model when using dropout.

small samples, dropout is less effective.

### 7.13 Adversarial Training

```python
show_image("fig7_8.png", figsize=[10, 8])
```

### 7.14 Tangent Distance, Tangent Prop, and Manifold Tangent Classifier

```python

```


# The Jupyter notebook

[IPython](https://ipython.org) provides a **kernel** for [Jupyter](https://jupyter.org).
Jupyter is the name for this notebook interface,
and the document format.

Notebooks can contain [Markdown](https://help.github.com/articles/markdown-basics/) like this cell here,
as well as mathematics rendered with [mathjax](https://mathjax.org):

$$
\frac{1}{\Bigl(\sqrt{\phi \sqrt{5}}-\phi\Bigr) e^{\frac25 \pi}} =
1+\frac{e^{-2\pi}} {1+\frac{e^{-4\pi}} {1+\frac{e^{-6\pi}}
{1+\frac{e^{-8\pi}} {1+\ldots} } } }
$$

```python
!head -n 32 "Intro to IPython.ipynb"
```

[nbviewer](https://nbviewer.org) is a service that renders notebooks to HTML,
for sharing and reading notebooks on the Internet.

[This notebook](https://nbviewer.jupyter.org/github/minrk/inf3331-ipython/blob/master/Intro%20to%20IPython.ipynb) on nbviewer.

You can also convert notebooks to HTML and other formats locally with `jupyter nbconvert`.

# IPython: beyond plain Python

Follow along: https://github.com/minrk/inf3331-ipython

When executing code in IPython, all valid Python syntax works as-is, but IPython provides a number of features designed to make the interactive experience more fluid and efficient.

## First things first: running code, getting help

In the notebook, to run a cell of code, hit `Shift-Enter`. This executes the cell and puts the cursor in the next cell below, or makes a new one if you are at the end.  Alternately, you can use:

- `Alt-Enter` (or `option-Enter`) to force the creation of a new cell unconditionally (useful when inserting new content in the middle of an existing notebook).
- `Control-Enter` executes the cell and keeps the cursor in the same cell, useful for quick experimentation of snippets that you don't need to keep permanently.

```python
print("Hi")
```

```python
import time

for i in range(10):
    print(i, end=' ')
    time.sleep(1)
```

```python
i
```

Getting help:

```python
?
```

Typing `object_name?` will print all sorts of details about any object, including docstrings, function definition lines (for call arguments) and constructor details for classes.

```python
import collections
collections.namedtuple?
```

```python
# copy/paste an example

```

```python
collections.Counter??
```

```python
c = collections.Counter('abcdeabcdabcaba')
```

```python
c.most_common?
```

```python
c.most_common(2)
```

with '\*', you can do a wildcard search:

```python
*int*?
```

```python
import numpy as np
np.*array*?
```

An IPython quick reference card:

```python
%quickref
```

## Tab completion

Tab completion, especially for attributes, is a convenient way to explore the structure of any object you’re dealing with. Simply type `object_name.<TAB>` to view the object’s attributes. Besides Python objects and keywords, tab completion also works on file and directory names.

```python
np.array_equal
```

## The interactive workflow: input, output, history

```python
2+10
```

```python
_+10
```

You can suppress the storage and rendering of output if you append `;` to the last cell (this comes in handy when plotting with matplotlib, for example):

```python
10+20;
```

```python
_
```

The output is stored in `_N` and `Out[N]` variables:

```python
Out[19]
```

`%history` lets you view and search your history

```python
%history -n 1-5
```

**Exercise**

Using this info, how could we write the last 10 lines of history to a file with `%history`? What would be the first thing to try?

```python

```

## Accessing the underlying operating system

[open myfile](myfile.py)

```python
!cat myfile.py
```

```python
import os
print(os.getcwd())
```

```python
import subprocess
p = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE)
stdout, _ = p.communicate()
print(stdout.decode())
```

```python
!pwd
```

```python
!ls -la
```

```python
ls
```

```python
files = !ls
print("My current directory's files:")
print(files)
```

```python
for f in files:
    print(f)
```

```python
!echo $files
```

```python
!echo {files[0].upper()}
```

Note that all this is available even in multiline blocks:

```python
import os
for i,f in enumerate(files):
    if f.endswith('ipynb'):
        !echo {"%02d" % i} - "{os.path.splitext(f)[0]}"
    else:
        print(f'-- {os.path.splitext(f)[0]}')
```

## Beyond Python: magic functions

The IPyhton 'magic' functions are a set of commands, invoked by prepending one or two `%` signs to their name, that live in a namespace separate from your normal Python variables and provide a more command-like interface.  They take flags with `--` and arguments without quotes, parentheses or commas. The motivation behind this system is two-fold:

- To provide an orthogonal namespace for controlling IPython itself and exposing other system-oriented functionality.

- To expose a calling mode that requires minimal verbosity and typing while working interactively.  Thus the inspiration taken from the classic Unix shell style for commands.

Line vs cell magics:

```python
%timeit list(range(1000))
```

```python
%%timeit
list(range(10))
list(range(100))
```

```python
%%html
<b>some bold text</b>
```

Line magics can be used even inside code blocks:

```python
for i in range(1, 4):
    size = i*100
    print('size:', size, end=' ')
    %timeit list(range(size))
```

```python
%timeit time.sleep(0.1)
```

```python
%timeit?
```

Magics can do anything they want with their input, so it doesn't have to be valid Python:

```bash
%%bash
echo "My shell is:" $SHELL
echo "My disk usage is:"
df -h
```

Another interesting cell magic: create any file you want locally from the notebook:

```python
%%writefile test.txt
This is a test file!
It can contain anything I want...

And more...
```

```python
!cat test.txt
```

Let's see what other magics are currently defined in the system:

```python
%lsmagic
```

```python
import math
math.pi
```

```python
%precision 1
math.pi
```

```python
np.random.random(10)
```

```python
%precision?
```

## Running normal Python code: execution and errors

Not only can you input normal Python code, you can even paste straight from a Python or IPython shell session:

```python
>>> # Fibonacci series:
... # the sum of two elements defines the next
... a, b = 0, 1
>>> while b < 10:
...     print(b)
...     a, b = b, a + b

```

And when your code produces errors, you can control how they are displayed with the `%xmode` magic:

```python
%%writefile mod.py

def f(x):
    return 1.0/(x-1)

def g(y):
    return f(y+1)
```

Now let's call the function `g` with an argument that would produce an error:

```python
import mod
mod.g(0)
```

```python
%xmode verbose
```

```python
mod.g(0)
```

## Raw Input in the notebook

Since 1.0 the IPython notebook web application support `raw_input` which for example allow us to invoke the `%debug` magic in the notebook:

```python
%debug
```

Don't foget to exit your debugging session. Raw input can of course be use to ask for user input:

```python
colour = input('What is your favourite colour? ')
print('colour is:', colour)
```

## Running code in other languages with special `%%` magics

```perl
%%perl
@months = ("July", "August", "September");
print $months[0];
```

```ruby
%%ruby
name = "world"
puts "Hello #{name.capitalize}!"
```

## Plotting in the notebook

This magic configures matplotlib to render its figures inline:

```python
%matplotlib inline
```

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
x = np.linspace(0, 2*np.pi, 300)
y = np.sin(x**2)
plt.plot(x, y)
plt.title("A little chirp")
fig = plt.gcf()  # let's keep the figure object around for later...
```

### Widgets

```python
from ipywidgets import interact

@interact
def show_args(num=5, text='hello', check=True):
    print(locals())
```

```python
import sympy
from sympy import Symbol, Eq, factor
x = Symbol('x')
sympy.init_printing(use_latex='mathjax')
x
```

```python
@interact(n=(1,21))
def factorit(n):
    return Eq(x**n-1, factor(x**n-1))
```

```python
@interact(T=(0.0,10), ω=(-1., 10.), n=(5, 512))
def plot_sin(T, ω, n):
    t = np.linspace(0, T, n)
    y = np.sin(ω * t)
    plt.plot(t, y)

```


# Financial Economics HW_03

**11510691 程远星$\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\bbspace}{\;\;\;\;\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\given}[1]{\left. #1 \right|}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N}\left( #1 \right)}
\newcommand{\Exp}[1]{\mathrm{E}\left[ #1 \right]}
\newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]}
\newcommand{\Avar}[1]{\mathrm{Avar}\left[ #1 \right]}
\newcommand{\Cov}[1]{\mathrm{Cov}\left( #1 \right)}
\newcommand{\Corr}[1]{\mathrm{Corr}\left( #1 \right)}
\newcommand{\ExpH}{\mathrm{E}}
\newcommand{\VarH}{\mathrm{Var}}
\newcommand{\AVarH}{\mathrm{Avar}}
\newcommand{\CovH}{\mathrm{Cov}}
\newcommand{\CorrH}{\mathrm{Corr}}
\newcommand{\ow}{\text{otherwise}}
\newcommand{\FSD}{\text{FSD}}
\void^\dagger$**

## Question 8.1

$\bspace$We first write $n$ Eular equations

$$u_0'\P{e_0 - \Tran S \theta} S_k = \sum_{\omega \in\Omega} \pi_\omega u_1'\P{e_{1\omega} + X_{\omega,\cdot} \theta} X_{\omega,k},\bspace k = 1,2,\dots,n$$

$\bspace$whose solution $\theta$ is the optimal portfolio. The fact that $\tilde r_i$ are $i.i.d.$ implies that the preceding equation can be rewritten as, after letting $e_{1\omega} = 0$

$$\begin{align}u_0'\P{e_0 - \Tran S \theta} S_k &= \Exp{u_1'\P{\SB{S_1+S_1\cdot\tilde r_1,S_2+S_2\cdot\tilde r_2, \cdots, S_n + S_n\cdot\tilde r_n}\theta\:} \cdot S_k\P{1+\tilde r_k}}\\
u_0'\P{e_0 - \Tran S \theta}&= \Exp{u_1'\P{\SB{S_1+S_1\cdot\tilde r_1,S_2+S_2\cdot\tilde r_2, \cdots, S_n + S_n\cdot\tilde r_n}\theta\:}} \cdot \Exp{1+\tilde r_k}
\end{align}$$

$\bspace$And now if we consider the variance, unbalanced allocation of securities will lead to higher covariance, and so is the variance, consequently. So the optimal portfolio is to find a $\theta^*$ such that

$$\theta = \theta^*\cdot \iota$$

## Question 8.2

$\P{1}$

$\bspace$Suppose we put $y\cdot w$ money in risk security and the rest in risk free security, then we write

$$\tilde w = y\cdot w \P{1+\tilde r} + \P{1-y}w\P{1+r_F} = w\P{1+r_F + y\P{\tilde r-r_F}}$$

$\bspace$With this we rewrite the optimization problem:

$$\begin{array}{cc}
\d{\max_y} & \Exp{w\P{1+r_F + y\P{\tilde r-r_F}} - \ffrac{1}{2}aw^2\P{1+r_F + y\P{\tilde r-r_F}}^2}
\end{array}$$

$\bspace$And take the derivative on $y$ we have the first order condition:

$$\begin{align}
\ffrac{\partial \Exp{\tilde w - \ffrac{1}{2}a\tilde w^2}}{\partial y} &= \Exp{w\P{\tilde r - r_F} - \ffrac{1}{2} aw^2\cdot 2\P{1+r_F + y\P{\tilde r - r_F}}\P{\tilde r - r_F}}\\
&= w\P{\Exp{\tilde r} - r_F} - aw^2\P{\P{1+r_F}\P{\Exp{\tilde r} - r_F} + y\P{r_F^2 - 2r_F\Exp{\tilde r} + \Exp{\tilde r^2}}}\\
&= w\P{\bar r - r_F} - aw^2\P{\P{1+r_F}\P{\bar r - r_F} + y\P{r_F^2 - 2r_F\bar r + \bar r^2 + \sigma^2}}=0\\
\Rightarrow y &= \ffrac{\ffrac{\bar r - r_F}{aw} - \P{1+r_F}\P{\bar r - r_F}}{\P{\bar r - r_F}^2 + \sigma^2} = \ffrac{\P{\bar r - r_F}\P{1-aw\P{1+r_F}}}{aw\P{\P{\bar r - r_F}^2 + \sigma^2}}
\end{align}$$

$\P{2}$

$\bspace$Here we give the intuitive answers.

- $r_F$ and $y$ are negatively correlated, because agent will tend to put more money in risk free security given they can get more interest
- $\bar r$ and $y$ are positively correlated, because agent will tend to put more money in risk security given they can get more interest, in a probabilistic sense
- $\sigma$ and $y$ are negatively correlated, because rational agent will withdraw money from risk security given higher risk, or volatility
- $a$ and $y$ are negatively correlated, not only can be seen from the explicit expression of $y$, but also the meaning of $a$. Given a increase in $a$ the absolute risk aversion $A\P{w} = \ffrac{a}{1-aw}$ increases meaning that the agent tend to withdraw some money from the risk security.

## Question 8.3

$\P 1$

$\bspace$The optimization problem is now

$$\begin{array}{cc}
\d{\max_y} & \Exp{ -\exp\CB{-aw\P{1+r_F + y\P{\tilde r-r_F}}}}
\end{array}$$

$\bspace$We first simplify this expression

$$\begin{align}
\Exp{ -\exp\CB{-aw\P{1+r_F + y\P{\tilde r-r_F}}}} &= -\exp\CB{-aw\P{1+r_F-yr_F}}\Exp{\exp\CB{-awy \tilde r}}\\
&= -\exp\CB{-aw\P{1+r_F-yr_F}} \cdot \exp\CB{-\mu \cdot awy + \ffrac{\sigma^2}{2}\P{awy}^2}\\
&= -\exp\CB{-aw\P{1+r_F} + aw\P{r_F - \mu}y  + \ffrac{\sigma^2}{2}\P{aw}^2y^2}
\end{align}$$

$\bspace$Since $\exp\CB{-aw\P{1+r_F}}$ is positive we have the simplified optimization problem

$$\begin{array}{cc}
\d{\max_y} & -\exp\CB{aw\P{r_F - \mu}y  + \ffrac{\sigma^2}{2}\P{aw}^2y^2}
\end{array}$$

$\bspace$And take the derivative on $y$ we have the first order condition:

$$\begin{align}
\ffrac{\partial \P{-\exp\CB{aw\P{r_F - \mu}y  + \ffrac{\sigma^2}{2}\P{aw}^2y^2}}}{\partial y} &= -\exp\CB{aw\P{r_F - \mu}y  + \ffrac{\sigma^2}{2}\P{aw}^2y^2}\cdot\P{aw\P{r_F - \mu} + \sigma^2\P{aw}^2y} = 0\\
\Rightarrow \P{aw\P{r_F - \mu} + \sigma^2\P{aw}^2y}&=0 \Rightarrow y = \ffrac{\mu - r_F}{\sigma^2aw}
\end{align}$$

$\P{2}$

$\bspace$Here we give the intuitive answers.

- $r_F$ and $y$ are negatively correlated, because agent will tend to put more money in risk free security given they can get more interest
- $\mu$ and $y$ are positively correlated, because agent will tend to put more money in risk security given they can get more interest, in a probabilistic sense
- $\sigma$ and $y$ are negatively correlated, because rational agent will withdraw money from risk security given higher risk, or volatility
- $a$ and $y$ are negatively correlated, not only can be seen from the explicit expression of $y$, but also the meaning of $a$. Given a increase in $a$ the absolute risk aversion $A\P{w} = a$ increases meaning that the agent tend to withdraw some money from the risk security.

## Question 9.1

$\bspace\newcommand{\FSD}{\text{FSD}}
\newcommand{\SSD}{\text{SSD}}$Consider first stochastic dominance, $A\not\succsim_{\FSD} B$ if $u\P x = x^3$, nor $B\not\succsim_{\FSD} A$ if $u\P x = x+1$.

$\bspace$Then the second stochastic dominance, using the theorem, we let $d=1$ and define $\tilde e$

$$\begin{array}{c|cccc}
& \omega_1 & \omega_2 &\omega_3&\omega_4\\\hline
\tilde e & 0.4 & 0.3 & -0.3 & -0.4\\
p & 0.25 & 0.25 & 0.25 & 0.25\\
\tilde x_A & 1.5 & 1.5 & 1.7 & 1.7
\end{array}$$

$$\begin{align}
\Exp{\tilde e \mid x_A} &= \sum_{i} \tilde e_i \cdot p\P{\tilde e_i\mid x_{A,i}}\\
&= 0.4\times0.25 + 0.3 \times 0.25 - 0.3 \times 0.25 - 0.4 \times 0.25 = 0
\end{align}$$

$\bspace$So $A\succsim_{\SSD} B$.

## Question 9.2

$\P{\text a.1}$

$$\bspace \tilde r_B = \tilde x + \tilde e = 1 \cdot \tilde r_A + \tilde e \Rightarrow \tilde x_B = 1 \cdot \tilde x_A + \tilde e $$

$\bspace$And due to the independency, $\Exp{\tilde e\mid \tilde x_A} = \Exp{\tilde e} = 0$, thus $A\succsim_{\void_\SSD} B$.

$\P{\text a.2}$

$$\begin{align}
\Exp{u\P{\tilde w}} &= \Exp{u\P{a\P{1+\tilde r_A} + \P{1-a}\P{1+\tilde r_B}}}\\
&= \Exp{u\P{1+ a\P{\tilde x - \tilde x - \tilde e} + \tilde x + \tilde e}}\\
&= \Exp{u\P{1+\P{1-a}\tilde e + \tilde x}}
\end{align}$$

$\bspace$Since $\Exp{\tilde e} = 0$, we take the derivative on $a$ and have the following:

$$\begin{align}
\ffrac{\partial^2 \Exp{u\P{\tilde w}}}{\partial a^2} &=\Exp{u''\P{1+\P{1-a}\tilde e + \tilde x}\tilde e^2}\leq 0\\
\ffrac{\partial \Exp{u\P{\tilde w}}}{\partial a} &= \Exp{-u'\P{1+\P{1-a}\tilde e + \tilde x}\tilde e}\\
&= \sum_{e<0}e\cdot P\CB{\tilde e = e}\cdot u'\P{1+\P{1-a}e + \tilde x} + \sum_{e>0}e\cdot P\CB{\tilde e = e}\cdot u'\P{1+\P{1-a}e + \tilde x}\\
&\geq \sum_{e<0}e\cdot P\CB{\tilde e = e}\cdot u'\P{1+\P{1-a}\cdot 0 + \tilde x} + \sum_{e>0}e\cdot P\CB{\tilde e = e}\cdot u'\P{1+\P{1-a}\cdot 0 + \tilde x}\\
& = u'\P{1 + \tilde x}\cdot\Exp{\tilde e} = 0
\end{align}$$

$\bspace$Thus $\Exp{u\P{\tilde w}}$ gets to its maximum at $a=1$. The conclusion is same when $\tilde e$ is continuous.

$\P{\text b.1}$

$\bspace$With the same reasoning, we can prove that $A\succsim_{\void_\SSD} B$.

$\P{\text b.2}$

$\bspace$Now $\Exp{u\P{\tilde w}} = \Exp{u\P{a\tilde x + \P{1-a}\P{\tilde y + \tilde e}}} = \Exp{u\P{\tilde y + \tilde e + a\P{\tilde x - \tilde y - \tilde e}}}$. Still we have its second order direvative on $a$ positive. Then

$$\begin{align}
\ffrac{\partial \Exp{u\P{\tilde w}}}{\partial a} &= \Exp{u'\P{\tilde y + \tilde e + a\P{\tilde x - \tilde y - \tilde e}}\P{\tilde x - \tilde y - \tilde e}}\\
\end{align}$$

$\bspace$At $a = 0.5$, this equals to $\Exp{u'\P{a\P{\tilde x + \tilde y + \tilde e}}\P{\tilde x - \tilde y - \tilde e}} = -\Exp{u'\P{a\P{\tilde x + \tilde y + \tilde e}}\tilde e}$, with the same form we've proved in $\P{\text a.2}$. Thus $\given{\ffrac{\partial \Exp{u\P{\tilde w}}}{\partial a}}_{a=0.5}\geq0$. In order to reach the maximum, we have now $a>0.5$.

$\P{\text{c}}$

$\bspace$The result is intuitively obvious. In $\P{\text a}$, $\tilde r_B$ has got a higher volatility than $\tilde r_A$, so the optimal strategy is to ignore asset $B$. In $\P{\text b}$, we know that in probabilistic sense asset $A$ has a lower volatility than asset $B$, thus, we put more money in $A$ and consequently $a>0.5$.

## Question 9.5

$\bspace$Using $Theorem\space 9.5$, when all $\tilde r_n$ are $i.i.d.$, whatever preference you have, the portfolio you hold is equivelent to a equal-weight portfolio. Thus the utility function of any agent, say $k$, can now be linearly transformed to the utility function of any other agent, say $j$, meaning that the utility functions of all agents, are equivalent, in linear sense. So then we have single fund separation.

## Question 9.6

$\P{1}$

$\bspace$The weights on all $N$ risky securities are $\alpha = \SB{a_1;a_2;\cdots;a_N}$. Then we write

$$\tilde w = w\P{1+r_F} + \sum_{i=1}^N a_i\P{\tilde r_i - r_F} = w\P{1+r_F} + \Tran\alpha\P{\tilde r - r_F\cdot \iota}$$

$\bspace$We then take the derivative on $\alpha$ to derive the first order condition

$\begin{align}
&\ffrac{\partial \Exp{\tilde w - \ffrac{1}{2}a\tilde w^2}}{\partial \alpha}\\
=\;& \ffrac{\partial \Exp{w\P{1+r_F} + \Tran{\P{\tilde r - r_F\cdot\iota}}\alpha - \ffrac{1}{2}a \P{w^2\P{1+r_F}^2 + \Tran\alpha\P{\tilde r - r_F\cdot\iota} \Tran{\P{\tilde r - r_F\cdot\iota}}\alpha + 2w\P{1+r_F}\Tran{\P{\tilde r - r_F\cdot\iota}}\alpha}}}{\partial \alpha}\\
=\;& \Exp{\Tran{\P{\tilde r - r_F\cdot\iota}} - \ffrac{1}{2} a\Tran\alpha\P{\P{\tilde r - r_F\cdot\iota}\Tran{\P{\tilde r - r_F\cdot\iota}} + \P{\tilde r - r_F\cdot\iota}\Tran{\P{\tilde r - r_F\cdot\iota}}} - aw\P{1+r_F}\Tran{\P{\tilde r - r_F\cdot\iota}}}\\
=\;& \Exp{\Tran{\P{\tilde r - r_F\cdot\iota}}\P{1 - aw\P{1+r_F}}- a\Tran\alpha\P{\P{\tilde r - r_F\cdot\iota}\Tran{\P{\tilde r - r_F\cdot\iota}}}}\\
=\;& \Tran{\P{\bar r - r_F \cdot \iota}}\P{1-aw\P{1+r_F}} - a\Tran\alpha \Sigma = 0
\end{align}$

$\bspace$So that $\alpha = \ffrac{1-aw\P{1+r_F}}{a} \Sigma^{-1}\P{\bar r - r_F \cdot \iota}$

$\P{2}$

$\bspace$From $Theorem\space 9.6$, the assertion is obviously true. And the proof is given in Chapter 12, so...

***


```javascript
%%javascript
    MathJax.Hub.Config({
      TeX: { equationNumbers: { autoNumber: "AMS" } }
    });
```

    <IPython.core.display.Javascript object>

```python
from IPython.display import HTML

HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Code Toggle"></form>''')
```

<form action="javascript:code_toggle()"><input type="submit" value="Code Toggle"></form>

```python
from IPython.display import HTML

HTML('''
<a href="{{ site.links.github }}/raw/nist-pages/benchmarks/benchmark7.ipynb"
   download>
<button type="submit">Download Notebook</button>
</a>
''')
```

<a href="{{ site.links.github }}/raw/nist-pages/benchmarks/benchmark7.ipynb"
   download>
<button type="submit">Download Notebook</button>
</a>

# Benchmark Problem 7: MMS Allen-Cahn

```python
from IPython.display import HTML

HTML('''{% include jupyter_benchmark_table.html num="[7]" revision=0 %}''')
```

{% include jupyter_benchmark_table.html num="[7]" revision=0 %}

* [Overview](#Overview)
* [Governing equation and manufactured solution](#Governing-equation-and-manufactured-solution)
* [Domain geometry, boundary conditions, initial conditions, and stopping condition](#Domain-geometry,-boundary-conditions,-initial-conditions,-and-stopping-condition)
* [Parameter values](#Parameter-values)
* [Benchmark simulation instructions](#Benchmark-simulation-instructions)
    * [Part (a)](#Part-%28a%29)
    * [Part (b)](#Part-%28b%29)
    * [Part (c)](#Part-%28c%29)
* [Results](#Results)
* [Feedback](#Feedback)
* [Appendix](#Appendix)
    * [Computer algebra systems](#Computer-algebra-systems)
    * [Source equation](#Source-equation)
    * [Code](#Code)

See the journal publication entitled ["Benchmark problems for numerical implementations of phase field models"][benchmark_paper] for more details about the benchmark problems. Furthermore, read [the extended essay][benchmarks] for a discussion about the need for benchmark problems.

[benchmarks]: ../
[benchmark_paper]: http://dx.doi.org/10.1016/j.commatsci.2016.09.022

# Overview

The Method of Manufactured Solutions (MMS) is a powerful technique for verifying the accuracy of a simulation code. In the MMS, one picks a desired solution to the problem at the outset, the "manufactured solution", and then determines the governing equation that will result in that solution. With the exact analytical form of the solution in hand, when the governing equation is solved using a particular simulation code, the deviation from the expected solution can be determined exactly. This deviation can be converted into an error metric to rigously quantify the error for a calculation. This error can be used to determine the order of accuracy of the simulation results to verify simulation codes. It can also be used to compare the computational efficiency of different codes or different approaches for a particular code at a certain level of error. Furthermore, the spatial/temporal distribution can give insight into the conditions resulting in the largest error (high gradients, changes in mesh resolution, etc.).

After choosing a manufactured solution, the governing equation must be modified to force the solution to equal the manufactured solution. This is accomplished by taking the nominal equation that is to be solved (e.g.  Allen-Cahn equation, Cahn-Hilliard equation, Fick's second law, Laplace equation) and adding a source term. This source term is determined by plugging the manufactured solution into the nominal governing equation and setting the source term equal to the residual. Thus, the manufactured solution satisfies the MMS governing equation (the nominal governing equation plus the source term). A more detailed discussion of MMS can be found in [the report by Salari and Knupp][mms_report].

In this benchmark problem, the objective is to use the MMS to rigorously verify phase field simulation codes and then provide a basis of comparison for the computational performance between codes and for various settings for a single code, as discussed above. To this end, the benchmark problem was chosen as a balance between two factors: simplicity, to minimize the development effort required to solve the benchmark, and transferability to a real phase field system of physical interest.

[mms_report]: http://prod.sandia.gov/techlib/access-control.cgi/2000/001444.pdf

# Governing equation and manufactured solution
For this benchmark problem, we use a simple Allen-Cahn equation as the governing equation

$$\begin{equation}
\frac{\partial \eta}{\partial t} = - \left[ 4 \eta \left(\eta - 1 \right) \left(\eta-\frac{1}{2} \right) - \kappa \nabla^2 \eta \right] + S(x,y,t)
\end{equation}$$

where $S(x,y,t)$ is the MMS source term and $\kappa$ is a constant parameter (the gradient energy coefficient).

The manufactured solution, $\eta_{sol}$ is a hyperbolic tangent function, shifted to vary between 0 and 1, with the $x$ position of the middle of the interface ($\eta_{sol}=0.5$) given by the function $\alpha(x,t)$:

$$\begin{equation}
\eta_{sol}(x,y,t) = \frac{1}{2}\left[ 1 - \tanh\left( \frac{y-\alpha(x,t)}{\sqrt{2 \kappa}} \right) \right]
\end{equation}$$

$$\begin{equation}
\alpha(x,t) = \frac{1}{4} + A_1 t \sin\left(B_1 x \right) + A_2 \sin \left(B_2  x + C_2 t \right)
\end{equation}$$

where $A_1$, $B_1$, $A_2$, $B_2$, and $C_2$ are constant parameters.

This manufactured solution is an equilbrium solution of the governing equation, when $S(x,y,t)=0$ and $\alpha(x,t)$ is constant. The closeness of this manufactured solution to a solution of the nominal governing equation increases the likihood that the behavior of simulation codes when solving this benchmark problem is representive of the solution of the regular Allen-Cahn equation (i.e. without the source term). The form of $\alpha(x,t)$ was chosen to yield complex behavior while still retaining a (somewhat) simple functional form. The two spatial sinusoidal terms introduce two controllable length scales to the interfacial shape. Summing them gives a  "beat" pattern with a period longer than the period of either individual term, permitting a domain size that is larger than the wavelength of the sinusoids without a repeating pattern. The temporal sinusoidal term introduces a controllable time scale to the interfacial shape in addition to the phase transformation time scale, while the linear temporal dependence of the other term ensures that the sinusoidal term can go through multiple periods without $\eta_{sol}$ repeating itself.

Inserting the manufactured solution into the governing equation and solving for $S(x,y,t)$ yields:

$$\begin{equation}
S(x,y,t) = \frac{\text{sech}^2 \left[ \frac{y-\alpha(x,t)}{\sqrt{2 \kappa}} \right]}{4 \sqrt{\kappa}} \left[-2\sqrt{\kappa} \tanh \left[\frac{y-\alpha(x,t)}{\sqrt{2 \kappa}} \right] \left(\frac{\partial \alpha(x,t)}{\partial x} \right)^2+\sqrt{2} \left[ \frac{\partial \alpha(x,t)}{\partial t}-\kappa \frac{\partial^2 \alpha(x,t)}{\partial x^2} \right] \right]
\end{equation}$$

where $\alpha(x,t)$ is given above and where:

$$\begin{equation}
\frac{\partial \alpha(x,t)}{\partial x} = A_1 B_1 t \cos\left(B_1 x\right) + A_2 B_2 \cos \left(B_2  x + C_2 t \right)
\end{equation}$$

$$\begin{equation}
\frac{\partial^2 \alpha(x,t)}{\partial x^2} = -A_1 B_1^2 t \sin\left(B_1 x\right) - A_2 B_2^2 \sin \left(B_2  x + C_2 t \right)
\end{equation}$$

$$\begin{equation}
\frac{\partial \alpha(x,t)}{\partial t} = A_1 \sin\left(B_1 x\right) + A_2 C_2 \cos \left(B_2  x + C_2 t \right)
\end{equation}$$

#### *N.B.*: Don't transcribe these equations. Please download the appropriate files from the [Appendix](#Appendix).

# Domain geometry, boundary conditions, initial conditions, and stopping condition
The domain geometry is a rectangle that spans [0, 1] in $x$ and [0, 0.5] in $y$. This elongated domain was chosen to allow multiple peaks and valleys in $\eta_{sol}$ without stretching the interface too much in the $y$ direction (which causes the thickness of the interface to change) or having large regions where $\eta_{sol}$  never deviates from 0 or 1. Periodic boundary conditions are applied along the $x = 0$ and the $x = 1$ boundaries to accomodate the periodicity of $\alpha(x,t)$. Dirichlet boundary conditions of $\eta$ = 0 and $\eta$ = 1 are applied along the $y = 0$ and the $y = 0.5$ boundaries, respectively. These boundary conditions are chosen to be consistent with $\eta_{sol}(x,y,t)$. The initial condition is the manufactured solution at $t = 0$:

$$
\begin{equation}
\eta_{sol}(x,y,0) = \frac{1}{2}\left[ 1 - \tanh\left( \frac{y-\left(\frac{1}{4}+A_2 \sin(B_2 x) \right)}{\sqrt{2 \kappa}} \right) \right]
\end{equation}
$$

The stopping condition for all calculations is when t = 8 time units, which was chosen to let $\alpha(x,t)$ evolve substantially, while still being slower than the characteristic time for the phase evolution (determined by the CFL condition for a uniform mesh with a reasonable level of resolution of $\eta_{sol}$).

# Parameter values
The nominal parameter values for the governing equation and manufactured solution are given below. The value of $\kappa$ will change in Part (b) in the following section and the values of $\kappa$ and $C_2$ will change in Part (c).

| Parameter | Value |
|-----------|-------|
| $\kappa$  | 0.0004|
| $A_1$     | 0.0075|
| $B_1$     | 0.03  |
| $A_2$     | 8.0   |
| $B_2$     | 22.0  |
| $C_2$     | 0.0625|

# Benchmark simulation instructions
This section describes three sets of tests to conduct using the MMS problem specified above. The primary purpose of the first test is provide a computationally inexpensive problem to verify a simulation code. The second and third tests are more computationally demanding and are primarily designed to serve as a basis for performance comparisons.

## Part (a)
The objective of this test is to verify the accuracy of your simulation code in both time and space. Here, we make use of convergence tests, where either the mesh size (or grid point spacing) or the time step size is systematically changed to determine the response of the error to these quantities. Once a convergence test is completed the order of accuracy can be calculated from the result. The order of accuracy can be compared to the theoretical order of accuracy for the numerical method employed in the simulation. If the two match (to a reasonable degree), then one can be confident that the simulation code is working as expected. The remainder of this subsection will give instructions for convergence tests for this MMS problem.

Implement the MMS problem specified above using the simulation code of your choice. Perform a spatial convergence test by running the simulation for a variety of mesh sizes. For each simulation, determine the discrete $L_2$ norm of the error at $t=8$:

$$\begin{equation}
    L_2 = \sqrt{\sum\limits_{x,y}\left(\eta^{t=8}_{x,y} - \eta_{sol}(x,y,8)\right)^2 \Delta x \Delta y}
\end{equation}$$

For all of these simulations, verify that the time step is small enough that any temporal error is much smaller that the total error. This can be accomplished by decreasing the time step until it has minimal effect on the error. Ensure that at least three simulation results have $L_2$ errors in the range $[5\times10^{-3}, 1\times10^{-4}]$, attempting to cover as much of that range as possible/practical. This maximum and minimum errors in the range roughly represent a poorly resolved simulation and a very well-resolved simulation.

For at least three simulations that have $L_2$ errors in the range $[5\times10^{-3}, 1\times10^{-4}]$, save the effective mesh size and $L_2$ error in a CSV or JSON file. Upload this file to the PFHub website as a 2D data set with the effective mesh size as the x-axis column and the $L_2$ error as the y-axis column. Calculate the effective element size as the square root of the area of the finest part of the mesh for nonuniform meshes. For irregular meshes with continous distributions of element sizes, approximate the effective mesh size as the average of the square root of the area of the smallest 5% of the elements.

Next, confirm that the observed order of accuracy is approximately equal to the expected value. Calculate the order of accuracy, $p$, with a least squares fit of the following function:

$$\begin{equation}
    \log(E)=p \log(R) + b
\end{equation}$$

where $E$ is the $L_2$ error, $R$ is the effective element size, and b is an intercept. Deviations of ±0.2 or more from the theoretical value are to be expected (depending on the range of errors considered and other factors).

Finally, perform a similar convergence test, but for the time step, systematically changing the time step and recording the $L_2$ error. Use a time step that does not vary over the course of any single simulation. Verify that the spatial discretization error is small enough that it does not substantially contribute to the total error. Once again, ensure that at least three simulations have $L_2$ errors in the range $[5\times10^{-3}, 1\times10^{-4}]$, attempting to cover as much of that range as possible/practical. Save the effective mesh size and $L_2$ error for each individual simulation in a CSV or JSON file. [Upload this file to the PFHub website](https://pages.nist.gov/pfhub/simulations/upload_form/) as a 2D data set with the time step size as the x-axis column and the $L_2$ error as the y-axis column. Confirm that the observed order of accuracy is approximately equal to the expected value.

## Part (b)
Now that your code has been verified in (a), the objective of this part is to determine the computational performance of your code at various levels of error. These results can then be used to objectively compare the performance between codes or settings within the same code. To make the problem more computationally demanding and stress solvers more than in (a), decrease $\kappa$ by a factor of $256$ to $1.5625\times10^{-6}$. This change will reduce the interfacial thickness by a factor of $16$.

Run a series of simulations, attempting to optimize solver parameters (mesh, time step, tolerances, etc.) to minimize the required computational resources for at least three levels of $L_2$ error in range  $[5\times10^{-3}, 1\times10^{-5}]$. Use the same CPU and processor type for all simulations. For the best of these simulations, save the wall time, number of computing cores, maximum memory usage, and $L_2$ error for each individual simulation in a CSV or JSON file. [Upload this to the PFHub website](https://pages.nist.gov/pfhub/simulations/upload_form/) as a 3D data set with the wall time as the x-axis column, the number of computing cores as the y-axis column, and the $L_2$ error as the z-axis column. (The PFHub upload system is currently limited to three columns of data. Once this constraint is relaxed, the maximum memory usage data will be incorporated as well.)

<!---For the best of these simulations, submit the wall time, number of computing cores, processor speed, maximum memory usage, and $L_2$ error at $t=8$ to the CHiMaD website.--->

## Part (c)
This final part is designed to stress time integrators even further by increasing the rate of change of $\alpha(x,t)$. Increase $C_2$ to $0.5$. Keep $\kappa= 1.5625\times10^{-6}$ from (b).

Repeat the process from (b), uploading the wall time, number of computing cores, processor speed, maximum memory usage, and $L_2$ error at $t=8$ to the PFHub website.

# Results
Results from this benchmark problem are displayed on the [simulation result page]({{ site.baseurl }}/simulations) for different codes.

# Feedback
Feedback on this benchmark problem is appreciated. If you have questions, comments, or seek clarification, please contact the [CHiMaD phase field community](https://pages.nist.gov/chimad-phase-field/community/) through the [Gitter chat channel](https://gitter.im/usnistgov/chimad-phase-field) or by [email](https://pages.nist.gov/chimad-phase-field/mailing_list/). If you found an error, please file an [issue on GitHub](https://github.com/usnistgov/chimad-phase-field/issues/new).

# Appendix

## Computer algebra systems
Rigorous verification of software frameworks using MMS requires posing the equation and manufacturing the solution with as much complexity as possible. This can be straight-forward, but interesting equations produce complicated source terms. To streamline the MMS workflow, it is strongly recommended that you use a CAS such as SymPy, Maple, or Mathematica to generate source equations and turn it into executable code automatically. For accessibility, we will use [SymPy](http://www.sympy.org/), but so long as vector calculus is supported, and CAS will do.

## Source term

```python
from sympy import Symbol, symbols, simplify
from sympy import Eq, sin, cos, cosh, sinh, tanh, sqrt
from sympy.physics.vector import divergence, gradient, ReferenceFrame, time_derivative
from sympy.printing import pprint
from sympy.abc import kappa, S, t, x, y

# Spatial coordinates: x=R[0], y=R[1], z=R[2]
R = ReferenceFrame('R')

# sinusoid amplitudes
A1, A2 = symbols('A1 A2')
B1, B2 = symbols('B1 B2')
C1, C2 = symbols('C1 C2')

# Define interface offset (alpha)
alpha = (1/4 + A1 * t * sin(B1 * R[0])
             + A2 * sin(B2 * R[0] + C2 * t)
        ).subs({R[0]: x, R[1]: y})

# Define the solution equation (eta)
eta = (1/2 * (1 - tanh((R[1] - alpha) /
                        sqrt(2*kappa)))
      ).subs({R[0]: x, R[1]: y})

# Compute the initial condition
eta0 = eta.subs({t: 0, R[0]: x, R[1]: y})

# Compute the source term from the equation of motion
S = simplify(time_derivative(eta, R)
             + 4 * eta * (eta - 1) * (eta - 1/2)
             - divergence(kappa * gradient(eta, R), R)
            ).subs({R[0]: x, R[1]: y})
```

```python
pprint(Eq(symbols('alpha'), alpha))
```

    α = A₁⋅t⋅sin(B₁⋅x) + A₂⋅sin(B₂⋅x + C₂⋅t) + 0.25

```python
pprint(Eq(symbols('eta'), eta))
```

                  ⎛√2⋅(-A₁⋅t⋅sin(B₁⋅x) - A₂⋅sin(B₂⋅x + C₂⋅t) + y - 0.25)⎞
    η = - 0.5⋅tanh⎜─────────────────────────────────────────────────────⎟ + 0.5
                  ⎝                         2⋅√κ                        ⎠

```python
pprint(Eq(symbols('eta0'), eta0))
```

                   ⎛√2⋅(-A₂⋅sin(B₂⋅x) + y - 0.25)⎞
    η₀ = - 0.5⋅tanh⎜─────────────────────────────⎟ + 0.5
                   ⎝             2⋅√κ            ⎠

```python
pprint(Eq(symbols('S'), S))
```

        ⎛           ⎛√2⋅(A₁⋅t⋅sin(B₁⋅x) + A₂⋅sin(B₂⋅x + C₂⋅t) - y + 0.25)⎞
        ⎜0.5⋅√κ⋅tanh⎜────────────────────────────────────────────────────⎟ - 0.25⋅
        ⎝           ⎝                        2⋅√κ                        ⎠
    S = ──────────────────────────────────────────────────────────────────────────

                                              ⎞ ⎛    2⎛√2⋅(A₁⋅t⋅sin(B₁⋅x) + A₂⋅sin
    √2⋅(A₁⋅sin(B₁⋅x) + A₂⋅C₂⋅cos(B₂⋅x + C₂⋅t))⎟⋅⎜tanh ⎜───────────────────────────
                                              ⎠ ⎝     ⎝                        2⋅√
    ──────────────────────────────────────────────────────────────────────────────
                     √κ

    (B₂⋅x + C₂⋅t) - y + 0.25)⎞    ⎞
    ─────────────────────────⎟ - 1⎟
    κ                        ⎠    ⎠
    ───────────────────────────────

## Code

### Python

Copy the first cell under Source Term directly into your program.
For a performance boost, convert the expressions into lambda functions:
```python
from sympy.utilities.lambdify import lambdify

apy = lambdify([x, y], alpha, modules='sympy')
epy = lambdify([x, y], eta,   modules='sympy')
ipy = lambdify([x, y], eta0,  modules='sympy')
Spy = lambdify([x, y], S,     modules='sympy')
```
#### *N.B.*: You may need to add coefficients to the variables list.

### C

```python
from sympy.utilities.codegen import codegen

[(c_name, c_code), (h_name, c_header)] = codegen([('alpha', alpha),
                                                  ('eta', eta),
                                                  ('eta0', eta0),
                                                  ('S', S)],
                                                 language='C', prefix='MMS', project='PFHub')
print(c_code)
```

    /******************************************************************************
     *                      Code generated with sympy 1.1.1                       *
     *                                                                            *
     *              See http://www.sympy.org/ for more information.               *
     *                                                                            *
     *                        This file is part of 'PFHub'                        *
     ******************************************************************************/
    #include "MMS.h"
    #include <math.h>

    double alpha(double A1, double A2, double B1, double B2, double C2, double t, double x) {

       double alpha_result;
       alpha_result = A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) + 0.25;
       return alpha_result;

    }

    double eta(double A1, double A2, double B1, double B2, double C2, double kappa, double t, double x, double y) {

       double eta_result;
       eta_result = -0.5*tanh((1.0L/2.0L)*sqrt(2)*(-A1*t*sin(B1*x) - A2*sin(B2*x + C2*t) + y - 0.25)/sqrt(kappa)) + 0.5;
       return eta_result;

    }

    double eta0(double A2, double B2, double kappa, double x, double y) {

       double eta0_result;
       eta0_result = -0.5*tanh((1.0L/2.0L)*sqrt(2)*(-A2*sin(B2*x) + y - 0.25)/sqrt(kappa)) + 0.5;
       return eta0_result;

    }

    double S(double A1, double A2, double B1, double B2, double C2, double kappa, double t, double x, double y) {

       double S_result;
       S_result = (0.5*sqrt(kappa)*tanh((1.0L/2.0L)*sqrt(2)*(A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) - y + 0.25)/sqrt(kappa)) - 0.25*sqrt(2)*(A1*sin(B1*x) + A2*C2*cos(B2*x + C2*t)))*(pow(tanh((1.0L/2.0L)*sqrt(2)*(A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) - y + 0.25)/sqrt(kappa)), 2) - 1)/sqrt(kappa);
       return S_result;

    }

### C++

```python
from sympy.printing.cxxcode import cxxcode
```

```python
print("α:")
cxxcode(alpha)
```

    α:

    'A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) + 0.25'

```python
print("η:")
cxxcode(eta)
```

    η:

    '-0.5*tanh((1.0L/2.0L)*std::sqrt(2)*(-A1*t*sin(B1*x) - A2*sin(B2*x + C2*t) + y - 0.25)/std::sqrt(kappa)) + 0.5'

```python
print("η₀:")
cxxcode(eta0)
```

    η₀:

    '-0.5*tanh((1.0L/2.0L)*std::sqrt(2)*(-A2*sin(B2*x) + y - 0.25)/std::sqrt(kappa)) + 0.5'

```python
print("S:")
cxxcode(S)
```

    S:

    '(0.5*std::sqrt(kappa)*tanh((1.0L/2.0L)*std::sqrt(2)*(A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) - y + 0.25)/std::sqrt(kappa)) - 0.25*std::sqrt(2)*(A1*sin(B1*x) + A2*C2*cos(B2*x + C2*t)))*(std::pow(tanh((1.0L/2.0L)*std::sqrt(2)*(A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) - y + 0.25)/std::sqrt(kappa)), 2) - 1)/std::sqrt(kappa)'

### Fortran

```python
from sympy.printing import fcode
```

```python
print("α:")
fcode(alpha)
```

    α:

    '      A1*t*sin(B1*x) + A2*sin(B2*x + C2*t) + 0.25d0'

```python
print("η:")
fcode(eta)
```

    η:

    '      -0.5d0*tanh(0.707106781186548d0*kappa**(-0.5d0)*(-A1*t*sin(B1*x) -\n     @ A2*sin(B2*x + C2*t) + y - 0.25d0)) + 0.5d0'

```python
print("η₀:")
fcode(eta0)
```

    η₀:

    '      -0.5d0*tanh(0.707106781186548d0*kappa**(-0.5d0)*(-A2*sin(B2*x) + y\n     @ - 0.25d0)) + 0.5d0'

```python
print("S:")
fcode(S)
```

    S:

    '      (0.5d0*sqrt(kappa)*tanh(0.707106781186548d0*kappa**(-0.5d0)*(A1*t*\n     @ sin(B1*x) + A2*sin(B2*x + C2*t) - y + 0.25d0)) - 0.25d0*sqrt(\n     @ 2.0d0)*(A1*sin(B1*x) + A2*C2*cos(B2*x + C2*t)))*(tanh(\n     @ 0.707106781186548d0*kappa**(-0.5d0)*(A1*t*sin(B1*x) + A2*sin(B2*x\n     @ + C2*t) - y + 0.25d0))**2 - 1)/sqrt(kappa)'

### Julia

```python
from sympy.printing import julia_code
```

```python
print("α:")
julia_code(alpha)
```

    α:

    'A1.*t.*sin(B1.*x) + A2.*sin(B2.*x + C2.*t) + 0.25'

```python
print("η:")
julia_code(eta)
```

    η:

    '-0.5*tanh(sqrt(2)*(-A1.*t.*sin(B1.*x) - A2.*sin(B2.*x + C2.*t) + y - 0.25)./(2*sqrt(kappa))) + 0.5'

```python
print("η₀:")
julia_code(eta0)
```

    η₀:

    '-0.5*tanh(sqrt(2)*(-A2.*sin(B2.*x) + y - 0.25)./(2*sqrt(kappa))) + 0.5'

```python
print("S:")
julia_code(S)
```

    S:

    '(0.5*sqrt(kappa).*tanh(sqrt(2)*(A1.*t.*sin(B1.*x) + A2.*sin(B2.*x + C2.*t) - y + 0.25)./(2*sqrt(kappa))) - 0.25*sqrt(2)*(A1.*sin(B1.*x) + A2.*C2.*cos(B2.*x + C2.*t))).*(tanh(sqrt(2)*(A1.*t.*sin(B1.*x) + A2.*sin(B2.*x + C2.*t) - y + 0.25)./(2*sqrt(kappa))).^2 - 1)./sqrt(kappa)'

### Mathematica

```python
from sympy.printing import mathematica_code
```

```python
print("α:")
mathematica_code(alpha)
```

    α:

    'A1*t*Sin[B1*x] + A2*Sin[B2*x + C2*t] + 0.25'

```python
print("η:")
mathematica_code(eta)
```

    η:

    '-0.5*Tanh[(1/2)*2^(1/2)*(-A1*t*Sin[B1*x] - A2*Sin[B2*x + C2*t] + y - 0.25)/kappa^(1/2)] + 0.5'

```python
print("η₀:")
mathematica_code(eta0)
```

    η₀:

    '-0.5*Tanh[(1/2)*2^(1/2)*(-A2*Sin[B2*x] + y - 0.25)/kappa^(1/2)] + 0.5'

```python
print("S:")
mathematica_code(S)
```

    S:

    '(0.5*kappa^(1/2)*Tanh[(1/2)*2^(1/2)*(A1*t*Sin[B1*x] + A2*Sin[B2*x + C2*t] - y + 0.25)/kappa^(1/2)] - 0.25*2^(1/2)*(A1*Sin[B1*x] + A2*C2*Cos[B2*x + C2*t]))*(Tanh[(1/2)*2^(1/2)*(A1*t*Sin[B1*x] + A2*Sin[B2*x + C2*t] - y + 0.25)/kappa^(1/2)]^2 - 1)/kappa^(1/2)'

```python

```


```python
%matplotlib inline
```

```python
# Write your imports here
import numpy as np
```

# High-School Maths Exercise
## Getting to Know Jupyter Notebook. Python Libraries and Best Practices. Basic Workflow

### Problem 1. Markdown
Jupyter Notebook is a very light, beautiful and convenient way to organize your research and display your results. Let's play with it for a while.

First, you can double-click each cell and edit its content. If you want to run a cell (that is, execute the code inside it), use Cell > Run Cells in the top menu or press Ctrl + Enter.

Second, each cell has a type. There are two main types: Markdown (which is for any kind of free text, explanations, formulas, results... you get the idea), and code (which is, well... for code :D).

Let me give you a...
#### Quick Introduction to Markdown
##### Text and Paragraphs
There are several things that you can do. As you already saw, you can write paragraph text just by typing it. In order to create a new paragraph, just leave a blank line. See how this works below:
```
This is some text.
This text is on a new line, but it will continue the same paragraph (so you can make your paragraphs more easily readable by just continuing on a new line, or just go on and on like this one line is ever continuing).

This text is displayed in a new paragraph.

And this is yet another paragraph.
```
**Result:**

This is some text.
This text is on a new line, but it will continue the same paragraph (so you can make your paragraphs more easily readable by just continuing on a new line, or just go on and on like this one line is ever continuing).

This text is displayed in a new paragraph.

And this is yet another paragraph.

##### Headings
There are six levels of headings. Level one is the highest (largest and most important), and level 6 is the smallest. You can create headings of several types by prefixing the header line with one to six "#" symbols (this is called a pound sign if you are ancient, or a sharp sign if you're a musician... or a hashtag if you're too young :D). Have a look:
```
# Heading 1
## Heading 2
### Heading 3
#### Heading 4
##### Heading 5
###### Heading 6
```

**Result:**

# Heading 1
## Heading 2
### Heading 3
#### Heading 4
##### Heading 5
###### Heading 6

It is recommended that you have **only one** H1 heading - this should be the header of your notebook (or scientific paper). Below that, you can add your name or just jump to the explanations directly.

##### Emphasis
You can create emphasized (stonger) text by using a **bold** or _italic_ font. You can do this in several ways (using asterisks (\*) or underscores (\_)). In order to "escape" a symbol, prefix it with a backslash (\). You can also strike thorugh your text in order to signify a correction.
```
**bold** __bold__
*italic* _italic_

This is \*\*not \*\* bold.

I ~~didn't make~~ a mistake.
```

**Result:**

**bold** __bold__
*italic* _italic_

This is \*\*not\*\* bold.

I ~~didn't make~~ a mistake.

##### Lists
You can add two types of lists: ordered and unordered. Lists can also be nested inside one another. To do this, press Tab once (it will be converted to 4 spaces).

To create an ordered list, just type the numbers. Don't worry if your numbers are wrong - Jupyter Notebook will create them properly for you. Well, it's better to have them properly numbered anyway...
```
1. This is
2. A list
10. With many
9. Items
    1. Some of which
    2. Can
        3. Be nested
42. You can also
    * Mix
    * list
    * types
```

**Result:**
1. This is
2. A list
10. With many
9. Items
    1. Some of which
    2. Can
        3. Be nested
42. You can also
    * Mix
    * list
    * types

To create an unordered list, type an asterisk, plus or minus at the beginning:
```
* This is
* An
    + Unordered
    - list
```

**Result:**
* This is
* An
    + Unordered
        - list

##### Links
There are many ways to create links but we mostly use one of them: we present links with some explanatory text. See how it works:
```
This is [a link](http://google.com) to Google.
```

**Result:**

This is [a link](http://google.com) to Google.

##### Images
They are very similar to links. Just prefix the image with an exclamation mark. The alt(ernative) text will be displayed if the image is not available. Have a look (hover over the image to see the title text):
```

```

**Result:**

If you want to resize images or do some more advanced stuff, just use HTML.

Did I mention these cells support HTML, CSS and JavaScript? Now I did.

##### Tables
These are a pain because they need to be formatted (somewhat) properly. Here's a good [table generator](http://www.tablesgenerator.com/markdown_tables). Just select File > Paste table data... and provide a tab-separated list of values. It will generate a good-looking ASCII-art table for you.
```
| Cell1 | Cell2 | Cell3 |
|-------|-------|-------|
| 1.1   | 1.2   | 1.3   |
| 2.1   | 2.2   | 2.3   |
| 3.1   | 3.2   | 3.3   |
```

**Result:**

| Cell1 | Cell2 | Cell3 |
|-------|-------|-------|
| 1.1   | 1.2   | 1.3   |
| 2.1   | 2.2   | 2.3   |
| 3.1   | 3.2   | 3.3   |

##### Code
Just use triple backtick symbols. If you provide a language, it will be syntax-highlighted. You can also use inline code with single backticks.

```python
def square(x):
    return x ** 2
```
This is `inline` code. No syntax highlighting here.

**Result:**
```python
def square(x):
    return x ** 2
```
This is `inline` code. No syntax highlighting here.

**Now it's your turn to have some Markdown fun.** In the next cell, try out some of the commands. You can just throw in some things, or do something more structured (like a small notebook).

# Math for devs
## Author: Antonio DIchev

This is a list:
* 1
* 2
* 3

| Cell1 | Cell2 | Cell3 |
|-------|-------|-------|
| 1.1   | 1.2   | 1.3   |
| 2.1   | 2.2   | 2.3   |
| 3.1   | 3.2   | 3.3   |

### Problem 2. Formulas and LaTeX
Writing math formulas has always been hard. But scientists don't like difficulties and prefer standards. So, thanks to Donald Knuth (a very popular computer scientist, who also invented a lot of algorithms), we have a nice typesetting system, called LaTeX (pronounced _lah_-tek). We'll be using it mostly for math formulas, but it has a lot of other things to offer.

There are two main ways to write formulas. You could enclose them in single `$` signs like this: `$ ax + b $`, which will create an **inline formula**: $ ax + b $. You can also enclose them in double `$` signs `$$ ax + b $$` to produce $$ ax + b $$.

Most commands start with a backslash and accept parameters either in square brackets `[]` or in curly braces `{}`. For example, to make a fraction, you typically would write `$$ \frac{a}{b} $$`: $$ \frac{a}{b} $$.

[Here's a resource](http://www.stat.pitt.edu/stoffer/freetex/latex%20basics.pdf) where you can look up the basics of the math syntax. You can also search StackOverflow - there are all sorts of solutions there.

You're on your own now. Research and recreate all formulas shown in the next cell. Try to make your cell look exactly the same as mine. It's an image, so don't try to cheat by copy/pasting :D.

Note that you **do not** need to understand the formulas, what's written there or what it means. We'll have fun with these later in the course.

$$y=ax+b$$

$$ax^2+bx+c=0$$

$$x_{1,2}= \frac{-b\pm \sqrt{b^2-4ac}}{2a}$$

$$f(x)|_{x=a} = f(a)+f'(a)(x-a)+\frac{f''(a)}{2!}(x-a)^2+...+\frac{f^(n)(a)}{n!}(x-a)^n+...$$

$$(x+y)^n=\begin{pmatrix}n\\0\end{pmatrix}x^ny^0+\begin{pmatrix}n\\1\end{pmatrix}x^1y^{n-1}+...\begin{pmatrix}n\\n\end{pmatrix}x^0y^{n}=\sum^n\limits_{k=0}\begin{pmatrix}n\\k\end{pmatrix}x^{n-k}y^{k} $$

$$\int_{-\infty}^{+\infty}e^{-x^2}dx=\sqrt\pi$$

$$\begin{pmatrix}
2 & 1 & 3 \\
2 & 6 & 8 \\
6 & 8 & 18
\end{pmatrix} $$

$$A = \begin{bmatrix}
    a_{11} & a_{12} & \dots & a_{1n}  \\
    a_{21} & a_{22} & \dots & a_{2n}  \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \dots & a_{mn}  \\
    \end{bmatrix}$$

### Problem 3. Solving with Python
Let's first do some symbolic computation. We need to import `sympy` first.

**Should your imports be in a single cell at the top or should they appear as they are used?** There's not a single valid best practice. Most people seem to prefer imports at the top of the file though. **Note: If you write new code in a cell, you have to re-execute it!**

Let's use `sympy` to give us a quick symbolic solution to our equation. First import `sympy` (you can use the second cell in this notebook):
```python
import sympy
```

Next, create symbols for all variables and parameters. You may prefer to do this in one pass or separately:
```python
x = sympy.symbols('x')
a, b, c = sympy.symbols('a b c')
```

Now solve:
```python
sympy.solve(a * x**2 + b * x + c)
```

Hmmmm... we didn't expect that :(. We got an expression for $a$ because the library tried to solve for the first symbol it saw. This is an equation and we have to solve for $x$. We can provide it as a second paramter:
```python
sympy.solve(a * x**2 + b * x + c, x)
```

Finally, if we use `sympy.init_printing()`, we'll get a LaTeX-formatted result instead of a typed one. This is very useful because it produces better-looking formulas.

How about a function that takes $a, b, c$ (assume they are real numbers, you don't need to do additional checks on them) and returns the **real** roots of the quadratic equation?

Remember that in order to calculate the roots, we first need to see whether the expression under the square root sign is non-negative.

If $b^2 - 4ac > 0$, the equation has two real roots: $x_1, x_2$

If $b^2 - 4ac = 0$, the equation has one real root: $x_1 = x_2$

If $b^2 - 4ac < 0$, the equation has zero real roots

Write a function which returns the roots. In the first case, return a list of 2 numbers: `[2, 3]`. In the second case, return a list of only one number: `[2]`. In the third case, return an empty list: `[]`.

```python
import math
def solve_quadratic_equation(a, b, c):
    """
    Returns the real solutions of the quadratic equation ax^2 + bx + c = 0
    """
    D = b**2 - 4*a*c

    if D < 0:
        return []
    elif D == 0:
        x = (-b+math.sqrt(b**2-4*a*c))/2*a
        return x
    else:
        x1 = (-b-math.sqrt(b**2-4*a*c))/2*a
        x2 = (-b+math.sqrt(b**2-4*a*c))/2*a
        return x1, x2
```

```python
# Testing: Execute this cell. The outputs should match the expected outputs. Feel free to write more tests
print(solve_quadratic_equation(1, -1, -2)) # [-1.0, 2.0]
print(solve_quadratic_equation(1, -8, 16)) # [4.0]
print(solve_quadratic_equation(1, 1, 1)) # []
```

    (-1.0, 2.0)
    4.0
    []

**Bonus:** Last time we saw how to solve a linear equation. Remember that linear equations are just like quadratic equations with $a = 0$. In this case, however, division by 0 will throw an error. Extend your function above to support solving linear equations (in the same way we did it last time).

### Problem 4. Equation of a Line
Let's go back to our linear equations and systems. There are many ways to define what "linear" means, but they all boil down to the same thing.

The equation $ax + b = 0$ is called *linear* because the function $f(x) = ax+b$ is a linear function. We know that there are several ways to know what one particular function means. One of them is to just write the expression for it, as we did above. Another way is to **plot** it. This is one of the most exciting parts of maths and science - when we have to fiddle around with beautiful plots (although not so beautiful in this case).

The function produces a straight line and we can see it.

How do we plot functions in general? Ww know that functions take many (possibly infinitely many) inputs. We can't draw all of them. We could, however, evaluate the function at some points and connect them with tiny straight lines. If the points are too many, we won't notice - the plot will look smooth.

Now, let's take a function, e.g. $y = 2x + 3$ and plot it. For this, we're going to use `numpy` arrays. This is a special type of array which has two characteristics:
* All elements in it must be of the same type
* All operations are **broadcast**: if `x = [1, 2, 3, 10]` and we write `2 * x`, we'll get `[2, 4, 6, 20]`. That is, all operations are performed at all indices. This is very powerful, easy to use and saves us A LOT of looping.

There's one more thing: it's blazingly fast because all computations are done in C, instead of Python.

First let's import `numpy`. Since the name is a bit long, a common convention is to give it an **alias**:
```python
import numpy as np
```

Import that at the top cell and don't forget to re-run it.

Next, let's create a range of values, e.g. $[-3, 5]$. There are two ways to do this. `np.arange(start, stop, step)` will give us evenly spaced numbers with a given step, while `np.linspace(start, stop, num)` will give us `num` samples. You see, one uses a fixed step, the other uses a number of points to return. When plotting functions, we usually use the latter. Let's generate, say, 1000 points (we know a straight line only needs two but we're generalizing the concept of plotting here :)).
```python
x = np.linspace(-3, 5, 1000)
```
Now, let's generate our function variable
```python
y = 2 * x + 3
```

We can print the values if we like but we're more interested in plotting them. To do this, first let's import a plotting library. `matplotlib` is the most commnly used one and we usually give it an alias as well.
```python
import matplotlib.pyplot as plt
```

Now, let's plot the values. To do this, we just call the `plot()` function. Notice that the top-most part of this notebook contains a "magic string": `%matplotlib inline`. This hints Jupyter to display all plots inside the notebook. However, it's a good practice to call `show()` after our plot is ready.
```python
plt.plot(x, y)
plt.show()
```

```python
import numpy as np
import matplotlib.pyplot as plt
x = np.linspace(-3, 5, 1000)
y = 2 * x + 3
plt.plot(x, y)
plt.show()
```

It doesn't look too bad bit we can do much better. See how the axes don't look like they should? Let's move them to zeto. This can be done using the "spines" of the plot (i.e. the borders).

All `matplotlib` figures can have many plots (subfigures) inside them. That's why when performing an operation, we have to specify a target figure. There is a default one and we can get it by using `plt.gca()`. We usually call it `ax` for "axis".
Let's save it in a variable (in order to prevent multiple calculations and to make code prettier). Let's now move the bottom and left spines to the origin $(0, 0)$ and hide the top and right one.
```python
ax = plt.gca()
ax.spines["bottom"].set_position("zero")
ax.spines["left"].set_position("zero")
ax.spines["top"].set_visible(False)
ax.spines["right"].set_visible(False)
```

**Note:** All plot manipulations HAVE TO be done before calling `show()`. It's up to you whether they should be before or after the function you're plotting.

This should look better now. We can, of course, do much better (e.g. remove the double 0 at the origin and replace it with a single one), but this is left as an exercise for the reader :).

```python
plt.plot(x, y)
ax = plt.gca()
ax.spines["bottom"].set_position("zero")
ax.spines["left"].set_position("zero")
ax.spines["top"].set_visible(False)
ax.spines["right"].set_visible(False)
plt.show()
```

### * Problem 5. Linearizing Functions
Why is the line equation so useful? The main reason is because it's so easy to work with. Scientists actually try their best to linearize functions, that is, to make linear functions from non-linear ones. There are several ways of doing this. One of them involves derivatives and we'll talk about it later in the course.

A commonly used method for linearizing functions is through algebraic transformations. Try to linearize
$$ y = ae^{bx} $$

Hint: The inverse operation of $e^{x}$ is $\ln(x)$. Start by taking $\ln$ of both sides and see what you can do. Your goal is to transform the function into another, linear function. You can look up more hints on the Internet :).

$$\ln y=b*x+\ln a$$

### * Problem 6. Generalizing the Plotting Function
Let's now use the power of Python to generalize the code we created to plot. In Python, you can pass functions as parameters to other functions. We'll utilize this to pass the math function that we're going to plot.

Note: We can also pass *lambda expressions* (anonymous functions) like this:
```python
lambda x: x + 2```
This is a shorter way to write
```python
def some_anonymous_function(x):
    return x + 2
```

We'll also need a range of x values. We may also provide other optional parameters which will help set up our plot. These may include titles, legends, colors, fonts, etc. Let's stick to the basics now.

Write a Python function which takes another function, x range and number of points, and plots the function graph by evaluating it at every point.

**BIG hint:** If you want to use not only `numpy` functions for `f` but any one function, a very useful (and easy) thing to do, is to vectorize the function `f` (e.g. to allow it to be used with `numpy` broadcasting):
```python
f_vectorized = np.vectorize(f)
y = f_vectorized(x)
```

```python
def plot_math_function(f, min_x, max_x, num_points):

        f_vectorized = np.vectorize(f)
        x=np.linspace(min_x,max_x,num_points)
        y = f_vectorized(x)

        plt.plot(x, y)
        ax = plt.gca()
        ax.spines["bottom"].set_position("zero")
        ax.spines["left"].set_position("zero")
        ax.spines["top"].set_visible(False)
        ax.spines["right"].set_visible(False)
        plt.show()
```

```python
#x = np.linspace(-15,15,100) # 100 linearly spaced numbers
#y = np.sin(x)/x # computing the values of sin(x)/x

# compose plot
#plt.plot(x,y) # sin(x)/x
#plt.plot(x,y,'co') # same function with cyan dots
#plt.plot(x,2*y,x,3*y) # 2*sin(x)/x and 3*sin(x)/x
#plt.show() # show the plot
```

```python

```

```python
plot_math_function(lambda x: 2 * x + 3, -3, 5, 1000)
plot_math_function(lambda x: -x + 8, -1, 10, 1000)
plot_math_function(lambda x: x**2 - x - 2, -3, 4, 1000)
plot_math_function(lambda x: np.sin(x), -np.pi, np.pi, 1000)
plot_math_function(lambda x: np.sin(x) / x, -4 * np.pi, 4 * np.pi, 1000)
```

### * Problem 7. Solving Equations Graphically
Now that we have a general plotting function, we can use it for more interesting things. Sometimes we don't need to know what the exact solution is, just to see where it lies. We can do this by plotting the two functions around the "=" sign ans seeing where they intersect. Take, for example, the equation $2x + 3 = 0$. The two functions are $f(x) = 2x + 3$ and $g(x) = 0$. Since they should be equal, the point of their intersection is the solution of the given equation. We don't need to bother marking the point of intersection right now, just showing the functions.

To do this, we'll need to improve our plotting function yet once. This time we'll need to take multiple functions and plot them all on the same graph. Note that we still need to provide the $[x_{min}; x_{max}]$ range and it's going to be the same for all functions.

```python
vectorized_fs = [np.vectorize(f) for f in functions]
ys = [vectorized_f(x) for vectorized_f in vectorized_fs]
```

```python
def plot_math_functions(functions, min_x, max_x, num_points):

        vectorized_fs = [np.vectorize(f) for f in functions]
        x=np.linspace(min_x,max_x,num_points)

        ys = [vectorized_f(x) for vectorized_f in vectorized_fs]

        #y = [y for y in ys]

        for i in ys: #ys is my range of y values on the chart
            plt.plot(x, i)
        ax = plt.gca()
        ax.spines["bottom"].set_position("zero")
        ax.spines["left"].set_position("zero")
        ax.spines["top"].set_visible(False)
        ax.spines["right"].set_visible(False)
        plt.show()

```

```python
plot_math_functions([lambda x: 2 * x + 3, lambda x: 0], -3, 5, 1000)
plot_math_functions([lambda x: 3 * x**2 - 2 * x + 5, lambda x: 3 * x + 7], -2, 3, 1000)
```

This is also a way to plot the solutions of systems of equation, like the one we solved last time. Let's actually try it.

```python
plot_math_functions([lambda x: (-4 * x + 7) / 3, lambda x: (-3 * x + 8) / 5, lambda x: (-x - 1) / -2], -1, 4, 1000)
```

### Problem 8. Trigonometric Functions
We already saw the graph of the function $y = \sin(x)$. But, how do we define the trigonometric functions once again? Let's quickly review that.

The two basic trigonometric functions are defined as the ratio of two sides:
$$ \sin(x) = \frac{\text{opposite}}{\text{hypotenuse}} $$
$$ \cos(x) = \frac{\text{adjacent}}{\text{hypotenuse}} $$

And also:
$$ \tan(x) = \frac{\text{opposite}}{\text{adjacent}} = \frac{\sin(x)}{\cos(x)} $$
$$ \cot(x) = \frac{\text{adjacent}}{\text{opposite}} = \frac{\cos(x)}{\sin(x)} $$

This is fine, but using this, "right-triangle" definition, we're able to calculate the trigonometric functions of angles up to $90^\circ$. But we can do better. Let's now imagine a circle centered at the origin of the coordinate system, with radius $r = 1$. This is called a "unit circle".

We can now see exactly the same picture. The $x$-coordinate of the point in the circle corresponds to $\cos(\alpha)$ and the $y$-coordinate - to $\sin(\alpha)$. What did we get? We're now able to define the trigonometric functions for all degrees up to $360^\circ$. After that, the same values repeat: these functions are **periodic**:
$$ \sin(k.360^\circ + \alpha) = \sin(\alpha), k = 0, 1, 2, \dots $$
$$ \cos(k.360^\circ + \alpha) = \cos(\alpha), k = 0, 1, 2, \dots $$

We can, of course, use this picture to derive other identities, such as:
$$ \sin(90^\circ + \alpha) = \cos(\alpha) $$

A very important property of the sine and cosine is that they accept values in the range $(-\infty; \infty)$ and produce values in the range $[-1; 1]$. The two other functions take values in the range $(-\infty; \infty)$ **except when their denominators are zero** and produce values in the same range.

#### Radians
A degree is a geometric object, $1/360$th of a full circle. This is quite inconvenient when we work with angles. There is another, natural and intrinsic measure of angles. It's called the **radian** and can be written as $\text{rad}$ or without any designation, so $\sin(2)$ means "sine of two radians".

It's defined as *the central angle of an arc with length equal to the circle's radius* and $1\text{rad} \approx 57.296^\circ$.

We know that the circle circumference is $C = 2\pi r$, therefore we can fit exactly $2\pi$ arcs with length $r$ in $C$. The angle corresponding to this is $360^\circ$ or $2\pi\ \text{rad}$. Also, $\pi rad = 180^\circ$.

(Some people prefer using $\tau = 2\pi$ to avoid confusion with always multiplying by 2 or 0.5 but we'll use the standard notation here.)

**NOTE:** All trigonometric functions in `math` and `numpy` accept radians as arguments. In order to convert between radians and degrees, you can use the relations $\text{[deg]} = 180/\pi.\text{[rad]}, \text{[rad]} =  \pi/180.\text{[deg]}$. This can be done using `np.deg2rad()` and `np.rad2deg()` respectively.

#### Inverse trigonometric functions
All trigonometric functions have their inverses. If you plug in, say $\pi/4$ in the $\sin(x)$ function, you get $\sqrt{2}/2$. The inverse functions (also called, arc-functions) take arguments in the interval $[-1; 1]$ and return the angle that they correspond to. Take arcsine for example:
$$ \arcsin(y) = x: sin(y) = x $$
$$ \arcsin\left(\frac{\sqrt{2}}{2}\right) = \frac{\pi}{4} $$

Please note that this is NOT entirely correct. From the relations we found:
$$\sin(x) = sin(2k\pi + x), k = 0, 1, 2, \dots $$

it follows that $\arcsin(x)$ has infinitely many values, separated by $2k\pi$ radians each:
$$ \arcsin\left(\frac{\sqrt{2}}{2}\right) = \frac{\pi}{4} + 2k\pi, k = 0, 1, 2, \dots $$

In most cases, however, we're interested in the first value (when $k = 0$). It's called the **principal value**.

Note 1: There are inverse functions for all four basic trigonometric functions: $\arcsin$, $\arccos$, $\arctan$, $\text{arccot}$. These are sometimes written as $\sin^{-1}(x)$, $cos^{-1}(x)$, etc. These definitions are completely equivalent.

Just notice the difference between $\sin^{-1}(x) := \arcsin(x)$ and $\sin(x^{-1}) = \sin(1/x)$.

#### Exercise
Use the plotting function you wrote above to plot the inverse trigonometric functions.

```python
plot_math_functions([lambda x:np.arcsin(x)], -1, 1, 1000)
plot_math_functions([lambda x:np.arccos(x)], -1, 1, 1000)
plot_math_functions([lambda x:np.arctan(x)], -1, 1, 1000)
plot_math_functions([lambda x:np.arctan(1/x)], -1, 1, 1000)
```

```python
def plot_circle(x_c, y_c, r):
    """
    Plots the circle with center C(x_c; y_c) and radius r.
    This corresponds to plotting the equation x^2 + y^2 = r^2
    """
    ##(x−x_c)**2 + (y−y_c)**2 = r**2

    x = np.linspace(-r+x_c,r+x_c,1000)
    y = y_c+np.sqrt(-(x-x_c)**2+r**2)
    y2 = y_c-np.sqrt(-(x-x_c)**2+r**2)
    plt.plot(x, y,'b')
    plt.plot(x, y2,'b')
    plt.gca().set_aspect('equal')
    plt.show()
```

```python
plot_circle(300, 300, 600)
```

### ** Problem 9. Perlin Noise
This algorithm has many applications in computer graphics and can serve to demonstrate several things... and help us learn about math, algorithms and Python :).
#### Noise
Noise is just random values. We can generate noise by just calling a random generator. Note that these are actually called *pseudorandom generators*. We'll talk about this later in this course.
We can generate noise in however many dimensions we want. For example, if we want to generate a single dimension, we just pick N random values and call it a day. If we want to generate a 2D noise space, we can take an approach which is similar to what we already did with `np.meshgrid()`.

$$ \text{noise}(x, y) = N, N \in [n_{min}, n_{max}] $$

This function takes two coordinates and returns a single number N between $n_{min}$ and $n_{max}$. (This is what we call a "scalar field").

Random variables are always connected to **distributions**. We'll talk about these a great deal but now let's just say that these define what our noise will look like. In the most basic case, we can have "uniform noise" - that is, each point in our little noise space $[n_{min}, n_{max}]$ will have an equal chance (probability) of being selected.

#### Perlin noise
There are many more distributions but right now we'll want to have a look at a particular one. **Perlin noise** is a kind of noise which looks smooth. It looks cool, especially if it's colored. The output may be tweaked to look like clouds, fire, etc. 3D Perlin noise is most widely used to generate random terrain.

#### Algorithm
... Now you're on your own :). Research how the algorithm is implemented (note that this will require that you understand some other basic concepts like vectors and gradients).

#### Your task
1. Research about the problem. See what articles, papers, Python notebooks, demos, etc. other people have created
2. Create a new notebook and document your findings. Include any assumptions, models, formulas, etc. that you're using
3. Implement the algorithm. Try not to copy others' work, rather try to do it on your own using the model you've created
4. Test and improve the algorithm
5. (Optional) Create a cool demo :), e.g. using Perlin noise to simulate clouds. You can even do an animation (hint: you'll need gradients not only in space but also in time)
6. Communicate the results (e.g. in the Softuni forum)

Hint: [This](http://flafla2.github.io/2014/08/09/perlinnoise.html) is a very good resource. It can show you both how to organize your notebook (which is important) and how to implement the algorithm.


##### Copyright 2020 The Cirq Developers

```
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Ion Device Class

    View on QuantumAI

    Run in Google Colab

    View source on GitHub

    Download notebook

The `IonDevice` represents a trapped ion quantum computer with all-to-all qubit connectivity.  The number of qubits as well as the duration of gates and measurements are specified by the user when creating an ion device.

Two-qubit gates are implemented by an Ising-type coupling known as the *Mølmer–Sørensen* gate. The Mølmer–Sørensen gate couples ions through the shared motional modes of the ion chain.  The ion motion and internal state decouples at the end of each gate. The `IonDevice` class assumes this decoupling is perfect and does not explicitly model the ion motion.

```
try:
    import cirq
except ImportError:
    print("installing cirq...")
    !pip install cirq --quiet
    print("installed cirq.")
    import cirq

import numpy as np
```

## Defining an `IonDevice`

To define an `IonDevice`, we specify

- The set of qubits in the device,
- The duration of single-qubit gates,
- The duration of two-qubit gates, and
- The duration of measurement gates.

The code below creates an `IonDevice` with four qubits in a linear array. The durations we use for each type of gate are reasonable order-of-magnitude estimates, though they will differ for different trapped ion computers.

```
"""Create an IonDevice."""
ion_device = cirq.IonDevice(
    qubits=cirq.LineQubit.range(4),
    oneq_gates_duration=cirq.Duration(micros=10),
    twoq_gates_duration=cirq.Duration(micros=200),
    measurement_duration=cirq.Duration(micros=100)
)
```

We can view some properties of the `ion_device` as shown below.

```
"""View some properties of the device."""
# Display the ion device.
print("Ion Device:\n", ion_device)

# Get all qubits in the device.
print("\nQubits in the IonDevice:\n", sorted(ion_device.qubits))

# Get a qubit at a certain position (if present).
pos = 2
print(f"\nQubit at position {pos}:\n", ion_device.at(pos))
```

    Ion Device:
     0───1───2───3

    Qubits in the IonDevice:
     [cirq.LineQubit(0), cirq.LineQubit(1), cirq.LineQubit(2), cirq.LineQubit(3)]

    Qubit at position 2:
     2

## Native Gate Set

An `IonDevice` can implement single-qubit rotations about the $X$, $Y$, and $Z$ axes of the Bloch sphere: namely, `cirq.rx`, `cirq.ry`, and `cirq.rz`.

An `IonDevice` can implement the two-qubit Mølmer–Sørensen gate, a rotation about the $XX$ axis in the two-qubit Bloch sphere defined as

\begin{equation}
    \exp(-i t XX) = \left[ \begin{matrix}
        \cos t & 0 & 0 & -i \sin t \\
        0 & \cos t & -i \sin t & 0 \\
        0 & -i \sin t & \cos t & 0 \\
        -i \sin t & 0 & 0 & \cos t
    \end{matrix} \right] .
\end{equation}

The Mølmer–Sørensen gate is defined in Cirq as `cirq.ms`.

One can check if a given gate is valid with `IonDevice.validate_gate`. This method raises an error if the gate is invalid (not supported by the device) and does nothing if the gate is valid (supported by the device).

```
"""Check if gates are valid. Invalid gates raise a ValueError."""
# Single-qubit X rotation of any angle is supported.
ion_device.validate_gate(cirq.rx(np.pi / 7))

# Single-qubit Z rotation of any angle is supported.
ion_device.validate_gate(cirq.rz(np.pi / 5))

# Mølmer–Sørensen gate of any angle is supported.
ion_device.validate_gate(cirq.ms(np.pi / 4))
```

One can also validate operations and circuits with `IonDevice.validate_operation` and `IonDevice.validate_circuit`, respectively.

We can get the duration of valid operations as follows.

```
"""Get the duration of valid operations."""
# Duration of a single-qubit operation.
ion_device.duration_of(cirq.ry(np.pi / 2).on(ion_device.at(0)))
```

    cirq.Duration(micros=10)

## Decomposing Operations and Circuits

Operations which are not valid on the device can be decomposed into a set of valid operations. For example, a CNOT gate is not supported but can be implemented with the following decomposition.

```
"""Decompose a CNOT operation into valid IonDevice operations."""
# Get a CNOT operation.
op = cirq.CNOT(ion_device.at(0), ion_device.at(1))

# Decompose it for the IonDevice.
ion_device_ops = cirq.ConvertToIonGates().convert_one(op)

# Print the sequence of operations to implement a CNOT.
print("Sequence of IonDevice operations for a CNOT:\n")
print(cirq.Circuit(ion_device_ops))
```

    Sequence of IonDevice operations for a CNOT:

    0: ───Ry(0.5π)───MS(0.25π)───Rx(-0.5π)───Ry(-0.5π)───
                     │
    1: ──────────────MS(0.25π)───Rx(-0.5π)───────────────

Circuits can also be decomposed in a similar manner using `IonDevice.decompose_circuit`.

```
"""Decompose a circuit into IonDevice operations."""
# Example circuit to decompose.
circuit = cirq.Circuit(
    cirq.H(cirq.LineQubit(0)),
    cirq.CNOT(cirq.LineQubit(0), cirq.LineQubit(1)),
    cirq.CNOT(cirq.LineQubit(0), cirq.LineQubit(2))
)

# Display it.
print("Circuit to decompose:\n")
print(circuit)

# Decompose the circuit.
ion_device_circuit = ion_device.decompose_circuit(circuit)

# Display the decomposed circuit.
print("\nIonDevice circuit:\n")
print(ion_device_circuit)
```

    Circuit to decompose:

    0: ───H───@───@───
              │   │
    1: ───────X───┼───
                  │
    2: ───────────X───

    IonDevice circuit:

    0: ───PhX(1)───────────MS(0.25π)───PhX(1)^0.5───────────MS(0.25π)───PhX(-0.5)^0.5───S^-1───
                           │                                │
    1: ────────────────────MS(0.25π)───PhX(1)^0.5───────────┼──────────────────────────────────
                                                            │
    2: ─────────────────────────────────────────────────────MS(0.25π)───PhX(1)^0.5─────────────