# Linear Algebra and Python Basics

In this chapter, I will be discussing some linear algebra basics that will provide sufficient linear algebra background for effective programming in Python for our purposes.  We will be doing very basic linear algebra that by no means covers the full breadth of this topic.  Why linear algebra?  Linear algebra allows us to express relatively complex linear expressions in a very compact way.

Being comfortable with the rules for scalar and matrix addition, subtraction, multiplication, and division (known as inversion) is important for our class.

Before we can implement any of these ideas in code, we need to talk a bit about python and how data is stored.

## Python Primer

There are numerous ways to run python code.  I will show you two and both are easily accessible after installing Anaconda:

1. The Spyder integrated development environment.  The major advantages of Spyder is that it provides a graphical way for viewing matrices, vectors, and other objects you want to check as you work on a problem.  It also has the most intuitive way of debugging code.

    Spyder looks like this:

    Code can be run by clicking the green arrow (runs the entire file) or by blocking a subset and running it.
    In Windows or Mac, you can launch the Spyder by looking for the icon in the newly installed Program Folder Anaconda.

2. The Ipython Notebook (now called Jupyter).  The major advantages of this approach is that you use your web browser for all of your python work and you can mix code, videos, notes, graphics from the web, and mathematical notation to tell the whole story of your python project. In fact, I am using the ipython notebook for writing these notes.
    The Ipython Notebook looks like this:

    In Windows or Mac, you can launch the Ipython Notebook by looking in the newly installed Program Folder Anaconda.

In my work flow, I usually only use the Ipython Notebook, but for some coding problems where I need access to the easy debugging capabilities of Spyder, I use it.  We will be using the Ipython Notebook interface (web browser) mostly in this class.

### Loading libraries

The python universe has a huge number of libraries that extend the capabilities of python. Nearly all of these are open source, unlike packages like stata or matlab where some key libraries are proprietary (and can cost lots of money).  In lots of my code, you will see this at the top:

```python
%matplotlib inline
import sympy as sympy
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sbn
from scipy import *
```

This code sets up Ipython Notebook environments (lines beginning with `%`), and loads several libraries and functions.  The core scientific stack in python consists of a number of free libraries.  The ones I have loaded above include:

1. sympy: provides for symbolic computation (solving algebra problems)
2. numpy: provides for linear algebra computations
3. matplotlib.pyplot: provides for the ability to graph functions and draw figures
4. scipy: scientific python provides a plethora of capabilities
5. seaborn: makes matplotlib figures even pretties (another library like this is called bokeh).  This is entirely optional and is purely for eye candy.

### Creating arrays, scalars, and matrices in Python

Scalars can be created easily like this:

```python
x = .5
print x
```

    0.5

#### Vectors and Lists

The numpy library (we will reference it by np) is the workhorse library for linear algebra in python.  To creat a vector simply surround a python list ($[1,2,3]$) with the np.array function:

```python
x_vector = np.array([1,2,3])
print x_vector
```

    [1 2 3]

We could have done this by defining a python list and converting it to an array:

```python
c_list = [1,2]
print "The list:",c_list
print "Has length:", len(c_list)

c_vector = np.array(c_list)
print "The vector:", c_vector
print "Has shape:",c_vector.shape
```

    The list: [1, 2]
    Has length: 2
    The vector: [1 2]
    Has shape: (2,)

```python
z = [5,6]
print "This is a list, not an array:",z
print type(z)
```

    This is a list, not an array: [5, 6]
    <type 'list'>

```python
zarray = np.array(z)
print "This is an array, not a list",zarray
print type(zarray)
```

    This is an array, not a list [5 6]
    <type 'numpy.ndarray'>

#### Matrices

```python
b = zip(z,c_vector)
print b
print "Note that the length of our zipped list is 2 not (2 by 2):",len(b)
```

    [(5, 1), (6, 2)]
    Note that the length of our zipped list is 2 not (2 by 2): 2

```python
print "But we can convert the list to a matrix like this:"
A = np.array(b)
print A
print type(A)
print "A has shape:",A.shape
```

    But we can convert the list to a matrix like this:
    [[5 1]
     [6 2]]
    <type 'numpy.ndarray'>
    A has shape: (2, 2)

## Matrix Addition and Subtraction

###Adding or subtracting a scalar value to a matrix

To learn the basics, consider a small matrix of dimension $2 \times 2$, where $2 \times 2$ denotes the number of rows $\times$ the number of columns.  Let $A$=$\bigl( \begin{smallmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{smallmatrix} \bigr)$.  Consider adding a scalar value (e.g. 3) to the A.
$$
\begin{equation}
	A+3=\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22}
	\end{bmatrix}+3
	=\begin{bmatrix}
	  a_{11}+3 & a_{12}+3 \\
	  a_{21}+3 & a_{22}+3
	\end{bmatrix}
\end{equation}
$$
The same basic principle holds true for A-3:
$$
\begin{equation}
	A-3=\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22}
	\end{bmatrix}-3
	=\begin{bmatrix}
	  a_{11}-3 & a_{12}-3 \\
	  a_{21}-3 & a_{22}-3
	\end{bmatrix}
\end{equation}
$$
Notice that we add (or subtract) the scalar value to each element in the matrix A.  A can be of any dimension.

This is trivial to implement, now that we have defined our matrix A:

```python
result = A + 3
#or
result = 3 + A
print result
```

    [[8 4]
     [9 5]]

###Adding or subtracting two matrices
Consider two small $2 \times 2$ matrices, where $2 \times 2$ denotes the \# of rows $\times$ the \# of columns.  Let $A$=$\bigl( \begin{smallmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{smallmatrix} \bigr)$ and $B$=$\bigl( \begin{smallmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{smallmatrix} \bigr)$.  To find the result of $A-B$, simply subtract each element of A with the corresponding element of B:

$$
\begin{equation}
	A -B =
	\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22}
	\end{bmatrix} -
	\begin{bmatrix} b_{11} & b_{12} \\
	  b_{21} & b_{22}
	\end{bmatrix}
	=
	\begin{bmatrix}
	  a_{11}-b_{11} & a_{12}-b_{12} \\
	  a_{21}-b_{21} & a_{22}-b_{22}
	\end{bmatrix}
\end{equation}
$$

Addition works exactly the same way:

$$
\begin{equation}
	A + B =
	\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22}
	\end{bmatrix} +
	\begin{bmatrix} b_{11} & b_{12} \\
	  b_{21} & b_{22}
	\end{bmatrix}
	=
	\begin{bmatrix}
	  a_{11}+b_{11} & a_{12}+b_{12} \\
	  a_{21}+b_{21} & a_{22}+b_{22}
	\end{bmatrix}
\end{equation}
$$

An important point to know about matrix addition and subtraction is that it is only defined when $A$ and $B$ are of the same size.  Here, both are $2 \times 2$.  Since operations are performed element by element, these two matrices must be conformable- and for addition and subtraction that means they must have the same numbers of rows and columns.  I like to be explicit about the dimensions of matrices for checking conformability as I write the equations, so write

$$
A_{2 \times 2} + B_{2 \times 2}= \begin{bmatrix}
  a_{11}+b_{11} & a_{12}+b_{12} \\
  a_{21}+b_{21} & a_{22}+b_{22}
\end{bmatrix}_{2 \times 2}
$$

Notice that the result of a matrix addition or subtraction operation is always of the same dimension as the two operands.

Let's define another matrix, B, that is also $2 \times 2$ and add it to A:

```python
B = np.random.randn(2,2)
print B
```

    [[ 0.65905256  1.8847017 ]
     [ 0.61068714  1.96506417]]

```python
result = A + B
result
```

    array([[ 5.65905256,  2.8847017 ],
           [ 6.61068714,  3.96506417]])

##Matrix Multiplication

###Multiplying a scalar value times a matrix

As before, let $A$=$\bigl( \begin{smallmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{smallmatrix} \bigr)$.  Suppose we want to multiply A times a scalar value (e.g. $3 \times A$)

$$
\begin{equation}
	3 \times A = 3 \times \begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22}
	\end{bmatrix}
	=
	\begin{bmatrix}
	  3a_{11} & 3a_{12} \\
	  3a_{21} & 3a_{22}
	\end{bmatrix}
\end{equation}
$$

is of dimension (2,2).  Scalar multiplication is commutative, so that $3 \times A$=$A \times 3$.  Notice that the product is defined for a matrix A of any dimension.

Similar to scalar addition and subtration, the code is simple:

```python
A * 3
```

    array([[15,  3],
           [18,  6]])

###Multiplying two matricies

Now, consider the $2 \times 1$ vector $C=\bigl( \begin{smallmatrix} c_{11} \\
  c_{21}
\end{smallmatrix} \bigr)$

Consider multiplying matrix $A_{2 \times 2}$ and the vector $C_{2 \times 1}$.  Unlike the addition and subtraction case, this product is defined.  Here, conformability depends not on the row **and** column dimensions, but rather on the column dimensions of the first operand and the row dimensions of the second operand.  We can write this operation as follows

$$
\begin{equation}
	A_{2 \times 2} \times C_{2 \times 1} =
	\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22}
	\end{bmatrix}_{2 \times 2}
    \times
    \begin{bmatrix}
	c_{11} \\
	c_{21}
	\end{bmatrix}_{2 \times 1}
	=
	\begin{bmatrix}
	  a_{11}c_{11} + a_{12}c_{21} \\
	  a_{21}c_{11} + a_{22}c_{21}
	\end{bmatrix}_{2 \times 1}
\end{equation}
$$

Alternatively, consider a matrix C of dimension $2 \times 3$ and a matrix A of dimension $3 \times 2$

$$
\begin{equation}
	A_{3 \times 2}=\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22} \\
	  a_{31} & a_{32}
	\end{bmatrix}_{3 \times 2}
	,
	C_{2 \times 3} =
	\begin{bmatrix}
		  c_{11} & c_{12} & c_{13} \\
		  c_{21} & c_{22} & c_{23} \\
	\end{bmatrix}_{2 \times 3}
	\end{equation}
$$

Here, A $\times$ C is

$$
\begin{align}
	A_{3 \times 2} \times C_{2 \times 3}=&
	\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22} \\
	  a_{31} & a_{32}
	\end{bmatrix}_{3 \times 2}
	\times
	\begin{bmatrix}
	  c_{11} & c_{12} & c_{13} \\
	  c_{21} & c_{22} & c_{23}
	\end{bmatrix}_{2 \times 3} \\
	=&
	\begin{bmatrix}
	  a_{11} c_{11}+a_{12} c_{21} & a_{11} c_{12}+a_{12} c_{22} & a_{11} c_{13}+a_{12} c_{23} \\
	  a_{21} c_{11}+a_{22} c_{21} & a_{21} c_{12}+a_{22} c_{22} & a_{21} c_{13}+a_{22} c_{23} \\
	  a_{31} c_{11}+a_{32} c_{21} & a_{31} c_{12}+a_{32} c_{22} & a_{31} c_{13}+a_{32} c_{23}
	\end{bmatrix}_{3 \times 3}
\end{align}
$$

So in general, $X_{r_x \times c_x} \times Y_{r_y \times c_y}$ we have two important things to remember:

* For conformability in matrix multiplication, $c_x=r_y$, or the columns in the first operand must be equal to the rows of the second operand.
* The result will be of dimension $r_x \times c_y$, or of dimensions equal to the rows of the first operand and columns equal to columns of the second operand.

Given these facts, you should convince yourself that matrix multiplication is not generally commutative, that the relationship $X \times Y = Y \times X$ does **not** hold in all cases.
For this reason, we will always be very explicit about whether we are pre multiplying ($X \times Y$) or post multiplying ($Y \times X$) the vectors/matrices $X$ and $Y$.

For more information on this topic, see this
http://en.wikipedia.org/wiki/Matrix_multiplication.

```python
# Let's redefine A and C to demonstrate matrix multiplication:
A = np.arange(6).reshape((3,2))
C = np.random.randn(2,2)

print A.shape
print C.shape
```

    (3, 2)
    (2, 2)

We will use the numpy dot operator to perform the these multiplications.  You can use it two ways to yield the same result:

```python
print A.dot(C)
print np.dot(A,C)
```

    [[-1.05731701 -0.93611763]
     [-3.11819033 -2.65506704]
     [-5.17906365 -4.37401644]]
    [[-1.05731701 -0.93611763]
     [-3.11819033 -2.65506704]
     [-5.17906365 -4.37401644]]

Suppose instead of pre-multiplying C by A, we post-multiply.  The product doesn't exist because we don't have conformability as described above:

```python
C.dot(A)
```

##Matrix Division
The term matrix division is actually a misnomer.  To divide in a matrix algebra world we first need to invert the matrix.  It is useful to consider the analog case in a scalar work.  Suppose we want to divide the $f$ by $g$.  We could do this in two different ways:
$$
\begin{equation}
	\frac{f}{g}=f \times g^{-1}.
\end{equation}
$$
In a scalar seeting, these are equivalent ways of solving the division problem.  The second one requires two steps: first we invert g and then we multiply f times g.  In a matrix world, we need to think about this second approach.  First we have to invert the matrix g and then we will need to pre or post multiply depending on the exact situation we encounter (this is intended to be vague for now).

###Inverting a Matrix

As before, consider the square $2 \times 2$ matrix $A$=$\bigl( \begin{smallmatrix} a_{11} & a_{12} \\ a_{21} & a_{22}\end{smallmatrix} \bigr)$.  Let the inverse of matrix A (denoted as $A^{-1}$) be

$$
\begin{equation}
	A^{-1}=\begin{bmatrix}
             a_{11} & a_{12} \\
		     a_{21} & a_{22}
           \end{bmatrix}^{-1}=\frac{1}{a_{11}a_{22}-a_{12}a_{21}}	\begin{bmatrix}
		             a_{22} & -a_{12} \\
				     -a_{21} & a_{11}
		           \end{bmatrix}
\end{equation}
$$

The inverted matrix $A^{-1}$ has a useful property:
$$
\begin{equation}
	A \times A^{-1}=A^{-1} \times A=I
\end{equation}
$$
where I, the identity matrix (the matrix equivalent of the scalar value 1), is
$$
\begin{equation}
	I_{2 \times 2}=\begin{bmatrix}
             1 & 0 \\
		     0 & 1
           \end{bmatrix}
\end{equation}
$$
furthermore, $A \times I = A$ and $I \times A = A$.

An important feature about matrix inversion is that it is undefined if (in the $2 \times 2$ case), $a_{11}a_{22}-a_{12}a_{21}=0$.  If this relationship is equal to zero the inverse of A does not exist.  If this term is very close to zero, an inverse may exist but $A^{-1}$ may be poorly conditioned meaning it is prone to rounding error and is likely not well identified computationally.  The term $a_{11}a_{22}-a_{12}a_{21}$ is the determinant of matrix A, and for square matrices of size greater than $2 \times 2$, if equal to zero indicates that you have a problem with your data matrix (columns are linearly dependent on other columns).  The inverse of matrix A exists if A is square and is of full rank (ie. the columns of A are not linear combinations of other columns of A).

For more information on this topic, see this
http://en.wikipedia.org/wiki/Matrix_inversion, for example, on inverting matrices.

```python
# note, we need a square matrix (# rows = # cols), use C:
C_inverse = np.linalg.inv(C)
print C_inverse
```

    [[-16.7544699   -1.37174182]
     [ 18.92367512   0.48109974]]

Check that $C\times C^{-1} = I$:

```python
print C.dot(C_inverse)
print "Is identical to:"
print C_inverse.dot(C)
```

    [[ 1.  0.]
     [ 0.  1.]]
    Is identical to:
    [[ 1.  0.]
     [ 0.  1.]]

##Transposing a Matrix

At times it is useful to pivot a matrix for conformability- that is in order to matrix divide or multiply, we need to switch the rows and column dimensions of matrices.  Consider the matrix
$$
\begin{equation}
	A_{3 \times 2}=\begin{bmatrix}
	  a_{11} & a_{12} \\
	  a_{21} & a_{22} \\
	  a_{31} & a_{32}
	\end{bmatrix}_{3 \times 2}
\end{equation}
$$
The transpose of A (denoted as $A^{\prime}$) is
$$
\begin{equation}
   A^{\prime}=\begin{bmatrix}
	  a_{11} & a_{21} & a_{31} \\
	  a_{12} & a_{22} & a_{32} \\
	\end{bmatrix}_{2 \times 3}
\end{equation}
$$

```python
A = np.arange(6).reshape((3,2))
B = np.arange(8).reshape((2,4))
print "A is"
print A

print "The Transpose of A is"
print A.T
```

    A is
    [[0 1]
     [2 3]
     [4 5]]
    The Transpose of A is
    [[0 2 4]
     [1 3 5]]

One important property of transposing a matrix is the transpose of a product of two matrices.  Let matrix A be of dimension $N \times M$ and let B of of dimension $M \times P$.  Then
$$
\begin{equation}
	(AB)^{\prime}=B^{\prime}A^{\prime}
\end{equation}
$$
For more information, see this http://en.wikipedia.org/wiki/Matrix_transposition on matrix transposition.  This is also easy to implement:

```python
print B.T.dot(A.T)
print "Is identical to:"
print (A.dot(B)).T
```

    [[ 4 12 20]
     [ 5 17 29]
     [ 6 22 38]
     [ 7 27 47]]
    Is identical to:
    [[ 4 12 20]
     [ 5 17 29]
     [ 6 22 38]
     [ 7 27 47]]

## More python tools

### Indexing

Python begins indexing at 0 (not 1), therefore the first row and first column is referenced by 0,0 **not** 1,1.

### Slicing

Accessing elements of numpy matrices and arrays. This code grabs the first column of A:

```python
print A
A[:,0]
```

    [[0 1]
     [2 3]
     [4 5]]

    array([0, 2, 4])

or, we could grab a particular element (in this case, the second column, last row):

```python
A[2,1]
```

    5

### Logical Checks to extract values from matrices/arrays:

```python
print A
```

    [[0 1]
     [2 3]
     [4 5]]

```python
print A[:,1]>4

A[A[:,1]>4]
```

    [False False  True]

    array([[4, 5]])

### For loops

Create a $12 \times 2$ matrix and print it out:

```python
A = np.arange(24).reshape((12,2))
print A
print A.shape
```

    [[ 0  1]
     [ 2  3]
     [ 4  5]
     [ 6  7]
     [ 8  9]
     [10 11]
     [12 13]
     [14 15]
     [16 17]
     [18 19]
     [20 21]
     [22 23]]
    (12, 2)

The code below loops over the rows (12 of them) of our matrix A.  For each row, it slices A and prints the row values across all columns.  Notice the form of the for loop.  The colon defines the statement we are looping over.  For each iteration of the loop **idented** lines will be executed:

```python
for rows in A:
    print rows
```

    [0 1]
    [2 3]
    [4 5]
    [6 7]
    [8 9]
    [10 11]
    [12 13]
    [14 15]
    [16 17]
    [18 19]
    [20 21]
    [22 23]

```python
for rows in A:
    print rows
```

    [0 1]
    [2 3]
    [4 5]
    [6 7]
    [8 9]
    [10 11]
    [12 13]
    [14 15]
    [16 17]
    [18 19]
    [20 21]
    [22 23]

```python
for cols in A.T:
    print cols
```

    [ 0  2  4  6  8 10 12 14 16 18 20 22]
    [ 1  3  5  7  9 11 13 15 17 19 21 23]

### If/then/else

The code below checks the value of x and categorizes it into one of three values.  Like the for loop, each logical if check is ended with a colon, and any commands to be applied to that particular if check (if true) must be indented.

```python
x=.4

if x<.5:
    print "Heads"
    print 100
elif x>.5:
    print "Tails"
    print 0
else:
    print "Tie"
    print 50
```

    Heads
    100

### While loops

Again, we have the same basic form for the statement (note the colons and indents).  Here we use the shorthand notation `x+=1` for performing the calculation `x = x + 1`:

```python
x=0
while x<10:
    x+=1
    print x<10

print x
```

    True
    True
    True
    True
    True
    True
    True
    True
    True
    False
    10


Manipulate Taylor series using the **SymPy** package in Python, following the tutorial at [scipy-lectures.org/packages/sympy.html](https://scipy-lectures.org/packages/sympy.html).  SymPy is used in [SageMath](https://www.sagemath.org/).

You can do this in Maple, which is a bit old-school, or Mathematica (which we have a license for at CU).  Wolfram Alpha is another choice (by the company that makes Mathematica) though the free version does have some limitations.

The nice thing about Python is that it is free and community supported, so if you take the time to use it, it won't go away. If you learn Mathematica now, you may not be able to use it in your job, since it's an expensive license.

```
# Load the package
import sympy as sym
from sympy import init_printing
init_printing()  # This will make output look very nice!

import math  # has things like math.pi (which is the numerical value of pi)
```

```
# First, all symbolic variables must be declared. This is in contrast to Mathematica
x = sym.Symbol('x')

# Now, let's ask for a Taylor series expansion
sym.series( sym.cos(x), x )
```

```
# We can see the Docstring for the function to see all the options
?sym.series
```

Before going further, let's play with cos(pi/4)

```
print( math.cos( math.pi/4 ) )
# or, ask to print out 20 decimal places
print( '%.20f' % math.cos( math.pi/4 ) )  # this is old-school: see https://docs.python.org/3/tutorial/inputoutput.html#old-string-formatting
print( math.cos( sym.pi/4) )
print( sym.cos( math.pi/4) )
print( sym.cos( sym.pi/4) )
sym.cos( sym.pi/4)  # same as above, but use fancy formatting
```

Now let's explore the series. In particular, let's try to evaluate cos( .75 ), using only addition and multiplication, and a few "memorized" values of sine/cosine (let's assume that we memorize the fact that $\cos( \pi/4 ) = \sqrt{2}/2$ and that $\sin( \pi/4 ) = \sqrt{2}/2$.

Since .75 is not that far from $\pi/4 \approx 0.785$, let's do a Taylor expansion of $\cos$ around $\pi/4$.

```
Taylor = sym.series( sym.cos(x), x, x0=sym.pi/4, n=5)
Taylor
```

```
Taylor.subs(x,.75) # Not what we want
```

```
Taylor.evalf( subs={x: .75})
```

```
# Numerically evaluating this was a problem, because of the O( x^5 ) term. Let's remove it.
# See https://docs.sympy.org/latest/tutorial/calculus.html
Taylor.removeO()
```

```
guess = Taylor.removeO().subs(x,.75).evalf()
print("Our guess is: \t%.15f" % guess)
print("True value is: \t%.15f" % math.cos( .75) )
print("Discrepency is: \t%.3e" % abs( guess - math.cos(.75) ))
```

    Our guess is: 	0.731688868548266
    True value is: 	0.731688868873821
    Discrepency is: 	3.256e-10

Our *absolute* error is about $3\times 10^{-10}$. Is this what we'd expect? Let's use math to be able to **guarantee** a small error (without having known the true answer first)

Let's look at the remainder term in the Taylor series

```
x0=sym.Symbol('x0')
sym.series( sym.cos(x), x, x0=x0, n=5)
```

```
xi = sym.Symbol('xi')
Remainder = -(x-x0)**5*sym.sin(xi)/120
Remainder
```

By Taylor's remainder theorem, we know $\xi \in [x,x_0]$ (or $\xi \in [x_0,x]$ if $x_0 < x$) such that the above remainder is the error in the Taylor series. We don't know exactly what $\xi$ is, but that's OK, because for any $\xi$, we can just bound $|\sin(\xi)| \le 1$.  So using this,...

```
RemainderBound = -(x-x0)**5/120
RemainderBound
```

```
# See https://docs.sympy.org/latest/tutorial/basic_operations.html
RemainderBound.subs( [ (x,.75), (x0,math.pi/4) ])
```

To summarize, our Taylor series approximation had error $3.25 \times 10^{-10}$.  We were able to use math to give an *a priori* bound that the error was no more than $4.63 \times 10^{-10}$ (though in that bound, we disregarded potentially cancellation errors when summing the series)


# PuLP : getting started

PuLP installation:

```
pip install pulp
```

## Simple PuLP Model (bonds_simple-PuLP.py)

$$
\begin{align}
    \max_{x_1,x_2} & \quad 4 x_1 + 3 x_2 \\
    \text{s.t.}    & \quad   x_1 +   x_2 \leq 100 \\
                   & \quad 2 x_1 +   x_2 \leq 150 \\
                   & \quad 3 x_1 + 4 x_2 \leq 360 \\
                   & \quad x_1, x_2 \geq 0
\end{align}
$$

Example taken from: https://www.ima.umn.edu/materials/2017-2018.2/W8.21-25.17/26306/PythonModeling.pdf

```python
from pulp import LpProblem, LpVariable, lpSum, LpMaximize, value
```

```python
prob = LpProblem("Dedication Model", LpMaximize)
```

```python
X1 = LpVariable("X1", 0, None)
X2 = LpVariable("X2", 0, None)
```

```python
# Objective function
prob += 4*X1 + 3*X2              # Objectives are nothing more than expressions without a right hand side

# Constraints
prob +=   X1 +   X2 <= 100
prob += 2*X1 +   X2 <= 150
prob += 3*X1 + 4*X2 <= 360
```

```python
prob.solve()
```

```python
print("Optimal total cost is: ", value(prob.objective))
```

```python
print("X1 :", X1.varValue)
print("X2 :", X2.varValue)
```


# Section 1.2.6 Example: Testing for Primality

```scheme
(if (> 1 2)
    (print "true")
    (print "false"))
```

    "false"

```scheme
(define (smallest-divisor n)
  (find-divisor n 2))

(define (find-divisor n test-divisor)
  (cond ((> (square test-divisor) n) n)
        ((divides? test-divisor n) test-divisor)
        (else (find-divisor n (+ test-divisor 1)))))

(define (divides? a b)
  (= (remainder b a) 0))

(define (square n)
  (* n n))
```

```scheme
(define (prime? n)
  (= n (smallest-divisor n)))

(print (prime? 4))
(print (prime? 5))
(print (prime? 13))
```

    #f
    #t
    #t

The order of this procedure is $\Theta(\sqrt{n})$

## The Fermat test

The $\Theta(\log{n})$ primality test is based on a result from number theory known as ***Fermat's Little Theorem***.

### Fermat's Little Theorem - フェルマーの小定理:

If $n$ is prime and $a$ is ***any positive interger*** less than $n$, the $a$ raised to the $n$-th power is congruent to $a$ modulo $n$.

I.e., $a \equiv a^n \pmod{n}$

- $a \equiv a^n$ でなかった場合は、即それは素数でないと判断できる。
- $a \equiv a^n$ だった場合は、素数の可能性があるので、ことなる $a$ でまた探索をする

```scheme
;; base = a
;; exp = 1,...,n
;; m = n
(define (expmod base exp m)
  (cond ((= exp 0) 1)
        ((even? exp)
         (remainder (square (expmod base (/ exp 2) m))
         m))
        (else
         (remainder (* base (expmod base (- exp 1) m))
                    m))))
```

$
\begin{align}
a^n\bmod{n} = \left\{
\begin{aligned}
\left[ (a^{n/2} \bmod{n})^2 \right] \bmod{n} &\quad \textrm{if $n$ is even} \\
\left[ (a^{n-1} \bmod{n}) \cdot a \right] \bmod{n} &\quad \textrm{overwise}
\end{aligned}
\right.
\end{align}
$

```scheme
(define (fermat-test n)
  (define (try-it a)
    (= (expmod a n n) a))
  (try-it (+ 1 (random (- n 1)))))
```

```scheme
(define (fast-prime? n times)
  (cond
   ((= times 0) #t)
   ((fermat-test n) (fast-prime? n (- times 1)))
   (else #f)
   )
  )
```

```scheme
(print (fast-prime? 2 10))
(print (fast-prime? 11 10))
(print (fast-prime? 443 10))
(print (fast-prime? 93 10))
```

    #t
    #t
    #t
    #f

```scheme

```


# Modeling and Simulation in Python

Chapter 9

Copyright 2017 Allen Downey

License: [Creative Commons Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0)

## Assignment 9
### Completed by: Philip Tanofsky

```python
# Configure Jupyter to display the assigned value after an assignment
%config InteractiveShell.ast_node_interactivity='last_expr_or_assign'

# import everything from SymPy.
from sympy import *

# Set up Jupyter notebook to display math.
init_printing()
```

The following displays SymPy expressions and provides the option of showing results in LaTeX format.

```python
from sympy.printing import latex

def show(expr, show_latex=False):
    """Display a SymPy expression.

    expr: SymPy expression
    show_latex: boolean
    """
    if show_latex:
        print(latex(expr))
    return expr
```

### Analysis with SymPy

Create a symbol for time.

```python
t = symbols('t')
```

If you combine symbols and numbers, you get symbolic expressions.

```python
expr = t + 1
```

The result is an `Add` object, which just represents the sum without trying to compute it.

```python
type(expr)
```

    sympy.core.add.Add

`subs` can be used to replace a symbol with a number, which allows the addition to proceed.

```python
expr.subs(t, 2)
```

`f` is a special class of symbol that represents a function.

```python
f = Function('f')
```

    f

The type of `f` is `UndefinedFunction`

```python
type(f)
```

    sympy.core.function.UndefinedFunction

SymPy understands that `f(t)` means `f` evaluated at `t`, but it doesn't try to evaluate it yet.

```python
f(t)
```

`diff` returns a `Derivative` object that represents the time derivative of `f`

```python
dfdt = diff(f(t), t)
```

```python
type(dfdt)
```

    sympy.core.function.Derivative

We need a symbol for `alpha`

```python
alpha = symbols('alpha')
```

Now we can write the differential equation for proportional growth.

```python
eq1 = Eq(dfdt, alpha*f(t))
```

And use `dsolve` to solve it.  The result is the general solution.

```python
solution_eq = dsolve(eq1)
```

We can tell it's a general solution because it contains an unspecified constant, `C1`.

In this example, finding the particular solution is easy: we just replace `C1` with `p_0`

```python
C1, p_0 = symbols('C1 p_0')
```

```python
particular = solution_eq.subs(C1, p_0)
```

In the next example, we have to work a little harder to find the particular solution.

### Solving the quadratic growth equation

We'll use the (r, K) parameterization, so we'll need two more symbols:

```python
r, K = symbols('r K')
```

Now we can write the differential equation.

```python
eq2 = Eq(diff(f(t), t), r * f(t) * (1 - f(t)/K))
```

And solve it.

```python
solution_eq = dsolve(eq2)
```

The result, `solution_eq`, contains `rhs`, which is the right-hand side of the solution.

```python
general = solution_eq.rhs
```

We can evaluate the right-hand side at $t=0$

```python
at_0 = general.subs(t, 0)
```

Now we want to find the value of `C1` that makes `f(0) = p_0`.

So we'll create the equation `at_0 = p_0` and solve for `C1`.  Because this is just an algebraic identity, not a differential equation, we use `solve`, not `dsolve`.

The result from `solve` is a list of solutions.  In this case, [we have reason to expect only one solution](https://en.wikipedia.org/wiki/Picard%E2%80%93Lindel%C3%B6f_theorem), but we still get a list, so we have to use the bracket operator, `[0]`, to select the first one.

```python
solutions = solve(Eq(at_0, p_0), C1)
type(solutions), len(solutions)
```

    (list, 1)

```python
value_of_C1 = solutions[0]
```

Now in the general solution, we want to replace `C1` with the value of `C1` we just figured out.

```python
particular = general.subs(C1, value_of_C1)
```

The result is complicated, but SymPy provides a method that tries to simplify it.

```python
particular = simplify(particular)
```

Often simplicity is in the eye of the beholder, but that's about as simple as this expression gets.

Just to double-check, we can evaluate it at `t=0` and confirm that we get `p_0`

```python
particular.subs(t, 0)
```

This solution is called the [logistic function](https://en.wikipedia.org/wiki/Population_growth#Logistic_equation).

In some places you'll see it written in a different form:

$f(t) = \frac{K}{1 + A e^{-rt}}$

where $A = (K - p_0) / p_0$.

We can use SymPy to confirm that these two forms are equivalent.  First we represent the alternative version of the logistic function:

```python
A = (K - p_0) / p_0
```

```python
logistic = K / (1 + A * exp(-r*t))
```

To see whether two expressions are equivalent, we can check whether their difference simplifies to 0.

```python
simplify(particular - logistic)
```

This test only works one way: if SymPy says the difference reduces to 0, the expressions are definitely equivalent (and not just numerically close).

But if SymPy can't find a way to simplify the result to 0, that doesn't necessarily mean there isn't one.  Testing whether two expressions are equivalent is a surprisingly hard problem; in fact, there is no algorithm that can solve it in general.

### Exercises

**Exercise:** Solve the quadratic growth equation using the alternative parameterization

$\frac{df(t)}{dt} = \alpha f(t) + \beta f^2(t) $

```python
# Create symbol for beta
beta = symbols('beta')
```

```python
# Now write the differential equation
eqX = Eq(diff(f(t), t), alpha*f(t) + beta*f(t)**2)
```

```python
# Solve it
solution_X = dsolve(eqX)
```

```python
# Extract the right hand side
general_X = solution_X.rhs
```

```python
# Evaluate the right-hand side at t = 0
at_0_X = general_X.subs(t, 0)
```

```python
# Now find the value of C1 that makes f(0) = p_0
# So create equation at_0_x = p_0 and solve for C1
# Use solve and not dsolve
solutions_X = solve(Eq(at_0_X, p_0), C1)
type(solutions_X), len(solutions_X)
```

    (list, 1)

```python
# Result of solve is a list of solutions
# Use bracket operator [0] to select first and only one solution
value_of_C1_X = solutions_X[0]
```

```python
# Replace C1 with the value of C1 just determined
particular_X = general_X.subs(C1, value_of_C1_X)
```

```python
# Simply above with SymPy
particular_X = simplify(particular_X)
```

```python
# Double check result by evaluating t=0 to confirm p_0
particular_X.subs(t, 0)
```

```python
# Above shows the equation solved
```

**Exercise:**  Use [WolframAlpha](https://www.wolframalpha.com/) to solve the quadratic growth model, using either or both forms of parameterization:

    df(t) / dt = alpha f(t) + beta f(t)^2

or

    df(t) / dt = r f(t) (1 - f(t)/K)

Find the general solution and also the particular solution where `f(0) = p_0`.

### General solution for

    df(t) / dt = alpha f(t) + beta f(t)^2

### Particular solution for

    df(t) / dt = alpha f(t) + beta f(t)^2 where f(0) = p_0

### General solution for

    df(t) / dt = r f(t) (1 - f(t)/K)

### Particular solution for

    df(t) / dt = r f(t) (1 - f(t)/K) where f(0) = p_0


```python
import numpy as np
from scipy import stats

import matplotlib.pyplot as plt
plt.style.use('fivethirtyeight')
```

# Inverse Transform sampling

## Rationale

**Inverse transform sampling** allows to transform samples from uniform distribution $U$ to any other distribution $D$, given the $CDF$ of $D$.

How can we do it?

Let's take

$$\large T(U) = X$$

where:

* $U$ is a uniform random variable
* $T$ is some kind of a transformation
* $X$ is the target random variable (let's use **exponential** distribution as an example)

Now, we said that to perform **inverse transformation sampling**, we need a $CDF$.

By definition $CDF$ (we'll call it $F_X(x)$ here) is given by:

$$\large F_X(x) \triangleq P(X \leq x)$$

We said before that to get $X$, we'll apply certain transformation $T$ to a uniform random variable.

We can then say, that:

$$\large P(X \leq x) = P(T(U) \leq x)$$

Now, let's apply an ibnverse of $T$ to the both sides of the inequality:

$$\large = P(U \leq T^{-1}(x))$$

Uniform distribution has a nice property that it's $CDF$ at any given point $x$ is equal to the value of $x$.

Therefore, we can say that:

$$\large = T^{-1}(x)$$

and conclude that:

$$\large F_X(x) = T^{-1}(x)$$

## Conclusion

We demonstrated how to sample from any density $D$ using a sample from a uniform distribution and an inverse of $CDF$ od $D$.

Now, let's apply it in practice!

## Code

Let's see how to apply this in Python.

We'll use **exponential distribution** as an example.

```python
# Define params
SAMPLE_SIZE = 100000
N_BINS = np.sqrt(SAMPLE_SIZE).astype('int') // 2
LAMBDA = 8
```

Let's instantiate distributions.

We will instantiate an exponential distribution expicitly for comparison purposes.

___________

Note that **`scipy.stats`** has a slightly **different parametrization** of exponential than the populuar $\lambda$ parametrization.

In the [documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.expon.html), we read:

*A common parameterization for expon is in terms of the rate parameter lambda, such that pdf = lambda * exp(-lambda * x). This parameterization corresponds to using scale = 1 / lambda.*

____________

Therefore, we're going to use **`scale=1/LAMBDA`** to parametrize our test **exponential distribution**.

```python
# Instantiate U(0, 1)
unif = stats.uniform(0, 1)

# Instantiate Exp(8) for comparison purposes
exp = stats.expon(loc=0, scale=1/LAMBDA)
```

Now, we need to define the inverse transformation $T^{-1}(x)$ that will allow us to translate between uniform and exponential samples.

The $CDF$ of exponential distribution is defined as:

$$\large
\begin{equation}
  F_X(x) \triangleq
    \begin{cases}
      1 - e^{-\lambda x} \ \text{  for  }\ x \geq 0\\
      0 \ \ \ \ \ \ \ \ \ \ \ \ \ \  \text{for }\ x<0 \\
    \end{cases}
\end{equation}
$$

Let's take the inverse of this function (solve for $x$):

$$\large y = 1 - e^{-\lambda x}$$

* subtract $1$ from both sides:

$$\large 1 - y = - e^{-\lambda x}$$

* take $ln$ of both sides:

$$\large ln(1 - y) = \lambda x$$

* divide both sides by $\lambda$:

$$\large x = \frac{ln(1 - y)}{\lambda}$$

**Et voilà!** 🎉🎉🎉

We've got it! 💪🏼

Let's translate it to Python code:

```python
# Define
def transform_to_exp(x, lmbd):

    """Transoforms a uniform sample into an exponential sample"""

    return -np.log(1 - x) / lmbd
```

Take samples:

```python
# Sample from uniform
sample_unif = unif.rvs(SAMPLE_SIZE)

# Sample from the true exponential
sample_exp = exp.rvs(SAMPLE_SIZE)

# Transform U -> Exp
sample_transform = transform_to_exp(sample_unif, LAMBDA)
```

A brief sanity check:

```python
# Sanity check -> U(0, 1)
plt.hist(sample_unif, bins=N_BINS, density=True)
plt.title('Histogarm of $U(0, 1)$')
plt.ylabel('$p(x)$')
plt.xlabel('$x$')
plt.show()
```

..and let's compare the resutls:

```python
plt.hist(sample_exp, bins=N_BINS, density=True, alpha=.5, label='Exponential')
plt.hist(sample_transform, bins=N_BINS, density=True, alpha=.5, label='$T(U)$')
plt.legend()
plt.title('Histogram of exponential and transformed distributions', fontsize=12)
plt.ylabel('$p(x)$')
plt.xlabel('$x$')
plt.show()
```

Beautiful! It worked as expected 🎉🎉🎉

sampling/00 - Inverse transform sampling.ipynb


# Binomial Trees

```python
import math
```

```python
def exp(x):
    return math.exp(x)
```

## Application of Binomial Trees

Binomial trees are a popular technique for pricing options. This method relies on the assumption that stock prices follow a random walk and at each time step, it has a certain probability of moving up or down by a certain percentage. Furthermore, it requires the assumption that no arbitrage opportunities exist. The binomial tree represents the different paths a stock price can follow. In the limit, this model converges to the Black-Scholes-Merton model, which is described [here](european_plain_vanilla_option.ipynb).

## Example - One-Step Binomial Tree

Consider a stock which currently trades at 50. We are interested in pricing a European call option with a strike of 50 and maturity in six months from now. We know that at maturity, the stock will either move up to 55 or move down to 45. Hence, if the stock price moves up, the option will have a value of 5; if the stock price moves down, the option will have the value zero.

Assuming a risk neutral world, we can set up a portfolio of the stock itself and the option so that we already know the value of the portfolio at maturity. Therefore, we need to calculate the delta of shares we need to hold to make the portfolio riskless. We can calculate delta as

$$\Delta=\frac{c_u-c_d}{S_0u-S_0d},$$

where $c_u$ and $c_d$ is the value of the option assuming the stock moves up respectively down and $S_0u$ and $S_0d$ is the value of the stock if it moves up respectively down. Hence, delta is given as

$$\Delta=\frac{5-0}{55-45}=0.5.$$

Now, we can create a riskless portfolio by holding a long position in $\Delta$ shares and short one call option. If the stock price moves up to 55, the value of the portfolio is $55\cdot0.5-5=22.5$; if the stock price moves down to 45, the value of the portfolio is $45\cdot0.5-0=22.5$. Discounting the portfolio value at maturity assuming a risk-free interest rate $r$ of 0.05 gives as a present value of the portfolio of

$$22.5e^{-0.05\cdot6/12}=21.94.$$

In order to determine the option price today, we know that the portfolio value is the value of the long position in $\Delta$ shares and the short position in one option and we further now that this value is $21.94$. Hence, it follows that

$$50\cdot0.5-c=21.94$$ or $$c = 3.06.$$

The following code shows how the option value can be calculated.

```python
S0 = 50
K = 50
S0_u = 55
S0_d = 45
r= 0.05
T = 0.5

c_u = max(S0_u-K,0)
c_d = max(S0_d-K, 0)

delta = (c_u-c_d)/(S0_u-S0_d)

pf_value_u = (S0_u*delta-c_u)*exp(-r*T)
pf_value_d = (S0_d*delta-c_d)*exp(-r*T)

c = -(pf_value_u-(S0*delta))
print(c)
```

## Generalization of the One-Step Binomial Tree

The following formula allows the pricing of an option using a one-step binomial tree. The value of an option $o$ is defined as

$$ o = e^{-rT}[po_u+(1-p)o_d]$$

where $o_u$ describes the value of the option in case of an upward-movement of the stock and $o_d$ in case of a downward-movement of the stock with the probability of an upward-movement $p$ defined as

$$p=\frac{e^{rT}-d}{u-d}$$

where $u$ is the upward-moving factor and $d$ is the downward-moving factor, i.e. $u-1$ is the percentage increase in case of an upward-movement and $1-d$ is the percentage decrease in case of an downward-movement.

For a detailed description how the formulas are derived please refer to Hull, *Options, futures, and other derivatives, 8th Edition,* 2012, p. 265.

We now can take the example from before using these formulas.

```python
S0 = 50
K = 50
r = 0.05
T = 0.5
u = 1.1
d = 0.9
S0_u = S0*u
S0_d = S0*d

#Call
o_u = max(S0_u-K,0)
o_d = max(S0_d-K, 0)

#Put - uncomment the following lines to calculate the value of a put option instead of a call option
#o_u = max(K-S0_u,0)
#o_d = max(K-S0_d, 0)

p = (exp(r*T)-d)/(u-d)

o = exp(-r*T)*(p*o_u+(1-p)*o_d)
print(o)
```

## Two-Step Binomial Tree

The analysis of the option value using the one-step binomial tree can be extended to a two-step binomial tree. Hence, let A denote the node at start $S_0$. In the first step, the stock price either moves up by $u$ or down by $d$ to $S_0u$ or $S_0d$ where $S_0u$ is node B and $S_0d$ is node C of the binomial tree. In the second step, the stock price again goes up by $u$ or down by $d$ from each node. Hence from node B, the stock price goes up to $S_0uu$ (node D), or down to $S_0ud$ (node E). From node C, the stock price either goes up to $S_0du$ which equals $S_0ud$ (node E) or down to $S_0dd$ node (F). Hence, we derive the following nodes from the two-step binomial model.

$A = S_0$,
$B = S_0u$,
$C = S_0d$,
$D = S_0uu$,
$E = S_0ud/S_0du$,
$F = S_0dd$.

In order to determine the option price today, we need to go backwards starting at the nodes at maturity to determine the option prices at the previous nodes. Therefore, we need the stock prices at each node A to F which are given as

$A = 50.00$,
$B = 55.00$,
$C = 45.00$,
$D = 60.50$,
$E = 49.50$,
$F = 40.50$.

Hence, at nodes D to F the option price is either $max(S_T-K, 0)$ in case of call option or $max(K-S_T, 0)$ in case of a put option. This leads to the following option values at the terminal nodes D to F in case of a call option:

$o_D = 10.50$,
$o_E = 0.00$,
$o_F = 0.00$.

Having calculated the payoffs at maturity, we can determine the option values at nodes B and C. The values are calculated using the formula given in section 1.3 as

$$o = e^{-rT}[po_u+(1-p)o_d].$$

The probability of an upward-movement $p$ is defined as shown in section 1.2 as

$$p=\frac{e^{rT}-d}{u-d}$$ which is approximately 0.63.

Using this probability of an upward movement, the option value at node B is $o_B=e^{-0.05\cdot{0.5}}[0.63\cdot{10.50}+(1-0.63)\cdot{0}]=6.42$ and the option value at node C is $o_C=e^{-0.05\cdot{0.5}}[0.63\cdot{0}+(1-0.63)\cdot{0}]=0$.

$o_B = 6.42$,
$o_C = 0.00$.

The same can be done for the last node A which is $o_A=e^{-0.05\cdot{0.5}}[0.63\cdot{6.42}+(1-0.63)\cdot{0}]=3.92$.

Hence, the option value today at node A is 3.92.

The procedure for deriving the option value can be reproduced using the following code cell.

```python
S0 = 50
K = 50
r = 0.05
T = 0.5
u = 1.1
d = 0.9

S0_u = S0*u
S0_d = S0*d

S0_uu = S0*u**2
S0_ud = S0*u*d #Equals S0_du == S0*d*u
S0_dd = S0*d**2

p = (exp(r*T)-d)/(u-d)

o_F = max(S0_dd-K, 0) #In the case of a put option this needs to be max(K-S0_dd, 0)
o_E = max(S0_ud-K, 0)
o_D = max(S0_uu-K, 0)

o_C = exp(-r*T)*(o_E*p+o_F*(1-p))
o_B = exp(-r*T)*(o_D*p+o_E*(1-p))

o_A = exp(-r*T)*(o_B*p+o_C*(1-p))

print('Value of the call option = ',o_A)
```

## Generalization of the Two-Step Binomial Tree

The formula for the two-step binomial tree can be generalized as follows: Let $r$ denote the risk-free interest rate and $\Delta{t}$ denote the length of a time step. The probability of an upward movement is $p$ and the option value is $o$ (e.g. the option value after an upward movement is $o_u$). The option value at each time step $\Delta{t}$ is defined as

$$o = e^{-r\Delta{t}}[po_u+(1-p)o_d]$$
with
$$p=\frac{e^{r\Delta{t}}-d}{u-d}$$

Repeating this step for the relevant nodes gives

\begin{align}
o_u &= e^{-r\Delta{t}}[po_uu+(1-p)o_ud] \\
\\
o_d &= e^{-r\Delta{t}}[po_ud+(1-p)o_dd] \\
\\
o &= e^{-r\Delta{t}}[po_u+(1-p)o_d] \\
\end{align}

which can be summarized to

$$o = e^{-2r\Delta{t}}[p^2o_{uu}+2p(1-p)o_{ud}+(1-p^2)o_{dd}].$$

## Application of Binomial Trees in Option Pricing

The most commonly application of binomial trees in option pricing is the case of American options. In contrast to European options, American options can be exercised at any time before expiry, and this give the holder of the option more rights than in the case of an European option. The valuation of American options using binomial trees is explained in the [American Plain Vanilla Option](american_plain_vanilla_option.ipynb) notebook.

---


# Dynamic programming

This notebook is based on the tutorial on Dynamic programming on this webpage (in Russian): https://bestprogrammer.ru/izuchenie/uchebnik-po-dinamicheskomu-programmirovaniyu-sozdanie-effektivnyh-programm-na-python

## Imports

```python
import time
import matplotlib.pyplot as plt
```

## Recursive algorithm for Fibonacci numbers

```python
%%timeit

def fib(n):
    if n <= 0:
        return 0
    if n == 1:
        return 1
    else:
        return fib(n-1) + fib(n-2)

fib(10)
```

    17.5 µs ± 228 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

## Recursive algorithm with memoization for Fibonacci numbers

```python
%%timeit

cache = {}

def fib(n):
    if n <= 0:
        return 0
    if n == 1:
        return 1
    elif n in cache:
        return cache[n]
    else:
        cache[n] = fib(n-1) + fib(n-2)
        return cache[n]

fib(10)
```

    2.95 µs ± 27.8 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

We can see that with memoization, the runtime decreases from 18 to 3 microseconds, that is, we get a six-fold improvement.

## Knapsack problem

Based on the description from https://en.wikipedia.org/wiki/Knapsack_problem as the original statement is incomprehensible.

We consider the problem of choosing some items from the given $n$ items such that the total value of these items is maximized but the total weight of these items does not exceed a given maximum capacity. This is called **0-1 knapsack problem** because we either take an item (1) or do not (0); we are not allowed to take fractions of the items.

Let's denote by $x_i$ if we take the $ith$ item or not: $x_i \in \{0, 1\}$. The value and the weight of the $i$th item are $v_i$ and $w_i$, respectively. The maximum allowed capacity is $M$.

Then, mathematically we solve the following problem:
$$
\begin{align}
\text{maximize}   & \sum_{i=1}^n x_i v_i \\
\text{subject to} & \sum_{i=1}^n x_i w_i \leq M,
\end{align}
$$
where $x_i \in \{0, 1\}$.

The following implementation and test problem follows https://en.wikipedia.org/wiki/Knapsack_problem

```python
def solve_knapsack_problem(v, w, M):
    """Solve knapsack problem with values `v`, weights `w`, and capacity `M`."""
    n = len(v)
    assert len(v) == len(w)

    cache = {}

    for i in (range(0, n)):
        cache[i, 0] = 0
    for j in (range(0, M+1)):
        cache[0, j] = 0

    for i in range(1, n):
        for j in range(0, M+1):
            if w[i] > j:
                cache[i, j] = cache[i-1, j]
            else:
                cache[i, j] = max(
                    cache[i-1, j],
                    cache[i-1, j-w[i]] + v[i]
                )

    return cache[n-1, M]
```

```python
v = [5, 4, 3, 2]
w = [4, 3, 2, 1]
M = 6

solve_knapsack_problem(v, w, M)
```

    9

## Coin change problem

```python

```


# Factorial digits

How many digits does the factorial of $n$ have?

```python
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import sympy
```

Compute the factorial of numbers ranging over several orders of magnitude.

```python
order = 5
x_values = np.logspace(0, order, 4*order + 1, dtype=np.int64)
fac_values = [sympy.factorial(x) for x in x_values]
nr_digits = list(map(lambda x: len(str(x)), fac_values))
```

Plot the result in a $\log$-$\log$ plot.

```python
figure, axes = plt.subplots(nrows=1, ncols=2, figsize=(12, 6))
axes[0].loglog(x_values, fac_values)
axes[0].set_xlabel(r'$n$')
axes[0].set_ylabel(r'$n!$')
axes[1].loglog(x_values, nr_digits)
axes[1].set_xlabel(r'$n$')
axes[1].set_ylabel(r'nr. digits of $n!$')
figure.tight_layout()
```

The number of digits grows approximately linear with $n$.  This is easy to see since $n! = 10^{\log_{10} n!}$, $n! = \prod_{i=1}^{n} i$ and hence $\log_{10} n! = \sum_{i=1}^{n} \log_{10} i$.  Now we can bound this sum from above since $\log i < 1$ for $i < 10$, $\log i < 2$ for $i < 100$, and so on.  So if $n = 10^p$, $\log_{10} n! < 1 \times 10 + 2 \times 100 + 3 \times 1000 + \ldots + p \times 10^p$.  So $\log_{10} 10^p! < (p + 1) \times 10^p$.  Since the number of digits of an integer $n$ is $\lceil \log_{10} n \rceil$, this proves the point.


# Homework 0

## Exercise

Get used to using python as a calculator. Try computing all of the following:

(i) $10 \sqrt{(2(11.4+12.6))}$

```python
import numpy as np
a = 10*np.sqrt(2*(11.4+12.6))
a
```

    69.28203230275508

(ii) $e^{0.5}$

```python
import numpy as np
a = np.exp(0.5)
a
```

    1.6487212707001282

(iii) the value of $y=(10+sin(x)+x^2+cosh(x^2+1))/(sinh(x)+x^3+3x^4+e^x)$ for $x=\pi$ and $x=0.2$

```python
import numpy as np

def value(xValue):
  x = xValue
  num = 10+np.sin(x)+x**2+np.cosh(x**2+1)
  dem = np.sinh(x) + x**3 + 3*x**4 +np.exp(x)
  y = num/dem
  return y

print("When x = pi, the value of y equals: " , value(np.pi))
print("When x = 0.2, the value of y equals: " , value(0.2))

```

    When x = pi, the value of y equals:  73.47132985890015
    When x = 0.2, the value of y equals:  8.240812092855817

## Exercise

Write a function that takes as arguement $r$ for the radius of a circle and returns the diameter $r$ of the circle, as well as the circle's circumference $c$ and the area $a$. Try the function for $r=6$.

```python
import numpy as np

def triangleInfo(radius):
  r = radius
  d = 2*r
  c = 2*np.pi*r
  a = np.pi*r**2
  infoList = [d,c,a]
  return infoList

info = triangleInfo(radius=6)
print("Given radius r=6, the circle's diameter d is", info[0],
      " circumference c is ", info[1], ", and area a is ", info[2])
```

    Given radius r=6, the circle's diameter d is 12  circumference c is  37.69911184307752 , and area a is  113.09733552923255

## Exercise

Write a for loop to sum up all natural numbers from $1$ to $1,000,000$.

```python
## Method 1
def sequenceSum(startVal,endVal, increment):
  sum = 0
  item = startVal
  while (item < endVal) | (item == endVal):
    sum += item
    item +=increment
  return sum

sum1 = sequenceSum(1,1000000,1)
print("method 1", sum1)

## Method 2
a= range(1,1000001,1) # notice the exclusive or range at the right boundary
sum2 = sum(a)
print("method 2", sum2)
```

    method 1 500000500000
    method 2 500000500000

## Exercise

Compute the sum $\sum_{i=1}^{10}(i+1)$.

```python
def sequenceSum(startVal,endVal,increment,bias):
  sum = 0
  item = startVal
  while (item < endVal) | (item == endVal):
    sum += item
    item +=increment
  return sum

sum1 = sequenceSum(1,10,1,bias = 1)
print("The sum of sequence here is ", sum1)
```

    The sum of sequence here is  55

## Exercise

The curve $f(x)=x^2+3x+\sqrt5$ intersects the curve $g(x)=-x^2-2x+1$ at two points. Find the coordinates of the intersections and the gradients of the tangents to each curve at those points.

```python
"""Method 1"""
import numpy as np
import sympy as sy

def find_intersections(x,line_1,line_2):
    intersection_set = set()
    # print(type(intersection_set))
    N = len(x)-1
    index = 0
    while index < N:
        current_x, current_y1, current_y2 = x[index], line_1[index], line_2[index]
        dif = np.abs(current_y2 - current_y1)
        if dif < 1e-3:
            # print(dif)
            intersection_set.add((round(current_x, 3), round(current_y1, 2)))
        index += 1
    intersection_list = list(intersection_set)
    return intersection_list

def fun1(x):
    y1 = np.square(x)+3*x+np.sqrt(5)
    return y1

def fun2(x):
    y2 = - np.square(x)-2*x+1
    return y2

def FindGradients(current_x,tag="fun1"):
    import sympy as sy

    x = sy.symbols('x')
    y1 = x**2 + 3*x + sy.sqrt(5)
    y2 = - x**2 - 2*x + 1
    y1_dif = sy.diff(y1, x)
    y2_dif = sy.diff(y2, x)

    if tag == "fun1":
        difValue = y1_dif.subs(x,current_x)
        difValue = round(difValue.evalf(),2)
        print("The gradient of function1 at x = ", current_x, " is ", difValue)
    if tag == "fun2":
        difValue = y2_dif.subs(x,current_x)
        difValue = round(difValue.evalf(),2)
        print("The gradient of function2 at x = ", current_x, " is ", difValue)

t = np.linspace(-10, 10, 200000)
curve_1 = fun1(t)
curve_2 = fun2(t)

intersection_points = find_intersections(t, curve_1, curve_2)
print("The coordinates of intersection points are ", intersection_points)

for point in intersection_points:
    x, y = point
    FindGradients(x,"fun1")
    FindGradients(x,"fun2")
```

    The coordinates of intersection points are  [(-0.278, 1.48), (-2.222, 0.51)]
    The gradient of function1 at x =  -0.278  is  2.44
    The gradient of function2 at x =  -0.278  is  -1.44
    The gradient of function1 at x =  -2.222  is  -1.44
    The gradient of function2 at x =  -2.222  is  2.44

## Exercise

The Taylor series for sinh(x) is given by,

$$sinh(x)=x+\frac{x^3}{3!} + \frac{x^5}{5!} + \frac{x^7}{7!}+ ...$$

Write a function that finds an approximate solution for $sinh(x)$. Try your function for $x=4$ and compute the error between your approximate answer and numpy's `sinh()` function. Report how many terms you need in order to achieve an error

$$\epsilon < 1e-4$$.

```python
import numpy as np
import sympy as sy

global x
x = sy.symbols('x')

def approximate_sinh(term_Num = 3):
    global x
    # x = sy.symbols('x')
    exp = 0
    n=1
    while n< term_Num+1:
        exp += x**(2*n-1)/sy.factorial(2*n-1)
        n +=1

    # print(exp)
    return exp

N = range(1, 10)
i = 1
while i < len(N):
    # print(i)
    sym_exp = approximate_sinh(term_Num = i)
    error = np.abs(np.sinh(4)-sym_exp.subs(x,4))
    if error < 1e-4:
        print("The required minimum number of terms to accurately approximating sinh() is ",i)
        print("And the current expression is ", sym_exp)
        break
    else:
        i += 1
```

    The required minimum number of terms to accurately approximating sinh() is  8
    And the current expression is  x**15/1307674368000 + x**13/6227020800 + x**11/39916800 + x**9/362880 + x**7/5040 + x**5/120 + x**3/6 + x

## Exercise

Consider the functions $f(x)=x^2$, $g(x)=sin(x)$, and $h(x)=sinh(x+1)-e^{2x}$. Write lambda functions that represents each of these functions. Then write another function that takes as its arguements two functions and the value of $x$ and returns the composition of the functions (e.g., $f(g(x))=f\circ g(x)$). Use this to find:

(i) $g\circ f(5.0)$

(ii) $f \circ g(2)$

(iii) $h \circ(g\circ f)(2\pi)$

```python
""" Lambda functions, also referred to as 'Anonymous function'
is same as a regular python function but can be defined without a name.
ref: https://www.machinelearningplus.com/python/lambda-function/#1.-What-is-Lambda-Function-in-Python?"""

import numpy as np

# calculate squares using lambda
f = lambda x: x*x
g = lambda x: np.sin(x)
h = lambda x: np.sinh(x+1)-np.exp(2*x)

def nesting_g_f(f, g, x_value):
    return g(f(x_value))

print(nesting_g_f(f, g, x_value=5))
```

    -0.13235175009777303

```python
""" Lambda functions, also referred to as 'Anonymous function'
is same as a regular python function but can be defined without a name.
ref: https://www.machinelearningplus.com/python/lambda-function/#1.-What-is-Lambda-Function-in-Python?"""

import numpy as np

# calculate squares using lambda
f = lambda x: x*x
g = lambda x: np.sin(x)
h = lambda x: np.sinh(x+1)-np.exp(2*x)

def nesting_f_g(f, g, x_value):
    return f(g(x_value))

print(nesting_f_g(f, g, x_value=2))
```

    0.826821810431806

```python
""" Lambda functions, also referred to as 'Anonymous function'
is same as a regular python function but can be defined without a name.
ref: https://www.machinelearningplus.com/python/lambda-function/#1.-What-is-Lambda-Function-in-Python?"""

import numpy as np

# calculate squares using lambda
f = lambda x: x*x
g = lambda x: np.sin(x)
h = lambda x: np.sinh(x+1)-np.exp(2*x)

def nesting_h_g_f(f, g, h,x_value):
    return h(g(f(x_value)))

print(nesting_h_g_f(f, g, h, x_value=2*np.pi))
```

    -3.5295864605536447

## Exercise

Implement the following matirx addition in Python using Numpy.

$$
 \mathbf{A} + \mathbf{B} = \begin{bmatrix}
1 & 2 \\
3 & 4 \\
\end{bmatrix}
+
\begin{bmatrix}
1 & 1  \\
1 & 1 \\
\end{bmatrix}
=
\begin{bmatrix}
2 & 3  \\
4 & 5 \\
\end{bmatrix}
$$

```python
""" Implementing matrix operation with numpy only
ref: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
"""
import numpy as np

A = np.matrix('1 2; 3 4')
B = np.matrix('1 1; 1 1')
C = A+B
print(C)
```

Write a double for-loop that iterates over the elements of the numpy arrays and adds the two matrices together.

```python
""" Implementing matrix operation with numpy only
ref: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
"""
import numpy as np

A = np.matrix('1 2; 3 4')
B = np.matrix('1 1; 1 1')
list_C = []
item_B = 1  # as all the elements of B equal to ONE
for row_A in A:
    print(row_A)
    for item_A in row_A:
        item_C = item_A + item_B
        list_C.append(item_C)

print(list_C)
print(np.array(list_C))
```

    [[1 2]]
    [[3 4]]
    [matrix([[2, 3]]), matrix([[4, 5]])]
    [[[2 3]]

     [[4 5]]]


# Introdução à Computação Simbólica com _sympy_

## Motivação

- O valor de $\pi$ que você usa é finito...

```python
from math import pi
print(pi)
3.141592653589793
```
- E se pudéssemos usá-lo com precisão infinita?

- 3.141592653589793 é um valor razoavelmente aceitável

- Exemplo: a equipe de engenharia da NASA explica que, usando este valor para calcular o perímetro de uma circunferência com diâmetro igual a 25 bilhões de milhas, o erro de cálculo é próximo de 1,5 polegada [[NASA]](https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimals-of-pi-do-we-really-need/).

## O que é computação simbólica

> *Computação Simbólica* (CS) é uma subárea de estudo da matemática e da ciência da computação que se preocupa em resolver problemas usando objetos simbólicos representáveis em um computador.

## Para que serve CS?

- Base de vários *sistemas de computação algébrica* (SCAs).

- Álgebra computacional, projetos assistidos por computação (CAD)

- Raciocínio automatizado, gestão do conhecimento, lógica computacional, sistemas formais de verificação etc.

### Como a CS está integrada?

Fonte: [[RISC/JKU]](https://risc.jku.at/studying-symbolic-computation/)

## Principais SCAs

- Maple

- Mathematica

- MuPad

- Sagemath

- ...

## Por que *sympy*?

> O objetivo principal do *sympy* é ser uma biblioteca de manipulação simbólica para Python.

- 2006 em diante (2019, v. 1.5.1)

Principais características:

- é gratuito;

- é baseado inteiramente em Python;

- é leve e independente.

## Objetos numéricos x objetos simbólicos

Importaremos os módulos `math` e `sympy` para ver diferenças

```python
import math as mt
import sympy as sy
sy.init_printing(pretty_print=True) # melhor impressão de símbolos
```

```python
mt.pi # numérico
```

```python
sy.pi # simbólico
```

Verifiquemos com `type`.

```python
type(mt.pi)
```

    float

```python
type(sy.pi) # é um objeto simbólico
```

    sympy.core.numbers.Pi

Vejamos mais um exemplo:

```python
mt.sqrt(2)
```

```python
sy.sqrt(2)
```

```python
type(mt.sqrt(2))
```

    float

```python
type(sy.sqrt(2)) # é um objeto simbólico
```

    sympy.core.power.Pow

### Função x método

A partir deste ponto, poderemos ver situações como as seguintes:

- `f(x)`: a função `f` é aplicada ao parâmetro `x`; ex. `print('a')`; `type('a')`

- `a.f()`: `f` é um método sem parâmetro do objeto `a`; ex. `z.conjugate()`

- `a.f(x)`: `f` é um método com parâmetro `x` do objeto `a`; ex. `mt.sqrt(2)`

A partir do último exemplo, podemos dizer que um método é, na verdade, uma função que pertence a um objeto.

### Atribuições com símbolos

Podemos atribuir símbolos a variáveis usando a função `symbols`.

```python
x = sy.symbols('x')
y = sy.symbols('y')
```

`x` e `y` são símbolos sem valor definido.

```python
x
```

```python
y
```

Podemos operar aritmeticamente com símbolos e obter uma expressão simbólica como resultado.

```python
z = sy.symbols('z')
x*y + z**2/3 + sy.sqrt(x*y - z)
```

**Exemplo**: escreva o produto notável $(x - y)^2$ como uma expressão simbólica.

```python
x**2 - 2*x*y + y**2
```

Note que o nome da variável não tem a ver com o nome do símbolo. Poderíamos fazer o seguinte:

```python
y = sy.symbols('x') # y é variável; x é símbolo
y
```

### Atribuição por desempacotamento

Também poderíamos realizar as atribuições anteriores da seguinte forma:

```python
x, y, z = sy.symbols('x y z')
```

### Alfabeto de símbolos

O *sympy* dispõe de um submódulo chamado `abc` do qual podemos importar símbolos para letras latinas (maiúsculas e minúsculas) e gregas (minúsculas).

```python
from sympy.abc import a,b,c,alpha,beta,gamma
(a + 2*b - 3*c)*(alpha/3 + beta/2 - gamma) # símbolico
```

```python
from sympy.abc import D,G,psi,theta
D**a * G**b * psi**c * theta**2 # símbolico
```

**Nota**: algumas letras já são usadas como símbolos especiais, tais como `O`, que indica "ordem" e `I`, que é o complexo $i$. Neste caso, cuidado deve ser tomado com nomes de variáveis

```python
sy.I # imaginário simbólico
```

```python
type(sy.I)
```

    sympy.core.numbers.ImaginaryUnit

### Símbolos com nomes genéricos

Para criar símbolos genéricos, temos de usar `symbols` ou `Symbol`.

```python
sem_nocao = sy.symbols('nada')
sem_nocao
```

```python
muito_louco = sy.Symbol('massa')
muito_louco
```

### Variáveis e símbolos

```python
sem_medo = sem_nocao + 2
sem_medo
```

```python
soma = muito_louco + 2
muito_louco = 3 # 'muito_louco' aqui não é o simbólico
soma
```

## Substituição

A operação de *substituição* permite que:

1. substituamos variáveis por valores numéricos para avaliar uma expressão ou calcular valores de uma função em um dado ponto.
2. substituamos uma subexpressão por outra.

Para tanto, procedemos da seguinte forma:

```python
expressao.subs(variavel,valor)
```

**Exemplo**: considere o polinômio $P(x) = 2x^3 - 4x -6$. Calcule o valor de $P(-1)$, $P(e/3)$, $P(\sqrt{3.2})$.

```python
from sympy.abc import x
P = 2*x**3 - 4*x - 6
P1 = P.subs(x,-1)
Pe3 = P.subs(x,mt.e/3)
P32 = P.subs(x,mt.sqrt(3.2))
print(P1, Pe3, P32)
```

    -4 -8.13655822141297 -1.70674948320040

**Exemplo:** sejam $f(x) = 4^x$ e $g(x) = 2x - 1$. Compute o valor da função composta $f(g(x))$ em $x = 3$.

```python
f = 4**x
fg = f.subs(x,2*x - 1)
```

```python
fg.subs(x,3)
```

Poderíamos também fazer isso com um estilo "Pythônico":

```python
fg = 4**x.subs(x,2*x - 1).subs(x,3)
fg
```

**Exemplo:** se $a(x) = 2^x$, $b(x) = 6^x$ e $c(x) = \cos(x)$, compute o valor de $a(x)b(c(x))$ em $x = 4$

```python
a = 2**x
b = 6**x
c = sy.cos(x)
(a * b.subs(x,c)).subs(x,4)
```

Ou, de modo direto:

```python
valor = ( 2**x * ( 6**x.subs(x,sy.cos(x))) ).subs(x,4)
valor
```

### Avaliação de expressão em ponto flutuante

Note que a expressão anterior não foi computada em valor numérico. Para obter seu valor numérico, podemos usar o método `evalf`.

```python
valor.evalf()
```

#### Precisão arbitrária

`evalf` permite que escolhamos a precisão do cálculo impondo o número de dígitos de precisão. Por exemplo, a última expressão com 20 dígitos de precisão seria:

```python
valor.evalf(20)
```

Com 55, seria:

```python
valor.evalf(55)
```

E com 90 seria:

```python
valor.evalf(90)
```

**Exemplo**: calcule o valor de $e$ com 200 dígitos de precisão.

```python
sy.exp(1).evalf(200)
```

## Funções predefinidas x funções regulares

Apresentaremos 3 grupos de funções que podem ser criadas em Python

- **funções predefinidas** (*built-in functions*): funções já prontas que podemos usar (ex. `print()`, `type()` , `int()`, `float()`

- ** funções regulares**, ou *normais*, *definidas pelo usuário* (do inglês *user-defined functions*, ou simplesmente *UDF*): aquelas que você cria!

Podemos fazer isto de uma maneira usando uma "palavra-chave" (*keyword*) chamada `def` da seguinte forma:

```python
def f(x):
    (...)
    return y
```
- uma UDF **pode ter zero ou mais argumentos**, tantos quantos se queira;
- uma UDF **pode ou não ter valor de retorno**;

Vamos entender as UDFs com exemplos.

**Exemplo:** Suponha que você é um(a) analista de dados do mercado imobiliário e está estudando o impacto do repasse de comissões pagas a corretores mediante vendas de imóveis. Você, então, começa a raciocinar e cria um modelo matemático bastante simples que, antes de tudo, precisa calcular o valor do repasse a partir do preço de venda.

Se $c$ for o percentual de comissão, $V$ o valor da venda do imóvel e $r$ o valor a ser repassado para o corretor, então, a função a ser definida é

$$r(V) = c\, V,$$

assumindo que $c$ seja um valor fixo.

Digamos que $c$ corresponda a 1.03% do valor da venda do imóvel. Neste caso podemos criar uma UDF para calcular $r$ para nós da seguinte forma:

```python
def repasse(V):
    r = 0.0103*V
    return r
```

Para $V = \, R\$ \, 332.130,00$:

```python
repasse(332130)
```

O que é necessário observar:

- `def` seguido pelo *nome* da função
- argumentos enclausurados por parênteses
- os dois-pontos (`:`) são obrigatórios
- *escopo* da função, que deve ser escrito em uma ou mais linhas indentadas (pressione `TAB` para isso, ou use 4 espaços)
- o valor de retorno, se houver, é posto na última linha do escopo.

Podemos atribuir os valores do argumento e resultado a variáveis:

```python
V = 332130
rep = repasse(V)
rep
```

Nomes iguais de variável e função são permissíveis.

```python
repasse = repasse(V) # 'repasse' à esquerda é uma variável; à direita, função
print(repasse)
```

    3420.939

Todavia, isto pode ser confuso e é bom evitar.

O estilo "Pythônico" de escrever permite que o valor de retorno não seja explicitamente declarado. No escopo

```python
...
    r = 0.0103*V
    return r
```
 a variável `r` não é necessária.

Python é inteligente para permitir o seguinte:

```python
def repasse(V):
    return 0.0103*V

# note que aqui não indentamos a linha.
# Logo esta instrução NÃO pertence ao escopo da função.
repasse(V)
```

Podemos criar uma função para diferentes valores de `c` e `V` usando *dois* argumentos:

```python
def repasse_c(c,V): # esta função tem outro nome
    return c*V
```

```python
c = 0.0234 # equivaleria a uma taxa de repasse de 2.34%
V = 197432 # o valor do imóvel agora é R$ 197.432,00
repasse_c(c,V)
```

A ordem dos argumentos importa:

```python
V = 0.0234 # este deveria ser o valor de c
c = 197432 # este deveria ser o valor de V
repasse_c(c,V)
```

Por que o valor resultante é o mesmo? Porque a operação no escopo da função é uma multiplicação, `c*V`, que é comutativa independentemente do valor das variáveis. Porém, digamos que um segundo modelo tenha uma forma de cálculo distinta para a comissão dada por

$$r_2(V) = c^{3/5} \, V$$

Neste caso:

```python
def repasse_2(c,V):
    return c**(3/5)*V

V = 197432
c = 0.0234

repasse_2(c,V)
```

Porém, se trocarmos o valor das variáveis, a função `repasse_2` calculará um valor distinto. Embora exista um produto também comutativo, o expoente `3/4` modifica apenas o valor de `c`.

```python
# variáveis com valores trocados
c = 197432
V = 0.0234

repasse_2(c,V)
```

A ordem com que escrevemos os argumentos tem importância relativa aos valores que passamos e ao que definimos:

```python
# variáveis com valores corretos
V = 197432
c = 0.0234

def repasse_2_trocada(V,c): # V vem antes de c
    return c**(3/5)*V

repasse_2_trocada(V,c)
```

Mas,

```python
# os valores das variáveis estão corretos,
# mas foram passados para a função na ordem errada
repasse_2_trocada(c,V)
```

e

```python
# a ordem dos argumentos está de acordo com a que foi definida
# mas os valores das variáveis foram trocados
V = 197432
c = 0.0234
repasse_2_trocada(c,V)
```

## Modelos matemáticos simbólicos

A partir do que aprendemos, podemos definir modelos matemáticos completamente simbólicos.

```python
from sympy.abc import c,V

def repasse_2_simbolica(c,V):
    return c**(3/5)*V
```

Se chamarmos esta função, ela será um objeto simbólico.

```python
repasse_2_simbolica(c,V)
```

Atribuindo em variável:

```python
rep_simb = repasse_2_simbolica(c,V)
```

```python
type(rep_simb) # é um objeto simbólico
```

    sympy.core.mul.Mul

**Exemplo:** Suponha, agora, que seu modelo matemático de repasse deva considerar não apenas um percentual $c$ pré-estabelecido, mas também um valor de "bônus" adicional concedido como prêmio pela venda do imóvel. Considere, então, que o valor deste bônus seja $b$. Diante disso, nosso novo modelo teria uma fórmula como a seguinte:

$$r_3(V) = c\,V + b$$

Simbolicamente:

```python
# importaremos apenas o símbolo b,
# uma vez que c e V já foram importados
# como símbolos anteriormente
from sympy.abc import b

def r3(V):
    return c*V + b

rep_3 = r3(V)
rep_3
```

### Substituindo valores

Podemos usar a função `subs` para atribuir quaisquer valores para o modelo.

**Exemplo:** $c = 0.119$

```python
rep_3.subs(c,0.119) # substituindo para c
```

**Exemplo:** $c = 0.222$

```python
rep_3.subs(c,0.222) # substituindo para c
```

**Exemplo:** $c = 0.222$ e $b = 12.0$

```python
rep_3.subs(c,0.222).subs(b,12.0) # substituindo para c, depois para b
```

### Substituição múltipla

O modo anterior de substituição não é "Pythônico". Para substituirmos mais de uma variável de uma vez, devemos usar *pares ordenados* separados por vírgula sequenciados entre colchetes como uma *lista*. Mais tarde, aprenderemos sobre pares ordenados e listas.

**Exemplo:** Modifique o modelo $r_3$ para que $c = 0.043$ e $b = 54.0$

```python
# espaços foram adicionados para dar legibilidade
rep_3.subs( [ (c,0.043), (b,54.0) ] )
```

#### Pares ordenados

$$X \times Y = \{ (x,y) ; x \in X \text{ e } y \in Y \}$$

onde $X$ e $Y$ são conjuntos quaisquer e $x$ e $y$ são as *coordenadas*.

- ex. $X = Y = \mathbb{R}$; $(3,2)$, $(-1,3)$, $(\pi,2.18)$ etc.

- Este é o caso de $\mathbb{R} \times \mathbb{R} = \mathbb{R}^2$, que é exatamente o *plano cartesiano*.

A substituição múltipla com `subs` ocorre da seguinte forma;

- a primeira coordenada é o *símbolo*;

- a segunda coordenada é o *valor* que você quer dar para o símbolo.

**Exemplo:** Calcule $r_3(V)$ considerando $c = 0.021$, $b = 34.0$ e $V = 432.000$.

```python
# armazenaremos o valor na variável 'valor'
valor = r3(V)

# subsituição
valor.subs( [ (c,0.021), (b,54.0) ] )
```

Com o estilo "Pythônico":

```python
valor = r3(V).subs( [ (c,0.021), (b,54.0) ] ) #
valor
```

Podemos seguir esta regra de pares para substituir todos os valores de um modelo simbólico genérico não necessariamente definido através de uma função. Veja o exemplo aplicado a seguir.

## Exemplo de aplicação: o índice de caminhabilidade

O *índice de caminhabilidade* $W$ para uma vizinhança de casas é uma medida matemática que assume valores no intervalo $[0,1]$. A fórmula é definida por:

$$W(d) = e^{-5 \left( \dfrac{d}{M} \right)^5},$$

onde $d$ é a distância medida entre a vizinhança (0 metro) e um dado ponto de referência, e $M$ é a distância máxima de avaliação considerada a partir da qual a caminhabilidade é assumida como nula.

### Interpretação

- quando estamos na vizinhança, $d = 0$, $W = 1$ e a caminhabilidade é considerada ótima.

- à medida que nos afastamos da vizinhança em direção ao local da amenidade, $d$ aumenta e o valor $W$ decai vertiginosamente até atingir o valor limite $M$ a partir do qual $W = 0$ e a caminhabilidade é considerada "péssima".

- W é calculado com relação a um ponto de destino definido
- A distância deve levar em consideração as vias de circulação (ruas, rodovias etc) e não a distância mais curta (raio do perímetro).
- ex. Para $M = 500 \, m$ , um bar a 100 metros da vizinhança teria um índice de caminhabilidade maior do que o de uma farmácia localizada a 300 m e muito maior do que o de um shopping localizado a 800 m, ainda que muito famoso.

Fonte: *De Nadai, M. and Lepri, B. [[The economic value of neighborhoods: Predicting real estate prices from the urban environment]](https://arxiv.org/pdf/1808.02547.pdf)*.

### Modelo simbólico

Podemos modelar $W$ simbolicamente e calcular seu valor para diferentes valores de $d$ e $M$ usando a substituição múltipla.

```python
from sympy.abc import d,M,W

W = sy.exp(-5*(d/M)**5) # função exponencial simbólica
W
```

**Exemplo:** A nossa corretora de imóveis gostaria de entender a relação de preços de imóveis para o Condomínio Pedras de Marfim. Considerando $M = 1 km$, calcule:

- o índice de caminhabilidade $W_1$ em relação à farmácia Dose Certa, localizada a 222 m do condomínio.

- o índice de caminhabilidade $W_2$ em relação ao restaurante Sabor da Arte, localizada a 628 m do condomínio.

- o índice de caminhabilidade $W_3$ em relação ao Centro Esportivo Physicalidade, localizada a 998 m do condomínio.

- o índice de caminhabilidade $W_4$ em relação à Padaria Dolce Panini, localizada a 1,5 km do condomínio.

```python
# note que 1 km = 1000 m
W1 = W.subs([ (d,222), (M,1000) ])
W2 = W.subs([ (d,628), (M,1000) ])
W3 = W.subs([ (d,998), (M,1000) ])
W4 = W.subs([ (d,1500), (M,1000) ])
```

Perceba, entretanto, que os valores calculados ainda não são numéricos, como esperado.

```python
W1
```

```python
W2
```

```python
W3
```

```python
W4
```

Lembre-se que podemos usar `evalf` para calcular esses valores. Faremos isso considerando 3 casas decimais.

```python
# reatribuindo todos os valores
W1n = W1.evalf(3)
W2n = W2.evalf(3)
W3n = W3.evalf(3)
W4n = W4.evalf(3)

print('W1 =', W1n, '; ' \
      'W2 =', W2n, '; ' \
      'W3 =', W3n, '; ' \
      'W4 =', W4n)
```

    W1 = 0.997 ; W2 = 0.614 ; W3 = 0.00708 ; W4 = 3.24e-17

Como era de se esperar, os valores decaem de 0.997 a 3.24e-17, que é um valor considerado nulo em termos de aproximação numérica.

#### Quebrando instruções com `\`

A contra-barra `\` pode ser usada para quebrar instruções e continuá-las nas próximas linhas, porém não poderá haver nenhum caracter após ela, nem mesmo espaços. Caso contrário, um erro será lançado.

```python
print('Continuando' \
      'na linha abaixo')
```

    Continuandona linha abaixo

```python
# neste exemplo, há um caracter de espaço após \
print('Continuando' \
      'na linha abaixo')
```

### O tipo `bool`

Em Python, temos mais um tipo de dado bastante útil, o `bool`, que é uma redução de "booleano". Objetos `bool`, que têm sua raiz na chamada Álgebra de Boole, são baseados nos conceitos *true* (verdadeiro) e *false*, ou *0* e *1* e são estudados em algumas disciplinas, tais como Circuitos Lógicos, Matemática Discreta, Lógica Aplicada, entre outras.

Aprenderemos sobre operadores lógicos mais à frente. Por enquanto, cabe mencionar as entidades fundamentais `True` e `False`.

```python
True
```

    True

```python
False
```

    False

```python
type(True)
```

    bool

```python
type(False)
```

    bool

Podemos realizar testes lógicos para concluir verdades ou falsidades quando temos dúvidas sobre objetos e relações entre eles. Por exemplo, retomemos os seguintes valores:

```python
W1
```

```python
W2
```

A princípio, é difícil determinar qual dos dois é o maior. Porém, podemos realizar "perguntas" lógicas para o interpretador Python com operadores lógicos. Mostraremos apenas dois exemplos com `>` e `<`.

```python
W1 > W2 # isto quer dizer: "W1 é maior do que W2?"
```

O valor `True` confirma que o valor de `W1` é maior do que `W2`.

```python
W4 < 0
```

> Note que, de acordo com nosso modelo de caminhabilidade, este valor deveria ser zero. Porém, numericamente, ele é uma aproximação para zero. Embora muito pequeno, não é exatamente zero! Por que isso ocorre? Porque o computador lida com uma matemática inexata e aproximada, mas com precisão satisfatória.


## Numeric vs symbolic computing

```python
1.01 - 1
```

    0.010000000000000009

Floating point numbers have limited accuracy (53 bits for double precision IEEE 754)

```python
import math
math.log10(2**53)
```

    15.954589770191003

That is, about 15 decimal places. Additionally, decimal numbers don't have exact representation in binary in general. In some cases, it may lead to accumulation of errors (numerically unstable algorithms)

```python
x = 1
for i in range(10):
    x = (1.01-x)*100
    print(x)
```

    1.0000000000000009
    0.9999999999999121
    1.0000000000087939
    0.9999999991206154
    1.0000000879384574
    0.9999912061542604
    1.0008793845739605
    0.912061542603948
    9.7938457396052
    -878.38457396052

## SymPy as a calculator

Symbolic computing uses exact mathematical rules. Numbers in sympy are represented by objects of type Integer, Rational, and Float and they have overloaded mathematical operators

```python
from sympy import *
```

```python
Rational(101, 100)-Integer(1)
```

    1/100

```python
sympify("101/100-1")
```

    1/100

```python
nsimplify("1.01") - 1
```

    1/100

```python
x = Integer(1)
for i in range(10):
    x = (Rational(101, 100)-x)*100
    print(x)
```

    1
    1
    1
    1
    1
    1
    1
    1
    1
    1

Sympy has some predefined mathematical constants

```python
E*I - pi
```

    -pi + E*I

and many functions...

```python
cos(0)
```

    1

```python
sin(pi/3)
```

    sqrt(3)/2

and you can evaluate the expressions numerically with function N() or method .evalf() with arbirtary precision (based on mpmath)

```python
N(pi+E*I, 30)
```

    3.14159265358979323846264338328 + 2.71828182845904523536028747135*I

```python
sqrt(2).evalf(1000)
```

    1.414213562373095048801688724209698078569671875376948073176679737990732478462107038850387534327641572735013846230912297024924836055850737212644121497099935831413222665927505592755799950501152782060571470109559971605970274534596862014728517418640889198609552329230484308714321450839762603627995251407989687253396546331808829640620615258352395054745750287759961729835575220337531857011354374603408498847160386899970699004815030544027790316454247823068492936918621580578463111596668713013015618568987237235288509264861249497715421833420428568606014682472077143585487415565706967765372022648544701585880162075847492265722600208558446652145839889394437092659180031138824646815708263010059485870400318648034219489727829064104507263688131373985525611732204024509122770022694112757362728049573810896750401836986836845072579936472906076299694138047565482372899718032680247442062926912485905218100445984215059112024944134172853147810580360337107730918286931471017111168391658172688941975871658215212822951848847

## Symbolic computing
Symbols are python objects

```python
x, y, z = symbols("x, y, z")
alpha, beta, delta = symbols('alpha, beta, delta')
```

symbolic expressions are created using standard python syntax

```python
expr = x**y + sqrt(alpha)/2
expr
```

    sqrt(alpha)/2 + x**y

SymPy can print expressions in various forms, including $\LaTeX$, unicode pretty printing, and others. The pretty printing is enabled with `init_printing()`

```python
init_printing()
```

```python
expr
```

Lists of symbols can be created with the following notation

```python
symbols("x0:10")
```

### Basic manipulation with the expressions
#### Polynomials and rational functions

```python
(x+y)**10
```

Polynomials can be expanded

```python
expand(_)
```

and, more importantly, factorized

```python
factor(_)
```

```python
_.subs(x+y, z)
```

We can collect the terms with the same power of a given variable

```python
collect(y*x**2 + 3*x**2 - x*y + x - 1, x)
```

Cancel rational functions

```python
cancel((x**2 + 2*x + 1)/(x**2 - 1))
```

or decompose them into partial fractions

```python
apart((x**3 + 4*x - 1)/(x**2 - 1))
```

#### Trigonometric functions

```python
sin(x+y)
```

```python
expand(_, trig=True)
```

```python
trigsimp(_)
```

```python
trigsimp(sin(x)**2 + cos(x)**2)
```

#### Exponentiation

```python
x**alpha * y**alpha
```

```python
powsimp(_)
```

Note that symbols in SymPy are complex by default. However, $x^\alpha y^\alpha$ does not equal $(xy)^\alpha$ in general. For example $\sqrt{-1}\sqrt{-1} \neq \sqrt{-1\cdot-1}$.

It is valid for $x, y \ge 0$ and $\alpha\in\mathbb R$ and we have to define the symbols as such.

Similarly $(x^a)^b = x^{ab}$ holds only for $b\in\mathbb Z$

```python
x_pos, y_pos = symbols("x_pos, y_pos", positive=True)
alpha_R = symbols("alpha_R", real=True)
beta_Z = symbols("beta_Z", integer=True)
```

```python
powsimp(x_pos**alpha_R * y_pos**alpha_R)
```

```python
(expand_power_base(_))
```

```python
(z**delta)**beta_Z
```

```python
log(x_pos**alpha_R * y_pos)
```

```python
expand_log(_)
```

```python
logcombine(_)
```

The need for assumptions can be avoided using `force=True` (not recommended :)

```python
expand_log(log(x**alpha * y))
```

```python
expand_log(log(x**alpha * y), force=True)
```

#### Rewriting functions

```python
tan(x).rewrite(sin)
```

Note that there are many special functions defined in SymPy...

```python
factorial(x).rewrite(gamma)
```

```python
k, m, n = symbols('k, m, n')
binomial(k, n).rewrite(factorial)
```

```python
binomial(k, k-2)
```

```python
combsimp(_)
```

#### Equation solving
Equations are defined using `Eq`. Note that `=` is assignment and `==` tests for exact identity of two expressions

```python
Eq(x**2, 1)
```

`solveset` attempts to find all solutions of the given equation. If it fails, you may try to use `solve`...

```python
solveset(_, x)
```

```python
solveset(sin(x), x)
```

```python
solveset(x**2+1, x)
```

```python
solveset(x**2+1, x, domain=S.Reals)
```

```python
solveset(exp(x)-1, x)
```

```python
solveset(exp(x) > 1, x, domain=S.Reals)
```

```python
linsolve([x + y + z - 1, x + y + 2*z - 3 ], (x, y, z))
```

```python
nonlinsolve([x**2 + 1, y**2 + 1], [x, y])
```

## Sympy internals
Expressions are represented as trees. This can be seen by printing with srepr

```python
x*y
```

```python
srepr(x*y)
```

    "Mul(Symbol('x'), Symbol('y'))"

```python
from IPython.display import Image
def show_tree(expr):
    with open("xy.dot", "w") as f:
        f.write(dotprint(expr))
    !dot xy.dot -Tpng > xy.png
    return Image(filename='xy.png')
```

```python
show_tree(x*y)
```

```python
expr = sin(x)/y+1-x
show_tree(expr)
```

```python
expr.func
```

    sympy.core.add.Add

```python
expr.args
```

```python
expr.args[1].func
```

    sympy.core.mul.Mul

```python
expr.args[1].args
```