  GEOS 639 Geodetic Imaging

 Lab 5: Volcano Source Modeling Using InSAR  -- [20 Points]

  Franz J Meyer; University of Alaska Fairbanks

 Due Date:  April 14, 2022

  This lab will introduce you to the intersection between Geodetic Displacement data created using InSAR and Geophysical Modeling. Radar Remote Sensing can provide you with geodetic observations of surface displacement. Inverse Modeling helps you understand the physical causes behind an observed displacement.

To illuminate the handoff from geodesy to geophysics, this lab will show how to use InSAR observations to determine the most likely parameters of a volcanic magma source underneath Okmok volcano, Alaska. You will use a Mogi source model to describe the physics behind observed surface displacement at Okmok. We will again use our **Jupyter Notebook** framework implemented within the Amazon Web Services (AWS) cloud to work on this exercise.

This Lab is part of the UAF course GEOS639 Geodetic Imaging. It will introduce the following data analysis concepts:

- A Mogi Source Model describing volcanic source geometry and physics
- How to use the "grid search" method to perform a pseudo-inversion of a Mogi source model
- How to solve for the best fitting source parameters using modeling with InSAR data

  THIS NOTEBOOK INCLUDES THREE HOMEWORK ASSIGNMENTS.

 Complete all assignments to achieve full score.

 To submit your homework, please download your completed Jupyter Notebook from the server both asf PDF (*.pdf) and Notebook file (*.ipynb) and submit them as a ZIP bundle via the GEOS 639 Canvas page. To download, please select the following options in the main menu of the notebook interface:

  Save your notebook with all of its content by selecting  File / Save and Checkpoint

 To export in Notebook format, click the radio button next to the notebook file in the main Jupyter Hub browser tab. Once clicked, a download field will appear near the top of the page.

 To export in PDF format, right-click on your browser window and print the browser content to PDF

Contact me at fjmeyer@alaska.edu should you run into any problems.

```python
import url_widget as url_w
notebookUrl = url_w.URLWidget()
display(notebookUrl)
```

```python
from IPython.display import Markdown
from IPython.display import display

notebookUrl = notebookUrl.value
user = !echo $JUPYTERHUB_USER
env = !echo $CONDA_PREFIX
if env[0] == '':
    env[0] = 'Python 3 (base)'
if env[0] != '/home/jovyan/.local/envs/unavco':
    display(Markdown(f'<text style=color:red><strong>WARNING:</strong></text>'))
    display(Markdown(f'<text style=color:red>This notebook should be run using the "unavco" conda environment.</text>'))
    display(Markdown(f'<text style=color:red>It is currently using the "{env[0].split("/")[-1]}" environment.</text>'))
    display(Markdown(f'<text style=color:red>Select "unavco" from the "Change Kernel" submenu of the "Kernel" menu.</text>'))
    display(Markdown(f'<text style=color:red>If the "unavco" environment is not present, use <a href="{notebookUrl.split("/user")[0]}/user/{user[0]}/notebooks/conda_environments/Create_OSL_Conda_Environments.ipynb"> Create_OSL_Conda_Environments.ipynb </a> to create it.</text>'))
    display(Markdown(f'<text style=color:red>Note that you must restart your server after creating a new environment before it is usable by notebooks.</text>'))
```

# 0. Importing Relevant Python Packages

 First step in any notebook is to import the required Python libraries into the Jupyter environment. In this notebooks we use the following scientific libraries:

 NumPy is one of the principal packages for scientific applications of Python. It is intended for processing large multidimensional arrays.

 Matplotlib is a low-level library for creating two-dimensional diagrams and graphs. With its help, you can build diverse charts, from histograms and scatterplots to non-Cartesian coordinates graphs.

The first step is to import all required python modules:

```python
import os # for chdir, getcwd, path.basename, path.exists
import copy
import subprocess # for check_call

import matplotlib.pylab as plt # for add_subplot, cm.jet, colorbar, figure, grid, imshow, rcParams.update, savefig,
                               # set_bad, set_clim, set_title, set_xlabel, set_ylabel
import numpy as np # for arange, arctan, concatenate, cos, fromfile, isnan, ma.masked_value, min, pi, power, reshape,
                   # sqrt, square, sin, sum, tile, transpose, where, zeros
```

set up matplotlib plotting inside the notebook:

```python
%matplotlib inline
```

# 1. Introduction to the Study Site: Okmok Volcano, Alaska

 Okmok is one of the more active volcanoes in Alaska’s Aleutian Chain. Its last (confirmed) eruption was in the summer of 2008. Okmok is interesting from an InSAR perspective as it inflates and deflates heavily as magma moves around in its magmatic source located roughly 2.5 km underneath the surface. To learn more about Okmok volcano and its eruptive history, please visit the very informative site of the Alaska Volcano Observatory.

This lab uses a pair of C-band ERS-2 SAR images acquired on Aug 18, 2000 and Jul 19, 2002 to analyze the properties of a volcanic source that was responsible for an inflation of Okmok volcano of more than 3 cm near its summit. The figure to the right shows the Okmok surface displacement as measured by GPS data from field campaigns conducted in 2000 and 2002. The plots show that the displacement measured at the site is consistent with that created by an inflating point (Mogi) source.

The primary goal of the problem set is to estimate values for four unknown model parameters describing a source process beneath a volcano. The lab uses real InSAR data from Okmok volcano, so you should get some sense for how remote sensing can be used to infer physical processes at volcanoes. We will assume that the source can be modeled as an inflating point source (a so-called Mogi source) and will use a grid-search method to find the source model parameters (3D source location and volume of magma influx) that best describe our InSAR-observed surface displacement.

# 2. Downloading and Visualizing the InSAR Data

## 2.1 Download Data from AWS S3 Storage Bucket and Prep for Further Processing

We are using a pre-calculated displacement map created from C-band ERS-2 SAR images acquired on Aug 18, 2000 and Jul 19, 2002. We will pull the displacement map from an Amazon Web Services (AWS) S3 storage bucket:

Create and move to a directory in which to store our Lab 5 files:"

```python
path = f"{os.getcwd()}/lab_6_data"
if not os.path.exists(path):
    os.makedirs(path)
os.chdir(path)
print(f"Current working directory: {os.getcwd()}")
```

Download the displacement map from the AWS-S3 bucket:

```python
displacement_map_path = 's3://asf-jupyter-data-west/E451_20000818_20020719.unw'
displacement_map = os.path.basename(displacement_map_path)
!aws --region=us-west-2 --no-sign-request s3 cp $displacement_map_path $displacement_map
```

Define some variables:

```python
sample = 1100
line = 980
posting = 40.0
half_wave = 28.3
```

Read the dataset into the notebook, storing our observed displacement map in the variable "observed_displacement_map":

```python
if os.path.exists(displacement_map):
    with open (displacement_map, 'rb') as f:
        coh = np.fromfile(f, dtype='>f', count=-1)
    observed_displacement_map = np.reshape(coh, (line, sample))
```

Now we scale the measured and unwrapped InSAR phase into surface displacement in *cm* units and replace all ```nans``` with 0

```python
observed_displacement_map = observed_displacement_map*half_wave/2.0/np.pi
where_are_NaNs = np.isnan(observed_displacement_map)
observed_displacement_map[where_are_NaNs] = 0
```

 Create a mask that removes invalid samples (low coherence) from the displacement map:

```python
observed_displacement_map_m = np.ma.masked_where(observed_displacement_map==0, observed_displacement_map)
```

## 2.2 Visualize The Surface Displacement Map

 We will visualize the displacement map both in units of [cm] and as a rewrapped interferogram.

Write a function that calculates the bounding box.

```python
def extents(vector_component):
        delta = vector_component[1] - vector_component[0]
        return [vector_component[0] - delta/2, vector_component[-1] + delta/2]
```

Create a directory in which to store the plots we are about to make, and move into it:

```python
os.chdir(path)
product_path = 'plots'
if not os.path.exists(product_path):
    os.makedirs(product_path)
if os.path.exists(product_path) and os.getcwd() != f"{path}/{product_path}":
    os.chdir(product_path)
print(f"Current working directory: {os.getcwd()}")
```

Write a plotting function:

```python
def plot_model(infile, line, sample, posting, output_filename=None, dpi=72):
    # Calculate the bounding box
    extent_xvec = extents((np.arange(1, sample*posting, posting)) / 1000)
    extent_yvec = extents((np.arange(1, line*posting, posting)) / 1000)
    extent_xy = extent_xvec + extent_yvec

    plt.rcParams.update({'font.size': 14})
    inwrapped = (infile/10 + np.pi) % (2*np.pi) - np.pi
    cmap = copy.copy(plt.cm.get_cmap("jet"))
    cmap.set_bad('white', 1.)

    # Plot displacement
    fig = plt.figure(figsize=(16, 8))
    ax1 = fig.add_subplot(1, 2, 1)
    im = ax1.imshow(infile, interpolation='nearest', cmap=cmap, extent=extent_xy, origin='upper')
    cbar = ax1.figure.colorbar(im, ax=ax1, orientation='horizontal')
    ax1.set_title("Displacement in look direction [mm]")
    ax1.set_xlabel("Easting [km]")
    ax1.set_ylabel("Northing [km]")
    plt.grid()

    # Plot interferogram
    im.set_clim(-30, 30)
    ax2 = fig.add_subplot(1, 2, 2)
    im = ax2.imshow(inwrapped, interpolation='nearest', cmap=cmap, extent=extent_xy, origin='upper')
    cbar = ax2.figure.colorbar(im, ax=ax2, orientation='horizontal')
    ax2.set_title("Interferogram phase [rad]")
    ax2.set_xlabel("Easting [km]")
    ax2.set_ylabel("Northing [km]")
    plt.grid()

    if output_filename:
        plt.savefig(output_filename, dpi=dpi)
```

Call plot_model() to plot our observed displacement map:

```python
plot_model(observed_displacement_map_m, line, sample, posting, output_filename='Okmok-inflation-observation.png', dpi=200)
```

# 3. The Mogi Source Forward Model for InSAR Observations

## 3.1 The Mogi Equation

The Mogi model provides the 3D ground displacement, $u(x,y,z)$, due to an inflating source at location $(x_s,y_s,z_s)$ with volume change $V$:

\begin{equation}
u(x,y,z)=\frac{1}{\pi}(1-\nu)\cdot V\Big(\frac{x-x_s}{r(x,y,z)^3},\frac{y-y_s}{r(x,y,z)^3},\frac{z-z_s}{r(x,y,z)^3}\Big)
\end{equation}

\begin{equation}
r(x,y,z)=\sqrt{(x-x_s)^2+(y-y_s)^2+(z-z_s)^2}
\end{equation}

where $r$ is the distance from the Mogi source to $(x,y,z)$, and $\nu$ is the Poisson's ratio of the halfspace. The Poisson ratio describes how rocks react when put under stress (e.g., pressure). It is affected by temperature, the quantity of liquid to solid, and the composition of the soil material. In our problem, we will assume that $\nu$ is fixed.

## 3.2 Projecting Mogi Displacement to InSAR Line-of-Sight

In our example, the $x$-axis points east, $y$ points north, and $z$ points up. However, in the code the input values for $z$ are assumed to be depth, such that the Mogi source is at depth $z_s > 0$. The observed interferogram is already corrected for the effect of topography, so the observations can be considered to be at $z = 0$.

The satellite “sees” a projection of the 3D ground displacement, $u$, onto the look vector, $\hat{L}$, which points from the satellite to the target. Therefore, we are actually interested in the (signed magnitude of the) projection of $u$ onto $\hat{L}$ (right). This is given by

\begin{array}{lcl} proj_{\hat{L}}u & = & (u^T\hat{L})\hat{L} \\ u^T\hat{L} & = & u \cdot \hat{L} = |u||\hat{L}|cos(\alpha) = |u|cos(\alpha) \\ & = & u_x\hat{L}_x+ u_y\hat{L}_y + u_z\hat{L}_z \end{array}

where the look vector is given by $\hat{L}=(sin(l) \cdot cos(t), -sin(l) \cdot sin(t), -cos(l))$, where $l$ is the look angle measured from the nadir direction and $t$ is the satellite track angle measured clockwise from geographic north. All vectors are represented in an east-north-up basis.

Our forward model takes a Mogi source, $(x_s,y_s,z_s,V)$, and computes the look displacement at any given $(x, y, z)$ point. If we represent the ith point on our surface grid by $x_i = (x_i,y_i,z_i)$ the the displacement vector is $u_i = u(x_i, y_i, z_i)$, and the look displacement is

\begin{equation}
d_i = u_i \cdot \hat{L}
\end{equation}

## 3.3 Defining the Mogi Forward Model

We can now represent the Mogi forward problem as

\begin{equation}
g(m) = d
\end{equation}

where $g(·)$ describes the forward model in the very first equation in this notebook, $m$ is the (unknown) Mogi model, and $d$ is the predicted interferogram. The following code cells calculate the Mogi forward model according to the equations given above:

Write a function to calculate a forward model for a Mogi source.

```python
def calc_forward_model_mogi(n1, e1, depth, delta_volume, northing, easting, plook):

    # This geophysical coefficient is needed to describe how pressure relates to volume change
    displacement_coefficient = (1e6*delta_volume*3)/(np.pi*4)

    # Calculating the horizontal distance from every point in the displacement map to the x/y source location
    d_mat = np.sqrt(np.square(northing-n1) + np.square(easting-e1))

    # denominator of displacement field for mogi source
    tmp_hyp = np.power(np.square(d_mat) + np.square(depth),1.5)

    # horizontal displacement
    horizontal_displacement = displacement_coefficient * d_mat / tmp_hyp

    # vertical displacement
    vertical_displacement = displacement_coefficient * depth / tmp_hyp

    # azimuthal angle
    azimuth = np.arctan2((easting-e1), (northing-n1))

    # compute north and east displacement from horizontal displacement and azimuth angle
    east_displacement = np.sin(azimuth) * horizontal_displacement
    north_displacement = np.cos(azimuth) * horizontal_displacement

    # project displacement field onto look vector
    temp = np.concatenate((east_displacement, north_displacement, vertical_displacement), axis=1)
    delta_range = temp.dot(np.transpose([plook]))
    delta_range = -1.0 * delta_range
    return delta_range
```

Write a function to create simulated displacement data based on Mogi Source Model parameters:

```python
def displacement_data_from_mogi(x, y, z, volume, iplot, imask):
    # Organizing model parameters
    bvc = [x, y, z, volume, 0, 0, 0, 0]
    bvc = np.asarray(bvc, dtype=object)
    bvc = np.transpose(bvc)

    # Setting acquisition parameters
    track =  -13.3*np.pi / 180.0
    look  = 23.0*np.pi / 180.0
    plook = [-np.sin(look)*np.cos(track), np.sin(look)*np.sin(track), np.cos(look)]

    # Defining easting and northing vectors
    northing = np.arange(0, (line)*posting, posting) / 1000
    easting = np.arange(0, (sample)*posting, posting) / 1000
    northing_mat = np.tile(northing, (sample, 1))
    easting_mat = np.transpose(np.tile(easting, (line, 1)))
    northing_vec = np.reshape(northing_mat, (line*sample, 1))
    easting_vec = np.reshape(easting_mat, (line*sample, 1))

    # Handing coordinates and model parameters over to the rngchg_mogi function
    calc_range = calc_forward_model_mogi(bvc[1], bvc[0], bvc[2], bvc[3], northing_vec, easting_vec, plook)

    # Reshaping surface displacement data derived via calc_forward_model_mogi()
    surface_displacement = np.reshape(calc_range, (sample,line))

    # return rotated surface displacement
    return np.transpose(np.fliplr(surface_displacement))
```

## 3.4 Plotting The Mogi Forward Model

The cell below plots several Mogi forward models by varying some of the four main Mogi modeling parameters $(x_s,y_s,z_s,V)$.

The examples below fix the depth parameter to $z_s = 2.58 km$ and the volume change parameter to $volume = 0.0034 km^3$. We then vary the easting and northing parameters $x_s$ and $y_s$ to demonstrate how the model predictions vary when model parameters are changed.

Run the first example:

```python
plt.rcParams.update({'font.size': 14})
extent_x = extents((np.arange(1, sample*posting, posting))/1000)
extent_y = extents((np.arange(1, line*posting, posting))/1000)
extent_xy = extent_x + extent_y
xs = np.arange(18, 24.2, 0.4)
ys = np.arange(20, 24.2, 0.4)

zs = 2.58;
volume = 0.0034;
xa = [0, 7, 15]
ya = [0 ,5, 10]

fig = plt.figure(figsize=(18, 18))
cmap = copy.copy(plt.cm.get_cmap("jet"))
subplot_index = 1

for k in xa:
    for l in ya:
        ax = fig.add_subplot(3, 3, subplot_index)
        predicted_displacement_map = displacement_data_from_mogi(xs[k], ys[l], zs, volume, 0, 0)
        predicted_displacement_map_m = np.ma.masked_where(observed_displacement_map==0, predicted_displacement_map)
        im = ax.imshow(predicted_displacement_map_m, interpolation='nearest', cmap=cmap, extent=extent_xy)
        cbar = ax.figure.colorbar(im, ax=ax, orientation='horizontal')
        plt.grid()
        im.set_clim(-30, 30)
        ax.plot(xs[k],ys[l], 'k*', markersize=25, markerfacecolor='w')
        ax.set_title('Source: X=%4.2fkm; Y=%4.2fkm' % (xs[k], ys[l]))
        ax.set_xlabel("Easting [km]")
        ax.set_ylabel("Northing [km]")
        subplot_index += 1

plt.savefig('Model-samples-3by3.png', dpi=200, transparent='false')
```

#  Homework Assignment #1

   ASSIGNMENT #1:   Experiment with the Mogi Forward Model   -- [8 Points]

 To get a feeling for the Mogi forward model, please run the following forward model experiments using the Python Function displacement_data_from_mogi and plot the results (using the code cell above):

Run a reference simulation using the code cell above by specifying the following model parameters for source depth $x_s$ and volume change $V$: $z_{s1} = 2.5 km$; $V_1 = 0.01 km^3$. The script will visualize the resulting simulated surface displacement maps. Change the name of the output figure (last line of the script) to something that you will recognize later on (e.g., ReferenceRun.png). -- [2 Points]

Change the depth of the source by a factor of three ($z_{s2} = 7.5 km$) while leaving the other model parameters unchanged. Modify name of the output figure in the last line of the script. Visualize the results. Discuss changes to the reference run. Describe how the strength and shape of the displacement signal has changed and provide a physical explanation. -- [2 Points]

Now change the source volume by a factor of three ($V_2 = 0.03 km^3$ – also reset source depth to $z_{s1} = 2.5 km$). Visualize the results and compare them to the reference run. -- [2 Points]

Finally change both source volume and depth by a factor of three ($z_{s2} = 7.5 km$ and $V_2 = 0.03 km^3$). Compare this result to the results of experiments 1–3. -- [2 Points]

  Question 1.1 [2 Points]: Experiment no. 1: Perform reference run in code cell above using source model parameters to $z_{s1} = 2.5 km$; $V_1 = 0.01 km^3$. Plot results.

ADD DISCUSSION HERE:

  Question 1.2 [2 Points]: Experiment no. 2: Set source model parameters to $z_{s2} = 7.5 km$; $V_1 = 0.01 km^3$. Plot and discuss the results in comparison to reference run.

ADD DISCUSSION HERE:

  Question 1.3 [2 Points]: Experiment no. 3: Set source model parameters to $V_2 = 0.03 km^3$ and $z_{s1} = 2.5 km$. Plot and discuss the results in comparison to reference run.

ADD DISCUSSION HERE:

  Question 1.4 [2 Points]: Experiment no. 4: Set source model parameters to $V_2 = 0.03 km^3$ and $z_{s2} = 7.5 km$. Plot and discuss the results in comparison to reference run.

ADD DISCUSSION HERE:

Modify the example script below to answer questions 1.2 - 1.4:

```python
plt.rcParams.update({'font.size': 14})
extent_x = extents((np.arange(1, sample*posting, posting))/1000)
extent_y = extents((np.arange(1, line*posting, posting))/1000)
extent_xy = extent_x + extent_y
xs = np.arange(18, 24.2, 0.4)
ys = np.arange(20, 24.2, 0.4)

# ------------ Change Variables HERE --------------- #
zs = 2.58;
volume = 0.0034;
# ------------------------------------------------- #

xa = [0, 7, 15]
ya = [0 ,5, 10]

fig = plt.figure(figsize=(18, 18))
cmap = copy.copy(plt.cm.get_cmap("jet"))
subplot_index = 1

for k in xa:
    for l in ya:
        ax = fig.add_subplot(3, 3, subplot_index)
        predicted_displacement_map = displacement_data_from_mogi(xs[k], ys[l], zs, volume, 0, 0)
        predicted_displacement_map_m = np.ma.masked_where(observed_displacement_map==0, predicted_displacement_map)
        im = ax.imshow(predicted_displacement_map_m, interpolation='nearest', cmap=cmap,extent=extent_xy)
        cbar = ax.figure.colorbar(im, ax=ax, orientation ='horizontal')
        plt.grid()
        im.set_clim(-30, 30)
        ax.plot(xs[k],ys[l], 'k*', markersize=25, markerfacecolor='w')
        ax.set_title(f"Source: X={xs[k]:.2f}km; Y={ys[l]:.2f}km")
        ax.set_xlabel("Easting [km]")
        ax.set_ylabel("Northing [km]")
        subplot_index += 1

# CHANGE THE NAME OF THE IMAGE THAT IS BEING SAVED TO YOUR CLOUD INSTANCE!!
plt.savefig('Model-samples-3by3.png', dpi=200, transparent='false')
```

# 4. Solving the Inverse Model

 The inverse problem seeks to determine the optimal parameters $(\hat{x_s},\hat{y_s},\hat{z_s},\hat{V})$ of the Mogi model $m$ by minimizing the misfit between predictions, $g(m)$, and observations $d^{obs}$ according to

\begin{equation}
\sum{\Big[g(m) - d^{obs}\Big]^2}
\end{equation}

This equation describes misfit using the method of least-squares, a standard approach to approximate the solution of an overdetermined equation system. We will use a grid-search approach to find the set of model parameters that minimize the the misfit function. The approach is composed of the following processing steps:

Loop through the mogi model parameters,

Calculate the forward model for each set of parameters,

Calculate the misfit $\sum{[g(m) - d^{obs}]^2}$, and

Find the parameter set that minimizes this misfit.

## 4.1 Experimenting with Misfit

Let's look at the misfit $\sum{[g(m) - d^{obs}]^2}$ for a number of different model parameter sets $(x_s,y_s,z_s,V)$:

```python
plt.rcParams.update({'font.size': 14})
extent_x = extents((np.arange(1, sample*posting, posting))/1000)
extent_y = extents((np.arange(1, line*posting, posting))/1000)
extent_xy = extent_x + extent_y
xs = np.arange(18, 24.2, 0.4)
ys = np.arange(20, 24.2, 0.4)

zs = 2.58;
volume = 0.0034;
xa = [0, 7, 15]
ya = [0 ,5, 10]

fig = plt.figure(figsize=(18, 18))
cmap = copy.copy(plt.cm.get_cmap("jet"))
subplot_index = 1

for k in xa:
    for l in ya:
        ax = fig.add_subplot(3, 3, subplot_index)
        predicted_displacement_map = displacement_data_from_mogi(xs[k], ys[l], zs, volume, 0, 0)
        predicted_displacement_map_m = np.ma.masked_where(observed_displacement_map==0, predicted_displacement_map)
        im = ax.imshow(observed_displacement_map_m-predicted_displacement_map_m, interpolation='nearest', cmap=cmap, extent=extent_xy)
        cbar = ax.figure.colorbar(im, ax=ax, orientation='horizontal')
        plt.grid()
        im.set_clim(-30, 30)
        ax.plot(xs[k], ys[l], 'k*', markersize=25, markerfacecolor='w')
        ax.set_title('Source: X=%4.2fkm; Y=%4.2fkm' % (xs[k], ys[l]))
        ax.set_xlabel("Easting [km]")
        ax.set_ylabel("Northing [km]")
        subplot_index += 1
plt.savefig('Misfit-samples-3by3.png', dpi=200, transparent='false')
```

## 4.2 Running Grid-Search to find Best Fitting Model Parameter $(\hat{x}_s,\hat{y}_s)$

The following code cell runs a grid-search approach to find the best fitting Mogi source parameters for the 2000-2002 displacement event at Okmok. To keep things simple, we will fix the depth $z_s$ and volume change $V$ parameters close to their "true" values and search only for the correct east/north source location ($x_s,y_s$).

Write a script using the grid-search approach in Python:

```python
# FIX Z AND dV, SEARCH OVER X AND Y

# Setting up search parameters
xs = np.arange(19, 22.2, 0.2)
ys = np.arange(21, 23.2, 0.2)
zs = 2.58;
volume = 0.0034;

nx = xs.size
ny = ys.size
ng = nx * ny;

print(f"fixed z = {zs}km, dV = {volume}, searching over (x,y)")

misfit = np.zeros((nx, ny))
subplot_index = 0

# Commence grid-search for best model parameters
for k, xv in enumerate(xs):
    for l, yv in enumerate(ys):
        subplot_index += 1
        predicted_displacement_map = displacement_data_from_mogi(xs[k], ys[l], zs, volume, 0, 0)
        predicted_displacement_map_m = np.ma.masked_where(observed_displacement_map==0, predicted_displacement_map)
        misfit[k,l] = np.sum(np.square(observed_displacement_map_m - predicted_displacement_map_m))
    print(f"Source {subplot_index:3d}/{ng:3d} is x = {xs[k]:.2f} km, y = {ys[l]:.2f} km")

# Searching for the minimum in the misfit matrix
mmf = np.where(misfit == np.min(misfit))
print(f"\n----------------------------------------------------------------")
print('Best fitting Mogi Source located at: X = %5.2f km; Y = %5.2f km' % (xs[mmf[0]], ys[mmf[1]]))
print(f"----------------------------------------------------------------")
```

## 4.3 Plot and Inspect the Misfit Function

The code cell below plots the misfit function ($\sum{[g(m) - d^{obs}]^2}$) describing the fit of different Mogi source parameterizations to the observed InSAR data. You should notice a clear minimum in the misfit plot at the location of the best fitting source location estimated above.

You may notice that, even for the best fitting solution, the misfit does not become zero. This could be due to other signals in the InSAR data (e.g., atmospheric effects or residual topography). Alternatively, it could also indicate that the observed displacement doesn't fully comply with Mogi theory.

Plot the misfit function ($\sum{[g(m) - d^{obs}]^2}$):

```python
plt.rcParams.update({'font.size': 18})
extent_xy = extents(xs) + extents(ys)
fig = plt.figure(figsize=(10, 10))
cmap = copy.copy(plt.cm.get_cmap("jet"))
ax1 = fig.add_subplot(1, 1 ,1)
im = ax1.imshow(np.transpose(misfit), origin='lower', interpolation='nearest', cmap=cmap, extent=extent_xy)
# USE THIS COMMAND TO CHANGE COLOR SCALING: im.set_clim(-30, 30)
ax1.set_aspect('auto')
cbar = ax1.figure.colorbar(im, ax=ax1, orientation='horizontal')
ax1.plot(xs[mmf[0]], ys[mmf[1]], 'k*', markersize=25, markerfacecolor='w')
ax1.set_title("Misfit Function for Mogi-Source Approximation")
ax1.set_xlabel("Easting [km]")
ax1.set_ylabel("Northing [km]")
plt.savefig('Misfit-function.png', dpi=200, transparent='false')
```

## 4.4 Plot Best-Fitting Mogi Forward Model and Compare to Observations

With the best-fitting model parameters defined, you can now analyze how well the model fits the InSAR-observed surface displacement. The best way to do that is to look at both the observed and predicted displacement maps and compare their spatial patterns. Additionally, we will also plot the residuals (observed_displacement_map - observed_displacement_map) to determine if there are additional signals in the data that are not modeled using Mogi theory.

Compare the observed and predicted displacement maps:

```python
# Calculate predicted displacement map for best-fitting Mogi parameters:
predicted_displacement_map = displacement_data_from_mogi(xs[mmf[0]], ys[mmf[1]], zs, volume, 0, 0)

# Mask the predicted displacement map to remove pixels incoherent in the observations:
predicted_displacement_map_m = np.ma.masked_where(observed_displacement_map==0, predicted_displacement_map)

# Plot observed displacement map
plot_model(observed_displacement_map_m, line, sample, posting)

# Plot simulated displacement map
plot_model(predicted_displacement_map_m, line, sample, posting)

plt.savefig('BestFittingMogiDefo.png', dpi=200, transparent='false')

# Plot simulated displacement map without mask applied
plot_model(predicted_displacement_map, line, sample, posting)
```

Determine if there are additional signals in the data that are not modeled using Mogi theory:

```python
# Plot residual between observed and predicted displacement maps
plot_model(observed_displacement_map_m-predicted_displacement_map_m, line, sample, posting)
plt.savefig('Residuals-ObsMinusMogi.png', dpi=200, transparent='false')
```

#  Homework Assignment #2

   ASSIGNMENT #2:   Run 2nd Grid-Search to Find Model Parameters $(\hat{z}_s,\hat{V})$    -- [8 Points]

 For this second grid-search run, we now switch out the model parameters we are trying to estimate. We will assume that the lateral location of the Mogi source is now fixed to its estimated value ($\hat{x}_s = 20.6 km$; $\hat{y}_s = 21.8 km$).

To perform a grid search for the best fitting model parameters $\hat{z}_s$ and $\hat{V}$, please complete the following steps:

Using the previous grid-search script as a template, write a new grid-search script to search for the best fitting source model depth ($z_s$) and volume change ($V$). -- [3 Points]

Provide a plot of the misfit function and provide the best-fitting values for $\hat{z}_s$ and $\hat{V}$. When plotting the misfit function, put $z_s$ on the vertical axis. You may want to adjust the color scale, in order to better see the shape of the misfit function. -- [2 Points]

Compare the $z_s$ vs. $V$ misfit function (misfit function 2) to the $y_s$ vs. $x_s$ misfit function (misfit function 1). You should see that the shape of the function is different. Misfit function 1 is largely of circular shape while misfit function 2 appears elongated. Interpret this pattern. -- [3 Points]

  Question 2.1 [3 Points]: Provide code to perform grid search over $z_s$ and $V$.

PROVIDE SCRIPT BY MODIFYING THE CODE IN THE CODE CELL BELOW:

```python
# !!!! MODIFY THIS SCRIPT TO PERFORM A GRID SEARCH OVER zs AND V: !!!!

# Setting up search parameters
xs = np.arange(19, 22.2, 0.2)
ys = np.arange(21, 23.2, 0.2)
zs = 2.58;
volume = 0.0034;

nx = xs.size
ny = ys.size
ng = nx * ny;

#print('fixed z = ',zs,' km, dV = ',volume, ' searching over (x,y)')
print(f"fixed z = {zs}km, dV = {volume}, searching over (x,y)")

misfit=np.zeros((nx,ny))
subplot_index = 0

# Commence grid-search for best model parameters
for k, xv in enumerate(xs):
    for l, yv in enumerate(ys):
        subplot_index = subplot_index+1
        predicted_displacement_map = displacement_data_from_mogi(xs[k],ys[l],zs,volume,0,0)
        predicted_displacement_map_m = np.ma.masked_where(observed_displacement_map == 0, predicted_displacement_map)
        misfit[k,l] = np.sum(np.square(observed_displacement_map_m - predicted_displacement_map_m))
    print(f"Source {subplot_index:3d}/{ng:3d} is x = {xs[k]:.2f} km, y = {ys[l]:.2f} km")

# Searching for the minimum in the misfit matrix
mmf = np.where(misfit == np.min(misfit))
print('')
print(f"\n----------------------------------------------------------------")
print('Best fitting Mogi Source located at: X = %5.2f km; Y = %5.2f km' % (xs[mmf[0]], ys[mmf[1]]))
print(f"----------------------------------------------------------------")
```

  Question 2.2-A [1 Points]: Provide the best fitting values for source depth ($\hat{z}_s$) and volume change ($\hat{V}$) according to your grid-search results.

PROVIDE ESTIMATES FOR $\hat{z}_s$ and $\hat{V}$ HERE:

  Question 2.2-B [1 Points]: Provide plot of $z_s$ on $V$ misfit function.

PROVIDE PLOT HERE:

  Question 2.3 [3 Points]: Compare the $z_s$ vs. $V$ misfit function (misfit function 2) to the initial $y_s$ vs. $x_s$ misfit function (misfit function 1). Interpret their difference in spatial pattern.

PROVIDE DISCUSSION HERE:

#  Homework Assignment #3

   ASSIGNMENT #3:   Error Discussion   -- [4 Points]

 In a perfect world where data are noise-free and geophysical models perfectly represent reality, there should be a set of model parameters that reduces the misfit function to zero. In our case, however, the misfit function still shows large values, even for the best fitting model parameters. Provide and explain three reasons for why the differences between the model and the data are not zero.

PROVIDE DISCUSSION HERE:

# 5. Version Log

 GEOS 639 Geodetic Imaging - Version 1.3.3 -  March 2022

        Version Changes:

remove obsolete asf_notebook functions

url_widget

Adjust some of the language in the notebook


# SNLP Assignment 3

Name 1: Sangeet Sagar

Student id 1: 7009050

Email 1: sasa00001@stud.uni-saarland.de

Name 2: Nikhil Paliwal

Student id 2: 7009915

Email 2: nipa00002@stud.uni-saarland.de

**Instructions:** Read each question carefully.

Make sure you appropriately comment your code wherever required. Your final submission should contain the completed Notebook and the respective Python files for exercises 2 and 3. There is no need to submit the data files.

Upload the zipped folder in Teams. Make sure to click on "Turn-in" after you upload your submission, otherwise the assignment will not be considered as submitted. Only one member of the group should make the submisssion.

---

## Exercise 1: Entropy Intuition (2 points)

### 1.1 (0.5 points)

Order the following three snippets by entropy (highest to lowest). Justify your answer (view it more intuitively rather than by using a specific character-level language model, though you would probably reach the same conclusion).

```
1:    A B A A A A B B A A A B A B B B B B A
2:    A B A B A B A B A B A B A B A B A B A
3:    A B A A A B A B A B A B A B A B A B A
```

**Answer**

Correct order: $2 > 3 > 1$

*Explanation*: Looking intuitively, Entropy is a measure of randomness in a probability distribution. We compare the entorpy in the above sequences by comparing radomness. **2** has the highest degre of randomness for the reason that no consecutive letters are same. This is followed by **3** as it has a repition of `A A A` in the beginning. This is observation is more prevelant in **1**, hence it has least entropy among all.

### 1.2 (0.5 point)

Words in natural language do not have the maximum entropy given the available alphabet. This creates a redundancy (e.g. the word `maximum` could be uniquely replaced by `mxmm` and everyone would still understand). If the development of natural languages leads to somewhat optimal solutions, why is it beneficial to have such redundancies in communication?

If you're uncertain, please refer to this well-written article: [www-math.ucdenver.edu/~wcherowi/courses/m5410/m5410lc1.html](http://www-math.ucdenver.edu/~wcherowi/courses/m5410/m5410lc1.html).

**Answer**

Having redundancies diminishes the uncertainty in communication. With more information on the goal of communication, more certain we become what the speaker refers.

### 1.3 (1 point)

1. Assume you were given a perfect language model that would always assign probability of $1$ to the next word. What would be the cross-entropy on any text? Motivate your answer with formal derivation. (0.5 points)
2. How does cross-entropy relate to perplexity? Is there a reason why would one be preferred over the other? (0.5 points)

**Answer**

Cross-entropy
$$ H(P, Q) = \quad –\sum_{x \in X} P(x) * \log(Q(x)) $$
1. In the given situation, $P(x) = Q(x) = 1$. Hence,
$$ H(P, Q) = \quad –\sum_{x \in X} 1 * \log(1) $$
$$ H(P, Q) = 0 $$
Intuitively, if the probablity of the next word is 1, we are always certain about the subsequent outcomes, hence there is no un-certainity and the Entropy is $0$.

2. Perplexity($M$) is given as $M = 2^{H(P, Q)}$. Hence it is equivalent to the exponentiation of the cross-entropy. Generally, perplexity is preferred over cross-entropy as it is easy to interpret (for the reason that perplexity is the avergae de-facto size of vocabluary).

## Exercise 2: Harry Potter and the Measure of Uncertainty (4 points)

#### 2.1 (2.5 points)

Harry, Hermione, and Ron are trying to save the Philosopher's Stone. To do this, they have to cross a series of hurdles to reach the room where the stone is kept. Currently, they are trapped in a chamber whose exit is blocked by fire. On a table before them are 7 potions.

|P1|P2|P3|P4|P5|P6|P7|
|---|---|---|---|---|---|---|

Of these, 6 potions are poisons and only one is the antidote that will get them through the exit. Drinking the poison will not kill them, but will weaken them considerably.

1. There is no way of knowing which potion is a poison and which an antidote. How many potions must they sample *on an average* to pick the antidote? (1 point)

**Answer**

We have $X$ = no. of potion sampled before picking up an antidote.

$P(X=1) = \frac{1}{7} \quad \quad \quad$ (antidote is picked up in the first sampling)

$P(X=2) = \frac{6}{7}\cdot\frac{1}{6} = \frac{1}{7}\quad$ (1 poison, 1 antidote)

$P(X=3) = \frac{6}{7} \cdot \frac{5}{6} \cdot \frac{1}{5} = \frac{1}{7} $ (2 poison, 1 antidote)

Similarly,

$P(X=4)= P(X=5)= P(X=6)= P(X=7) =\frac{1}{7} $

$$ E[x] = \sum_{n=1}^{7} n\cdot\left(\frac{1}{7}\right)$$
$$ E[x] = \frac{1}{7}\sum_{n=1}^{7} n$$
$$ E[x] = 4$$

Therefore, we must take sample 4 potions on an avergage to pick the antidote.

Hermione notices a scroll lying near the potions. The scroll contains an intricate riddle written by Professor Snape that will help them determine which potion is the antidote. With the help of the clues provided, Hermione cleverly deduces that each potion can be the antidote with a certain probability.

|P1|P2|P3|P4|P5|P6|P7|
|---|---|---|---|---|---|---|
|1/16|1/4|1/64|1/2|1/64|1/32|1/8|

2. In this situation, how many potions must they now sample *on an average* to pick the antidote correctly? (1 point)

3. What is the most efficient sequence of potions they must sample to discover the antidote? Why do you claim that in terms of how uncertain you are about guessing right? (0.5 point)

**Answer**

**P4 > P2 > P7 > P1 > P6 > {P3, P4}**

The sequence of potion sampling given above diminishes the uncertainity (to maximum extent) as we take the potion with highest probablity of being an antidote at first.

#### 2.2 (1.5 points)

1. Extend your logic from 2.1 to a Shannon's Game where you have to correctly guess the next word in a sentence. Assume that a word is any possible permutation and combination of 26 letters of the alphabet, and all the words have a length of at most *n*.
How many guesses will one have to make to guess the correct word? (1 point)

(**Hint**: Think of how many words can exist in this scenario)

**Answer**

Let $k$ be the total number of words of length atmost $n$ be present in the corpus:
$$ k = \sum_{n=1}^{26} 26^{n}$$

Sum of a GP

$$ k = \frac{26}{25}(26^{n}-1)$$

Using similar logic form 2.1, we have

$E[X=1] = \frac{1}{k} $ ; (expectation of a correct guess in the 1st sampling)

$E[X=2] = \frac{k-1}{k} \frac{1}{k-1} = \frac{1}{k}$ ; (expectation of a correct guess in the 2nd sampling)

And so on,

$E[X=k] = \frac{1}{k}$ ; (we sample as many times as we have totat number of words in the corpus)

$$ E[x] = \sum_{m=1}^{k} m\cdot\left(\frac{1}{k}\right)$$
$$ E[x] = \frac{1}{k}\sum_{m=1}^{k} m$$
$$ E[x] = \frac{1}{k}\cdot \frac{k(k+1)}{2}$$

Therefore, one has to make $\frac{1}{k}\cdot \frac{k(k+1)}{2}$ (where $k = \sum_{n=1}^{26} 26^{n}$) guesses to to guess the correct word.

2. Why is the entropy lower in real-world languages? How do language models help to reduce the uncertainty of guessing the correct word? (2-3 sentences) (0.5 point)

**Answer**

Entropy is lower for real-world languages for the reason that the language is defined under a particular set of rules (e.g. Grammatical rules) and is confined to follow them. We as a speaker or a writer of the language follow a specific sentence structure.

A statistical language model is learned from raw text and predicts the probability of the next word in the sequence given the words already present in the sequence. Hence, given a word the LM will have certain choices to choose
from to make the next prediction and it will select the one with maximum probablity, thus reducing the uncertainity of guessing the correct word.

## Exercise 3: Kullback-Leibler Divergence (4 points)

Another metric (besides perplexity and cross-entropy) to compare two probability distributions is the Kullback-Leibler Divergence $D_{KL}$. It is defined as:

\begin{equation}
D_{KL}(P\|Q) = \sum_{x \in X}P(x) \cdot \log \frac{P(x)}{Q(x)}
\end{equation}

Where $P$ is the empirical or observed distribution, and Q is the estimated distribution over a common probabilitiy space $X$.
Answer the following questions:

#### 3.1. (0.5 points)

How is $D_{KL}$ related to Cross-Entropy? Derive a mathematical expression that describes the relationship.

**Answer**

$$ D_{KL}(P\|Q) = \sum_{x \in X}P(x) \cdot \log \frac{P(x)}{Q(x)} $$
$$ D_{KL}(P\|Q) = -\sum_{x \in X}P(x)\cdot \log(Q(x)) + \sum_{x \in X}P(x)\cdot \log(Q(x)) $$
$$ D_{KL}(P\|Q) = E_P[-\log(Q)]- E_P[-\log(P)]$$
$$ D_{KL}(P\|Q) = H(P, Q)- H(P)$$

Where:

$ H(P,Q)$ = cross entropy of distributions $P$ and $Q$

$ H(P)$ = entropy of distribution $P$

#### 3.2. (0.5 points)

Is minimizing $D_{KL}$ the same thing as minimizing Cross-Entropy?  Support your answer using your answer to 1.

**Answer**

Yes, minimizing cross-entrpy is same as minimizing $D_{KL}$ becuase entropy remains unchanged for a true distribution. Changes in the distribution are reflected only in the cross-entropy.

#### 3.3 (3 points)

For a function $d$ to be considered a distance metric, the following three properties must hold:

$\forall x,y,z \in U:$

1. $d(x,y) = 0 \Leftrightarrow x = y$
2. $d(x,y) = d(y,x)$
3. $d(x,z) \le d(x,y) + d(y,z)$

Is $D_{KL}$ a distance metric? ($U$ in this case is the set of all distributions over the same possible states).
For each of the three points either prove that it holds for $K_{DL}$ or show a counterexample proving why it does not.

**Answer**

1. Let $x=p$, $y=q$
$$ D(p \|q) = H(p, q) - H(p) \quad \quad \quad  \quad \dots (1)$$
$$ D(p \|q) = H(p,p) - H(p)$$
$$ D(p \|q) = H(p) - H(p)$$
$$ D(p \|q) = 0$$
$D_{KL}$ holds here.

2. $$ D(q \|p) = H(q, p) - H(q) \quad \quad \quad  \quad \dots (2)$$
From 1 and 2,
$$D(q \|p) \neq D(p \|q)$$
$D_{KL}$ does not hold here.

Counterexample:
\begin{align}
D(x\|y) &= \frac{1}{3} \log\left(\frac{1/3}{1/6}\right) + \frac{2}{3} \log\left(\frac{2/3}{5/6}\right) \\
D(x\|y) &= 0.035 \\
\\
D(y\|x) &= \frac{1}{6} \log\left(\frac{1/6}{1/3}\right) + \frac{5}{6} \log\left(\frac{5/6}{2/3}\right) \\
D(y\|x) &= 0.03 \\
\end{align}
Hence
$$D(x \|y) \neq D(y \|x) $$

3. $D_{KL}$ does not hold here.

Counterexample:
Sample space : ${0, 1}$

$x(0) = \frac{1}{3}$

$y(0) = \frac{1}{6}$

$z(0) = \frac{1}{12}$

$$ D(p\|q) = \sum p_i \log\left(\frac{p_i}{q_i}\right) $$
\begin{align}
D(x\|z) &= \sum x_i \log\left(\frac{x_i}{z_i}\right) \\
D(x\|z) &= x(0) \log\left(\frac{x(0)}{z(0)}\right) + x(1) \log\left(\frac{x(1)}{z(1)}\right) \\
D(x\|z) &= \frac{1}{3} \log\left(\frac{1/3}{1/12}\right) + \frac{2}{3} \log\left(\frac{2/3}{11/12}\right) \\
D(x\|z) &= 0.108 \\
\\
D(x\|y) &= \frac{1}{3} \log\left(\frac{1/3}{1/6}\right) + \frac{2}{3} \log\left(\frac{2/3}{5/6}\right) \\
D(x\|y) &= 0.035 \\
\\
D(y\|z) &= \frac{1}{6} \log\left(\frac{1/6}{1/12}\right) + \frac{5}{6} \log\left(\frac{5/6}{1/12}\right) \\
D(x\|z) &= 0.015 \\
\end{align}

Hence,
$$D(x\|z) \ge D(x\|y) + D(x\|y)$$

Therefore, $D_{KL}$ is not a distance metric.

## Bonus (1.5 points)

1. Compute $D_{KL}(Q_1\|P_1)$ for the following pair of sentences based on a unigram language model (word level).

```
p1: to be or not to be
q1: to be or to be or not or to be be be
```

 Do so by implementing the function `dkl` in `bonus.py`. You will also have to calculate the distributions $P_1$, $Q_1$; for this, you can either reuse your code from the last assignment or implement a new function in `bonus.py`. (1 point)

2. Suppose the sentences in 1. would be replaced by the following sequences of symbols. You can imagine them to be sequences of nucleobases in a [coding](https://en.wikipedia.org/wiki/Coding_region) region of a gene in your genome.

```
p2: ACTGACACTGAC
q2: ACTACTGACCCACTACTGACCC
```

Let $P_2$, $Q_2$ be the character-level unigram LMs derived from these sequences. What values will $D_{KL}(P_1\|P_2)$, $D_{KL}(Q_1\|Q_2)$ take? Does the quantity hold any information? Would computing $D_{KL}$ between distributions over two different natural languages hold any information? (0.5 points)

No mathematical explanation nor coding required for the second part.

```python
from importlib import reload
import bonus
bonus = reload(bonus)

# TODO: estimate LMs
P =
Q =

# TODO: DKL
print(bonus.dkl(p,q))
```


Escuela de Ciencias Básicas, Tecnología e Ingeniería

ECBTI

Curso: Métodos Numéricos

Unidad 1: Error

Febrero 28 de 2020

***

> **Tutor:** Carlos Alberto Álvarez Henao, I.C. D.Sc.

> **skype:** carlos.alberto.alvarez.henao

> **Herramienta:** [Jupyter](http://jupyter.org/)

> **Kernel:** Python 3.7

***

***Comentario:*** estas notas están basadas en el curso del profesor [Kyle T. Mandli](https://github.com/mandli/intro-numerical-methods) (en inglés)

# Fuentes de error

Los cálculos numéricos, que involucran el uso de máquinas (análogas o digitales) presentan una serie de errores que provienen de diferentes fuentes:

- del Modelo
- de los datos
- de truncamiento
- de representación de los números (punto flotante)
- $\ldots$

***Meta:*** Categorizar y entender cada tipo de error y explorar algunas aproximaciones simples para analizarlas.

# Error en el modelo y los datos

Errores en la formulación fundamental

- Error en los datos: imprecisiones en las mediciones o incertezas en los parámetros

Infortunadamente no tenemos control de los errores en los datos y el modelo de forma directa pero podemos usar métodos que pueden ser más robustos en la presencia de estos tipos de errores.

# Error de truncamiento

Los errores surgen de la expansión de funciones con una función simple, por ejemplo, $sin(x) \approx x$ para $|x|\approx0$.

# Error de  representación de punto fotante

Los errores surgen de aproximar números reales con la representación  en precisión finita de números en el computador.

# Definiciones básicas

Dado un valor verdadero de una función $f$ y una solución aproximada $F$, se define:

- Error absoluto

$$e_a=|f-F|$$

- Error relativo

$$e_r = \frac{e_a}{|f|}=\frac{|f-F|}{|f|}$$

# Notación $\text{Big}-\mathcal{O}$

sea $$f(x)= \mathcal{O}(g(x)) \text{ cuando } x \rightarrow a$$

si y solo si

$$|f(x)|\leq M|g(x)| \text{ cuando } |x-a| < \delta \text{ donde } M, a > 0$$

En la práctica, usamos la notación $\text{Big}-\mathcal{O}$ para decir algo sobre cómo se pueden comportar los términos que podemos haber dejado fuera de una serie. Veamos el siguiente ejemplo de la aproximación de la serie de Taylor:

***Ejemplo:***

sea $f(x) = \sin x$ con $x_0 = 0$ entonces

$$T_N(x) = \sum^N_{n=0} (-1)^{n} \frac{x^{2n+1}}{(2n+1)!}$$

Podemos escribir $f(x)$ como

$$f(x) = x - \frac{x^3}{6} + \frac{x^5}{120} + \mathcal{O}(x^7)$$

Esto se vuelve más útil cuando lo vemos como lo hicimos antes con $\Delta x$:

$$f(x) = \Delta x - \frac{\Delta x^3}{6} + \frac{\Delta x^5}{120} + \mathcal{O}(\Delta x^7)$$

# Reglas para el error de propagación basado en la notación $\text{Big}-\mathcal{O}$

En general, existen dos teoremas que no necesitan prueba y se mantienen cuando el valor de $x$ es grande:

Sea

$$\begin{aligned}
    f(x) &= p(x) + \mathcal{O}(x^n) \\
    g(x) &= q(x) + \mathcal{O}(x^m) \\
    k &= \max(n, m)
\end{aligned}$$

Entonces

$$
    f+g = p + q + \mathcal{O}(x^k)
$$

y

\begin{align}
    f \cdot g &= p \cdot q + p \mathcal{O}(x^m) + q \mathcal{O}(x^n) + O(x^{n + m}) \\
    &= p \cdot q + \mathcal{O}(x^{n+m})
\end{align}

De otra forma, si estamos interesados en valores pequeños de $x$, $\Delta x$, la expresión puede ser modificada como sigue:

\begin{align}
    f(\Delta x) &= p(\Delta x) + \mathcal{O}(\Delta x^n) \\
    g(\Delta x) &= q(\Delta x) + \mathcal{O}(\Delta x^m) \\
    r &= \min(n, m)
\end{align}

entonces

$$
    f+g = p + q + O(\Delta x^r)
$$

y

\begin{align}
    f \cdot g &= p \cdot q + p \cdot \mathcal{O}(\Delta x^m) + q \cdot \mathcal{O}(\Delta x^n) + \mathcal{O}(\Delta x^{n+m}) \\
    &= p \cdot q + \mathcal{O}(\Delta x^r)
\end{align}

***Nota:*** En este caso, supongamos que al menos el polinomio con $k=max(n,m)$ tiene la siguiente forma:

$$
    p(\Delta x) = 1 + p_1 \Delta x + p_2 \Delta x^2 + \ldots
$$

o

$$
    q(\Delta x) = 1 + q_1 \Delta x + q_2 \Delta x^2 + \ldots
$$

para que $\mathcal{O}(1)$

de modo que hay un término $\mathcal{O}(1)$ que garantiza la existencia de $\mathcal{O}(\Delta x^r)$ en el producto final.

Para tener una idea de por qué importa más la potencia en $\Delta x$ al considerar la convergencia, la siguiente figura muestra cómo las diferentes potencias en la tasa de convergencia pueden afectar la rapidez con la que converge nuestra solución. Tenga en cuenta que aquí estamos dibujando los mismos datos de dos maneras diferentes. Graficar el error como una función de $\Delta x$ es una forma común de mostrar que un método numérico está haciendo lo que esperamos y muestra el comportamiento de convergencia correcto. Dado que los errores pueden reducirse rápidamente, es muy común trazar este tipo de gráficos en una escala log-log para visualizar fácilmente los resultados. Tenga en cuenta que si un método fuera realmente del orden $n$, será una función lineal en el espacio log-log con pendiente $n$.

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
dx = np.linspace(1.0, 1e-4, 100)

fig = plt.figure()
fig.set_figwidth(fig.get_figwidth() * 2.0)
axes = []
axes.append(fig.add_subplot(1, 2, 1))
axes.append(fig.add_subplot(1, 2, 2))

for n in range(1, 5):
    axes[0].plot(dx, dx**n, label="$\Delta x^%s$" % n)
    axes[1].loglog(dx, dx**n, label="$\Delta x^%s$" % n)

axes[0].legend(loc=2)
axes[1].set_xticks([10.0**(-n) for n in range(5)])
axes[1].set_yticks([10.0**(-n) for n in range(16)])
axes[1].legend(loc=4)
for n in range(2):
    axes[n].set_title("Crecimiento del Error vs. $\Delta x^n$")
    axes[n].set_xlabel("$\Delta x$")
    axes[n].set_ylabel("Error Estimado")
    axes[n].set_title("Crecimiento de las diferencias")
    axes[n].set_xlabel("$\Delta x$")
    axes[n].set_ylabel("Error Estimado")

plt.show()
```

# Error de truncamiento

***Teorema de Taylor:*** Sea $f(x) \in C^{m+1}[a,b]$ y $x_0 \in [a,b]$, para todo $x \in (a,b)$ existe un número $c = c(x)$ que se encuentra entre $x_0$ y $x$ tal que

$$ f(x) = T_N(x) + R_N(x)$$

donde $T_N(x)$ es la aproximación del polinomio de Taylor

$$T_N(x) = \sum^N_{n=0} \frac{f^{(n)}(x_0)\times(x-x_0)^n}{n!}$$

y $R_N(x)$ es el residuo (la parte de la serie que obviamos)

$$R_N(x) = \frac{f^{(n+1)}(c) \times (x - x_0)^{n+1}}{(n+1)!}$$

Otra forma de pensar acerca de estos resultados consiste en reemplazar $x - x_0$ con $\Delta x$. La idea principal es que el residuo $R_N(x)$ se vuelve mas pequeño cuando $\Delta x \rightarrow 0$.

$$T_N(x) = \sum^N_{n=0} \frac{f^{(n)}(x_0)\times \Delta x^n}{n!}$$

y $R_N(x)$ es el residuo (la parte de la serie que obviamos)

$$ R_N(x) = \frac{f^{(n+1)}(c) \times \Delta x^{n+1}}{(n+1)!} \leq M \Delta x^{n+1}$$

***Ejemplo 1:***

$f(x) = e^x$ con $x_0 = 0$

Usando esto podemos encontrar expresiones para el error relativo y absoluto en función de $x$ asumiendo $N=2$.

Derivadas:
$$\begin{aligned}
    f'(x) &= e^x \\
    f''(x) &= e^x \\
    f^{(n)}(x) &= e^x
\end{aligned}$$

Polinomio de Taylor:
$$\begin{aligned}
    T_N(x) &= \sum^N_{n=0} e^0 \frac{x^n}{n!} \Rightarrow \\
    T_2(x) &= 1 + x + \frac{x^2}{2}
\end{aligned}$$

Restos:
$$\begin{aligned}
    R_N(x) &= e^c \frac{x^{n+1}}{(n+1)!} = e^c \times \frac{x^3}{6} \quad \Rightarrow \\
    R_2(x) &\leq \frac{e^1}{6} \approx 0.5
\end{aligned}$$

Precisión:
$$
    e^1 = 2.718\ldots \\
    T_2(1) = 2.5 \Rightarrow e \approx 0.2 ~~ r \approx 0.1
$$

¡También podemos usar el paquete `sympy` que tiene la capacidad de calcular el polinomio de *Taylor* integrado!

```python
import sympy
x = sympy.symbols('x')
f = sympy.symbols('f', cls=sympy.Function)

f = sympy.exp(x)
f.series(x0=0, n=5)
```

$\displaystyle 1 + x + \frac{x^{2}}{2} + \frac{x^{3}}{6} + \frac{x^{4}}{24} + O\left(x^{5}\right)$

Graficando

```python
x = np.linspace(-1, 1, 100)
T_N = 1.0 + x + x**2 / 2.0
R_N = np.exp(1) * x**3 / 6.0

plt.plot(x, T_N, 'r', x, np.exp(x), 'k', x, R_N, 'b')
plt.plot(0.0, 1.0, 'o', markersize=10)
plt.grid(True)
plt.xlabel("x")
plt.ylabel("$f(x)$, $T_N(x)$, $R_N(x)$")
plt.legend(["$T_N(x)$", "$f(x)$", "$R_N(x)$"], loc=2)
plt.show()
```

***Ejemplo 2:***

Aproximar

$$ f(x) = \frac{1}{x} \quad x_0  = 1,$$

usando $x_0 = 1$ para el tercer termino de la serie de Taylor.

$$\begin{aligned}
    f'(x) &= -\frac{1}{x^2} \\
    f''(x) &= \frac{2}{x^3} \\
    f^{(n)}(x) &= \frac{(-1)^n n!}{x^{n+1}}
\end{aligned}$$

$$\begin{aligned}
    T_N(x) &= \sum^N_{n=0} (-1)^n (x-1)^n \Rightarrow \\
    T_2(x) &= 1 - (x - 1) + (x - 1)^2
\end{aligned}$$

$$\begin{aligned}
    R_N(x) &= \frac{(-1)^{n+1}(x - 1)^{n+1}}{c^{n+2}} \Rightarrow \\
    R_2(x) &= \frac{-(x - 1)^{3}}{c^{4}}
\end{aligned}$$

```python
x = np.linspace(0.8, 2, 100)
T_N = 1.0 - (x-1) + (x-1)**2
R_N = -(x-1.0)**3 / (1.1**4)

plt.plot(x, T_N, 'r', x, 1.0 / x, 'k', x, R_N, 'b')
plt.plot(1.0, 1.0, 'o', markersize=10)
plt.grid(True)
plt.xlabel("x")
plt.ylabel("$f(x)$, $T_N(x)$, $R_N(x)$")

plt.legend(["$T_N(x)$", "$f(x)$", "$R_N(x)$"], loc=8)
plt.show()
```

# En esta celda haz tus comentarios

Esta cosa con esta vaina quizas tal vez-.-.--

## Error de punto flotante

Errores surgen de aproximar números reales con números de precisión finita

$$\pi \approx 3.14$$

o $\frac{1}{3} \approx 0.333333333$ en decimal, los resultados forman un número finito de registros para representar cada número.

### Sistemas de punto flotante

Los números en sistemas de punto flotante se representan como una serie de bits que representan diferentes partes de un número. En los sistemas de punto flotante normalizados, existen algunas convenciones estándar para el uso de estos bits. En general, los números se almacenan dividiéndolos en la forma

$$F = \pm d_1 . d_2 d_3 d_4 \ldots d_p \times \beta^E$$

donde

1. $\pm$ es un bit único y representa el signo del número.

2. $d_1 . d_2 d_3 d_4 \ldots d_p$ es la *mantisa*. observe que, técnicamente, el decimal se puede mover, pero en general, utilizando la notación científica, el decimal siempre se puede colocar en esta ubicación. Los digitos $d_2 d_3 d_4 \ldots d_p$ son llamados la *fracción* con $p$ digitos de precisión. Los sistemas normalizados específicamente ponen el punto decimal en el frente y asume $d_1 \neq 0$ a menos que el número sea exactamente $0$.

3. $\beta$ es la *base*. Para el sistema binario $\beta = 2$, para decimal $\beta = 10$, etc.

4. $E$ es el *exponente*, un entero en el rango $[E_{\min}, E_{\max}]$

Los puntos importantes en cualquier sistema de punto flotante es

1. Existe un conjunto discreto y finito de números representables.

2. Estos números representables no están distribuidos uniformemente en la línea real

3. La aritmética en sistemas de punto flotante produce resultados diferentes de la aritmética de precisión infinita (es decir, matemática "real")

### Propiedades de los sistemas de punto flotante

Todos los sistemas de punto flotante se caracterizan por varios números importantes

- Número normalizado reducido (underflow si está por debajo, relacionado con números sub-normales alrededor de cero)

- Número normalizado más grande (overflow)

- Cero

- $\epsilon$ o $\epsilon_{mach}$

- `Inf` y `nan`

***Ejemplo: Sistema de juguete***

Considere el sistema decimal de 2 digitos de  precisión (normalizado)

$$f = \pm d_1 . d_2 \times 10^E$$

con $E \in [-2, 0]$.

**Numero y distribución de números**

1. Cuántos números pueden representarse con este sistema?

2. Cuál es la distribución en la línea real?

3. Cuáles son los límites underflow y overflow?

Cuántos números pueden representarse con este sistema?

$$f = \pm d_1 . d_2 \times 10^E ~~~ \text{with} E \in [-2, 0]$$

$$2 \times 9 \times 10 \times 3 + 1 = 541$$

Cuál es la distribución en la línea real?

```python
d_1_values = [1, 2, 3, 4, 5, 6, 7, 8, 9]
d_2_values = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
E_values = [0, -1, -2]

fig = plt.figure(figsize=(10.0, 1.0))
axes = fig.add_subplot(1, 1, 1)

for E in E_values:
    for d1 in d_1_values:
        for d2 in d_2_values:
            axes.plot( (d1 + d2 * 0.1) * 10**E, 0.0, 'r+', markersize=20)
            axes.plot(-(d1 + d2 * 0.1) * 10**E, 0.0, 'r+', markersize=20)

axes.plot(0.0, 0.0, '+', markersize=20)
axes.plot([-10.0, 10.0], [0.0, 0.0], 'k')

axes.set_title("Distribución de Valores")
axes.set_yticks([])
axes.set_xlabel("x")
axes.set_ylabel("")
axes.set_xlim([-0.1, 0.1])
plt.show()
```

Cuáles son los límites superior (overflow) e inferior (underflow)?

- El menor número que puede ser representado (underflow) es: $1.0 \times 10^{-2} = 0.01$

- El mayor número que puede ser representado (overflow) es:  $9.9 \times 10^0 = 9.9$

### Sistema Binario

Considere el sistema en base 2 de 2 dígitos de precisión

$$f=\pm d_1 . d_2 \times 2^E \quad \text{with} \quad E \in [-1, 1]$$

#### Numero y distribución de números**

1. Cuántos números pueden representarse con este sistema?

2. Cuál es la distribución en la línea real?

3. Cuáles son los límites underflow y overflow?

Cuántos números pueden representarse en este sistema?

$$f=\pm d_1 . d_2 \times 2^E ~~~~ \text{con} ~~~~ E \in [-1, 1]$$

$$ 2 \times 1 \times 2 \times 3 + 1 = 13$$

Cuál es la distribución en la línea real?

```python
d_1_values = [1]
d_2_values = [0, 1]
E_values = [1, 0, -1]

fig = plt.figure(figsize=(10.0, 1.0))
axes = fig.add_subplot(1, 1, 1)

for E in E_values:
    for d1 in d_1_values:
        for d2 in d_2_values:
            axes.plot( (d1 + d2 * 0.5) * 2**E, 0.0, 'r+', markersize=20)
            axes.plot(-(d1 + d2 * 0.5) * 2**E, 0.0, 'r+', markersize=20)

axes.plot(0.0, 0.0, 'r+', markersize=20)
axes.plot([-4.5, 4.5], [0.0, 0.0], 'k')

axes.set_title("Distribución de Valores")
axes.set_yticks([])
axes.set_xlabel("x")
axes.set_ylabel("")
axes.set_xlim([-3.5, 3.5])
plt.show()
```

Cuáles son los límites superior (*overflow*) e inferior (*underflow*)?

- El menor número que puede ser representado (*underflow*) es: $1.0 \times 2^{-1} = 0.5$

- El mayor número que puede ser representado (*overflow*) es:  $1.1 \times 2^1 = 3$

Observe que estos números son en sistema binario.

Una rápida regla de oro:

$$2^3 2^2 2^1 2^0 . 2^{-1} 2^{-2} 2^{-3}$$

corresponde a

8s, 4s, 2s, 1s . mitades, cuartos, octavos, $\ldots$

### Sistema real - IEEE 754 sistema binario de punto flotante

#### Precisión simple

- Almacenamiento total es de 32 bits

- Exponente de 8 bits $\Rightarrow E \in [-126, 127]$

- Fracción 23 bits ($p = 24$)

```
s EEEEEEEE FFFFFFFFFFFFFFFFFFFFFFF
0 1      8 9                     31
```

Overflow $= 2^{127} \approx 3.4 \times 10^{38}$

Underflow $= 2^{-126} \approx 1.2 \times 10^{-38}$

$\epsilon_{\text{machine}} = 2^{-23} \approx 1.2 \times 10^{-7}$

#### Precisión doble

- Almacenamiento total asignado es 64 bits

- Exponenete de 11 bits $\Rightarrow E \in [-1022, 1024]$

- Fracción de 52 bits ($p = 53$)

```
s EEEEEEEEEE FFFFFFFFFF FFFFFFFFFF FFFFFFFFFF FFFFFFFFFF FFFFFFFFFF FF
0 1       11 12                                                      63
```
Overflow $= 2^{1024} \approx 1.8 \times 10^{308}$

Underflow $= 2^{-1022} \approx 2.2 \times 10^{-308}$

$\epsilon_{\text{machine}} = 2^{-52} \approx 2.2 \times 10^{-16}$

### Acceso de Python a números de la IEEE

Accede a muchos parámetros importantes, como el epsilon de la máquina

```python
import numpy
numpy.finfo(float).eps
```

```python
import numpy
numpy.finfo(float).eps

print(numpy.finfo(numpy.float16))
print(numpy.finfo(numpy.float32))
print(numpy.finfo(float))
print(numpy.finfo(numpy.float128))
```

## Por qué debería importarnos esto?

- Aritmética de punto flotante no es conmutativa o asociativa

- Errores de punto flotante compuestos, No asuma que la precisión doble es suficiente

- Mezclar precisión es muy peligroso

### Ejemplo 1: Aritmética simple

Aritmética simple $\delta < \epsilon_{\text{machine}}$

   $$(1+\delta) - 1 = 1 - 1 = 0$$

   $$1 - 1 + \delta = \delta$$

### Ejemplo 2: Cancelación catastrófica

Miremos qué sucede cuando sumamos dos números $x$ y $y$ cuando $x+y \neq 0$. De hecho, podemos estimar estos límites haciendo un análisis de error. Aquí necesitamos presentar la idea de que cada operación de punto flotante introduce un error tal que

$$
    \text{fl}(x ~\text{op}~ y) = (x ~\text{op}~ y) (1 + \delta)
$$

donde $\text{fl}(\cdot)$ es una función que devuelve la representación de punto flotante de la expresión encerrada, $\text{op}$ es alguna operación (ex. $+, -, \times, /$), y $\delta$ es el error de punto flotante debido a  $\text{op}$.

De vuelta a nuestro problema en cuestión. El error de coma flotante debido a la suma es

$$\text{fl}(x + y) = (x + y) (1 + \delta).$$

Comparando esto con la solución verdadera usando un error relativo tenemos

$$\begin{aligned}
    \frac{(x + y) - \text{fl}(x + y)}{x + y} &= \frac{(x + y) - (x + y) (1 + \delta)}{x + y} = \delta.
\end{aligned}$$

entonces si $\delta = \mathcal{O}(\epsilon_{\text{machine}})$ no estaremos muy preocupados.

Que pasa si consideramos un error de punto flotante en la representación de $x$ y $y$, $x \neq y$, y decimos que $\delta_x$ y $\delta_y$ son la magnitud de los errores en su representación. Asumiremos que esto constituye el error de punto flotante en lugar de estar asociado con la operación en sí.

Dado todo esto, tendríamos

$$\begin{aligned}
    \text{fl}(x + y) &= x (1 + \delta_x) + y (1 + \delta_y) \\
    &= x + y + x \delta_x + y \delta_y \\
    &= (x + y) \left(1 + \frac{x \delta_x + y \delta_y}{x + y}\right)
\end{aligned}$$

Calculando nuevamente el error relativo, tendremos

$$\begin{aligned}
    \frac{x + y - (x + y) \left(1 + \frac{x \delta_x + y \delta_y}{x + y}\right)}{x + y} &= 1 - \left(1 + \frac{x \delta_x + y \delta_y}{x + y}\right) \\
    &= \frac{x}{x + y} \delta_x + \frac{y}{x + y} \delta_y \\
    &= \frac{1}{x + y} (x \delta_x + y \delta_y)
\end{aligned}$$

Lo importante aquí es que ahora el error depende de los valores de $x$ y $y$, y más importante aún, su suma. De particular preocupación es el tamaño relativo de $x + y$. A medida que se acerca a cero en relación con las magnitudes de $x$ y $y$, el error podría ser arbitrariamente grande. Esto se conoce como ***cancelación catastrófica***.

```python
dx = numpy.array([10**(-n) for n in range(1, 16)])
x = 1.0 + dx
y = -numpy.ones(x.shape)
error = numpy.abs(x + y - dx) / (dx)

fig = plt.figure()
fig.set_figwidth(fig.get_figwidth() * 2)

axes = fig.add_subplot(1, 2, 1)
axes.loglog(dx, x + y, 'o-')
axes.set_xlabel("$\Delta x$")
axes.set_ylabel("$x + y$")
axes.set_title("$\Delta x$ vs. $x+y$")

axes = fig.add_subplot(1, 2, 2)
axes.loglog(dx, error, 'o-')
axes.set_xlabel("$\Delta x$")
axes.set_ylabel("$|x + y - \Delta x| / \Delta x$")
axes.set_title("Diferencia entre $x$ y $y$ vs. Error relativo")

plt.show()
```

### Ejemplo 3: Evaluación de una función

Considere la función

$$
    f(x) = \frac{1 - \cos x}{x^2}
$$

con $x\in[-10^{-4}, 10^{-4}]$.

Tomando el límite cuando $x \rightarrow 0$ podemos ver qué comportamiento esperaríamos ver al evaluar esta función:

$$
    \lim_{x \rightarrow 0} \frac{1 - \cos x}{x^2} = \lim_{x \rightarrow 0} \frac{\sin x}{2 x} = \lim_{x \rightarrow 0} \frac{\cos x}{2} = \frac{1}{2}.
$$

¿Qué hace la representación de punto flotante?

```python
x = numpy.linspace(-1e-3, 1e-3, 100, dtype=numpy.float32)
error = (0.5 - (1.0 - numpy.cos(x)) / x**2) / 0.5

fig = plt.figure()
axes = fig.add_subplot(1, 1, 1)
axes.plot(x, error, 'o')
axes.set_xlabel("x")
axes.set_ylabel("Error Relativo")
```

### Ejemplo 4: Evaluación de un Polinomio

 $$f(x) = x^7 - 7x^6 + 21 x^5 - 35 x^4 + 35x^3-21x^2 + 7x - 1$$

```python
x = numpy.linspace(0.988, 1.012, 1000, dtype=numpy.float16)
y = x**7 - 7.0 * x**6 + 21.0 * x**5 - 35.0 * x**4 + 35.0 * x**3 - 21.0 * x**2 + 7.0 * x - 1.0

fig = plt.figure()
axes = fig.add_subplot(1, 1, 1)
axes.plot(x, y, 'r')
axes.set_xlabel("x")
axes.set_ylabel("y")
axes.set_ylim((-0.1, 0.1))
axes.set_xlim((x[0], x[-1]))
plt.show()
```

### Ejemplo 5: Evaluación de una función racional

Calcule $f(x) = x + 1$ por la función $$F(x) = \frac{x^2 - 1}{x - 1}$$

¿Cuál comportamiento esperarías encontrar?

```python
x = numpy.linspace(0.5, 1.5, 101, dtype=numpy.float16)
f_hat = (x**2 - 1.0) / (x - 1.0)

fig = plt.figure()
axes = fig.add_subplot(1, 1, 1)
axes.plot(x, numpy.abs(f_hat - (x + 1.0)))
axes.set_xlabel("$x$")
axes.set_ylabel("Error Absoluto")
plt.show()
```

## Combinación de error

En general, nos debemos ocupar de la combinación de error de truncamiento con el error de punto flotante.

- Error de Truncamiento: errores que surgen de la aproximación de una función, truncamiento de una serie.

$$\sin x \approx x - \frac{x^3}{3!} + \frac{x^5}{5!} + O(x^7)$$

- Error de punto flotante: errores derivados de la aproximación de números reales con números de precisión finita

$$\pi \approx 3.14$$

o $\frac{1}{3} \approx 0.333333333$ en decimal, los resultados forman un número finito de registros para representar cada número.

### Ejemplo 1:

Considere la aproximación de diferencias finitas donde $f(x) = e^x$ y estamos evaluando en $x=1$

$$f'(x) \approx \frac{f(x + \Delta x) - f(x)}{\Delta x}$$

Compare el error entre disminuir $\Delta x$ y la verdadera solucion $f'(1) = e$

```python
delta_x = numpy.linspace(1e-20, 5.0, 100)
delta_x = numpy.array([2.0**(-n) for n in range(1, 60)])
x = 1.0
f_hat_1 = (numpy.exp(x + delta_x) - numpy.exp(x)) / (delta_x)
f_hat_2 = (numpy.exp(x + delta_x) - numpy.exp(x - delta_x)) / (2.0 * delta_x)

fig = plt.figure()
axes = fig.add_subplot(1, 1, 1)
axes.loglog(delta_x, numpy.abs(f_hat_1 - numpy.exp(1)), 'o-', label="Unilateral")
axes.loglog(delta_x, numpy.abs(f_hat_2 - numpy.exp(1)), 's-', label="Centrado")
axes.legend(loc=3)
axes.set_xlabel("$\Delta x$")
axes.set_ylabel("Error Absoluto")
plt.show()
```

### Ejemplo 2:

Evalúe $e^x$ con la serie de *Taylor*

$$e^x = \sum^\infty_{n=0} \frac{x^n}{n!}$$

podemos elegir $n< \infty$ que puede aproximarse $e^x$ en un rango dado $x \in [a,b]$ tal que el error relativo $E$ satisfaga $E<8 \cdot \varepsilon_{\text{machine}}$?

¿Cuál podría ser una mejor manera de simplemente evaluar el polinomio de Taylor directamente por varios $N$?

```python
import scipy.special

def my_exp(x, N=10):
    value = 0.0
    for n in range(N + 1):
        value += x**n / scipy.special.factorial(n)

    return value

x = numpy.linspace(-2, 2, 100, dtype=numpy.float32)
for N in range(1, 50):
    error = numpy.abs((numpy.exp(x) - my_exp(x, N=N)) / numpy.exp(x))
    if numpy.all(error < 8.0 * numpy.finfo(float).eps):
        break

print(N)

fig = plt.figure()
axes = fig.add_subplot(1, 1, 1)
axes.plot(x, error)
axes.set_xlabel("x")
axes.set_ylabel("Error Relativo")
plt.show()
```

### Ejemplo 3: Error relativo

Digamos que queremos calcular el error relativo de dos valores $x$ y $y$ usando $x$ como valor de normalización

$$
    E = \frac{x - y}{x}
$$
y
$$
    E = 1 - \frac{y}{x}
$$

son equivalentes. En precisión finita, ¿qué forma pidría esperarse que sea más precisa y por qué?

Ejemplo tomado de [blog](https://nickhigham.wordpress.com/2017/08/14/how-and-how-not-to-compute-a-relative-error/) posteado por Nick Higham*

Usando este modelo, la definición original contiene dos operaciones de punto flotante de manera que

$$\begin{aligned}
    E_1 = \text{fl}\left(\frac{x - y}{x}\right) &= \text{fl}(\text{fl}(x - y) / x) \\
    &= \left[ \frac{(x - y) (1 + \delta_+)}{x} \right ] (1 + \delta_/) \\
    &= \frac{x - y}{x}  (1 + \delta_+) (1 + \delta_/)
\end{aligned}$$

Para la otra formulación tenemos

$$\begin{aligned}
    E_2 = \text{fl}\left( 1 - \frac{y}{x} \right ) &= \text{fl}\left(1 - \text{fl}\left(\frac{y}{x}\right) \right) \\
    &= \left(1 - \frac{y}{x} (1 + \delta_/) \right) (1 + \delta_-)
\end{aligned}$$

Si suponemos que todos las $\text{op}$s tienen magnitudes de error similares, entonces podemos simplificar las cosas dejando que

$$
    |\delta_\ast| \le \epsilon.
$$

Para comparar las dos formulaciones, nuevamente usamos el error relativo entre el error relativo verdadero $e_i$ y nuestras versiones calculadas $E_i$

Definición original

$$\begin{aligned}
    \frac{e - E_1}{e} &= \frac{\frac{x - y}{x} - \frac{x - y}{x}  (1 + \delta_+) (1 + \delta_/)}{\frac{x - y}{x}} \\
    &\le 1 - (1 + \epsilon) (1 + \epsilon) = 2 \epsilon + \epsilon^2
\end{aligned}$$

Definición manipulada:

$$\begin{aligned}
    \frac{e - E_2}{e} &= \frac{e - \left[1 - \frac{y}{x}(1 + \delta_/) \right] (1 + \delta_-)}{e} \\
    &= \frac{e - \left[e - \frac{y}{x} \delta_/) \right] (1 + \delta_-)}{e} \\
    &= \frac{e - \left[e + e\delta_- - \frac{y}{x} \delta_/ - \frac{y}{x} \delta_/ \delta_-)) \right] }{e} \\
    &= - \delta_- + \frac{1}{e} \frac{y}{x} \left(\delta_/ + \delta_/ \delta_- \right) \\
    &= - \delta_- + \frac{1 -e}{e} \left(\delta_/ + \delta_/ \delta_- \right) \\
    &\le \epsilon + \left |\frac{1 - e}{e}\right | (\epsilon + \epsilon^2)
\end{aligned}$$

Vemos entonces que nuestro error de punto flotante dependerá de la magnitud relativa de $e$

```python
# Based on the code by Nick Higham
# https://gist.github.com/higham/6f2ce1cdde0aae83697bca8577d22a6e
# Compares relative error formulations using single precision and compared to double precision

N = 501    # Note: Use 501 instead of 500 to avoid the zero value
d = numpy.finfo(numpy.float32).eps * 1e4
a = 3.0
x = a * numpy.ones(N, dtype=numpy.float32)
y = [x[i] + numpy.multiply((i - numpy.divide(N, 2.0, dtype=numpy.float32)), d, dtype=numpy.float32) for i in range(N)]

# Compute errors and "true" error
relative_error = numpy.empty((2, N), dtype=numpy.float32)
relative_error[0, :] = numpy.abs(x - y) / x
relative_error[1, :] = numpy.abs(1.0 - y / x)
exact = numpy.abs( (numpy.float64(x) - numpy.float64(y)) / numpy.float64(x))

# Compute differences between error calculations
error = numpy.empty((2, N))
for i in range(2):
    error[i, :] = numpy.abs((relative_error[i, :] - exact) / numpy.abs(exact))

fig = plt.figure()
axes = fig.add_subplot(1, 1, 1)
axes.semilogy(y, error[0, :], '.', markersize=10, label="$|x-y|/|x|$")
axes.semilogy(y, error[1, :], '.', markersize=10, label="$|1-y/x|$")

axes.grid(True)
axes.set_xlabel("y")
axes.set_ylabel("Error Relativo")
axes.set_xlim((numpy.min(y), numpy.max(y)))
axes.set_ylim((5e-9, numpy.max(error[1, :])))
axes.set_title("Comparasión Error Relativo")
axes.legend()
plt.show()
```

Algunos enlaces de utilidad con respecto al punto flotante IEEE:

- [What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)

- [IEEE 754 Floating Point Calculator](http://babbage.cs.qc.edu/courses/cs341/IEEE-754.html)

- [Numerical Computing with IEEE Floating Point Arithmetic](http://epubs.siam.org/doi/book/10.1137/1.9780898718072)

## Operaciones de conteo

- ***Error de truncamiento:*** *¿Por qué no usar más términos en la serie de Taylor?*

- ***Error de punto flotante:*** *¿Por qué no utilizar la mayor precisión posible?*

### Ejemplo 1: Multiplicación matriz - vector

Sea $A, B \in \mathbb{R}^{N \times N}$ y $x \in \mathbb{R}^N$.

1. Cuenta el número aproximado de operaciones que tomará para calcular $Ax$

2. Hacer lo mismo para $AB$

***Producto Matriz-vector:*** Definiendo $[A]_i$ como la $i$-ésima fila de $A$ y $A_{ij}$ como la $i$,$j$-ésima entrada entonces

$$
    A x = \sum^N_{i=1} [A]_i \cdot x = \sum^N_{i=1} \sum^N_{j=1} A_{ij} x_j
$$

Tomando un caso en particular, siendo $N=3$, entonces la operación de conteo es

$$
    A x = [A]_1 \cdot v + [A]_2 \cdot v + [A]_3 \cdot v = \begin{bmatrix}
        A_{11} \times v_1 + A_{12} \times v_2 + A_{13} \times v_3 \\
        A_{21} \times v_1 + A_{22} \times v_2 + A_{23} \times v_3 \\
        A_{31} \times v_1 + A_{32} \times v_2 + A_{33} \times v_3
    \end{bmatrix}
$$

Esto son 15 operaciones (6 sumas y 9 multiplicaciones)

Tomando otro caso, siendo $N=4$, entonces el conteo de operaciones es:

$$
    A x = [A]_1 \cdot v + [A]_2 \cdot v + [A]_3 \cdot v = \begin{bmatrix}
        A_{11} \times v_1 + A_{12} \times v_2 + A_{13} \times v_3 + A_{14} \times v_4 \\
        A_{21} \times v_1 + A_{22} \times v_2 + A_{23} \times v_3 + A_{24} \times v_4 \\
        A_{31} \times v_1 + A_{32} \times v_2 + A_{33} \times v_3 + A_{34} \times v_4 \\
        A_{41} \times v_1 + A_{42} \times v_2 + A_{43} \times v_3 + A_{44} \times v_4 \\
    \end{bmatrix}
$$

Esto lleva a 28 operaciones (12 sumas y 16 multiplicaciones).

Generalizando, hay $N^2$ mutiplicaciones y $N(N-1)$ sumas para un total de

$$
    \text{operaciones} = N (N - 1) + N^2 = \mathcal{O}(N^2).
$$

***Producto Matriz-Matriz ($AB$):*** Definiendo $[B]_j$ como la $j$-ésima columna de $B$ entonces

$$
    (A B)_{ij} = \sum^N_{i=1} \sum^N_{j=1} [A]_i \cdot [B]_j
$$

El producto interno de dos vectores es representado por

$$
    a \cdot b = \sum^N_{i=1} a_i b_i
$$

conduce a $\mathcal{O}(3N)$ operaciones. Como hay $N^2$ entradas en la matriz resultante, tendríamos $\mathcal{O}(N^3)$ operaciones

Existen métodos para realizar la multiplicación matriz - matriz más rápido. En la siguiente figura vemos una colección de algoritmos a lo largo del tiempo que han podido limitar el número de operaciones en ciertas circunstancias
$$
    \mathcal{O}(N^\omega)
$$

### Ejemplo 2: Método de Horner para evaluar polinomios

Dado

$$P_N(x) = a_0 + a_1 x + a_2 x^2 + \ldots + a_N x^N$$

o

$$P_N(x) = p_1 x^N + p_2 x^{N-1} + p_3 x^{N-2} + \ldots + p_{N+1}$$

queremos encontrar la mejor vía para evaluar $P_N(x)$

Primero considere dos vías para escribir $P_3$

$$ P_3(x) = p_1 x^3 + p_2 x^2 + p_3 x + p_4$$

y usando multiplicación anidada

$$ P_3(x) = ((p_1 x + p_2) x + p_3) x + p_4$$

Considere cuántas operaciones se necesitan para cada...

$$ P_3(x) = p_1 x^3 + p_2 x^2 + p_3 x + p_4$$

$$P_3(x) = \overbrace{p_1 \cdot x \cdot x \cdot x}^3 + \overbrace{p_2 \cdot x \cdot x}^2 + \overbrace{p_3 \cdot x}^1 + p_4$$

Sumando todas las operaciones, en general podemos pensar en esto como una pirámide

podemos estimar de esta manera que el algoritmo escrito de esta manera tomará aproximadamente $\mathcal{O}(N^2/2)$ operaciones para completar.

Mirando nuetros otros medios de evaluación

$$ P_3(x) = ((p_1 x + p_2) x + p_3) x + p_4$$

Aquí encontramos que el método es $\mathcal{O}(N)$ (el 2 generalmente se ignora en estos casos). Lo importante es que la primera evaluación es $\mathcal{O}(N^2)$ y la segunda $\mathcal{O}(N)$!

### Algoritmo

Complete la función e implemente el método de *Horner*

```python
def eval_poly(p, x):
    """Evaluates polynomial given coefficients p at x

    Function to evaluate a polynomial in order N operations.  The polynomial is defined as

    P(x) = p[0] x**n + p[1] x**(n-1) + ... + p[n-1] x + p[n]

    The value x should be a float.
    """
    pass
```

```python
def eval_poly(p, x):
    """Evaluates polynomial given coefficients p at x

    Function to evaluate a polynomial in order N operations.  The polynomial is defined as

    P(x) = p[0] x**n + p[1] x**(n-1) + ... + p[n-1] x + p[n]

    The value x should be a float.
    """
    ### ADD CODE HERE
    pass
```

```python
# Scalar version
def eval_poly(p, x):
    """Evaluates polynomial given coefficients p at x

    Function to evaluate a polynomial in order N operations.  The polynomial is defined as

    P(x) = p[0] x**n + p[1] x**(n-1) + ... + p[n-1] x + p[n]

    The value x should be a float.
    """

    y = p[0]
    for coefficient in p[1:]:
        y = y * x + coefficient

    return y

# Vectorized version
def eval_poly(p, x):
    """Evaluates polynomial given coefficients p at x

    Function to evaluate a polynomial in order N operations.  The polynomial is defined as

    P(x) = p[0] x**n + p[1] x**(n-1) + ... + p[n-1] x + p[n]

    The value x can by a NumPy ndarray.
    """

    y = numpy.ones(x.shape) * p[0]
    for coefficient in p[1:]:
        y = y * x + coefficient

    return y

p = [1, -3, 10, 4, 5, 5]
x = numpy.linspace(-10, 10, 100)
plt.plot(x, eval_poly(p, x))
plt.show()
```

```python

```


# Econometrics HW_08

**11510691 程远星$\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\Exp}{\mathrm{E}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\AcA}{\mathscr{A}}
\newcommand{\FcF}{\mathscr{F}}
\newcommand{\Var}[2][\,\!]{\mathrm{Var}_{#1}\left[#2\right]}
\newcommand{\Avar}[2][\,\!]{\mathrm{Avar}_{#1}\left[#2\right]}
\newcommand{\Cov}[2][\,\!]{\mathrm{Cov}_{#1}\left(#2\right)}
\newcommand{\Corr}[2][\,\!]{\mathrm{Corr}_{#1}\left(#2\right)}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N} \left( #1 \right)}
\newcommand{\ow}{\text{otherwise}}
\void^\dagger$**

## Question 5

$\P{1}$

$\bspace$The two set of standard errors are so close and there's really no needs to distinguish them from each other.

$\P{2}$

$\bspace$Holding all other variables fixed, the probability of smoking changes by about $-0.029\times4=-0.116$.

$\P{3}$

$$\abs{\ffrac{0.02} {2\times 0.00026}}\approx 38.46153846153846$$

$\P{4}$

$\bspace$Holding other factors in the equation fixed, the probability to smoke will decrease $0.101$ for a person in a state with restaurant smoking restrictions.

$\P{5}$

$$\begin{align}\widehat{\text{smoke}} &= 0.656-0.069\times\log\P{67.44} + 0.012\times\log\P{6500} - 0.029 \times 16\\
&\bspace+ 0.0207\times77 -0.00026\times77^2 -0.101\times0-0.026\times 0\\
&\approx 0.0052
\end{align}$$

## Question 6

$\P{1}$

$\bspace$The numerator has $k+1$ regressors, and that's the $df$ for it. For the denominator, its $df$ is $n-\P{k-2}$

$\P{2}$

$\bspace$In BP test, there's one more regressor thus it's got a higher $R$-squared. In White test, the model has more restrictions and thus the $R$-squared will be higher.

$\P{3}$

$\bspace$For $t$ test statistic will be a little bit smaller however the change of $F$ statistic is unpredictable. The $\text{SSR}$s will be larger while $df$s also do.

$\P{4}$

$\bspace$Collinearity. Since the estimated equation will be a linear combination of all variables.

## Question 7

$\P{1}$

$\bspace$Since the two are uncorrelated, we have $\Var{u_{i,e}} = \Var{f_i} + \Var{v_{i,e}} = \sigma_f^2 + \sigma_v^2$

$\P{2}$

$$\begin{align}
\Cov{u_{i,e},u_{i,g}} &= \Cov{f_i + v_{i,e},f_i + v_{i,g}}\\
&= \Cov{f_i,f_i} + \Cov{f_i,v_{i,g}} + \Cov{v_{i,e},f_i} + \Cov{v_{i,e},v_{i,g}}\\
&= \Cov{f_i,f_i} + 0 + 0 + 0 = \Var{f_i}
\end{align}$$

$\P{3}$

$$\begin{align}
\Var{\bar u_i} &= \Var{\ffrac{1} {m_i}\sum_{e=1}^{m_i} u_{i,e}} \\
&= \Var{f_i + \ffrac{1} {m_i}\sum_{e=1}^{m_i} v_{i,e}}\\
&= \Var{f_i} + \Var{\ffrac{1} {m_i}\sum_{e=1}^{m_i} v_{i,e}}\\
&= \sigma_f^2 + \ffrac{\sigma_v^2} {m_i}
\end{align}$$

$\P{4}$

$\bspace$From the weighted OLS method, our target is to find some specific weight so that $\Var{\bar u_i} = \ffrac{\sigma_f^2} {m_i}$. If we take the weight so that the data are simply averaged, then like the preceding problem, we failed.