# Section 0 问题描述与完成项目流程

## 1. 问题描述

</img>

在该项目中，你将使用强化学习算法，实现一个自动走迷宫机器人。

1. 如上图所示，智能机器人显示在右上角。在我们的迷宫中，有陷阱（红色炸弹）及终点（蓝色的目标点）两种情景。机器人要尽量避开陷阱、尽快到达目的地。
2. 小车可执行的动作包括：向上走 `u`、向右走 `r`、向下走 `d`、向左走 `l`。
3. 执行不同的动作后，根据不同的情况会获得不同的奖励，具体而言，有以下几种情况。
    - 撞到墙壁：-10
    - 走到终点：50
    - 走到陷阱：-30
    - 其余情况：-0.1
4. 我们需要通过修改 `robot.py` 中的代码，来实现一个 Q Learning 机器人，实现上述的目标。

## 2. 完成项目流程

1. 配置环境，使用 `envirnment.yml` 文件配置名为 `robot-env` 的 conda 环境，具体而言，你只需转到当前的目录，在命令行/终端中运行如下代码，稍作等待即可。
```
conda env create -f envirnment.yml
```
安装完毕后，在命令行/终端中运行 `source activate robot-env`（Mac/Linux 系统）或 `activate robot-env`（Windows 系统）激活该环境。

2. 阅读 `main.ipynb` 中的指导完成项目，并根据指导修改对应的代码，生成、观察结果。
3. 导出代码与报告，上传文件，提交审阅并优化。
---

---

# Section 1 算法理解

## 1. 1 强化学习总览

强化学习作为机器学习算法的一种，其模式也是让智能体在“训练”中学到“经验”，以实现给定的任务。但不同于监督学习与非监督学习，在强化学习的框架中，我们更侧重通过智能体与环境的**交互**来学习。通常在监督学习和非监督学习任务中，智能体往往需要通过给定的训练集，辅之以既定的训练目标（如最小化损失函数），通过给定的学习算法来实现这一目标。然而在强化学习中，智能体则是通过其与环境交互得到的奖励进行学习。这个环境可以是虚拟的（如虚拟的迷宫），也可以是真实的（自动驾驶汽车在真实道路上收集数据）。

在强化学习中有五个核心组成部分，它们分别是：**环境（Environment）**、**智能体（Agent）**、**状态（State）**、**动作（Action）**和**奖励（Reward）**。在某一时间节点 $t$：

- 智能体在从环境中感知其所处的状态 $s_t$
- 智能体根据某些准则选择动作 $a_t$
- 环境根据智能体选择的动作，向智能体反馈奖励 $r_{t+1}$

通过合理的学习算法，智能体将在这样的问题设置下，成功学到一个在状态 $s_t$ 选择动作 $a_t$ 的策略 $\pi (s_t) = a_t$。

---

**问题 1**：请参照如上的定义，描述出 “机器人走迷宫这个问题” 中强化学习五个组成部分对应的实际对象：

- **环境** : 有陷阱的迷宫
- **状态** : 机器人在迷宫中的位置

- **动作** : 机器人选择向上走 u、向右走 r、向下走 d、向左走 l
- **奖励** : 撞到墙壁：-10，走到终点：50，走到陷阱：-30，其余情况：-0.1

$$T(s^{'}, a, s) = P(s^{'}|a,s)$$

---

## 1.2 计算 Q 值

在我们的项目中，我们要实现基于 Q-Learning 的强化学习算法。Q-Learning 是一个值迭代（Value Iteration）算法。与策略迭代（Policy Iteration）算法不同，值迭代算法会计算每个”状态“或是”状态-动作“的值（Value）或是效用（Utility），然后在执行动作的时候，会设法最大化这个值。因此，对每个状态值的准确估计，是我们值迭代算法的核心。通常我们会考虑**最大化动作的长期奖励**，即不仅考虑当前动作带来的奖励，还会考虑动作长远的奖励。

在 Q-Learning 算法中，我们把这个长期奖励记为 Q 值，我们会考虑每个 ”状态-动作“ 的 Q 值，具体而言，它的计算公式为：

$$
q(s_{t},a) = R_{t+1} + \gamma \times\max_a q(a,s_{t+1})
$$

也就是对于当前的“状态-动作” $(s_{t},a)$，我们考虑执行动作 $a$ 后环境给我们的奖励 $R_{t+1}$，以及执行动作 $a$ 到达 $s_{t+1}$后，执行任意动作能够获得的最大的Q值 $\max_a q(a,s_{t+1})$，$\gamma$ 为折扣因子。

不过一般地，我们使用更为保守地更新 Q 表的方法，即引入松弛变量 $alpha$，按如下的公式进行更新，使得 Q 表的迭代变化更为平缓。

$$
q(s_{t},a) = (1-\alpha) \times q(s_{t},a) + \alpha \times(R_{t+1} + \gamma \times\max_a q(a,s_{t+1}))
$$

---
</img>

**问题 2**：根据已知条件求 $q(s_{t},a)$，在如下模板代码中的空格填入对应的数字即可。

已知：如上图，机器人位于 $s_1$，行动为 `u`，行动获得的奖励与题目的默认设置相同。在 $s_2$ 中执行各动作的 Q 值为：`u`: -24，`r`: -13，`d`: -0.29、`l`: +40，$\gamma$ 取0.9。

$$
\begin{align}
q(s_{t},a) & = R_{t+1} + \gamma \times\max_a q(a,s_{t+1}) \\
 & =(-0.1) + (0.9)*(40) \\
 & =(35.9)
\end{align}
$$

---

## 1.3 如何选择动作

在强化学习中，「探索-利用」问题是非常重要的问题。具体来说，根据上面的定义，我们会尽可能地让机器人在每次选择最优的决策，来最大化长期奖励。但是这样做有如下的弊端：
1. 在初步的学习中，我们的 Q 值会不准确，如果在这个时候都按照 Q 值来选择，那么会造成错误。
2. 学习一段时间后，机器人的路线会相对固定，则机器人无法对环境进行有效的探索。

因此我们需要一种办法，来解决如上的问题，增加机器人的探索。由此我们考虑使用 epsilon-greedy 算法，即在小车选择动作的时候，以一部分的概率随机选择动作，以一部分的概率按照最优的 Q 值选择动作。同时，这个选择随机动作的概率应当随着训练的过程逐步减小。

---

**问题 3**：在如下的代码块中，实现 epsilon-greedy 算法的逻辑，并运行测试代码。

```python
import random
import numpy as np

actions = ['u','r','d','l']
qline = {'u':1.2, 'r':-2.1, 'd':-24.5, 'l':27}
epsilon = 0.3 # 以0.3的概率进行随机选择

def epsilon_greedy_probs(Q_s, epsilon, nA):
    """ obtains the action probabilities corresponding to epsilon-greedy policy """
    policy_s = np.ones(nA) * epsilon / nA
    best_a = np.argmax(Q_s)
    policy_s[best_a] = 1 - epsilon + (epsilon / nA)
    return policy_s

def choose_action(epsilon):

    action = None
    if epsilon < random.random(): # 以某一概率
        #pass # 实现对动作的随机选择
        p = np.random.choice(np.arange(len(actions)), p=epsilon_greedy_probs(qline, epsilon, len(actions)))
        action = actions[p]
    else:
        #pass # 否则选择具有最大 Q 值的动作
        action = max(qline,key=qline.get)
    return action

```

---

---

# Section 2 代码实现

## 2.1. `Maze` 类理解

我们首先引入了迷宫类 `Maze`，这是一个非常强大的函数，它能够根据你的要求随机创建一个迷宫，或者根据指定的文件，读入一个迷宫地图信息。

1. 使用 `Maze("file_name")` 根据指定文件创建迷宫，或者使用 `Maze(maze_size=(height,width))` 来随机生成一个迷宫。
2. 使用 `trap_number` 参数，在创建迷宫的时候，设定迷宫中陷阱的数量。
3. 直接键入迷宫变量的名字按回车，展示迷宫图像（如 `g=Maze("xx.txt")`，那么直接输入 `g` 即可。
4. 建议生成的迷宫尺寸，长在 6~12 之间，宽在 10～12 之间。

---

**问题 4**：在如下的代码块中，创建你的迷宫并展示。

```python
from Maze import Maze
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

## todo: 创建迷宫并展示
mm = Maze(maze_size=(10,12),trap_number=6)
mm
```

---

你可能已经注意到，在迷宫中我们已经默认放置了一个机器人。实际上，我们为迷宫配置了相应的 API，来帮助机器人的移动与感知。其中你随后会使用的两个 API 为 `maze.sense_robot()` 及 `maze.move_robot()`。

1. `maze.sense_robot()` 为一个无参数的函数，输出机器人在迷宫中目前的位置。
2. `maze.move_robot(direction)` 对输入的移动方向，移动机器人，并返回对应动作的奖励值。

---

**问题 5**：随机移动机器人，并记录下获得的奖励，展示出机器人最后的位置。

```python
rewards = []

## 循环、随机移动机器人10次，记录下奖励
for i in range(10):
    rewards.append(mm.move_robot(choose_action(i)))

## 输出机器人最后的位置
print(mm.sense_robot())
print(rewards)

## 打印迷宫，观察机器人位置
mm
```

## 2.2. `Robot` 类实现

`Robot` 类是我们需要重点实现的部分。在这个类中，我们需要实现诸多功能，以使得我们成功实现一个强化学习智能体。总体来说，之前我们是人为地在环境中移动了机器人，但是现在通过实现 `Robot` 这个类，机器人将会自己移动。通过实现学习函数，`Robot` 类将会学习到如何选择最优的动作，并且更新强化学习中对应的参数。

首先 `Robot` 有多个输入，其中 `alpha=0.5, gamma=0.9, epsilon0=0.5` 表征强化学习相关的各个参数的默认值，这些在之前你已经了解到，`Maze` 应为机器人所在迷宫对象。

随后观察 `Robot.update` 函数，它指明了在每次执行动作时，`Robot` 需要执行的程序。按照这些程序，各个函数的功能也就明了了。

最后你需要实现 `Robot.py` 代码中的8段代码，他们都在代码中以 `#TODO` 进行标注，你能轻松地找到他们。

---

**问题 6**：实现 `Robot.py` 中的8段代码，并运行如下代码检查效果（记得将 `maze` 变量修改为你创建迷宫的变量名）。

```python
from Robot import Robot
robot = Robot(mm) # 记得将 maze 变量修改为你创建迷宫的变量名
robot.set_status(learning=True,testing=False)
print(robot.update())

mm
```

---

## 2.3 用 `Runner` 类训练 Robot

在实现了上述内容之后，我们就可以开始对我们 `Robot` 进行训练并调参了。我们为你准备了又一个非常棒的类 `Runner`，来实现整个训练过程及可视化。使用如下的代码，你可以成功对机器人进行训练。并且你会在当前文件夹中生成一个名为 `filename` 的视频，记录了整个训练的过程。通过观察该视频，你能够发现训练过程中的问题，并且优化你的代码及参数。

---

**问题 7**：尝试利用下列代码训练机器人，并进行调参。可选的参数包括：

- 训练参数
    - 训练次数 `epoch`
- 机器人参数：
    - `epsilon0` (epsilon 初值)
    - `epsilon`衰减（可以是线性、指数衰减，可以调整衰减的速度），你需要在 Robot.py 中调整
    - `alpha`
    - `gamma`
- 迷宫参数:
    - 迷宫大小
    - 迷宫中陷阱的数量

```python
## 可选的参数：
epoch = 20

epsilon0 = 0.5
alpha = 0.5
gamma = 0.9

maze_size = (6,6)
trap_number = 1
```

```python
from Runner import Runner
%matplotlib inline

g = Maze(maze_size=maze_size,trap_number=trap_number)
r1 = Robot(g,alpha=0.5, epsilon0=0.5, gamma=0.9)
r2 = Robot(g,alpha=0.01, epsilon0=0.5, gamma=0.9)
r3 = Robot(g,alpha=0.5, epsilon0=0.01, gamma=0.9)
r4 = Robot(g,alpha=0.5, epsilon0=0.5, gamma=0.1)
r1.set_status(learning=True)
r2.set_status(learning=True)
r3.set_status(learning=True)
r4.set_status(learning=True)

runner1 = Runner(r1, g)
runner2 = Runner(r2, g)
runner3 = Runner(r3, g)
runner4 = Runner(r4, g)
runner1.run_training(epoch, display_direction=True)
runner2.run_training(epoch, display_direction=True)
runner3.run_training(epoch, display_direction=True)
runner4.run_training(epoch, display_direction=True)
#runner.generate_movie(filename = "final1.mp4") # 你可以注释该行代码，加快运行速度，不过你就无法观察到视频了。
```

---

使用 `runner.plot_results()` 函数，能够打印机器人在训练过程中的一些参数信息。

- Success Times 代表机器人在训练过程中成功的累计次数，这应当是一个累积递增的图像。
- Accumulated Rewards 代表机器人在每次训练 epoch 中，获得的累积奖励的值，这应当是一个逐步递增的图像。
- Running Times per Epoch 代表在每次训练 epoch 中，小车训练的次数（到达终点就会停止该 epoch 转入下次训练），这应当是一个逐步递减的图像。

---

**问题 8**：使用 `runner.plot_results()` 输出训练结果，根据该结果对你的机器人进行分析。

- 指出你选用的参数如何，选用参数的原因。
- 建议你比较不同参数下机器人的训练的情况。
- 训练的结果是否满意，有何改进的计划。

```python
runner1.plot_results()
runner2.plot_results()
runner3.plot_results()
runner4.plot_results()
```

(回答区)
epsilon0：epsilon-greedy 算法中的epsilon值，表示随机选择动作概率
alpha：当前知识权重，alpha越大，当前知识对策略影响越大，之前知识影响越小
gamma：为折扣因子，表示执行动作s(t+1)状态后，执行任意动作能够获得的最大的Q值对当前Q值影响程度，gamma越大，对当前Q值影响越大

对比上图，alpha较小时，Runtime比较大，训练需要更长时间；epsilon0较低，按照已经学到的qtable执行，前期训练可能会出现转圈，达不到终点；gamma较小，终点处的正奖励不能够“扩散”到周围，机器人很有可能无法学习到一个到达终点的策略

对于epsilon0 = 0.5，alpha = 0.5，gamma = 0.9，曲线比较平滑，效果较好。

---

**问题 9：** 请将如下的文件打包，提交文件给审阅者。

- `robot.py`
- `robot_maze.ipynb`
- 由 `robot_maze.ipynb` 导出的 `robot_maze.html`


# DIY ZHL

## References etc.

  * Mark Powell's "Deco for Divers" has all the background information in a succinct and easy to read form. The book should be available on interlibrary loan, ask your librarian.

  * Erik Baker's papers: "Decolessons", "Understanding M-values", "Clearing up the confusion about Deep Stops", are more technical and tend to omit much of the background info. PDFs are freely available on-line from several sources.

  * Buhlmann's "Decompression - Decompression sickness".

  * Several open-source implementations are out there, most notable Subsurface software, but you have to read C.

  * Plenty of other on-line sources, e.g. Stuart Morrison's "DIY Decompresion"

## Primer

### Dissolved gas

Gas is driven in and out of tissue by the difference in pressure: "outside" aka ambient $ P_{amb} $   and "inside" aka dissolved gas pressure, $ P_{t} $. The rate is assumed to be $ ln(2) $

**Partial pressure** refers to pressures of individual gases in the breathing mix: oxygen, nitrogen, etc.

### Tissue compartments

To simulate real tissue the model uses hypothetical "compartments". They are defined by the time $ T_{ h } $ it takes for the pressure difference (above) to get to -- since we're using log 2 -- half (or double, depending on whether the tissues in on- or off-gassing) its intial value. That time is callled ``half time``. It is usually given in minutes.

The body is modelled as a set of "tissue compartments" with different half-times. They have nothing to do with real tissues, but to illustrate the concept it is usually said that blood is "fast": gas gets in and out very quickly, whereas bone is very "slow".

### M-values

The greater the difference between $ P_{amb} $   and $ P_t $, the faster the gas flows. There is, however, such thing as too fast. To recreational divers "too fast" is only relevant on the way up.

The model defines "maximum allowed pressure difference" value, ``M-value`` for short. The model does not attempt to quantify the risk, it only works in binary "above the M-value" (bad) or "below the M-value" (OK) terms.

Haldane originally set his M-value as a fixed ratio (2:1). Workman replaced it with a function: a straight line $ M = \Delta M + M_0 $ with greater overpressure tolerated at greater depths.

## Water vapour etc. in alveolae

Breathing gas in respiratory passages is diluted with water vapour and expelled carbon dioxide. The fraction of water vapour and carbon dioxide is subtracted from the partial pressure of inert gas.

$
\begin{align}
P_{alv} = ( P_{amb} - P_{H_{2}O} + ( 1 - RQ ) / RQ * P_{CO_{2}} ) Q
\end{align}
$

where
  * $ P_{ CO_2 } $ is pressure of carbon dioxide, 40 mm Hg or 0.0534 bar,
  * $ RQ $ is respiratory quotient,
  * $ P_{ H_{2}O } $ is pressure of water vapour, 47 mm Hg or 0.0627 bar,
  * $ Q $ is the fraction of inert gas,
  * $ P_{amb} $ is ambient pressure.

$ RQ $ is a ratio of oxygen consumed to carbon dioxide produced in metabolism, it depends on nutrition and exertion and is in the range 0.7..1. Smaller number results in greater calculated gas loading. Buhlmann uses 1 whereas Shreiner uses 0.8 and USN uses 0.9.

## Schreiner equation to calculate inert gas pressure in a tissue compartment

$
\begin{align}
P_{t}(t) = P_{alv0} + R * (t - 1 / k) - (P_{alv0} - P_{t0} - R / k) * e^{-k * t}
\end{align}
$

*where*

  * $ P_{t}(t) $ : partial pressure of inert gas in the tissue (bar) at time $ t $

  * $ P_{alv0} $ : initial alveolar partial pressure of the gas in the breathing mix (bar)

  * $ P_{t0} $ : initial partial pressure of the gas in the tissue (bar) (at time $ t0 $)

  * $ R $ is rate of change in inert gas pressure in the breathing mix, bar/min. This is rate of change in ambient pressure, $ \Delta P $ times fraction of inert gas. $ \Delta P $ is negative for ascent, positive for descent, zero for constant depth.

  * $ t $ : time of exposure in minutes

  * $ k $ : gas decay constant for a tissue compartment: $ k = \frac  {\ln(2)} {T_{h}} $

    * $ \quad T_{h} $ : gas "half-life" time for tissue compartment in minutes

  * $ e $ : 2.7182818284590452353602874713527 and counting

Note that for constant depth this reduces to a "short form" given in many sources, the code below just uses the full form with $ R = 0 $

## M-values

### Workman

Workman-style M-value $ M $ is defined in terms of $ \Delta M $ and $ M_{ 0 } $, where

  * $ M_{0} $ is the surfacing M-value and
  * $ \Delta M $ is the slope of M-value line: multiply by depth to get tha actual number at that depth.

$ M = \Delta M * Depth + M_{0} $

$ Tolerated Depth = ( P - M_{0} ) / \Delta M $

where Depth is measured as pressure from sea level and Tolerated Depth is how high you can ascend without exceeding the M-value.

### Buhlmann

In Buhlmann model the M-values are expresed in terms of two coeficients: $ a $ and $ b $. In addition, it is using absolute instead od sea-level pressure for diving at altitude, and comes with separate numbers for different inert gases. Buhlmann's model calls "Tolerated Depth" an "ascent ceiling".

### Conversion

  * Workman to Buhlmann:

   * $ a = M_{0} - \Delta M * P_{ \text{ amb. (surface at sea level) }} $

   * $ b = 1 / \Delta M $

  * Buhlmann to Workman:

   * $ \Delta M = 1 / b $

   * $ M_{0} = a + \frac { P_{ \text{ amb. (surface at sea level) }}} { b } $

**Note** that this only really works with $ \Delta M $ greater than 1. See Erik Baker's "M-values" for the why.

## No-stop time

### by (variant of) Schreiner equation

This needs Workman-style M-values:

$
\begin{align}
P_{no-deco} \ = ( M_{0} - P_{alv0} - R * (t_{asc} - 1/k) ) * e ^ {k * t_{asc}} \ + P_{alv0} - R / k
\end{align}
$
  * $ P_{no-deco} $   is the pressure at which ascent must be started (bar),
  * $ M_{0} $ is the surfacing M-value,
  * $ P_{alv0} $ is alveolar partial pressure (bar),
  * $ t_{asc} $ is the time needed for ascent (minutes): depth divided by speed.
  * $ R, k, e $ are the same as in Schreiner's equation above.

You have to figure out how long it'll take to get from the current inspired pressure $ P_{t} $ to $ P_{no-deco} $   : run `schreiner()` in a loop adding a minute of exposure on each iteration until you hit $ P_{no-deco} $   (or do a fancy binary search from, say, 99 minutes down, etc.)

### by Buhlmann equation

Buhlmann equation calculates Tolerated Depth in absolute pressure and calls it "safe ascent ceiling":

$
\begin{align}
P_{amb.tol.} = ( P_{t} - a ) * b
\end{align}
$

where
  * $ P_{amb.tol.} $   is the ascent ceiling,
  * $ P_{t} $ is current inert gas pressure in the tisue, and
  * $ a $ and $ b $ are Buhlmann's coefficients.

When the calculated $ P_{amb.tol.} $   is above 1, i.e. under the water surface (assuming surface pressure is 1 bar of course), that means we have a deco ceiling and must stop and off-gas there.

Buhlmann equation modified by Erik Baker's Gradient Factors is

$
\begin{align}
P_{amb.tol.} = ( P_{t} - a * GF ) / ( GF / b + 1.0 - GF )
\end{align}
$

where $ GF $ is the "current" gradient factor.

(Without the gradient factor: $ GF = 1 $, that reduces to $ ( P_{t} - a * 1 ) / ( 1/b + 1 - 1 ) => ( P_{t} - a ) / ( 1/b ) =>  ( P_{t} - a ) * b $: Buhlmann's original.)

Gradient Factors plural are a pair of numbers: $ GFLo $ that applies at first decompression stop and $ GFHi $ that applies at the last stop: the surface. The "current" $ GF $ at any given depth between those two points is calculated from a simple proportion. **Note** that this only works when you have a relatively deep decompression stop.

### You can calculate no-stop time
by running `shreiner()` and `buhlmann()` in a loop, adding a minute of exposure at each iteration, until the latter produces a ceiling. **Note** however, that neither this method nor the equation above factor in off-gassing that will take place during ascent.

### Helium

With multiple inert gases, each has to be tracked separately by Schreiner equation.

For Buhlmann's formula you add up the partial pressures and proportionally adjust the $ a $ and $ b $:

$ P_{t} = P_{iN_2} + P_{iHe} $

$ a = (a_{N_2} * P_{N_2} + a_{He} * P_{He}) / P_{t} $

$ b = (b_{N_2} * P_{N_2} + b_{He} * P_{He}) / P_{t} $

## Decompression stop

... is ascent ceiling rounded to next 3 metres down.

The best I could find, in Buhlmann's "Decompression", is in open water it's hard to maintain stops at smaller intervals. Buhlmann suggests using smaller intervals or even "ride the M-value line" continuously for in-chamber decompression there. (**Note** this assumes no gas switching.)

Once you're at deco stop, run `schreiner()` and `buhlmann()` in a loop adding a minute on each iteration, as usual, until your next ceiling is at your next stop: 3 metres above you current depth if using the 3 m spacing.

## Tissue compartments

Tables are coded as python dicts: keys for each compartment are ``t``, ``a``, and ``b`` for half-time and the coefficients resp. E.g. ZH-L12's $ b $ coefficient for nitrogen in TC 3 is ``ZHL12N[3]["b"]``

### ZH-L16

**Note** that you have two options for the 1st (fastest) tissue compartment: 4 minutes or 5. Pick one. The 5-minute TC is usually called "1b" and is keyed as "1.1" in the table below.

The ''a'' key in the nitrogen tables is a dict itself with three sub-keys: ''A'', ''B'', adn ''C'' for the three versions of ZH-L16 resp. Helium numbers seem to be unobtainable for ''A'' or ''C'', nor for the 4-minute tissue compartment. I.e. if you want to use this for trimix, ZH-L16B with 5-minute TC is your only option.

M-values for "-A" were calculated as

$
\begin{align}
a = 2 * T_{h} ^ {- \frac{1}{3}}
\end{align}
$

$
\begin{align}
b = 1.005 - T_{h} ^ {- \frac{1}{2}}
\end{align}
$

  * $ T_{h} $ is tissue compartment half-time

### Workman

Not useful as is, would need to be recalculated for $ \Delta M $ instead of $ M $ first, then converted to Buhlmann's `a` and `b` coefficients.

### DSAT

Convert to Buhlmann's `a` and `b` using $ \Delta M = 1 $

## Code

Run the cell below to load python code (assuming you have the `diyzhl.py` in the same directory as the notebook file).

The functions are written to closely follow the above formulae. That includes calculating constants on the fly, like e.g. $ K $ or $ P_{alv} $   -- you'd normally use pre-calculated numbers for your chosen parameters.

About the only added features are
 * some of them save intermediate results in variables: mainly because I wanted to play with rounding at different steps. Note that although I round the numbers to 4 decimal digits, they're probably only meaningful to 2 at best.

 * Typecasts to `float` are there so that passing in not-a-number will throw an exception early.

 * Similarly, `asserts` are used to catch division by zero early.

```python
# %load diyzhl.py
#!/usr/bin/python -u
#
# (K) Copy Rites Reversed: reuse what you like (but give credit)
#
# Credits:
#
# Mark Powell's "Deco for Divers"
# Erik Baker's papers: "Decolessons", "Understanding M-values", and "Clearing up the confusion about Deep Stops" in particular
# Buhlmann's "Decompression - Decompression Sickness", English edition
# Several open-source implementations, most notably Subsurface software (and people, Robert in particular)
# Plenty of other on-line sources, e.g. Stuart Morrison's "DIY Decompresion"
#
# The goal here is "by the book" implementation to use for learning this stuff.
# There's a lot of things that can be done differently, more efficeint, and so on... and
#  that's not what this code is for.
#

import sys
import math

# ZH-L12 from "Decompression"
#
ZHL12N = {
    1 :  { "t" : 2.65,  "a" : 2.2,   "b" : 0.82 },
    2 :  { "t" : 7.94,  "a" : 1.5,   "b" : 0.82 },
    3 :  { "t" : 12.2,  "a" : 1.08,  "b" : 0.825 },
    4 :  { "t" : 18.5,  "a" : 0.9,   "b" : 0.835 },
    5 :  { "t" : 26.5,  "a" : 0.75,  "b" : 0.845 },
    6 :  { "t" : 37.0,  "a" : 0.58,  "b" : 0.86 },
    7 :  { "t" : 53.0,  "a" : 0.47,  "b" : 0.87 },
    8 :  { "t" : 79.0,  "a" : 0.45,  "b" : 0.89 },
    9 :  { "t" : 114.0, "a" : 0.45,  "b" : 0.89 },
    10 : { "t" : 146.0, "a" : 0.455, "b" : 0.934 },
    11 : { "t" : 185.0, "a" : 0.455, "b" : 0.934 },
    12 : { "t" : 238.0, "a" : 0.38,  "b" : 0.944 },
    13 : { "t" : 304.0, "a" : 0.255, "b" : 0.962 },
    14 : { "t" : 397.0, "a" : 0.255, "b" : 0.962 },
    15 : { "t" : 503.0, "a" : 0.255, "b" : 0.962 },
    16 : { "t" : 635.0, "a" : 0.255, "b" : 0.962 }
}

ZHL12He = {
    1 :  { "t" : 1.0,   "a" : 2.2,   "b" : 0.82 },
    2 :  { "t" : 3.0,   "a" : 1.5,   "b" : 0.82 },
    3 :  { "t" : 4.6,   "a" : 1.08,  "b" : 0.825 },
    4 :  { "t" : 7.0,   "a" : 0.9,   "b" : 0.835 },
    5 :  { "t" : 10.0,  "a" : 0.75,  "b" : 0.845 },
    6 :  { "t" : 14.0,  "a" : 0.58,  "b" : 0.86 },
    7 :  { "t" : 20.0,  "a" : 0.47,  "b" : 0.87 },
    8 :  { "t" : 30.0,  "a" : 0.45,  "b" : 0.89 },
    9 :  { "t" : 43.0,  "a" : 0.45,  "b" : 0.89 },
    10 : { "t" : 55.0,  "a" : 0.515, "b" : 0.926 },
    11 : { "t" : 70.0,  "a" : 0.515, "b" : 0.926 },
    12 : { "t" : 90.0,  "a" : 0.515, "b" : 0.926 },
    13 : { "t" : 115.0, "a" : 0.515, "b" : 0.926 },
    14 : { "t" : 150.0, "a" : 0.515, "b" : 0.926 },
    15 : { "t" : 190.0, "a" : 0.515, "b" : 0.926 },
    16 : { "t" : 240.0, "a" : 0.515, "b" : 0.926 },
}

# ZH_L16: several sources incl. a photo of a page from Tauchmedizin @
# http://www.nigelhewitt.co.uk/stuff/aab.jpg
# It appears nobody has Helium numbers for "-A" and "-C", nor for the 4-minute TC
# 5-minute TC is keyed as 1.1
#
ZHL16N = {
    1   : { "t" : 4.0,   "b" : 0.505,  "a" : { "A" : 1.2599, "B" : 1.2599, "C" : 1.2599 } },
    1.1 : { "t" : 5.0,   "b" : 0.5578, "a" : { "A" : 1.1696, "B" : 1.1696, "C" : 1.1696 } },
    2   : { "t" : 8.0,   "b" : 0.6514, "a" : { "A" : 1.0,    "B" : 1.0,    "C" : 1.0 } },
    3   : { "t" : 12.5,  "b" : 0.7222, "a" : { "A" : 0.8618, "B" : 0.8618, "C" : 0.8618 } },
    4   : { "t" : 18.5,  "b" : 0.7825, "a" : { "A" : 0.7562, "B" : 0.7562, "C" : 0.7562 } },
    5   : { "t" : 27.0,  "b" : 0.8126, "a" : { "A" : 0.6667, "B" : 0.6667, "C" : 0.62 } },
    6   : { "t" : 38.3,  "b" : 0.8434, "a" : { "A" : 0.5933, "B" : 0.56,   "C" : 0.5043 } },
    7   : { "t" : 54.3,  "b" : 0.8693, "a" : { "A" : 0.5282, "B" : 0.4947, "C" : 0.441 } },
    8   : { "t" : 77.0,  "b" : 0.891,  "a" : { "A" : 0.4701, "B" : 0.45,   "C" : 0.4 } },
    9   : { "t" : 109.0, "b" : 0.9092, "a" : { "A" : 0.4187, "B" : 0.4187, "C" : 0.375 } },
    10  : { "t" : 146.0, "b" : 0.9222, "a" : { "A" : 0.3798, "B" : 0.3798, "C" : 0.35 } },
    11  : { "t" : 187.0, "b" : 0.9319, "a" : { "A" : 0.3497, "B" : 0.3497, "C" : 0.3295 } },
    12  : { "t" : 239.0, "b" : 0.9403, "a" : { "A" : 0.3223, "B" : 0.3223, "C" : 0.3065 } },
    13  : { "t" : 305.0, "b" : 0.9477, "a" : { "A" : 0.2971, "B" : 0.285,  "C" : 0.2835 } },
    14  : { "t" : 390.0, "b" : 0.9544, "a" : { "A" : 0.2737, "B" : 0.2737, "C" : 0.261 } },
    15  : { "t" : 498.0, "b" : 0.9602, "a" : { "A" : 0.2523, "B" : 0.2523, "C" : 0.248 } },
    16  : { "t" : 635.0, "b" : 0.9653, "a" : { "A" : 0.2327, "B" : 0.2327, "C" : 0.2327 } }
}

ZHL16He = {
    1.1 : { "t" : 1.88,   "a" : { "B" : 1.6189 }, "b" : 0.477 },
    2 :   { "t" : 3.02,   "a" : { "B" : 1.383 },  "b" : 0.5747 },
    3 :   { "t" : 4.72,   "a" : { "B" : 1.1919 }, "b" : 0.6527 },
    4 :   { "t" : 6.99,   "a" : { "B" : 1.0458 }, "b" : 0.7223 },
    5 :   { "t" : 10.21,  "a" : { "B" : 0.922 },  "b" : 0.7582 },
    6 :   { "t" : 14.48,  "a" : { "B" : 0.8205 }, "b" : 0.7957 },
    7 :   { "t" : 20.53,  "a" : { "B" : 0.7305 }, "b" : 0.8279 },
    8 :   { "t" : 29.11,  "a" : { "B" : 0.6502 }, "b" : 0.8553 },
    9 :   { "t" : 41.20,  "a" : { "B" : 0.595 },  "b" : 0.8757 },
    10 :  { "t" : 55.19,  "a" : { "B" : 0.5545 }, "b" : 0.8903 },
    11 :  { "t" : 70.69,  "a" : { "B" : 0.5333 }, "b" : 0.8997 },
    12 :  { "t" : 90.34,  "a" : { "B" : 0.5189 }, "b" : 0.9073 },
    13 :  { "t" : 115.29, "a" : { "B" : 0.5181 }, "b" : 0.9122 },
    14 :  { "t" : 147.42, "a" : { "B" : 0.5176 }, "b" : 0.9171 },
    15 :  { "t" : 188.24, "a" : { "B" : 0.5172 }, "b" : 0.9217 },
    16 :  { "t" : 240.03, "a" : { "B" : 0.5119 }, "b" : 0.9267 }
}

# From Deco for Divers
# Do not use:
#  need to convert to bar and delta-M to use,
#  M0 is in msw and M calculated as Delta M * Depth + M0,
#  Depth is unknown
#
WORKMAN = {
    1 : { "t" : 5.0, "M0" : 31.5, "M" : 1.8 },
    2 : { "t" : 10.0, "M0" : 26.8, "M" : 1.6 },
    3 : { "t" : 20.0, "M0" : 21.9, "M" : 1.5 },
    4 : { "t" : 40.0, "M0" : 17.0, "M" : 1.4 },
    5 : { "t" : 80.0, "M0" : 16.4, "M" : 1.3 },
    6 : { "t" : 120.0, "M0" : 15.8, "M" : 1.2 },
    7 : { "t" : 160.0, "M0" : 15.5, "M" : 1.15 },
    8 : { "t" : 200.0, "M0" : 15.5, "M" : 1.1 },
    9 : { "t" : 240.0, "M0" : 15.2, "M" : 1.1 }
}

# Also from Deco for Divers
#
# Since DSAT's primary concern is no-stop diving, it only uses M0
#  -- there is no Delta M i.e. Delta M = 1
# values are in msw
#
# Conveert to Buhlmann with
#  m_w2b( M0 = NNN / 10, dM = 1, P = 1 )
# and run ZHL with DSAT compartments and M-values
#
DSAT = {
    1 : { "t" : 5.0, "M0" : 30.42 },
    2 : { "t" : 10.0, "M0" : 25.37 },
    3 : { "t" : 20.0, "M0" : 20.54 },
    4 : { "t" : 30.0, "M0" : 18.34 },
    5 : { "t" : 40.0, "M0" : 17.11 },
    6 : { "t" : 60.0, "M0" : 15.79 },
    7 : { "t" : 80.0, "M0" : 15.11 },
    8 : { "t" : 100.0, "M0" : 14.69 },
    9 : { "t" : 120.0, "M0" : 14.41 },
    10 : { "t" : 160.0, "M0" : 14.06 },
    11 : { "t" : 200.0, "M0" : 13.84 },
    12 : { "t" : 240.0, "M0" : 13.69 },
    13 : { "t" : 360.0, "M0" : 13.45 },
    14 : { "t" : 480.0, "M0" : 13.33 }
}

# return alveolar inert gas pressure
# with P amb = 1 bar, fraction of inert gas = 0.79, and RQ = 0.9
# this should return 0.79 - 0.0567 = 0.7451 or 0.7452 dep. on where you round it
#
def palv( Pamb = 1, Q = 0.79, RQ = 0.9 ) :
    assert float( RQ ) != 0.0
    vw = float( Pamb ) - 0.0627 + (1.0 - float( RQ )) / float( RQ ) * 0.0534
    return round( vw * float( Q ), 4 )

# return k: constant for tissue compartment (min^-1)
# Th : tissue compartment half-time in minutes
# for 5-minute compartment it's 0.8452
#
def kay( Th = 5 ) :
    assert float( Th ) > 0.0
    return round( math.log( 2 ) / float( Th ), 4 )

# return rate of pressure change in bar/min
# d0 : start pressure, bar
# dt : end pressure, bar
# t : time, min
# Q : fraction of inert gas (same Q as in palv()
#
def arr( d0 = 1.0, dt = 1.0, t = 1, Q = 0.79 ) :
    assert float( t ) > 0.0
    dP = (float( dt ) - float( d0 )) / float( t )
    rc = dP * float( Q )
    return round( rc, 4 )

# Schreiner equation
# Palv + R * (t - 1/k) - (Palv - Pi - R/k) * e^(-k * t)
#
# returns pressure in tissue compartment after time t at depth Pa & dP
#
# Pi: initial pressure of inert gas in tissue (bar)
# Palv: initial pressure of inert gas in the lungs (bar, output of palv())
# t: time (minutes)\n",
# R: rate of pressure change (output of arr()),
# k: gas decay constant (output of kay()
#
# (Intermediate variables b/c I was playing with rounding)
#

def schreiner( Pi = 0.7451, Palv = 0.7451, t = 1, R = 0, k = 0.1386, verbose = False ) :

    assert float( k ) != 0.0
    x1 = float( R ) * (float( t ) - 1.0 / float( k ))
    x2 = float( Palv ) - float( Pi ) - float( R ) / float( k )
    x3 = math.e ** (float( -k ) * float( t ))
    rc = round( float( Palv ) + x1 - x2 * x3, 4 )
    if verbose : sys.stdout.write( "x1: %f, x2: %f, x3: %f, rc: %f\n" % (x1, x2, x3, rc,) )
    return round( rc, 4 )

# M-value: workman to buhlmann
# P is ambient pressure in bar
# returns pair ( a, b )
#
# TODO: add GF
#
def m_w2b( M0 = 2.9624, dM = 1.7928, P = 1 ) :
    assert float( dM ) >= 1.0
    a = float( M0 ) - float( dM ) * float( P )
    b = 1.0 / float( dM )
    return (round( a, 4 ), round( b, 4 ))

# M-value: buhlmann to workman
# returns pair ( M0, dM )
#
def m_b2w( a = 1.1696, b = 0.5578, P = 1 ) :
    assert float( b ) > 0.0
    M0 = float( a ) + float( P ) / float( b )
    dM = 1.0 / float( b )
    return (round( M0, 4 ), round( dM, 4 ))

# no-stop time by Schreiner
#
# Palv: initial pressure of inert gas in the lungs (bar, output of palv())
# t: time (minutes)\n",
# R: rate of pressure change (output of arr()),
# k: gas decay constant (output of kay()
#  -- same as schreiner()
# M0: surfacing M-value as per Workman
#
def ndl( Palv = 0.7451, M0 = 2.9624, t = 0, R = 0, k = 0.1386, verbose = False ) :

# (M0 - Palv - R * (t - 1/k)) * math.e ** (k * t) + Palv - R / k
    assert float( k ) != 0.0
    x1 = float( M0 ) - float( Palv ) - float( R ) * (float( t ) - 1.0 / float( k ))
    x2 = math.e ** (float( k ) * float( t ))
    rc = x1 * x2 + float( Palv ) - float( R ) / float( k )
    if verbose : sys.stdout.write( "x1: %f, x2: %f, rc: %f\n" % (x1, x2, rc,) )
    return round( rc, 4 )

# Buhlman formula with GF and Helium
# returns safe ascent ceiling
#
# Pn is N2 pressure in tissue compartment
# Phe is He pressure in tissue compartment
# an is N2 a coefficient
# bn is N2 b coefficient
# ahe is He a coefficient
# bhe is He b coefficient
# gf is current gradient factor
#
def buhlmann( Pn, an, bn, Phe = 0, ahe = 0, bhe = 0, gf = 1 ) :

    P = float( Pn ) + float( Phe )
    assert float( P ) != 0.0
    a = (float( an ) * float( Pn ) + float( ahe ) * float( Phe )) / P
    b = (float( bn ) * float( Pn ) + float( bhe ) * float( Phe )) / P
    num = float( P ) - float( a ) * float( gf )
    den = float( gf ) / float( b ) + 1.0 - float( gf )
    assert den != 0.0
    rc = num / den
    return round( rc, 4 )

# eof
#

```

```python

```


# Textual entailment classifier using an MLP plus attention

In textual entailment,
the input is 2 sentences (premise and hypothesis), and the output
is a label, specifying if P entails H, P contradicts H, or neither.
(This is also called "natural language inference".)
We use attention to align hypothesis to premise and vice versa,
then compare the aligned words to estimate similarity between the sentences, and pass the weighted similarities to an MLP.

Based on sec 15.5 of http://d2l.ai/chapter_natural-language-processing-applications/natural-language-inference-attention.html

```python
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(seed=1)
import math
import collections
import os

import torch
from torch import nn
from torch.nn import functional as F

!mkdir figures # for saving plots

!wget https://raw.githubusercontent.com/d2l-ai/d2l-en/master/d2l/torch.py -q -O d2l.py
import d2l
```

    mkdir: cannot create directory ‘figures’: File exists

# Data

We use SNLI (Stanford Natural Language Inference) dataset described in sec 15.4 of http://d2l.ai/chapter_natural-language-processing-applications/natural-language-inference-and-dataset.html.

```python

import re

d2l.DATA_HUB['SNLI'] = ('https://nlp.stanford.edu/projects/snli/snli_1.0.zip',
                        '9fcde07509c7e87ec61c640c1b2753d9041758e4')

data_dir = d2l.download_extract('SNLI')
```

    Downloading ../data/snli_1.0.zip from https://nlp.stanford.edu/projects/snli/snli_1.0.zip...

```python
def read_snli(data_dir, is_train):
    """Read the SNLI dataset into premises, hypotheses, and labels."""
    def extract_text(s):
        # Remove information that will not be used by us
        s = re.sub('\\(', '', s)
        s = re.sub('\\)', '', s)
        # Substitute two or more consecutive whitespace with space
        s = re.sub('\\s{2,}', ' ', s)
        return s.strip()

    label_set = {'entailment': 0, 'contradiction': 1, 'neutral': 2}
    file_name = os.path.join(
        data_dir, 'snli_1.0_train.txt' if is_train else 'snli_1.0_test.txt')
    with open(file_name, 'r') as f:
        rows = [row.split('\t') for row in f.readlines()[1:]]
    premises = [extract_text(row[1]) for row in rows if row[0] in label_set]
    hypotheses = [extract_text(row[2]) for row in rows if row[0] in label_set]
    labels = [label_set[row[0]] for row in rows if row[0] in label_set]
    return premises, hypotheses, labels
```

Show first 3 training examples and their labels (“0”, “1”, and “2” correspond to “entailment”, “contradiction”, and “neutral”, respectively ).

```python
train_data = read_snli(data_dir, is_train=True)
for x0, x1, y in zip(train_data[0][:3], train_data[1][:3], train_data[2][:3]):
    print('premise:', x0)
    print('hypothesis:', x1)
    print('label:', y)
```

    premise: A person on a horse jumps over a broken down airplane .
    hypothesis: A person is training his horse for a competition .
    label: 2
    premise: A person on a horse jumps over a broken down airplane .
    hypothesis: A person is at a diner , ordering an omelette .
    label: 1
    premise: A person on a horse jumps over a broken down airplane .
    hypothesis: A person is outdoors , on a horse .
    label: 0

```python
test_data = read_snli(data_dir, is_train=False)
for data in [train_data, test_data]:
    print([[row for row in data[2]].count(i) for i in range(3)])
```

    [183416, 183187, 182764]
    [3368, 3237, 3219]

```python
class SNLIDataset(torch.utils.data.Dataset):
    """A customized dataset to load the SNLI dataset."""
    def __init__(self, dataset, num_steps, vocab=None):
        self.num_steps = num_steps
        all_premise_tokens = d2l.tokenize(dataset[0])
        all_hypothesis_tokens = d2l.tokenize(dataset[1])
        if vocab is None:
            self.vocab = d2l.Vocab(all_premise_tokens + all_hypothesis_tokens,
                                   min_freq=5, reserved_tokens=['<pad>'])
        else:
            self.vocab = vocab
        self.premises = self._pad(all_premise_tokens)
        self.hypotheses = self._pad(all_hypothesis_tokens)
        self.labels = torch.tensor(dataset[2])
        print('read ' + str(len(self.premises)) + ' examples')

    def _pad(self, lines):
        return torch.tensor([
            d2l.truncate_pad(self.vocab[line], self.num_steps,
                             self.vocab['<pad>']) for line in lines])

    def __getitem__(self, idx):
        return (self.premises[idx], self.hypotheses[idx]), self.labels[idx]

    def __len__(self):
        return len(self.premises)
```

```python
def load_data_snli(batch_size, num_steps=50):
    """Download the SNLI dataset and return data iterators and vocabulary."""
    num_workers = d2l.get_dataloader_workers()
    data_dir = d2l.download_extract('SNLI')
    train_data = read_snli(data_dir, True)
    test_data = read_snli(data_dir, False)
    train_set = SNLIDataset(train_data, num_steps)
    test_set = SNLIDataset(test_data, num_steps, train_set.vocab)
    train_iter = torch.utils.data.DataLoader(train_set, batch_size,
                                             shuffle=True,
                                             num_workers=num_workers)
    test_iter = torch.utils.data.DataLoader(test_set, batch_size,
                                            shuffle=False,
                                            num_workers=num_workers)
    return train_iter, test_iter, train_set.vocab
```

```python
train_iter, test_iter, vocab = load_data_snli(128, 50)
len(vocab)
```

    read 549367 examples
    read 9824 examples

    /usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
      cpuset_checked))

    18678

```python
for X, Y in train_iter:
    print(X[0].shape)
    print(X[1].shape)
    print(Y.shape)
    break
```

    /usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
      cpuset_checked))

    torch.Size([128, 50])
    torch.Size([128, 50])
    torch.Size([128])

# Model

The model is described in the book. Below we just give the code.

## Attending

We define attention weights
$$
e_{ij} = f(a_i)^T f(b_j)
$$
where $a_i \in R^E$ is the embedding of the $i$'th token from the premise,
$b_j \in R^E$ is the embedding of the $j$'th token from the hypothesis,
and $f: R^E \rightarrow R^H$ is an MLP that maps from the embedding space to another hidden space.

```python
def mlp(num_inputs, num_hiddens, flatten):
    net = []
    net.append(nn.Dropout(0.2))
    net.append(nn.Linear(num_inputs, num_hiddens))
    net.append(nn.ReLU())
    if flatten:
        net.append(nn.Flatten(start_dim=1))
    net.append(nn.Dropout(0.2))
    net.append(nn.Linear(num_hiddens, num_hiddens))
    net.append(nn.ReLU())
    if flatten:
        net.append(nn.Flatten(start_dim=1))
    return nn.Sequential(*net)
```

The $i$'th word in A computes a weighted average of "relevant" words in B, and vice versa, as follows:
$$
\begin{align}
\beta_i &= \sum_{j=1}^n
\frac{\exp(e_{ij})}{\sum_{k=1}^n \exp(e_{ik})}
  b_j
  \\
\alpha_j &= \sum_{i=1}^m \frac{\exp(e_{ij})}{\sum_{k=1}^m \exp(e_{kj})}
  a_i
  \end{align}
  $$

```python
class Attend(nn.Module):
    def __init__(self, num_inputs, num_hiddens, **kwargs):
        super(Attend, self).__init__(**kwargs)
        self.f = mlp(num_inputs, num_hiddens, flatten=False)

    def forward(self, A, B):
        # Shape of `A`/`B`: (`batch_size`, no. of words in sequence A/B,
        # `embed_size`)
        # Shape of `f_A`/`f_B`: (`batch_size`, no. of words in sequence A/B,
        # `num_hiddens`)
        f_A = self.f(A)
        f_B = self.f(B)
        # Shape of `e`: (`batch_size`, no. of words in sequence A,
        # no. of words in sequence B)
        e = torch.bmm(f_A, f_B.permute(0, 2, 1))
        # Shape of `beta`: (`batch_size`, no. of words in sequence A,
        # `embed_size`), where sequence B is softly aligned with each word
        # (axis 1 of `beta`) in sequence A
        beta = torch.bmm(F.softmax(e, dim=-1), B)
        # Shape of `alpha`: (`batch_size`, no. of words in sequence B,
        # `embed_size`), where sequence A is softly aligned with each word
        # (axis 1 of `alpha`) in sequence B
        alpha = torch.bmm(F.softmax(e.permute(0, 2, 1), dim=-1), A)
        return beta, alpha
```

## Comparing

We concatenate word $i$ in A, $a_i$, with its "soft counterpart" in B, $\beta_i$, and vice versa, and then pass this through another MLP $g$
to get a "comparison vector" for each input location.
$$
\begin{align}
  v_{A,i} &= g([a_i, \beta_i]), \; i=1,\ldots, m \\
  v_{B,j} &= g([b_j, \alpha_j]), \; j=1,\ldots, n
\end{align}
$$

```python
class Compare(nn.Module):
    def __init__(self, num_inputs, num_hiddens, **kwargs):
        super(Compare, self).__init__(**kwargs)
        self.g = mlp(num_inputs, num_hiddens, flatten=False)

    def forward(self, A, B, beta, alpha):
        V_A = self.g(torch.cat([A, beta], dim=2))
        V_B = self.g(torch.cat([B, alpha], dim=2))
        return V_A, V_B
```

## Aggregation

We sum-pool the "comparison vectors" for each input sentence, and then pass the pair of poolings to yet another MLP $h$ to generate the final classification.

$$
\begin{align}
  v_A &= \sum_{i=1}^m v_{A,i} \\
  v_B &= \sum_{j=1}^n v_{B,j} \\
  \hat{y} &= h([v_A, v_B])
\end{align}
$$

```python
class Aggregate(nn.Module):
    def __init__(self, num_inputs, num_hiddens, num_outputs, **kwargs):
        super(Aggregate, self).__init__(**kwargs)
        self.h = mlp(num_inputs, num_hiddens, flatten=True)
        self.linear = nn.Linear(num_hiddens, num_outputs)

    def forward(self, V_A, V_B):
        # Sum up both sets of comparison vectors
        V_A = V_A.sum(dim=1)
        V_B = V_B.sum(dim=1)
        # Feed the concatenation of both summarization results into an MLP
        Y_hat = self.linear(self.h(torch.cat([V_A, V_B], dim=1)))
        return Y_hat
```

## Putting it altogether

We use a pre-trained embedding of size E=100.
The $f$ (attend) function maps from $E=100$  to $H=200$ hiddens.
The $g$ (compare) function maps $2E=200$ to $H=200$.
The $h$ (aggregate) function maps $2H=400$ to 3 outputs.

```python
class DecomposableAttention(nn.Module):
    def __init__(self, vocab, embed_size, num_hiddens, num_inputs_attend=100,
                 num_inputs_compare=200, num_inputs_agg=400, **kwargs):
        super(DecomposableAttention, self).__init__(**kwargs)
        self.embedding = nn.Embedding(len(vocab), embed_size)
        self.attend = Attend(num_inputs_attend, num_hiddens)
        self.compare = Compare(num_inputs_compare, num_hiddens)
        # There are 3 possible outputs: entailment, contradiction, and neutral
        self.aggregate = Aggregate(num_inputs_agg, num_hiddens, num_outputs=3)

    def forward(self, X):
        premises, hypotheses = X
        A = self.embedding(premises)
        B = self.embedding(hypotheses)
        beta, alpha = self.attend(A, B)
        V_A, V_B = self.compare(A, B, beta, alpha)
        Y_hat = self.aggregate(V_A, V_B)
        return Y_hat
```

```python
embed_size, num_hiddens, devices = 100, 200, d2l.try_all_gpus()
net = DecomposableAttention(vocab, embed_size, num_hiddens)
# get pre-trained GloVE embeddings of size 100
glove_embedding = d2l.TokenEmbedding('glove.6b.100d')
embeds = glove_embedding[vocab.idx_to_token]
net.embedding.weight.data.copy_(embeds);
```

    Downloading ../data/glove.6B.100d.zip from http://d2l-data.s3-accelerate.amazonaws.com/glove.6B.100d.zip...

# Training

```python
lr, num_epochs = 0.001, 4
trainer = torch.optim.Adam(net.parameters(), lr=lr)
loss = nn.CrossEntropyLoss(reduction="none")
d2l.train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices)
```

    loss 0.497, train acc 0.804, test acc 0.824
    14685.9 examples/sec on [device(type='cuda', index=0)]

# Testing

```python
def predict_snli(net, vocab, premise, hypothesis):
    net.eval()
    premise = torch.tensor(vocab[premise], device=d2l.try_gpu())
    hypothesis = torch.tensor(vocab[hypothesis], device=d2l.try_gpu())
    label = torch.argmax(
        net([premise.reshape((1, -1)),
             hypothesis.reshape((1, -1))]), dim=1)
    return 'entailment' if label == 0 else 'contradiction' if label == 1 \
            else 'neutral'
```

```python
predict_snli(net, vocab, ['he', 'is', 'good', '.'], ['he', 'is', 'bad', '.'])
```

    'contradiction'

```python
predict_snli(net, vocab, ['he', 'is', 'very', 'naughty', '.'], ['he', 'is', 'bad', '.'])
```

    'neutral'

```python
predict_snli(net, vocab, ['he', 'is', 'awful', '.'], ['he', 'is', 'bad', '.'])
```

    'entailment'

```python
predict_snli(net, vocab, ['he', 'is', 'handsome', '.'], ['he', 'is', 'bad', '.'])
```

    'contradiction'

## Examples from training set

```python
predict_snli(net, vocab,
             ['a', 'person', 'on', 'a', 'horse', 'jumps', 'over', 'a', 'log' '.'],
              ['a', 'person', 'is', 'outdoors', 'on', 'a', 'horse', '.'])
```

    'entailment'

```python
predict_snli(net, vocab,
             ['a', 'person', 'on', 'a', 'horse', 'jumps', 'over', 'a', 'log' '.'],
              ['a', 'person', 'is', 'at', 'a', 'diner', 'ordering', 'an', 'omelette', '.'])
```

    'contradiction'

```python
predict_snli(net, vocab,
             ['a', 'person', 'on', 'a', 'horse', 'jumps', 'over', 'a', 'log' '.'],
              ['a', 'person', 'is', 'training', 'a', 'horse', 'for', 'a', 'competition', '.'])
```

    'neutral'


# The Specific Factors or Ricardo-Viner Model

### Background

The SF model is a workhorse model in trade, growth, political economy and development.  We will see variants of the model used to describe rural to urban migration, the Lewis model and other dual sector models of sectoral misallocation, models such as the Harris-Todaro model that explain migration and the urban informal sector. The specific factors model predicts that, in the absence of political redistribution mechanisms, specific factors in declining sectors will organize strong opposition to policies that might otherwise raise growth. The initial distribution of factor endowments may therefore make a big difference in terms of what types of political coalitions mobilize for and against different policies.  This is the basic driving force in Moav-Galor's (2006) growth model on why some regions made public investments in human capital which sped the transition from agriculture to manufacturing and enhanced growth, whereas similar policies were delayed in other regions where political/economic resistance was stronger, for instance where landlords had stronger voice in political decisions.

These are just a few of the applications.  The model is relatively easy to analyze -- it can be described compactly in terms of diagrams and yet is very rich in predictions.

The Specific Factors (SF) or Ricardo-Viner model is a close relative of the Hecksher-Ohlin-Samuelson (HOS) neoclassical trade model. The 2x2 HOS model assumes production in each of two sectors takes place by firms using constant returns to scale technologies with capital and labor as inputs and that both capital and labor are mobile across sectors. In the SF model only labor is inter-sectorally mobile and the fixed amounts of capital become 'specific' to the sector they are trapped within.

In effect the SF model therefore consists of three factors of production: mobile labor and two types of capital, one specific to each sector.  Let's label the two sectors are Agriculture and Manufacturing. In agriculture competitive firms bid to hire land and labor. In manufacturing competitive firms bid to hire capital and labor. Each factor of production will be competitively priced in equilibrium but only labor is priced on a national labor market.

The SF model is often described as a short-run version of the HOS model.  For example suppose we start with a HOS model equilibrium where labor wage and rental rate of capital have equalized across sectors (which also implies marginal products are equalized across sectors -- no productivity differences across sectors).  Now suppose that the relative price of manufacturing products suddenly rises (due to a change of world price, or government trade protection or other policies that favor manufacturing).  The higher relative product price should lead firms in the booming manufacturing sector to demand both more capital and labor. Correspondingly, demand for land and labor decline in the agricultural sector.  In the short run however only labor can be move from agriculture to manufacturing.  Agricultural workers can become factory workers but agricultural capital (say 'land' or 'tractors') cannot be easily converted to manufacturing capital (say 'weaving machines'). So labor moves from manufacturing to the agriculture, lowering the capital labor ratio in manufacturing and raising the land to labor ratio in agriculture.  The model thus predicts a surge in the real return to capital in the expanding sector and a real decline in the real return to capital in agriculture (land).  Hence the measured average and marginal product of capital will now diverge across sectors.  What happens to the real wage is more ambiguous: it rises measured in terms of purchasing power over agricultural goods but falls in terms of purchasing power over manufacturing goods.   Whether workers are better off or worse off following this price/policy change therefore comes down to how important agricultural and manufacturing goods are in their consumption basket. This result is labeled the neo-classical ambiguity.

Over the longer-term weaving machines cannot be transformed into tractors but over time new capital accumulation will build tractors and old weaving machines can be sold overseas or as scrap metal. Hence over time more capital will arrive into the manufacturing sector and leave the agricultural sector, whcih in turn will lead to even more movement of labor to manufacturing.  If this process continues capital has, in effect become mobile over time, and we end up getting closer to the predictions of the HOS model.

### Technology and Endowments
There are two sectors Agriculture and Manufacturing.  Production in each sector takes place with a linear homogenous (constant returns to scale) production functions. Agricultural production requires land which is non-mobile or specific to the sector and mobile labor.

$$Q_a = F(\bar T, L_a)$$

Manufacturing production requires specific capital $K$ and mobile labor.

$$Q_m = G(\bar K, L_m)$$

The quantity of land in the agricultural sector and the quantity of capital in the manufacturing sector are in fixed supply  during the period of analysis.  That means that firms within the agricultural (manufacturing) sector may compete with one another for the limited supply of the factor, but no new land (capital) can be supplied in the short-run period of analysis.  This of course means that the price of the factor will rise (or fall) quickly in response to swings in factor demand compared to the wage of labor whose supply is more elastic.

The market for mobile labor is competitive and the market clears at a wage where the sum of labor demands from each sector equals total labor supply. While the total labor supply in the economy is inelastic, the supply of labor to each sector will be elastic, since a rise in the wage in one sector will attract workers from the other sector.

$$L_a + L_m = \bar{L}$$

Notice that we can invert the two production function to get minimum labor requirement functions $L_a(Q_a)$ and $L_m(Q_m)$ which tell us the minimum amount of labor $L_i$ required in sector $i$ to produce quantity $Q_i$.  If we take these expressions and substitute them into the labor resource constraint we get an expression for the **production possibility frontier (PPF)** which summarizes the tradeoffs between sectors.

### Assumptions and parameters for visualizations

Let's get concrete and assume each sector employs a CRS Cobb-Douglas production function:

$$F(\bar T, L_a)=\bar T^{1-\alpha} \cdot  L_a^\alpha$$

$$G(\bar K, L_m)=\bar K^{1-\beta} \cdot  L_m^\beta$$

If $\alpha = \beta = \frac{1}{2}$ and $\bar T= \bar K = 100$.  Then

$$Q_a =  \sqrt{\bar T} \sqrt{L_a}  $$

$$Q_m = \sqrt{\bar K} \sqrt{L_m}$$

Substituting these into the labor resource constraint yields:

$$\frac{Q_a^2}{\bar T}+\frac{Q_m^2}{\bar K} = \bar L$$

or

$$Q_m = \sqrt{\bar K \bar L - \frac{\bar K Q_a^2}{\bar T} }   $$

If we make the further assumption that $\bar K = \bar L  = 100$ and $\bar L = 400$ then the PPF would look like this:

**NOTE:** If you are running this as a live jupyter notebook please first go to the [code section](#codesection) below and execute all the code cells there. Then return and run the code cells that follow sequentially.

```python
ppf(Tbar=100, Kbar=100, Lbar=400)
```

## Labor market equilibrium

Profit maximizing firms in each sector will hire labor up to the point where the marginal value product of labor (MVPL) equals the market wage. Since labor is mobile across sectors in equilibrium workers must be paid the same nominal wage $w$ in either sector:

$$ P_a \cdot MPL_a(\bar T, L_a) = w = P_m \cdot MPL_m(\bar K, L_m) $$

It will be useful to express the wage in real terms.  Divide each expression above by $P_m$ to get

$$ p \cdot MPL_a = \frac{w}{p_m} = MPL_m $$

$$\text{    where   } p = \frac{P_a}{P_m}$$

In the plots below we will place the real wage measured in terms of manufactured goods on the vertical axis of the labor demand and supply diagrams.

### Labor allocation across sectors

In a competitive market firms take the market prices of goods $p$ and wages $w$ as given. Since in equilibrium the labor market clears we can write $L_m = \bar L - L_a$ and substitute into the equilibrium condition above to get:

$$p \cdot MPL_a(L_A) = w = MPL_m(\bar L-L_A) $$

The left hand side $p \cdot MPL_a(L_A)=w$ can be solved to give us demand for labor in the agricultural sector as a function of the wage $L_a(w/p)$ and is plotted below.  The right hand side $w = MPL_m(\bar L-L_a)$ gives us demand for labor in the manufacturing sector $L_m$ as a function of the real wage but this in turn also gives us the supply of labor to the agricultural sector $L_a^s = \bar L - L_m(w/p)$ since firms in the agricultural sector can only attract workers to their sector by paying those workers just a bit above what they would be paid for the jobs they have to leave in the manufacturing sector.

The diagram below can therefore be interpreted as showing labor demand and supply in the agricultural sector and their intersection gives us the equilibrium real wage (measured on the vertical in terms of purchasing power over manufactured goods).

```python
sfmplot(p=1)
```

We can solve for the equilibrium labor allocation $L_a$ as a function of the relative price $p$, and this in turn also gives us the equilibrium real wage. When $\alpha=\beta = \frac{1}{2}$ the equilibrium condition $p \cdot MPL_a(L_a) = MPL_m(\bar L-L_a)$ becomes:

$$p \cdot \frac{\sqrt{\bar T}}{\sqrt{L_a}} = \frac{\sqrt{\bar K}}{\sqrt{\bar L - L_a}}$$

which we solve to find:

$$L_a^e = \frac{p^2 \bar T \bar L}{\bar K - p^2 \bar T}$$

For cases where $\alpha \ne \beta$ we find the optimal $L_a$ that solves this equilibrium condition numerically.

## Autarky prices

Thus far we have a pretty complete model of how production allocations and real wages would be determined in a small open economy where producers face world price ratio $p$.  we explore comparative statics in this economy in more detail below.

If however the economy is inititally in autarky or closed to the world then we must also consider the role of domestic consumer preferences in the determination of domestic equilibrium product prices.

With Cobb-Douglas preferences $u(x,y) = \gamma \ln(x) + (1-\gamma) \ln(y)$ consumers demands can be written as a function of relative price $p=\frac{P_a}{P_m}$. Setting $P_m=1$ to make manufacturing the numeraire good, this can be written:

$$\begin{align}
C_a(p) =& \gamma \cdot \frac{I(p)}{p} \\
C_m(p) =& (1-\gamma) \cdot I(p)
\end{align}$$

Income $I$ in the expressions is given by the value of national production or GDP at these prices. Measured in manufactured goods:

$$I(p) = p \cdot F(\bar T, L_a(p)) + G(\bar K, \bar L - L_a(p)$$

By Walras' law we only need to find the relative price at which output equals demand in one of the two product markets so in the the code below we solve for equilibrium domestic prices from the condition $Q_a(p)=C_a(p)$.

For parameters $\alpha=\beta=\gamma = \frac{1}{2}$, $\bar T = \bar K =100$ and $\bar L=400$ the domestic equilibrium prices are unitary:

```python
p_autarky(Lbar, Tbar, Kbar)
```

    1.0

And this in turn leads to an equilibrium autarky allocation with $L_a = L_m = 200$ and a real wage $\frac{w}{p}=0.35$.

```python
eqn(p_autarky())
```

    (200.0, 0.35355339059327373)

The plot below shows the autarky production and consumption point (marked by an 'X') as well as the new production point (marked by a circle) and consumption point (marked by a square) if the country opened to trade with a world relative price $p=\frac{7}{4}$.

Opening to trade leads the country to expand agricultural production by re-allocating labor from manufacturing to agriculture. We confirm the expected **neoclassical ambiguity** result which is that an increase in the relative price of agricultural goods will lead to an *increase* in the real wage measured in terms of manufactured goods  $\frac{w}{p_m}$ ($=w$ since we have $P_m=1$) but a *decrease* in real wage measured in terms of agricultural goods or $\frac{w}{P_a} = \frac{w}{p}$ in our notation (since $\frac{w}{P_a} = \frac{w}{p_m} \cdot \frac{P_m}{P_a}$).

Diagrammatically

$$p \uparrow \rightarrow \frac{w}{p_m} \uparrow , \frac{w}{p_a} \downarrow$$

```python
pw = 7/4
Lao, wo = eqn(p=pw)
Lao, wo, wo/pw
```

    (301.53846153846149, 0.50389110926865943, 0.28793777672494825)

```python
sfmtrade(p=7/4)
```

## Effects of a price change on the income distribution

More explanations to be placed here...

```python
sfmplot2(2)
```

## Code Section

Make sure you run the cells below FIRST.  Then run the cells above

## Python simulation and plots

```python
import numpy as np
from scipy.optimize import fsolve
np.seterr(divide='ignore', invalid='ignore')
import matplotlib.pyplot as plt
from ipywidgets import interact, fixed
import seaborn
%matplotlib inline
```

```python
plt.style.use('seaborn-colorblind')
plt.rcParams["figure.figsize"] = [7,7]
plt.rcParams["axes.spines.right"] = True
plt.rcParams["axes.spines.top"] = False
plt.rcParams["font.size"] = 18
plt.rcParams['figure.figsize'] = (10, 6)
plt.rcParams['axes.grid']=True
```

```python
Tbar = 100       # Fixed specific land in ag.
Kbar = 100       # Fixed specific capital in manuf
Lbar = 400       # Total number of mobile workers
LbarMax = 400    # Lbar will be on slider, max value.

p    = 1.00      # initial rel price of ag goods, p = Pa/Pm
alpha, beta = 0.5, 0.5  # labor share in ag, manuf
```

For the plots we want to plot over $L_a$ and $L_m = \bar L -l_a$:

```python
La = np.linspace(1, LbarMax-1,LbarMax)
Lm = Lbar - La
```

The production functions in each sector:

```python
def F(La, Tbar = Tbar):
    return (Tbar**(1-alpha) * La**alpha)

def G(Lm, Kbar =Kbar):
    return (Kbar**(1-beta) * Lm**beta)
```

```python
def MPLa(La):
    return alpha*Tbar**(1-alpha) * La**(alpha-1)

def MPLm(Lm):
    return beta*Kbar**(1-beta) * Lm**(beta-1)

def MPT(La, Tbar=Tbar):
    return (1-alpha)*Tbar**(-alpha) * La**alpha

def MPK(Lm, Kbar=Kbar):
    return (1-beta)*Kbar**(-beta) * Lm**beta
```

We have enough to plot a production possibility Frontier (and how it varies with factor supplies):

```python
def ppf(Tbar=Tbar, Kbar=Kbar, Lbar=Lbar):
    Qa = F(La, Tbar) * (La<Lbar)
    Qm = G(Lm, Kbar)
    plt.title('Production Possibility Frontier')
    plt.xlabel(r'$Q_a$')
    plt.ylabel(r'$Q_m$')
    plt.plot(Qa, Qm)
    plt.gca().set_aspect('equal');
```

```python
ppf(Tbar=100, Kbar=100, Lbar=400)
```

Labor demand in each sector as a function of $p=\frac{P_a}{P_m}$:

```python
LDa = p * MPLa(La) *(La<Lbar)         # for Cobb-Douglas MPL can be written this way
LDm = MPLm(Lbar-La)
```

The following function returns the equilibrium allocation of labor to agriculture and equilibrium nominal wage for any initial relative price of agricultural goods $p=\frac{P_a}{P_m}$

```python
def eqn(p, Lbar=Lbar):
    '''returns numerically found equilibrium labor allocation and wage'''
    def func(La):
        return p*MPLa(La) - MPLm(Lbar-La)
    Laeq = fsolve(func, 1)[0]
    return Laeq, p*MPLa(Laeq)
```

```python
def u(x,y):
    '''Utility function'''
    return x*y

def XD(p, Lbar = Lbar):
    '''Cobb-Douglas demand for goods given world prices (national income computed)'''
    LAe, we = eqn(p)
    # gdp at world prices measured in manuf goods
    gdp = p*F(LAe, Tbar = Tbar) + G(Lbar -LAe, Kbar=Kbar)
    return (1/2)*gdp/p, (1/2)*gdp

def indif(x, ubar):
    return ubar/x
```

```python
def p_autarky(Lbar=Lbar, Tbar=Tbar, Kbar=Kbar):
    '''Find autarky product prices. By Walras' law enough to find price that
    sets excess demand in just one market'''
    def excessdemandA(p):
        LAe, _ = eqn(p)
        QA = F(LAe, Tbar = Tbar)
        CA, CM = XD(p, Lbar=Lbar)
        return QA-CA
    peq = fsolve(excessdemandA, 1)[0]
    return peq

```

```python
p_autarky()
```

    1.0

```python
def sfmtrade(p):
    Ca = np.linspace(0,200,200)
    LAe, we = eqn(p)
    X, Y = F(LAe, Tbar = Tbar), G(Lbar -LAe)
    gdp = p*X + Y
    plt.scatter(F(LAe, Tbar), G(Lbar-LAe, Tbar), label='Trade produce')
    plt.scatter(*XD(p),label='Trade consume', marker='s')
    plt.scatter(*XD(p_autarky()), marker='x', label='Autarky')
    plt.plot([0,gdp/p],[gdp, 0])
    ppf(100)
    ub = u(*XD(p))
    plt.ylim(0,gdp)
    plt.xlim(0,gdp)
    plt.plot(Ca, indif(Ca, ub))
    plt.grid(False)
    plt.legend()
    plt.gca().spines['bottom'].set_position('zero')
    plt.gca().spines['left'].set_position('zero')
```

```python
sfmtrade(p=7/4)
```

Simple example with $\bar L$ at $\bar L^{max}$

```python
La = np.arange(0, LbarMax)   # this is always over the LbarMax range

def sfmplot(p, Lbar=LbarMax, show=True):
    Lm = Lbar - La
    Qa = (Tbar**(1-alpha) * La**alpha) * (La<Lbar)
    Qm = Kbar**(1-beta) * (Lbar - La)**beta
    pMPLa = (p * alpha * Qa/La)*(La<Lbar)         # for Cobb-Douglass MPL can be written this way
    MPLm = beta * Qm/(Lbar-La)
    LA, weq = eqn(p)
    ymax = 1.0
    plt.ylim(0,ymax)
    plt.xlim(0,LbarMax)
    plt.title('Specific Factors Model')
    plt.plot(La, pMPLa, linewidth = 3, label='AG labor demand')
    plt.plot(La, MPLm, linewidth=3, label ='MF labor demand')
    plt.scatter(LA, weq, s=100, color='black')
    plt.axhline(weq, linestyle='dashed')
    plt.axvline(Lbar, linewidth = 3)
    plt.plot([LA, LA],[0, weq], linestyle='dashed')
    plt.xlabel('Labor')
    plt.ylabel('Real wage --' + r'$\frac{w}{p_M}$')
    plt.grid()
    if show:
        plt.legend(loc=(1.04,0.9))
        print("(La, Lm) = ({0:3.0f}, {1:3.0f})   (w/Pm, w/Pa) =({2:3.2f}, {3:3.2f})"
              .format(LA, Lbar-LA, weq, weq/p))
        print("(v/Pa, v/Pm) = ({0:3.1f}, {1:3.1f})   (r/Pa, r/Pm) = ({2:3.1f}, {3:3.1f})"
              .format(MPT(LA), p*MPT(LA), MPK(Lbar-LA)/p, MPK(Lbar-LA)    )   )
        plt.show()

```

If you are wondering about why we skip the plt.show() when show=True, it's because for figures that use the ipywidgets interact function to allow parameters to change with sliders we will need to have a plt.show() to redraw the plot in every function call (otherwise we would end up with a stack of plots rather than a changing plot).

```python
sfmplot(p=2)
```

Suppose that by opening to trade the relative price of agricultural goods $p$ doubles from $p=1$ to $p=2$.  This greatly shifts the demand for agricultural workers:

```python
def sfmplot2(p):
    sfmplot(1, show=False)
    sfmplot(p, show=False)
    plt.grid(False)
    if p == 1:
        plt.title('SF Model');
    else:
        La0, w0 = eqn(1)
        plt.scatter(La0,w0*p, s=100, color='black')  #where wage would rise to without labor movement
        if p>1:
            plt.title(r'$\frac{P_a}{P_m} \uparrow  \rightarrow  \frac{w}{P_m} \uparrow, \frac{w}{P_a} \downarrow $'  );
        elif p<1:
            plt.title(r'$\frac{P_a}{P_m} \downarrow  \rightarrow  \frac{w}{P_m} \downarrow, \frac{w}{P_a} \uparrow $'  );
    plt.show();

```

```python
sfmplot2(2)
```

```python
interact(sfmplot2, p=(0.1, 2,0.1));
```

## Before and after plots

The trick here is to define a new plotting function that first plots a static plot and then allows an interaction.

```python
interact(sfmplot2, p=(0,2,0.2), Lbar=fixed(400), show=fixed(True))
```

    <function __main__.sfmplot2>