# Calculation of the Feasible Joint Reaction Loads

The importance of evaluating the feasible muscle forces is demonstrated in the
context of joint reaction analysis. An accurate estimation of the muscle forces
is essential for the assessment of joint reaction loads. Consequently, the null
space muscle forces can significantly alter the reaction forces without
affecting the movement. The process is separated into four steps:

1. Perform an inverse kinematics (IK), static optimization (SO) and joint
reaction analysis (JRA) using OpenSim to generate the required data for the next
steps.

2. Calculate the feasible muscle forces that satisfy both the action and the
physiological constraints of the muscles.

3. For each distinct muscle force realization perform a joint reaction analysis.

4. Post-process the reaction loads.

```python
# notebook general configuration
%load_ext autoreload
%autoreload 2

import os
import pickle
import numpy as np
import sympy as sp
from IPython.display import display
sp.interactive.printing.init_printing()
from tqdm import tqdm
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (10.0, 6.0)
%matplotlib inline
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

## Subject Data

Please provide the following information for the subject under study:

- **.osim** the scaled OpenSim model (after scaling the model using a static
    pose)

- **.trc** the marker trajectories of the movement (based on OpenSim file
    requirements)

- **.mot** the ground reaction forces (based on the OpenSim file format)

- **.xml** describing how to apply the ground reaction forces (based on OpenSim
  force requirements)

- **.xml** file with the reserve actuator for the SO (based on OpenSim requirements)

- **subject mass** (used for scaling the reaction loads, and is not a
    requirement)

- **heel strike** and **toe off** events (used for annotation of the results,
  and are not so important)

The following calculations are performed for the *Gait10dof18musc* dataset.

```python
# subject data
mass = 72.6  # kg
g = 9.8  # m/s^2
body_weight = mass * g

# time events of toe off and heel strike
heel_strike_right = [0.65, 1.85]
toe_off_right = [0.15, 1.4]
heel_strike_left = [0.0, 1.25]
toe_off_left = [0.8, 2]

# required files and directories
subject_dir = os.getcwd() + '/../dataset/Gait10dof18musc/'
model_file = subject_dir + 'subject01.osim'
trc_file = subject_dir + 'subject01_walk.trc'
grf_file = subject_dir + 'subject01_walk_grf.mot'
grf_xml_file = subject_dir + 'subject01_walk_grf.xml'
reserve_actuators_file = subject_dir + 'reserve_actuators.xml'
results_dir = subject_dir + 'notebook_results/'
feasible_set_dir = results_dir + 'feasible_force_set/'
jra_results_dir = results_dir + 'joint_reaction_analyses/'
figures_dir = results_dir + 'fig/'

if not (os.path.isfile(model_file) and
        os.path.isfile(trc_file) and
        os.path.isfile(grf_file) and
        os.path.isfile(grf_xml_file) and
        os.path.isfile(reserve_actuators_file)):
    raise RuntimeError('required files do not exist')

if not (os.path.isdir(results_dir) and
        os.path.isdir(feasible_set_dir) and
        os.path.isdir(jra_results_dir) and
        os.path.isdir(figures_dir)):
    raise RuntimeError('required folders do not exist')
```

## Step 1: Perform Required OpenSim Analyses

In order to perform the feasible muscle force calculations, the kinematics and
kinetics that satisfy the experimental measured motion and ground reaction
forces must be calculated. Furthermore, OpenSim JRA is performed so as to
compare the feasible reaction loads in step 4. The following functions make use
of the OpenSim IK, SO and RJA analyses.

```python
from opensim_utils import perform_ik, perform_so, perform_jra, plot_sto

# perform OpenSim inverse kinematics
ik_file = perform_ik(model_file, trc_file, results_dir)
plot_sto(ik_file, 4, save=True)

# perform OpenSim static optimization
(so_force_file, so_activation_file) = perform_so(model_file, ik_file, grf_file,
                                                 grf_xml_file, reserve_actuators_file,
                                                 results_dir)
plot_sto(so_force_file, 4, save=True)

# perform OpenSim joint reaction analysis
jra_file = perform_jra(model_file, ik_file, grf_file, grf_xml_file,
                       reserve_actuators_file, so_force_file, results_dir)

# store file names so that they can be loaded without running this section
pickle.dump((ik_file, so_force_file, jra_file), file(results_dir + 'opensim_files.dat', 'w'))
```

```python
# get file names from step 1
(ik_file, so_force_file, jra_file) = pickle.load(file(results_dir + 'opensim_files.dat', 'r'))
```

## Step 2: Calculate the Feasible Muscle Forces

The feasible muscle forces are calculated below. Initially, the moment arm and
maximum muscle force quantities are computed for each instance of the
movement. Then the following inequality is formed assuming a linear muscle model

\begin{equation}\label{equ:linear-muscle-null-space-inequality}
  \begin{gathered}
    f_m = f_{max} \circ a_m = f_m^{\parallel} +
    N_{R} f_{m0},\; 0 \preceq a_m \preceq 1
    \rightarrow \\
    \begin{bmatrix}
      - N_{R} \\
      \hdashline
      N_{R}
    \end{bmatrix}
    f_{m0} \preceq
    \begin{bmatrix}
      f_m^{\parallel} \\
      \hdashline
      f_{max} - f_m^{\parallel}
    \end{bmatrix} \\
    Z f_{m0} \preceq \beta
  \end{gathered}
\end{equation}

where $a_m \in \Re^{m}$ represents a vector of muscle activations, $f_{max} \in
\Re^{m}$ a vector specifying the maximum muscle forces, $\circ$ the Hadamard
(elementwise) product, $f_m^{\parallel}$ the particular muscle force solution
that satisfies the action, $N_{R}$ the moment arm null space and $f_{m0}$ the
null space forces.

```python
# since this section of code may be called independently after step 1
# ensure that cells 2, 5 and 9 are evaluated
from IPython.display import Javascript
Javascript("Jupyter.notebook.execute_cells([2, 5, 9])")
```

```python
from opensim_utils import calculate_muscle_data
from util import null_space, construct_muscle_space_inequality, \
    convex_bounded_vertex_enumeration, readMotionFile, \
    index_containing_substring, write_as_sto_file

# calculate the moment arm and maximum muscle forces assuming a linear muscle
# model (in a future version this can calculate the properties for a nonlinear
# muscle model)
moment_arm, max_force = calculate_muscle_data(model_file, ik_file)

# read SO results
so_header, so_labels, so_data = readMotionFile(so_force_file)
so_data = np.array(so_data)
coordinates = moment_arm[0].shape[0]
muscles = moment_arm[0].shape[1]
time = so_data[:, 0]
entries = time.shape[0]
np.
# collect quantities for computing the feasible muscle forces
NR = []
Z = []
b = []
fm_par = []
for t in tqdm(range(0, entries)):
    # get tau, R, Fmax
    fm = so_data[t, 1:(muscles + 1)]  # time the first column
    RT_temp = moment_arm[t, :, :]
    fmax_temp = max_force[t, :]

    # calculate the reduced rank (independent columns) null space to avoid
    # singularities
    NR_temp = null_space(RT_temp)

    # fm_par = fm is used instead of fm_par = -RBarT * tau because the
    # muscle may not be able to satisfy the action. In OpenSim reserve
    # actuators are used to ensure that Static Optimization can satisfy the
    # action. In this case, we ignore the residual forces and assume that fm
    # is the minimum effort solution. If the model is able to satisfy the
    # action without needing reserve forces then we can use fm_par = -RBarT
    # * tau as obtained form Inverse Dynamics.
    fm_par_temp = fm

    Z_temp, b_temp = construct_muscle_space_inequality(NR_temp,
                                                       fm_par_temp,
                                                       fmax_temp)

    # append results
    NR.append(NR_temp)
    Z.append(Z_temp)
    b.append(b_temp)
    fm_par.append(fm_par_temp)
```

The next step is to sample the inequality $Z f_{m0} \leq \beta$. This is the
bottleneck of the analysis. The *convex_bounded_vertex_enumeration* uses the
lsr method, which is a vertex enumeration algorithm for finding the vertices
of a polytope in $O(v m^3)$.

```python
# calculate the feasible muscle force set by sampling the inequality
f_set = []
for t in tqdm(range(0, entries)):
    try:
        fs = convex_bounded_vertex_enumeration(Z[t], b[t][:, 0], 0, method='lrs')
    except:
        print('inequlity is infeasible thus append previous iteration')
        f_set.append(f_set[-1])
        continue

    temp = []
    for i in range(0, fs.shape[0]):
        temp.append(fm_par[t] + NR[t].dot(fs[i, :]))

    f_set.append(temp)

# serialization f_set -> [time x feasible force set set x muscles] so as to
# avoid recomputing it
pickle.dump(f_set, file(feasible_set_dir + 'f_set.dat', 'w'))
```

Finally, store the feasible muscle force set into multiple .sto files that can
be used by the JRA.

```python
# keep only muscle forces
idx = index_containing_substring(so_labels, 'FX')[0]
labels = so_labels[:idx]

# find largest feasible set
S = len(f_set[0])
for fs in f_set:
    if len(fs) > S:
        S = len(fs)

# export muscle force realizations
for j in tqdm(range(0, S)):
    data_temp = []
    for i in range(0, so_data.shape[0]):
        data_temp.append(f_set[i][j % len(f_set[i])])

    write_as_sto_file(feasible_set_dir + str(j) + '.sto', labels,
                      time, np.array(data_temp))
```

## Step 3: Perform Joint Reaction Analyses on Feasible Muscle Forces

For each distinct muscle force realization computed in the previous step
initiate a JRA so as to calculate the influence of the muscle forces on the
joint reaction loads. Due to memory leaks in OpenSim JRA implementation this
process must be restarted since it can cause problems with the RAM usage. The if
statement is used so as to continue from the last performed analysis.

```python
# since this section of code may be called independently after step 2
# ensure that cells 2, 5 and 9 are evaluated
from IPython.display import Javascript
Javascript("Jupyter.notebook.execute_cells([2, 5, 9])")
```

```python
from opensim_utils import perform_jra

# get all files in the directory
feasible_force_files = os.listdir(feasible_set_dir)

# remove files that are not .sto
feasible_force_files = [e for e in feasible_force_files if '.sto' in e]

# perform joint reaction analyses
previous_iteration = 0
for i, force_file in enumerate(tqdm(feasible_force_files)):
    # change the previous_iteration variable
    if i < previous_iteration:
         continue

    if i > previous_iteration + 200:
        print('please shutdown this notebook and reopen (RAM usage problem)')
        print('set previous_iteration=' + str(previous_iteration + 200))
        break

    perform_jra(model_file, ik_file, grf_file, grf_xml_file,
                reserve_actuators_file, feasible_set_dir + force_file,
                jra_results_dir, prefix=str(i) + '_')
```

## Step 4: Postprocess Joint Reaction Loads

This section collects the joint reaction loads calculated in the previous step.

```python
# since this section of code may be called independently after step 3
# ensure that cells 2, 5 and 9 are evaluated
from IPython.display import Javascript
Javascript("Jupyter.notebook.execute_cells([2, 5, 9])")
```

    <IPython.core.display.Javascript object>

```python
from util import readMotionFile, index_containing_substring

# select the joint of interest
joint = 'hip_r'

# load OpenSim JRA results (step 1)
os_header, os_labels, os_data = readMotionFile(jra_file)
os_data = np.array(os_data)
joint_index = index_containing_substring(os_labels, joint)
assert(joint_index != -1)

# get all files from the JRA batch simulation (step 3)
jra_files = os.listdir(jra_results_dir)

# remove files that are not joint reactions
jra_files = [e for e in jra_files if 'ReactionLoads' in e]

# allocate the necessary space to collect all results
solutions_to_keep = len(jra_files)
simulationData = np.empty([solutions_to_keep,
                           os_data.shape[0],
                           os_data.shape[1]],
                          dtype=float)

# collect reaction loads
for i, f in enumerate(tqdm(jra_files)):
    header, labels, data = readMotionFile(jra_results_dir + f)
    simulationData[i, :, :] = np.array(data)
```

    100%|██████████| 16524/16524 [01:59<00:00, 138.28it/s]

The following section of code compares the feasible and OpenSim reaction loads
for the joint of interest. The reaction loads are normalized using the body
weight of the subject. The heel strike and toe off events are annotated
accordingly.

```python
# select the heel strike and toe off events
if '_l' in joint:
    heel_strike = heel_strike_left
    toe_off = toe_off_left
else:
    heel_strike = heel_strike_right
    toe_off = toe_off_right

# plot data min/max reactions vs OpenSim JRA
joints = 3
fig, ax = plt.subplots(nrows=1, ncols=joints, figsize=(15, 5))
for i in range(0, joints):
    # plot feasible reaction loads
    min_reaction = np.min(simulationData[1:, 1:, joint_index[i]] / body_weight,
                          axis=0)
    max_reaction = np.max(simulationData[1:, 1:, joint_index[i]] / body_weight,
                          axis=0)
    ax[i].fill_between(os_data[1:, 0], min_reaction, max_reaction,
                       color='b', alpha=0.2, label='Feasible Reactions')
    # plot OpenSim reaction loads
    ax[i].plot(os_data[1:, 0], os_data[1:, joint_index[i]] / body_weight,
               '-.r', label='OpenSim JRA')
    # annotate the heel strike and toe off regions
    min_min = np.min(min_reaction)
    max_max = np.max(max_reaction)
    ax[i].vlines(x=heel_strike, ymin=min_min, ymax=max_max,
                 color='c', linestyle='--', label='HS')
    ax[i].vlines(x=toe_off, ymin=min_min, ymax=max_max,
                 color='m', linestyle=':', label='TO')
    # figure settings
    ax[i].set_title(os_labels[joint_index[i]])
    ax[i].set_xlabel('time (s)')
    ax[i].set_ylabel('reaction / body weight')
    if i == joints - 1:
        ax[i].legend()

# export results
fig.tight_layout()
fig.savefig(figures_dir + joint + '.pdf',
            format='pdf', dpi=300)
fig.savefig(figures_dir + joint + '.png',
            format='png', dpi=300)
# fig.show()
```

We observe that the results obtained from OpenSim joint reaction analysis
predict low reaction load levels, since the minimum muscle effort criterion is
used to compute the muscle forces, ignoring muscle co-contraction. On the
contrary, it is possible to calculate the feasible reactions without making any
prior assumption, which can limit the scope and extent of the analysis. Last and
perhaps most importantly, the large range of possible values confirms that
misinterpretation of the results is possible if the null space solutions are
ignored.


```python
%matplotlib inline
```

단어 임베딩: 어휘의 의미를 인코딩하기
===========================================
**번역**: `임성연 <http://github.com/sylim2357>`_

단어 임베딩(word embedding)이란 말뭉치(혹은 코퍼스, corpus) 내 각 단어에 일대일로 대응하는 밀집된 실수 벡터(dense vector)의 집합, 혹은 이 벡터를
구하는 행위를 가리킵니다. 주로 단어를 피처(feature)로 사용하는 자연어 처리 분야에서는 단어를 컴퓨터 친화적인
형태로 바꾸어 주는 작업이 필수적입니다. 컴퓨터가 단어를 바로 이해하기는 상당히 어렵기 때문이죠.
그렇다면, 단어를 어떻게 표현하는 것이 좋을까요? 물론 각 문자에 해당하는 ASCII코드를 사용할 수 있겠지만,
ASCII코드는 이 단어가 *무엇* 인지를 알려줄 뿐, 단어가 어떤 *의미* 를 가지는지는 알려주지 않습니다.
(룰베이스로 어미 등 문법적 특징을 활용하거나 영어의 경우 대문자를 사용할 수 있겠지만 충분하지 않습니다.)
단어를 어떻게 표현할지 뿐 아니라, 이 표현법을 어떠한 방식으로 연산해야 할 지 또한 큰 문제입니다.
보통 이러한 밀도 높은 벡터를 얻기 위해 사용하는 뉴럴넷 모델은 $|V|$ (말뭉치의 단어 개수)의
큰 입력 차원과 몇 안되는 (텍스를 분류하는 문제라고 할 경우) 작은 출력 차원을 가집니다.
즉, 단어들 간의 연산이 필수입니다. 어떻게 이 큰 차원의 공간을 작은 공간으로 변형시킬 수 있을까요?

먼저, 상기한 ASCII코드 대신 원핫 인코딩(one-hot encoding)을 사용해보는 것은 어떨까요? 원핫 인코딩이란
하나의 단어 $w$ 를 아래의 벡터로 표현하는 것을 말합니다.

\begin{align}\overbrace{\left[ 0, 0, \dots, 1, \dots, 0, 0 \right]}^\text{|V| elements}\end{align}

여기서 1은 해당 벡터가 표현하고자 하는 단어에 해당하는 위치 1곳에 자리합니다. (나머지는 전부
0입니다.) 다른 단어를 나타내는 벡터에선 1이 다른 곳에 위치해 있겠죠.

원핫 인코딩은 만들기가 쉽다는 장점이 있지만, 단순한 만큼 단점도 있습니다. 일단 단어 벡터 한 개는
모든 단어를 표현할 수 있을 만한 크기가 되어야 합니다. 우리가 얼마나 많은 종류의 단어를
사용하는지를 생각 한다면 어마어마하게 큰 벡터라는 것을 알 수 있죠. 이 뿐만이 아닙니다.
원핫 벡터는 모든 단어를 독립적인 개체로 가정하는 것을 볼 수 있습니다. 즉, 공간상에서
완전히 다른 축에 위치해 있어서 단어간의 관계를 나타낼 수가 없습니다. 하지만 우리는 단어
사이의 *유사도* 를 어떻게든 계산하고 싶은거죠. 왜 유사도가 중요하냐구요? 다음 예제를 봅시다.

우리의 목표가 언어 모델을 만드는 것이라고 가정하고 다음의 문장이 학습 데이터로써 주어졌다고 해봅시다.

* 수학자가 가게로 뛰어갔다.
* 물리학자가 가게로 뛰어갔다.
* 수학자가 리만 가설을 증명했다.

또한 학습 데이터에는 없는 아래 문장이 있다고 생각해봅시다.

* 물리학자가 리만 가설을 증명했다.

ASCII 코드나 원핫 인코딩 기반 언어 모델은 위 문장을 어느정도 다룰 수 있겠지만, 개선의 여지가 있지 않을까요?
먼저 아래의 두 사실을 생각해봅시다.

* '수학자'와 '물리학자'가 문장 내에서 같은 역할을 맡고 있습니다. 이 두 단어는 어떻게든 의미적인 연관성이 있을 겁니다.
* 새로운 문장에서 '물리학자'가 맡은 역할을 '수학자'가 맡는 것을 학습 데이터에서 본 적이 있습니다.

우리 모델이 위의 사실을 통해 '물리학자'가 새 문장에 잘 들어 맞는다는 것을 추론할 수
있다면 참 좋을 것입니다. 이것이 위에서 언급한 유사도의 의미입니다. 철자적 유사도 뿐
아니라 *의미적 유사도* 인 것입니다. 이것이야말로 언어 데이터에 내재하는 희박성(sparsity)에
대한 처방이 될 것입니다. 우리가 본 것과 아직 보지 않은 것 사이를 이어주는 것이죠.
앞으로는 다음의 언어학적 기본 명제를 가정하도록 합시다. 바로 비슷한 맥락에서 등장하는
단어들은 서로 의미적 연관성을 가진다는 것입니다. 언어학적으로는 `분산 의미 가설(distributional
hypothesis) <https://en.wikipedia.org/wiki/Distributional_semantics>`__ 이라고도 합니다.

밀집된 단어 임베딩 구하기
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

어떻게 단어의 의미적 유사도를 인코딩 할 수 있을까요? 다시 말해, 어떻게 해야 단어의 유사도를
단어 벡터에 반영할 수 있을까요? 단어 데이터에 의미적 속성(attribute)을 부여하는 건 어떤가요?
예를 들어 '수학자'와 '물리학자'가 모두 뛸 수 있다면, 해당 단어의 '뛸 수 있음' 속성에 높은 점수를 주는 겁니다.
계속 해봅시다. 다른 단어들에 대해서는 어떠한 속성을 만들 수 있을지 생각해봅시다.

만약 각 속성을 하나의 차원이라고 본다면 하나의 단어에 아래와 같은 벡터를 배정할 수 있을겁니다.

\begin{align}q_\text{수학자} = \left[ \overbrace{2.3}^\text{뛸 수 있음},
   \overbrace{9.4}^\text{커피를 좋아함}, \overbrace{-5.5}^\text{물리 전공임}, \dots \right]\end{align}

\begin{align}q_\text{물리학자} = \left[ \overbrace{2.5}^\text{뛸 수 있음},
   \overbrace{9.1}^\text{커피를 좋아함}, \overbrace{6.4}^\text{물리 전공임}, \dots \right]\end{align}

그러면 아래와 같이 두 단어 사이의 유사도를 구할 수 있습니다. ('유사도'라는 함수를 정의하는 겁니다)

\begin{align}\text{유사도}(\text{물리학자}, \text{수학자}) = q_\text{물리학자} \cdot q_\text{수학자}\end{align}

물론 보통은 이렇게 벡터의 길이로 나눠주지만요.

\begin{align}\text{유사도}(\text{물리학자}, \text{수학자}) = \frac{q_\text{물리학자} \cdot q_\text{수학자}}
   {\| q_\text{물리학자} \| \| q_\text{수학자} \|} = \cos (\phi)\end{align}

$\phi$ 는 두 벡터 사이의 각입니다. 이런 식이면 정말 비슷한 단어는 유사도 1을 갖고,
정말 다른 단어는 유사도 -1을 갖겠죠. 비슷한 의미를 가질수록 같은 방향을 가리키고 있을 테니까요.

이 글 초반에 나온 희박한 원핫 벡터가 사실은 우리가 방금 정의한 의미 벡터의
특이 케이스라는 것을 금방 알 수 있습니다. 단어 벡터의 각 원소는 그 단어의 의미적 속성을
표현하고, 모든 단어 쌍의 유사도는 0이기 때문이죠. 위에서 정의한 의미 벡터는 *밀집* 되어 있습니다.
즉, 원핫 벡터에 비해 0 원소의 수가 적다고 할 수 있습니다.

하지만 이 벡터들은 구하기가 진짜 어렵습니다. 단어간의 유사도를 결정 지을 만한
의미적 속성은 어떻게 결정할 것이며, 속성을 결정했다고 하더라도 각 속성에
해당하는 값은 도대체 어떠한 기준으로 정해야 할까요? 속성과 값을 데이터에 기반해
만들고 자동으로 단어 벡터를 만들 수는 없을까요? 있습니다. 딥러닝을 사용하면 말이죠.
딥러닝은 인공신경망을 이용하여 사람의 개입 없이 속성의 표현 방법을 자동으로 학습합니다.
이를 이용해 단어 벡터를 모델 모수로 설정하고 모델 학습시에 단어 벡터도 함께 업데이트 하면
될 것입니다. 이렇게 우리 신경망 모델은 적어도 이론상으로는 충분히 학습할 수 있는
*잠재 의미 속성* 을 찾을 것입니다. 여기서 말하는 잠재 의미 속성으로 이루어진 벡터는 사람이
해석하기 상당히 어렵다는 점을 기억해 두세요. 위에서 수학자와 물리학자에게 커피를 좋아한다는
등 사람이 임의적으로 단어에 부여한 속성과는 달리, 인공신경망이 자동으로 단어의 속성을 찾는다면
그 속성과 값이 의미하는 바를 알기가 어려울 것입니다. 예를 들어서 신경망 모델이 찾은 '수학자'와
'물리학자'의 표현 벡터 둘 다 두번째 원소가 크다고 가정해 봅시다. 둘이 비슷하다는 건 알겠지만,
도대체 두번째 원소가 무엇을 의미하는지는 알기가 매우 힘든 것입니다. 표현 벡터 공간상에서
비슷하다는 정보 외에는 아마 많은 정보를 주긴 어려울 것입니다.

요약하자면, **단어 임베딩은 단어의 *의미* 를 표현하는 방법이며, 차후에 임베딩을 사용해서
풀고자 하는 문제에 유용할 의미 정보를 효율적으로 인코딩한 것입니다.** 품사 태그, 파스 트리(parse tree) 등
단어의 의미 외에 다른 것도 인코딩 할 수 있습니다! 피처 임베딩의 개념을 잡는 것이 중요합니다.

파이토치에서 단어 임베딩 하기
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

실제로 코드와 예시를 보기 전에, 파이토치를 비롯해 딥러닝 관련 프로그래밍을 할 때
단어 임베딩을 어떻게 사용하는지에 대해 조금 알아봅시다. 맨 위에서 원핫 벡터를
정의했던 것 처럼, 단어 임베딩을 사용할 때에도 각 단어에 인덱스를 부여해야 합니다.
이 인덱스를 참조 테이블(look-up table)에서 사용할 것입니다. 즉, $|V| \times D$ 크기의 행렬에
단어 임베딩을 저장하는데, $D$ 차원의 임베딩 벡터가 행렬의 $i$ 번째 행에
저장되어있어 $i$ 를 인덱스로 활용해 임베딩 벡터를 참조하는 것입니다.
아래의 모든 코드에서는 단어와 인덱스를 매핑해주는 딕셔너리를 word\_to\_ix라 칭합니다.

파이토치는 임베딩을 손쉽게 사용할 수 있게 torch.nn.Embedding에 위에서 설명한 참조 테이블
기능을 지원합니다. 이 모듈은 단어의 개수와 임베딩의 차원, 총 2개의 변수를 입력 변수로 받습니다.

torch.nn.Embedding 테이블의 임베딩을 참조하기 위해선 torch.LongTensor 타입의 인덱스 변수를
꼭 사용해야 합니다. (인덱스는 실수가 아닌 정수이기 때문입니다.)

```python
# Author: Robert Guthrie

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)
```

```python
word_to_ix = {"hello": 0, "world": 1}
embeds = nn.Embedding(2, 5)  # 2 words in vocab, 5 dimensional embeddings
lookup_tensor = torch.tensor([word_to_ix["hello"]], dtype=torch.long)
hello_embed = embeds(lookup_tensor)
print(hello_embed)
```

예시: N그램 언어 모델링
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

N그램 언어 모델링에선 단어 시퀀스 $w$ 가 주어졌을 때 아래의 것을 얻고자
함을 상기해 봅시다.

\begin{align}P(w_i | w_{i-1}, w_{i-2}, \dots, w_{i-n+1} )\end{align}

$w_i$ 는 시퀀스에서 i번째 단어입니다.

이 예시에서는 학습 데이터를 바탕으로 손실 함수를 계산하고 역전파를 통해
모수를 업데이트 해보겠습니다.

```python
CONTEXT_SIZE = 2
EMBEDDING_DIM = 10
# 셰익스피어 소네트(Sonnet) 2를 사용하겠습니다.
test_sentence = """When forty winters shall besiege thy brow,
And dig deep trenches in thy beauty's field,
Thy youth's proud livery so gazed on now,
Will be a totter'd weed of small worth held:
Then being asked, where all thy beauty lies,
Where all the treasure of thy lusty days;
To say, within thine own deep sunken eyes,
Were an all-eating shame, and thriftless praise.
How much more praise deserv'd thy beauty's use,
If thou couldst answer 'This fair child of mine
Shall sum my count, and make my old excuse,'
Proving his beauty by succession thine!
This were to be new made when thou art old,
And see thy blood warm when thou feel'st it cold.""".split()
# 원래는 입력을 제대로 토크나이즈(tokenize) 해야하지만 이번엔 간소화하겠습니다.
# 튜플로 이루어진 리스트를 만들겠습니다. 각 튜플은 ([ i-2 번째 단어, i-1 번째 단어 ], 목표 단어)입니다.
trigrams = [([test_sentence[i], test_sentence[i + 1]], test_sentence[i + 2])
            for i in range(len(test_sentence) - 2)]
# 첫 3개의 튜플을 출력하여 데이터가 어떻게 생겼는지 보겠습니다.
print(trigrams[:3])

vocab = set(test_sentence)
word_to_ix = {word: i for i, word in enumerate(vocab)}

class NGramLanguageModeler(nn.Module):

    def __init__(self, vocab_size, embedding_dim, context_size):
        super(NGramLanguageModeler, self).__init__()
        self.embeddings = nn.Embedding(vocab_size, embedding_dim)
        self.linear1 = nn.Linear(context_size * embedding_dim, 128)
        self.linear2 = nn.Linear(128, vocab_size)

    def forward(self, inputs):
        embeds = self.embeddings(inputs).view((1, -1))
        out = F.relu(self.linear1(embeds))
        out = self.linear2(out)
        log_probs = F.log_softmax(out, dim=1)
        return log_probs

losses = []
loss_function = nn.NLLLoss()
model = NGramLanguageModeler(len(vocab), EMBEDDING_DIM, CONTEXT_SIZE)
optimizer = optim.SGD(model.parameters(), lr=0.001)

for epoch in range(10):
    total_loss = 0
    for context, target in trigrams:

        # 첫번째. 모델에 넣어줄 입력값을 준비합니다. (i.e, 단어를 정수 인덱스로
        # 바꾸고 파이토치 텐서로 감싸줍시다.)
        context_idxs = torch.tensor([word_to_ix[w] for w in context], dtype=torch.long)

        # 두번째. 토치는 기울기가 *누적* 됩니다. 새 인스턴스를 넣어주기 전에
        # 기울기를 초기화합니다.
        model.zero_grad()

        # 세번째. 순전파를 통해 다음에 올 단어에 대한 로그 확률을 구합니다.
        log_probs = model(context_idxs)

        # 네번째. 손실함수를 계산합니다. (파이토치에서는 목표 단어를 텐서로 감싸줘야 합니다.)
        loss = loss_function(log_probs, torch.tensor([word_to_ix[target]], dtype=torch.long))

        # 다섯번째. 역전파를 통해 기울기를 업데이트 해줍니다.
        loss.backward()
        optimizer.step()

        # tensor.item()을 호출하여 단일원소 텐서에서 숫자를 반환받습니다.
        total_loss += loss.item()
    losses.append(total_loss)
print(losses)  # 반복할 떄마다 손실이 줄어드는 것을 봅시다!
```

예시: 단어 임베딩 계산하기: Continuous Bag-of-Words
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Continuous Bag-of-Words (CBOW) 모델은 NLP 딥러닝에서 많이 쓰입니다.
이 모델은 문장 내에서 주변 단어, 즉 앞 몇 단어와 뒤 몇 단어를 보고 특정
단어를 예측하는데, 언어 모델링과는 달리 순차적이지도 않고 확률적이지도 않습니다.
주로 CBOW는 복잡한 모델의 초기 입력값으로 쓰일 단어 임베딩을 빠르게 학습하는
데에 쓰입니다. 이것을 *사전 훈련된(pre-trained) 임베딩* 이라고 부르죠.
몇 퍼센트 정도의 성능 향상을 기대할 수 있는 기법입니다.

CBOW 모델은 다음과 같습니다. 목표 단어 $w_i$ 와 그 양쪽에 $N$ 개의
문맥 단어 $w_{i-1}, \dots, w_{i-N}$ 와 $w_{i+1}, \dots, w_{i+N}$
가 주어졌을 때, (문맥 단어를 총칭해 $C$ 라고 합시다.)

\begin{align}-\log p(w_i | C) = -\log \text{Softmax}(A(\sum_{w \in C} q_w) + b)\end{align}

위 식을 최소화하는 것이 CBOW의 목적입니다. 여기서 $q_w$ 는 단어 $w$ 의
임베딩 입니다.

아래의 클래스 템플릿을 보고 파이토치로 CBOW를 구현해 보세요. 힌트는 다음과 같습니다.

* 어떤 모수를 정의해야 하는지 생각해보세요.
* 각 작업에서 다루어지는 변수의 차원이 어떤지 꼭 생각해보세요.
  텐서의 모양을 바꿔야 한다면 .view()를 사용하세요.

```python
CONTEXT_SIZE = 2  # 왼쪽으로 2단어, 오른쪽으로 2단어
raw_text = """We are about to study the idea of a computational process.
Computational processes are abstract beings that inhabit computers.
As they evolve, processes manipulate other abstract things called data.
The evolution of a process is directed by a pattern of rules
called a program. People create programs to direct processes. In effect,
we conjure the spirits of the computer with our spells.""".split()

# 중복된 단어를 제거하기 위해 `raw_text` 를 집합(set) 자료형으로 바꿔줍니다.
vocab = set(raw_text)
vocab_size = len(vocab)

word_to_ix = {word: i for i, word in enumerate(vocab)}
data = []
for i in range(2, len(raw_text) - 2):
    context = [raw_text[i - 2], raw_text[i - 1],
               raw_text[i + 1], raw_text[i + 2]]
    target = raw_text[i]
    data.append((context, target))
print(data[:5])

class CBOW(nn.Module):

    def __init__(self):
        pass

    def forward(self, inputs):
        pass

# 모델을 만들고 학습해 보세요.
# 아래는 데이터 준비를 원활하게 돕기 위한 함수입니다.

def make_context_vector(context, word_to_ix):
    idxs = [word_to_ix[w] for w in context]
    return torch.tensor(idxs, dtype=torch.long)

make_context_vector(data[0][0], word_to_ix)  # 예시
```


```python
%matplotlib inline
```

Adversarial Example Generation
==============================

**Author:** `Nathan Inkawhich <https://github.com/inkawhich>`__

If you are reading this, hopefully you can appreciate how effective some
machine learning models are. Research is constantly pushing ML models to
be faster, more accurate, and more efficient. However, an often
overlooked aspect of designing and training models is security and
robustness, especially in the face of an adversary who wishes to fool
the model.

This tutorial will raise your awareness to the security vulnerabilities
of ML models, and will give insight into the hot topic of adversarial
machine learning. You may be surprised to find that adding imperceptible
perturbations to an image *can* cause drastically different model
performance. Given that this is a tutorial, we will explore the topic
via example on an image classifier. Specifically we will use one of the
first and most popular attack methods, the Fast Gradient Sign Attack
(FGSM), to fool an MNIST classifier.

Threat Model
------------

For context, there are many categories of adversarial attacks, each with
a different goal and assumption of the attacker’s knowledge. However, in
general the overarching goal is to add the least amount of perturbation
to the input data to cause the desired misclassification. There are
several kinds of assumptions of the attacker’s knowledge, two of which
are: **white-box** and **black-box**. A *white-box* attack assumes the
attacker has full knowledge and access to the model, including
architecture, inputs, outputs, and weights. A *black-box* attack assumes
the attacker only has access to the inputs and outputs of the model, and
knows nothing about the underlying architecture or weights. There are
also several types of goals, including **misclassification** and
**source/target misclassification**. A goal of *misclassification* means
the adversary only wants the output classification to be wrong but does
not care what the new classification is. A *source/target
misclassification* means the adversary wants to alter an image that is
originally of a specific source class so that it is classified as a
specific target class.

In this case, the FGSM attack is a *white-box* attack with the goal of
*misclassification*. With this background information, we can now
discuss the attack in detail.

Fast Gradient Sign Attack
-------------------------

One of the first and most popular adversarial attacks to date is
referred to as the *Fast Gradient Sign Attack (FGSM)* and is described
by Goodfellow et. al. in `Explaining and Harnessing Adversarial
Examples <https://arxiv.org/abs/1412.6572>`__. The attack is remarkably
powerful, and yet intuitive. It is designed to attack neural networks by
leveraging the way they learn, *gradients*. The idea is simple, rather
than working to minimize the loss by adjusting the weights based on the
backpropagated gradients, the attack *adjusts the input data to maximize
the loss* based on the same backpropagated gradients. In other words,
the attack uses the gradient of the loss w.r.t the input data, then
adjusts the input data to maximize the loss.

Before we jump into the code, let’s look at the famous
`FGSM <https://arxiv.org/abs/1412.6572>`__ panda example and extract
some notation.

.. figure:: /_static/img/fgsm_panda_image.png
   :alt: fgsm_panda_image

From the figure, $\mathbf{x}$ is the original input image
correctly classified as a “panda”, $y$ is the ground truth label
for $\mathbf{x}$, $\mathbf{\theta}$ represents the model
parameters, and $J(\mathbf{\theta}, \mathbf{x}, y)$ is the loss
that is used to train the network. The attack backpropagates the
gradient back to the input data to calculate
$\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)$. Then, it adjusts
the input data by a small step ($\epsilon$ or $0.007$ in the
picture) in the direction (i.e.
$sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y))$) that will
maximize the loss. The resulting perturbed image, $x'$, is then
*misclassified* by the target network as a “gibbon” when it is still
clearly a “panda”.

Hopefully now the motivation for this tutorial is clear, so lets jump
into the implementation.

```python
from __future__ import print_function
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
import numpy as np
import matplotlib.pyplot as plt
```

Implementation
--------------

In this section, we will discuss the input parameters for the tutorial,
define the model under attack, then code the attack and run some tests.

Inputs
~~~~~~

There are only three inputs for this tutorial, and are defined as
follows:

-  **epsilons** - List of epsilon values to use for the run. It is
   important to keep 0 in the list because it represents the model
   performance on the original test set. Also, intuitively we would
   expect the larger the epsilon, the more noticeable the perturbations
   but the more effective the attack in terms of degrading model
   accuracy. Since the data range here is $[0,1]$, no epsilon
   value should exceed 1.

-  **pretrained_model** - path to the pretrained MNIST model which was
   trained with
   `pytorch/examples/mnist <https://github.com/pytorch/examples/tree/master/mnist>`__.
   For simplicity, download the pretrained model `here <https://drive.google.com/drive/folders/1fn83DF14tWmit0RTKWRhPq5uVXt73e0h?usp=sharing>`__.

-  **use_cuda** - boolean flag to use CUDA if desired and available.
   Note, a GPU with CUDA is not critical for this tutorial as a CPU will
   not take much time.

```python
epsilons = [0, .05, .1, .15, .2, .25, .3]
pretrained_model = "data/lenet_mnist_model.pth"
use_cuda=True
```

Model Under Attack
~~~~~~~~~~~~~~~~~~

As mentioned, the model under attack is the same MNIST model from
`pytorch/examples/mnist <https://github.com/pytorch/examples/tree/master/mnist>`__.
You may train and save your own MNIST model or you can download and use
the provided model. The *Net* definition and test dataloader here have
been copied from the MNIST example. The purpose of this section is to
define the model and dataloader, then initialize the model and load the
pretrained weights.

```python
# LeNet Model definition
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

# MNIST Test dataset and dataloader declaration
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=False, download=True, transform=transforms.Compose([
            transforms.ToTensor(),
            ])),
        batch_size=1, shuffle=True)

# Define what device we are using
print("CUDA Available: ",torch.cuda.is_available())
device = torch.device("cuda" if (use_cuda and torch.cuda.is_available()) else "cpu")

# Initialize the network
model = Net().to(device)

# Load the pretrained model
model.load_state_dict(torch.load(pretrained_model, map_location='cpu'))

# Set the model in evaluation mode. In this case this is for the Dropout layers
model.eval()
```

FGSM Attack
~~~~~~~~~~~

Now, we can define the function that creates the adversarial examples by
perturbing the original inputs. The ``fgsm_attack`` function takes three
inputs, *image* is the original clean image ($x$), *epsilon* is
the pixel-wise perturbation amount ($\epsilon$), and *data_grad*
is gradient of the loss w.r.t the input image
($\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)$). The function
then creates perturbed image as

\begin{align}perturbed\_image = image + epsilon*sign(data\_grad) = x + \epsilon * sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y))\end{align}

Finally, in order to maintain the original range of the data, the
perturbed image is clipped to range $[0,1]$.

```python
# FGSM attack code
def fgsm_attack(image, epsilon, data_grad):
    # Collect the element-wise sign of the data gradient
    sign_data_grad = data_grad.sign()
    # Create the perturbed image by adjusting each pixel of the input image
    perturbed_image = image + epsilon*sign_data_grad
    # Adding clipping to maintain [0,1] range
    perturbed_image = torch.clamp(perturbed_image, 0, 1)
    # Return the perturbed image
    return perturbed_image
```

Testing Function
~~~~~~~~~~~~~~~~

Finally, the central result of this tutorial comes from the ``test``
function. Each call to this test function performs a full test step on
the MNIST test set and reports a final accuracy. However, notice that
this function also takes an *epsilon* input. This is because the
``test`` function reports the accuracy of a model that is under attack
from an adversary with strength $\epsilon$. More specifically, for
each sample in the test set, the function computes the gradient of the
loss w.r.t the input data ($data\_grad$), creates a perturbed
image with ``fgsm_attack`` ($perturbed\_data$), then checks to see
if the perturbed example is adversarial. In addition to testing the
accuracy of the model, the function also saves and returns some
successful adversarial examples to be visualized later.

```python
def test( model, device, test_loader, epsilon ):

    # Accuracy counter
    correct = 0
    adv_examples = []

    # Loop over all examples in test set
    for data, target in test_loader:

        # Send the data and label to the device
        data, target = data.to(device), target.to(device)

        # Set requires_grad attribute of tensor. Important for Attack
        data.requires_grad = True

        # Forward pass the data through the model
        output = model(data)
        init_pred = output.max(1, keepdim=True)[1] # get the index of the max log-probability

        # If the initial prediction is wrong, dont bother attacking, just move on
        if init_pred.item() != target.item():
            continue

        # Calculate the loss
        loss = F.nll_loss(output, target)

        # Zero all existing gradients
        model.zero_grad()

        # Calculate gradients of model in backward pass
        loss.backward()

        # Collect datagrad
        data_grad = data.grad.data

        # Call FGSM Attack
        perturbed_data = fgsm_attack(data, epsilon, data_grad)

        # Re-classify the perturbed image
        output = model(perturbed_data)

        # Check for success
        final_pred = output.max(1, keepdim=True)[1] # get the index of the max log-probability
        if final_pred.item() == target.item():
            correct += 1
            # Special case for saving 0 epsilon examples
            if (epsilon == 0) and (len(adv_examples) < 5):
                adv_ex = perturbed_data.squeeze().detach().cpu().numpy()
                adv_examples.append( (init_pred.item(), final_pred.item(), adv_ex) )
        else:
            # Save some adv examples for visualization later
            if len(adv_examples) < 5:
                adv_ex = perturbed_data.squeeze().detach().cpu().numpy()
                adv_examples.append( (init_pred.item(), final_pred.item(), adv_ex) )

    # Calculate final accuracy for this epsilon
    final_acc = correct/float(len(test_loader))
    print("Epsilon: {}\tTest Accuracy = {} / {} = {}".format(epsilon, correct, len(test_loader), final_acc))

    # Return the accuracy and an adversarial example
    return final_acc, adv_examples
```

Run Attack
~~~~~~~~~~

The last part of the implementation is to actually run the attack. Here,
we run a full test step for each epsilon value in the *epsilons* input.
For each epsilon we also save the final accuracy and some successful
adversarial examples to be plotted in the coming sections. Notice how
the printed accuracies decrease as the epsilon value increases. Also,
note the $\epsilon=0$ case represents the original test accuracy,
with no attack.

```python
accuracies = []
examples = []

# Run test for each epsilon
for eps in epsilons:
    acc, ex = test(model, device, test_loader, eps)
    accuracies.append(acc)
    examples.append(ex)
```

Results
-------

Accuracy vs Epsilon
~~~~~~~~~~~~~~~~~~~

The first result is the accuracy versus epsilon plot. As alluded to
earlier, as epsilon increases we expect the test accuracy to decrease.
This is because larger epsilons mean we take a larger step in the
direction that will maximize the loss. Notice the trend in the curve is
not linear even though the epsilon values are linearly spaced. For
example, the accuracy at $\epsilon=0.05$ is only about 4% lower
than $\epsilon=0$, but the accuracy at $\epsilon=0.2$ is 25%
lower than $\epsilon=0.15$. Also, notice the accuracy of the model
hits random accuracy for a 10-class classifier between
$\epsilon=0.25$ and $\epsilon=0.3$.

```python
plt.figure(figsize=(5,5))
plt.plot(epsilons, accuracies, "*-")
plt.yticks(np.arange(0, 1.1, step=0.1))
plt.xticks(np.arange(0, .35, step=0.05))
plt.title("Accuracy vs Epsilon")
plt.xlabel("Epsilon")
plt.ylabel("Accuracy")
plt.show()
```

Sample Adversarial Examples
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Remember the idea of no free lunch? In this case, as epsilon increases
the test accuracy decreases **BUT** the perturbations become more easily
perceptible. In reality, there is a tradeoff between accuracy
degredation and perceptibility that an attacker must consider. Here, we
show some examples of successful adversarial examples at each epsilon
value. Each row of the plot shows a different epsilon value. The first
row is the $\epsilon=0$ examples which represent the original
“clean” images with no perturbation. The title of each image shows the
“original classification -> adversarial classification.” Notice, the
perturbations start to become evident at $\epsilon=0.15$ and are
quite evident at $\epsilon=0.3$. However, in all cases humans are
still capable of identifying the correct class despite the added noise.

```python
# Plot several examples of adversarial samples at each epsilon
cnt = 0
plt.figure(figsize=(8,10))
for i in range(len(epsilons)):
    for j in range(len(examples[i])):
        cnt += 1
        plt.subplot(len(epsilons),len(examples[0]),cnt)
        plt.xticks([], [])
        plt.yticks([], [])
        if j == 0:
            plt.ylabel("Eps: {}".format(epsilons[i]), fontsize=14)
        orig,adv,ex = examples[i][j]
        plt.title("{} -> {}".format(orig, adv))
        plt.imshow(ex, cmap="gray")
plt.tight_layout()
plt.show()
```

Where to go next?
-----------------

Hopefully this tutorial gives some insight into the topic of adversarial
machine learning. There are many potential directions to go from here.
This attack represents the very beginning of adversarial attack research
and since there have been many subsequent ideas for how to attack and
defend ML models from an adversary. In fact, at NIPS 2017 there was an
adversarial attack and defense competition and many of the methods used
in the competition are described in this paper: `Adversarial Attacks and
Defences Competition <https://arxiv.org/pdf/1804.00097.pdf>`__. The work
on defense also leads into the idea of making machine learning models
more *robust* in general, to both naturally perturbed and adversarially
crafted inputs.

Another direction to go is adversarial attacks and defense in different
domains. Adversarial research is not limited to the image domain, check
out `this <https://arxiv.org/pdf/1801.01944.pdf>`__ attack on
speech-to-text models. But perhaps the best way to learn more about
adversarial machine learning is to get your hands dirty. Try to
implement a different attack from the NIPS 2017 competition, and see how
it differs from FGSM. Then, try to defend the model from your own
attacks.


# Multi-TX/TL

Matthieu Kratz

## Why this model?

Like most genetic circuit models, you typically start with a model that captures the two fundamental processes of transcription and translation. These processes can modelled with varying degrees of complexity, ranging from a basic tx/tl model (__Equation 1__) to complex models that simulate tx/tl at a base pair level. For my project, it was of particular importance that I accurately model the sequestration of transcriptional machinery. Further, I was working without RPU data, making typical models that assume some maximum steady state saturation e.g. positive proportional hill intractable for my system.

\begin{align}
\\ \\
&G \xrightarrow{ktx} T + G \\ \\
&\textbf{Equation 1.} \ \ \text{Simple Transcription}
\\
\end{align}

Hence, I needed a model that gave reasonable steady state transcription and translation dynamics from non-RPU parameters, all while accurately tracking the sequestration of the relevant machinery.

## The Multi-TX Model

The model CRN is below and the mechanism effectively relies on accounting for all possible RNAp occupancy states, including the various transitions between these occupancy states.

\begin{align}
\\ \\
&\textbf{1. Binding} \\
&T7_p + T7_p:G_n  \underset{kT72}{\overset{kT71}\rightleftharpoons} T7_p:G_{n\alpha}  \ \ , \   \text{n $\neq$ $n_{max}$} \\ \\
&\textbf{2. Closed --> Open} \\
&T7_p:G_{n\alpha}  \xrightarrow{k_{iso}} T7_p:G_{n+1}  \ \ , \   \text{n $\neq$ $n_{max}$} \\ \\
&\textbf{3. TX} \\
&T7_p:G_{n\alpha} \xrightarrow{ktx} nT7_m + nT7_p + T7_p:G_{0\alpha} \ \ , \  \text{n $\neq$ 0} \\
&T7_p:G_n \xrightarrow{ktx} nT7_m + nT7_p + T7_p:G_{0} \ \ , \  \text{n $\neq$ 0} \\
\end{align}

kT71 --> Promoter binding rate constant (bimolecular)

kT72 --> Promoter unbinding rate constant (unimolecular)

k_iso --> Closed to open complex transition rate constant (unimolecular)

ktx --> Single polymerase mRNA synthesis rate constant (unimolecular)

A couple of comments:
- We explicitly model the process of transitioning from the closed ($T7_p:G_{n\alpha}$) to open complex ($T7_p:G_{n+1}$). This a pretty slow reaction in vivo, and in the various iterations of this model, including this seemed to be key in accurately reflecting transcription dynamics.

- $n_{max}$ refers to the maximum possible occupancy of the gene. This is the physical limit which is ultimately determined by the footprint of the polymerase and the length of the gene. In relation to the point above, without isomerization, this physical limit is always met. With the explicit isomerization, this physical saturation is not met (with my parameter set at least).

- Polymerase can only be added one at a time to existing genes i.e. cannot have multiple binding events or closed complexes simultaneously.

- We have two TX reactions, one from the closed state and the other from the open state, both with $N$ actively transcribing polymerases. We decided to allow release from the closed state as there is no reason why one polymerase in the closed form should inhibit of other activitely transcribing polymerases. Further, at a modelling level, not allowing release from the closed state results in excessive sequestration of polymerase due to the long time scale of isomerization.

- In both TX reactions, the entirety of the $N$ polymerases (bar the one in the closed state) are simultaneously release, along with $N$ transcripts and the unoccupied gene ($T7_p:G_{0}$)

## Biocrnpyler multi_tx Mechanism subclass

Subclass should be availabe via `from biocrnpyler import mechanism`, code is below for clarity

Couple of comments:
- The subclass has been designed to be used in concert with the Promoter and DNA assemblies subclasses

- Have to define a maximum occupancy (int) and cognate polymerase (species or str) when instantiating

- The various complex species are generated within the subclass from the names of the polymerase and dna objects in the DNAassembly object. They are defined as species with DNA material types.

## Example: T7 Polymerase Transcription of GFP mRNA

Now we define a DNA assembly that use our mechanism in the following steps:
- Create a species for the relevant polymerase
- Create multi_tx Mechanism, give a maximum occupancy and polymerase (must be species)
- Associate this mechanism with a promoter
- Place this promoter into a DNA assembly

And voila, the cassette regulated by T7p is ready to use!

This DNAassembly will be placed in a SimpleTxTlExtract which only dilutes mRNA, not proteins. I haven't used protein dilution as it makes it easier to glean the degree of sequestration present with this mechanism. In practical use, you would of course allow your polymerase to be diluted.

```python
from biocrnpyler import *

#the most important parameters for multi_tx
#max_occ is the number of ribosomes that can bind to a transcript
#k_iso is the rate of isomerization
parameters = {"max_occ":10, "k_iso":10}

# Define Polymerase, and max occupancy and instatiate Mechanism Object
T7P = Species('T7p','protein')
#Multi-tx mechanism
MX = multi_tx(pol=T7P,name='MX')

# create promoter object, associated MX and params with it
pT7_mx = Promoter(name='pT7',mechanisms={'transcription':MX})

# place promoter object into DNA assembly with GFP reporter
GFP = Species("GFP", material_type = "protein")
PFL_mx = DNAassembly('PFL', promoter=pT7_mx, rbs = "weak", protein=GFP)

# create simple promoter and DNA assembly objects that synthesize polymerase
SC = DNAassembly('T7_const', promoter="weak", rbs = "weak", protein=T7P)

# make extract with T7p source and GFP and compile CRN
mixture = SimpleTxTlDilutionMixture(components=[PFL_mx, SC], parameter_file = "default_parameters.txt", parameters = parameters)

CRN1 = mixture.compile_crn()

print(CRN1.pretty_print(show_rates = False))
print(mixture.components)
```

    Species (26) = {0. dna[PFL] init_conc = 0, 1. complex[dna[PFL]:protein[T7p]] init_conc = 0, 2. complex[dna[PFL]:2x_protein[T7p]] init_conc = 0, 3. complex[dna[PFL]:3x_protein[T7p]] init_conc = 0, 4. complex[dna[PFL]:4x_protein[T7p]] init_conc = 0, 5. complex[dna[PFL]:5x_protein[T7p]] init_conc = 0, 6. complex[dna[PFL]:6x_protein[T7p]] init_conc = 0, 7. complex[dna[PFL]:7x_protein[T7p]] init_conc = 0, 8. complex[dna[PFL]:8x_protein[T7p]] init_conc = 0, 9. complex[dna[PFL]:9x_protein[T7p]] init_conc = 0, 10. complex[dna[PFL]:10x_protein[T7p]] init_conc = 0, 11. complex[dna[PFL]:protein[T7p](closed)] init_conc = 0, 12. complex[dna[PFL]:2x_protein[T7p](closed)] init_conc = 0, 13. complex[dna[PFL]:3x_protein[T7p](closed)] init_conc = 0, 14. complex[dna[PFL]:4x_protein[T7p](closed)] init_conc = 0, 15. complex[dna[PFL]:5x_protein[T7p](closed)] init_conc = 0, 16. complex[dna[PFL]:6x_protein[T7p](closed)] init_conc = 0, 17. complex[dna[PFL]:7x_protein[T7p](closed)] init_conc = 0, 18. complex[dna[PFL]:8x_protein[T7p](closed)] init_conc = 0, 19. complex[dna[PFL]:9x_protein[T7p](closed)] init_conc = 0, 20. complex[dna[PFL]:10x_protein[T7p](closed)] init_conc = 0, 21. protein[T7p] init_conc = 0, 22. rna[PFL] init_conc = 0, 23. protein[GFP] init_conc = 0, 24. dna[T7_const] init_conc = 0, 25. rna[T7_const] init_conc = 0}

    Reactions (48) = [
    0. protein[T7p]+complex[dna[PFL]:protein[T7p]] <--> complex[dna[PFL]:2x_protein[T7p](closed)]
    1. protein[T7p]+complex[dna[PFL]:2x_protein[T7p]] <--> complex[dna[PFL]:3x_protein[T7p](closed)]
    2. protein[T7p]+complex[dna[PFL]:3x_protein[T7p]] <--> complex[dna[PFL]:4x_protein[T7p](closed)]
    3. protein[T7p]+complex[dna[PFL]:4x_protein[T7p]] <--> complex[dna[PFL]:5x_protein[T7p](closed)]
    4. protein[T7p]+complex[dna[PFL]:5x_protein[T7p]] <--> complex[dna[PFL]:6x_protein[T7p](closed)]
    5. protein[T7p]+complex[dna[PFL]:6x_protein[T7p]] <--> complex[dna[PFL]:7x_protein[T7p](closed)]
    6. protein[T7p]+complex[dna[PFL]:7x_protein[T7p]] <--> complex[dna[PFL]:8x_protein[T7p](closed)]
    7. protein[T7p]+complex[dna[PFL]:8x_protein[T7p]] <--> complex[dna[PFL]:9x_protein[T7p](closed)]
    8. protein[T7p]+complex[dna[PFL]:9x_protein[T7p]] <--> complex[dna[PFL]:10x_protein[T7p](closed)]
    9. complex[dna[PFL]:protein[T7p](closed)] --> complex[dna[PFL]:protein[T7p]]
    10. complex[dna[PFL]:2x_protein[T7p](closed)] --> complex[dna[PFL]:2x_protein[T7p]]
    11. complex[dna[PFL]:3x_protein[T7p](closed)] --> complex[dna[PFL]:3x_protein[T7p]]
    12. complex[dna[PFL]:4x_protein[T7p](closed)] --> complex[dna[PFL]:4x_protein[T7p]]
    13. complex[dna[PFL]:5x_protein[T7p](closed)] --> complex[dna[PFL]:5x_protein[T7p]]
    14. complex[dna[PFL]:6x_protein[T7p](closed)] --> complex[dna[PFL]:6x_protein[T7p]]
    15. complex[dna[PFL]:7x_protein[T7p](closed)] --> complex[dna[PFL]:7x_protein[T7p]]
    16. complex[dna[PFL]:8x_protein[T7p](closed)] --> complex[dna[PFL]:8x_protein[T7p]]
    17. complex[dna[PFL]:9x_protein[T7p](closed)] --> complex[dna[PFL]:9x_protein[T7p]]
    18. complex[dna[PFL]:10x_protein[T7p](closed)] --> complex[dna[PFL]:10x_protein[T7p]]
    19. complex[dna[PFL]:protein[T7p]] --> protein[T7p]+rna[PFL]+dna[PFL]
    20. complex[dna[PFL]:2x_protein[T7p]] --> 2protein[T7p]+2rna[PFL]+dna[PFL]
    21. complex[dna[PFL]:3x_protein[T7p]] --> 3protein[T7p]+3rna[PFL]+dna[PFL]
    22. complex[dna[PFL]:4x_protein[T7p]] --> 4protein[T7p]+4rna[PFL]+dna[PFL]
    23. complex[dna[PFL]:5x_protein[T7p]] --> 5protein[T7p]+5rna[PFL]+dna[PFL]
    24. complex[dna[PFL]:6x_protein[T7p]] --> 6protein[T7p]+6rna[PFL]+dna[PFL]
    25. complex[dna[PFL]:7x_protein[T7p]] --> 7protein[T7p]+7rna[PFL]+dna[PFL]
    26. complex[dna[PFL]:8x_protein[T7p]] --> 8protein[T7p]+8rna[PFL]+dna[PFL]
    27. complex[dna[PFL]:9x_protein[T7p]] --> 9protein[T7p]+9rna[PFL]+dna[PFL]
    28. complex[dna[PFL]:10x_protein[T7p]] --> 10protein[T7p]+10rna[PFL]+dna[PFL]
    29. complex[dna[PFL]:2x_protein[T7p](closed)] --> protein[T7p]+rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    30. complex[dna[PFL]:3x_protein[T7p](closed)] --> 2protein[T7p]+2rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    31. complex[dna[PFL]:4x_protein[T7p](closed)] --> 3protein[T7p]+3rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    32. complex[dna[PFL]:5x_protein[T7p](closed)] --> 4protein[T7p]+4rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    33. complex[dna[PFL]:6x_protein[T7p](closed)] --> 5protein[T7p]+5rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    34. complex[dna[PFL]:7x_protein[T7p](closed)] --> 6protein[T7p]+6rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    35. complex[dna[PFL]:8x_protein[T7p](closed)] --> 7protein[T7p]+7rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    36. complex[dna[PFL]:9x_protein[T7p](closed)] --> 8protein[T7p]+8rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    37. complex[dna[PFL]:10x_protein[T7p](closed)] --> 9protein[T7p]+9rna[PFL]+complex[dna[PFL]:protein[T7p](closed)]
    38. dna[PFL]+protein[T7p] <--> complex[dna[PFL]:protein[T7p](closed)]
    39. rna[PFL] --> rna[PFL]+protein[GFP]
    40. dna[T7_const] --> dna[T7_const]+rna[T7_const]
    41. rna[T7_const] --> rna[T7_const]+protein[T7p]
    42. protein[T7p] -->
    43. rna[PFL] -->
    44. protein[GFP] -->
    45. rna[T7_const] -->
    46. rna[PFL] -->
    47. rna[T7_const] -->
    ]
    [DNAassembly: PFL
    	Promoter: pT7
    	transcript = rna_PFL
    	RBS: weak
    	protein = protein_GFP, DNAassembly: T7_const
    	Promoter: weak
    	transcript = rna_T7_const
    	RBS: weak
    	protein = protein_T7p]

```python
try:
    import numpy as np
    import matplotlib.pyplot as plt

    ts = np.arange(0,5000,1)

    x0_dict = {repr(PFL_mx.dna):1.,repr(SC.dna):.1}

    R1 = CRN1.simulate_with_bioscrape_via_sbml(ts, initial_condition_dict = x0_dict, stochastic = False)
    if R1 is not None:
        fig, ax = plt.subplots(1,2,figsize=(18,8))
        ax[0].set_title('Polymerase Levels',pad=20,fontdict={'fontsize':18})
        ax[0].plot(R1[str(T7P)],linewidth=3)
        ax[0].set_xlabel('Time (sec)',labelpad=15,fontdict={'fontsize':14})
        ax[0].set_ylabel('Polymerase Count',labelpad=15,fontdict={'fontsize':14})

        ax[1].set_title('GFP Levels',pad=20,fontdict={'fontsize':18})
        ax[1].plot(R1[str(GFP)],linewidth=3,c='k')
        ax[1].set_xlabel('Time (sec)',labelpad=15,fontdict={'fontsize':14})
        ax[1].set_ylabel('[GFP]',labelpad=15,fontdict={'fontsize':14})
except ModuleNotFoundError:
    print('please install the plotting libraries: pip install biocrnpyler[all]')

```

## Comparison of Transcript SS with RPU Data

Here we will do a head to head comparison with a simple transcription model built using RPU data. I consider this to effectively be the ground truth for the SS Transcript Count, although it should be noted this comparison does not necessary validate any of the pre-SS dynamics as the simple transcription model assumes immediate saturation. RPU data is from Qi et al.(2012) and RPU standard is from supplement of Nielsen et al. (2016).

```python
# place promoter object into DNA assembly
mech_default = Transcription_MM(rnap=T7P,name='MX')
pT7 = Promoter("pT7", mechanisms = [mech_default])
PFL_default = DNAassembly('PFL',dna='T7',promoter=pT7,rbs = "weak", protein = GFP)

# make extract with T7p source and GFP and compile CRN
Test_EX = SimpleTxTlDilutionMixture(components=[PFL_default, SC],parameter_file = "default_parameters.txt")
CRN2 = Test_EX.compile_crn()
```

```python
try:
    import numpy as np
    import matplotlib.pyplot as plt

    ts = np.arange(0,5000,1)
    x0_dict = {PFL_default.dna:1, SC.dna:.1}

    R2 = CRN2.simulate_with_bioscrape_via_sbml(ts, initial_condition_dict = x0_dict, stochastic = False,)
    if R2 is not None:
        fig = plt.figure(figsize=(10,6))
        plt.title('GFP Levels',pad=20,fontdict={'fontsize':18})
        plt.plot(R1[str(GFP)],linewidth=3,c='k',label='MTX Model')
        plt.plot(R2[str(GFP)],linewidth=3,c='b', label='RPU Model')
        plt.xlabel('Time (sec)',labelpad=15,fontdict={'fontsize':14})
        plt.ylabel('[GFP]',labelpad=15,fontdict={'fontsize':14})
        plt.legend(fontsize=14)

        print('\n \n')
        print(f"MX Model predicts {np.round(R1[str(str(GFP))].iloc[-1])} GFP and RPU Model predicts {np.round(R2[str(str(GFP))].iloc[-1])} GFP at SS \n \n")
except ModuleNotFoundError:
    print('please install the plotting libraries: pip install biocrnpyler[all]')

```

The two models seem to agree pretty well, with some minor difference. This is pretty cool given the fact that MX model uses indirect parameters (promoter affinity, isomerization and tx rates) to predict this SS value. Of course, it's also important to keep in mind that alternative models typically do to not account for the constant sequestration of the polymerase machinery to sustain this SS transcript level. As such, they may not reflect key dynamics resulting from machinery allocation/sharing e.g. retroactivity in tx/tl.

## Putative Multi-TL Model

The general model of binding, isomerization and production can be readily extended to the process of translation. However, there are several caveats for translation.

- First of all, I don't have any direct data like RPU experiments to compare my model with. All I can say is that with biologically reasonable parameter sets, you get something like a few 100 proteins per mRNA (RBZ in excess) and e. coli has a protein to mRNA ratio in that range (100:1 - 1000:1) (Taniguchi et al.(2010) or bionumbers book).

- Second of all, since we're no longer working with DNA complexes, the mRNA-RBZ complexes should be subject to degredation/dilution. However, it seems that at the level of the biology, initiation can have a stabilizing effect and for elongation it is uncertain (Roy et al. (2013)). In my simulations, I've effectively done as if only dilution is applied to these complexes, but there is an argument to include some form of active degredation (at a reduced rate). Depending on how this implemented, one may need to make a new degredation mechanism where the complexes effectively releases the ribosomes i.e. only mRNA degraded. This is also complicated by the fact that using subclasses like ComplexSpecies() results in inheritance of degredation/material properties.

## Quick Example
PFL is transcribed through simple transcription and translated through my multi_tl mechanism. We also have a constitutive source of our RBZ being produced to provide a constant saturating concentration of RBZ.

```python
#the most important parameters for multi_tl
#max_occ is the number of ribosomes that can bind to a transcript
#k_iso is the rate of isomerization
parameters = {"max_occ":2, "k_iso":10}

#Create a DNA assembly and Mixture
PFL = DNAassembly('PFL', dna='T7', rbs='medium', promoter='medium',transcript='GFP',protein='GFP')
EM = TxTlDilutionMixture(name = 'e coli',components=[PFL], parameter_file = "default_parameters.txt", parameters = parameters)

# Instantiate mechanism and pass it the ribosome
ML = multi_tl(EM.ribosome.get_species())

#Overwrite the translation mechanism
EM.add_mechanism(ML, overwrite = True)

CRN3 = EM.compile_crn()
print(CRN3.pretty_print(show_rates = False))
```

    Species (29) = {0. dna[T7] init_conc = 0, 1. protein[RNAP(machinery)] init_conc = 0, 2. rna[GFP] init_conc = 0, 3. complex[dna[T7]:protein[RNAP]] init_conc = 0, 4. complex[protein[Ribo]:rna[GFP]] init_conc = 0, 5. complex[2x_protein[Ribo]:rna[GFP]] init_conc = 0, 6. complex[protein[Ribo]:rna[GFP](closed)] init_conc = 0, 7. complex[2x_protein[Ribo]:rna[GFP](closed)] init_conc = 0, 8. protein[Ribo(machinery)] init_conc = 0, 9. protein[GFP] init_conc = 0, 10. protein[RNAase(machinery)] init_conc = 0, 11. dna[cellular_processes] init_conc = 0, 12. rna[cellular_processes] init_conc = 0, 13. complex[dna[cellular_processes]:protein[RNAP]] init_conc = 0, 14. complex[protein[Ribo]:rna[cellular_processes]] init_conc = 0, 15. complex[2x_protein[Ribo]:rna[cellular_processes]] init_conc = 0, 16. complex[protein[Ribo]:rna[cellular_processes](closed)] init_conc = 0, 17. complex[2x_protein[Ribo]:rna[cellular_processes](closed)] init_conc = 0, 18. protein[cellular_processes] init_conc = 0, 19. complex[protein[RNAase]:rna[GFP]] init_conc = 0, 20. complex[complex[protein[Ribo]:rna[GFP]]:protein[RNAase]] init_conc = 0, 21. complex[complex[2x_protein[Ribo]:rna[GFP]]:protein[RNAase]] init_conc = 0, 22. complex[complex[protein[Ribo]:rna[GFP]]:protein[RNAase]] init_conc = 0, 23. complex[complex[2x_protein[Ribo]:rna[GFP]]:protein[RNAase]] init_conc = 0, 24. complex[protein[RNAase]:rna[cellular_processes]] init_conc = 0, 25. complex[complex[protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] init_conc = 0, 26. complex[complex[2x_protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] init_conc = 0, 27. complex[complex[protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] init_conc = 0, 28. complex[complex[2x_protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] init_conc = 0}

    Reactions (42) = [
    0. dna[T7]+protein[RNAP(machinery)] <--> complex[dna[T7]:protein[RNAP]]
    1. complex[dna[T7]:protein[RNAP]] --> dna[T7]+rna[GFP]+protein[RNAP(machinery)]
    2. protein[Ribo(machinery)]+complex[protein[Ribo]:rna[GFP]] <--> complex[2x_protein[Ribo]:rna[GFP](closed)]
    3. complex[protein[Ribo]:rna[GFP](closed)] --> complex[protein[Ribo]:rna[GFP]]
    4. complex[2x_protein[Ribo]:rna[GFP](closed)] --> complex[2x_protein[Ribo]:rna[GFP]]
    5. complex[protein[Ribo]:rna[GFP]] --> protein[Ribo(machinery)]+protein[GFP]+rna[GFP]
    6. complex[2x_protein[Ribo]:rna[GFP]] --> 2protein[Ribo(machinery)]+2protein[GFP]+rna[GFP]
    7. complex[2x_protein[Ribo]:rna[GFP](closed)] --> protein[Ribo(machinery)]+protein[GFP]+complex[protein[Ribo]:rna[GFP](closed)]
    8. rna[GFP]+protein[Ribo(machinery)] <--> complex[protein[Ribo]:rna[GFP](closed)]
    9. dna[cellular_processes]+protein[RNAP(machinery)] <--> complex[dna[cellular_processes]:protein[RNAP]]
    10. complex[dna[cellular_processes]:protein[RNAP]] --> dna[cellular_processes]+rna[cellular_processes]+protein[RNAP(machinery)]
    11. protein[Ribo(machinery)]+complex[protein[Ribo]:rna[cellular_processes]] <--> complex[2x_protein[Ribo]:rna[cellular_processes](closed)]
    12. complex[protein[Ribo]:rna[cellular_processes](closed)] --> complex[protein[Ribo]:rna[cellular_processes]]
    13. complex[2x_protein[Ribo]:rna[cellular_processes](closed)] --> complex[2x_protein[Ribo]:rna[cellular_processes]]
    14. complex[protein[Ribo]:rna[cellular_processes]] --> protein[Ribo(machinery)]+protein[cellular_processes]+rna[cellular_processes]
    15. complex[2x_protein[Ribo]:rna[cellular_processes]] --> 2protein[Ribo(machinery)]+2protein[cellular_processes]+rna[cellular_processes]
    16. complex[2x_protein[Ribo]:rna[cellular_processes](closed)] --> protein[Ribo(machinery)]+protein[cellular_processes]+complex[protein[Ribo]:rna[cellular_processes](closed)]
    17. rna[cellular_processes]+protein[Ribo(machinery)] <--> complex[protein[Ribo]:rna[cellular_processes](closed)]
    18. rna[GFP]+protein[RNAase(machinery)] <--> complex[protein[RNAase]:rna[GFP]]
    19. complex[protein[RNAase]:rna[GFP]] --> protein[RNAase(machinery)]
    20. complex[protein[Ribo]:rna[GFP]]+protein[RNAase(machinery)] <--> complex[complex[protein[Ribo]:rna[GFP]]:protein[RNAase]]
    21. complex[complex[protein[Ribo]:rna[GFP]]:protein[RNAase]] --> protein[Ribo(machinery)]+protein[RNAase(machinery)]
    22. complex[2x_protein[Ribo]:rna[GFP]]+protein[RNAase(machinery)] <--> complex[complex[2x_protein[Ribo]:rna[GFP]]:protein[RNAase]]
    23. complex[complex[2x_protein[Ribo]:rna[GFP]]:protein[RNAase]] --> 2protein[Ribo(machinery)]+protein[RNAase(machinery)]
    24. complex[protein[Ribo]:rna[GFP](closed)]+protein[RNAase(machinery)] <--> complex[complex[protein[Ribo]:rna[GFP]]:protein[RNAase]]
    25. complex[complex[protein[Ribo]:rna[GFP]]:protein[RNAase]] --> protein[Ribo(machinery)]+protein[RNAase(machinery)]
    26. complex[2x_protein[Ribo]:rna[GFP](closed)]+protein[RNAase(machinery)] <--> complex[complex[2x_protein[Ribo]:rna[GFP]]:protein[RNAase]]
    27. complex[complex[2x_protein[Ribo]:rna[GFP]]:protein[RNAase]] --> 2protein[Ribo(machinery)]+protein[RNAase(machinery)]
    28. rna[cellular_processes]+protein[RNAase(machinery)] <--> complex[protein[RNAase]:rna[cellular_processes]]
    29. complex[protein[RNAase]:rna[cellular_processes]] --> protein[RNAase(machinery)]
    30. complex[protein[Ribo]:rna[cellular_processes]]+protein[RNAase(machinery)] <--> complex[complex[protein[Ribo]:rna[cellular_processes]]:protein[RNAase]]
    31. complex[complex[protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] --> protein[Ribo(machinery)]+protein[RNAase(machinery)]
    32. complex[2x_protein[Ribo]:rna[cellular_processes]]+protein[RNAase(machinery)] <--> complex[complex[2x_protein[Ribo]:rna[cellular_processes]]:protein[RNAase]]
    33. complex[complex[2x_protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] --> 2protein[Ribo(machinery)]+protein[RNAase(machinery)]
    34. complex[protein[Ribo]:rna[cellular_processes](closed)]+protein[RNAase(machinery)] <--> complex[complex[protein[Ribo]:rna[cellular_processes]]:protein[RNAase]]
    35. complex[complex[protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] --> protein[Ribo(machinery)]+protein[RNAase(machinery)]
    36. complex[2x_protein[Ribo]:rna[cellular_processes](closed)]+protein[RNAase(machinery)] <--> complex[complex[2x_protein[Ribo]:rna[cellular_processes]]:protein[RNAase]]
    37. complex[complex[2x_protein[Ribo]:rna[cellular_processes]]:protein[RNAase]] --> 2protein[Ribo(machinery)]+protein[RNAase(machinery)]
    38. rna[GFP] -->
    39. protein[GFP] -->
    40. rna[cellular_processes] -->
    41. protein[cellular_processes] -->
    ]

```python
try:

    import numpy as np
    import matplotlib.pyplot as plt

    ts = np.arange(0,10000,1)
    x0_dict = {PFL.dna:1, EM.ribosome.get_species():100,  EM.rnap.get_species():20}

    R3 = CRN3.simulate_with_bioscrape_via_sbml(ts, initial_condition_dict = x0_dict, stochastic = False,)

    if R3 is not None:
        fig, ax = plt.subplots(1,2,figsize=(18,8))
        ax[0].set_title('GFP Protein Levels',pad=20,fontdict={'fontsize':18})
        ax[0].plot(R3[str(PFL.protein)],linewidth=3)
        #ax[0].plot(R3[str(EM.ribosome.get_species())],linewidth=3)
        ax[0].set_xlabel('Time (sec)',labelpad=15,fontdict={'fontsize':14})
        ax[0].set_ylabel('GFP Protein Count',labelpad=15,fontdict={'fontsize':14})

        ax[1].set_title('GFP Transcript Levels',pad=20,fontdict={'fontsize':18})
        ax[1].plot(R3[str(PFL.transcript)],linewidth=3,c='k')
        ax[1].set_xlabel('Time (sec)',labelpad=15,fontdict={'fontsize':14})
        ax[1].set_ylabel('GFP Transcript Count',labelpad=15,fontdict={'fontsize':14})
except ModuleNotFoundError:
    print('please install the plotting libraries: pip install biocrnpyler[all]')

```

# Future Work

- Want to do more rigorous validation of multi-tx mechanism. As part of that, I first plan on comparing RPU data of existing consitutive promoters and non-native polymerase expression systems e.g. T5 with what the multi-tx model would predict.
- Is there data out there that would help validate the predicted RNAp occupancy of genes from the multi-tx model (same story for multi-tl model)?
- Eventually want to develop a multi-tx mechanism that can be used for TF-mediated transcription.
- Start looking towards validating multi-tl model, currently plan on seeing what data and parameters I could scrape from existing resource e.g. BCD RBS binding rates from biocrnpyler.
- Do some model comparison using bioscrape inference, where I generate parameters with RPU data model and fit parameters (vary known and unknown parameters) from the MTX model. Definitely very interesting for deriving isomerization rates as that seems hard to come by in literature and seems to be a key part of the model in determining system behaviour (Pre-SS and SS dynamics).

# Mentioned Papers and Parameter Resources

T7 parameters:
- Promoter Binding and Unbinding: Jia, Y., Kumar, A., & Patel, S. S. (1996). Equilibrium and Stopped-flow Kinetic Studies of Interaction between T7 RNA Polymerase and Its Promoters Measured by Protein and 2-Aminopurine Fluorescence Changes. Journal of Biological Chemistry , 271(48), 30451–30458. https://doi.org/10.1074/jbc.271.48.30451
- Isomerization Rate: Skinner, G. M., Baumann, C. G., Quinn, D. M., Molloy, J. E., & Hoggett, J. G. (2004). Promoter Binding, Initiation, and Elongation By Bacteriophage T7 RNA Polymerase: A SINGLE-MOLECULE VIEW OF THE TRANSCRIPTION CYCLE . Journal of Biological Chemistry , 279(5), 3239–3244. https://doi.org/10.1074/jbc.M310471200
- Translation Rate (T7p is VERY fast): Kochetkov, S. N., Rusakova, E. E., & Tunitskaya, V. L. (1998). Recent studies of T7 RNA polymerase mechanism. FEBS Letters, 440(3), 264–267. https://doi.org/https://doi.org/10.1016/S0014-5793(98)01484-7

Translation Parameters:
- RBS Binding and Unbinding: Chandra, F., & Del Vecchio, D. (2016). The Effects of Ribosome Autocatalysis and Negative Feedback in Resource Competition. bioRxiv. https://doi.org/10.1101/042127
- Isomerization Rate: Draper, D. E. (1993). Mechanisms of Translational Initiation and Repression in Prokaryotes BT  - The Translational Apparatus: Structure, Function, Regulation, Evolution. In K. H. Nierhaus, F. Franceschi, A. R. Subramanian, V. A. Erdmann, & B. Wittmann-Liebold (Eds.) (pp. 197–207). Boston, MA: Springer US. https://doi.org/10.1007/978-1-4615-2407-6_19

Misc:
- mRNA Degredation: Roy, B., & Jacobson, A. (2013). The intimate relationships of mRNA decay and translation. Trends in Genetics : TIG, 29(12), 691–699. https://doi.org/10.1016/j.tig.2013.09.002
- RPU Data: Nielsen, A. A. K., Der, B. S., Shin, J., Vaidyanathan, P., Paralanov, V., Strychalski, E. A., … Voigt, C. A. (2016). Genetic circuit design automation. Science, 352(6281), aac7341. https://doi.org/10.1126/science.aac7341 and Qi, L., Haurwitz, R. E., Shao, W., Doudna, J. A., & Arkin, A. P. (2012). RNA processing enables predictable programming of gene expression. Nature Biotechnology, 30(10), 1002–1006. https://doi.org/10.1038/nbt.2355
- Protein:mRNA ratios: Taniguchi, Y., Choi, P. J., Li, G.-W., Chen, H., Babu, M., Hearn, J., … Xie, X. S. (2010). Quantifying E. coli Proteome and Transcriptome with Single-Molecule Sensitivity in Single Cells. Science, 329(5991), 533 LP – 538. https://doi.org/10.1126/science.1188308


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)
```

Toggle cell visibility here.

```python
%matplotlib notebook
import pylab
import matplotlib.pyplot as plt
import math
import sympy as sym
import numpy as np
import ipywidgets as widgets
import control as control
import math as math
from ipywidgets import interact
from IPython.display import Latex, display, Markdown
```

## Linearization of a function

### Introduction

> Linearization is defined as a process of finding a linear approximation of a function at a certain point. The linear approximation of a function is obtained by the Taylor expansion around the point of interest in which only the first two terms are kept. Linearization is an effective method for approximating the output of a function $y=f(x)$ at any $x=x_0+\Delta x$ based on the value and the slope of the function at $x=x_0+\Delta x$, given that $f(x)$ is differentiable on $[x_0,x_0+\Delta x]$ (or $[x_0+\Delta x,x_0]$) and that $x_0$ is close to $x_0+\Delta x$. In short, linearization approximates the output of a function near $x=x_0$. (source: [Wikipedia](https://en.wikipedia.org/wiki/Linearization))

In this example, linearization is defined as:

\begin{equation}
    f(x)\approx f(x_0)+f^{\prime}(x_0) \cdot (x-x_0),
\end{equation}

where $f^{\prime}=\frac{f(x_0+h)-f(x_0)}{h}$ ($h$ is set to $0.01$ in order to calculate the derivative).

Unit step function is defined as:

\begin{equation}
    u(x) =
    \begin{cases}
      0; & \text{$x<0$}\\
      1; & \text{$x\geq0$}
    \end{cases},
\end{equation}

and unit ramp function:

\begin{equation}
    r(x) =
    \begin{cases}
      0; & \text{$x<0$}\\
      x; & \text{$x\geq0$}
    \end{cases}.
\end{equation}

---

### How to use this notebook?
Move the slider to change the value of $x_0$, i.e. the $x$ value at which you want to linearize the function.

```python
# sinus, step, ramp, x^2, sqrt(x)
functionSelect = widgets.ToggleButtons(
    options=[('sine function', 0), ('unit step function', 1), ('unit ramp function', 2), ('parabolic function', 3), ('square root function', 4)],
    description='Select: ')

fig = plt.figure(num='Linearization of a function')
fig.set_size_inches((9.8, 3))
fig.set_tight_layout(True)
f1 = fig.add_subplot(1, 1, 1)

f1.grid(which='both', axis='both', color='lightgray')

f1.set_xlabel('$x$')
f1.set_ylabel('$f(x)$')

f1.axhline(0,Color='black',linewidth=0.5)
f1.axvline(0,Color='black',linewidth=0.5)

func_plot, = f1.plot([],[])
tang_plot, = f1.plot([],[])
point_plot, = f1.plot([],[])

f1.set_xlim((-5,5))
f1.set_ylim((-6,6))

def create_draw_functions(x0,index):
    x=np.linspace(-5,5,1001)
    h=0.001 # equal to \Delta x

    global func_plot, tang_plot, point_plot

    if index==0:
        y=np.sin(x)
        fprime=(np.sin(x0+h)-np.sin(x0))/h
        tang=np.sin(x0)+fprime*(x-x0)
        fx0=np.sin(x0)
    elif index==1:
        y=np.zeros(1001)
        y[510:1001]=1
    elif index==2:
        y=np.zeros(1001)
        y[500:1001]=np.linspace(0,5,501)
    elif index==3:
        y=x*x
        fprime=((x0+h)*(x0+h)-(x0*x0))/h
        tang=x0*x0+fprime*(x-x0)
        fx0=x0*x0
    elif index==4:
        x1=np.linspace(0,5,500)
        y=np.sqrt(x1)
        if x0>=0:
            fprime=(np.sqrt(x0+h)-np.sqrt(x0))/h
            tang=np.sqrt(x0)+fprime*(x-x0)
            fx0=np.sqrt(x0)

    f1.lines.remove(func_plot)
    f1.lines.remove(tang_plot)
    f1.lines.remove(point_plot)

    if index == 0:
        func_plot, = f1.plot(x,y,label='$f(x)=sin(x)$',color='C0')
        tang_plot, = f1.plot(x,tang,'--r',label='tangent')
        point_plot, = f1.plot(x0,fx0,'om',label='$x_0$')
        for txt in f1.texts:
                txt.set_visible(False)
    elif index == 1: # in case of the unit step function
        if x0==0:
            func_plot, = f1.step(x,y,label='$f(x)=u(x)$',color='C0')
            tang_plot, = f1.plot([],[])
            point_plot, = f1.plot([],[])
            f1.text(0.1,1.3,'Linearization at $x_0=0$ is not possible!',fontsize=14)
        elif x0<0:
            tang=np.zeros(1001)
            func_plot, = f1.step(x,y,label='$f(x)=u(x)$',color='C0')
            tang_plot, = f1.plot(x,tang,'--r',label='tangent')
            point_plot, = f1.plot(x0,[0],'om',label='$x_0$')
            for txt in f1.texts:
                txt.set_visible(False)
        elif x0>0:
            tang=np.ones(1001)
            func_plot, = f1.step(x,y,label='$f(x)=u(x)$',color='C0')
            tang_plot, = f1.plot(x,tang,'--r',label='tangent')
            point_plot, = f1.plot(x0,[1],'om',label='$x_0$')
            for txt in f1.texts:
                txt.set_visible(False)
    elif index==2: # in case of the ramp
        if x0<0:
            tang=np.zeros(1001)
            func_plot, = f1.plot(x,y,label='$f(x)=R(x)$',color='C0')
            tang_plot, = f1.plot(x,np.zeros(1001),'--r',label='tangent')
            point_plot, = f1.plot(x0,[0],'om',label='$x_0$')
            for txt in f1.texts:
                txt.set_visible(False)
        elif x0>=0:
            tang=x
            func_plot, = f1.plot(x,y,label='$f(x)=R(x)$',color='C0')
            tang_plot, = f1.plot(x,tang,'--r',label='tangent')
            point_plot, = f1.plot(x0,x0,'om',label='$x_0$')
            for txt in f1.texts:
                txt.set_visible(False)
    elif index==3:
        func_plot, = f1.plot(x,y,label='$f(x)=x^2$',color='C0')
        tang_plot, = f1.plot(x,tang,'--r',label='tangent')
        point_plot, = f1.plot(x0,fx0,'om',label='$x_0$')
        for txt in f1.texts:
                txt.set_visible(False)
    elif index==4: #in case of the square root function
        if x0<0:
            for txt in f1.texts:
                txt.set_visible(False)
            func_plot, = f1.plot(x1,y,label='$f(x)=\sqrt{x}$',color='C0')
            tang_plot, = f1.plot([],[])
            point_plot, = f1.plot([],[])
            f1.text(-4.9,1.3,'Square root function is not defined for $x<0$!',fontsize=14)
        else:
            func_plot, = f1.plot(x1,y,label='$f(x)=\sqrt{x}$',color='C0')
            tang_plot, = f1.plot(x,tang,'--r',label='tangent')
            point_plot, = f1.plot(x0,fx0,'om',label='$x_0$')
            for txt in f1.texts:
                txt.set_visible(False)

    if (index==1) and x0==0 or (index==4 and x0<0):
        display(Markdown('See comment on the figure.'))
    else:
        k=round(((tang[-1]-tang[0])/(x[-1]-x[0])),3)
        n=round(((tang[-1]-k*x[-1])),3)
        display(Markdown('Equation of the tangent: $y=%.3fx+%.3f$.'%(k,n)))

    f1.legend()

    f1.relim()
    f1.relim()
    f1.autoscale_view()
    f1.autoscale_view()

x0_slider = widgets.FloatSlider(value=1, min=-5, max=5, step=0.2, description='$x_0$',
                          continuous_update=True, layout=widgets.Layout(width='auto', flex='5 5 auto'),readout_format='.1f')

input_data = widgets.interactive_output(create_draw_functions, {'x0':x0_slider, 'index':functionSelect})

def update_sliders(index):
    global x0_slider

    x0val = [0.5, 0.5, 1, 1, 5, 10]
    x0slider.value = x0val[index]

input_data2 = widgets.interactive_output(update_sliders, {'index':functionSelect})

display(functionSelect)

display(x0_slider,input_data)

# display(Markdown("The system can be represented as $f(x)=5$ for small excursions of x about x0."))
```

    <IPython.core.display.Javascript object>

    ToggleButtons(description='Select: ', options=(('sine function', 0), ('unit step function', 1), ('unit ramp fu…

    FloatSlider(value=1.0, description='$x_0$', layout=Layout(flex='5 5 auto', width='auto'), max=5.0, min=-5.0, r…

    Output()

```python

```


```python
%run notebook_setup
```

# Getting started with The Joker

*The Joker* (pronounced Yo-kurr) is a highly specialized Monte Carlo (MC) sampler that is designed to generate converged posterior samplings for Keplerian orbital parameters, even when your data are sparse, non-uniform, or very noisy. This is *not* a general MC sampler, and this is *not* a Markov Chain MC sampler like `emcee`, or `pymc3`: This is fundamentally a [rejection sampler](https://en.wikipedia.org/wiki/Rejection_sampling) with some tricks that help improve performance for the two-body problem.

*The Joker* shines over more conventional MCMC sampling methods when your radial velocity data is imprecise, non-uniform, sparse, or has a short baseline: In these cases, your likelihood function will have many, approximately equal-height modes that are often spaced widely, all properties that make conventional MCMC bork when applied to this problem. In this tutorial, we will not go through the math behind the sampler (most of that is covered [in the original paper](https://arxiv.org/abs/1610.07602)). However, some terminology is important to know for the tutorial below or for reading the documentation. Most relevant, the parameters in the two-body problem (Kepler orbital parameters) split into two sets: nonlinear and linear parameters. The nonlinear parameters are always the same in each run of The Joker: period $P$, eccentricity $e$, argument of pericenter $\omega$, and a phase $M_0$. The default linear parameters are the velocity semi-ampltude $K$, and a systemtic velocity $v_0$. However, there are ways to add additional linear parameters into the model (as described in other tutorials).

For this tutorial, we will set up an inference problem that is common to binary star or exoplanet studies, show how to generate posterior orbit samples from the data, and then demonstrate how to visualize the samples. Other tutorials demonstrate more advanced or specialized functionality included in *The Joker*, like:
- [fully customizing the parameter prior distributions](2-Customize-prior.ipynb),
- [allowing for a long-term velocity trend in the data](3-Polynomial-velocity-trend.ipynb),
- [continuing sampling with standard MCMC methods](4-Continue-sampling-mcmc.ipynb) when *The Joker* returns one or few samples,
- [simultaneously inferring constant offsets between data sources](5-Calibration-offsets.ipynb) (i.e. when using data from multiple instruments that may have calibration offsets)

But let's start here with the most basic functionality!

First, imports we will need later:

```python
import astropy.table as at
from astropy.time import Time
import astropy.units as u
from astropy.visualization.units import quantity_support
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline

import thejoker as tj
```

```python
# set up a random generator to ensure reproducibility
rnd = np.random.default_rng(seed=42)
```

## Loading radial velocity data

To start, we need some radial velocity data to play with. Our ultimate goal is to construct or read in a `thejoker.RVData` instance, which is the main data container object used in *The Joker*. For this tutorial, we will use a simulated RV curve that was generated using a separate script and saved to a CSV file, and we will create an `RVData` instance manually.

Because we previously saved this data as an Astropy [ECSV](http://docs.astropy.org/en/latest/api/astropy.io.ascii.Ecsv.html#astropy.io.ascii.Ecsv) file, the units are provided with the column data and read in automatically using the [astropy.table read/write interface](http://docs.astropy.org/en/latest/table/index.html):

```python
data_tbl = at.QTable.read('data.ecsv')
data_tbl[:2]
```

The full simulated data table has many rows (256), so let's randomly grab 4 rows to work with:

```python
sub_tbl = data_tbl[rnd.choice(len(data_tbl), size=4, replace=False)]
```

```python
sub_tbl
```

It looks like the time column is given in Barycentric Julian Date (BJD), so in order to create an `RVData` instance, we will need to create an `astropy.time.Time` object from this column:

```python
t = Time(sub_tbl['bjd'], format='jd', scale='tcb')
data = tj.RVData(t=t, rv=sub_tbl['rv'], rv_err=sub_tbl['rv_err'])
```

We now have an `RVData` object, so we could continue on with the tutorial. But as a quick aside, there is an alternate, more automatic (automagical?) way to create an `RVData` instance from tabular data: `RVData.guess_from_table`. This classmethod attempts to guess the time format and radial velocity column names from the columns in the data table. It is very much an experimental feature, so if you think it can be improved, please open an issue in the [GitHub repo for The Joker](https://github.com/adrn/thejoker/issues). In any case, here it successfully works:

```python
data = tj.RVData.guess_from_table(sub_tbl)
```

One of the handy features of `RVData` is the `.plot()` method, which generates a quick view of the data:

```python
_ = data.plot()
```

The data are clearly variable! But what orbits are consistent with these data? I suspect many, given how sparse they are! Now that we have the data in hand, we need to set up the sampler by specifying prior distributions over the parameters in *The Joker*.

## Specifying the prior distributions for The Joker parameters

The prior *pdf* (probability distribution function) for *The Joker* is controlled and managed through the `thejoker.JokerPrior` class. The prior for *The Joker* is fairly customizable and the initializer for `JokerPrior` is therefore pretty flexible; usually too flexible for typical use cases. We will therefore start by using an alternate initializer defined on the class, `JokerPrior.default()`, that provides a simpler interface for creating a `JokerPrior` instance that uses the default prior distributions assumed in *The Joker*. In the default prior:

$$
\begin{align}
&p(P) \propto \frac{1}{P} \quad ; \quad P \in (P_{\rm min}, P_{\rm max})\\
&p(e) = B(a_e, b_e)\\
&p(\omega) = \mathcal{U}(0, 2\pi)\\
&p(M_0) = \mathcal{U}(0, 2\pi)\\
&p(K) = \mathcal{N}(K \,|\, \mu_K, \sigma_K)\\
&\sigma_K = \sigma_{K, 0} \, \left(\frac{P}{P_0}\right)^{-1/3} \, \left(1 - e^2\right)^{-1/2}\\
&p(v_0) = \mathcal{N}(v_0 \,|\, \mu_{v_0}, \sigma_{v_0})\\
\end{align}
$$

where $B(.)$ is the beta distribution, $\mathcal{U}$ is the uniform distribution, and $\mathcal{N}$ is the normal distribution.

Most parameters in the distributions above are set to reasonable values, but there are a few required parameters for the default case: the range of allowed period values (``P_min`` and ``P_max``), the scale of the ``K`` prior variance ``sigma_K0``, and the standard deviation of the $v_0$ prior ``sigma_v``. Let's set these to some arbitrary numbers. Here, I chose the value for ``sigma_K0`` to be typical of a binary star system; if using The Joker for exoplanet science, you will want to adjust this correspondingly.

```python
prior = tj.JokerPrior.default(
    P_min=2*u.day, P_max=1e3*u.day,
    sigma_K0=30*u.km/u.s,
    sigma_v=100*u.km/u.s)
```

Once we have the prior instance, we need to generate some prior samples that we will then use *The Joker* to rejection sample down to a set of posterior samples. To generate prior samples, use the `JokerSamples.sample()` method. Here, we'll generate a lare number of samples to use:

```python
prior_samples = prior.sample(size=250_000,
                             random_state=rnd)
prior_samples
```

This object behaves like a Python dictionary in that the parameter values can be accessed via their key names:

```python
prior_samples['P']
```

```python
prior_samples['e']
```

They can also be written to disk or re-loaded using this same class. For example, to save these prior samples to the current directory to the file "prior_samples.hdf5":

```python
prior_samples.write("prior_samples.hdf5", overwrite=True)
```

We could then load the samples from this file using:

```python
tj.JokerSamples.read("prior_samples.hdf5")
```

## Running The Joker

Now that we have a set of prior samples, we can create an instance of The Joker and use the rejection sampler:

```python
joker = tj.TheJoker(prior, random_state=rnd)
joker_samples = joker.rejection_sample(data, prior_samples,
                                       max_posterior_samples=256)
```

This works by either passing in an instance of `JokerSamples` containing the prior samples, or by passing in a filename that contains `JokerSamples` written to disk. So, for example, this is equivalent:

```python
joker_samples = joker.rejection_sample(data, "prior_samples.hdf5",
                                       max_posterior_samples=256)
```

The ``max_posterior_samples`` argument above specifies the maximum number of posterior samples to return. It is often helpful to set a threshold here in cases when your data are very uninformative to avoid generating huge numbers of samples (which can slow down the sampler considerably).

In either case above, the ``joker_samples`` object returned from ``rejection_sample()`` is also an instance of the ``JokerSamples`` class, but now contains posterior samples for all nonlinear and linear parameters in the model:

```python
joker_samples
```

## Plotting The Joker orbit samples over the input data

With posterior samples in Keplerian orbital parameters in hand for our data set, we can now plot the posterior samples over the input data to get a sense for how constraining the data are. *The Joker* comes with a convenience plotting function, ``plot_rv_curves``, for doing just this:

```python
_ = tj.plot_rv_curves(joker_samples, data=data)
```

It has various options to allow customizing the style of the plot:

```python
fig, ax = plt.subplots(1, 1, figsize=(8, 4))
_ = tj.plot_rv_curves(joker_samples, data=data,
                      plot_kwargs=dict(color='tab:blue'),
                      data_plot_kwargs=dict(color='tab:red'),
                      relative_to_t_ref=True, ax=ax)
ax.set_xlabel(f'BMJD$ - {data.t.tcb.mjd.min():.3f}$')
```

Another way to visualize the samples is to plot 2D projections of the sample values, for example, to plot period against eccentricity:

```python
fig, ax = plt.subplots(1, 1, figsize=(8, 5))

with quantity_support():
    ax.scatter(joker_samples['P'],
               joker_samples['e'],
               s=20, lw=0, alpha=0.5)

ax.set_xscale('log')
ax.set_xlim(prior.pars['P'].distribution.a,
            prior.pars['P'].distribution.b)
ax.set_ylim(0, 1)

ax.set_xlabel('$P$ [day]')
ax.set_ylabel('$e$')
```

But is the true period value included in those distinct period modes returned by *The Joker*? When generating the simulated data, I also saved the true orbital parameters used to generate the data, so we can load and over-plot it:

```python
import pickle
with open('true-orbit.pkl', 'rb') as f:
    truth = pickle.load(f)
```

```python
fig, ax = plt.subplots(1, 1, figsize=(8, 5))

with quantity_support():
    ax.scatter(joker_samples['P'],
               joker_samples['e'],
               s=20, lw=0, alpha=0.5)

    ax.axvline(truth['P'], zorder=-1, color='tab:green')
    ax.axhline(truth['e'], zorder=-1, color='tab:green')
    ax.text(truth['P'], 0.95, 'truth', fontsize=20,
            va='top', ha='left', color='tab:green')

ax.set_xscale('log')
ax.set_xlim(prior.pars['P'].distribution.a,
            prior.pars['P'].distribution.b)
ax.set_ylim(0, 1)

ax.set_xlabel('$P$ [day]')
ax.set_ylabel('$e$')
```

It indeed looks like there are posterior samples from *The Joker* in the vicinity of the true value. Deciding what to do next depends on the problem you would like to solve. For example, if you just want to get a sense of how multi-modal the posterior *pdf* over orbital parameters is, you might be satisfied with the number of samples we generated and the plots we made in this tutorial. However, if you want to fully propagate the uncertainty in these orbital parameters through some other inference (for example, to transform the samples into constraints on companion mass or other properties), you may want or need to generate a lot more samples. To start, you could change ``max_posterior_samples`` to be a much larger number in the ``rejection_sample()`` step above. But I have found that in many cases, you need to run with many, many more (e.g., 500 million) prior samples. To read more, check out the next tutorial!