# Encoding of categorical variables

## Sources

* [Encoding categorical variables](https://kiwidamien.github.io/encoding-categorical-variables.html)
* [Benchmarking Categorical Encoders](https://towardsdatascience.com/benchmarking-categorical-encoders-9c322bd77ee8)
* [Smarter Ways to Encode Categorical Data for Machine Learning](https://towardsdatascience.com/smarter-ways-to-encode-categorical-data-for-machine-learning-part-1-of-3-6dca2f71b159)
* [Encoding Categorical Variables in Practice](https://medium.com/epfl-extension-school/encoding-categorical-variables-in-practice-a536907f2013)
* [Types of Categorical Data Encoding Schemes](https://medium.com/analytics-vidhya/types-of-categorical-data-encoding-schemes-a5bbeb4ba02b)
* [Guide to Encoding Categorical Values in Python ](https://pbpython.com/categorical-encoding.html)
* [All about Categorical Variable Encoding](https://towardsdatascience.com/all-about-categorical-variable-encoding-305f3361fd02)
* [Encoding categorical variables: one-hot and beyond](http://www.win-vector.com/blog/2017/04/encoding-categorical-variables-one-hot-and-beyond/)

# Background

Many machine learning algorithms are not able to use non-numeric data. While many features we might use, such as a person's age, or height, are numeric there are many that are not. Usually these features are represented by strings, and we need some way of transforming them to numbers before using scikit-learn's algorithms. The different ways of doing this are called encodings.

**Note**:This notebook does not address the translation of text into numbers/vectors. This is the topic of __word embeddings__ where the aim is to generate vectors such that words with similar meaning are close to one another and that the arithmetic of vectors corresponds to ''arithmetic'' of meaning, e.g. embedding('queen') ~ embedding('king') - embedding('man') + embedding('woman').

## Terminology

**Levels**: A levels of a non-numeric feature are the number of distinct values. The examples listed above are all examples of levels. The number of levels can vary wildly: the number of races for a patient is typically four (asian, black, hispanic, and white), the number of states for the US is 51 (if we include DC separately), while the number of professions is in the thousands.

**Ordinal**: If the levels are ordered, then we call the feature ordinal. For example, if a class grade such as "B+" or "A" is a non-numeric feature, but the letters are not just different, they are ordered (an "A" is better than a "B+", which is better than a "C-" etc).

**Nominal**: If the levels are just different without an ordering, we call the feature categorical. For example, professions or car brands are categorical. If we use an encoding that maps levels to numbers, we introduce an ordering on the categories, which may not be desirable. Most of this article will be about encoding categorical variables.

## Considerations when choosing encodings

**Do you have many levels?** If so, using an encoding that has a level-per-feature is difficult for tree-based models. Trees separate on features that "split" the data into different classes effectively. If there are many levels, it is likely only a tiny fraction of the data belong to one level, so it will be hard for trees to "find" that feature to split on. Typically this isn't a problem for linear models.

**Are there many examples of each level?** If there are only 5 doctors in your dataset, you probably are not going to know the doctor category very well (nor will it generalize). Some encoders deal with this gracefully, while others won't. You might consider making an explicit "other" category for levels, or grouping categories together. This is a problem for all models.

**Could you have new categories at test time?** Some categorical variables can be completely specified at training time (e.g. the levels for race or blood type would be known even with zero training examples). Other categories, such as profession, are so broad that we probably learn the levels from the training data. Some encoders deal with levels that are only in the test set better than others.

**Are the categories related?** Many encoding schemes treat two different levels as "equally different" from one another. If looking at color of a car, a typical encoding has no idea that "brick red" and "red" are more related than "red" and "yellow". One way of solving this problem is to cluster the categories into higher levels, and then encode that category as well.

**Is it reversible?** Does it store a lookup table? Given the encoding of a feature, can you recover the original value? If so, we call the encoding reversible. Generally, reversible features also require a lot of storage if there are a lot of levels (to figure out how to go backward). I would generally see reversible as a negative if it requries storing a lookup table as well.

## Example data

```python
import requests
import category_encoders as ce
import numpy as np
import pandas as pd

def encode_var(var, encoder, y=None):
    if y is None:
        encoder.fit(var)
    else:
        encoder.fit(var, y)
    new_var = encoder.transform(var)
    if isinstance(new_var, pd.DataFrame):
        new_var.insert(0, 'original', var)
        return new_var
    else:
        return pd.DataFrame({'original': var, 'encoder': new_var})

def print_res(res, rows_per_level=2):
    out = pd.DataFrame(columns=res.columns)
    for lvl in res.original.unique():
        out = out.append(res[res.original==lvl].head(rows_per_level))
    return out
```

```python
download_data = False
```

```python
if download_data:
    url = "http://mlr.cs.umass.edu/ml/machine-learning-databases/autos/imports-85.data"
    r = requests.get(url)
    with open('imports-85.data', 'wb') as f:
        f.write(r.content)
```

```python
# Define the headers since the data does not have any
headers = ["symboling", "normalized_losses", "make", "fuel_type", "aspiration",
           "num_doors", "body_style", "drive_wheels", "engine_location",
           "wheel_base", "length", "width", "height", "curb_weight",
           "engine_type", "num_cylinders", "engine_size", "fuel_system",
           "bore", "stroke", "compression_ratio", "horsepower", "peak_rpm",
           "city_mpg", "highway_mpg", "price"]
# Read in the CSV file and convert "?" to NaN
df = pd.read_csv("imports-85.data",
                  header=None, names=headers, na_values="?" )
```

```python
df.head()
```

      symboling
      normalized_losses
      make
      fuel_type
      aspiration
      num_doors
      body_style
      drive_wheels
      engine_location
      wheel_base
      ...
      engine_size
      fuel_system
      bore
      stroke
      compression_ratio
      horsepower
      peak_rpm
      city_mpg
      highway_mpg
      price

      0
      3
      NaN
      alfa-romero
      gas
      std
      two
      convertible
      rwd
      front
      88.6
      ...
      130
      mpfi
      3.47
      2.68
      9.0
      111.0
      5000.0
      21
      27
      13495.0

      1
      3
      NaN
      alfa-romero
      gas
      std
      two
      convertible
      rwd
      front
      88.6
      ...
      130
      mpfi
      3.47
      2.68
      9.0
      111.0
      5000.0
      21
      27
      16500.0

      2
      1
      NaN
      alfa-romero
      gas
      std
      two
      hatchback
      rwd
      front
      94.5
      ...
      152
      mpfi
      2.68
      3.47
      9.0
      154.0
      5000.0
      19
      26
      16500.0

      3
      2
      164.0
      audi
      gas
      std
      four
      sedan
      fwd
      front
      99.8
      ...
      109
      mpfi
      3.19
      3.40
      10.0
      102.0
      5500.0
      24
      30
      13950.0

      4
      2
      164.0
      audi
      gas
      std
      four
      sedan
      4wd
      front
      99.4
      ...
      136
      mpfi
      3.19
      3.40
      8.0
      115.0
      5500.0
      18
      22
      17450.0

5 rows × 26 columns

```python
df.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 205 entries, 0 to 204
    Data columns (total 26 columns):
    symboling            205 non-null int64
    normalized_losses    164 non-null float64
    make                 205 non-null object
    fuel_type            205 non-null object
    aspiration           205 non-null object
    num_doors            203 non-null object
    body_style           205 non-null object
    drive_wheels         205 non-null object
    engine_location      205 non-null object
    wheel_base           205 non-null float64
    length               205 non-null float64
    width                205 non-null float64
    height               205 non-null float64
    curb_weight          205 non-null int64
    engine_type          205 non-null object
    num_cylinders        205 non-null object
    engine_size          205 non-null int64
    fuel_system          205 non-null object
    bore                 201 non-null float64
    stroke               201 non-null float64
    compression_ratio    205 non-null float64
    horsepower           203 non-null float64
    peak_rpm             203 non-null float64
    city_mpg             205 non-null int64
    highway_mpg          205 non-null int64
    price                201 non-null float64
    dtypes: float64(11), int64(5), object(10)
    memory usage: 41.8+ KB

# Notation

$$ \begin{align}
N: & \text{Number of observations}\\
N_i: & \text{Number of observations with level i}\\
n: & \text{Number of levels}\\
i: & \text{The i-th level}\\
k: & \text{The k-th variable of the encoding}\\
x_k(i): & \text{Value for the k-th variable of the encoding when the original variable has label $i$.}\\
\bar{y}: & \text{The average value of the target}\\
\bar{y}_i: & \text{The average value of the target for level i}
\end{align}$$

```python

```

# Unsupervised Encoding Methods

## Label Encoding

**Description**

Label encoding replaces the *n* labels with values from *0* to *n-1*, in lexicographical order.

**Construction**

$$x(i) = i$$

**When to use**

Never (only for ordinal variables, for which there is the `OrdinalEncoder` class.

```python
df["num_cylinders"].value_counts().sort_index()
```

    eight       5
    five       11
    four      159
    six        24
    three       1
    twelve      1
    two         4
    Name: num_cylinders, dtype: int64

```python
from sklearn.preprocessing import LabelEncoder
```

```python
encoder = LabelEncoder()
res = encode_var(df["num_cylinders"], encoder)
print_res(res)
```

      original
      encoder

      0
      four
      2

      1
      four
      2

      2
      six
      3

      12
      six
      3

      4
      five
      1

      5
      five
      1

      71
      eight
      0

      72
      eight
      0

      55
      two
      6

      56
      two
      6

      18
      three
      4

      49
      twelve
      5

## Ordinal Encoding

**Description**

Same as label encoding, but values are ordered in the order of labels.

**When to use**

For ordinal variables, where the difference between all labels represents the same distance.

```python
df["num_cylinders"].value_counts(sort=False)
```

    three       1
    five       11
    two         4
    six        24
    four      159
    twelve      1
    eight       5
    Name: num_cylinders, dtype: int64

```python
df["num_cylinders"] = df["num_cylinders"].astype('category').cat.reorder_categories(ordered=True, new_categories=['two', 'three', 'four', 'five', 'six', 'eight', 'twelve'])
```

```python
df["num_cylinders"].value_counts(sort=False)
```

    two         4
    three       1
    four      159
    five       11
    six        24
    eight       5
    twelve      1
    Name: num_cylinders, dtype: int64

```python
encoder = ce.ordinal.OrdinalEncoder()
res = encode_var(df[["num_cylinders"]], encoder)
print_res(res)
```

      original
      num_cylinders

      0
      four
      3

      1
      four
      3

      2
      six
      5

      12
      six
      5

      4
      five
      4

      5
      five
      4

      71
      eight
      6

      72
      eight
      6

      55
      two
      1

      56
      two
      1

      18
      three
      2

      49
      twelve
      7

## Dummy/One Hot Encoding

**Description**

Creates a binary indicator for each level.

**Construction**

$$x_k(i) =
\begin{cases}
1 & \text{if } k=i \\
0 & \text{otherwise}
\end{cases}$$

**When to use**

When the main interest is in differences in average values for each level and there are sufficient observations for each level.

```python
df['drive_wheels'].value_counts()
```

    fwd    120
    rwd     76
    4wd      9
    Name: drive_wheels, dtype: int64

```python
encoder = ce.one_hot.OneHotEncoder()
res = encode_var(df[["drive_wheels"]], encoder)
print_res(res)
```

      original
      drive_wheels_1
      drive_wheels_2
      drive_wheels_3

      3
      fwd
      0
      1
      0

      5
      fwd
      0
      1
      0

      0
      rwd
      1
      0
      0

      1
      rwd
      1
      0
      0

      4
      4wd
      0
      0
      1

      9
      4wd
      0
      0
      1

## Binary Encoding

**Description**

Similar to One-Hot Encoding, but values of categories are stored as binary bitstrings. Each binary digit create one encoding column, i.e. if there are $n$ levels then there are $\log_2n$ features.

**Construction**

Each level is associated with its order, which is then translated into bitstrings. The encoding variables are the different valeus in the bitstring.

**When to use**

For categorical variables with many levels.

```python
encoder = ce.binary.BinaryEncoder()
res = encode_var(df[["drive_wheels"]], encoder)
print_res(res)
```

      original
      drive_wheels_0
      drive_wheels_1
      drive_wheels_2

      3
      fwd
      0
      1
      0

      5
      fwd
      0
      1
      0

      0
      rwd
      0
      0
      1

      1
      rwd
      0
      0
      1

      4
      4wd
      0
      1
      1

      9
      4wd
      0
      1
      1

## BaseN Encoding

**Description**

Base-N encoder encodes the categories into arrays of their base-N representation. A base of $1$ is equivalent to one-hot encoding (not really base-1, but useful), a base of $2$ is equivalent to binary encoding. A base of $n$ is equivalent to vanilla ordinal encoding.

**Construction**

Each level is associated with its order, which is then translated into bitstrings. The encoding variables are the different valeus in the bitstring.

**When to use**

For categorical variables with many levels.

```python
encoder = ce.basen.BaseNEncoder(base=2)
res = encode_var(df[["make"]], encoder)
print_res(res)
```

      original
      make_0
      make_1
      make_2
      make_3
      make_4
      make_5

      150
      toyota
      0
      1
      0
      1
      0
      0

      151
      toyota
      0
      1
      0
      1
      0
      0

      89
      nissan
      0
      0
      1
      1
      0
      1

      90
      nissan
      0
      0
      1
      1
      0
      1

      50
      mazda
      0
      0
      1
      0
      0
      1

      51
      mazda
      0
      0
      1
      0
      0
      1

      30
      honda
      0
      0
      0
      1
      1
      0

      31
      honda
      0
      0
      0
      1
      1
      0

      76
      mitsubishi
      0
      0
      1
      1
      0
      0

      77
      mitsubishi
      0
      0
      1
      1
      0
      0

      182
      volkswagen
      0
      1
      0
      1
      0
      1

      183
      volkswagen
      0
      1
      0
      1
      0
      1

      138
      subaru
      0
      1
      0
      0
      1
      1

      139
      subaru
      0
      1
      0
      0
      1
      1

      194
      volvo
      0
      1
      0
      1
      1
      0

      195
      volvo
      0
      1
      0
      1
      1
      0

      107
      peugot
      0
      0
      1
      1
      1
      0

      108
      peugot
      0
      0
      1
      1
      1
      0

      21
      dodge
      0
      0
      0
      1
      0
      1

      22
      dodge
      0
      0
      0
      1
      0
      1

      10
      bmw
      0
      0
      0
      0
      1
      1

      11
      bmw
      0
      0
      0
      0
      1
      1

      67
      mercedes-benz
      0
      0
      1
      0
      1
      0

      68
      mercedes-benz
      0
      0
      1
      0
      1
      0

      3
      audi
      0
      0
      0
      0
      1
      0

      4
      audi
      0
      0
      0
      0
      1
      0

      118
      plymouth
      0
      0
      1
      1
      1
      1

      119
      plymouth
      0
      0
      1
      1
      1
      1

      132
      saab
      0
      1
      0
      0
      1
      0

      133
      saab
      0
      1
      0
      0
      1
      0

      125
      porsche
      0
      1
      0
      0
      0
      0

      126
      porsche
      0
      1
      0
      0
      0
      0

      43
      isuzu
      0
      0
      0
      1
      1
      1

      44
      isuzu
      0
      0
      0
      1
      1
      1

      0
      alfa-romero
      0
      0
      0
      0
      0
      1

      1
      alfa-romero
      0
      0
      0
      0
      0
      1

      18
      chevrolet
      0
      0
      0
      1
      0
      0

      19
      chevrolet
      0
      0
      0
      1
      0
      0

      47
      jaguar
      0
      0
      1
      0
      0
      0

      48
      jaguar
      0
      0
      1
      0
      0
      0

      130
      renault
      0
      1
      0
      0
      0
      1

      131
      renault
      0
      1
      0
      0
      0
      1

      75
      mercury
      0
      0
      1
      0
      1
      1

## Simple Encoder

**Description**

Compares each level to the reference level, with the intercept as the grand mean.

**Construction**

$$x_k(i) = \begin{cases}
\frac{n-1}{n} & \text{if } k = i \\
-\frac{1}{n} & \text{otherwise}
\end{cases}$$

**When to use**

Same as dummy encoding, but when the interest is in deviations from the grand mean rather than deviations from the reference level.

```python
### Not available in Python
from simple_coding import SimpleEncoder
```

```python
encoder = SimpleEncoder()
res = encode_var(df[["drive_wheels"]], encoder)
print_res(res)
```

      original
      intercept
      drive_wheels_0
      drive_wheels_1

      3
      fwd
      1
      0.666667
      -0.333333

      5
      fwd
      1
      0.666667
      -0.333333

      0
      rwd
      1
      -0.333333
      -0.333333

      1
      rwd
      1
      -0.333333
      -0.333333

      4
      4wd
      1
      -0.333333
      0.666667

      9
      4wd
      1
      -0.333333
      0.666667

## Sum encoding / Deviation Encoding / Effect Encoding

**Description**

Sum encoding compares each group effect to the grand mean, i.e. the mean of group means (which is not the overall mean). The encoding representation is the difference of indicator variables minus the indicator for one baseline group. Sum encoding is similar to One-Hot encoding, but the interpretation of effects is different (effect relative to the grand mean vs. effect relative to a baseline group).

**Construction**

$$x_k(i) = \begin{cases}
-1 & \text{if } i = 1\\
1 & \text{if } i+1 = k\\
0 & \text{otherwise}
\end{cases}$$

**When to use**

When the main interest is in differences in average values for each level compared to the grand mean and there are sufficient observations for each level.

```python
encoder = ce.sum_coding.SumEncoder()
res = encode_var(df[["drive_wheels"]], encoder)
print_res(res)
```

      original
      intercept
      drive_wheels_0
      drive_wheels_1

      3
      fwd
      1
      0.0
      1.0

      5
      fwd
      1
      0.0
      1.0

      0
      rwd
      1
      1.0
      0.0

      1
      rwd
      1
      1.0
      0.0

      4
      4wd
      1
      -1.0
      -1.0

      9
      4wd
      1
      -1.0
      -1.0

## (Orthogonal) Polynomial Encoding

**Description**
Orthogonal polynomial coding is a form of trend analysis in that it is looking for the linear, quadratic and cubic trends in the categorical variable. Orthogonal polynomials are equations such that each is associated with a power of the variable.

**Construction**

Each encoding variable are the coefficients of the orthogonal polynomials of order $n-1$. See this [Post](https://stats.stackexchange.com/questions/105115/polynomial-contrasts-for-regression) for an explanation of the construction.

**When to use**
This type of coding system should be used only with an ordinal variable in which the levels are equally spaced.

```python
encoder = ce.polynomial.PolynomialEncoder()
res = encode_var(df[["num_cylinders"]], encoder)
print_res(res)
```

      original
      intercept
      num_cylinders_0
      num_cylinders_1
      num_cylinders_2
      num_cylinders_3
      num_cylinders_4
      num_cylinders_5

      0
      four
      1
      -1.889822e-01
      -3.273268e-01
      4.082483e-01
      0.080582
      -5.455447e-01
      0.493464

      1
      four
      1
      -1.889822e-01
      -3.273268e-01
      4.082483e-01
      0.080582
      -5.455447e-01
      0.493464

      2
      six
      1
      1.889822e-01
      -3.273268e-01
      -4.082483e-01
      0.080582
      5.455447e-01
      0.493464

      12
      six
      1
      1.889822e-01
      -3.273268e-01
      -4.082483e-01
      0.080582
      5.455447e-01
      0.493464

      4
      five
      1
      1.617449e-17
      -4.364358e-01
      -1.109626e-16
      0.483494
      -6.714569e-16
      -0.657952

      5
      five
      1
      1.617449e-17
      -4.364358e-01
      -1.109626e-16
      0.483494
      -6.714569e-16
      -0.657952

      71
      eight
      1
      3.779645e-01
      1.195122e-17
      -4.082483e-01
      -0.564076
      -4.364358e-01
      -0.197386

      72
      eight
      1
      3.779645e-01
      1.195122e-17
      -4.082483e-01
      -0.564076
      -4.364358e-01
      -0.197386

      55
      two
      1
      -5.669467e-01
      5.455447e-01
      -4.082483e-01
      0.241747
      -1.091089e-01
      0.032898

      56
      two
      1
      -5.669467e-01
      5.455447e-01
      -4.082483e-01
      0.241747
      -1.091089e-01
      0.032898

      49
      twelve
      1
      5.669467e-01
      5.455447e-01
      4.082483e-01
      0.241747
      1.091089e-01
      0.032898

      18
      three
      1
      -3.779645e-01
      9.521795e-17
      4.082483e-01
      -0.564076
      4.364358e-01
      -0.197386

## Helmert Encoding

**Description**

Helmert coding compares each level of a categorical variable to the mean of the subsequent levels. The first contrast compares the mean of the dependent variable for the second level with the mean of the dependent variable for the first level. The second contrast compares the mean for the third level with the mean for the first two levels, etc.

**Construction**

$$x_k(i) = \begin{cases}
-1 & \text{if } i < k\\
k & \text{if } i = k\\
0 & \text{otherwise}
\end{cases}$$

The representation for level k has -1 for each level before k and then k as value for the current level, starting at *k=2* up to *k=n*.

**When to use**

When levels of a categorical variable are ordered from lowest to highest or from smallest to largest.

```python
encoder = ce.helmert.HelmertEncoder()
res = encode_var(df[["body_style"]], encoder)
print_res(res)
```

      original
      intercept
      body_style_0
      body_style_1
      body_style_2
      body_style_3

      3
      sedan
      1
      0.0
      2.0
      -1.0
      -1.0

      4
      sedan
      1
      0.0
      2.0
      -1.0
      -1.0

      2
      hatchback
      1
      1.0
      -1.0
      -1.0
      -1.0

      9
      hatchback
      1
      1.0
      -1.0
      -1.0
      -1.0

      7
      wagon
      1
      0.0
      0.0
      3.0
      -1.0

      28
      wagon
      1
      0.0
      0.0
      3.0
      -1.0

      69
      hardtop
      1
      0.0
      0.0
      0.0
      4.0

      74
      hardtop
      1
      0.0
      0.0
      0.0
      4.0

      0
      convertible
      1
      -1.0
      -1.0
      -1.0
      -1.0

      1
      convertible
      1
      -1.0
      -1.0
      -1.0
      -1.0

```python
df["body_style"].unique()
```

    array(['convertible', 'hatchback', 'sedan', 'wagon', 'hardtop'],
          dtype=object)

## Backward Difference Encoding

**Description**

Similar to Helmert encoding, but differences are taken with respect to the prior adjacent level (not all prior levels).

**Construction**

$$x_k(i) = \begin{cases}
-\frac{n-k}{n} & \text{if } i \leq k\\
\frac{n}{k} & \text{otherwise}
\end{cases}$$

**When to use**

When levels of a categorical variable are ordered from lowest to highest or from smallest to largest, but the interest is in step-wise differences.

```python
encoder = ce.backward_difference.BackwardDifferenceEncoder()
res = encode_var(df[["body_style"]], encoder)
print_res(res)
```

      original
      intercept
      body_style_0
      body_style_1
      body_style_2
      body_style_3

      0
      convertible
      1
      -0.8
      -0.6
      -0.4
      -0.2

      1
      convertible
      1
      -0.8
      -0.6
      -0.4
      -0.2

      2
      hatchback
      1
      0.2
      -0.6
      -0.4
      -0.2

      9
      hatchback
      1
      0.2
      -0.6
      -0.4
      -0.2

      3
      sedan
      1
      0.2
      0.4
      -0.4
      -0.2

      4
      sedan
      1
      0.2
      0.4
      -0.4
      -0.2

      7
      wagon
      1
      0.2
      0.4
      0.6
      -0.2

      28
      wagon
      1
      0.2
      0.4
      0.6
      -0.2

      69
      hardtop
      1
      0.2
      0.4
      0.6
      0.8

      74
      hardtop
      1
      0.2
      0.4
      0.6
      0.8

## Frequency / Count Encoding

**Description**

Frequency encoding replaces the levels of a categorical variable with their absolute or relative frequency.

**Construction**

$$x(i) = \frac{N_i}{N}$$

**When to use**

When common or uncommon levels have similar influences.

```python
encoder = ce.count.CountEncoder(normalize=True)
res = encode_var(df[["drive_wheels"]], encoder)
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      0.585366

      5
      fwd
      0.585366

      0
      rwd
      0.370732

      1
      rwd
      0.370732

      4
      4wd
      0.043902

      9
      4wd
      0.043902

## Hashing

**Description**

Hashing converts categorical variables to a higher dimensional space of integers, where the distance between two vectors of categorical variables in approximately maintained the transformed numerical dimensional space.

**Construction**

Any hash funtion from the `hashlib` package.

**When to use**

This method is advantageous when the cardinality of categorical is very high.

```python
encoder = ce.hashing.HashingEncoder()
res = encode_var(df[["drive_wheels"]], encoder)
print_res(res)
```

      original
      col_0
      col_1
      col_2
      col_3
      col_4
      col_5
      col_6
      col_7

      0
      rwd
      0
      0
      0
      0
      0
      1
      0
      0

      1
      rwd
      0
      0
      0
      0
      0
      1
      0
      0

      3
      fwd
      0
      0
      0
      0
      1
      0
      0
      0

      5
      fwd
      0
      0
      0
      0
      1
      0
      0
      0

      4
      4wd
      0
      0
      0
      0
      0
      0
      1
      0

      9
      4wd
      0
      0
      0
      0
      0
      0
      1
      0

# Supervised Encoding Methods / Bayesian Encoders

## Target / Mean / Impact / Likelihood Encoding

**Description**

Target encoding replaces each category with the average value of all ovservations for that category, mixed with a prior.

For the case of *categorical target*: features are replaced with a blend of posterior probability of the target given particular categorical value and the prior probability of the target over all the training data.

For the case of *continuous target*: features are replaced with a blend of the expected value of the target given particular categorical value and the expected value of the target over all the training data.

**Construction**

$$x(i) = \bar{y} (1-s) + s \bar{y}_i$$
where $s$ is a smoothing parameter calculated as:
$$s=\frac{1}{1+\exp{\left(-\frac{n-mdl}{a}\right)}}$$
with 'mdl' standing for 'min data in leaf' and $a$ is a regularization parameter. The mdl defines a threshold where prior, i.e. the mean of the target, and target mean for a given category value have the same weight.

**When to use**

This encoding method brings out the relation between similar categories, but the connections are bounded within the categories and target itself. The advantages of the mean target encoding are that it does not affect the volume of the data and helps in faster learning. Target encoding is powerful in prediction tasks, but runs the risk of target leakage. The leakage can be controlled via regularization, data augmentation by adding noise to the encoding representation, or through double validation.

```python
encoder = ce.target_encoder.TargetEncoder()
res = encode_var(df[["drive_wheels"]], encoder, df['price'])
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      9244.779661

      5
      fwd
      9244.779661

      0
      rwd
      19757.613333

      1
      rwd
      19757.613333

      4
      4wd
      10243.702296

      9
      4wd
      10243.702296

## James-Stein Encoding

**Description**

Target encoding based on the James-Stein mean estimator, rather than the sample mean. The idea is to improve the estimation of the category's mean target by shrinking them towards the central average.

**Construction**

$$x(i) = \bar{y} B + (1-B) \bar{y}_i$$
where $B$ is a shrinkage parameter. A common value is
$$B=\frac{Var\left[y_k\right]}{Var\left[y_k\right]+Var\left[y\right]}$$
but the value could also be set via cross-valdiation. The intuition behind the equation is that if the mean estiamte of a category is uncertain (high variance), then stronger shrinkage should be applied.

**When to use**

Same as for target encoder.

```python
encoder = ce.james_stein.JamesSteinEncoder()
res = encode_var(df[["drive_wheels"]], encoder, df['price'])
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      9244.779661

      5
      fwd
      9244.779661

      0
      rwd
      19757.613333

      1
      rwd
      19757.613333

      4
      4wd
      10241.000000

      9
      4wd
      10241.000000

## M-estimator Encoding

**Description**

A simplified version of the target encoder as it has only one hyerparameter.

**Construction**

$$x(i) = \frac{N_i + m \times \bar{y}}{\bar{y}_i + m}$$
where $m$ is a smoothing parameter.

**When to use**

Same as for target encoder.

```python
encoder = ce.m_estimate.MEstimateEncoder()
res = encode_var(df[["drive_wheels"]], encoder, df['price'])
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      9278.076717

      5
      fwd
      9278.076717

      0
      rwd
      19671.422755

      1
      rwd
      19671.422755

      4
      4wd
      10570.569928

      9
      4wd
      10570.569928

## Leave One Out Encoding

**Description**

The encoding is calculated for each observation $j$ by calculating the average value of the target of all observations with the same target value as observation $j$ except $j$.

**Construction**

$$ x_k^{(j)} = \frac{a}{b}\frac{\sum_{i\neq j}(y_i(x_i==k)-y_j}{\sum_{i\neq j}x_i==k}$$

**When to use**

Same as for target encoder.

```python
encoder = ce.leave_one_out.LeaveOneOutEncoder()
res = encode_var(df[["drive_wheels"]], encoder, df['price'])
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      9244.779661

      5
      fwd
      9244.779661

      0
      rwd
      19757.613333

      1
      rwd
      19757.613333

      4
      4wd
      10241.000000

      9
      4wd
      10241.000000

## Catboost Encoding

**Decription**

Catboost is an improvement over the leave-one-out encoder. It is intended to overcome the target leackage problems.

**Construction**
Same as LOO encoder, but the sums only range up to the current observations:
$$ x_k(i) = \frac{a}{b}\frac{\sum_{j\neq i, j\leq k}(y_j(x_j==k)-y_i}{\sum_{j\neq i, j\leq k}x_j==k}$$

**When to use**

```python
encoder = ce.cat_boost.CatBoostEncoder()
res = encode_var(df[["drive_wheels"]], encoder, df['price'])
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      9278.076717

      5
      fwd
      9278.076717

      0
      rwd
      19671.422755

      1
      rwd
      19671.422755

      4
      4wd
      10570.569928

      9
      4wd
      10570.569928

## Weight of Evidence Encoder

**Description**

The encoding is a measure of the *strength* that seperates effect from no effect. The encoding can only be used for binary targets.

**Construction**

The encoding is calculated from a modified odds ratio, which is intended to prevent target leakage:
$$ \begin{align}
numerator =& \ \frac{N_i+a}{N_i\bar{y}_i + 2a} \\
denominator =& \ \frac{N-N_i+a}{N\bar{y}-N_i\bar{y}_i  2a} \\
x(i) =& \ \log\left(\frac{numerator}{denominator}\right)
\end{align}
$$

**When to use**

For binary targets

```python
encoder = ce.woe.WOEEncoder()
res = encode_var(df[["drive_wheels"]], encoder, df['fuel_type']=='gas')
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      0.275848

      5
      fwd
      0.275848

      0
      rwd
      -0.435318

      1
      rwd
      -0.435318

      4
      4wd
      0.162519

      9
      4wd
      0.162519

## Probability Ratio Encoding

**Description**

Probability Ratio Encoding is similar to Weight Of Evidence encoding, but the encoding is based on the ratio of probabilities of the positive to the negative class.

**Construction**

$$x(i) = \frac{y^+/N}{y_k^-/N}=\frac{y_k^+}{y_k^-}$$
This is the same as WoE encoding with no regularization, i.e. $a=0$.

**When to use**

Same as weight of evidence encoding.

```python
encoder = ce.woe.WOEEncoder(regularization=0.)
res = encode_var(df[["drive_wheels"]], encoder, df['fuel_type']=='gas')
print_res(res)
```

      original
      drive_wheels

      3
      fwd
      0.287682

      5
      fwd
      0.287682

      0
      rwd
      -0.448132

      1
      rwd
      -0.448132

      4
      4wd
      inf

      9
      4wd
      inf

# Other encodings

* DRACuLa
* Reverse Helmert
* Forward Differences
* Thermometer Encoder

# Cheat Sheet

# Exercises

**Exercise 1**: Calculate the regressions of 'num_cylinders' on price for each possible encoding. What is the correct interpretation for the coefficients?

**Exercise 2**: Try to find the best encoding for each variable to maximize the generalization performance of a linear regression model to predict the price of a car.


# Python 101

**ENGSCI233: Computational Techniques and Computer Systems**

*Department of Engineering Science, University of Auckland*

The purpose of this notebook is to give you the **very basics** of Python (and computer code in general). You will not be an expert by the end. But you will be on [The Path](https://zen-of-python.info/).

When writing computer code, there are ***rules*** and there are ***conventions***.

- If you break a ***rule***, the code will not work.
- If you break a ***convention***, someone, somewhere puts a mark against your name in a book. At the end, you will be called to account.

Some Python rules:

- ***Syntax*** - we have *very* precise expectations about how you write computer code. If you type an opening bracket, `(`, you must close it again later, `)`. Some control structures are terminated by a colon, `:` - if you omit this, the code will not work. ***Yes, learning syntax for a new code is very pedantic and a total pain.*** But you have to do it anyway, and at least Python returns readable error messages to help you understand your missteps...
- ***Indentation*** - Python uses this to determine when control structures begin and end. More on what a **control structure** is later.

Some Python conventions:

- ***Commenting*** - it is helpful for the poor soul who has to read your poorly written Python (sometimes that is you, weeks or months later) if you have included little **'sign-posts'** in the code, articulating what you are doing. These are called comments. They begin with a `#` symbol, after which you can write whatever you like and it will not be executed as a command.
- ***Sensible variable names*** - relating to the thing the variable represents but not too long. For example, if a variable contains the mean temperature, then `Tmean` is a sensible variable name, where as `the_mean_temperature_of_the_profile` or `a1` are not sensible variable names.

Hang-on, what's a **variable**?

**Execute the cell below by clicking inside it and hitting Ctrl+Enter**

```python
# this is a comment, nothing happens when Python reads this line
hello_string = 'Hello, world'
print(hello_string)
```

The snippet of Python code above does a number of things.

1. The first **line** is a comment - Python sees the `#` symbol and ignores everything that follows **on that line**.
2. The second line creates a **variable** called `hello_string` and **assigns to it** the ***value*** `'Hello, world'`.
3. The third line uses a **function** called `print` to display the value of `hello_string` to the screen. `hello_string` was **passed** to `print` as an **input** or **argument**.

Ooooooh. LOTS of terminology there. Let's list and define the terms:

- ***line*** - just as you read a book one sentence at a time, Python executes a computer program one line at a time. If Python is executing the 4th line of a program, it *will* have knowledge of the three lines preceding it, *but no* knowledge of the lines that follow.
- ***variable*** - think of this as a 'container' inside the computer code. A variable has a **name** (in this case, `hello_string`) and a **value**.
- ***value*** - the thing sitting inside the 'container', in this case it is `'Hello, world'`, which is a particular **type** of variable called a string.
- ***type*** - a classification of the variable, e.g., `'hi'` and `'Hello, world'` are both *strings*, `3.2`, `5.0` and `-0.12e5` are *floats*, and `2`, `-3412` and `0` are *integers*. There are other types, we will get to some of them later.
- ***assign to*** - the act of giving a *value* to a *variable*, usually accomplished by an 'equals sign', e.g., `variable = value`.
- ***function*** - a sequence of Python commands, written down somewhere else, to achieve some small (or large) task. Some functions are given to you as part of Python and its modules. Others you will have to define yourself. Functions inputs are given inside of **round brackets**.
- ***input/argument*** - a variable that is used by a function as it executes its tasks.

## Learning by doing

Obviously, the best way to master any new skill is to practice it. Work your way through the cells and exercises below to grow your Python foundation.

## 1 We can use Python as a glorified calculator

In the example below, we create a few variables, assign them some simple numbers, and confirm that Python can manage basic arithmetic.

```python
a = 2           # a variable named a, assigned a value of 2, which is of type 'integer'
b = 3
c = a+b         # adding two variables together to create a third one
print(c)

# Can you make a change? Define the variable d as the sum of a and b and c. Print it out.
```

See also

```python
print(a-b)      # Python does subtraction
print(a*b)      # and multiplication
print(a/b)      # and division
print(a**b)     # and exponentiation

# Can you make a change? Print the value of b minus a, instead of a minus b .
```

There are heaps of other mathematical operations we can perform with Python, many of which are made available through the NumPy module.

```python
import numpy as np
print(np.sin(a))
```

In the cell above we **imported** `numpy` and told Python to make the module available as a variable called `np`. Then, if I want to use one of its special functions, I use the **syntax** `module.function`.

Some more examples below

```python
print(np.cos(b))
print(np.log(a))
print(np.log10(b))
print(np.sinh(a))
print(np.arctan2(a,b))
```

Not sure what a particular function does? Write it down and replace the round brackets with a question mark `?`

```python
np.arctan2?

# what does np.exp do?
```

#### <homework>

Calculate viscosity of magma according to the forumula given by [Costa et al. [2007]](http://onlinelibrary.wiley.com/doi/10.1029/2008GC002138/abstract)

\begin{equation}
\eta(\phi) = \frac{1+\varphi^\delta}{\left[1-F(\varphi, \varepsilon, \gamma)^{B\cdot\phi_*}\right]},\quad\text{where}\quad F=(1-\xi)\cdot\text{erf}\left[\frac{\sqrt{\pi}}{2\cdot(1-\xi)}\varphi\cdot(1+\varphi^\gamma)\right]\quad\text{with}\quad\varphi=\frac{\phi}{\phi_*}.
\end{equation}

```python
# parameters
phi = 0.5
phi_star = 0.66
pi = 3.14159
xi = 0.0009
gamma = 9.8
B = 2.5          # Einstein coefficient
delta = 1.3

# note, the error function is given by erf(x)
from scipy.special import erf           # call this as a function, e.g., erf(2)

# **complete the code below**
# uncomment the partially written code below
# calculate viscosity in three steps
# - calculate new variable phi_scaled
# - calculate F
# - calculate viscosity

#phi_scaled = phi/phi_star

#F = ____

#print(visc)

```

***How does increasing each of the parameters, $\phi$, $\phi_*$, $\xi$ (`xi`), $\gamma$, $B$, $\delta$, change the viscosity, $\eta$?***

#### </homework>

## 2 Whoops!

I have replicated the very first example in this notebook, **except** that the variable names have been changed AND I have **reversed the order** of the commands.

***Run the cell below***

```python
# this is a comment, nothing happens when Python reads this line
print(night_string)
night_string = 'Good night, world'
```

What you see above is a Python error message. ***Reading these messages and using them to debug your code is an invaluable skill.***

There are two key pieces of information:

1. **WHERE** is the error. In this case, the problem occurs on Line 2. We know this, because there is an arrow pointing to it, i.e., `----> 2 print(night_string)`
2. **WHAT** is the error. In this case, it is that we are attempting to use a variable called `night_string` before it has been created. We know this from the very last piece of information in the error message, i.e., `name 'night_string' is not defined`.

Deciphering these errors **gets easier with practice**, because there are a handful of easy-to-make mistakes that will crop up frequently.

Another tip, sometimes the error is not on the line that `---->` points to, but instead the command immediately above or below it.

***Fix the error above so that the cell runs correctly.***

To do this, swap the order of the two commands. The key here is that `night_string` has to be defined (`night_string = 'Good night, world'`) before it can be used in a function (`print(night_string)`).

***See if you can fix the errors in the code below.***

```python
# calculate the harmonic average of the numbers a and b
hamonic_avg = 2/(1/a+1/b
a = 2
b = 3
print(hamonic_avg)
```

## 3 Lists and arrays

There are these things called lists. They are as you would expect, literally a list of ordered items. An example is given below.

```python
my_list = [1,2,3]                                            # this list has three items
empty_list = []                                              # this list has no items
mixed_list = [-0.2, 300, 'a string', 5.3, True, my_list]     # this list has four items of different types, one is another list

print(mixed_list)
```

Note, lists use square brackets [] whereas functions use round brackets () - ***syntax!***

We can **access** the items of a list by passing an **index** to the variable name. The indices begin at 0 (the first item in a list) and increment by 1. For example

```python
print(my_list[0])
```

```python
print(my_list[1]+my_list[2])
```

We can also use indices to 'count backward' from the end of the list, for example

```python
print(my_list[2], my_list[-1])                 # these access the same element of the list
print(my_list[1], my_list[-2])                 # so do these
```

We can pull out a smaller list from the list using **index slicing**. This uses the colon, `:`, which essentially says 'and everything in between'. For example,

```python
print(mixed_list)
print(mixed_list[1:3])                         # all items between index 1 and 3 NOT including 3
print(mixed_list[:3])                          # all items from the START of the list up to index 3 NOT including 3
print(mixed_list[3:])                          # all items from index 3 up to the END of the list
```

Or we can even reverse a list.

```python
print(mixed_list[::-1])
```

An **array** is a list of numbers. It has special properties

```python
T = np.array([1,2,3])                          # array of 1st order temperature measurements
dT = np.array([0.1, 0.1, -0.1])                # array of corresponding 2nd order temperature deviation
print(T+dT)                                   # total temperature change
print(T*dT)                                   # product of 1st and 2nd order effects
```

Make sure your arrays are the same length though! ***Execute the cell below and interpret the error message***

```python
b1 = np.array([4,5,6])
b2 = np.array([0.1,-0.1])
print(b1+b2)
```

## 4 Doing things over and over

Scripting is useful to automate tasks that have to be performed over and over. For instance, consider calculating the value of $e$ using the exponential series

$$ e = 1+\frac{1}{1!}+\frac{1}{2!}+\frac{1}{3!}+\frac{1}{4!}+\cdots $$

We could do it the long way...

```python
from math import factorial
e = 1 + 1/factorial(1) + 1/factorial(2) + 1/factorial(3) + 1/factorial(4) + 1/factorial(5)    # I got tired and gave up here
print(e)
```

Or we could write a **for loop** to do it for us.

```python
e = 1        # the initial value
for i in range(1,21):        # create a variable called i, initially assign it the value of 1, then 2, then 3, ... then 20
    e = e + 1/factorial(i)   # each time the loop 'goes around', execute the commands 'in the loop'
print(e)
```

What's happening above? The loop we have written is identical to the sequence of commands below

```python
e = 1
i = 1
e = e + 1/factorial(i)
i = 2
e = e + 1/factorial(i)
i = 3
e = e + 1/factorial(i)
i = 4
e = e + 1/factorial(i)
i = 5
e = e + 1/factorial(i)
i = 6
e = e + 1/factorial(i)
i = 7
e = e + 1/factorial(i)
i = 8
e = e + 1/factorial(i)
i = 9
e = e + 1/factorial(i)
i = 10
e = e + 1/factorial(i)
i = 11
e = e + 1/factorial(i)
i = 12
e = e + 1/factorial(i)
i = 13
e = e + 1/factorial(i)
i = 14
e = e + 1/factorial(i)
i = 15
e = e + 1/factorial(i)
i = 16
e = e + 1/factorial(i)
i = 17
e = e + 1/factorial(i)
i = 18
e = e + 1/factorial(i)
i = 19
e = e + 1/factorial(i)
i = 20
e = e + 1/factorial(i)
print(e)
```

but with A LOT less repetition. Can you see which command is 'inside the loop' and gets executed over and over? Can you see which variable has its value changed with each iteration of the loop?

***In the cell below, write a for loop to calculate the "sum of squares" of the list of pressure observations.***

```python
a = [7,3,-2,0,4.5,9.0,-1,-1,15,0.1]              # pressure deviations (in kPa), from p0 = 101 kPa
sum_squares = 0
# **your code here**
```

## 5 Making the computer program a little bit smart

Humans are allegedly an intelligent species. One aspect of this intelligence is the ability to **see how things are and act accordingly**. What?

As an example, to cross a busy street, you first check for cars. ***If*** there is a car coming, you do not cross, ***else*** you do.

We can write this in Python.

```python
street = 'busy'
if street == 'busy':
    print('dont cross the street')
else:
    print('cross the street')
```

The `if` statement above evaluates a **condition** (essentially a question asked of Python, is the 'value' of `street` equal to `'busy'`?).

If the condition evaluates to `True`, then the command `print('dont cross the street')` is executed. If it evaluates to `'False'`, then the command `print('cross the street')` is executed instead.

The key here though is that it is **one or the other** and the outcome depends on stuff that happened earlier in the code (in this case, on the first line when I assigned a value to the variable `street`).

Have another play with the example below.

```python
street = 'busy'
attempt_to_cross = True     # this is a type of variable called a 'boolean' - it is either True or False, no other options

if street == 'busy' and attempt_to_cross is False:      # the first condition to check
    print('dont cross, good decision')
elif street != 'busy' and attempt_to_cross is True:     # if the first condition is False, then check this one
    print('safe to cross, well done')
elif street == 'busy' and attempt_to_cross is True:     # if the first AND second conditions are False, check this one
    print('you dead')
```

In the example above, we see that the **special statement**, `and`, allows us to evaluate two conditions at once and require that they BOTH be true. Alternatively, we could have used `or`, which allows either one, or both to be true.

***Make changes to the cell above to generate a "successful crossing" outcome.***

The example below demonstrates different conditions and combinations of conditions:

```python
print('1',True and True)
print('2',True and False)
print('3',True or False)
print('4',False or False)
print('5',not False)
print('6', False or not False)
print('6b', False or not (True or not True))
print('7', 3>4)
print('8', 3<4)
print('9', 4<4)
print('10', 4<=4)
print('11', 4==4)
density = 2650.
print('12', density>2400. and density<2800.)
print('13', 2400.<density<2800.)
```

#### <neat> List Comprehension

This concept is subtle, and you should ideally be already quite comfortable with for loops before experimenting with [list comprehensions](http://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/).

Compare the following snippets of codes and verify that they generate the same output.

```python
a1 = []                 # an empty list
for i in range(10):     # for all integers up to (but not including) ten
    a1.append(i**2)           # append to the end of the list, the integer**2
print(a1)

a2 = [i**2 for i in range(10)]   # the same list, now constructed using list comprehension (one line!)
print(a2)
```

"*Can you make it fancier?*"

Oh, yes. Consider the following, equivalent, list transformations.

```python
b1 = []
for ai in a1:
    if (ai%2 == 1):        # if the value is odd, append it to the new list
        b1.append(ai)      #  (% is the 'remainder' operator, i.e., what is the remainder when ai is divided by 2)
    else:
        b1.append(-ai)     # otherwise, make the value negative and append it
print(b1)

b2 = [ai if ai%2 == 1 else -ai for ai in a1]    # (one line!)
print(b2)

# or if there's not 'else' case to worry about, the if condition comes at the end
# e.g., just keep the odd numbers
# b3 = [ai for ai in a1 if ai%2 == 1]
# print(b3)
```

#### </neat>

### 5.1 Making a loop a little bit smart

We have [seen](python101.ipynb#0.4-Doing-things-over-and-over) how a for loop can be used to repeat a computation over and over a **fixed number of times**. But what if we don't know ahead of time how many times a loop should go around?

Taking the example from before, say we want to calculate the value of $e$ to a fixed level of accuracy, say 4 decimal places. That is, as soon as the next term in the series - $1/i!$ - is smaller than $10^{-5}$, we should stop the loop.

To do this, we can use a **while loop**, which makes use of a condition to determine when the loop should stop.

```python
from math import factorial
e = 1        # the initial value
i = 1        # unlike the for loop, we have to initialise and increment the counter manually
de = 1/factorial(i)   # calculate the update to e

# initialise the while loop with a condition, each time the loop goes around the value of de changes,
# and the condition is rechecked. The loop stops when the condition evaluates to False.

while de > 1.e-5:                 # check the condition
    e = e + de                    # update the estimate of e
    i = i + 1                     # increment the counter
    de = 1/factorial(i)           # calculate a new value for de
    print('i =',i,', e =', e)     # print progress so far

# MAKE A CHANGE
# - modify the loop so that it exits when e has been calculated to 6 decimal places
# - modify the loop so that it exits when e has been calculated to 8 decimal places, or when 12 terms
# have been calculated, whichever comes first
```

## 6 Classes, objects, attributes and methods

Python is an [**object-oriented**](https://en.wikipedia.org/wiki/Object-oriented_programming) programming language. Most people initially become familiar with **procedural, structured programming**, computer code organised into a logical procession of statements, loops, control blocks and functions. We can build on that understanding and introduce the idea of **objects, with attributes and methods**.

The best introduction is perhaps a direct demonstration.

**Execute the cell below to define a new *Class* for a geothermal well.**

```python
class Well(object):          # defining a class is similar to defining a function in that there is precise syntax
    ''' An object to represent an arbitrary Well.
    '''
    def __init__(self):
        ''' Define what properties the object should have when it is brought into existence
        '''
        self.location = []                  # these are called attributes, we have defined 3: location, name and depth
        self.name = 'unnamed'               # they are like variables, but they *belong* to the object
        self.depth = None                   # we can access and change them using the notation OBJECT.ATTRIBUTE
```

Think of the **Class** as a new "kind" or a "type" of object (along with floats, integers, strings, and arrays). Much like a function, once it is defined, we can begin to use it.

```python
# Create an *instance* of the Well object. A duplicate, to be modified independently of other instances.
well1 = Well()               # note the use of brackets in creating the object
well2 = Well()               # now we have two 'instances' of the Well object

print(well1.name, well2.name)
```

We can modify their **attributes** in the usual way a variable is modified.

```python
# let's make the first object personal (change for yourself)
well1.location = [0.5, 8.2]
well1.name = 'tvz25'
well1.depth = 2305.

# let's make the second object a beloved pet (change for yourself)
well2.location = [6.3, -2.4]
well2.name = 'tvz32'
well2.depth = 1680.

print(well1.name, well2.name)               # verifying we have changed the attributes
print(well1.depth > well2.depth)            # verifying attributes are subject to the usual computer arithmetic
```

#### <neat> `__repr__`

Try **printing** an object directly.

```python
print(well2)
```

The standard output is not very **informative**...

We can modify this by including a **specialised method** called '__repr__' in the class definition

```python
class Well(object):
    ''' An object to represent an arbitrary Well.
    '''
    def __init__(self):
        ''' Define what properties the object should have when it is brought into existence
        '''
        self.location = []
        self.name = 'unnamed'
        self.depth = None
    def __repr__(self):
        ''' What information to print to the screen when the object is printed.
        '''
        return '{:s}'.format(self.name)

# create and print the new object
well2 = Well()
well2.location = [6.3, -2.4]
well2.name = 'tvz32'
well2.depth = 1680.
print(well2)
```

#### </neat>

An object's attributes are **specific** to it. For example,

```python
print(well1.name)                   # the 'name' *attribute* has been defined for the well1 object
print(well2.name)                   # the 'name' *attribute* has been defined for the well2 object
print(name)                         # 'name' on its own is a *variable* that has yet to be defined
```

In much the same way that attributes are just variables **specific to an object**, we can define **methods**, which are functions **specific to an object**.

***Execute the cell below to update the Well class with a method `bhp` that computes bottomhole pressure.***

```python
class Well(object):
    ''' An object to represent an arbitrary Well.
    '''
    def __init__(self):
        ''' Define what properties the object should have when it is brought into existence
        '''
        self.location = []
        self.name = 'unnamed'
        self.depth = None
    def __repr__(self):
        ''' What information to print to the screen when the object is printed.
        '''
        return '{:s}'.format(self.name)

    def bhp(self, density=1000., whp=0.1):
        ''' Computes the bottomhole pressure (in MPa) for given fluid density and wellhead pressure.
        '''
        # set gravity
        g = 9.81
        # compute pressure due to water column, in Pa
        dP = density*self.depth*g
        # convert to MPa
        dP = dP/1.e6
        # add wellhead pressure
        bhp = whp + dP
        # return value
        return bhp
```

***Execute the cell below to call the `bhp` method for a range of inputs.***

```python
# define a well
well2 = Well()
well2.location = [6.3, -2.4]
well2.name = 'tvz32'
well2.depth = 1680.

# compute downhole pressure assuming density of 1000. and whp of 12 bar (1.2 MPa)
bottomholepressure = well2.bhp(density=1000., whp=1.2)
print('at 1000 kg/m^3 and 12 bar WHP, the BHP is', bottomholepressure)

# compute downhole pressure with density of 980 and atmospheric pressure (default 0.1)
bottomholepressure = well2.bhp(density=980.)
print('at 980 kg/m^3 and open, the BHP is', bottomholepressure)

# compute downhole pressure using all defaults
bottomholepressure = well2.bhp()
print('at 1000 kg/m^3 and open, the BHP is', bottomholepressure)
```

# What now?

There you go. That's your crash course in computer coding with Python. Obviously, we're only scraping the surface, and you shouldn't expect to really feel like you "*know what you're doing*" until you've been writing Python for a month or so.

The rewards though. So great.


Table of Contents

1  Question

1.1  Answer

2  Question

2.1  Answer

3  Question

3.1  Answer

4  Question

4.1  Answer

# Question
**Question 1.1** Write a Python Program(with class concepts) to find the area of the triangle using the below formula.

`area = (s*(s-a)*(s-b)*(s-c)) ** 0.5`

Function to take the length of the sides of triangle from user should be defined in the parent class and function to calculate the area should be defined in subclass.

**BreakDown points**

`Class Parent : Should Read inputs from User fr the sides of triangle`

`Class Child  : Should Calculate the Area Of Triangle with Variables from parent`

**Heron's Formula**
\begin{align}
s = \frac{a + b + c}{2}
\end{align}

\begin{align}
Area Of Triangle = \sqrt{(s*(s-a)*(s-b)*(s-c))}
\end{align}

## Answer

```python
# Parent Class : Will get the input from User and display
class LengthOfTheSides():
    def __init__(self):
        print('Please provide length of the sides of triangle : \n')
        self.a = float(input('side1 : '))
        self.b = float(input('side2 : '))
        self.c = float(input('side3 : '))

    def __repr__(self):
        return 'Provided Values are ...\nside1 = {}, side2 = {}, side3 = {}\n\n'.format(str(self.a),str(self.b), str(self.c))

# Child Class : Will Calculate the Area Of Triangle
class Area(LengthOfTheSides):
    def __init__(self):
        LengthOfTheSides.__init__(self)

    def AreaOfTriangle(self):
        s, a, b, c = (self.a + self.b + self.c)/2, self.a, self.b, self.c
        return (s*(s-a)*(s-b)*(s-c)) ** 0.5

AreaOfTriangle = Area()
print('\nArea Of Triangle : {}'.format(AreaOfTriangle.AreaOfTriangle()))
```

    Please provide length of the sides of triangle :

    side1 : 5
    side2 : 4
    side3 : 3

    Area Of Triangle : 6.0

# Question

**Question 1.2** Write a function filter_long_words() that takes a list of words and an integer n and returns the list of words that are longer than n.

## Answer

```python
def filter_long_words(words_to_check, max_length):
    # Check Provided Arg[0] is List
    if not isinstance(words_to_check, list):
        raise Exception('Please enter arg[0] as list of words')
    # Check Provided Arg[1] is int
    if not isinstance(max_length, int):
        raise Exception('Please enter arg[1] as int')

    return [i for i in words_to_check if len(i)>max_length]
```

```python
# Correct Arguments Passed

filter_long_words(['Vignesh', 'Vicky'], 5)
```

    ['Vignesh']

```python
# Arg[0] is String

filter_long_words('Vignesh', 5)
```

```python
# Arg[1] is String

filter_long_words(['Vignesh', 'Vicky'], "5")
```

# Question

**Question 2.1** Write a Python program using function concept that maps list of words into a list of integers representing the lengths of the corresponding words.

**Hint:** If a list [ ab,cde,erty] is passed on to the python function output should come as [2,3,4]. Here 2,3 and 4 are the lengths of the words in the list.

## Answer

```python
def words_to_int(words_to_check):
    # Check Provided Arg[0] is List
    if not isinstance(words_to_check, list):
        raise Exception('Please enter arg[0] as list of words')

    return [len(i) for i in words_to_check]
```

```python
words_to_int(['Vignesh', 'Vicky'])
```

    [7, 5]

# Question

**Question 2.2** Write a Python function which takes a character (i.e. a string of length 1) and returns True if it is a vowel, False otherwise.

## Answer

```python
def check_vowels(char_to_check):
    vowels = ('a', 'e', 'i', 'o', 'u')

    # Check Provided Arg[0] is a Single Charecter String
    if isinstance(char_to_check, str) and len(char_to_check) == 1:
        if char_to_check.lower() in vowels:
            return True
        else:
            return False
    else:
        raise Exception('Please enter arg[0] as a Single Charecter String')
```

```python
check_vowels('V')
```

    False

```python
check_vowels('I')
```

    True


# pAPRika tutorial 5 - APR/Amber with Plumed restraints

In this tutorial, we will perform APR calculations for the butane (BUT)--cucurbit[6]uril (CB6) host-guest system. This is a repeat of [Tutorial 1](01-tutorial-cb6-but.ipynb) using `Plumed`-based restraints and the `AMBER` MD engine. `Plumed` is a plugin for MD codes that can analyze trajectories and perform free-energy calculations on collective variables. It is a versatile plugin that can interface with a number of MD engines. Here, we will go through the process of converting APR restraints constructed with pAPRika to a `Plumed` file and run a short calculation with `sander`.

## Initialize

### Before you start

We will run the simulations in this tutorial using `sander` (*Ambertools*) and `Plumed`. Both of these should be installed in your `conda` environment if you installed *pAPRika* though the `conda` route. However, for `Plumed` to work with `sander` we first need to make sure the `PLUMED_KERNEL` environment variable is loaded (the library is called `libplumedKernel.so`). *pAPRika* should load the `Plumed` kernel automatically but let's make sure it is loaded and run the cell below.

```python
import os
'PLUMED_KERNEL' in os.environ.keys()
```

    False

If it does not exists we will load the environment in this Jupyter Notebook. Since `Plumed` is installed through `conda` the kernel will be located in your conda environment library folder. If you are running this on a Mac replace the kernel library in the cell below to `libplumedKernel.dylib`. If you compiled `Plumed` yourself and then you will need to change the path below.

```python
os.environ['PLUMED_KERNEL'] = f"{os.environ['CONDA_PREFIX']}/lib/libplumedKernel.so"
```

For running `Plumed` with Amber outside of this notebook it might be better to export the `PLUMED_KERNEL` variable into your `.bashrc` file.

**Note:** we can run `Plumed` with `AMBER` versions 18 and 20, but version 18 requires you to patch the source code first and recompile. Older versions of Amber are not supported. See the `Plumed` documentation for more details https://www.plumed.org/doc-v2.6/user-doc/html/index.html.

🔵 Since we have a prepared the host-guest-dummy setup from the first tutorial, we will skip the initial tleap steps and go right into initializing the restraints.

### Define names
We will store the files created in this tutorial in a folder called `plumed` so we don't mix files with the previous tutorial.

```python
base_name = "cb6-but-dum"
work_dir = "plumed"
complex_dir = "complex"
```

## Configure APR Restraints

### Define anchor atoms
See [tutorial 1](01-tutorial-cb6-but.ipynb) for the choice of selection

```python
# Guest atoms
G1 = ":BUT@C"
G2 = ":BUT@C3"

# Host atoms
H1 = ":CB6@C"
H2 = ":CB6@C31"
H3 = ":CB6@C18"

# Dummy atoms
D1 = ":DM1"
D2 = ":DM2"
D3 = ":DM3"
```

### Determine the number of windows
Before we add the restraints, it is helpful to set the $\lambda$ fractions that control the strength of the force constants during attach and release, and to define the distances for the pulling phase.

The attach fractions go from 0 to 1 and we place more points at the bottom of the range to sample the curvature of $dU/d \lambda$. Next, we generally apply a distance restraint until the guest is ~18 Angstroms away from the host, in increments of 0.4 Angstroms. This distance should be at least twice the Lennard-Jones cutoff in the system. These values have worked well for us, but this is one aspect that should be carefully checked for new systems.

```python
import numpy as np
```

```python
attach_string = "0.00 0.40 0.80 1.60 2.40 4.00 5.50 8.65 11.80 18.10 24.40 37.00 49.60 74.80 100.00"
attach_fractions = [float(i) / 100 for i in attach_string.split()]

initial_distance = 6.0
pull_distances = np.arange(0.0 + initial_distance, 18.0 + initial_distance, 1.0)

release_fractions = []

windows = [len(attach_fractions), len(pull_distances), len(release_fractions)]
print(f"There are {windows} windows in this attach-pull-release calculation.")
```

    There are [15, 18, 0] windows in this attach-pull-release calculation.

### Load structure

```python
import parmed as pmd
```

* Load complex structure

```python
structure = pmd.load_file(
    os.path.join(complex_dir, f"{base_name}.prmtop"),
    os.path.join(complex_dir, f"{base_name}.rst7"),
    structure = True,
)
```

### Host Static Restraints
See [tutorial 1](01-tutorial-cb6-but.ipynb#host_static) for an explanation of the static restraints

```python
import paprika.restraints as restraints
```

```python
static_restraints = []
```

```python
r = restraints.static_DAT_restraint(restraint_mask_list = [D1, H1],
                                    num_window_list = windows,
                                    ref_structure = structure,
                                    force_constant = 5.0,
                                    amber_index=True)

static_restraints.append(r)
```

```python
r = restraints.static_DAT_restraint(restraint_mask_list = [D2, D1, H1],
                                    num_window_list = windows,
                                    ref_structure = structure,
                                    force_constant = 100.0,
                                    amber_index=True)

static_restraints.append(r)
```

```python
r = restraints.static_DAT_restraint(restraint_mask_list = [D3, D2, D1, H1],
                                    num_window_list = windows,
                                    ref_structure = structure,
                                    force_constant = 100.0,
                                    amber_index=True)

static_restraints.append(r)
```

```python
r = restraints.static_DAT_restraint(restraint_mask_list = [D1, H1, H2],
                                    num_window_list = windows,
                                    ref_structure = structure,
                                    force_constant = 100.0,
                                    amber_index=True)

static_restraints.append(r)
```

```python
r = restraints.static_DAT_restraint(restraint_mask_list = [D2, D1, H1, H2],
                                    num_window_list = windows,
                                    ref_structure = structure,
                                    force_constant = 100.0,
                                    amber_index=True)

static_restraints.append(r)
```

```python
r = restraints.static_DAT_restraint(restraint_mask_list = [D1, H1, H2, H3],
                                    num_window_list = windows,
                                    ref_structure = structure,
                                    force_constant = 100.0,
                                    amber_index=True)

static_restraints.append(r)
```

### Guest translational and rotational restraints
See [tutorial 1](01-tutorial-cb6-but.ipynb#guest) for an explanation of the guest restraints

```python
guest_restraints = []
```

```python
r = restraints.DAT_restraint()
r.mask1 = D1
r.mask2 = G1
r.topology = structure
r.auto_apr = True
r.continuous_apr = True
r.amber_index = True

r.attach["target"] = pull_distances[0]          # Angstroms
r.attach["fraction_list"] = attach_fractions
r.attach["fc_final"] = 5.0                      # kcal/mol/Angstroms**2

r.pull["target_final"] = 24.0                   # Angstroms
r.pull["num_windows"] = windows[1]

r.initialize()
guest_restraints.append(r)
```

```python
r = restraints.DAT_restraint()
r.mask1 = D2
r.mask2 = D1
r.mask3 = G1
r.topology = structure
r.auto_apr = True
r.continuous_apr = True
r.amber_index = True

r.attach["target"] = 180.0                      # Degrees
r.attach["fraction_list"] = attach_fractions
r.attach["fc_final"] = 100.0                    # kcal/mol/radian**2

r.pull["target_final"] = 180.0                  # Degrees
r.pull["num_windows"] = windows[1]

r.initialize()
guest_restraints.append(r)
```

```python
r = restraints.DAT_restraint()
r.mask1 = D1
r.mask2 = G1
r.mask3 = G2
r.topology = structure
r.auto_apr = True
r.continuous_apr = True
r.amber_index = True

r.attach["target"] = 180.0                      # Degrees
r.attach["fraction_list"] = attach_fractions
r.attach["fc_final"] = 100.0                    # kcal/mol/radian**2

r.pull["target_final"] = 180.0                  # Degrees
r.pull["num_windows"] = windows[1]

r.initialize()
guest_restraints.append(r)
```

### Create APR windows
We use the guest restraints to create a list of windows with the appropriate names and then create the directories.

```python
from paprika.restraints.restraints import create_window_list
```

```python
window_list = create_window_list(guest_restraints)
```

```python
if not os.path.isdir(work_dir):
    os.makedirs(work_dir)

for window in window_list:
    folder = os.path.join(work_dir, window)
    if not os.path.isdir(folder):
        os.makedirs(os.path.join(work_dir, window))
```

### Write APR restraints to Plumed format
In this section we create an instance of `Plumed()` from `paprika.restraints.plumed`, which is a class to generate the `Plumed` restraint files. We need to specify the list of restraints used throughout the APR calculations and the corresponding windows list. In this tutorial we will print the **host static** restraints and the **guest** restraints. The `Plumed` class includes a method to add restraints to dummy atoms (`add_dummy_atoms_to_file`) but we will not do that here. Instead we will use the built-in position restraints feature in Amber (see [Simulation](#simulate) section below).

**Note**: be careful when specifiying the force constants in `DAT_restraints`. We follow the Amber (and CHARMM) convention where the force constant is already multiplied by a factor of 1/2 but `Plumed` requires the user to specify the force constant without this factor, i.e.

$$
\begin{align}
U_{amber} &= K_{amber} (r-r_{0})^2 \\
U_{plumed} &= \frac{1}{2} k_{plumed} (r - r_{0})^2
\end{align}
$$

thus $k_{plumed} = 2 \times  K_{amber}$. If Amber force constants was used in generating the `DAT_restraints` (the case in this tutorial) we need to set the variable `uses_legacy_k` to `True` (this is on by default).

```python
from paprika.restraints.plumed import Plumed
```

```python
restraints_list = (static_restraints + guest_restraints)

plumed = Plumed()
plumed.file_name = 'plumed.dat'
plumed.path = work_dir
plumed.window_list = window_list
plumed.restraint_list = restraints_list
plumed.uses_legacy_k = True

plumed.dump_to_file()
```

## Prepare host-guest system

### Translate guest molecule
For the attach windows, we will use the initial, bound coordinates for the host-guest complex. Only the force constants change during this phase, so a single set of coordinates is sufficient. For the pull windows, we will translate the guest to the target value of the restraint before solvation, and for the release windows, we will use the coordinates from the final pull window.

```python
import shutil
```

```python
for window in window_list:
    if window[0] == "a":
        shutil.copy(os.path.join(complex_dir, f"{base_name}.prmtop"),
                    os.path.join(work_dir, window, f"{base_name}.prmtop"))
        shutil.copy(os.path.join(complex_dir, f"{base_name}.rst7"),
                    os.path.join(work_dir, window, f"{base_name}.rst7"))

    elif window[0] == "p":
        structure = pmd.load_file(
            os.path.join(complex_dir, f"{base_name}.prmtop"),
            os.path.join(complex_dir, f"{base_name}.rst7"),
            structure = True
        )
        target_difference = guest_restraints[0].phase['pull']['targets'][int(window[1:])] -\
                            guest_restraints[0].pull['target_initial']
        print(f"In window {window} we will translate the guest {target_difference.magnitude:0.1f}.")

        for atom in structure.atoms:
            if atom.residue.name == "BUT":
                atom.xz += target_difference.magnitude

        structure.save(os.path.join(work_dir, window, f"{base_name}.prmtop"), overwrite=True)
        structure.save(os.path.join(work_dir, window, f"{base_name}.rst7"), overwrite=True)
```

    In window p000 we will translate the guest 0.0 Angstroms.
    In window p001 we will translate the guest 1.1 Angstroms.
    In window p002 we will translate the guest 2.1 Angstroms.
    In window p003 we will translate the guest 3.2 Angstroms.
    In window p004 we will translate the guest 4.2 Angstroms.
    In window p005 we will translate the guest 5.3 Angstroms.
    In window p006 we will translate the guest 6.4 Angstroms.
    In window p007 we will translate the guest 7.4 Angstroms.
    In window p008 we will translate the guest 8.5 Angstroms.
    In window p009 we will translate the guest 9.5 Angstroms.
    In window p010 we will translate the guest 10.6 Angstroms.
    In window p011 we will translate the guest 11.6 Angstroms.
    In window p012 we will translate the guest 12.7 Angstroms.
    In window p013 we will translate the guest 13.8 Angstroms.
    In window p014 we will translate the guest 14.8 Angstroms.
    In window p015 we will translate the guest 15.9 Angstroms.
    In window p016 we will translate the guest 16.9 Angstroms.
    In window p017 we will translate the guest 18.0 Angstroms.

## Simulation

Since we are going to run an implicit solvent simulation, we have everything ready to go. **pAPRika** has an `AMBER` module that can help setting default parameters for the simulation. There are some high level options that we set directly, like `simulation.path`, and then we call the function `config_gb_min()` to setup reasonable default simulation parameters for a minimization in the Generalized-Born ensemble. After that, we directly modify the simulation `cntrl` section to apply the positional restraints on the dummy atoms.

We will run the simulations with `sander` but it is also possible and faster to run this with `pmemd` or `pmemd.cuda` if you have them installed.

**Note**: The difference here compared to [Tutorial 1](01-tutorial-cb6-but.ipynb#simulate) is that instead of specifying a `simulation.restraint_file` we will specify `simulation.plumed_file`.

**Note**: as explained at the [start](#start) of this tutorial, make sure that the `PLUMED_KERNEL` environment variable is set otherwise the simulation will fail to run.

```python
from paprika.simulate import AMBER
```

Initialize logger

```python
import logging
from importlib import reload
reload(logging)

logger = logging.getLogger()
logging.basicConfig(
    format='%(asctime)s %(message)s',
    datefmt='%Y-%m-%d %I:%M:%S %p',
    level=logging.INFO
)
```

### Energy Minimization
Run a quick minimization in every window. Note that we need to specify `simulation.cntrl["ntr"] = 1` to enable the positional restraints on the dummy atoms.

```python
for window in window_list:
    simulation = AMBER()
    simulation.executable = "sander"

    simulation.path = f"{work_dir}/{window}/"
    simulation.prefix = "minimize"

    simulation.topology = "cb6-but-dum.prmtop"
    simulation.coordinates = "cb6-but-dum.rst7"
    simulation.ref = "cb6-but-dum.rst7"
    simulation.plumed_file = "plumed.dat"

    simulation.config_gb_min()
    simulation.cntrl["ntr"] = 1
    simulation.cntrl["restraint_wt"] = 50.0
    simulation.cntrl["restraintmask"] = "'@DUM'"

    logger.info(f"Running minimization in window {window}...")
    simulation.run(overwrite=True)
```

    2020-10-01 11:18:20 AM Running minimization in window a000...
    2020-10-01 11:18:31 AM Running minimization in window a001...
    2020-10-01 11:18:42 AM Running minimization in window a002...
    2020-10-01 11:18:54 AM Running minimization in window a003...
    2020-10-01 11:19:05 AM Running minimization in window a004...
    2020-10-01 11:19:17 AM Running minimization in window a005...
    2020-10-01 11:19:28 AM Running minimization in window a006...
    2020-10-01 11:19:42 AM Running minimization in window a007...
    2020-10-01 11:19:54 AM Running minimization in window a008...
    2020-10-01 11:20:06 AM Running minimization in window a009...
    2020-10-01 11:20:18 AM Running minimization in window a010...
    2020-10-01 11:20:30 AM Running minimization in window a011...
    2020-10-01 11:20:42 AM Running minimization in window a012...
    2020-10-01 11:20:53 AM Running minimization in window a013...
    2020-10-01 11:21:04 AM Running minimization in window p000...
    2020-10-01 11:21:14 AM Running minimization in window p001...
    2020-10-01 11:21:26 AM Running minimization in window p002...
    2020-10-01 11:21:38 AM Running minimization in window p003...
    2020-10-01 11:21:50 AM Running minimization in window p004...
    2020-10-01 11:22:01 AM Running minimization in window p005...
    2020-10-01 11:22:15 AM Running minimization in window p006...
    2020-10-01 11:22:28 AM Running minimization in window p007...
    2020-10-01 11:22:40 AM Running minimization in window p008...
    2020-10-01 11:22:54 AM Running minimization in window p009...
    2020-10-01 11:23:06 AM Running minimization in window p010...
    2020-10-01 11:23:19 AM Running minimization in window p011...
    2020-10-01 11:23:31 AM Running minimization in window p012...
    2020-10-01 11:23:44 AM Running minimization in window p013...
    2020-10-01 11:23:55 AM Running minimization in window p014...
    2020-10-01 11:24:07 AM Running minimization in window p015...
    2020-10-01 11:24:19 AM Running minimization in window p016...
    2020-10-01 11:24:30 AM Running minimization in window p017...

### Production Run
Here we will skip the equilibration step and go straight to production!

```python
for window in window_list:
    simulation = AMBER()
    simulation.executable = "sander"

    simulation.path = f"{work_dir}/{window}/"
    simulation.prefix = "production"

    simulation.topology = "cb6-but-dum.prmtop"
    simulation.coordinates = "minimize.rst7"
    simulation.ref = "cb6-but-dum.rst7"
    simulation.plumed_file = "plumed.dat"

    simulation.config_gb_md()
    simulation.cntrl["ntr"] = 1
    simulation.cntrl["restraint_wt"] = 50.0
    simulation.cntrl["restraintmask"] = "'@DUM'"

    logger.info(f"Running production in window {window}...")
    simulation.run(overwrite=True)
```

    2020-10-01 11:12:15 AM Running production in window a000...
    2020-10-01 11:12:25 AM Running production in window a001...
    2020-10-01 11:12:34 AM Running production in window a002...
    2020-10-01 11:12:44 AM Running production in window a003...
    2020-10-01 11:12:55 AM Running production in window a004...
    2020-10-01 11:13:06 AM Running production in window a005...
    2020-10-01 11:13:19 AM Running production in window a006...
    2020-10-01 11:13:29 AM Running production in window a007...
    2020-10-01 11:13:39 AM Running production in window a008...
    2020-10-01 11:13:49 AM Running production in window a009...
    2020-10-01 11:13:59 AM Running production in window a010...
    2020-10-01 11:14:10 AM Running production in window a011...
    2020-10-01 11:14:19 AM Running production in window a012...
    2020-10-01 11:14:29 AM Running production in window a013...
    2020-10-01 11:14:39 AM Running production in window p000...
    2020-10-01 11:14:49 AM Running production in window p001...
    2020-10-01 11:14:59 AM Running production in window p002...
    2020-10-01 11:15:10 AM Running production in window p003...
    2020-10-01 11:15:21 AM Running production in window p004...
    2020-10-01 11:15:33 AM Running production in window p005...
    2020-10-01 11:15:43 AM Running production in window p006...
    2020-10-01 11:15:55 AM Running production in window p007...
    2020-10-01 11:16:05 AM Running production in window p008...
    2020-10-01 11:16:18 AM Running production in window p009...
    2020-10-01 11:16:31 AM Running production in window p010...
    2020-10-01 11:16:45 AM Running production in window p011...
    2020-10-01 11:16:56 AM Running production in window p012...
    2020-10-01 11:17:05 AM Running production in window p013...
    2020-10-01 11:17:16 AM Running production in window p014...
    2020-10-01 11:17:26 AM Running production in window p015...
    2020-10-01 11:17:37 AM Running production in window p016...
    2020-10-01 11:17:49 AM Running production in window p017...

## Analysis

Once the simulation is completed, we can using the `analysis` module to determine the binding free energy. We supply the location of the parameter information, a string or list for the file names (wildcards supported), the location of the windows, and the restraints on the guest.

In this example, we use the method `ti-block` which determines the free energy using **t**hermodynamic **i**integration and then estimates the standard error of the mean at each data point using blocking analysis. Bootstrapping it used to determine the uncertainty of the  full thermodynamic integral for each phase.

After running `compute_free_energy()`, a dictionary called `results` will be populated, that contains the free energy and SEM for each phase of the simulation.

```python
import paprika.analysis as analysis
```

```python
free_energy = analysis.fe_calc()
free_energy.topology = "cb6-but-dum.prmtop"
free_energy.trajectory = 'production*.nc'
free_energy.path = work_dir
free_energy.restraint_list = guest_restraints
free_energy.collect_data()
free_energy.methods = ['ti-block']
free_energy.ti_matrix = "full"
free_energy.bootcycles = 1000
free_energy.compute_free_energy()
```

We also need to calculate the free-energy cost of releasing the restraints on the guest molecule.

```python
free_energy.compute_ref_state_work([
    guest_restraints[0], guest_restraints[1], None, None,
    guest_restraints[2], None
])
```

Then we add the free-energies together and combine the uncertainties to get the binding-free energy

```python
binding_affinity = -1 * (
    free_energy.results["attach"]["ti-block"]["fe"] + \
    free_energy.results["pull"]["ti-block"]["fe"] + \
    free_energy.results["ref_state_work"]
)

sem = np.sqrt(
    free_energy.results["attach"]["ti-block"]["sem"]**2 + \
    free_energy.results["pull"]["ti-block"]["sem"]**2
)
```

```python
print(f"The binding affinity of butane to cucurbit[6]uril = {binding_affinity.magnitude:0.2f} +/- {sem.magnitude:0.2f} kcal/mol")
```

    The binding affinity of butane to cucurbit[6]uril = -7.24 +/- 6.52 kcal/mol


# Markdown 與 LaTeX簡介

每個範例內容將範例語法與輸出分為不同的 Cell 放置，雙擊 Cell 也可以查看原始碼，進行進一步的研究與實驗。

## 1. Markdown 主要語法

### 1.1 段落和斷行

_範例語法：_
```
前軍至夏口，周瑜問：「荊州有人在前面接否？」人報：「劉皇叔使糜竺來見都督。」瑜喚至，問勞軍如何。糜竺曰：「主公皆準備安排下了。」瑜曰：「皇叔何在？」竺曰：「在荊州城門相等，與都督把盞。」瑜曰：「今為汝家之事，出兵遠征；勞軍之禮，休得輕易。」糜竺領了言語先回。<br />戰船密密排在江上，依次而進。
```
_輸出：(可以看到在"戰船"前面有換行)_

前軍至夏口，周瑜問：「荊州有人在前面接否？」人報：「劉皇叔使糜竺來見都督。」瑜喚至，問勞軍如何。糜竺曰：「主公皆準備安排下了。」瑜曰：「皇叔何在？」竺曰：「在荊州城門相等，與都督把盞。」瑜曰：「今為汝家之事，出兵遠征；勞軍之禮，休得輕易。」糜竺領了言語先回。
戰船密密排在江上，依次而進。

### 1.2 標題 (Heading)

_範例語法：_

```
# 主標題 (等同於 HTML `<h1></h1>`)

## 次標題 (等同於 `<h2></h12`)

### 第三階 (等同於 `<h3></h3>`)

#### 第四階 (等同於 `<h4></h4>`)

##### 第五階 (等同於 `<h5></h5>`)

###### 第六階 (等同於 `<h6></h6>`)
```

_輸出：_

# 主標題 (等同於 HTML `<h1></h1>`)

## 次標題 (等同於 `<h2></h12`)

### 第三階 (等同於 `<h3></h3>`)

#### 第四階 (等同於 `<h4></h4>`)

##### 第五階 (等同於 `<h5></h5>`)

###### 第六階 (等同於 `<h6></h6>`)

### 1.3 無序號列表 (Bullet list)

_範例語法：_

```
六都清單：
- 台北市
    - 大安區
    - 大同區
- 新北市
* 桃園市
* 台中市
+ 台南市
+ 高雄市
```

_輸出：_

六都清單：
- 台北市
    - 大安區
    - 大同區
- 新北市
* 桃園市
* 台中市
+ 台南市
+ 高雄市

### 1.4 序號列表 (Numbered list)

_範例語法：_

```
1. 第一項
2. 第二項
3. 第三項
```

_輸出：_

1. 第一項
2. 第二項
3. 第三項

### 1.5 區塊引言 (blockquoting)

_範例語法：_

```
> 卻說魯肅回見周瑜，說玄德，孔明歡喜不疑，準備出城勞軍。周瑜大笑曰：「原來今番也中了吾計！」便教魯肅稟報吳侯，並遣程普引兵接應。周瑜此時箭瘡已漸平愈，身軀無事，使甘寧為先鋒，自與徐盛，丁奉為第二；淩統，呂蒙為後隊。水陸大兵五百萬，望荊州而來。

周瑜在船中，時復歡笑，以為孔明中計。

> 前軍至夏口，周瑜問：「荊州有人在前面接否？」人報：「劉皇叔使糜竺來見都督。」瑜喚至，問勞軍如何。
```

_輸出：_

> 卻說魯肅回見周瑜，說玄德，孔明歡喜不疑，準備出城勞軍。周瑜大笑曰：「原來今番也中了吾計！」便教魯肅稟報吳侯，並遣程普引兵接應。周瑜此時箭瘡已漸平愈，身軀無事，使甘寧為先鋒，自與徐盛，丁奉為第二；淩統，呂蒙為後隊。水陸大兵五百萬，望荊州而來。

周瑜在船中，時復歡笑，以為孔明中計。

> 前軍至夏口，周瑜問：「荊州有人在前面接否？」人報：「劉皇叔使糜竺來見都督。」瑜喚至，問勞軍如何。

### 1.6 程式碼區塊

_範例語法：_

\`\`\`julia

println("Hello Julia")

\`\`\`

_輸出：_

```julia
println("Hello Julia")
```

### 1.7 分隔線

_範例語法：_

```
卻說魯肅回見周瑜，說玄德，孔明歡喜不疑，準備出城勞軍。周瑜大笑曰：「原來今番也中了吾計！」便教魯肅稟報吳侯，並遣程普引兵接應。周瑜此時箭瘡已漸平愈，身軀無事，使甘寧為先鋒，自與徐盛，丁奉為第二；淩統，呂蒙為後隊。水陸大兵五百萬，望荊州而來。周瑜在船中，時復歡笑，以為孔明中計。

---

前軍至夏口，周瑜問：「荊州有人在前面接否？」人報：「劉皇叔使糜竺來見都督。」瑜喚至，問勞軍如何。糜竺曰：「主公皆準備安排下了。」瑜曰：「皇叔何在？」竺曰：「在荊州城門相等，與都督把盞。」瑜曰：「今為汝家之事，出兵遠征；勞軍之禮，休得輕易。」糜竺領了言語先回。<br />戰船密密排在江上，依次而進。
```

_輸出：_

卻說魯肅回見周瑜，說玄德，孔明歡喜不疑，準備出城勞軍。周瑜大笑曰：「原來今番也中了吾計！」便教魯肅稟報吳侯，並遣程普引兵接應。周瑜此時箭瘡已漸平愈，身軀無事，使甘寧為先鋒，自與徐盛，丁奉為第二；淩統，呂蒙為後隊。水陸大兵五百萬，望荊州而來。周瑜在船中，時復歡笑，以為孔明中計。

---

前軍至夏口，周瑜問：「荊州有人在前面接否？」人報：「劉皇叔使糜竺來見都督。」瑜喚至，問勞軍如何。糜竺曰：「主公皆準備安排下了。」瑜曰：「皇叔何在？」竺曰：「在荊州城門相等，與都督把盞。」瑜曰：「今為汝家之事，出兵遠征；勞軍之禮，休得輕易。」糜竺領了言語先回。
戰船密密排在江上，依次而進。

### 1.8 超連結

_範例語法：_

```
[Cupoy - 為你探索世界的新知](https://www.cupoy.com)
```

_輸出：_

[Cupoy - 為你探索世界的新知](https://www.cupoy.com)

### 1.9 嵌入圖片

_範例語法：_

```

```

_輸出：_

### 1.10 表格

_範例語法：_

```
|姓名|國家|地址|年齡|
|---|---|---|---|
|John Doe|中華民國台灣|台北市大安區敦化街1號|25|
```

_輸出：_

|姓名|國家|地址|年齡|
|---|---|---|---|
|John Doe|中華民國台灣|台北市大安區敦化街1號|25|

## 2. 用 LaTeX 寫數學公式

### 2.1 Inline 模式

_範例語法：_

```
$\LARGE f(\displaystyle\sum_i w_i x_i + b)$
```

_輸出：_

$\LARGE f(\displaystyle\sum_i w_i x_i + b)$

### 2.2 Block 模式

_範例語法：_

```
$$\LARGE f(\displaystyle\sum_i w_i x_i + b)$$
```

_輸出：_

$$\LARGE f(\displaystyle\sum_i w_i x_i + b)$$

_範例語法：_

```
\begin{equation}
\LARGE f(\displaystyle\sum_i w_i x_i + b)
\end{equation}
```

_輸出：_

\begin{equation}
\LARGE f(\displaystyle\sum_i w_i x_i + b)
\end{equation}

_範例語法：_

```
\begin{align}
\LARGE f(\displaystyle\sum_i w_i x_i + b)
\end{align}
```

_輸出：_

\begin{align}
\LARGE f(\displaystyle\sum_i w_i x_i + b)
\end{align}

## 3. 結合 Markdown 和 LaTeX 數學公式

_範例語法：_

```
公式 $\LARGE f(\displaystyle\sum_i w_i x_i + b)$ 是 Deep Learning 課程中常會見到的基本公式

結合 Markdown 和 LaTeX 數學公式，我們可以撰寫出漂亮的文件筆記，讓學習更有效率。
```

_輸出：_

公式 $\LARGE f(\displaystyle\sum_i w_i x_i + b)$ 是 Deep Learning 課程中常會見到的基本公式

結合 Markdown 和 LaTeX 數學公式，我們可以撰寫出漂亮的文件筆記，讓學習更有效率。

```julia

```