## Configurations for Colab

```python
import sys
IN_COLAB = 'google.colab' in sys.modules

if IN_COLAB:
    !apt-get install -y xvfb python-opengl > /dev/null 2>&1
    !pip install gym pyvirtualdisplay > /dev/null 2>&1
    !pip install JSAnimation==0.1
    !pip install pyglet==1.3.2

    from pyvirtualdisplay import Display

    # Start virtual display
    dis = Display(visible=0, size=(400, 400))
    dis.start()
```

# 05. Noisy Networks for Exploration

[M. Fortunato et al., "Noisy Networks for Exploration." arXiv preprint arXiv:1706.10295, 2017.](https://arxiv.org/pdf/1706.10295.pdf)

NoisyNet is an exploration method that learns perturbations of the network weights to drive exploration. The key insight is that a single change to the weight vector can induce a consistent, and potentially very complex, state-dependent change in policy over multiple time steps.

Firstly, let's take a look into a linear layer of a neural network with $p$ inputs and $q$ outputs, represented by

$$
y = wx + b,
$$

where $x \in \mathbb{R}^p$ is the layer input, $w \in \mathbb{R}^{q \times p}$, and $b \in \mathbb{R}$ the bias.

The corresponding noisy linear layer is defined as:

$$
y = (\mu^w + \sigma^w \odot \epsilon^w) x + \mu^b + \sigma^b \odot \epsilon^b,
$$

where $\mu^w + \sigma^w \odot \epsilon^w$ and $\mu^b + \sigma^b \odot \epsilon^b$ replace $w$ and $b$ in the first linear layer equation. The parameters $\mu^w \in \mathbb{R}^{q \times p}, \mu^b \in \mathbb{R}^q, \sigma^w \in \mathbb{R}^{q \times p}$ and $\sigma^b \in \mathbb{R}^q$ are learnable, whereas $\epsilon^w \in \mathbb{R}^{q \times p}$ and $\epsilon^b \in \mathbb{R}^q$ are noise random variables which can be generated by one of the following two ways:

1. **Independent Gaussian noise**: the noise applied to each weight and bias is independent, where each random noise entry is drawn from a unit Gaussian distribution. This means that for each noisy linear layer, there are $pq + q$ noise variables (for $p$ inputs to the layer and $q$ outputs).
2. **Factorised Gaussian noise:** This is a more computationally efficient way. It produces 2 random Gaussian noise vectors ($p, q$) and makes $pq + q$ noise entries by outer product as follows:

$$
\begin{align}
\epsilon_{i,j}^w &= f(\epsilon_i) f(\epsilon_j),\\
\epsilon_{j}^b &= f(\epsilon_i),\\
\text{where } f(x) &= sgn(x) \sqrt{|x|}.
\end{align}
$$

In all experiements of the paper, the authors used Factorised Gaussian noise, so we will go for it as well.

```python
import math
import os
from typing import Dict, List, Tuple

import gym
import matplotlib.pyplot as plt
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from IPython.display import clear_output
```

## Replay buffer

Please see *01.dqn.ipynb* for detailed description.

```python
class ReplayBuffer:
    """A simple numpy replay buffer."""

    def __init__(self, obs_dim: int, size: int, batch_size: int = 32):
        self.obs_buf = np.zeros([size, obs_dim], dtype=np.float32)
        self.next_obs_buf = np.zeros([size, obs_dim], dtype=np.float32)
        self.acts_buf = np.zeros([size], dtype=np.float32)
        self.rews_buf = np.zeros([size], dtype=np.float32)
        self.done_buf = np.zeros(size, dtype=np.float32)
        self.max_size, self.batch_size = size, batch_size
        self.ptr, self.size, = 0, 0

    def store(
        self,
        obs: np.ndarray,
        act: np.ndarray,
        rew: float,
        next_obs: np.ndarray,
        done: bool,
    ):
        self.obs_buf[self.ptr] = obs
        self.next_obs_buf[self.ptr] = next_obs
        self.acts_buf[self.ptr] = act
        self.rews_buf[self.ptr] = rew
        self.done_buf[self.ptr] = done
        self.ptr = (self.ptr + 1) % self.max_size
        self.size = min(self.size + 1, self.max_size)

    def sample_batch(self) -> Dict[str, np.ndarray]:
        idxs = np.random.choice(self.size, size=self.batch_size, replace=False)
        return dict(obs=self.obs_buf[idxs],
                    next_obs=self.next_obs_buf[idxs],
                    acts=self.acts_buf[idxs],
                    rews=self.rews_buf[idxs],
                    done=self.done_buf[idxs])

    def __len__(self) -> int:
        return self.size
```

## Noisy Layer

**References:**
- https://github.com/higgsfield/RL-Adventure/blob/master/5.noisy%20dqn.ipynb
- https://github.com/Kaixhin/Rainbow/blob/master/model.py

```python
class NoisyLinear(nn.Module):
    """Noisy linear module for NoisyNet.

    Attributes:
        in_features (int): input size of linear module
        out_features (int): output size of linear module
        std_init (float): initial std value
        weight_mu (nn.Parameter): mean value weight parameter
        weight_sigma (nn.Parameter): std value weight parameter
        bias_mu (nn.Parameter): mean value bias parameter
        bias_sigma (nn.Parameter): std value bias parameter

    """

    def __init__(self, in_features: int, out_features: int, std_init: float = 0.5):
        """Initialization."""
        super(NoisyLinear, self).__init__()

        self.in_features = in_features
        self.out_features = out_features
        self.std_init = std_init

        self.weight_mu = nn.Parameter(torch.Tensor(out_features, in_features))
        self.weight_sigma = nn.Parameter(
            torch.Tensor(out_features, in_features)
        )
        self.register_buffer(
            "weight_epsilon", torch.Tensor(out_features, in_features)
        )

        self.bias_mu = nn.Parameter(torch.Tensor(out_features))
        self.bias_sigma = nn.Parameter(torch.Tensor(out_features))
        self.register_buffer("bias_epsilon", torch.Tensor(out_features))

        self.reset_parameters()
        self.reset_noise()

    def reset_parameters(self):
        """Reset trainable network parameters (factorized gaussian noise)."""
        mu_range = 1 / math.sqrt(self.in_features)
        self.weight_mu.data.uniform_(-mu_range, mu_range)
        self.weight_sigma.data.fill_(
            self.std_init / math.sqrt(self.in_features)
        )
        self.bias_mu.data.uniform_(-mu_range, mu_range)
        self.bias_sigma.data.fill_(
            self.std_init / math.sqrt(self.out_features)
        )

    def reset_noise(self):
        """Make new noise."""
        epsilon_in = self.scale_noise(self.in_features)
        epsilon_out = self.scale_noise(self.out_features)

        # outer product
        self.weight_epsilon.copy_(epsilon_out.ger(epsilon_in))
        self.bias_epsilon.copy_(epsilon_out)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """Forward method implementation.

        We don't use separate statements on train / eval mode.
        It doesn't show remarkable difference of performance.
        """
        return F.linear(
            x,
            self.weight_mu + self.weight_sigma * self.weight_epsilon,
            self.bias_mu + self.bias_sigma * self.bias_epsilon,
        )

    @staticmethod
    def scale_noise(size: int) -> torch.Tensor:
        """Set scale to make noise (factorized gaussian noise)."""
        x = torch.FloatTensor(np.random.normal(loc=0.0, scale=1.0, size=size))

        return x.sign().mul(x.abs().sqrt())
```

## Noisy Network

We use NoisyLinear for the last two FC layers, and there is a method to reset noise at every step.
These are the only differences from the example of *01.dqn.ipynb*.

```python
class Network(nn.Module):
    def __init__(self, in_dim: int, out_dim: int):
        """Initialization."""
        super(Network, self).__init__()

        self.feature = nn.Linear(in_dim, 128)
        self.noisy_layer1 = NoisyLinear(128, 128)
        self.noisy_layer2 = NoisyLinear(128, out_dim)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """Forward method implementation."""
        feature = F.relu(self.feature(x))
        hidden = F.relu(self.noisy_layer1(feature))
        out = self.noisy_layer2(hidden)

        return out

    def reset_noise(self):
        """Reset all noisy layers."""
        self.noisy_layer1.reset_noise()
        self.noisy_layer2.reset_noise()
```

## DQN + NoisyNet Agent (w/o DuelingNet)

Here is a summary of DQNAgent class.

| Method           | Note                                                 |
| ---              | ---                                                  |
|select_action     | select an action from the input state.               |
|step              | take an action and return the response of the env.   |
|compute_dqn_loss  | return dqn loss.                                     |
|update_model      | update the model by gradient descent.                |
|target_hard_update| hard update from the local model to the target model.|
|train             | train the agent during num_frames.                   |
|test              | test the agent (1 episode).                          |
|plot              | plot the training progresses.                        |

In the paper, NoisyNet is used as a component of the Dueling Network Architecture, which includes Double-DQN and Prioritized Experience Replay. However, we don't implement them to simplify the tutorial. One thing to note is that NoisyNet is an alternertive to $\epsilon$-greedy method, so all $\epsilon$ related lines are removed. Please check all comments with *NoisyNet*.

```python
class DQNAgent:
    """DQN Agent interacting with environment.

    Attribute:
        env (gym.Env): openAI Gym environment
        memory (ReplayBuffer): replay memory to store transitions
        batch_size (int): batch size for sampling
        target_update (int): period for target model's hard update
        gamma (float): discount factor
        dqn (Network): model to train and select actions
        dqn_target (Network): target model to update
        optimizer (torch.optim): optimizer for training dqn
        transition (list): transition information including
                           state, action, reward, next_state, done
    """

    def __init__(
        self,
        env: gym.Env,
        memory_size: int,
        batch_size: int,
        target_update: int,
        gamma: float = 0.99,
    ):
        """Initialization.

        Args:
            env (gym.Env): openAI Gym environment
            memory_size (int): length of memory
            batch_size (int): batch size for sampling
            target_update (int): period for target model's hard update
            gamma (float): discount factor
        """
        # NoisyNet: All attributes related to epsilon are removed
        obs_dim = env.observation_space.shape[0]
        action_dim = env.action_space.n

        self.env = env
        self.memory = ReplayBuffer(obs_dim, memory_size, batch_size)
        self.batch_size = batch_size
        self.target_update = target_update
        self.gamma = gamma

        # device: cpu / gpu
        self.device = torch.device(
            "cuda" if torch.cuda.is_available() else "cpu"
        )
        print(self.device)

        # networks: dqn, dqn_target
        self.dqn = Network(obs_dim, action_dim).to(self.device)
        self.dqn_target = Network(obs_dim, action_dim).to(self.device)
        self.dqn_target.load_state_dict(self.dqn.state_dict())
        self.dqn_target.eval()

        # optimizer
        self.optimizer = optim.Adam(self.dqn.parameters())

        # transition to store in memory
        self.transition = list()

        # mode: train / test
        self.is_test = False

    def select_action(self, state: np.ndarray) -> np.ndarray:
        """Select an action from the input state."""
        # NoisyNet: no epsilon greedy action selection
        selected_action = self.dqn(
            torch.FloatTensor(state).to(self.device)
        ).argmax()
        selected_action = selected_action.detach().cpu().numpy()

        if not self.is_test:
            self.transition = [state, selected_action]

        return selected_action

    def step(self, action: np.ndarray) -> Tuple[np.ndarray, np.float64, bool]:
        """Take an action and return the response of the env."""
        next_state, reward, done, _ = self.env.step(action)

        if not self.is_test:
            self.transition += [reward, next_state, done]
            self.memory.store(*self.transition)

        return next_state, reward, done

    def update_model(self) -> torch.Tensor:
        """Update the model by gradient descent."""
        samples = self.memory.sample_batch()

        loss = self._compute_dqn_loss(samples)

        self.optimizer.zero_grad()
        loss.backward()
        self.optimizer.step()

        # NoisyNet: reset noise
        self.dqn.reset_noise()
        self.dqn_target.reset_noise()

        return loss.item()

    def train(self, num_frames: int, plotting_interval: int = 200):
        """Train the agent."""
        self.is_test = False

        state = self.env.reset()
        update_cnt = 0
        losses = []
        scores = []
        score = 0

        for frame_idx in range(1, num_frames + 1):
            action = self.select_action(state)
            next_state, reward, done = self.step(action)

            state = next_state
            score += reward

            # NoisyNet: removed decrease of epsilon

            # if episode ends
            if done:
                state = env.reset()
                scores.append(score)
                score = 0

            # if training is ready
            if len(self.memory) >= self.batch_size:
                loss = self.update_model()
                losses.append(loss)
                update_cnt += 1

                # if hard update is needed
                if update_cnt % self.target_update == 0:
                    self._target_hard_update()

            # plotting
            if frame_idx % plotting_interval == 0:
                self._plot(frame_idx, scores, losses)

        self.env.close()

    def test(self) -> List[np.ndarray]:
        """Test the agent."""
        self.is_test = True

        state = self.env.reset()
        done = False
        score = 0

        frames = []
        while not done:
            frames.append(self.env.render(mode="rgb_array"))
            action = self.select_action(state)
            next_state, reward, done = self.step(action)

            state = next_state
            score += reward

        print("score: ", score)
        self.env.close()

        return frames

    def _compute_dqn_loss(self, samples: Dict[str, np.ndarray]) -> torch.Tensor:
        """Return dqn loss."""
        device = self.device  # for shortening the following lines
        state = torch.FloatTensor(samples["obs"]).to(device)
        next_state = torch.FloatTensor(samples["next_obs"]).to(device)
        action = torch.LongTensor(samples["acts"].reshape(-1, 1)).to(device)
        reward = torch.FloatTensor(samples["rews"].reshape(-1, 1)).to(device)
        done = torch.FloatTensor(samples["done"].reshape(-1, 1)).to(device)

        # G_t   = r + gamma * v(s_{t+1})  if state != Terminal
        #       = r                       otherwise
        curr_q_value = self.dqn(state).gather(1, action)
        next_q_value = self.dqn_target(next_state).max(
            dim=1, keepdim=True
        )[0].detach()
        mask = 1 - done
        target = (reward + self.gamma * next_q_value * mask).to(self.device)

        # calculate dqn loss
        loss = F.smooth_l1_loss(curr_q_value, target)

        return loss

    def _target_hard_update(self):
        """Hard update: target <- local."""
        self.dqn_target.load_state_dict(self.dqn.state_dict())

    def _plot(
        self,
        frame_idx: int,
        scores: List[float],
        losses: List[float],
    ):
        """Plot the training progresses."""
        clear_output(True)
        plt.figure(figsize=(20, 5))
        plt.subplot(131)
        plt.title('frame %s. score: %s' % (frame_idx, np.mean(scores[-10:])))
        plt.plot(scores)
        plt.subplot(132)
        plt.title('loss')
        plt.plot(losses)
        plt.show()
```

## Environment

You can see the [code](https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py) and [configurations](https://github.com/openai/gym/blob/master/gym/envs/__init__.py#L53) of CartPole-v0 from OpenAI's repository.

```python
# environment
env_id = "CartPole-v0"
env = gym.make(env_id)
```

## Set random seed

```python
seed = 777

def seed_torch(seed):
    torch.manual_seed(seed)
    if torch.backends.cudnn.enabled:
        torch.backends.cudnn.benchmark = False
        torch.backends.cudnn.deterministic = True

np.random.seed(seed)
seed_torch(seed)
env.seed(seed)
```

    [777]

## Initialize

```python
# parameters
num_frames = 10000
memory_size = 1000
batch_size = 32
target_update = 100

# train
agent = DQNAgent(env, memory_size, batch_size, target_update)
```

    cuda

## Train

```python
agent.train(num_frames)
```

## Test

Run the trained agent (1 episode).

```python
frames = agent.test()
```

    score:  200.0

## Render

```python
# Imports specifically so we can render outputs in Colab.
from matplotlib import animation
from JSAnimation.IPython_display import display_animation
from IPython.display import display

def display_frames_as_gif(frames):
    """Displays a list of frames as a gif, with controls."""
    patch = plt.imshow(frames[0])
    plt.axis('off')

    def animate(i):
        patch.set_data(frames[i])

    anim = animation.FuncAnimation(
        plt.gcf(), animate, frames = len(frames), interval=50
    )
    display(display_animation(anim, default_mode='loop'))

# display
display_frames_as_gif(frames)
```

    <input id="_anim_sliderDVREKFGCMTZZTQWF" type="range" style="width:350px" name="points" min="0" max="1" step="1" value="0" onchange="animDVREKFGCMTZZTQWF.set_frame(parseInt(this.value));"></input>

    <button onclick="animDVREKFGCMTZZTQWF.slower()">–</button>
    <button onclick="animDVREKFGCMTZZTQWF.first_frame()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.previous_frame()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.reverse_animation()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.pause_animation()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.play_animation()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.next_frame()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.last_frame()"></button>
    <button onclick="animDVREKFGCMTZZTQWF.faster()">+</button>
  <form action="#n" name="_anim_loop_selectDVREKFGCMTZZTQWF" class="anim_control">
    <input type="radio" name="state" value="once" > Once </input>
    <input type="radio" name="state" value="loop" checked> Loop </input>
    <input type="radio" name="state" value="reflect" > Reflect </input>
  </form>


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)
```

Toggle cell visibility here.

```python
%matplotlib notebook
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
import ipywidgets as widgets
```

## Linearizacija - nihalo

V okviru modeliranja sistemov predstavlja linearizacija proces modeliranja nelinearnega sistema z linearno diferencialno enačbo v bližini izbrane obratovalne točke (običajno je to ravnovesna točka). V tem interaktivnem primeru je opisan proces prikazan na primeru preprostega nihala. Sila, ki povzroča oscilatorno gibanje nihala (prikazano na sliki spodaj), je definirana kot $-mgsin\theta$. Gibalno enačbo nihala lahko tako zapišemo kot:

\begin{equation}
    mL^2\frac{d^2\theta}{dt^2}=-mgsin\theta L.
\end{equation}

Po preureditvi zgornje enačbi dobimo naslednjo nelinearno diferencialno enačbo drugega reda:

\begin{equation}
    \frac{d^2\theta}{dt^2}+\frac{g}{L}sin\theta=0.
\end{equation}

V primeru majhnih kotnih pomikov lahko uporabimo izraz za aproksimacijo majhnih kotov ($sin\theta\approx\theta$) in zgornjo enačbo preoblikujemo v linearno diferencialno enačbo drugega reda:

\begin{equation}
    \frac{d^2\theta}{dt^2}+\frac{g}{L}\theta=0.
\end{equation}

---

        Preprosto nihalo

### Kako upravljati s tem interaktivnim primerom?

Z uporabo drsnikov spreminjaj dolžino nihala $L$ in vrednosti začetnih pogojev $\theta_0$ in $\dot{\theta_0}$.

```python
# create figure
fig = plt.figure(figsize=(9.8, 3),num='Linearizacija - nihalo')

# add sublot
ax = fig.add_subplot(111)
ax.set_title('Časovni odziv')
ax.set_ylabel('izhod')
ax.set_xlabel('$t$ [s]')
ax.axhline(y=0, xmin=-1, xmax=6, color='k', linewidth=1)

ax.grid(which='both', axis='both', color='lightgray')

nonlinear, = ax.plot([], [])
linear, = ax.plot([], [])

style = {'description_width': 'initial'}

g=9.81 # m/s^2

def model_nonlinear(ic,t,L):
    fi, fidot = ic
    return [fidot,-g/L*np.sin(fi)]

def model_linear(ic,t,L):
    fi, fidot = ic
    return [fidot,-g/L*fi]

def build_model(y0,ypika0,L):
    ic=[y0,ypika0]
    t=np.linspace(0,5,num=500)
    fi=odeint(model_nonlinear,ic,t,args=(L,))
    ys=fi[:,0]

    fi_linear=odeint(model_linear,ic,t,args=(L,))
    ys_linear=fi_linear[:,0]

    global nonlinear, linear

    ax.lines.remove(nonlinear)
    ax.lines.remove(linear)

    nonlinear, = ax.plot(t,ys,label='nelinearen sistem',color='C0', linewidth=5)
    linear, = ax.plot(t,ys_linear,label='lineariziran sistem',color='C3', linewidth=2)

    ax.legend()

    ax.relim()
    ax.autoscale_view()

L_slider=widgets.FloatSlider(value=0.3, min=.01, max=2., step=.01,
                                        description='$L$ [m]:',disabled=False,continuous_update=False,
                                        orientation='horizontal',readout=True,readout_format='.2f',)

ypika0_slider=widgets.FloatSlider(value=1, min=-3, max=3, step=0.1,
                                        description='$\dot \\theta_0$ [rad/s]:',disabled=False,continuous_update=False,
                                        orientation='horizontal',readout=True,readout_format='.2f',)

y0_slider=widgets.FloatSlider(value=1, min=-3, max=3, step=0.1,
                                        description='$\\theta_0$ [rad]:',disabled=False,continuous_update=False,
                                        orientation='horizontal',readout=True,readout_format='.2f',)

input_data=widgets.interactive_output(build_model, {'y0':y0_slider,'ypika0':ypika0_slider,'L':L_slider})

display(L_slider,y0_slider,ypika0_slider,input_data)
```

    <IPython.core.display.Javascript object>

    FloatSlider(value=0.3, continuous_update=False, description='$L$ [m]:', max=2.0, min=0.01, step=0.01)

    FloatSlider(value=1.0, continuous_update=False, description='$\\theta_0$ [rad]:', max=3.0, min=-3.0)

    FloatSlider(value=1.0, continuous_update=False, description='$\\dot \\theta_0$ [rad/s]:', max=3.0, min=-3.0)

    Output()


# CSNAnalysis Tutorial
### A brief introduction to the use of the CSNAnalysis package
---
**Updated Aug 19, 2020**
*Dickson Lab, Michigan State University*

## Overview

The CSNAnalysis package is a set of tools for network-based analysis of molecular dynamics trajectories.
  CSNAnalysis is an easy interface between enhanced sampling algorithms
  (e.g. WExplore implemented in `wepy`), molecular clustering programs (e.g. `MSMBuilder`), graph analysis packages (e.g. `networkX`) and graph visualization programs (e.g. `Gephi`).

### What are conformation space networks?

A conformation space network is a visualization of a free energy landscape, where each node is a cluster of molecular conformations, and the edges show which conformations can directly interconvert during a molecular dynamics simulation. A CSN can be thought of as a visual representation of a transition matrix, where the nodes represent the row / column indices and the edges show the off-diagonal elements. `CSNAnalysis` offers a concise set of tools for the creation, analysis and visualization of CSNs.

**This tutorial will give quick examples for the following use cases:**

1. Initializing CSN objects from count matrices
2. Trimming CSNs
2. Obtaining steady-state weights from a transition matrix
  * By eigenvalue
  * By iterative multiplication
3. Computing committor probabilities to an arbitrary set of basins
4. Exporting gexf files for visualization with the Gephi program

## Getting started

Clone the CSNAnalysis repository:

```
git clone https://github.com/ADicksonLab/CSNAnalysis.git```

Navigate to the examples directory and install using pip:

```
cd CSNAnalysis
pip install --user -e
```

Go to the examples directory and open this notebook (`examples.ipynb`):

```
cd examples; jupyter notebook```

## Dependencies

I highly recommend using Anaconda and working in a `python3` environment. CSNAnalysis uses the packages `numpy`, `scipy` and `networkx`.  If these are installed then the following lines of code should run without error:

```python
import numpy as np
import networkx as nx
import scipy
```

If `CSNAnalysis` was installed (i.e. added to your `sys.path`), then this should also work:

```python
from csnanalysis.csn import CSN
from csnanalysis.matrix import *
```

This notebook also uses `matplotlib`, to visualize output.

```python
import matplotlib
```

Great!  Now let's load in the count matrix that we'll use for all the examples here:

```python
count_mat = scipy.sparse.load_npz('matrix.npz')
```

## Background: Sparse matrices

It's worth knowing a little about sparse matrices before we start. If we have a huge $N$ by $N$ matrix, where $N > 1000$, but most of the elements are zero, it is more efficient to store the data as a sparse matrix.

```python
type(count_mat)
```

    scipy.sparse.coo.coo_matrix

`coo_matrix` refers to "coordinate format", where the matrix is essentially a set of lists of matrix "coordinates" (rows, columns) and data:

```python
rows = count_mat.row
cols = count_mat.col
data = count_mat.data

for r,c,d in zip(rows[0:10],cols[0:10],data[0:10]):
    print(r,c,d)
```

    0 0 382.0
    0 651 2.0
    0 909 2.0
    0 920 2.0
    0 1363 1.0
    0 1445 2.0
    0 2021 5.0
    0 2022 7.0
    0 2085 4.0
    0 2131 1.0

Although it can be treated like a normal matrix ($4000$ by $4000$ in this case):

```python
count_mat.shape
```

    (4000, 4000)

It only needs to store non-zero elements, which are much fewer than $4000^2$:

```python
len(rows)
```

    44163

**OK, let's get started building a Conformation Space Network!**

---

## 1) Initializing CSN objects from count matrices

To get started we need a count matrix, which can be a `numpy` array, or a `scipy.sparse` matrix, or a list of lists:

```python
our_csn = CSN(count_mat,symmetrize=True)
```

Any of the `CSNAnalysis` functions can be queried using "?"

```python
CSN?
```

The `our_csn` object now holds three different representations of our data.  The original counts can now be found in `scipy.sparse` format:

```python
our_csn.countmat
```

    <4000x4000 sparse matrix of type '<class 'numpy.float64'>'
    	with 62280 stored elements in COOrdinate format>

A transition matrix has been computed from this count matrix according to:
\begin{equation}
t_{ij} = \frac{c_{ij}}{\sum_j c_{ij}}
\end{equation}

```python
our_csn.transmat
```

    <4000x4000 sparse matrix of type '<class 'numpy.float64'>'
    	with 62280 stored elements in COOrdinate format>

where the elements in each column sum to one:

```python
our_csn.transmat.sum(axis=0)
```

    matrix([[1., 1., 1., ..., 1., 1., 1.]])

Lastly, the data has been stored in a `networkx` directed graph:

```python
our_csn.graph
```

    <networkx.classes.digraph.DiGraph at 0x81f64e050>

that holds the nodes and edges of our csn, and we can use in other `networkx` functions.  For example, we can calculate the shortest path between nodes 0 and 10:

```python
nx.shortest_path(our_csn.graph,0,10)
```

    [0, 1445, 2125, 2043, 247, 1780, 10]

---
## 2) Trimming CSNs

A big benefit of coupling the count matrix, transition matrix and graph representations is that elements can be "trimmed" from all three simultaneously.  The `trim` function will eliminate nodes that are not connected to the main component (by inflow, outflow, or both), and can also eliminate nodes that do not meet a minimum count requirement:

```python
our_csn.trim(by_inflow=True, by_outflow=True, min_count=20)
```

The trimmed graph, count matrix and transition matrix are stored as `our_csn.trim_graph`, `our_csn.trim_countmat` and `our_csn.trim_transmat`, respectively.

```python
our_csn.trim_graph.number_of_nodes()
```

    2282

```python
our_csn.trim_countmat.shape
```

    (2282, 2282)

```python
our_csn.trim_transmat.shape
```

    (2282, 2282)

## 3) Obtaining steady-state weights from the transition matrix

Now that we've ensured that our transition matrix is fully-connected, we can compute its equilibrium weights.  This is implemented in two ways.

First, we can compute the eigenvector of the transition matrix with eigenvalue one:

```python
wt_eig = our_csn.calc_eig_weights()
```

This can exhibit some instability, especially for low-weight states, so we can also calculate weights by iterative multiplication of the transition matrix, which can take a little longer:

```python
wt_mult = our_csn.calc_mult_weights()
```

```python
import matplotlib.pyplot as plt
%matplotlib inline

plt.scatter(wt_eig,wt_mult)
plt.plot([0,wt_mult.max()],[0,wt_mult.max()],'r-')
plt.xlabel("Eigenvalue weight")
plt.ylabel("Mult weight")
plt.show()
```

These weights are automatically added as attributes to the nodes in `our_csn.graph`:

```python
our_csn.graph.node[0]
```

    {'label': 0,
     'count': 482,
     'trim': 0.0,
     'eig_weights': 0.002595528367725156,
     'mult_weights': 0.0025955283677248217}

## 4) Committor probabilities to an arbitrary set of basins

We are often doing simulations in the presence of one or more high probability "basins" of attraction.  When there more than one basin, it can be useful to find the probability that a simulation started in a given state will visit (or "commit to") a given basin before the others.

`CSNAnalysis` calculates committor probabilities by creating a sink matrix ($S$), where each column in the transition matrix that corresponds to a sink state is replaced by an identity vector. This turns each state into a "black hole" where probability can get in, but not out.

By iteratively multiplying this matrix by itself, we can approximate $S^\infty$.  The elements of this matrix reveal the probability of transitioning to any of the sink states, upon starting in any non-sink state, $i$.

Let's see this in action.  We'll start by reading in a set of three basins:  $A$, $B$ and $U$.

```python
Astates = [2031,596,1923,3223,2715]
Bstates = [1550,3168,476,1616,2590]
Ustates = list(np.loadtxt('state_U.dat',dtype=int))
```

We can then use the `calc_committors` function to calculate committors between this set of three basins. This will calculate $p_A$, $p_B$, and $p_U$ for each state, which sum to one.

```python
basins = [Astates,Bstates,Ustates]
labels = ['pA','pB','pU']
comms = our_csn.calc_committors(basins,labels=labels)
```

The committors can be interpreted as follows:

```python
i = our_csn.trim_indices[0]
print('comms['+str(i)+'] = ',comms[i])
print('\nIn other words, if you start in state {0:d}:'.format(i))
print('You will reach basin A first with probability {0:.2f}, basin B with probability {1:.2f} and basin U with probability {2:.2f}'.format(comms[i,0],comms[i,1],comms[i,2]))
```

    comms[0] =  [0.26406217 0.29477873 0.44115911]

    In other words, if you start in state 0:
    You will reach basin A first with probability 0.26, basin B with probability 0.29 and basin U with probability 0.44

## 5) Exporting graph for visualization in Gephi

`NetworkX` is great for doing graph-based analyses, but not stellar at greating graph layouts for large(r) networks. However, they do have excellent built-in support for exporting graph objects in a variety of formats.

Here we'll use the `.gexf` format to save our network, as well as all of the attributes we've calculated, to a file that can be read into [Gephi](https://gephi.org/), a powerful graph visualization program.  While support for Gephi has been spotty in the recent past, it is still one of the best available options for graph visualization.

Before exporting to `.gexf`, let's use the committors we've calculated to add colors to the nodes:

```python
rgb = our_csn.colors_from_committors(comms)
our_csn.set_colors(rgb)
```

Now we have added some properties to our nodes under 'viz', which will be interpreted by Gephi:

```python
our_csn.graph.node[0]
```

    {'label': 0,
     'count': 482,
     'trim': 0.0,
     'eig_weights': 0.002595528367725156,
     'mult_weights': 0.0025955283677248217,
     'pA': 0.26406216543613925,
     'pB': 0.2947787254045238,
     'pU': 0.4411591091593356,
     'viz': {'color': {'r': 152, 'g': 170, 'b': 255, 'a': 0}}}

And we can use an internal `networkx` function to write all of this to a `.gexf` file:

```python
nx.readwrite.gexf.write_gexf(our_csn.graph.to_undirected(),'test.gexf')
```

After opening this file in Gephi, I recommend creating a layout using the "Force Atlas 2" algorithm in the layout panel.  I set the node sizes to the "eig_weights" variable, and after exporting to pdf and adding some labels, I get the following:

**That's the end of our tutorial!**  I hope you enjoyed it and you find `CSNAnalysis` useful in your research.  If you are having difficulties with the installation or running of the software, feel free to create an [issue on the Github page](https://github.com/ADicksonLab/CSNAnalysis).

```python

```


### License (MIT)
Copyright (c) 2016 Kirill Moskovtsev

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

# Model for Si with deformation potential $\Xi_u$ shear strain effects ($\Xi_{sh}$)
Run the 'Definitions' cell first, then run the cells for Si or Ge.
## Definitions:

```python
'''Python 2.7'''
from sympy import *
import numpy as np
from itertools import combinations_with_replacement, permutations
from IPython.display import Math, display

def avg_wm(Xis, structure, eta_structure):
    res = sum_wetas_etam(Xis, structure, eta_structure)
    T_power = 0
    for i in structure:
        T_power = T_power + i
    return res/M**len(structure)/T**T_power

def sum_wetas_etam(Xis, structure, eta_structure):
    '''
    This is the most important function in the code.
    See explanation in a cell below.
    The version includes shear strain term, used when including mass effects.
    '''
    prod = 1
    M = len(Xis)
    for ns, ne in zip(structure, eta_structure):
        sumi = 0
        for i in range(M):
            #sumi = sumi + (einsum('ij,ij', Xis[i], eta_t) + Ds*eta_minus[i]**2)**ns*eta_minus[i]**ne
            sumi = sumi + (einsum('ij,ij', Xis[i], eta_t) + Ds*eta_minus[i]**2 + Ds4*eta_minus[i]**4)**ns*eta_minus[i]**ne
        prod = prod*sumi
    return expand(prod)

def einsum(string, *args):
    '''
    Analog of np.einsum which works for 'object' data,
    not only numbers
    Posted by Krastanov at http://stackoverflow.com/questions/15606937/how-do-i-get-numpy-einsum-to-play-well-with-sympy
    '''
    index_groups = map(list, string.split(','))
    assert len(index_groups) == len(args)
    tensor_indices_tuples = zip(index_groups, args)
    return reduce(einsum_for_two, tensor_indices_tuples)[1]

def einsum_for_two(tensor_indices1, tensor_indices2):
    '''
    component of einsum
    '''
    string1, tensor1 = tensor_indices1
    string2, tensor2 = tensor_indices2
    sum_over_indices = set(string1).intersection(set(string2))
    new_string = string1 + string2
    axes = ([], [])
    for i in sum_over_indices:
        new_string.remove(i)
        new_string.remove(i)
        axes[0].append(string1.index(i))
        axes[1].append(string2.index(i))
    return new_string, np.tensordot(tensor1, tensor2, axes)

def show_math(x, s = '', prnt = False):
    display(Math(s + latex(x)))
    if prnt:
        print s + latex(x)

def R(i,j):
    return Rational(i,j)

sd, su, N, T, F0, F1, F2, F3, F4, F32, J, Ds, Ds4= symbols("\\Xi_d \\Xi_u N T F_{1/2} {F_{1/2}}' {F_{1/2}}'' {F_{1/2}}'''" + \
                                                  " {F_{1/2}}'''' F_{3/2} J \\Xi_{s2} \\Xi_{s4}")
eta1, eta2, eta3, eta4, eta5, eta6, theta0, x, K = symbols("\\eta_1 \\eta_2 \\eta_3 \\eta_4 \\eta_5 \\eta_6 \\theta_0 x K")
eta_t = [[eta1, Rational(1,2)*eta6, Rational(1,2)*eta5],[Rational(1,2)*eta6, eta2, Rational(1,2)*eta4],\
         [Rational(1,2)*eta5, Rational(1,2)*eta4, eta3]]
eta_minus = [Rational(1,2)*eta4, Rational(1,2)*eta4, Rational(1,2)*eta5, \
             Rational(1,2)*eta5, Rational(1,2)*eta6, Rational(1,2)*eta6]
M = 6
unit = [[1,0,0], [0,1,0], [0,0,1]]
unitm = Matrix(unit)

Xis = []
for i in range(3):
    for j in [-1, 1]:
        a = np.reshape(np.array([0,0,0]), (1,3))
        a[0,i] = j
        at = np.reshape(a, (3,1))
        Xis.append(sd*unitm + su*np.dot(at, a))

```

**'sum_wetas_etam**(Xis, structure, eta_structure)':

Calculates the following expression:

$$
\left(\sum_\alpha \left(\delta E^{(\alpha)}\right)^{\text{structure[0]}}\eta_\alpha^{eta\_structure[0]}\right)...\left(\sum_\alpha \left(\delta E^{(\alpha)}\right)^{\text{structure[r]}}\eta_\alpha^{eta\_structure[r]}\right)
$$
where $\delta E^{(\alpha)}$ is the shift of valley $\alpha$, $\eta_\alpha$ defined as $\varepsilon_\alpha$ in the appendix of the paper, but $\eta_\alpha$ is written in Voigt notation.

If we ignore *'eta_structure'*, then $\Delta_\varepsilon$ structures from the paper are written as follows, for example:
$$
    \overline{\Delta_\varepsilon^k}\overline{\Delta_\varepsilon^n} = T^{-(k+n)}M^{-2}\left(\text{sum_wetas_etam}(Xis, (k,n), (0,0))\right)
$$

## Calculate expressions for $\delta c_{ij...}$ for doped Si:
$\Xi_{s2} \equiv \Xi_{sh}$;

$\Xi_{s4}$ is the hypothetical deformation potential for a quartic in strain shift of valleys, and it is assumed small.

```python
#Free energy terms expressed through strain
Free1 = N*T*avg_wm(Xis, (1,), (0,))
Free2 = -N*T/F0*(Rational(1,2)*F1*(avg_wm(Xis, (2,), (0,)) - avg_wm(Xis, (1,1), (0,0)) ))

Free3 = N*T/F0*(Rational(1,6)*F2*(avg_wm(Xis, (3,),(0,)) - 3*avg_wm(Xis, (2,1),(0,0)) + \
                                  2*avg_wm(Xis, (1,1,1),(0,0,0))) )

Free4_1 = Rational(1,24)*N*T*F3/F0*(-6*avg_wm(Xis, (2,1,1), (0,0,0)) + 4*avg_wm(Xis, (3,1),(0,0)) - avg_wm(Xis, (4,),(0,)) +\
                                    3*avg_wm(Xis, (1,1,1,1),(0,0,0,0)))
Free4_2 = Rational(1,8)*N*T*F2**2/F0/F1*(avg_wm(Xis, (2,2),(0,0)) - 2*avg_wm(Xis, (2,1,1),(0,0,0)) + \
                                         avg_wm(Xis, (1,1,1,1), (0,0,0,0)) )

Free4 = expand(Free4_1 + Free4_2)

Free_all = expand(Free1 + Free2 + Free3 + Free4)
subs_dict = {eta1: eta1*x, eta2: eta2*x, eta3: eta3*x, eta4: eta4*x, eta5: eta5*x, eta6: eta6*x}
Free_all_x = Free_all.subs(subs_dict)
Free_xdict = collect(Free_all_x, x, evaluate=False)
Free2 = Free_xdict[x**2]
Free3 = Free_xdict[x**3]
Free4 = Free_xdict[x**4]

#Differentiate to find elastic constants:

dc11 = simplify(diff(Free2, eta1, eta1))
dc12 = simplify(diff(Free2, eta1, eta2))
dc44 = simplify(diff(Free2, eta4, eta4))
print "Second order:"
show_math(dc11, '\\delta c_{11} = ')
show_math(dc12, '\\delta c_{12} = ')
show_math(dc44, '\\delta c_{44} = ')

dc111 = diff(Free3, eta1, eta1, eta1)
dc112 = diff(Free3, eta1, eta1, eta2)
dc123 = diff(Free3, eta1, eta2, eta3)
dc144 = diff(Free3, eta1, eta4, eta4)
dc155 = diff(Free3, eta1, eta5, eta5)
dc456 = diff(Free3, eta4, eta5, eta6)

print "Third order:"
show_math(dc111, '\\delta c_{111} = ')
show_math(dc112, '\\delta c_{112} = ')
show_math(dc123, '\\delta c_{123} = ')
show_math(dc144, '\\delta c_{144} = ')
show_math(dc155, '\\delta c_{155} = ')
show_math(dc456, '\\delta c_{456} = ', prnt = False)

dc1111 = expand(diff(Free4, eta1, eta1, eta1, eta1))
dc1112 = expand(diff(Free4, eta1, eta1, eta1, eta2))
dc1122 = expand(diff(Free4, eta1, eta1, eta2, eta2))
dc1123 = expand(diff(Free4, eta1, eta1, eta2, eta3))
dc1144 = expand(diff(Free4, eta1, eta1, eta4, eta4))
dc1155 = expand(diff(Free4, eta1, eta1, eta5, eta5))
dc1244 = expand(diff(Free4, eta1, eta2, eta4, eta4))
dc1266 = expand(diff(Free4, eta1, eta2, eta6, eta6))
dc1456 = expand(diff(Free4, eta1, eta4, eta5, eta6))
dc4444 = expand(diff(Free4, eta4, eta4, eta4, eta4))
dc4455 = expand(diff(Free4, eta4, eta4, eta5, eta5))
print "Fourth order:"
show_math(dc1111, '\\delta c_{1111} = ')
show_math(dc1112, '\\delta c_{1112} = ')
show_math(dc1122, '\\delta c_{1122} = ')
show_math(dc1123, '\\delta c_{1123} = ')
show_math(dc1144, '\\delta c_{1144} = ')
show_math(dc1155, '\\delta c_{1155} = ')
show_math(dc1244, '\\delta c_{1244} = ')
show_math(dc1266, '\\delta c_{1266} = ')
show_math(dc1456, '\\delta c_{1456} = ')
show_math(dc4444, '\\delta c_{4444} = ', prnt = False)
show_math(dc4455, '\\delta c_{4455} = ', prnt = False)
```

    Second order:

$$\delta c_{11} = - \frac{2 N \Xi_u^{2} {F_{1/2}}'}{9 F_{1/2} T}$$

$$\delta c_{12} = \frac{N \Xi_u^{2} {F_{1/2}}'}{9 F_{1/2} T}$$

$$\delta c_{44} = \frac{N \Xi_{s2}}{6}$$

    Third order:

$$\delta c_{111} = \frac{2 N \Xi_u^{3} {F_{1/2}}''}{27 F_{1/2} T^{2}}$$

$$\delta c_{112} = - \frac{N \Xi_u^{3} {F_{1/2}}''}{27 F_{1/2} T^{2}}$$

$$\delta c_{123} = \frac{2 N \Xi_u^{3} {F_{1/2}}''}{27 F_{1/2} T^{2}}$$

$$\delta c_{144} = - \frac{N \Xi_u \Xi_{s2} {F_{1/2}}'}{9 F_{1/2} T}$$

$$\delta c_{155} = \frac{N \Xi_u \Xi_{s2} {F_{1/2}}'}{18 F_{1/2} T}$$

$$\delta c_{456} = 0$$

    Fourth order:

$$\delta c_{1111} = - \frac{2 N \Xi_u^{4} {F_{1/2}}'''}{27 F_{1/2} T^{3}} + \frac{4 N \Xi_u^{4} {F_{1/2}}''^{2}}{27 F_{1/2} T^{3} {F_{1/2}}'}$$

$$\delta c_{1112} = \frac{N \Xi_u^{4} {F_{1/2}}'''}{27 F_{1/2} T^{3}} - \frac{2 N \Xi_u^{4} {F_{1/2}}''^{2}}{27 F_{1/2} T^{3} {F_{1/2}}'}$$

$$\delta c_{1122} = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{27 F_{1/2} T^{3}} + \frac{2 N \Xi_u^{4} {F_{1/2}}''^{2}}{27 F_{1/2} T^{3} {F_{1/2}}'}$$

$$\delta c_{1123} = 0$$

$$\delta c_{1144} = \frac{N \Xi_u^{2} \Xi_{s2} {F_{1/2}}''}{27 F_{1/2} T^{2}}$$

$$\delta c_{1155} = - \frac{N \Xi_u^{2} \Xi_{s2} {F_{1/2}}''}{54 F_{1/2} T^{2}}$$

$$\delta c_{1244} = - \frac{N \Xi_u^{2} \Xi_{s2} {F_{1/2}}''}{54 F_{1/2} T^{2}}$$

$$\delta c_{1266} = \frac{N \Xi_u^{2} \Xi_{s2} {F_{1/2}}''}{27 F_{1/2} T^{2}}$$

$$\delta c_{1456} = 0$$

$$\delta c_{4444} = \frac{N \Xi_{s4}}{2} - \frac{N \Xi_{s2}^{2} {F_{1/2}}'}{6 F_{1/2} T}$$

$$\delta c_{4455} = \frac{N \Xi_{s2}^{2} {F_{1/2}}'}{36 F_{1/2} T}$$

## Save single-crystal $c$-expressions
to reuse them in the 'mode_calculation.ipynb'

```python
import pickle
c11, c12, c44, c111, c112, c123, c144, c155, c456 = \
symbols('c11 c12 c44 c111 c112 c123 c144 c155 c456')
c1111, c1112, c1122, c1123, c1144, c1155, c1244, c1266, c1456, c4444, c4455 = \
symbols('c1111 c1112 c1122 c1123 c1144 c1155 c1244 c1266 c1456 c4444 c4455')

c = {c11:dc11, c12:dc12, c44:dc44, c111:dc111, c112:dc112, c123:dc123, c144:dc144, \
            c155:dc155, c456:dc456, c1111:dc1111, c1112:dc1112, c1122:dc1122, c1123:dc1123, \
             c1144:dc1144,\
            c1155:dc1155, c1244:dc1244, c1266:dc1266, c1456:dc1456, c4444:dc4444, c4455:dc4455}
fname = './single_expressions/c_values_single.dat'
if not os.path.exists(os.path.dirname(fname)):
        os.makedirs(os.path.dirname(fname))
        print './single_expressions/ created'
ff = open(fname, 'w')
pickle.dump(c, ff)
ff.close()
```

# Germanium

```python
def sum_wetas_etam(Xis, structure, eta_structure):
    prod = 1
    M = len(Xis)
    for ns, ne in zip(structure, eta_structure):
        sumi = 0
        for i in range(M):
            #sumi = sumi + (einsum('ij,ij', Xis[i], eta_t) + Ds*eta_minus[i]**2)**ns*eta_minus[i]**ne
            sumi = sumi + (einsum('ij,ij', Xis[i], eta_t))**ns
        prod = prod*sumi
    return expand(prod)

M = 4
unit_vectors = np.array([[1,1,1], [-1,1,1], [1,-1, 1], [-1, -1, 1]])/sqrt(3)
unitm = np.array([[1,0,0], [0,1,0], [0,0,1]])
#unitm = Matrix(unit)

Xis = []
for a in unit_vectors:
    #for j in [-1/sqrt(3), 1/sqrt(3)]:
    a = np.reshape(a, (1,3))
    #a[0,2] = j
    at = np.reshape(a, (3,1))
    Xis.append(sd*unitm + su*np.dot(at, a))

Free1 = N*T*avg_wm(Xis, (1,), (0,))
Free2 = -N*T/F0*(Rational(1,2)*F1*(avg_wm(Xis, (2,), (0,)) - avg_wm(Xis, (1,1), (0,0)) ))
#Free2 = simplify(Free2)

Free3 = N*T/F0*(Rational(1,6)*F2*(avg_wm(Xis, (3,),(0,)) - 3*avg_wm(Xis, (2,1),(0,0)) + \
                                  2*avg_wm(Xis, (1,1,1),(0,0,0))) )

Free4_1 = Rational(1,24)*N*T*F3/F0*(-6*avg_wm(Xis, (2,1,1), (0,0,0)) + 4*avg_wm(Xis, (3,1),(0,0)) - avg_wm(Xis, (4,),(0,)) +\
                                    3*avg_wm(Xis, (1,1,1,1),(0,0,0,0)))
Free4_2 = Rational(1,8)*N*T*F2**2/F0/F1*(avg_wm(Xis, (2,2),(0,0)) - 2*avg_wm(Xis, (2,1,1),(0,0,0)) + \
                                         avg_wm(Xis, (1,1,1,1), (0,0,0,0)) )

Free4 = expand(Free4_1 + Free4_2)

Free_all = expand(Free1 + Free2 + Free3 + Free4).subs({Ds: 0, Ds4: 0}) #no \Xi_sh for Ge
subs_dict = {eta1: eta1*x, eta2: eta2*x, eta3: eta3*x, eta4: eta4*x, eta5: eta5*x, eta6: eta6*x}
Free_all_x = Free_all.subs(subs_dict)
Free_xdict = collect(Free_all_x, x, evaluate=False)
Free2 = Free_xdict[x**2]
Free3 = Free_xdict[x**3]
Free4 = Free_xdict[x**4]

dc11 = simplify(diff(Free2, eta1, eta1))
dc12 = simplify(diff(Free2, eta1, eta2))
dc44 = simplify(diff(Free2, eta4, eta4))
print "Second order:"
show_math(dc11, '\\delta c_{11} = ')
show_math(dc12, '\\delta c_{12} = ')
show_math(dc44, '\\delta c_{44} = ')

dc111 = diff(Free3, eta1, eta1, eta1)
dc112 = diff(Free3, eta1, eta1, eta2)
dc123 = diff(Free3, eta1, eta2, eta3)
dc144 = diff(Free3, eta1, eta4, eta4)
dc155 = diff(Free3, eta1, eta5, eta5)
dc456 = diff(Free3, eta4, eta5, eta6)

print "Third order:"
show_math(dc111, '\\delta c_{111} = ')
show_math(dc112, '\\delta c_{112} = ')
show_math(dc123, '\\delta c_{123} = ')
show_math(dc144, '\\delta c_{144} = ')
show_math(dc155, '\\delta c_{155} = ')
show_math(dc456, '\\delta c_{456} = ', prnt = True)

dc1111 = expand(diff(Free4, eta1, eta1, eta1, eta1))
dc1112 = expand(diff(Free4, eta1, eta1, eta1, eta2))
dc1122 = expand(diff(Free4, eta1, eta1, eta2, eta2))
dc1123 = expand(diff(Free4, eta1, eta1, eta2, eta3))
dc1144 = expand(diff(Free4, eta1, eta1, eta4, eta4))
dc1155 = expand(diff(Free4, eta1, eta1, eta5, eta5))
dc1244 = expand(diff(Free4, eta1, eta2, eta4, eta4))
dc1266 = expand(diff(Free4, eta1, eta2, eta6, eta6))
dc1456 = expand(diff(Free4, eta1, eta4, eta5, eta6))
dc4444 = expand(diff(Free4, eta4, eta4, eta4, eta4))
dc4455 = expand(diff(Free4, eta4, eta4, eta5, eta5))
print "Fourth order:"
show_math(dc1111, '\\delta c_{1111} = ')
show_math(dc1112, '\\delta c_{1112} = ')
show_math(dc1122, '\\delta c_{1122} = ')
show_math(dc1123, '\\delta c_{1123} = ')
show_math(dc1144, '\\delta c_{1144} = ')
show_math(dc1155, '\\delta c_{1155} = ')
show_math(dc1244, '\\delta c_{1244} = ')
show_math(dc1266, '\\delta c_{1266} = ')
show_math(dc1456, '\\delta c_{1456} = ')
show_math(dc4444, '\\delta c_{4444} = ', prnt = True)
show_math(dc4455, '\\delta c_{4455} = ', prnt = True)
```

    Second order:

$$\delta c_{11} = 0$$

$$\delta c_{12} = 0$$

$$\delta c_{44} = - \frac{N \Xi_u^{2} {F_{1/2}}'}{9 F_{1/2} T}$$

    Third order:

$$\delta c_{111} = 0$$

$$\delta c_{112} = 0$$

$$\delta c_{123} = 0$$

$$\delta c_{144} = 0$$

$$\delta c_{155} = 0$$

$$\delta c_{456} = \frac{N \Xi_u^{3} {F_{1/2}}''}{27 F_{1/2} T^{2}}$$

    \delta c_{456} = \frac{N \Xi_u^{3} {F_{1/2}}''}{27 F_{1/2} T^{2}}
    Fourth order:

$$\delta c_{1111} = 0$$

$$\delta c_{1112} = 0$$

$$\delta c_{1122} = 0$$

$$\delta c_{1123} = 0$$

$$\delta c_{1144} = 0$$

$$\delta c_{1155} = 0$$

$$\delta c_{1244} = 0$$

$$\delta c_{1266} = 0$$

$$\delta c_{1456} = 0$$

$$\delta c_{4444} = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{81 F_{1/2} T^{3}} + \frac{N \Xi_u^{4} {F_{1/2}}''^{2}}{27 F_{1/2} T^{3} {F_{1/2}}'}$$

    \delta c_{4444} = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{81 F_{1/2} T^{3}} + \frac{N \Xi_u^{4} {F_{1/2}}''^{2}}{27 F_{1/2} T^{3} {F_{1/2}}'}

$$\delta c_{4455} = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{81 F_{1/2} T^{3}} + \frac{N \Xi_u^{4} {F_{1/2}}''^{2}}{81 F_{1/2} T^{3} {F_{1/2}}'}$$

    \delta c_{4455} = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{81 F_{1/2} T^{3}} + \frac{N \Xi_u^{4} {F_{1/2}}''^{2}}{81 F_{1/2} T^{3} {F_{1/2}}'}

```python

```

```python
print '\delta c_{{11}} &= {}'.format(latex(dc11)) + '\\\\'
print '\delta c_{{12}} & = {}'.format(latex(dc12)) + '\\\\'
print '\delta c_{{44}} & = {}'.format(latex(dc44)) + '\\\\'
print '\delta c_{{111}} & = {}'.format(latex(dc111)) + '\\\\'
print '\delta c_{{112}} & = {}'.format(latex(dc112)) + '\\\\'
print '\delta c_{{123}} & = {}'.format(latex(dc123)) + '\\\\'
print '\delta c_{{144}} & = {}'.format(latex(dc144)) + '\\\\'
print '\delta c_{{155}} & = {}'.format(latex(dc155)) + '\\\\'
print '\delta c_{{456}} & = {}'.format(latex(dc456)) + '\\\\'
print '\delta c_{{1111}} & = {}'.format(latex(dc1111)) + '\\\\'
print '\delta c_{{1112}} & = {}'.format(latex(dc1112)) + '\\\\'
print '\delta c_{{1122}} & = {}'.format(latex(dc1122)) + '\\\\'
print '\delta c_{{1123}} & = {}'.format(latex(dc1123)) + '\\\\'
print '\delta c_{{1144}} & = {}'.format(latex(dc1144)) + '\\\\'
print '\delta c_{{1155}} & = {}'.format(latex(dc1155)) + '\\\\'
print '\delta c_{{1244}} & = {}'.format(latex(dc1244)) + '\\\\'
print '\delta c_{{1266}} & = {}'.format(latex(dc1266)) + '\\\\'
print '\delta c_{{1456}} & = {}'.format(latex(dc1456)) + '\\\\'
print '\delta c_{{4444}} & = {}'.format(latex(dc4444)) + '\\\\'
print '\delta c_{{4455}} & = {}'.format(latex(dc4455))
```

    \delta c_{11} &= 0\\
    \delta c_{12} & = 0\\
    \delta c_{44} & = - \frac{N \Xi_u^{2} {F_{1/2}}'}{9 F_{1/2} T}\\
    \delta c_{111} & = 0\\
    \delta c_{112} & = 0\\
    \delta c_{123} & = 0\\
    \delta c_{144} & = 0\\
    \delta c_{155} & = 0\\
    \delta c_{456} & = \frac{N \Xi_u^{3} {F_{1/2}}''}{27 F_{1/2} T^{2}}\\
    \delta c_{1111} & = 0\\
    \delta c_{1112} & = 0\\
    \delta c_{1122} & = 0\\
    \delta c_{1123} & = 0\\
    \delta c_{1144} & = 0\\
    \delta c_{1155} & = 0\\
    \delta c_{1244} & = 0\\
    \delta c_{1266} & = 0\\
    \delta c_{1456} & = 0\\
    \delta c_{4444} & = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{81 F_{1/2} T^{3}} + \frac{N \Xi_u^{4} {F_{1/2}}''^{2}}{27 F_{1/2} T^{3} {F_{1/2}}'}\\
    \delta c_{4455} & = - \frac{N \Xi_u^{4} {F_{1/2}}'''}{81 F_{1/2} T^{3}} + \frac{N \Xi_u^{4} {F_{1/2}}''^{2}}{81 F_{1/2} T^{3} {F_{1/2}}'}


Histograms of Oriented Gradients (HOG)

Introduction

正如我们在ORB算法中看到的那样，我们可以使用图像中的关键点来进行基于关键点的匹配，以此来检测图像中的对象。如果你想检测具有许多不受背景影响的一致内部特征的对象，这些类型的算法就会非常有用。比如，这些算法适用于人脸检测，因为人脸具有许多不受图像背景影响的一致内部特征，例如眼睛、鼻子和嘴巴。然而，如果要尝试进行更一般的对象识别（例如，图像中的行人检测），这些类型的算法就不会那么有用了。原因是行人与人脸不同，没有与人脸一样的内部特征，毕竟每个行人的体形和走路风格都不同（见图1）。这意味着每个行人都有不同的内部特征，因此我们需要一些可以用于更一般地描述行人的方法。

Fig. 1. - Pedestrians.

其中，有一种方法是尝试通过轮廓来检测行人。通过轮廓（边界）可以检测到图像中的对象，但这种方法非常具有难度，因为我们必须处理背景和前景之间的对比，而完成这一点是相当困难的。例如，假设你想要检测在白色建筑物前行走的图像中的行人，并且她穿着白色外套和黑色裤子（见图2）。我们可以在图2中看到，由于图像的背景大多是白色的，因此黑色裤子具有非常高的对比度，但是白色外套具有非常低的对比度。在这种情况下，非常容易就可以检测到裤子的边缘，但要检测外套的边缘，就会非常困难。在这个时候，就需要使用**HOG** 了。HOG代表**方向梯度直方图** ，它是由Navneet Dalal和Bill Triggs于2005年首次提出的。

Fig. 2. - High and Low Contrast.

HOG算法的工作原理是通过创建图像中梯度方向分布的直方图，然后以非常特殊的方式对它们进行归一化。即使在对比度非常低的情况下，这种特殊的归一化也使得HOG在检测物体边缘方面非常有效。这些归一化的直方图会被放在一个特征向量中，称为HOG描述子。HOG可用于训练机器学习算法，如支持向量机（SVM），使其根据它们的边界（边缘）检测图像中的对象 。由于其巨大的成功和可靠性，HOG已成为用于对象检测的计算机视觉中最广泛使用的算法之一。

在这个notebook中，你将要学习的内容如下：

* HOG算法的工作原理
* 如何使用OpenCV创建HOG描述子
* 如何可视化HOG描述子。

# HOG算法

顾名思义，HOG算法是一种基于从图像梯度的方向创建直方图的算法。 HOG算法是通过一系列步骤实现的，具体如下：

1. 对于某个特定对象的图像，设置一个可以覆盖图像中整个对象的检测窗口或者你感兴趣的区域（参见图3）。

2. 计算检测窗口中每个像素的梯度的大小和方向。

3. 将检测窗口划分为连接的像素*单元格* ，其中，所有单元格的大小相同（参见图3）。该单元格的大小是一个自由参数，通常选择它的目的在于匹配想要检测的特征的范围。例如，在64×128像素检测窗口中，6至8像素宽的正方形单元格适合于检测人体肢体。

4. 为每个单元格创建直方图，首先将每个单元格中所有像素的渐变方向分组为特定数量的方向（角度）区间（bin）；然后将每个角度区间中梯度的梯度大小相加（见图3）。直方图中的区间数是一个自由参数，通常设置为9个角度区间。

5. 将相邻的单元格分组成*块*（见图3）。每个块中的单元格数是一个自由参数，所有块必须具有相同的大小。每个块之间的距离（称为步长）是一个自由参数，但通常设置为块大小的一半，在这种情况下，你将会获得重叠块（*请查看下面的视频*）。这样，HOG算法会根据经验更好地与重叠块一起运行。

6. 使用每个块中包含的单元格来归一化该块中的单元格直方图（参见图3）。如果你有重叠块，这说明大多数单元格将根据不同的块进行归一化（*请查看下面的视频*）。因此，相同的单元格可以具有几种不同的归一化。

7. 将所有块中的所有归一化直方图收集到称为HOG描述子的单个特征向量中。

8. 使用来自相同类型对象的大量图像的结果HOG描述子来训练机器学习算法（例如SVM），进而检测图像中那些类型的对象。 例如，你可以使用来自行人的大量图像的HOG描述子来训练SVM，进而检测图像中的行人。其中，训练是通过你想要在图像中检测到的对象的正反面示例完成的。

9. 训练完SVM之后，你需要使用滑动窗口方法来尝试检测和定位图像中的对象。 要想检测图像中的对象，你需要找到看起来类似于SVM所学习的HOG类型的图像部分。

Fig. 3. - HOG Diagram.

Vid. 1. - HOG Animation.

# 为什么HOG算法如此有效

如上所述，HOG通过在称为*单元格*的图像的局部部分中添加特定方向的梯度的大小来创建直方图。这样做，我们保证了更强的梯度会对其各自的角度区间的大小产生更多影响，同时由噪声导致的较弱与随机定向梯度的影响被最小化。总之，直方图会以这种方式告诉我们每个单元格的主导梯度方向。

### 处理对比

现在，由于局部照明的变化以及背景和前景之间的对比度，主导方向的梯度差异会非常大。

为了弄清楚背景与前景对比度差异，HOG算法尝试在本地检测边缘。为了做到这一点，它定义了一组称为**块**的单元格，并使用这个本地单元格组对该直方图进行归一化。通过局部归一化，HOG算法可以非常可靠地检测每个块中的边缘。这就叫做**块归一化**。

除了使用块归一化之外，HOG算法还使用重叠块来提高其性能。通过使用重叠块，每个单元格向最终HOG描述子贡献若干独立分量，其中每个分量对应于相对于不同块归一化的单元格。这似乎是多余的，但是，经验证明，通过相对于不同的本地块对每个单元格进行几次归一化，HOG算法的性能会显著提高。

### 加载图像并导入资源

要创建HOG描述子，第一步是将所需的包加载到Python中并加载图像。

我们首先使用OpenCV加载三角形图块的图像。因为，`cv2.imread()`函数会将图像加载为BGR，而我们需要将图像转换为RGB，这样我们就可以使用正确的颜色显示它。与之前一样，我们会将BGR图像转换为灰度图像进行分析。

```python
import cv2
import numpy as np
import matplotlib.pyplot as plt

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Load the image
image = cv2.imread('./images/triangle_tile.jpeg')

# Convert the original image to RGB
original_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Convert the original image to gray scale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Print the shape of the original and gray scale images
print('The original image has shape: ', original_image.shape)
print('The gray scale image has shape: ', gray_image.shape)

# Display the images
plt.subplot(121)
plt.imshow(original_image)
plt.title('Original Image')
plt.subplot(122)
plt.imshow(gray_image, cmap='gray')
plt.title('Gray Scale Image')
plt.show()
```

    The original image has shape:  (250, 250, 3)
    The gray scale image has shape:  (250, 250)

    <matplotlib.figure.Figure at 0x7f4b49d45e80>

# 创建HOG描述子

我们将使用OpenCV的`HOGDescriptor`类来创建HOG描述子， 使用`HOGDescriptor()`函数设置HOG描述子的参数。`HOGDescriptor()`函数的参数及其默认值如下：

`cv2.HOGDescriptor(win_size = (64, 128),
                  block_size = (16, 16),
                  block_stride = (8, 8),
                  cell_size = (8, 8),
                  nbins = 9,
                  win_sigma = DEFAULT_WIN_SIGMA,
                  threshold_L2hys = 0.2,
                  gamma_correction = true,
                  nlevels = DEFAULT_NLEVELS)`

参数：

* **win_size** – *Size*
  检测窗口的大小（以像素为单位）（*高度，宽度*）。定义一个感兴趣的区域。它必须是单元格尺寸的整数倍。

* **block_size** – *Size*
  块大小（以像素为单位）（*宽度，高度*）。定义每个块中有多少个单元格。它必须是单元格尺寸的整数倍，并且必须小于检测窗口。块越小，你将获得的细节越详细。

* **block_stride** – *Size*
  以像素为单位块步长（*水平，垂直*）。它必须是单元格尺寸的整数倍。 `block_stride`定义了相邻块之间的距离，例如，水平8个像素和垂直8个像素。`block_stride`越长，算法就会运行得越快（因为评估的块会越少），但算法也可能不再运行。

* **cell_size** – *Size*
  单元格大小，以像素为单位（*宽度，高度*）。确定格单元的大小。单元格越小，你将获得的细节越详细。

* **nbins** – *int*
  直方图的区间数。确定用于制作直方图的角度区间数量。使用的区间越多，就可以获取更多的渐变方向。 HOG使用无符号渐变，因此角度区间的值将介于0和180度之间。

* **win_sigma** – *double*
  高斯平滑窗口参数。通过在计算直方图之前对每个像素应用高斯空间窗口来平滑块边缘附近的像素，从而改善HOG算法的性能。

* **threshold_L2hys** – *double*
   L2-Hys（Lowe式限幅L2范数）归一化方法收缩率。 L2-Hys方法用于归一化块，它由L2范数和剪切以及重归一化组成。限幅会将每个块的描述子向量的最大值限制为具有给定阈值的值（默认为0.2）。限幅之后，如*IJCV*，60（2）：91-110,2004中所述，重新归一化描述子矢量。

* **gamma_correction** – *bool*
  用于指定是否需要伽马校正预处理的标志。执行伽马校正可以在一定程度上提高HOG算法的性能。

* **nlevels** – *int*
  检测窗口增加的最大数量。

我们可以看到，`cv2.HOGDescriptor()`函数支持各种参数。前几个参数（`block_size, block_stride, cell_size`以及`nbins`）是最有可能更改的参数。其他参数都可以放心地保留其默认值，这样你会获得很不错的结果。

在下面的代码中，我们将使用 `cv2.HOGDescriptor()`函数来设置单元格大小、块大小、块步长以及HOG描述子的直方图区间数。然后，我们将使用`.compute(image)`方法计算给定`image`的HOG描述子（特征向量）。

```python
# Specify the parameters for our HOG descriptor

# Cell Size in pixels (width, height). Must be smaller than the size of the detection window
# and must be chosen so that the resulting Block Size is smaller than the detection window.
cell_size = (6, 6)

# Number of cells per block in each direction (x, y). Must be chosen so that the resulting
# Block Size is smaller than the detection window
num_cells_per_block = (2, 2)

# Block Size in pixels (width, height). Must be an integer multiple of Cell Size.
# The Block Size must be smaller than the detection window
block_size = (num_cells_per_block[0] * cell_size[0],
              num_cells_per_block[1] * cell_size[1])

# Calculate the number of cells that fit in our image in the x and y directions
x_cells = gray_image.shape[1] // cell_size[0]
y_cells = gray_image.shape[0] // cell_size[1]

# Horizontal distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (x_cells - num_cells_per_block[0]) / h_stride = integer.
h_stride = 1

# Vertical distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (y_cells - num_cells_per_block[1]) / v_stride = integer.
v_stride = 1

# Block Stride in pixels (horizantal, vertical). Must be an integer multiple of Cell Size
block_stride = (cell_size[0] * h_stride, cell_size[1] * v_stride)

# Number of gradient orientation bins
num_bins = 9

# Specify the size of the detection window (Region of Interest) in pixels (width, height).
# It must be an integer multiple of Cell Size and it must cover the entire image. Because
# the detection window must be an integer multiple of cell size, depending on the size of
# your cells, the resulting detection window might be slightly smaller than the image.
# This is perfectly ok.
win_size = (x_cells * cell_size[0] , y_cells * cell_size[1])

# Print the shape of the gray scale image for reference
print('\nThe gray scale image has shape: ', gray_image.shape)
print()

# Print the parameters of our HOG descriptor
print('HOG Descriptor Parameters:\n')
print('Window Size:', win_size)
print('Cell Size:', cell_size)
print('Block Size:', block_size)
print('Block Stride:', block_stride)
print('Number of Bins:', num_bins)
print()

# Set the parameters of the HOG descriptor using the variables defined above
hog = cv2.HOGDescriptor(win_size, block_size, block_stride, cell_size, num_bins)

# Compute the HOG Descriptor for the gray scale image
hog_descriptor = hog.compute(gray_image)
```

#  HOG描述子中的元素数量

由此得到的HOG描述子（特征向量）包含来自在一个长向量中连接的检测窗口中的所有块的所有单元格的归一化直方图。因此，HOG特征向量的大小将由检测窗口中的块总数乘以每个块的单元格数乘以定向区间的数量来得出，公式如下：

\begin{equation}
\mbox{total_elements} = (\mbox{total_number_of_blocks})\mbox{ } \times \mbox{ } (\mbox{number_cells_per_block})\mbox{ } \times \mbox{ } (\mbox{number_of_bins})
\end{equation}

如果没有重叠块（*即* ` block_stride`等于`block_size`），则可以通过将检测窗口的大小除以块大小来计算块的总数。但是，在一般情况下，必须考虑到我们有重叠块的事实。要想知道一般情况下的块总数（*即* 对于任何`block_stride`和`block_size`），可以使用下面给出的公式：

\begin{equation}
\mbox{Total}_i = \left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right)\left( \frac{\mbox{window_size}_i}{\mbox{block_size}_i} \right) - \left [\left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right) - 1 \right]; \mbox{  for  } i = x,y
\end{equation}

其中， Total$_x$ ，是沿检测窗口宽度的块总数， Total$_y$，是沿着检测窗口高度的块总数。这个  Total$_x$ 和 Total$_y$公式考虑了重叠产生的额外块。在计算 Total$_x$ 和 Total$_y$之后，我们可以通过 Total$_x$ 乘以 Total$_y$来获取检测窗口中的块总数。其实，上述公式可以进行相当大的简化，因为`block_size`，`block_stride`和`window_size`都是根据`cell_size`定义的。通过进行适当的替换和删除，上述公式简化为：

\begin{equation}
\mbox{Total}_i = \left(\frac{\mbox{cells}_i - \mbox{num_cells_per_block}_i}{N_i}\right) + 1\mbox{  }; \mbox{  for  } i = x,y
\end{equation}

其中， cells$_x$是沿检测窗口宽度的单元格总数， cells$_y$，是沿检测窗口高度的单元格总数。  $N_x$是以`cell_size`为单位的水平块步长，$N_y$是以cell_size`cell_size`为单位的垂直块步长。

接下来，让我们计算一下HOG特征向量的元素数量应该是多少，并检查它是否与上面计算的HOG描述子的形状相匹配。

```python
# Calculate the total number of blocks along the width of the detection window
tot_bx = np.uint32(((x_cells - num_cells_per_block[0]) / h_stride) + 1)

# Calculate the total number of blocks along the height of the detection window
tot_by = np.uint32(((y_cells - num_cells_per_block[1]) / v_stride) + 1)

# Calculate the total number of elements in the feature vector
tot_els = (tot_bx) * (tot_by) * num_cells_per_block[0] * num_cells_per_block[1] * num_bins

# Print the total number of elements the HOG feature vector should have
print('\nThe total number of elements in the HOG Feature Vector should be: ',
      tot_bx, 'x',
      tot_by, 'x',
      num_cells_per_block[0], 'x',
      num_cells_per_block[1], 'x',
      num_bins, '=',
      tot_els)

# Print the shape of the HOG Descriptor to see that it matches the above
print('\nThe HOG Descriptor has shape:', hog_descriptor.shape)
print()
```

# 可视化HOG描述子

我们可以通过将与每个单元格相关联的直方图绘制为矢量集合来可视化HOG描述子。为此，我们将直方图中的每个区间绘制为单个向量，其大小由区间的高度而定，其方向由与其关联的角度区间而定。由于任何给定的单元格，可能有多个与之关联的直方图。由于重叠块的存在，我们将选择平均分配每个单元格的所有直方图，这样便于为每个单元格生成单个直方图。

要可视化HOG描述子，OpenCV没有简单的方法可用，因此我们必须先进行一些操作才能使其可视化。这要从改变HOG描述子开始，这样可以使我们的计算更容易。然后，我们将计算每个单元格的平均直方图，最后我们将直方图区间转换为矢量。只要有了矢量，就可以为图像中的每个单元格绘制相应的矢量图。

下面的代码会生成一个交互式图，你可以与之进行交互。该图包含：
* 灰度图像，
* HOG描述子（特征向量），
* HOG描述子的放大部分，
* 所选单元格的直方图。

**你可以通过灰度图像或HOG描述子图像上的任意位置来选择特定单元格。** 单击任一图像后，将出现一个*洋红色*矩形，它会显示你选择的单元。缩放窗口将显示所选单元格周围的HOG描述子的一个放大版本，而直方图将显示所选单元格的相应直方图。交互式窗口底部还有按钮，你可以选择其他功能，例如平移，并根据需要提供了保存图形的选项。点击Home按钮，会将图形返回到其默认值。

**注意**: 如果你在优达学城工作区中运行此notebook，则交互式图大约会有2秒的延迟。这就是说，如果单击图像进行放大，则交互图的刷新将需要大约2秒钟。

```python
%matplotlib notebook

import copy
import matplotlib.patches as patches

# Set the default figure size
plt.rcParams['figure.figsize'] = [9.8, 9]

# Reshape the feature vector to [blocks_y, blocks_x, num_cells_per_block_x, num_cells_per_block_y, num_bins].
# The blocks_x and blocks_y will be transposed so that the first index (blocks_y) referes to the row number
# and the second index to the column number. This will be useful later when we plot the feature vector, so
# that the feature vector indexing matches the image indexing.
hog_descriptor_reshaped = hog_descriptor.reshape(tot_bx,
                                                 tot_by,
                                                 num_cells_per_block[0],
                                                 num_cells_per_block[1],
                                                 num_bins).transpose((1, 0, 2, 3, 4))

# Print the shape of the feature vector for reference
print('The feature vector has shape:', hog_descriptor.shape)

# Print the reshaped feature vector
print('The reshaped feature vector has shape:', hog_descriptor_reshaped.shape)

# Create an array that will hold the average gradients for each cell
ave_grad = np.zeros((y_cells, x_cells, num_bins))

# Print the shape of the ave_grad array for reference
print('The average gradient array has shape: ', ave_grad.shape)

# Create an array that will count the number of histograms per cell
hist_counter = np.zeros((y_cells, x_cells, 1))

# Add up all the histograms for each cell and count the number of histograms per cell
for i in range (num_cells_per_block[0]):
    for j in range(num_cells_per_block[1]):
        ave_grad[i:tot_by + i,
                 j:tot_bx + j] += hog_descriptor_reshaped[:, :, i, j, :]

        hist_counter[i:tot_by + i,
                     j:tot_bx + j] += 1

# Calculate the average gradient for each cell
ave_grad /= hist_counter

# Calculate the total number of vectors we have in all the cells.
len_vecs = ave_grad.shape[0] * ave_grad.shape[1] * ave_grad.shape[2]

# Create an array that has num_bins equally spaced between 0 and 180 degress in radians.
deg = np.linspace(0, np.pi, num_bins, endpoint = False)

# Each cell will have a histogram with num_bins. For each cell, plot each bin as a vector (with its magnitude
# equal to the height of the bin in the histogram, and its angle corresponding to the bin in the histogram).
# To do this, create rank 1 arrays that will hold the (x,y)-coordinate of all the vectors in all the cells in the
# image. Also, create the rank 1 arrays that will hold all the (U,V)-components of all the vectors in all the
# cells in the image. Create the arrays that will hold all the vector positons and components.
U = np.zeros((len_vecs))
V = np.zeros((len_vecs))
X = np.zeros((len_vecs))
Y = np.zeros((len_vecs))

# Set the counter to zero
counter = 0

# Use the cosine and sine functions to calculate the vector components (U,V) from their maginitudes. Remember the
# cosine and sine functions take angles in radians. Calculate the vector positions and magnitudes from the
# average gradient array
for i in range(ave_grad.shape[0]):
    for j in range(ave_grad.shape[1]):
        for k in range(ave_grad.shape[2]):
            U[counter] = ave_grad[i,j,k] * np.cos(deg[k])
            V[counter] = ave_grad[i,j,k] * np.sin(deg[k])

            X[counter] = (cell_size[0] / 2) + (cell_size[0] * i)
            Y[counter] = (cell_size[1] / 2) + (cell_size[1] * j)

            counter = counter + 1

# Create the bins in degress to plot our histogram.
angle_axis = np.linspace(0, 180, num_bins, endpoint = False)
angle_axis += ((angle_axis[1] - angle_axis[0]) / 2)

# Create a figure with 4 subplots arranged in 2 x 2
fig, ((a,b),(c,d)) = plt.subplots(2,2)

# Set the title of each subplot
a.set(title = 'Gray Scale Image\n(Click to Zoom)')
b.set(title = 'HOG Descriptor\n(Click to Zoom)')
c.set(title = 'Zoom Window', xlim = (0, 18), ylim = (0, 18), autoscale_on = False)
d.set(title = 'Histogram of Gradients')

# Plot the gray scale image
a.imshow(gray_image, cmap = 'gray')
a.set_aspect(aspect = 1)

# Plot the feature vector (HOG Descriptor)
b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
b.invert_yaxis()
b.set_aspect(aspect = 1)
b.set_facecolor('black')

# Define function for interactive zoom
def onpress(event):

    #Unless the left mouse button is pressed do nothing
    if event.button != 1:
        return

    # Only accept clicks for subplots a and b
    if event.inaxes in [a, b]:

        # Get mouse click coordinates
        x, y = event.xdata, event.ydata

        # Select the cell closest to the mouse click coordinates
        cell_num_x = np.uint32(x / cell_size[0])
        cell_num_y = np.uint32(y / cell_size[1])

        # Set the edge coordinates of the rectangle patch
        edgex = x - (x % cell_size[0])
        edgey = y - (y % cell_size[1])

        # Create a rectangle patch that matches the the cell selected above
        rect = patches.Rectangle((edgex, edgey),
                                  cell_size[0], cell_size[1],
                                  linewidth = 1,
                                  edgecolor = 'magenta',
                                  facecolor='none')

        # A single patch can only be used in a single plot. Create copies
        # of the patch to use in the other subplots
        rect2 = copy.copy(rect)
        rect3 = copy.copy(rect)

        # Update all subplots
        a.clear()
        a.set(title = 'Gray Scale Image\n(Click to Zoom)')
        a.imshow(gray_image, cmap = 'gray')
        a.set_aspect(aspect = 1)
        a.add_patch(rect)

        b.clear()
        b.set(title = 'HOG Descriptor\n(Click to Zoom)')
        b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
        b.invert_yaxis()
        b.set_aspect(aspect = 1)
        b.set_facecolor('black')
        b.add_patch(rect2)

        c.clear()
        c.set(title = 'Zoom Window')
        c.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 1)
        c.set_xlim(edgex - cell_size[0], edgex + (2 * cell_size[0]))
        c.set_ylim(edgey - cell_size[1], edgey + (2 * cell_size[1]))
        c.invert_yaxis()
        c.set_aspect(aspect = 1)
        c.set_facecolor('black')
        c.add_patch(rect3)

        d.clear()
        d.set(title = 'Histogram of Gradients')
        d.grid()
        d.set_xlim(0, 180)
        d.set_xticks(angle_axis)
        d.set_xlabel('Angle')
        d.bar(angle_axis,
              ave_grad[cell_num_y, cell_num_x, :],
              180 // num_bins,
              align = 'center',
              alpha = 0.5,
              linewidth = 1.2,
              edgecolor = 'k')

        fig.canvas.draw()

# Create a connection between the figure and the mouse click
fig.canvas.mpl_connect('button_press_event', onpress)
plt.show()
```

# 了解直方图

下面，让我们看一下上图的几个快照，看看所选单元格的直方图是否有意义。首先，看一个在三角形内部而不是边缘附近的一个单元格：

Fig. 4. - Histograms Inside a Triangle.

在这个示例中，由于三角形几乎都是相同的颜色，因此在所选单元格中不应存在任何主导梯度。我们在缩放窗口和直方图中也可以清楚地看到，情况确实如此。梯度有很多，但没有一个在另一个上占明显的主导地位。

现在，让我们看一下靠近水平边缘的一个单元格：

Fig. 5. - Histograms Near a Horizontal Edge.

请牢记，边缘是图像中强度变化很突然的一个区域。在这些区域，我们将在某个特定方向上具有高强度梯度。这正是我们在所选单元格的相应直方图和缩放窗口中看到的内容。在缩放窗口中，我们可以看到主导梯度呈几乎90度的向上趋势，因为这是强度急剧变化的方向。因此，我们应该期望直方图中的90度区域比其他区域更强。实际上，这就是我们所看到的。

接下来，让我们看一下靠近垂直边缘的一个单元格：

Fig. 6. - Histograms Near a Vertical Edge.

在这个示例中，我们希望该单元格中的主导梯度是水平的，接近180度，因为这是强度急剧变化的方向。因此，我们应该期望直方图中的170度区域会强烈支配其他区域。实际上，这就是我们在直方图中看到的，但我们也看到该单元格中还有另一个显性梯度，即10度区间中的梯度。原因在于HOG算法使用无符号梯度，这意味着0度和180度被认为是相同的。因此，当创建直方图时，160度和180度之间的角度与10度区间和170度区间成比例。这就会导致在垂直边缘附近的单元格中，存在两个主导梯度，而不是只有一个。

最后，让我们看一下靠近对角线边缘的一个单元格。

Fig. 7. - Histograms Near a Diagonal Edge.

要理解我们所看到的东西，首先，我们要知道，梯度有一个x分量和一个y分量，就像矢量一样。因此，一个梯度的最终方向将由其分量的矢量和而定。出于这个原因，在垂直边缘上，梯度都是水平的，因为它们只有一个x分量，如图4所示。而在水平边缘上，梯度都是垂直的，因为它们只有一个y分量，如图3所示。因此，在对角线边缘，梯度将会是对角线，因为x和y分量都是非零。由于图像中的对角线边缘接近45度，我们应该期望在50度区间中看到一个主导的梯度方向。实际上，这就是我们在直方图中看到的，但是，如图4所示，我们看到有两个主导梯度而不是一个。其原因在于，创建直方图时，靠近区间边缘的角度成比例地给相邻区间提供贡献。例如，角度为40度的梯度位于30度和50度区间的中间。因此，梯度的大小被均匀地分成30度和50度的区间。这就会导致在对角线边缘附近的单元中，存在两个主导梯度，而不是只有一个。

现在你已经了解了HOG的实现方式，接下来，在工作区中找到名为*Examples*的notebook。你可以在那里为各种图像的HOG描述子设置自己的参数。祝你学得开心哦！