# IllinoisGRMHD: Basic equations and modules

## Authors: Leo Werneck & Zach Etienne

**This module is currently under development**

## This module introduces the basic equations solved by IllinoisGRMHD and provides an introduction to the modules inside the code.

## Introduction:

[`IllinoisGRMHD`](http://arxiv.org/abs/1501.07276) solves the equations of General Relativistic MagnetoHydroDynamics (GRMHD) using a high-resolution shock capturing scheme. It is a rewrite of the Illinois Numerical Relativity (ILNR) group's GRMHD code, and generates results that agree to roundoff error with that original code. Its feature set coincides with the features of the ILNR group's recent code (ca. 2009-2014), which was used in their modeling of the following systems:

1. Magnetized circumbinary disk accretion onto binary black holes
2. Magnetized black hole-neutron star mergers
3. Magnetized Bondi flow, Bondi-Hoyle-Littleton accretion
4. White dwarf-neutron star mergers

`IllinoisGRMHD` is particularly good at modeling GRMHD flows into black holes without the need for excision. Its [HARM-based conservative-to-primitive solver](https://arxiv.org/abs/astro-ph/0512420) has also been modified to check the physicality of conservative variables prior to primitive inversion, and move them into the physical range if they become unphysical.

Currently IllinoisGRMHD consists of

1. the Piecewise Parabolic Method (PPM) for reconstruction,
2. the Harten, Lax, van Leer (HLL/HLLE) approximate Riemann solver, and
3. a modified HARM Conservative-to-Primitive solver.

`IllinoisGRMHD` evolves the vector potential $A_{\mu}$ (on staggered grids) instead of the magnetic fields ($B^i$) directly, to guarantee that the magnetic fields will remain divergenceless even at AMR boundaries. On uniform resolution grids, this vector potential formulation produces results equivalent to those generated using the standard, staggered flux-CT scheme. This scheme is based on that of [Del Zanna *et al.* (2003)](https://arxiv.org/abs/astro-ph/0210618).

### Required and recommended citations:
* **(Required)** Etienne, Z. B., Paschalidis, V., Haas R., Mösta P., and Shapiro, S. L. IllinoisGRMHD: an open-source, user-friendly GRMHD code for dynamical spacetimes. Class. Quantum Grav. 32 (2015) 175009. ([arxiv:1501.07276](http://arxiv.org/abs/1501.07276)).
* **(Required)** Noble, S. C., Gammie, C. F., McKinney, J. C., Del Zanna, L. Primitive Variable Solvers for Conservative General Relativistic Magnetohydrodynamics. Astrophysical Journal, 641, 626 (2006) ([astro-ph/0512420](https://arxiv.org/abs/astro-ph/0512420)).
* **(Recommended)** Del Zanna, L., Bucciantini N., Londrillo, P. An efficient shock-capturing central-type scheme for multidimensional relativistic flows - II. Magnetohydrodynamics. A&A 400 (2) 397-413 (2003). DOI: 10.1051/0004-6361:20021641 ([astro-ph/0210618](https://arxiv.org/abs/astro-ph/0210618)).

# Table of Contents
$$\label{toc}$$

This module is organized as follows

1. [Step 1](#basic_equations): **The equations of GRMHD**
1. [Step 2](#suitable_equations): **Recasting the equations of GRMHD into a more useful form**
    1. [Step 2.1](#einstein_equations): Einstein's equations
    1. [Step 2.2](#conserv_of_baryon_number): Conservation of baryon number
    1. [Step 2.3](#conserv_of_em_tensor): Conservation of energy-momentum
    1. [Step 2.4](#em_evo_equations): The evolution equations for the electromagnetic field
    1. [Step 2.5](#equation_of_state): The equation of state (EOS)
1. [Step 3](#latex_pdf_output): **Output this notebook to $\LaTeX$-formatted PDF file**

# Step 1: The equations of GRMHD \[Back to [top](#toc)\]
$$\label{basic_equations}$$

The basic equations solved by `IllinoisGRMHD` are

1. Einstein's equations: $R^{\mu\nu} + \frac{1}{2}g^{\mu\nu}R = 8\pi T^{\mu\nu} = 8\pi \left(T^{\mu\nu}_{\rm matter} + T^{\mu\nu}_{\rm EM}\right)$ ;
1. Conservation of baryon number: $\nabla_{\mu}\left(\rho_{0} u^{\mu}\right) = 0$ ;
1. Conservation of energy-momentum: $\nabla_{\mu}T^{\mu\nu} = \nabla_{\mu}\left(T^{\mu\nu}_{\rm matter} + T^{\mu\nu}_{\rm EM}\right) = 0$ ;
1. Dual of homogeneous Maxwell's equations: $\nabla_{\nu} F^{*\mu\nu}=\frac{1}{\sqrt{-g}}\partial_{\nu}\left(\sqrt{-g}F^{*\mu\nu}\right)=0$ ,

where $g_{\mu\nu}$ is the ADM 4-metric, $R_{\mu\nu}$ and $R$ are the Ricci tensor and scalar, respectively, constructed from $g_{\mu\nu}$, $\nabla_{\mu}$ is the covariant derivative compatible with $g_{\mu\nu}$, $g\equiv\det\left(g_{\mu\nu}\right)$, $F^{\mu\nu}$ is the Faraday tensor and $F^{*\mu\nu}=\frac{1}{2}\epsilon^{\mu\nu\rho\sigma}F_{\mu\nu}$ its dual ($\epsilon^{\mu\nu\rho\sigma}$ is the Levi-Civita symbol).

The final equation that must be solved is the equation of state (EOS) for the matter. Currently `IllinoisGRMHD` implements a hybrid EOS of the form (**add citation**)

5. Equation of State (EOS): $P(\rho_{0},\epsilon) = P_{\rm cold}(\rho_{0}) + \left(\Gamma_{\rm th}-1\right) \rho_{0} \left[\epsilon-\epsilon_{\rm cold}(\rho_{0})\right]$ ,

where P is the pressure, $\epsilon$ the specific internal energy (the subscript cold indicate the cold components of these quantities), and $\Gamma_{\rm th}$ is a constant parameter which determines the conversion efficiency of kinetic to thermal energy at shocks.

# Step 2: Recasting the equations of GRMHD into a more useful form \[Back to [top](#toc)\]
$$\label{suitable_equations}$$

In this step we will write down the equations used by `IllinoisGRMHD` in the form they are implemented. To give an example, the GRMHD equations are written in the *conservative form*

$$
\partial_{t}\boldsymbol{C} + \boldsymbol{\nabla}\cdot\boldsymbol{F} = \boldsymbol{S}\ ,
$$

where $\boldsymbol{C} = \left\{\rho_{\star},\tilde\tau,\tilde{S}_{i},\tilde{B}^{i}\right\}$ is the vector of conservative variables, $\boldsymbol{F}$ is the flux vector, and $\boldsymbol{S}$ the vector of source terms (explict expressions for the components of $\boldsymbol{C}$, $\boldsymbol{F}$, and $\boldsymbol{S}$ can be found below).

## Step 2.1: Einstein's equations \[Back to [top](#toc)\]
$$\label{einstein_equations}$$

`IllinoisGRMHD` solves Einstein's field equations (with $G_{\rm N}=1=c$) in the presence of matter sources,

$$
G^{\mu\nu} = 8\pi T^{\mu\nu}\ ,
$$

where $G^{\mu\nu} \equiv R^{\mu\nu} + \frac{1}{2}g^{\mu\nu}R$ is the Einstein tensor and the total energy-momentum tensor, $T^{\mu\nu}$, is the sum of the matter and electromagnetic energy-momentum tensors:

$$
T^{\mu\nu} = T^{\mu\nu}_{\rm matter} + T^{\mu\nu}_{\rm EM}\ .
$$

Einstein's field equations are solved using the BSSN formalism (see [this tutorial module](../../Tutorial-BSSN_formulation.ipynb) for an overview).

## Step 2.2: Conservation of baryon number \[Back to [top](#toc)\]
$$\label{conserv_of_baryon_number}$$

The conservation of baryon number equation is written as (cf. eqs. (6), (7), and (18) in [Etienne *et al.*](https://arxiv.org/pdf/1501.07276.pdf))

$$
\boxed{\partial_{t}\rho_{\star} + \partial_{j}\left(\rho_{\star}v^{j}\right) = 0}\ ,
$$

where $\rho_{\star}\equiv \alpha \sqrt{\gamma}\rho_{0}u^{0}$ and $v^{i}\equiv u^{i}/u^{0}$.

## Step 2.3: Conservation of energy-momentum \[Back to [top](#toc)\]
$$\label{conserv_of_em_tensor}$$

In the ideal MHD limit, we can write down the total energy momentum tensor as (cf. eq. (8) in [Etienne *et al.*](https://arxiv.org/pdf/1501.07276.pdf) and the discussion before it)

$$
T^{\mu \nu} = (\rho_0 h +b^2) u^{\mu} u^{\nu} + \left( P + \frac{b^2}{2}\ .
\right) g^{\mu \nu} - b^{\mu} b^{\nu}
$$

The spatial components of the energy-momentum conservation equation give (cf. eq. (18) of https://arxiv.org/pdf/astro-ph/0503420.pdf and eqs. (35) and (36) of [Duez et al.](https://arxiv.org/pdf/astro-ph/0503420.pdf))

$$
\boxed{\partial_{t}\tilde{S}_{i} + \partial_{j}\left(\alpha\sqrt{\gamma} T^{j}_{\ i}\right) = \frac{1}{2}\sqrt{\gamma} T^{\alpha\beta}\partial_{i}g_{\alpha\beta}}\ ,
$$

where

$$
\tilde{S}_{i} = \sqrt{\gamma}S_{i} = \alpha \sqrt{\gamma} T^{0}_{\ i} = \left(\rho_{\star}h + \alpha u^{0}\sqrt{\gamma}b^{2}\right)u_{i} - \alpha \sqrt{\gamma} b^{0} b^{i}\ .
$$

The time component of the energy-momentum conservation equation gives (cf. eq. (18) of [Etienne *et al.*](https://arxiv.org/pdf/1501.07276.pdf))

$$
\boxed{\partial_{t}\tilde{\tau} + \partial_{i}\left(\alpha^{2}\sqrt{\gamma}T^{0i} - \rho_{\star}v^{i}\right) = s}\ ,
$$

where

\begin{align}
\tilde{\tau} &= \sqrt{\gamma}n_{\mu}n_{\nu} - \rho_{\star} = \alpha^{2}\sqrt{\gamma}T^{00} - \rho_{\star}\ ,\\
s &= -\alpha \sqrt{\gamma} T^{\mu\nu} \nabla_{\nu} n_{\mu} = \alpha\sqrt{\gamma}\left[\left(T^{00}\beta^{i}\beta^{j} + 2T^{0i}\beta^{j} + T^{ij}\right)K_{ij} - \left(T^{00}\beta^{i} + T^{0i}\right)\partial_{i}\alpha\right]\ ,
\end{align}

with $n_{\mu} = \left(\alpha,0,0,0\right)$ being the normal vector and $K_{ij}$ the extrinsic curvature.

## Step 2.4: The evolution equations for the electromagnetic field \[Back to [top](#toc)\]
$$\label{em_evo_equations}$$

From the spatial components of the dual of Maxwell's equations,

$$
\nabla_{\nu} F^{*\mu\nu}=\frac{1}{\sqrt{-g}}\partial_{\nu}\left(\sqrt{-g}F^{*\mu\nu}\right)=0\ ,
$$

we get the magnetic induction equation, which in conservative form may be written as (cf. eq. (12) [Etienne *et al.*](https://arxiv.org/pdf/1501.07276.pdf))

$$
\partial_{t}\tilde{B}^{i} + \partial_{j}\left(v^{j}\tilde{B}^{i} - v^{i}\tilde{B}^{j}\right) = 0\ ,
$$

where $\tilde{B}^{i} = \sqrt{\gamma}B^{i}$. We must also guarantee that no magnetic mopoles form, via the constraint

$$
\partial_{i}\tilde{B}^{i} = 0\ ,
$$

which is the time component of the dual of Maxwell's equations. Satisfying this constraint equation while evolving the magnetic field forward in time via the evolution equation above turns out to be a nontrivial endeavor, *particularly* on AMR grids. Instead, we choose to evolve the magnetic 4-vector potential $\mathcal{A}_{\mu}$ instead of the magnetic fields directly, so that

\begin{align}
\mathcal{A}_{\mu} &= \Phi n_{\mu} + A_{\mu}\ ,\\
\tilde{B}^{i} &= \tilde{\epsilon}^{ijk}\partial_{j}A_{k}\ ,
\end{align}

where $A_{\mu}$ is purely spatial ($A_{\mu}n^{\mu} = 0$) and $\Phi$ is the EM scalar potential.

Special finite difference operators for the vector potential are defined in IllinoisGRMHD so that the divergence of a curl is zero to roundoff error, which implies that the divergence of $\tilde{B}^{i}$ (as defined above) is zero and the condition $\partial_{i}\tilde{B}^{i}=0$ is satisfied automatically, *even on AMR grids*.

In terms of $A_{i}$, the induction equation becomes

$$
\boxed{\partial_t A_i = \tilde\epsilon_{ijk} v^j \tilde{B}^k - \partial_i (\alpha \Phi - \beta^j A_j)}\ .
$$

The final equation comes from choosing the covariant version of the "[generalized Lorenz gauge condition](https://arxiv.org/pdf/1207.3354.pdf)" that was introduced by the Illinois Relativity group,

$$
\nabla_{\mu}\mathcal{A}^{\mu} = \xi n_{\mu} \mathcal{A}^{\mu}\ ,
$$

where $\xi$ is a parameter with dimensions 1/Length, chosen so that the CFL condition remains satisfied. This gauge choice yields the additional evolution equation

$$
\boxed{\partial_t [\sqrt{\gamma} \Phi] + \partial_j (\alpha \sqrt{\gamma} A^j - \beta^j [\sqrt{\gamma}\Phi]) = -\xi\alpha \sqrt{\gamma}\Phi}\ .
$$

Note that in `IllinoisGRMHD` the evolved variable is $\sqrt{\gamma}\Phi$, not $\Phi$.

## Step 2.5: The equation of state (EOS) \[Back to [top](#toc)\]
$$\label{equation_of_state}$$

`IllinoisGRMHD` currently implements a hybrid EOS of the form

$$
\boxed{P(\rho_{0},\epsilon) = P_{\rm cold}(\rho_{0}) + \left(\Gamma_{\rm th}-1\right) \rho_{0} \left[\epsilon-\epsilon_{\rm cold}(\rho_{0})\right]}\ .
$$

The function $\epsilon_{\rm cold}$ is related to $P_{\rm cold}$ by the fist law of thermodynamics,

$$
\epsilon_{\rm cold}(\rho_{0}) = \int \frac{P_{\rm cold}(\rho_{0})}{\rho_{0}^{2}}d\rho_{0}\ .
$$

Currently, all functions within `IllinoisGRMHD` support piecewise-defined $P_{\rm cold}(\rho_{0})$ (the so-called "piecewise polytrope" EOS) with up to nine different polytropic indices, except for the conservatives-to-primitives solver, which currently supports only one.

In typical runs of the code, particularly on the ones presented in the [release paper](https://arxiv.org/pdf/1501.07276.pdf), the $\Gamma$-law EOS, $P = (\Gamma - 1)\rho_{0}\epsilon$, is adopted. This corresponds to setting $P_{\rm cold} = (\Gamma - 1)\rho_{0}\epsilon_{\rm cold}$ in the boxed equation above, which is equivalent to $P_{\rm cold} = \kappa\rho_{0}^{\Gamma}$ (with constant $\kappa$), and $\Gamma_{\rm th} = \Gamma$. In the absence of shocks, $\epsilon = \epsilon_{\rm cold}$ so that $P = P_{\rm cold}$.

# Step 3: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

```python
#!jupyter nbconvert --to latex --template ../../latex_nrpy_style.tplx Tutorial-IllinoisGRMHD__Overview.ipynb
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__Overview.tex
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__Overview.tex
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__Overview.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

```python

```


```python
import mmf_setup;mmf_setup.nbinit()
```

    <IPython.core.display.Javascript object>

This cell contains some definitions for equations and some CSS for styling
  the notebook. If things look a bit strange, please try the following:

Choose "Trust Notebook" from the "File" menu.

Re-execute this cell.

Reload the notebook.

# Overview

Here we demonstrate how gravitational wave data from a set of observations might constrain nuclear parameters.  These results were generated as follows:

1. A unified equation of state (EoS) was defined in terms of $N_p = 18$ parameters $\vect{p}$ as follows:
   * An outer crust is fixed by tabulated data. (No parameters.)
   * A compressible liquid drop model (CLDM) modeling spherical nuclei embedded in a gas of neutrons and electrons. (2 parameters)
   * A homogeneous interior with a homogeneous nuclear equation of state $\mathcal{E}(n_n, n_p)$ in conjunction with $\beta$-equilibrium. (13 parameters = 4 neutron matter parameters + 3 proton polaron parameters + 6 symmetric nuclear matter parameters)
   * A core characterized by the speed of sound (3 parameters).
2. The Tolman-Oppenheimer-Volkoff (TOV) equations were solved for non-rotating neutron stars, giving the neutron star masses $M(P_c, \vect{p})$, radii $R(P_c, \vect{p})$, second Love number $k_2(P_c, \vect{p})$ as functions of the central pressure $P_c$ and the 18 parameters $\vect{p}$.
3. Numerical derivatives of these functions were obtained using Richardson extrapolation.  These are tabulated for a set of $N_m$ masses spanning a range of neutron stars.  These derivatives are stored in dimensionless form:

   $$
     \frac{p_i}{M_j} \pdiff{M_j}{p_i} = p_j \pdiff \ln M_j.
   $$

4. Using a post-Newtonian analysis, these derivatives are used to obtain a set of Fisher information matrices $\mat{F}$ corresponding to the expected aLIGO signal sensitivity for all pairs of masses $m_1$ and $m_2$.  This is summarized in a $N_m\times N_m\times N_p\times N_p$ array where each entry $F[i, j,:,:]$ corresponds to the expected Fisher information matrix (FIM) obtained from the inspiral of a binary with masses $m_i$ and $m_j$ at a fiducial distance of $D_0=40$Mpc.  For an event at distance $D$, this must be weighted by a factor $(D_0/D)^2$.
5. From this set of FIM, a collective FIM can be obtained by summing over a sample population of binaries, weighed appropriately.
6. Nuclear uncertainties can be input directly with an information matrix:

   $$
     \mat{F}_{\text{nuclear}} = \mat{C}^{-1}
     \approx \diag\left(\frac{1}{\sigma_{0}^{2}}, \frac{1}{\sigma_{1}^2}, \cdots\right)
   $$

   where $\sigma_i$ are the estimated 1-$\sigma$ relative Gaussian errors for the $i$th parameter $p_i$.  If parameter covariances are known, they can be introduced directly into the covariance matrix $\mat{C}$.  *(All of this analysis assumes that errors are small enough and Gaussian so that linear error analysis is a reasonable approximation.  As above, the tabulated FIM is scaled by the parameter values to be dimensionless, so the errors $\sigma_i$ represent the relative errors in parameter.)*
7. From this combined FIM, we perform various analyses such as a principal component analysis, looking at the largest eigenvalues and eigenvectors of $\mat{F}$, or computing the parameter covariance matrix $\mat{C} = \mat{F}^{-1}$.

The summary of this analysis for a sample equation of state roughly matching th$ ALF4 equation of state is presented in the following notebook:

* [`ALF4.ipynb`](ALF4.ipynb)

*(As additional equations of state are added, they will be analyzed in appropriately named notebooks.)*

# Equation of State

## Outer Crust

The outer crust is simply tabulated from results like those of Negele and Vautherin.  There are no parameters introduced here.

## CLDM

Once the nuclear equation of state $\mathcal{E}(n_n, n_p)$ is specified (see below), we make a unified transition from the outer crust to homogeneous matter using a compressible liquid drop model with the following components:

* A spherical nucleus in a spherical Wigner-Seitz cell.  (Energy density $\mathcal{E}(n_n^(i), n_p^(i))$ where $n_{n,p}^{i}$ are the average densities of the protons and neutrons inside the drop.)
* A surrounding neutron gas.  (Described by $\mathcal{E}(n_n^{o}, 0)$ where $n_n^{o}$ is the density of the neutron gas.)
* A homogeneous electron cloud.  (Described as a free Fermi gas, relativistic if needed, maintaining beta-equilibrium by establishing zero total charge.)
* A surface term.

The only parameters that enter this approximation are those describing the surface of the drop.   For this we use the approach of [Lattimer:1985] (also in [Steiner:2012a]) which is characterized in terms of the proton fraction $x = n_p/n_b$ and the isospin asymmetry $\beta = (n_n - n_p)/(n_n + n_p)$:

$$
  \sigma = \sigma_0\mathcal{B}(x), \qquad
  \mathcal{B}(x) = \frac{16+b}{x^{-3} + b + \frac{1}{(1-x)^3}}, \qquad
  C_{\mathrm{sym}} = \frac{\sigma_{\delta}}{\sigma_0} = \frac{96}{b+16}.
$$

This introduces two parameters `sigma_delta`$=\sigma_\delta$ and `sigma_0`$=\sigma_0$.  Note that for small asymmetries $x\approx 1$, $\beta \approx 0$, we have:

$$
  \sigma = \sigma_0\left(
    1 -
    \frac{C_{\mathrm{sym}}}
         {C_{\mathrm{sym}} - \frac{12}{2 - \frac{1}{(1+\beta)^3} - \frac{1}{(1-\beta)^3}}}
  \right)
  \approx
  \sigma_0\left(
    1 - C_{\mathrm{sym}}\beta^2 + \order(\beta^4)
  \right)
$$

Thus, we see that there are two independent parameters in this expression.  We fix the parameter $\sigma_0$ so that the CLDM matches the baryon density $n_B$ and energy-density $\mathcal{E}(n_B)$ at the upper end of the tabulated set of outer-crust data, ensuring a smooth transition (at most, second-order).

Following [Steiner:2012a], we introduce one more parameter `C_C`$=\mathcal{C}$ which is a suppression factor in the Coulomb energy intended to model the diffuseness of the proton distribution.  Finally, one could in principle vary the dimension `d_C`$=d$ of the Wigner-Seitz cell to allow for a crude estimate of pasta.  We have tried this in some cases, but generally leave $d=3$.

Once these parameters are fixed, and $\sigma_0$ is found, we must find the transition from a droplet to homogeneous matter by finding the point where the chemical potentials and pressures are equal.  This ensures that the transition from the CLDM to homogeneous matter remains convex.  To summarize, we have the following parameters:

**CLDM**
* `C_C`: Coulomb suppression factor $\mathcal{C}$.
* `d_C`: Dimension $d$ of the cells.  (Generally held fixed at $d=3$.)
* `sigma_delta`: Surface tension isospin dependence $\sigma_\delta$ [MeV/fm$^2$].

[Lattimer:1985]: http://dx.doi.org/10.1016/0375-9474(85)90006-5 (J.M. Lattimer, C.J. Pethick, D.G. Ravenhall, and D.Q. Lamb, "Physical properties of hot, dense matter: The general case", Nucl. Phys. A 432(3), 646 - 742 (1985) )

[Steiner:2012a]: http://dx.doi.org/10.1103/PhysRevC.85.055804 (Andrew W. Steiner, "Deep crustal heating in a multicomponent accreted neutron star crust", Phys. Rev. C 85, 055804 (2012) )

## Homogeneous Matter

As motivated by our work with fitting a nuclear energy density functional (NEDF) called SeaLL1:

* [Bulgac, Forbes, Jin, Perez, and Schunck: A Minimal Nuclear Energy Density Functional (accepted for PRC)](https://arxiv.org/abs/1708.08771)

we acknowledge that the connection between neutron matter and symmetric nuclear matter might be quite weak.  We thus base our parameterization of homogeneous matter on an expansion in proton fraction starting from pure neutron matter.  For pure neutron matter, we use the following energy per particle $E_n$ [MeV] as a function of neutron density $n_n$ [1/fm$^3$]:

$$
   E_n(n_n) = \frac{\mathcal{E}_{n}(n_n)}{n_n}
   = m_nc^2 + a\left(\frac{n_n}{\bar{n}_0}\right)^{\alpha} + b\left(\frac{n_n}{\bar{n}_0}\right)^{\beta}
$$

where $m_n=939.565$MeV/$c^2$ is the neutron mass, $\bar{n}_0 = 0.16$/fm$^3$ is a constant (approximately the nucleon saturation density) and $a$, $b$, $\alpha$, and $\beta$ are four EoS parameters `a`, `alpha`, `b`, and `beta` in the code.

Adding a small proton fraction, we extend this using the following form:

\begin{align}
       \text{energy-per-particle}&& E_{np}(n_n, n_p) &= (1-x_p)E_n(n_n) + x_p\left(m_pc^2 + \Sigma^p(n_B)\right)
       + \frac{(2\pi^2)^{2/3}}{2m^*}x_p^{5/3}n_B^{2/3} +x_p^2f_2(n_B) + x_p^3f_3(n_B)+\cdots\\
       &&n_B &= n_n + n_p, \qquad x_p = \frac{n_p}{n_B},&&\\
       \text{proton self-energy}&&\Sigma^p(n_B) &= \mu_p\frac{n_B}{\bar{n}_0}\frac{2u_p-\frac{n_B}{n_0}}{2u_p - 1}&&
\end{align}

where $m_p = 938.272$MeV/$c^2$ is the bare proton mass, and $m^*$ is the effective proton mass.  The properties of the proton polaron are characterized by the parameters $\mu_p$ [MeV], the proton chemical potential at $n_B = \bar{n}_0$ and $u_p$ which specifies at which $n_B = u_p\bar{n}_0$ the proton self-energy $\Sigma^p(n_B)$ reaches its minimum value as shown below:

```python
%pylab inline --no-import-all
import constants as u
n_0 = 0.16/u.fm**3
mu_p = -104.5*u.MeV
u_p = 3.136
n_B = np.linspace(0, (u_p+1)*n_0)
Sigma_p = mu_p*n_B/n_0*(2*u_p - n_B/n_0)/(2*u_p-1)
plt.figure(figsize=(8,5))
plt.plot(n_B, Sigma_p)
plt.grid(True)
plt.xticks([0, n_0, u_p*n_0], ['0', r'$\bar{n}_0$', r'$u_p\bar{n}_0$'])
plt.yticks([0, mu_p], ['0', r'$\mu_p$'])
plt.xlabel(r'$n_B$')
plt.ylabel(r'$\Sigma^p(n_B)$')
plt.title("Demonstration of the parametrization of $\Sigma^p(n_B)$")
```

The expansion above depends on the functions $f_2(n_B)$, $f_3(n_B)$ etc. which we fix by matching the energy-per-particle of symmetric nuclear matter:

$$
  E_{np}(n_n, n_p) = \varepsilon_0 + \frac{K_0}{2}(\delta\rho)^2
  + \left((S_2 + L_2(\delta\rho) + \frac{K_2}{2}(\delta\rho)^2\right)\delta^2\\
  \delta = \frac{n_n-n_p}{n_B}, \qquad
  \delta\rho = \frac{n_B - n_0}{3n_0}.
$$

This introduces the following nuclear parameters: the saturation density $n_0$ [$1/$fm$^{3}$], the saturation energy-per-particle $\varepsilon_0$ [MeV], the symmetric incompressibility $K_0$ [MeV], and the following symmetry parameters at second order in the iso-spin asymmetry $\delta$: the symmetry energy $S_2$ [MeV], its slope $L_2$ [MeV] and its incompressibility $K_2$ [MeV].  Demanding that $E_{np}(n_n, n_p)$ match this expression order-by-order in the powers of $x_p$ up to order $x_p^5$ fixes the functions $f_2(n_B)$ through $f_5(n_B)$, completing the definition of the homogeneous equation of state.  To summarize, we have the following parameters:

**Nuclear Matter near Saturation**
* `n_0`: Saturation density $n_0$ [$1/$fm$^3$].
* `e_0`: Saturation energy density $\varepsilon_0$ [MeV].
* `K_0`: Symmetric incompressibility $K_0$ [MeV].
* `S_2`: Second-order symmetry energy $S_2$ [MeV].
* `L_2`: Second-order slope of the symmetry energy $L_2$ [MeV].
* `K_2`: Second-order incompressibility $K_2$ [MeV].

**Pure Neutron Matter**
* `a`: Coefficient $a$ [MeV] of the low-density polytrope.
* `alpha`: Power $\alpha$ of the low-density polytrope.
* `b`: Coefficient $b$ [MeV] of the high-density polytrope.
* `beta`: Power $\beta$ of the high-density polytrope.

**Proton Polaron**
* `mu_p0`: Proton chemical potential at the saturation density $\mu_p(\bar{n}_0)$ [MeV].
* `u_p`: Location $u_p\bar{n}_0$ of the minimum of the polaron dispersion.
* `m_eff_m_p`: Effective mass `m_p^*/m_p` of the proton in the polaron dispersion.

From these and the energy-density of a free lepton gas, we compute the equation of state by enforcing $\beta$-equilibrium through the condition of bulk charge neutrality and chemical equilibrium.

## Core

Finally, we require a parametrization of the inner core, which we characterize by the behaviour of the square of the speed-of-sound $C = c_s^2/c^2$.  We parametrize this as a polynomial starting at an energy density $\mathcal{E}_c$ up to a transition energy-density $\mathcal{E}_t$ as a function of the energy density $\mathcal{E} = n_BE_{np}(n_B)$.  Above $\mathcal{E}_t$ we simply adopt the asymptotic form $C_0 = 1/3$ predicted by quark matter.

In our code we specify that the polynomial form for $\mathcal{E}_c \leq \mathcal{E}\leq \mathcal{E}_t$ satisfy:

1. $C(\mathcal{E})$ is quadratic.
2. $C(\mathcal{E}_c) = C_c$ where $C_c$ is determined from the homogeneous equation of state.
3. $C(\mathcal{E}_\max) = C_\max$ is the maximum of the polynomial.  The two parameters $\mathcal{E}_\max$ and $C_\max$ are free.

These uniquely fix the equation of state of the core in terms of the following parameters:

**Core**
* `E_c`: Transition energy-density $\mathcal{E}_c$ [MeV/fm$^3$] at which we transition from the homogeneous equation of state to the core.
* `C_max`: Maximum value $C_\max$.
* `E_max`: Energy-density $\mathcal{E}_\max$ [MeV/fm$^3$] at which the polynomial obtains its maximum value.


```python
%matplotlib inline
```

분산 데이터 병렬 처리와 병렬 처리 파이프라인을 사용한 트랜스포머 모델 학습
=======================================================================

**Author**: `Pritam Damania <https://github.com/pritamdamania87>`_
  **번역**: `백선희 <https://github.com/spongebob03>`_

이 튜토리얼은 `분산 데이터 병렬처리(Distributed Data Parallel) <https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html>`__ 와
`병렬 처리 파이프라인 <https://pytorch.org/docs/stable/pipeline.html>`__
를 사용하여 여러 GPU에 걸친 거대한 트랜스포머(Transformer) 모델을 어떻게 학습시키는지 보여줍니다.
이번 튜토리얼은 `nn.Transformer 와 TorchText 로 시퀀스-투-시퀀스(Sequence-to-Sequence) 모델링하기 <https://tutorials.pytorch.kr/beginner/transformer_tutorial.html>`__ 의
확장판이며 분산 데이터 병렬 처리와 병렬 처리 파이프라인이 어떻게 트랜스포머 모델 학습에 쓰이는지 보여주기 위해 이전 튜토리얼에서의
모델 규모를 증가시켰습니다.

선수과목(Prerequisites):

    * `Pipeline Parallelism <https://pytorch.org/docs/stable/pipeline.html>`__
    * `nn.Transformer 와 TorchText 로 시퀀스-투-시퀀스(Sequence-to-Sequence) 모델링하기 <https://tutorials.pytorch.kr/beginner/transformer_tutorial.html>`__
    * `분산 데이터 병렬 처리 시작하기 <https://tutorials.pytorch.kr/intermediate/ddp_tutorial.html>`__

모델 정의하기
-------------

``PositionalEncoding`` 모듈은 시퀀스에서 토큰의 상대적, 절대적 위치에 대한
몇몇 정보를 주입합니다.
위치 인코딩은 임베딩과 같은 차원을 가지므로
둘을 합칠 수 있습니다. 여기서, 주파수가 다른 ``sine`` 과 ``cosine`` 함수를
사용합니다.

```python
import sys
import os
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
import tempfile
from torch.nn import TransformerEncoder, TransformerEncoderLayer

class PositionalEncoding(nn.Module):

    def __init__(self, d_model, dropout=0.1, max_len=5000):
        super(PositionalEncoding, self).__init__()
        self.dropout = nn.Dropout(p=dropout)

        pe = torch.zeros(max_len, d_model)
        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
        pe[:, 0::2] = torch.sin(position * div_term)
        pe[:, 1::2] = torch.cos(position * div_term)
        pe = pe.unsqueeze(0).transpose(0, 1)
        self.register_buffer('pe', pe)

    def forward(self, x):
        x = x + self.pe[:x.size(0), :]
        return self.dropout(x)
```

이번 튜토리얼에서는, 트랜스포머 모델을 두 개의 GPU에 걸쳐서 나누고
병렬 처리 파이프라인으로 학습시켜 보겠습니다. 추가로,
`분산 데이터 병렬 처리 <https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html>`__
를 사용하여 이 파이프라인의 두 복제를 훈련시킵니다. 한 프로세스는
GPUs 0, 1에 거쳐 파이프를 구동하고 다른 프로세스는 GPUs 2, 3에서 파이프를 구동합니다. 그 다음, 이 두
프로세스는 분산 데이터 병렬처리로 두 복제본(replica)을 학습시킵니다.
모델은 바로 `nn.Transformer 와 TorchText 로 시퀀스-투-시퀀스(Sequence-to-Sequence) 모델링하기
<https://tutorials.pytorch.kr/beginner/transformer_tutorial.html>`__ 튜토리얼과
똑같은 모델이지만 두 단계로 나뉩니다. 대부분 파라미터(parameter)들은
`nn.TransformerEncoder <https://pytorch.org/docs/stable/generated/torch.nn.TransformerEncoder.html>`__ 계층(layer)에 포함됩니다.
`nn.TransformerEncoder <https://pytorch.org/docs/stable/generated/torch.nn.TransformerEncoder.html>`__ 는
`nn.TransformerEncoderLayer <https://pytorch.org/docs/stable/generated/torch.nn.TransformerEncoderLayer.html>`__ 의 ``nlayers`` 로 구성되어 있습니다.
결과적으로, 이 튜토리얼에서는 ``nn.TransformerEncoder`` 에 중점을 두고 있으며
``nn.TransformerEncoderLayer`` 의 절반은 한 GPU에 두고
나머지 절반은 다른 GPU에 있도록 모델을 분할합니다. 이를 위해서 ``Encoder`` 와
``Decoder`` 섹션을 분리된 모듈로 빼낸 다음, 원본 트랜스포머 모듈을
나타내는 nn.Sequential을 빌드 합니다.

```python
if sys.platform == 'win32':
    print('Windows platform is not supported for pipeline parallelism')
    sys.exit(0)
if torch.cuda.device_count() < 4:
    print('Need at least four GPU devices for this tutorial')
    sys.exit(0)

class Encoder(nn.Module):
    def __init__(self, ntoken, ninp, dropout=0.5):
        super(Encoder, self).__init__()
        self.pos_encoder = PositionalEncoding(ninp, dropout)
        self.encoder = nn.Embedding(ntoken, ninp)
        self.ninp = ninp
        self.init_weights()

    def init_weights(self):
        initrange = 0.1
        self.encoder.weight.data.uniform_(-initrange, initrange)

    def forward(self, src):
        # Need (S, N) format for encoder.
        src = src.t()
        src = self.encoder(src) * math.sqrt(self.ninp)
        return self.pos_encoder(src)

class Decoder(nn.Module):
    def __init__(self, ntoken, ninp):
        super(Decoder, self).__init__()
        self.decoder = nn.Linear(ninp, ntoken)
        self.init_weights()

    def init_weights(self):
        initrange = 0.1
        self.decoder.bias.data.zero_()
        self.decoder.weight.data.uniform_(-initrange, initrange)

    def forward(self, inp):
        # Need batch dimension first for output of pipeline.
        return self.decoder(inp).permute(1, 0, 2)
```

학습을 위한 다중 프로세스 시작
------------------------------

각각 두 개의 GPU에서 자체 파이프라인을 구동하는 두 개의 프로세스를 시작합니다.
``run_worker`` 는 각 프로세스에 실행됩니다.

```python
def run_worker(rank, world_size):
```

데이터 로드하고 배치 만들기
---------------------------

학습 프로세스는 ``torchtext`` 의 Wikitext-2 데이터셋을 사용합니다.
torchtext 데이터셋에 접근하기 전에, https://github.com/pytorch/data 을 참고하여 torchdata를
설치하시기 바랍니다.
단어 오브젝트는 훈련 데이터셋으로 만들어지고, 토큰을 텐서(tensor)로 수치화하는데 사용됩니다.
시퀀스 데이터로부터 시작하여, ``batchify()`` 함수는 데이터셋을 열(column)들로 정리하고,
``batch_size`` 사이즈의 배치들로 나눈 후에 남은 모든 토큰을 버립니다.
예를 들어, 알파벳을 시퀀스(총 길이 26)로 생각하고 배치 사이즈를 4라고 한다면,
알파벳을 길이가 6인 4개의 시퀀스로 나눌 수 있습니다:

\begin{align}\begin{bmatrix}
  \text{A} & \text{B} & \text{C} & \ldots & \text{X} & \text{Y} & \text{Z}
  \end{bmatrix}
  \Rightarrow
  \begin{bmatrix}
  \begin{bmatrix}\text{A} \\ \text{B} \\ \text{C} \\ \text{D} \\ \text{E} \\ \text{F}\end{bmatrix} &
  \begin{bmatrix}\text{G} \\ \text{H} \\ \text{I} \\ \text{J} \\ \text{K} \\ \text{L}\end{bmatrix} &
  \begin{bmatrix}\text{M} \\ \text{N} \\ \text{O} \\ \text{P} \\ \text{Q} \\ \text{R}\end{bmatrix} &
  \begin{bmatrix}\text{S} \\ \text{T} \\ \text{U} \\ \text{V} \\ \text{W} \\ \text{X}\end{bmatrix}
  \end{bmatrix}\end{align}

이 열들은 모델에 의해서 독립적으로 취급되며, 이는
``G`` 와 ``F`` 의 의존성이 학습될 수 없다는 것을 의미하지만, 더 효율적인
배치 프로세싱(batch processing)을 허용합니다.

```python
# In 'run_worker'
    def print_with_rank(msg):
        print('[RANK {}]: {}'.format(rank, msg))

    from torchtext.datasets import WikiText2
    from torchtext.data.utils import get_tokenizer
    from torchtext.vocab import build_vocab_from_iterator

    train_iter = WikiText2(split='train')
    tokenizer = get_tokenizer('basic_english')
    vocab = build_vocab_from_iterator(map(tokenizer, train_iter), specials=["<unk>"])
    vocab.set_default_index(vocab["<unk>"])

    def data_process(raw_text_iter):
      data = [torch.tensor(vocab(tokenizer(item)), dtype=torch.long) for item in raw_text_iter]
      return torch.cat(tuple(filter(lambda t: t.numel() > 0, data)))

    train_iter, val_iter, test_iter = WikiText2()
    train_data = data_process(train_iter)
    val_data = data_process(val_iter)
    test_data = data_process(test_iter)

    device = torch.device(2 * rank)

    def batchify(data, bsz, rank, world_size, is_train=False):
        # Divide the dataset into bsz parts.
        nbatch = data.size(0) // bsz
        # Trim off any extra elements that wouldn't cleanly fit (remainders).
        data = data.narrow(0, 0, nbatch * bsz)
        # Evenly divide the data across the bsz batches.
        data = data.view(bsz, -1).t().contiguous()
        # Divide the data across the ranks only for training data.
        if is_train:
            data_per_rank = data.size(0) // world_size
            data = data[rank * data_per_rank : (rank + 1) * data_per_rank]
        return data.to(device)

    batch_size = 20
    eval_batch_size = 10
    train_data = batchify(train_data, batch_size, rank, world_size, True)
    val_data = batchify(val_data, eval_batch_size, rank, world_size)
    test_data = batchify(test_data, eval_batch_size, rank, world_size)
```

입력과 타겟 시퀀스를 생성하기 위한 함수들
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``get_batch()`` 함수는 트랜스포머 모델을 위한 입력과 타겟 시퀀스를
생성합니다. 이 함수는 소스 데이터를 ``bptt`` 길이를 가진 덩어리로 세분화합니다.
언어 모델링 과제를 위해서, 모델은
다음 단어인 ``Target`` 이 필요합니다. 예를 들어 ``bptt`` 의 값이 2라면,
``i`` = 0 일 때 다음의 2 개 변수(Variable)를 얻을 수 있습니다:

청크가 차원 0에 속하며
트랜스포머 모델의 ``S`` 차원과 일치한다는 것을 유의해야 합니다.
배치 차원 ``N`` 은 1 차원에 해당합니다.

```python
# In 'run_worker'
    bptt = 35
    def get_batch(source, i):
        seq_len = min(bptt, len(source) - 1 - i)
        data = source[i:i+seq_len]
        target = source[i+1:i+1+seq_len].view(-1)
        # Need batch dimension first for pipeline parallelism.
        return data.t(), target
```

모델 규모와 파이프 초기화
-------------------------

병렬 처리 파이프라인을 활용한 대형 트랜스포머 모델 학습을 보이기 위해,
트랜스포머 계층 규모를 적절히 확장시킵니다.
4096차원의 임베딩 벡터, 4096의 은닉 사이즈, 16개의 어텐션 헤드(attention head)와 총 8 개의
트랜스포머 계층 (``nn.TransformerEncoderLayer``)를 사용합니다. 이는 최대
**~1 억** 개의 파라미터를 갖는 모델을 생성합니다.

`RPC 프레임워크 <https://pytorch.org/docs/stable/rpc.html>`__ 를 초기화해야 합니다.
Pipe가  향후 호스트 파이프라인을 교차 확장할 수 있도록 하는 `RRef <https://pytorch.org/docs/stable/rpc.html#rref>`__ 를 통해
RPC 프레임워크에 의존하기 때문입니다.
이때 RPC 프레임워크는 오직 하나의 하나의 worker로 초기화를 해야 하는데,
여러 GPU를 다루기 위해 프로세스 하나만 사용하고 있기 때문입니다.

그런 다음 파이프라인은 한 GPU에 8개의 트랜스포머와
다른 GPU에 8개의 트랜스포머 계층으로 초기화됩니다. 한 파이프는 GPU 0, 1에 거쳐 설정되고
다른 하나는 GPU 2, 3에 설정됩니다. 그런 다음 DistributedDataParallel을 사용하여 두 파이프가 모두 복제됩니다.

```python
# In 'run_worker'
    ntokens = len(vocab) # the size of vocabulary
    emsize = 4096 # embedding dimension
    nhid = 4096 # the dimension of the feedforward network model in nn.TransformerEncoder
    nlayers = 8 # the number of nn.TransformerEncoderLayer in nn.TransformerEncoder
    nhead = 16 # the number of heads in the multiheadattention models
    dropout = 0.2 # the dropout value

    from torch.distributed import rpc
    tmpfile = tempfile.NamedTemporaryFile()
    rpc.init_rpc(
        name="worker",
        rank=0,
        world_size=1,
        rpc_backend_options=rpc.TensorPipeRpcBackendOptions(
            init_method="file://{}".format(tmpfile.name),
            # Specifying _transports and _channels is a workaround and we no longer
            # will have to specify _transports and _channels for PyTorch
            # versions >= 1.8.1
            _transports=["ibv", "uv"],
            _channels=["cuda_ipc", "cuda_basic"],
        )
    )

    # Num gpus for model parallelism.
    num_gpus = 2
    partition_len = ((nlayers - 1) // num_gpus) + 1

    # Add encoder in the beginning.
    tmp_list = [Encoder(ntokens, emsize, dropout).cuda(2 * rank)]
    module_list = []

    # Add all the necessary transformer blocks.
    for i in range(nlayers):
        transformer_block = TransformerEncoderLayer(emsize, nhead, nhid, dropout)
        if i != 0 and i % (partition_len) == 0:
            module_list.append(nn.Sequential(*tmp_list))
            tmp_list = []
        device = i // (partition_len)
        tmp_list.append(transformer_block.to(2 * rank + device))

    # Add decoder in the end.
    tmp_list.append(Decoder(ntokens, emsize).cuda(2 * rank + num_gpus - 1))
    module_list.append(nn.Sequential(*tmp_list))

    # Need to use 'checkpoint=never' since as of PyTorch 1.8, Pipe checkpointing
    # doesn't work with DDP.
    from torch.distributed.pipeline.sync import Pipe
    chunks = 8
    model = Pipe(torch.nn.Sequential(
        *module_list), chunks = chunks, checkpoint="never")

    # Initialize process group and wrap model in DDP.
    from torch.nn.parallel import DistributedDataParallel
    import torch.distributed as dist
    os.environ['MASTER_ADDR'] = 'localhost'
    os.environ['MASTER_PORT'] = '29500'
    dist.init_process_group(
                backend="nccl", rank=rank, world_size=world_size)
    model = DistributedDataParallel(model)

    def get_total_params(module: torch.nn.Module):
        total_params = 0
        for param in module.parameters():
            total_params += param.numel()
        return total_params

    print_with_rank('Total parameters in model: {:,}'.format(get_total_params(model)))
```

모델 실행하기
-------------

손실(loss)을 추적하기 위해 `CrossEntropyLoss <https://pytorch.org/docs/master/nn.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss>`__ 가
적용되며, 옵티마이저(optimizer)로서 `SGD <https://pytorch.org/docs/master/optim.html?highlight=sgd#torch.optim.SGD>`__
는 확률적 경사하강법(stochastic gradient descent method)을 구현합니다. 초기
학습률(learning rate)은 5.0로 설정됩니다. `StepLR <https://pytorch.org/docs/master/optim.html?highlight=steplr#torch.optim.lr_scheduler.StepLR>`__ 는
에폭(epoch)에 따라서 학습률을 조절하는 데 사용됩니다. 학습하는 동안,
기울기 폭발(gradient exploding)을 방지하기 위해 모든 기울기를 함께 조정(scale)하는 함수
`nn.utils.clip_grad_norm\_ <https://pytorch.org/docs/master/nn.html?highlight=nn%20utils%20clip_grad_norm#torch.nn.utils.clip_grad_norm_>`__
을 이용합니다.

```python
# In 'run_worker'
    criterion = nn.CrossEntropyLoss()
    lr = 5.0 # learning rate
    optimizer = torch.optim.SGD(model.parameters(), lr=lr)
    scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma=0.95)

    import time
    def train():
        model.train() # Turn on the train mode
        total_loss = 0.
        start_time = time.time()
        ntokens = len(vocab)

        # Train only for 50 batches to keep script execution time low.
        nbatches = min(50 * bptt, train_data.size(0) - 1)

        for batch, i in enumerate(range(0, nbatches, bptt)):
            data, targets = get_batch(train_data, i)
            optimizer.zero_grad()
            # Since the Pipe is only within a single host and process the ``RRef``
            # returned by forward method is local to this node and can simply
            # retrieved via ``RRef.local_value()``.
            output = model(data).local_value()
            # Need to move targets to the device where the output of the
            # pipeline resides.
            loss = criterion(output.view(-1, ntokens), targets.cuda(2 * rank + 1))
            loss.backward()
            torch.nn.utils.clip_grad_norm_(model.parameters(), 0.5)
            optimizer.step()

            total_loss += loss.item()
            log_interval = 10
            if batch % log_interval == 0 and batch > 0:
                cur_loss = total_loss / log_interval
                elapsed = time.time() - start_time
                print_with_rank('| epoch {:3d} | {:5d}/{:5d} batches | '
                      'lr {:02.2f} | ms/batch {:5.2f} | '
                      'loss {:5.2f} | ppl {:8.2f}'.format(
                        epoch, batch, nbatches // bptt, scheduler.get_last_lr()[0],
                        elapsed * 1000 / log_interval,
                        cur_loss, math.exp(cur_loss)))
                total_loss = 0
                start_time = time.time()

    def evaluate(eval_model, data_source):
        eval_model.eval() # Turn on the evaluation mode
        total_loss = 0.
        ntokens = len(vocab)
        # Evaluate only for 50 batches to keep script execution time low.
        nbatches = min(50 * bptt, data_source.size(0) - 1)
        with torch.no_grad():
            for i in range(0, nbatches, bptt):
                data, targets = get_batch(data_source, i)
                output = eval_model(data).local_value()
                output_flat = output.view(-1, ntokens)
                # Need to move targets to the device where the output of the
                # pipeline resides.
                total_loss += len(data) * criterion(output_flat, targets.cuda(2 * rank + 1)).item()
        return total_loss / (len(data_source) - 1)
```

에폭을 반복합니다. 만약 검증 오차(validation loss)가 지금까지 관찰한 것 중 최적이라면
모델을 저장합니다. 각 에폭 이후에 학습률을 조절합니다.

```python
# In 'run_worker'
    best_val_loss = float("inf")
    epochs = 3 # The number of epochs
    best_model = None

    for epoch in range(1, epochs + 1):
        epoch_start_time = time.time()
        train()
        val_loss = evaluate(model, val_data)
        print_with_rank('-' * 89)
        print_with_rank('| end of epoch {:3d} | time: {:5.2f}s | valid loss {:5.2f} | '
              'valid ppl {:8.2f}'.format(epoch, (time.time() - epoch_start_time),
                                         val_loss, math.exp(val_loss)))
        print_with_rank('-' * 89)

        if val_loss < best_val_loss:
            best_val_loss = val_loss
            best_model = model

        scheduler.step()
```

평가 데이터셋으로 모델 평가하기
-------------------------------

평가 데이터셋에서의 결과를 확인하기 위해 최고의 모델을 적용합니다.

```python
# In 'run_worker'
    test_loss = evaluate(best_model, test_data)
    print_with_rank('=' * 89)
    print_with_rank('| End of training | test loss {:5.2f} | test ppl {:8.2f}'.format(
        test_loss, math.exp(test_loss)))
    print_with_rank('=' * 89)

# Main execution
import torch.multiprocessing as mp

if __name__=="__main__":
    world_size = 2
    mp.spawn(run_worker, args=(world_size, ), nprocs=world_size, join=True)
```

Output
------

.. code-block:: py

    [RANK 0]: | epoch   1 |    10/   50 batches | lr 5.00 | ms/batch 778.97 | loss 43.31 | ppl 6432469059895903232.00
    [RANK 1]: | epoch   1 |    10/   50 batches | lr 5.00 | ms/batch 778.90 | loss 44.50 | ppl 21245447128217366528.00
    [RANK 0]: | epoch   1 |    20/   50 batches | lr 5.00 | ms/batch 699.89 | loss 44.50 | ppl 21176949187407757312.00
    [RANK 1]: | epoch   1 |    20/   50 batches | lr 5.00 | ms/batch 699.87 | loss 44.62 | ppl 23975861229620961280.00
    [RANK 0]: | epoch   1 |    30/   50 batches | lr 5.00 | ms/batch 698.86 | loss 41.62 | ppl 1193312915629888256.00
    [RANK 1]: | epoch   1 |    30/   50 batches | lr 5.00 | ms/batch 698.87 | loss 40.69 | ppl 471605759847546240.00
    [RANK 0]: | epoch   1 |    40/   50 batches | lr 5.00 | ms/batch 698.34 | loss 45.20 | ppl 42812308420836458496.00
    [RANK 1]: | epoch   1 |    40/   50 batches | lr 5.00 | ms/batch 698.33 | loss 45.68 | ppl 68839569686012223488.00
    [RANK 1]: -----------------------------------------------------------------------------------------
    [RANK 1]: | end of epoch   1 | time: 40.08s | valid loss  0.80 | valid ppl     2.22
    [RANK 1]: -----------------------------------------------------------------------------------------
    [RANK 0]: -----------------------------------------------------------------------------------------
    [RANK 0]: | end of epoch   1 | time: 40.09s | valid loss  0.80 | valid ppl     2.22
    [RANK 0]: -----------------------------------------------------------------------------------------
    [RANK 0]: | epoch   2 |    10/   50 batches | lr 4.75 | ms/batch 768.51 | loss 36.34 | ppl 6063529544668166.00
    [RANK 1]: | epoch   2 |    10/   50 batches | lr 4.75 | ms/batch 769.23 | loss 37.41 | ppl 17651211266236086.00
    [RANK 0]: | epoch   2 |    20/   50 batches | lr 4.75 | ms/batch 699.57 | loss 28.97 | ppl 3798441739584.11
    [RANK 1]: | epoch   2 |    20/   50 batches | lr 4.75 | ms/batch 699.56 | loss 29.28 | ppl 5203636967575.47
    [RANK 0]: | epoch   2 |    30/   50 batches | lr 4.75 | ms/batch 699.04 | loss 28.43 | ppl 2212498693571.25
    [RANK 1]: | epoch   2 |    30/   50 batches | lr 4.75 | ms/batch 699.05 | loss 28.33 | ppl 2015144761281.48
    [RANK 0]: | epoch   2 |    40/   50 batches | lr 4.75 | ms/batch 699.10 | loss 23.30 | ppl 13121380184.92
    [RANK 1]: | epoch   2 |    40/   50 batches | lr 4.75 | ms/batch 699.09 | loss 23.41 | ppl 14653799192.87
    [RANK 0]: -----------------------------------------------------------------------------------------
    [RANK 0]: | end of epoch   2 | time: 39.97s | valid loss  0.24 | valid ppl     1.27
    [RANK 0]: -----------------------------------------------------------------------------------------
    [RANK 1]: -----------------------------------------------------------------------------------------
    [RANK 1]: | end of epoch   2 | time: 39.98s | valid loss  0.24 | valid ppl     1.27
    [RANK 1]: -----------------------------------------------------------------------------------------
    [RANK 0]: | epoch   3 |    10/   50 batches | lr 4.51 | ms/batch 769.36 | loss 12.80 | ppl 361681.11
    [RANK 1]: | epoch   3 |    10/   50 batches | lr 4.51 | ms/batch 768.97 | loss 12.57 | ppl 287876.61
    [RANK 0]: | epoch   3 |    20/   50 batches | lr 4.51 | ms/batch 698.27 | loss 12.01 | ppl 164364.60
    [RANK 1]: | epoch   3 |    20/   50 batches | lr 4.51 | ms/batch 698.30 | loss 11.98 | ppl 159095.89
    [RANK 0]: | epoch   3 |    30/   50 batches | lr 4.51 | ms/batch 697.75 | loss 10.90 | ppl 54261.91
    [RANK 1]: | epoch   3 |    30/   50 batches | lr 4.51 | ms/batch 697.72 | loss 10.89 | ppl 53372.39
    [RANK 0]: | epoch   3 |    40/   50 batches | lr 4.51 | ms/batch 699.49 | loss 10.78 | ppl 47948.35
    [RANK 1]: | epoch   3 |    40/   50 batches | lr 4.51 | ms/batch 699.50 | loss 10.79 | ppl 48664.42
    [RANK 0]: -----------------------------------------------------------------------------------------
    [RANK 0]: | end of epoch   3 | time: 39.96s | valid loss  0.38 | valid ppl     1.46
    [RANK 0]: -----------------------------------------------------------------------------------------
    [RANK 1]: -----------------------------------------------------------------------------------------
    [RANK 1]: | end of epoch   3 | time: 39.96s | valid loss  0.38 | valid ppl     1.46
    [RANK 1]: -----------------------------------------------------------------------------------------
    [RANK 0]: =========================================================================================
    [RANK 0]: | End of training | test loss  0.33 | test ppl     1.39
    [RANK 0]: =========================================================================================
    [RANK 1]: =========================================================================================
    [RANK 1]: | End of training | test loss  0.33 | test ppl     1.39
    [RANK 1]: =========================================================================================


```python
%matplotlib inline
```

nn.Transformer 와 TorchText 로 시퀀스-투-시퀀스(Sequence-to-Sequence) 모델링하기
==========================================================================

이 튜토리얼에서는
`nn.Transformer <https://pytorch.org/docs/master/nn.html?highlight=nn%20transformer#torch.nn.Transformer>`__ 모듈을
이용하는 시퀀스-투-시퀀스(Sequence-to-Sequence) 모델을 학습하는 방법을 배워보겠습니다.

PyTorch 1.2 버젼에는
`Attention is All You Need <https://arxiv.org/pdf/1706.03762.pdf>`__ 논문에
기반한 표준 트랜스포머(transformer) 모듈을 포함하고 있습니다.
트랜스포머 모델은 더 높은 수준의 병렬화(parallelization)가 가능하면서도
많은 시퀀스-투-시퀀스 문제들에서 품질이 우수함이 입증되었습니다.
``nn.Transformer`` 모듈은 입력(input) 과 출력(output) 사이의 전역적인 의존성(global dependencies)
을 나타내기 위하여 전적으로 어텐션(attention) 메커니즘에 의존합니다.
(최근에 또 다른 모듈이 `nn.MultiheadAttention <https://pytorch.org/docs/master/nn.html?highlight=multiheadattention#torch.nn.MultiheadAttention>`__ 으로 구현되었습니다.)
``nn.Transformer`` 모듈은 현재 모듈화가 매우 잘 되어 있어,
다음과 같은 단일 컴포넌트 (이 튜토리얼의 `nn.TransformerEncoder <https://pytorch.org/docs/master/nn.html?highlight=nn%20transformerencoder#torch.nn.TransformerEncoder>`__ 와 같은)
는 쉽게 적용 및 구성될 수 있습니다.

모델 정의하기
----------------

이 튜토리얼에서, 우리는 ``nn.TransformerEncoder`` 모델을 언어 모델링(language modeling) 과제에 대해서 학습시킬 것입니다.
언어 모델링 과제는 주어진 단어 (또는 단어의 시퀀스) 가 다음에 이어지는 단어 시퀀스를 따를 가능성(likelihood)에 대한 확률을 할당하는 것입니다.
먼저, 토큰(token) 들의 시퀀스가 임베딩(embedding) 레이어로 전달되며, 이어서 포지셔널 인코딩(positional encoding) 레이어가 각 단어의 순서를 설명합니다.
(더 자세한 설명은 다음 단락을 참고해주세요.)
``nn.TransformerEncoder`` 는 여러 개의
`nn.TransformerEncoderLayer <https://pytorch.org/docs/master/nn.html?highlight=transformerencoderlayer#torch.nn.TransformerEncoderLayer>`__
레이어로 구성되어 있습니다.
``nn.TransformerEncoder`` 내부의 셀프-어텐션(self-attention) 레이어들은 시퀀스 안에서의 이전 포지션에만 집중하도록 허용되기 때문에,
입력(input) 순서와 함께, 정사각 형태의 어텐션 마스크(attention mask) 가 필요합니다.
언어 모델링 과제를 위해서, 미래의 포지션에 있는 모든 토큰들은 마스킹 되어야(가려져야) 합니다.
실제 단어를 얻기 위해서, ``nn.TransformerEncoder`` 의 출력은 로그-소프트맥스(log-Softmax) 로 이어지는 최종 선형(Linear) 레이어로 전달 됩니다.

```python
import math
import torch
import torch.nn as nn
import torch.nn.functional as F

class TransformerModel(nn.Module):

    def __init__(self, ntoken, ninp, nhead, nhid, nlayers, dropout=0.5):
        super(TransformerModel, self).__init__()
        from torch.nn import TransformerEncoder, TransformerEncoderLayer
        self.model_type = 'Transformer'
        self.src_mask = None
        self.pos_encoder = PositionalEncoding(ninp, dropout)
        encoder_layers = TransformerEncoderLayer(ninp, nhead, nhid, dropout)
        self.transformer_encoder = TransformerEncoder(encoder_layers, nlayers)
        self.encoder = nn.Embedding(ntoken, ninp)
        self.ninp = ninp
        self.decoder = nn.Linear(ninp, ntoken)

        self.init_weights()

    def _generate_square_subsequent_mask(self, sz):
        mask = (torch.triu(torch.ones(sz, sz)) == 1).transpose(0, 1)
        mask = mask.float().masked_fill(mask == 0, float('-inf')).masked_fill(mask == 1, float(0.0))
        return mask

    def init_weights(self):
        initrange = 0.1
        self.encoder.weight.data.uniform_(-initrange, initrange)
        self.decoder.bias.data.zero_()
        self.decoder.weight.data.uniform_(-initrange, initrange)

    def forward(self, src):
        if self.src_mask is None or self.src_mask.size(0) != len(src):
            device = src.device
            mask = self._generate_square_subsequent_mask(len(src)).to(device)
            self.src_mask = mask

        src = self.encoder(src) * math.sqrt(self.ninp)
        src = self.pos_encoder(src)
        output = self.transformer_encoder(src, self.src_mask)
        output = self.decoder(output)
        return output
```

``PositionalEncoding`` 모듈은 시퀀스 안에서 토큰의 상대적인 또는 절대적인 포지션에 대한 어떤 정보를 주입합니다.
포지셔널 인코딩은 임베딩과 합칠 수 있도록 똑같은 차원을 가집니다.
여기에서, 우리는 다른 주파수(frequency) 의 ``sine`` 과 ``cosine`` 함수를 사용합니다.

```python
class PositionalEncoding(nn.Module):

    def __init__(self, d_model, dropout=0.1, max_len=5000):
        super(PositionalEncoding, self).__init__()
        self.dropout = nn.Dropout(p=dropout)

        pe = torch.zeros(max_len, d_model)
        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
        pe[:, 0::2] = torch.sin(position * div_term)
        pe[:, 1::2] = torch.cos(position * div_term)
        pe = pe.unsqueeze(0).transpose(0, 1)
        self.register_buffer('pe', pe)

    def forward(self, x):
        x = x + self.pe[:x.size(0), :]
        return self.dropout(x)
```

데이터 로드하고 배치 만들기
-------------------

학습 과정에서는 ``torchtext`` 의 Wikitext-2 데이터셋을 이용합니다.
단어 오브젝트는 훈련 데이터셋(train dataset) 에 의하여 만들어지고, 토큰을 텐서(tensor)로 수치화하는데 사용됩니다.
시퀀스 데이터로부터 시작하여, ``batchify()`` 함수는 데이터셋을 컬럼들로 배열하고, ``batch_size`` 사이즈의 배치들로 나눈 후에 남은 모든 토큰을 버립니다.
예를 들어, 알파벳을 시퀀스(총 길이 26) 로 생각하고 배치 사이즈를 4라고 한다면, 우리는 알파벳을 길이가 6인 4개의 시퀀스로 나눌 수 있습니다.

\begin{align}\begin{bmatrix}
  \text{A} & \text{B} & \text{C} & \ldots & \text{X} & \text{Y} & \text{Z}
  \end{bmatrix}
  \Rightarrow
  \begin{bmatrix}
  \begin{bmatrix}\text{A} \\ \text{B} \\ \text{C} \\ \text{D} \\ \text{E} \\ \text{F}\end{bmatrix} &
  \begin{bmatrix}\text{G} \\ \text{H} \\ \text{I} \\ \text{J} \\ \text{K} \\ \text{L}\end{bmatrix} &
  \begin{bmatrix}\text{M} \\ \text{N} \\ \text{O} \\ \text{P} \\ \text{Q} \\ \text{R}\end{bmatrix} &
  \begin{bmatrix}\text{S} \\ \text{T} \\ \text{U} \\ \text{V} \\ \text{W} \\ \text{X}\end{bmatrix}
  \end{bmatrix}\end{align}

이 컬럼들은 모델에 의해서 독립적으로 취급되며, 이것은 더 효율적인 배치 프로세싱(batch processing) 이 가능하지만, ``G`` 와 ``F`` 의 의존성이 학습될 수 없다는 것을 의미합니다.

```python
import torchtext
from torchtext.data.utils import get_tokenizer
TEXT = torchtext.data.Field(tokenize=get_tokenizer("basic_english"),
                            init_token='<sos>',
                            eos_token='<eos>',
                            lower=True)
train_txt, val_txt, test_txt = torchtext.datasets.WikiText2.splits(TEXT)
TEXT.build_vocab(train_txt)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

def batchify(data, bsz):
    data = TEXT.numericalize([data.examples[0].text])
    # 데이터셋을 bsz 파트들로 나눕니다.
    nbatch = data.size(0) // bsz
    # 깔끔하게 나누어 떨어지지 않는 추가적인 부분(나머지들) 은 잘라냅니다.
    data = data.narrow(0, 0, nbatch * bsz)
    # 데이터에 대하여 bsz 배치들로 동등하게 나눕니다.
    data = data.view(bsz, -1).t().contiguous()
    return data.to(device)

batch_size = 20
eval_batch_size = 10
train_data = batchify(train_txt, batch_size)
val_data = batchify(val_txt, eval_batch_size)
test_data = batchify(test_txt, eval_batch_size)
```

입력(input) 과 타겟(target) 시퀀스를 생성하기 위한 함수들
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``get_batch()`` 함수는 트랜스포머 모델을 위한 입력과 타겟 시퀀스를 생성합니다.
이 함수는 소스 데이터를 ``bptt`` 길이를 가진 덩어리로 세분화 합니다.
언어 모델링 과제를 위해서, 모델은 다음 단어인 ``Target`` 이 필요 합니다.
예를 들어, ``bptt`` 의 값이 2 라면, 우리는 ``i`` = 0 일 때 다음의 2 개의 변수(Variable) 를 얻을 수 있습니다:

변수 덩어리는 트랜스포머 모델의 ``S`` 차원과 일치하는 0 차원에 해당합니다.
배치 차원 ``N`` 은 1 차원에 해당합니다.

```python
bptt = 35
def get_batch(source, i):
    seq_len = min(bptt, len(source) - 1 - i)
    data = source[i:i+seq_len]
    target = source[i+1:i+1+seq_len].view(-1)
    return data, target
```

인스턴스(instance) 초기화하기
--------------------

모델은 아래와 같은 하이퍼파라미터(hyperparameter) 로 세팅 됩니다.
단어 사이즈는 단어 오브젝트의 길이와 일치 합니다.

```python
ntokens = len(TEXT.vocab.stoi) # 단어 사전의 크기
emsize = 200 # 임베딩 차원
nhid = 200 # nn.TransformerEncoder 에서 피드포워드 네트워크(feedforward network) 모델의 차원
nlayers = 2 # nn.TransformerEncoder 내부의 nn.TransformerEncoderLayer 개수
nhead = 2 # 멀티헤드 어텐션(multi-head attention) 모델의 헤드 개수
dropout = 0.2 # 드랍아웃(dropout) 값
model = TransformerModel(ntokens, emsize, nhead, nhid, nlayers, dropout).to(device)
```

모델 실행하기
-------------

손실(loss) 을 추적하는 데에는
`CrossEntropyLoss <https://pytorch.org/docs/master/nn.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss>`__
가 적용되며, 옵티마이저(optimizer) 로서
`SGD <https://pytorch.org/docs/master/optim.html?highlight=sgd#torch.optim.SGD>`__
는 확률적 경사 하강법(stochastic gradient descent method) 을 구현합니다.
초기 학습률(learning rate) 은 5.0 으로 설정됩니다.
`StepLR <https://pytorch.org/docs/master/optim.html?highlight=steplr#torch.optim.lr_scheduler.StepLR>`__
은 에포크(epoch) 에 따라서 학습률을 조절하는데 사용됩니다.
학습하는 동안에, 우리는 기울기 폭발(gradient exploding) 을 방지하기 위하여 모든 기울기를 함께 스케일(scale) 하는 함수인
`nn.utils.clip_grad_norm\_ <https://pytorch.org/docs/master/nn.html?highlight=nn%20utils%20clip_grad_norm#torch.nn.utils.clip_grad_norm_>`__
을 이용합니다.

```python
criterion = nn.CrossEntropyLoss()
lr = 5.0 # 학습률
optimizer = torch.optim.SGD(model.parameters(), lr=lr)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma=0.95)

import time
def train():
    model.train() # 학습 모드를 시작합니다.
    total_loss = 0.
    start_time = time.time()
    ntokens = len(TEXT.vocab.stoi)
    for batch, i in enumerate(range(0, train_data.size(0) - 1, bptt)):
        data, targets = get_batch(train_data, i)
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output.view(-1, ntokens), targets)
        loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 0.5)
        optimizer.step()

        total_loss += loss.item()
        log_interval = 200
        if batch % log_interval == 0 and batch > 0:
            cur_loss = total_loss / log_interval
            elapsed = time.time() - start_time
            print('| epoch {:3d} | {:5d}/{:5d} batches | '
                  'lr {:02.2f} | ms/batch {:5.2f} | '
                  'loss {:5.2f} | ppl {:8.2f}'.format(
                    epoch, batch, len(train_data) // bptt, scheduler.get_lr()[0],
                    elapsed * 1000 / log_interval,
                    cur_loss, math.exp(cur_loss)))
            total_loss = 0
            start_time = time.time()

def evaluate(eval_model, data_source):
    eval_model.eval() # 평가 모드를 시작합니다.
    total_loss = 0.
    ntokens = len(TEXT.vocab.stoi)
    with torch.no_grad():
        for i in range(0, data_source.size(0) - 1, bptt):
            data, targets = get_batch(data_source, i)
            output = eval_model(data)
            output_flat = output.view(-1, ntokens)
            total_loss += len(data) * criterion(output_flat, targets).item()
    return total_loss / (len(data_source) - 1)
```

에포크 내에서 반복됩니다. 만약 검증 오차(validation loss) 가 우리가 지금까지 관찰한 것 중 최적이라면 모델을 저장합니다.
매 에포크 이후에 학습률을 조절합니다.

```python
best_val_loss = float("inf")
epochs = 3 # 에포크 수
best_model = None

for epoch in range(1, epochs + 1):
    epoch_start_time = time.time()
    train()
    val_loss = evaluate(model, val_data)
    print('-' * 89)
    print('| end of epoch {:3d} | time: {:5.2f}s | valid loss {:5.2f} | '
          'valid ppl {:8.2f}'.format(epoch, (time.time() - epoch_start_time),
                                     val_loss, math.exp(val_loss)))
    print('-' * 89)

    if val_loss < best_val_loss:
        best_val_loss = val_loss
        best_model = model

    scheduler.step()
```

평가 데이터셋(test dataset) 으로 모델을 평가하기
-------------------------------------

평가 데이터셋에 대한 결과를 확인하기 위해서 최고의 모델을 적용합니다.

```python
test_loss = evaluate(best_model, test_data)
print('=' * 89)
print('| End of training | test loss {:5.2f} | test ppl {:8.2f}'.format(
    test_loss, math.exp(test_loss)))
print('=' * 89)
```


# Code stuff - not slides!

```python
%run ../ML_plots.ipynb
```

    ERROR:root:File `'../ML_plots.ipynb.py'` not found.

# Session 12:
## Supervised learning, part 1

*Andreas Bjerre-Nielsen*

## Agenda
1. [Modelling data](#Modelling-data)
1. [A familiar regression model](#A-familiar-regression-model)
1. [The curse of overfitting](#The-curse-of-overfitting)
1. [Important details](#Implementation-details)

## Vaaaamos

```python
import warnings
from sklearn.exceptions import ConvergenceWarning
warnings.filterwarnings(action='ignore', category=ConvergenceWarning)

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

plt.style.use('default') # set style (colors, background, size, gridlines etc.)
plt.rcParams['figure.figsize'] = 10, 4 # set default size of plots
plt.rcParams.update({'font.size': 18})
```

## Supervised problems (1)
*How do we distinguish between problems?*

```python
f_identify_question
```

## Supervised problems (2)
*The two canonical problems*

```python
f_identify_answer
```

## Supervised problems (3)
*Which models have we seen for classification?*

- .

- .

- .

# Modelling data

## Model complexity (1)
*What does a model of low complexity look like?*

```python
f_complexity[0]
```

## Model complexity (2)
*What does medium model complexity look like?*

```python
f_complexity[1]
```

## Model complexity (3)
*What does high model complexity look like?*

```python
f_complexity[2]
```

## Model fitting (1)
*Quiz (1 min.): Which model fitted the data best?*

```python
f_bias_var['regression'][2]
```

## Model fitting (2)
*What does underfitting and overfitting look like for classification?*

```python
f_bias_var['classification'][2]
```

## Two agendas (1)

What are the objectives of empirical research?

1. *causation*: what is the effect of a particular variable on an outcome?
2. *prediction*: find some function that provides a good prediction of $y$ as a function of $x$

## Two agendas (2)

How might we express the agendas in a model?

$$ y = \alpha + \beta x + \varepsilon $$

- *causation*: interested in $\hat{\beta}$

- *prediction*: interested in $\hat{y}$

## Two agendas (3)

Might these two agendas be related at a deeper level?

Can prediction quality inform us about how to make causal models?

# A familiar regression model

## Estimation (1)
*Do we know already some ways to estimate regression models?*

- Social scientists know all about the Ordinary Least Squares (OLS).
    - OLS estimate both parameters and their standard deviation.
    - Is best linear unbiased estimator under regularity conditions.

*How is OLS estimated?*

- $\beta=(\textbf{X}^T\textbf{X})^{-1}\textbf{X}^T\textbf{y}$
- computation requires non perfect multicollinarity.

## Estimation (2)
*How might we estimate a linear regression model?*

- first order method (e.g. gradient descent)
- second order method (e.g. Newton-Raphson)

*So what the hell was gradient descent?*

- compute errors, multiply with features and update

## Estimation (3)
*Can you explain that in details?*

- Yes, like with Adaline, we minimize the sum of squared errors (SSE):
\begin{align}SSE&=\boldsymbol{e}^{T}\boldsymbol{e}\\\boldsymbol{e}&=\textbf{y}-\textbf{X}\textbf{w}\end{align}

```python
X = np.random.normal(size=(3,2))
y = np.random.normal(size=(3))
w = np.random.normal(size=(3))

e = y-(w[0]+X.dot(w[1:]))
SSE = e.T.dot(e)
```

## Estimation (4)
*And what about the updating..? What is it something about the first order deritative?*

\begin{align}
\frac{\partial SSE}{\partial\hat{w}}=&\textbf{X}^T\textbf{e},\\
 \Delta\hat{w}=&\eta\cdot\textbf{X}^T\textbf{e}=\eta\cdot\textbf{X}^T(\textbf{y}-\hat{\textbf{y}})
\end{align}

```python
eta = 0.001 # learning rate
fod = X.T.dot(e)
update_vars = eta*fod
update_bias = eta*e.sum()
```

## Estimation (5)
*What might some advantages be relative to OLS?*

- Works despite high multicollinarity
- Speed
    - OLS has $\mathcal{O}(K^2N)$ computation time ([read more](https://math.stackexchange.com/questions/84495/computational-complexity-of-least-square-regression-operation))
        - Quadratic scaling in number of variables ($K$).
    - Stochastic gradient descent
        - Likely to converge faster with many observations ($N$)

## Fitting a polynomial (1)
Polyonomial: $f(x) = 2+8*x^4$

Try models of increasing order polynomials.

- Split data into train and test (50/50)

- For polynomial order 0 to 9:
    - Iteration n: $y = \sum_{k=0}^{n}(\beta_k\cdot x^k)+\varepsilon$.
    - Estimate order n model on training data
    - Evaluate with on test data with RMSE:
        - $log RMSE = \log (\sqrt{MSE})$

## Fitting a polynomial (2)
We generate samples of data from true model.

```python
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

def true_fct(X):
    return 2+X**4

n_samples = 25
n_degrees = 15

np.random.seed(0)

X_train = np.random.normal(size=(n_samples,1))
y_train = true_fct(X_train).reshape(-1) + np.random.randn(n_samples)

X_test = np.random.normal(size=(n_samples,1))
y_test = true_fct(X_test).reshape(-1) + np.random.randn(n_samples)
```

## Fitting a polynomial (3)
We estimate the polynomials

```python
from sklearn.metrics import mean_squared_error as mse

test_mse = []
train_mse = []
parameters = []
degrees = range(n_degrees+1)

for p in degrees:
    X_train_p = PolynomialFeatures(degree=p).fit_transform(X_train)
    X_test_p = PolynomialFeatures(degree=p).fit_transform(X_train)
    reg = LinearRegression().fit(X_train_p, y_train)
    train_mse += [mse(reg.predict(X_train_p),y_train)]
    test_mse += [mse(reg.predict(X_test_p),y_test)]
    parameters.append(reg.coef_)
```

## Fitting a polynomial (4)
*So what happens to the model performance in- and out-of-sample?*

```python
degree_index = pd.Index(degrees,name='Polynomial degree ~ model complexity')
ax = pd.DataFrame({'Train set':train_mse, 'Test set':test_mse})\
    .set_index(degree_index)\
    .plot(figsize=(10,4))
ax.set_ylabel('Mean squared error')
```

## Fitting a polynomial (4)
*Why does it go wrong?*
- more spurious parameters
- the coefficient size increases

## Fitting a polynomial (5)
*What do you mean coefficient size increase?*

```python
order_idx = pd.Index(range(n_degrees+1),name='Polynomial order')
ax = pd.DataFrame(parameters,index=order_idx)\
.abs().mean(1)\
.plot(logy=True)
ax.set_ylabel('Mean parameter size')
```

## Fitting a polynomial (6)
*How else could we visualize this problem?*

```python
f_bias_var['regression'][2]
```

# The curse of overfitting

## Looking for a remedy
*How might we solve the overfitting problem?*

By reducing
- the number of variables
- the coefficient size of variables

## Regularization (1)

*Why do we regularize?*

- To mitigate overfitting > better model predictions

*How do we regularize?*

- We make models which are less complex:
  - reducing the **number** of coefficient;
  - reducing the **size** of the coefficients.

## Regularization (2)

*What does regularization look like?*

We add a penalty term our optimization procedure:

$$ \text{arg min}_\beta \, \underset{\text{MSE}}{\underbrace{E[(y_0 - \hat{f}(x_0))^2]}} + \underset{\text{penalty}}{\underbrace{\lambda \cdot R(\beta)}}$$

Introduction of penalties implies that increased model complexity has to be met with high increases precision of estimates.

## Regularization (3)

*What are some used penalty functions?*

The two most common penalty functions are L1 and L2 regularization.

- L1 regularization (***Lasso***): $R(\beta)=\sum_{j=1}^{p}|\beta_j|$
    - Makes coefficients sparse, i.e. selects variables by removing some (if $\lambda$ is high)

- L2 regularization (***Ridge***): $R(\beta)=\sum_{j=1}^{p}\beta_j^2$
    - Reduce coefficient size
    - Fast due to analytical solution

*To note:* The *Elastic Net* uses a combination of L1 and L2 regularization.

## Regularization (4)

*How the Lasso (L1 reg.) deviates from OLS*

## Regularization (5)

*How the Ridge regression (L2 reg.) deviates from OLS*

## Regularization (6)

*How might we describe the $\lambda$ of Lasso and Ridge?*

These are hyperparameters that we can optimize over.

- More about this tomorrow.

# Implementation details

## The devils in the details (1)

*So we just run regularization?*

# NO

We need to rescale our features:
- convert to zero mean:
- standardize to unit std:

Compute in Python:
- option 1: `StandardScaler` in `sklearn`
- option 2: `(X - np.mean(X)) / np.std(X)`

## The devils in the details (2)
*So we just scale our test and train?*

# NO

Fit to the distribution in the training data first, then rescale train and test! See more [here](https://stats.stackexchange.com/questions/174823/how-to-apply-standardization-normalization-to-train-and-testset-if-prediction-i).

## The devils in the details (3)
*So we just rescale before using polynomial features?*

# NO

Otherwise the interacted varaibles are not gaussian distributed.

## The devils in the details (4)
*Does sklearn's `PolynomialFeatures` work for more than variable?*

# YES!

# The end
[Return to agenda](#Agenda)


# XGBoost原理与应用

作者：杨岱川

时间：2019年11月

github：https://github.com/DrDavidS/basic_Machine_Learning

开源协议：[MIT](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/LICENSE)

参考文章：

- [XGBoost PPT](https://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf)
- [XGBoost: A Scalable Tree Boosting System](https://arxiv.org/abs/1603.02754)
- [Introduction to Boosted Trees](https://xgboost.readthedocs.io/en/latest/tutorials/model.html)
- [XGBoost原理](https://www.zhihu.com/question/58883125/answer/206813653)
- [XGBoost原理及目标函数推导详解](https://blog.csdn.net/htbeker/article/details/91517805)

## XGBoost

在[2.10 提升方法](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/02机器学习基础/2.10%20提升方法.ipynb)中我们简单提到过，在SKLearn之外还有很多优秀的，基于Boosting Tree方法的衍生框架。包括XGBoost，LightGBM等，而各种开源比赛方案对它们的应用，也从事实上证明了这些框架的优秀性能。

因此这一节会简要介绍XGBoost这一新框架的原理和用法。

### 简介

**XGBoost** 全称为“eXtreme Gradient Boosting”，是一种基于决策树的集成机器学习算法，使用梯度提升框架，适用于分类和回归问题。XGBoost 在各种比赛及比赛平台中大放异彩，比如Kaggle，天池，DataCastle等等，可以说是所有选手的必备武器之一。XGBoost 的项目主页参见[这里](https://xgboost.ai)。

XGBoost 最初由陈天奇开发。陈天奇本科毕业于上海交通大学 ACM 班，博士毕业于华盛顿大学计算机系，研究方向为大规模机器学习，2020年他将加入 CMU 出任助理教授。有兴趣的可以看看他的文章[《陈天奇：机器学习科研的十年》](http://www.sohu.com/a/328234576_129720)以及[访谈](https://cosx.org/2015/06/interview-of-tianqi)，也可以看看他的[知乎主页](https://www.zhihu.com/people/crowowrk/activities)。

下图展示了从决策树到 XGBoost 算法的发展过程：

图片刷不出来的请看[这里](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/back_up_images/xgboost发展历程.jpeg)

在 [XGBoost.ai](https://xgboost.ai) 官网有对 XGBoost原理有通俗易懂的解释。

现在简要介绍如下，部分基础知识之前也已经讨论过了，现在就当复习。其中的各种符号跟这里和论文保持一致，所以和之前的 GBDT 介绍可能有少许差别。

### 目标函数：经验损失 + 正则化

训练模型的目标是找到最好的参数 $\theta$ ，能够最佳拟合训练数据 $x_i$ 和标签 $y_i$。所以XGBoost为了训练模型，定义了一个**目标函数（objective function）** 来告诉模型如何去拟合训练数据。

目标函数由两部分组成：**经验损失**（又叫**训练损失**，**training loss**）和**正则项（regularization term）**。

$$\large {\rm obj}(\theta)=L(\theta)+\Omega(\theta)$$

其中， $L$ 就是训练损失，而 $\Omega$ 就是正则项。

通常，$L$ 会选择均方误差，如下：

$$\large L(\theta)=\sum_i(y_i-\hat y_i)^2$$

其中，$\hat y_i=\sum_j \theta_j x_{ij}$，也就是模型的预测值。

另一种常见的损失函数叫做**logistic loss**，也就是之前在逻辑回归中讲过的损失函数：

$$\large L(\theta) = \sum_i \left[ y_i \ln(1+e^{-\hat y_i})+(1-y_i)\ln(1+e^{-\hat y_i}) \right]$$

至于正则项$\Omega$，就是控制模型复杂度的，让模型不至于过拟合，以前的章节讲过多次，这里不再赘述。

### 决策树的集成

XGBoost模型是基于决策树的集成（**decision tree ensembles**）而来，这个集成模型是由一组**分类与回归树（CART）**所构成的，CART 的分裂基于 gini 系数，关于 CART 的详细描述可以参见《统计学习方法》中决策树一章。

这里给了一个简单的例子，比如我们要运用 CART 来判断某人是否喜欢电脑游戏。我们输入一家人的年龄、性别、职业等信息，然后得到一颗决策树：

图片刷不出来的看这里：[CART](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/back_up_images/cart.png)

我们将家庭成员分为不同的叶子结点（leaves），然后在相应的叶子上给他们分配分数（score）。

>**score** 怎么来的？
>
>由于这里使用的是 CART，所以当作是回归树而不要当作分类树来看，叶子结点上给出的是回归预测值，而非分类预测值。
>
>在每个叶子结点采用这种真实分数，可以提供超越“分类”的更丰富的解释，比如给出分类概率等，也会更方便优化，在后面会有展示。

之前有提到，一棵决策树树是不够强大的，不管是随机森林、AdaBoost还是GBDT，都是集成模型，我们需要把多个**弱分类器**（基本分类器）的预测结果汇总在一起，形成一个**强分类器**，这就是**集成模型（ensemble model）**。

在这里，我们汇总的方法就是把多棵树的 score 加起来，如图：

图片刷不出来的看这里：[two CART](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/back_up_images/twocart.png)

以上图两棵 CART 树为例，可以看出模型的集成就是简单地把两棵树叶子结点对实例的预测分数相加即可。上面这个例子表达了一个重要的点，就是两棵 CART 互为补充，以数学的形式展示就是:

$$\large \hat y_i=\sum^K_{k=1}f_k(x_i),\quad f_k\in \Bbb F$$

其中 $K$ 是树的数量， $f$ 就是一棵CART树的函数，属于函数空间 $\Bbb F$，而 $\Bbb F$ 是所有可能的 CART 的集合。

我们待优化的目标函数在一开始已经给出了，这里对其稍作细化：

$$\large {\rm obj}(\theta)=\sum_i^n l(y_i,\hat y_i) + \sum^K_{k=1}\Omega(f_k)$$

左边的 $\sum_i^n l(y_i,\hat y_i)$ 是损失函数，右边的 $\sum^K_{k=1}\Omega(f_k)$ 是正则项。

现在的回顾一下，随机森林中使用的 *模型* 是也是集成的树（Tree ensembles），所以 boosted trees 和 random forests 的核心原理还是一样的，只不过训练过程中有点点区别。这也是我们在讲Boost算法的时候叙述过的。

### 树的Boosting

如何训练模型？相信大家肯定不陌生，老样子，那就是 *定义目标函数然后对其优化* ！

将下式作为目标函数。需要说明的是，记住它始终包含了正则项，而在我们 2.10节中讲的 AdaBoost 和 GBDT，是没有涉及正则项的。

$$\large {\rm obj}= \sum_{i=1}^n l(y_i,\hat y_i^{(t)}) + \sum^t_{i=1}\Omega(f_k)$$

#### Additive Training

第一个问题：树的 **参数** 是什么？我们要学习的是那些 $f_i$ 函数，每个函数都包含树的结构和叶子得分（leaf scores）。

在这里针对树的训练，采用加法策略（前向分步算法），也就是和之前 Boosting 讲的一样，根据学习结果进行权重修正，然后新增一棵树。

我们定义在时间步 $t$ 时候的预测值为 $\large \hat y_i^{(t)}$，有:

$$
\large
\begin{equation}\begin{split}
\hat y_i^{(0)}&=0 \\
\hat y_i^{(1)}&=f_1(x_i)=\hat y_i^{(0)}+f_1(x_i)\\
\hat y_i^{(2)}&=f_1(x_i)+f_2(x_i)=\hat y_i^{(1)}+f_2(x_i)\\
\cdots\\
\hat y_i^{(t)}&=\sum^t_{k-1}f_k(x_i)=\hat y_i^{(t-1)}+f_t(x_i)
\end{split}\end{equation}
$$

还有一个问题就是，我们在每一步要的是那棵树？我们自然要选择能优化我们**目标函数**的那棵。

在 $t$ 步的时候，目标函数 ${\rm obj}^{(t)}$ 为：

$$
\large
\begin{equation}\begin{split}
{\rm obj}^{(t)} &=  \sum_{i=1}^n l(y_i,\hat y_i^{(t)}) + \sum^t_{i=1}\Omega(f_i)\\
&= \sum_{i=1}^n l(y_i,\hat y_i^{(t-1)}+f_t({x_i})) + \Omega(f_t)+\sum^{t-1}_{k=1}\Omega(f_t)\\
&= \sum_{i=1}^n l(y_i,\hat y_i^{(t-1)}+f_t({x_i})) + \Omega(f_t)+{\rm constant}
\end{split}\end{equation}
$$

> 这个 $\rm constant$ 就是常数的意思。由于前向分步算法，在 $t$ 步时候，前 $t-1$ 步的树都是已经确定了的，因此其结构是一个常数。
>
> 相当于 $\large \sum^{t-1}_{k=1}\Omega(f_t)={\rm constant}$

如果我们使用**均方误差（mean squared error ，MSE）**作为我们的损失函数，即替换 $l(y_i,\hat y_i^{(t)})$，那么目标函数就变成了：

$$
\large
\begin{equation}\begin{split}
{\rm obj}^{(t)} &=  \sum_{i=1}^n \left(y_i - \left(\hat y_i^{(t-1)}+f_t(x_i)\right)\right)^2 + \sum^t_{i=1}\Omega(f_i)\\
&= \sum_{i=1}^n \left[ 2\left( \hat y_i^{(t-1)}-y_i \right)f_t(x_i)+f_t(x_i)^2 \right] + \Omega(f_t)+{\rm constant}
\end{split}\end{equation}
$$

> 注意这里 $\large \hat y_i^{(t)}$ 实际上就是 $t-1$ 步之前的 CART 森林加上第 $t$ 步生成的 CART 树的集成模型的预测结果，即:
>
> $$\large \hat y_i^{(t)} = \hat y_i^{(t-1)}+f_t(x_i)$$
>
>故均方误差为：
>
> $$\large \left(y_i-\hat y_i^{(t)}\right)^2$$

均方误差的表示形式非常方便，存在一次项和二次项。但是如果损失函数换为其他损失，比如 logistic loss，要获得这样的直观形式就比较困难了。所以，在一般情况下，我们将损失函数的**泰勒展开式（Taylor expansion）**扩展到二阶：

$$\large {\rm obj}^{(t)}=\sum_{i=1}^n \left[l(y_i,\hat y_i^{(t-1)}) + g_if_t(x_i) + \frac{1}{2}h_if_t^2(x_i)\right] + \Omega(f_t)+{\rm constant}$$

其中 $g_i$ 和 $h_i$ 定义为：

$$\large g_i=\partial_{\hat y_i^{(t-1)}}l(y_i,\hat y_i^{(t-1)})$$

$$\large h_i=\partial^2_{\hat y_i^{(t-1)}}l(y_i,\hat y_i^{(t-1)})$$

就是对损失函数求预测值 $\large \hat y_i^{(t-1)}$ 的一阶和二阶偏导数。

> 泰勒公式的二阶近似可以表示为：
>
>$$\large f(x_0+\Delta x) \approx f(x_0)+f'(x_0)\Delta x+\frac{1}{2}f''(x_0)(\Delta x)^2$$
>
> 这里的增量 $\Delta x$ 相当于 $t-1$ 步的森林中新增一棵 $t$ 时间步的树，也就是 $f_t(x_i)$
>
> 至于为什么要求偏导数，因为 $\hat y_i^{(t)}$ 是我们要优化的目标，而不是 $y_i$，可以看看**梯度下降**相关内容。

现在我们去掉所有常数项，因为在优化过程中保留常数没啥必要，目标函数在 $t$ 步时候就变成了：

$$\large \sum^n_{i=1}\left[ g_if_t(x_i) + \frac{1}{2}h_if_t^2(x_i) \right] + \Omega(f_t)$$

这就是我们对新的树的优化目标。

我们新优化目标的优点在于目标函数的取值仅仅取决于 $g_i$ 和 $h_i$，也就是损失函数的一二阶导数，所以 XGBoost 支持自定义损失函数，只要它二阶可导。

> XGBoost 的特点就是将损失函数泰勒展开到了二阶，GBDT只用到了一阶，可以回头看看[2.10 提升方法](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/%E6%9D%AD%E7%94%B5%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E8%AF%BE%E7%A8%8B%E5%8F%8A%E4%BB%A3%E7%A0%81/2.10%20%E6%8F%90%E5%8D%87%E6%96%B9%E6%B3%95.ipynb)中求负梯度那部分。

### 模型复杂度

刚刚已经介绍过了训练的过程，但是还没有涉及非常重要的**正则项（regularization term）**，所以这里我们先定义树的复杂度 $\Omega(f)$。

首先将树的函数 $f(x)$ 细化为：

$$\large f_t(x)=w_{q(x)},\quad w \in R^T,q:R_d \rightarrow \{1,2,\cdots,T\}$$

其中 $w$ 是叶子上分数的矢量，$q$ 是将每个数据实例分配给相应叶子结点的函数，$T$ 是叶子的总数。

换句话说，$q(x)$ 就是输出的叶子节点的序号，$w_{q(x)}$ 表示对应的叶子节点的得分。

在 XGBoost中，我们将复杂度定义为：

$$\large \Omega(f)=\gamma T+\frac{1}{2}\lambda\sum^T_{j=1}w^2_j$$

相当于**叶子的总数** $T$ 加上 **叶子分数的L2正则**，$j$ 是叶子结点的编号。

定义模型复杂度的方法很多，但是这种定义方法在实践中表现的很好。在以前的很多基于决策树的工具中不是很在意这部分，一般都是把复杂度的控制留给了一些启发式算法。在 XGBoost 中给出了一个正式的定义，我们可以更好地了解模型的训练过程。

### The Structure Score

定义了模型的复杂度之后，我们将模型的目标函数重写：

$$
\large
\begin{equation}\begin{split}
{\rm obj}^{(t)} &\approx  \sum_{i=1}^n \left[ g_iw_{q(x_i)}+\frac{1}{2}h_iw^2_{q(x_i)} \right] + \gamma T+\frac{1}{2}\lambda\sum^T_{j=1}w^2_j \\
&= \sum_{j=1}^T \left[ \left( \sum_{i\in I_j}g_i \right)w_j + \frac{1}{2}\left( \sum_{i\in I_j}h_i + \lambda \right)w_j^2 \right] + \gamma T
\end{split}\end{equation}
$$

其中，$I_j=\{ i|q(x_i)=j \}$，表示分配给第 $j$ 个叶子的样本 $x_i$ 的索引，换句话说就是回归树中的叶结点区域。

然后上式把所有的样本点做了一下合并，因为同一个叶子结点上所有样本的分数都是一样的。

定义：

$$\large G_j=\sum_{j\in I_j}g_i$$

$$\large H_j=\sum_{j\in I_j}h_i$$

我们可以进一步简化目标函数 ${\rm obj}^{(t)}$：

$$\large {\rm obj}^{(t)}=\sum^T_{j=1}\left[ G_jw_j + \frac{1}{2}(H_j+\lambda)w_j^2 \right]+\gamma T$$

在这个等式中，$w_j$ 是彼此独立的，$G_jw_j + \cfrac{1}{2}(H_j+\lambda)w_j^2$ 是关于 $w_j$ 的一元二次函数，对于给定的 $q(x)$，对 $w_j$ 求导可以得到最优的 $w_j$ 取值：

$$\large w_j^*=-\frac{G_j}{H_j+\lambda}$$

把上式代入回 ${\rm obj}^{(t)}$：

$$\large {\rm obj}^*=-\frac{1}{2}\sum^T_{j=1}\frac{G_j^2}{H_j+\lambda}+\gamma T$$

得到最终的目标函数 ${\rm obj}^*$，它也称为**打分函数（scoring function）**，用以衡量树结构 $q(x)$ 的好坏，值越小代表结构越好。我们采用这个打分函数来选择 CART 的最佳切分点。

图片刷不出来的看这里：[struct_score](https://github.com/DrDavidS/basic_Machine_Learning/blob/master/back_up_images/struct_score.png)

如图，基本上对于给定的树结构，我们将统计实例的一二阶导数，也就是梯度信息 $g_i$ 和 $h_i$ ，然后放入它们所对应的叶子结点。再对叶子上实例的梯度信息求和，使用上面推出的公式计算树的质量。其实有点类似于计算一棵 CART 决策树的 gini 不纯度，但是不同之处在于它还考虑的模型的复杂度。

关于**gini不纯度**可以参考《统计学习方法》中决策树一章对 CART 的讲解。

### 学习树的结构

到目前为止，我们有了一种衡量一棵决策树质量的方法，理想情况下，我们会列举出所有可能的决策树，然后选择最好的一棵。但是实际上这是几乎不可能的。所以我们会试着一次优化树的一层（level），层层深入。

具体来说，我们把一个结点切分为两个子结点，其得分为：

$$\large Gain=\frac{1}{2}\left[ \frac{G^2_L}{H_L+\lambda}+\frac{G^2_R}{H_R+\lambda}-\frac{(G_L+G_R)^2}{H_L+H_R+\lambda} \right]-\gamma$$

该公式可以理解为新切分的左叶子结点的分数加上新切分的右叶子结点的分数减去原来叶子结点的分数，外加一个叶上的正则化。

这里有一个重要的事实，就是如果新切分的增益小于 $\gamma$，就不要添加该分支，这相当于是决策树的剪枝技术。这个 $\gamma$ 值是一个超参数，人工设定。

我们希望搜索到一个最佳的切分，而为了达成这个目标，将所有实例按照排序顺序放置，如图,按照年龄排序：

然后我们从左到右开始扫描分裂点，计算这些 CART 树的得分，就可以高效地找到最佳拆分点。

> 前向算法的局限性
>
> 在少部分边缘情况下，前向算法可能会失败，扩展阅读：[Can Gradient Boosting Learn Simple Arithmetic?](http://mariofilho.com/can-gradient-boosting-learn-simple-arithmetic/)

### 结语

到现在我们已经完成了对 XGBoost 原理的学习，而 XGBoost 也根据以上原理编写成了 XGBoost 工具，记得尝试一下这一款优秀的机器学习工具吧！

## XGBoost 安装和使用

参考[Get Started with XGBoost](https://xgboost.readthedocs.io/en/latest/get_started.html)

示例代码请先参考官方文档，本 NoteBook 待补充。

```python

```


```python
pip install tensorflow==2.4.1 tensorflow-quantum
```

```python
# Update package resources to account for version changes.
import importlib, pkg_resources
importlib.reload(pkg_resources)
```

    <module 'pkg_resources' from '/usr/local/lib/python3.7/dist-packages/pkg_resources/__init__.py'>

```python
from google.colab import drive
drive.mount('/content/drive')
```

    Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).

```python
import sys
sys.path.append('/content/drive/My Drive')
```

```python
import eecs598
import torch
import torchvision
import matplotlib.pyplot as plt
import statistics
import numpy as np
```

```python
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np
import seaborn as sns
import collections

# visualization tools
%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

```python

# Control qrid size for visualization
plt.rcParams['figure.figsize'] = (10.0, 8.0)
plt.rcParams['font.size'] = 16
```

```python
x_train, y_train, x_test, y_test = eecs598.data.cifar10()

print('Training set:', )
print('  data shape:', x_train.shape)
print('  labels shape: ', y_train.shape)
print('Test set:')
print('  data shape: ', x_test.shape)
print('  labels shape', y_test.shape)
```

    Training set:
      data shape: torch.Size([50000, 3, 32, 32])
      labels shape:  torch.Size([50000])
    Test set:
      data shape:  torch.Size([10000, 3, 32, 32])
      labels shape torch.Size([10000])

```python
#Visualization of  the dataset

import random
from torchvision.utils import make_grid

classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
samples_per_class = 12
samples = []
for y, cls in enumerate(classes):
    plt.text(-4, 34 * y + 18, cls, ha='right')
    idxs, = (y_train == y).nonzero(as_tuple=True)
    for i in range(samples_per_class):
        idx = idxs[random.randrange(idxs.shape[0])].item()
        samples.append(x_train[idx])
img = torchvision.utils.make_grid(samples, nrow=samples_per_class)
plt.imshow(eecs598.tensor_to_image(img))
plt.axis('off')
plt.show()
```

```python

```

```python
from tensorflow.keras.datasets import cifar10
(train_images, train_labels), (test_images, test_labels) = cifar10.load_data()
```

```python
y_train=train_labels.flatten()
y_test=test_labels.flatten()
# We have reduced the dimension
```

```python
x_train=tf.image.rgb_to_grayscale(train_images)
x_test=tf.image.rgb_to_grayscale(test_images)
#to convert grayscale
```

```python
#Filter the dataset to keep just the cat  and frog, remove the other classes
def filter_36(x, y):
    keep = (y == 3) | (y == 6)
    x, y = x[keep], y[keep]
    y = y == 3
    return x,y
```

```python
x_train, y_train = filter_36(x_train, y_train)
x_test, y_test = filter_36(x_test, y_test)

print("Number of filtered training examples:", len(x_train))
print("Number of filtered test examples:", len(x_test))
```

    Number of filtered training examples: 10000
    Number of filtered test examples: 2000

```python
plt.imshow(x_train[0, :, :, 0])
plt.colorbar()
```

```python
x_train_s = tf.image.resize(x_train, (4,4)).numpy()
x_test_s = tf.image.resize(x_test, (4,4)).numpy()
```

```python

THRESHOLD = 0.5

x_train_bin = np.array(x_train_s > THRESHOLD, dtype=np.float32)
x_test_bin = np.array(x_test_s > THRESHOLD, dtype=np.float32)
```

```python
def convert_to_circuit(image):
    """Encode truncated classical image into quantum datapoint."""
    values = np.ndarray.flatten(image)
    qubits = cirq.GridQubit.rect(4, 4)
    circuit = cirq.Circuit()
    for i, value in enumerate(values):
        if value:

            circuit.append(cirq.Circuit(cirq.X(qubits[i]), cirq.Y(qubits[i]),cirq.Z(qubits[i]) ))

    return circuit

x_train_circ = [convert_to_circuit(x) for x in x_train_bin]
x_test_circ = [convert_to_circuit(x) for x in x_test_bin]
```

```python
SVGCircuit(x_train_circ[0])

```

```python
x_train_tfcirc = tfq.convert_to_tensor(x_train_circ)
x_test_tfcirc = tfq.convert_to_tensor(x_test_circ)
```

```python
class CircuitLayerBuilder():
    def __init__(self, data_qubits, readout):
        self.data_qubits = data_qubits
        self.readout = readout

    def add_layer(self, circuit, gate, prefix):
        for i, qubit in enumerate(self.data_qubits):
            symbol = sympy.Symbol(prefix + '-' + str(i))
            circuit.append(gate(qubit, self.readout)**symbol)
```

```python
demo_builder = CircuitLayerBuilder(data_qubits = cirq.GridQubit.rect(4,1),
                                   readout=cirq.GridQubit(-1,-1))

circuit = cirq.Circuit()
demo_builder.add_layer(circuit, gate = cirq.XX, prefix='xx')
SVGCircuit(circuit)
```

```python
def create_quantum_model():
    """Create a QNN model circuit and readout operation to go along with it."""
    data_qubits = cirq.GridQubit.rect(4, 4)  # a 4x4 grid.
    readout = cirq.GridQubit(-1, -1)         # a single qubit at [-1,-1]
    circuit = cirq.Circuit()

    # Prepare the readout qubit.
    circuit.append(cirq.X(readout))
    circuit.append(cirq.H(readout))

    builder = CircuitLayerBuilder(
        data_qubits = data_qubits,
        readout=readout)

    # Then add layers (experiment by adding more).
    builder.add_layer(circuit, cirq.XX, "xx1")
    builder.add_layer(circuit, cirq.ZZ, "zz1")

    # Finally, prepare the readout qubit.
    circuit.append(cirq.H(readout))

    return circuit, cirq.Z(readout)
```

```python
model_circuit, model_readout = create_quantum_model()
```

```python
model = tf.keras.Sequential([
    # The input is the data-circuit, encoded as a tf.string
    tf.keras.layers.Input(shape=(), dtype=tf.string),
    # The PQC layer returns the expected value of the readout gate, range [-1,1].
    tfq.layers.PQC(model_circuit, model_readout),
])
```

```python
y_train_hinge = 2.0*y_train-1.0
y_test_hinge = 2.0*y_test-1.0

```

```python
def hinge_accuracy(y_true, y_pred):
    y_true = tf.squeeze(y_true) > 0.0
    y_pred = tf.squeeze(y_pred) > 0.0
    result = tf.cast(y_true == y_pred, tf.float32)

    return tf.reduce_mean(result)
```

```python
model.compile(
    loss=tf.keras.losses.Hinge(),
    optimizer=tf.keras.optimizers.Adam(),
    metrics=[hinge_accuracy])

```

```python
print(model.summary())
```

    Model: "sequential_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    pqc_1 (PQC)                  (None, 1)                 18
    =================================================================
    Total params: 18
    Trainable params: 18
    Non-trainable params: 0
    _________________________________________________________________
    None

```python
EPOCHS = 3
BATCH_SIZE = 128

NUM_EXAMPLES = len(x_train_tfcirc)

```

```python
x_train_tfcirc_sub = x_train_tfcirc[:NUM_EXAMPLES]
y_train_hinge_sub = y_train_hinge[:NUM_EXAMPLES]

```

```python
import time
start_time = time.time()
```

```python
qnn_history = model.fit(
      x_train_tfcirc_sub, y_train_hinge_sub,
      batch_size=32,
      epochs=EPOCHS,
      verbose=1,
      validation_data=(x_test_tfcirc, y_test_hinge))

qnn_results = model.evaluate(x_test_tfcirc, y_test)

```

    Epoch 1/3
    313/313 [==============================] - 487s 2s/step - loss: 1.0004 - hinge_accuracy: 0.5010 - val_loss: 1.0021 - val_hinge_accuracy: 0.5015
    Epoch 2/3
    313/313 [==============================] - 483s 2s/step - loss: 1.0041 - hinge_accuracy: 0.4933 - val_loss: 1.0021 - val_hinge_accuracy: 0.5015
    Epoch 3/3
    313/313 [==============================] - 484s 2s/step - loss: 1.0015 - hinge_accuracy: 0.4947 - val_loss: 1.0022 - val_hinge_accuracy: 0.5015
    63/63 [==============================] - 19s 302ms/step - loss: 1.0022 - hinge_accuracy: 0.5015

```python
qnn_accuracy = qnn_results[1]
qnn_accuracy
```

    0.5014880895614624

```python
model.predict_classes(x_train_tfcirc[0:7])
```

    /usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/sequential.py:450: UserWarning: `model.predict_classes()` is deprecated and will be removed after 2021-01-01. Please use instead:* `np.argmax(model.predict(x), axis=-1)`,   if your model does multi-class classification   (e.g. if it uses a `softmax` last-layer activation).* `(model.predict(x) > 0.5).astype("int32")`,   if your model does binary classification   (e.g. if it uses a `sigmoid` last-layer activation).
      warnings.warn('`model.predict_classes()` is deprecated and '

    array([[0],
           [0],
           [0],
           [0],
           [0],
           [0],
           [0]], dtype=int32)

```python
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.gridspec import GridSpec

def format_axes(fig):
    for i, ax in enumerate(fig.axes):
        ax.tick_params(labelbottom=False, labelleft=False)

fig = plt.figure(figsize=(10, 10))

gs = GridSpec(3, 3, figure=fig)
ax1 = fig.add_subplot(gs[0, 0])
# identical to ax1 = plt.subplot(gs.new_subplotspec((0, 0), colspan=3))

for i in range(3):
  for j in range(3):
    ax = fig.add_subplot(gs[i, j])

    ax.imshow(x_train[i+j, :, :, 0])

fig.suptitle("GridSpec")
format_axes(fig)

plt.show()
```

```python
plt.plot(qnn_history.history['hinge_accuracy'], label='QNN')
#plt.plot(hybrid_history.history['val_custom_accuracy'], label='Hybrid CNN')
#plt.title('Quantum vs Hybrid CNN performance')
plt.xlabel('Epochs')
plt.legend()
plt.ylabel('Validation Accuracy')
plt.show()
```