# Coupled Cluster theory and infinite nuclear matter

 **Morten Hjorth-Jensen**, [National Superconducting Cyclotron Laboratory](http://www.nscl.msu.edu/) and [Department of Physics and Astronomy](https://www.pa.msu.edu/), [Michigan State University](http://www.msu.edu/), East Lansing, MI 48824, USA and Department of Physics, University of Oslo, N-0316 Oslo, Norway

Date: **28th Indian-Summer School on Ab Initio Methods in Nuclear Physics**

# Introduction
Coester and Kummel first developed the ideas that led to coupled-cluster
theory in the late 1950s. The basic idea is that the correlated wave function
of a many-body system $\mid\Psi\rangle$
can be formulated as an exponential of correlation
operators $T$ acting on a reference state $\mid\Phi\rangle$

$$
\mid\Psi\rangle = \exp\left(-\hat{T}\right)\mid\Phi\rangle\ .
$$

We will discuss how to define the operators later in this work. This simple
ansatz carries enormous power. It leads to a non-perturbative many-body
theory that includes summation of ladder diagrams , ring
diagrams, and an infinite-order
generalization of many-body perturbation theory.

# Introduction
Developments and applications
of coupled-cluster theory took different routes in chemistry
and nuclear physics. In quantum chemistry,
coupled-cluster developments
and applications have proven to be extremely useful, see for example the review by [Barrett and Musial](http://journals.aps.org/rmp/abstract/10.1103/RevModPhys.79.291) as well as the recent
textbook by [Shavitt and Barrett](http://www.cambridge.org/fr/academic/subjects/chemistry/physical-chemistry/many-body-methods-chemistry-and-physics-mbpt-and-coupled-cluster-theory?format=HB).  Many previous applications to nuclear physics struggled with the repulsive character of the nuclear forces and limited basis sets used in the computations. Most of these problems have been overcome during the last decade and coupled-cluster
theory is one of the computational methods of preference for doing nuclear physics, with applications ranging from light nuclei to medium-heavy nuclei,
see for example the recent review by [Hagen, Papenbrock, Hjorth-Jensen and Dean](http://iopscience.iop.org/0034-4885/77/9/096302).

## A non-practical way of solving the eigenvalue problem

Before we proceed with the derivation of the Coupled cluster equations, let us repeat some of the arguments we presented during our FCI lectures.
In our FCI discussions, we rewrote the solution of the Schroedinger equation as a set of coupled equationsin the unknown coefficients $C$. Let us repeat some of these arguments.
To obtain the eigenstates and eigenvalues in terms of non-linear equations is not a very practical approach. However, it serves the scope of linking FCI theory with approximative solutions to the many-body problem
like Coupled cluster (CC) theory

## A non-practical way of solving the eigenvalue problem
If we assume that we have a two-body operator at most, the Slater-Condon rule
gives then an equation for the
correlation energy in terms of $C_i^a$ and $C_{ij}^{ab}$ only.  We get then

$$
\langle \Phi_0 | \hat{H} -E| \Phi_0\rangle + \sum_{ai}\langle \Phi_0 | \hat{H} -E|\Phi_{i}^{a} \rangle C_{i}^{a}+
\sum_{abij}\langle \Phi_0 | \hat{H} -E|\Phi_{ij}^{ab} \rangle C_{ij}^{ab}=0,
$$

or

$$
E-E_0 =\Delta E=\sum_{ai}\langle \Phi_0 | \hat{H}|\Phi_{i}^{a} \rangle C_{i}^{a}+
\sum_{abij}\langle \Phi_0 | \hat{H}|\Phi_{ij}^{ab} \rangle C_{ij}^{ab},
$$

where the energy $E_0$ is the reference energy and $\Delta E$ defines the so-called correlation energy.
The single-particle basis functions  could be the results of a Hartree-Fock calculation or just the eigenstates of the non-interacting part of the Hamiltonian.

## A non-practical way of solving the eigenvalue problem
In our notes on Hartree-Fock calculations,
we have already computed the matrix $\langle \Phi_0 | \hat{H}|\Phi_{i}^{a}\rangle $ and $\langle \Phi_0 | \hat{H}|\Phi_{ij}^{ab}\rangle$.  If we are using a Hartree-Fock basis, then the matrix elements
$\langle \Phi_0 | \hat{H}|\Phi_{i}^{a}\rangle=0$ and we are left with a *correlation energy* given by

$$
E-E_0 =\Delta E^{HF}=\sum_{abij}\langle \Phi_0 | \hat{H}|\Phi_{ij}^{ab} \rangle C_{ij}^{ab}.
$$

## A non-practical way of solving the eigenvalue problem
Inserting the various matrix elements we can rewrite the previous equation as

$$
\Delta E=\sum_{ai}\langle i| \hat{f}|a \rangle C_{i}^{a}+
\sum_{abij}\langle ij | \hat{v}| ab \rangle C_{ij}^{ab}.
$$

This equation determines the correlation energy but not the coefficients $C$.
We need more equations. Our next step is to set up

$$
\langle \Phi_i^a | \hat{H} -E| \Phi_0\rangle + \sum_{bj}\langle \Phi_i^a | \hat{H} -E|\Phi_{j}^{b} \rangle C_{j}^{b}+
\sum_{bcjk}\langle \Phi_i^a | \hat{H} -E|\Phi_{jk}^{bc} \rangle C_{jk}^{bc}+
\sum_{bcdjkl}\langle \Phi_i^a | \hat{H} -E|\Phi_{jkl}^{bcd} \rangle C_{jkl}^{bcd}=0,
$$

as this equation will allow us to find an expression for the coefficents $C_i^a$ since we can rewrite this equation as

$$
\langle i | \hat{f}| a\rangle +\langle \Phi_i^a | \hat{H}|\Phi_{i}^{a} \rangle C_{i}^{a}+ \sum_{bj\ne ai}\langle \Phi_i^a | \hat{H}|\Phi_{j}^{b} \rangle C_{j}^{b}+
\sum_{bcjk}\langle \Phi_i^a | \hat{H}|\Phi_{jk}^{bc} \rangle C_{jk}^{bc}+
\sum_{bcdjkl}\langle \Phi_i^a | \hat{H}|\Phi_{jkl}^{bcd} \rangle C_{jkl}^{bcd}=EC_i^a.
$$

## A non-practical way of solving the eigenvalue problem
We see that on the right-hand side we have the energy $E$. This leads to a non-linear equation in the unknown coefficients.
These equations are normally solved iteratively ( that is we can start with a guess for the coefficients $C_i^a$). A common choice is to use perturbation theory for the first guess, setting thereby

$$
C_{i}^{a}=\frac{\langle i | \hat{f}| a\rangle}{\epsilon_i-\epsilon_a}.
$$

The observant reader will however see that we need an equation for $C_{jk}^{bc}$ and $C_{jkl}^{bcd}$ as well.
To find equations for these coefficients we need then to continue our multiplications from the left with the various
$\Phi_{H}^P$ terms.

## A non-practical way of solving the eigenvalue problem
For $C_{jk}^{bc}$ we need then

8

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

$$
\sum_{cdkl}\langle \Phi_{ij}^{ab} | \hat{H} -E|\Phi_{kl}^{cd} \rangle C_{kl}^{cd}+\sum_{cdeklm}\langle \Phi_{ij}^{ab} | \hat{H} -E|\Phi_{klm}^{cde} \rangle C_{klm}^{cde}+\sum_{cdefklmn}\langle \Phi_{ij}^{ab} | \hat{H} -E|\Phi_{klmn}^{cdef} \rangle C_{klmn}^{cdef}=0,
$$

and we can isolate the coefficients $C_{kl}^{cd}$ in a similar way as we did for the coefficients $C_{i}^{a}$.
A standard choice for the first iteration is to set

$$
C_{ij}^{ab} =\frac{\langle ij \vert \hat{v} \vert ab \rangle}{\epsilon_i+\epsilon_j-\epsilon_a-\epsilon_b}.
$$

## A non-practical way of solving the eigenvalue problem
At the end we can rewrite our solution of the Schroedinger equation in terms of $n$ coupled equations for the coefficients $C_H^P$.
This is a very cumbersome way of solving the equation. However, by using this iterative scheme we can illustrate how we can compute the
various terms in the wave operator or correlation operator $\hat{C}$. We will later identify the calculation of the various terms $C_H^P$
as parts of different many-body approximations to full CI. In particular, we can  relate this non-linear scheme with Coupled Cluster theory and
many-body perturbation theory.

## Summarizing FCI and bringing in approximative methods
If we can diagonalize large matrices, FCI is the method of choice since:
* It gives all eigenvalues, ground state and excited states

* The eigenvectors are obtained directly from the coefficients $C_H^P$ which result from the diagonalization

* We can compute easily expectation values of other operators, as well as transition probabilities

* Correlations are easy to understand in terms of contributions to a given operator beyond the Hartree-Fock contribution. This is the standard approach in  many-body theory.

## Summarizing FCI and bringing in approximative methods
The correlation energy is defined as, with a two-body Hamiltonian,

$$
\Delta E=\sum_{ai}\langle i| \hat{f}|a \rangle C_{i}^{a}+
\sum_{abij}\langle ij | \hat{v}| ab \rangle C_{ij}^{ab}.
$$

The coefficients $C$ result from the solution of the eigenvalue problem.
The energy of say the ground state is then

$$
E=E_{ref}+\Delta E,
$$

where the so-called reference energy is the energy we obtain from a Hartree-Fock calculation, that is

$$
E_{ref}=\langle \Phi_0 \vert \hat{H} \vert \Phi_0 \rangle.
$$

## Summarizing FCI and bringing in approximative methods
However, as we have seen, even for a small case like the four first major shells and a nucleus like oxygen-16, the dimensionality becomes quickly intractable. If we wish to include single-particle states that reflect weakly bound systems, we need a much larger single-particle basis. We need thus approximative methods that sum specific correlations to infinite order.

Popular methods are
* [Many-body perturbation theory (in essence a Taylor expansion)](http://www.sciencedirect.com/science/journal/03701573/261/3-4)

* [Coupled cluster theory (coupled non-linear equations)](http://iopscience.iop.org/0034-4885/77/9/096302)

* [Green's function approaches (matrix inversion)](http://www.worldscientific.com/worldscibooks/10.1142/6821)

* [Similarity group transformation methods (coupled ordinary differential equations)](http://journals.aps.org/prc/abstract/10.1103/PhysRevC.85.061304)

All these methods start normally with a Hartree-Fock basis as the calculational basis.

## A quick tour of Coupled Cluster theory

The ansatz for the wavefunction (ground state) is given by

$$
\vert \Psi\rangle = \vert \Psi_{CC}\rangle = e^{\hat{T}} \vert \Phi_0\rangle =
  \left( \sum_{n=1}^{A} \frac{1}{n!} \hat{T}^n \right) \vert \Phi_0\rangle,
$$

where $A$ represents the maximum number of particle-hole excitations and $\hat{T}$ is the cluster operator defined as

$$
\begin{align*}
            \hat{T} &= \hat{T}_1 + \hat{T}_2 + \ldots + \hat{T}_A \\
            \hat{T}_n &= \left(\frac{1}{n!}\right)^2
                \sum_{\substack{
                        i_1,i_2,\ldots i_n \\
                        a_1,a_2,\ldots a_n}}
                t_{i_1i_2\ldots i_n}^{a_1a_2\ldots a_n} a_{a_1}^\dagger a_{a_2}^\dagger \ldots a_{a_n}^\dagger a_{i_n} \ldots a_{i_2} a_{i_1}.
        \end{align*}
$$

## A quick tour of Coupled Cluster theory
The energy is given by

$$
E_{\mathrm{CC}} = \langle\Phi_0\vert  \overline{H}\vert \Phi_0\rangle,
$$

where $\overline{H}$ is a similarity transformed Hamiltonian

$$
\begin{align*}
        \overline{H}&= e^{-\hat{T}} \hat{H}_N e^{\hat{T}} \\
        \hat{H}_N &= \hat{H} - \langle\Phi_0\vert \hat{H} \vert \Phi_0\rangle.
    \end{align*}
$$

## A quick tour of Coupled Cluster theory
The coupled cluster energy is a function of the unknown cluster amplitudes $t_{i_1i_2\ldots i_n}^{a_1a_2\ldots a_n}$,
given by the solutions to the amplitude equations

$$
0 = \langle\Phi_{i_1 \ldots i_n}^{a_1 \ldots a_n}\vert \overline{H}\vert \Phi_0\rangle.
$$

The similarity transformed   Hamiltonian  $\overline{H}$ is expanded using the Baker-Campbell-Hausdorff expression,

$$
\begin{align*}
        \overline{H}&= \hat{H}_N + \left[ \hat{H}_N, \hat{T} \right] +
            \frac{1}{2} \left[\left[ \hat{H}_N, \hat{T} \right], \hat{T}\right] + \ldots \\
            & \quad \frac{1}{n!} \left[ \ldots \left[ \hat{H}_N, \hat{T} \right], \ldots \hat{T} \right] +\dots
    \end{align*}
$$

and simplified using the connected cluster theorem

$$
\overline{H}= \hat{H}_N + \left( \hat{H}_N \hat{T}\right)_c + \frac{1}{2} \left( \hat{H}_N \hat{T}^2\right)_c
            + \dots + \frac{1}{n!} \left( \hat{H}_N \hat{T}^n\right)_c +\dots
$$

## A quick tour of Coupled Cluster theory
A much used approximation is to  truncate the cluster operator $\hat{T}$ at the $n=2$ level. This defines the so-called singes and doubles approximation to the Coupled Cluster wavefunction, normally shortened to CCSD..

The coupled cluster wavefunction is now given by

$$
\vert \Psi_{CC}\rangle = e^{\hat{T}_1 + \hat{T}_2} \vert \Phi_0\rangle
$$

where

$$
\begin{align*}
            \hat{T}_1 &=
            \sum_{ia}
                t_{i}^{a} a_{a}^\dagger a_i \\
            \hat{T}_2 &= \frac{1}{4}
            \sum_{ijab}
                t_{ij}^{ab} a_{a}^\dagger a_{b}^\dagger a_{j} a_{i}.
        \end{align*}
$$

## A quick tour of Coupled Cluster theory
The amplutudes $t$ play a role similar to the coefficients $C$ in the shell-model calculations. They are obtained by solving a set of non-linear equations
similar to those discussed above in connection withe FCI discussion.

If we truncate our equations at the CCSD level, it corresponds to performing a transformation of the Hamiltonian matrix of the following type for a six particle problem (with a two-body Hamiltonian):

       	   $0p-0h$  	   $1p-1h$  	   $2p-2h$  	   $3p-3h$  	   $4p-4h$  	   $5p-5h$  	   $6p-6h$

   $0p-0h$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    0              	    0              	    0              	    0

   $1p-1h$    	    0              	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    0              	    0              	    0

   $2p-2h$    	    0              	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    0              	    0

   $3p-3h$    	    0              	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    0

   $4p-4h$    	    0              	    0              	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$

   $5p-5h$    	    0              	    0              	    0              	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$

   $6p-6h$    	    0              	    0              	    0              	    0              	    $\tilde{x}$    	    $\tilde{x}$    	    $\tilde{x}$

## A quick tour of Coupled Cluster theory
In our FCI discussion the correlation energy is defined as, with a two-body Hamiltonian,

$$
\Delta E=\sum_{ai}\langle i| \hat{f}|a \rangle C_{i}^{a}+
\sum_{abij}\langle ij | \hat{v}| ab \rangle C_{ij}^{ab}.
$$

In Coupled cluster theory it becomes (irrespective of level of truncation of $T$)

$$
\Delta E=\sum_{ai}\langle i| \hat{f}|a \rangle t_{i}^{a}+
\sum_{abij}\langle ij | \hat{v}| ab \rangle t_{ij}^{ab}.
$$

## A quick tour of Coupled Cluster theory
Coupled cluster theory has several interesting computational features and is the method of choice in quantum chemistry. The method was originally proposed by Coester and Kummel, two nuclear physicists (way back in the fifties). It came back in full strength in nuclear physics during the last decade.

There are several interesting features:
* With a truncation like CCSD or CCSDT, we can include to infinite order correlations like $2p-2h$.

* We can include a large basis of single-particle states, not possible in standard FCI calculations

However, Coupled Cluster theory is
* non-variational

* if we want to find properties of excited states, additional calculations via for example equation of motion methods are needed

* if correlations are strong, a single-reference ansatz may not be the best starting point

* we cannot quantify properly the error we make when truncations are made in the cluster operator

## The CCD approximation

We will now approximate the cluster operator $\hat{T}$ to include only $2p-2h$ correlations. This leads to the so-called CCD approximation, that is

$$
\hat{T}\approx \hat{T}_2=\frac{1}{4}\sum_{abij}t_{ij}^{ab}a^{\dagger}_aa^{\dagger}_ba_ja_i,
$$

meaning that we have

$$
\vert \Psi_0 \rangle \approx \vert \Psi_{CCD} \rangle = \exp{\left(\hat{T}_2\right)}\vert \Phi_0\rangle.
$$

## The CCD approximation
Inserting these equations in the expression for the computation of the energy we have,
with a Hamiltonian defined with respect to a general vacuum (see the exercises in the second quantization part)

$$
\hat{H}=\hat{H}_N+E_{\mathrm{ref}},
$$

with

$$
\hat{H}_N=\sum_{pq}\langle p \vert \hat{f} \vert q \rangle  a^{\dagger}_pa_q + \frac{1}{4}\sum_{pqrs}\langle pq \vert \hat{v} \vert rs \rangle a^{\dagger}_pa^{\dagger}_qa_sa_r,
$$

we obtain that the energy can be written as

$$
\langle \Phi_0 \vert \exp{-\left(\hat{T}_2\right)}\hat{H}_N\exp{\left(\hat{T}_2\right)}\vert \Phi_0\rangle =
\langle \Phi_0 \vert \hat{H}_N(1+\hat{T}_2)\vert \Phi_0\rangle = E_{CCD}.
$$

## The CCD approximation
This quantity becomes

$$
E_{CCD}=E_{\mathrm{ref}}+\frac{1}{4}\sum_{abij}\langle ij \vert \hat{v} \vert ab \rangle t_{ij}^{ab},
$$

where the latter is the correlation energy from this level of approximation of CC theory.
Similarly, the expression for the amplitudes reads

$$
\langle \Phi_{ij}^{ab} \vert \exp{-\left(\hat{T}_2\right)}\hat{H}_N\exp{\left(\hat{T}_2\right)}\vert \Phi_0\rangle = 0.
$$

## The CCD approximation
These equations can be reduced to (after several applications of Wick's theorem) to, for all $i > j$ and all $a  > b$,

$$
0 = \langle ab \vert \hat{v} \vert ij \rangle + \left(\epsilon_a+\epsilon_b-\epsilon_i-\epsilon_j\right)t_{ij}^{ab}  \nonumber
$$

$$
+\frac{1}{2}\sum_{cd} \langle ab \vert \hat{v} \vert cd \rangle t_{ij}^{cd}+\frac{1}{2}\sum_{kl} \langle kl \vert \hat{v} \vert ij \rangle t_{kl}^{ab}+\hat{P}(ij\vert ab)\sum_{kc} \langle kb \vert \hat{v} \vert cj \rangle t_{ik}^{ac}  \nonumber
$$

$$
+\frac{1}{4}\sum_{klcd} \langle kl \vert \hat{v} \vert cd \rangle t_{ij}^{cd}t_{kl}^{ab}+\hat{P}(ij)\sum_{klcd} \langle kl \vert \hat{v} \vert cd \rangle t_{ik}^{ac}t_{jl}^{bd} \nonumber
$$

$$
\begin{equation}
-\frac{1}{2}\hat{P}(ij)\sum_{klcd} \langle kl \vert \hat{v} \vert cd \rangle t_{ik}^{dc}t_{lj}^{ab}-\frac{1}{2}\hat{P}(ab)\sum_{klcd} \langle kl \vert \hat{v} \vert cd \rangle t_{lk}^{ac}t_{ij}^{db},
\label{eq:ccd} \tag{1}
\end{equation}
$$

where we have defined

$$
\hat{P}\left(ab\right)= 1-\hat{P}_{ab},
$$

where $\hat{P}_{ab}$ interchanges two particles occupying the quantum numbers $a$ and $b$.

## The CCD approximation
The operator $\hat{P}(ij\vert ab)$  is defined as

$$
\hat{P}(ij\vert ab) = (1-\hat{P}_{ij})(1-\hat{P}_{ab}).
$$

Recall also that the unknown amplitudes $t_{ij}^{ab}$
represent anti-symmetrized matrix elements, meaning that they obey the same symmetry relations as the two-body interaction, that is

$$
t_{ij}^{ab}=-t_{ji}^{ab}=-t_{ij}^{ba}=t_{ji}^{ba}.
$$

The two-body matrix elements are also anti-symmetrized, meaning that

$$
\langle ab \vert \hat{v} \vert ij \rangle = -\langle ab \vert \hat{v} \vert ji \rangle= -\langle ba \vert \hat{v} \vert ij \rangle=\langle ba \vert \hat{v} \vert ji \rangle.
$$

The non-linear equations for the unknown amplitudes  $t_{ij}^{ab}$ are solved iteratively. We discuss the implementation of these equations below.

## Approximations to the full CCD equations

It is useful to make approximations to the equations for the amplitudes. The standard method for solving these equations is to set up an iterative scheme where method's like Newton's method or similar root searching methods are used to find the amplitudes.
Itreative solvers need a guess for the amplitudes. A good starting point is to use the correlated wave operator from perturbation theory to
first order in the interaction.
This means that we define the zeroth approximation to the amplitudes as

$$
t^{(0)}=\frac{\langle ab \vert \hat{v} \vert ij \rangle}{\left(\epsilon_i+\epsilon_j-\epsilon_a-\epsilon_b\right)},
$$

leading to our first approximation for the correlation energy at the CCD level to be equal to second-order perturbation theory without $1p-1h$ excitations, namely

$$
\Delta E_{\mathrm{CCD}}^{(0)}=\frac{1}{4}\sum_{abij} \frac{\langle ij \vert \hat{v} \vert ab \rangle \langle ab \vert \hat{v} \vert ij \rangle}{\left(\epsilon_i+\epsilon_j-\epsilon_a-\epsilon_b\right)}.
$$

## Approximations to the full CCD equations
With this starting point, we are now ready to solve Eq. ([eq:ccd](#eq:ccd)) iteratively. Before we attack the full equations, it is however instructive to study a truncated version of the equations. We will first study the following approximation where we take away all terms except the linear terms that involve the single-particle energies and the the two-particle intermediate excitations, that is

$$
\begin{equation}
0 = \langle ab \vert \hat{v} \vert ij \rangle + \left(\epsilon_a+\epsilon_b-\epsilon_i-\epsilon_j\right)t_{ij}^{ab}+\frac{1}{2}\sum_{cd} \langle ab \vert \hat{v} \vert cd \rangle t_{ij}^{cd}.
\label{eq:ccd1} \tag{2}
\end{equation}
$$

## Approximations to the full CCD equations
Setting the single-particle energies for the hole states equal to an energy variable $\omega = \epsilon_i+\epsilon_j$, Eq. ([eq:ccd1](#eq:ccd1)) reduces to the
well-known equations for the so-called $G$-matrix, widely used in [infinite matter and finite nuclei studies](http://www.sciencedirect.com/science/journal/03701573/261/3-4). The equation can then be reordered and solved by matrix inversion.  To see this let us define the following quantity

$$
\tau_{ij}^{ab}= \left(\omega-\epsilon_a-\epsilon_b\right)t_{ij}^{ab},
$$

and inserting

$$
1=\frac{\left(\omega-\epsilon_c-\epsilon_d\right)}{\left(\omega-\epsilon_c-\epsilon_d\right)},
$$

in the intermediate sums over $cd$ in Eq. ([eq:ccd1](#eq:ccd1)), we can rewrite the latter equation as

$$
\tau_{ij}^{ab}(\omega)= \langle ab \vert \hat{v} \vert ij \rangle + \frac{1}{2}\sum_{cd} \langle ab \vert \hat{v} \vert cd \rangle \frac{1}{\omega-\epsilon_c-\epsilon_d}\tau_{ij}^{cd}(\omega),
$$

where we have indicated an explicit energy dependence. This equation, transforming a two-particle configuration into a single index, can be transformed into a matrix inversion problem.  Solving the equations for a fixed energy $\omega$ allows us to compare directly with results from Green's function theory when only two-particle intermediate states are included.

## Approximations to the full CCD equations
To solve Eq. ([eq:ccd1](#eq:ccd1)), we would thus start with a guess for the unknown amplitudes, typically using the wave operator defined by first order in perturbation theory, leading to a zeroth approximation to the energy given by second-order perturbation theory for the correlation energy.
A simple approach to the solution of  Eq. ([eq:ccd1](#eq:ccd1)), is to thus to
1. Start with a guess for the amplitudes and compute the zeroth approximation to the correlation energy

2. Use the ansatz for the amplitudes to solve Eq. ([eq:ccd1](#eq:ccd1)) via for example your root-finding method of choice (Newton's method or modifications thereof can be used) and continue these iterations till the correlation energy does not change more than a prefixed quantity $\lambda$; $\Delta E_{\mathrm{CCD}}^{(i)}-\Delta E_{\mathrm{CCD}}^{(i-1)} \le \lambda$.

3. It is common during the iterations to scale the amplitudes with a parameter $\alpha$, with $\alpha \in (0,1]$ as  $t^{(i)}=\alpha t^{(i)}+(1-\alpha)t^{(i-1)}$.

## Approximations to the full CCD equations
The next approximation is to include the two-hole term in Eq. ([eq:ccd](#eq:ccd)), a term which allow us to make a link with Green's function theory with two-particle and two-hole correlations. This means that we solve

$$
\begin{equation}
0 = \langle ab \vert \hat{v} \vert ij \rangle + \left(\epsilon_a+\epsilon_b-\epsilon_i-\epsilon_j\right)t_{ij}^{ab}+\frac{1}{2}\sum_{cd} \langle ab \vert \hat{v} \vert cd \rangle t_{ij}^{cd}+\frac{1}{2}\sum_{kl} \langle kl \vert \hat{v} \vert ij \rangle t_{kl}^{ab}.
\label{eq:ccd2} \tag{3}
\end{equation}
$$

This equation is solved the same way as we would do for Eq. ([eq:ccd1](#eq:ccd1)). The final step is then to include all terms in Eq. ([eq:ccd](#eq:ccd)).

## Introduction to studies of infinite matter

Studies of infinite nuclear matter play an important role  in nuclear physics. The aim of this part of the lectures is to provide the necessary ingredients for perfoming studies of neutron star matter (or matter in $\beta$-equilibrium) and symmetric nuclear matter.

Here we will study infinite neutron matter
* at the Hartree-Fock with realistic nuclear forces and

* using many-body methods like coupled-cluster theory or many-body perturbation theory

## Infinite nuclear matter and neutron star matter

Studies of dense baryonic matter are of central importance to our basic understanding
of the stability of nuclear matter, spanning from matter at high densities and temperatures
to matter as found within dense astronomical objects  like neutron stars.

Neutron star matter
at densities of  0.1 fm$^{-3}$ and greater, is often assumed to
be made of mainly neutrons, protons, electrons and
muons in beta equilibrium. However, other baryons like various hyperons may exist, as well as possible mesonic condensates and transitions to quark degrees of freedom at higher densities.
Here we focus on specific definitions of various phases and focus
on distinct phases of matter such as pure baryonic
matter and/or quark matter.
The composition of matter is then
determined by the requirements of chemical and electrical equilibrium.
Furthermore, we will also consider matter at temperatures much lower
than the typical Fermi energies.

## Properties of infinite nuclear matter

The equilibrium conditions are governed by the weak processes
(normally referred to as the processes
for $\beta$-equilibrium)

$$
\begin{equation}
      b_1 \rightarrow b_2 + l +\bar{\nu}_l \hspace{1cm} b_2 +l \rightarrow b_1
+\nu_l,
\label{eq:betadecay} \tag{4}
\end{equation}
$$

where $b_1$ and $b_2$ refer to for example the baryons being a neutron and a proton,
respectively,
$l$ is either an electron or a muon and  $\bar{\nu}_l $
and $\nu_l$ their respective anti-neutrinos and neutrinos. Muons typically
appear at
a density close to nuclear matter saturation density, the latter being

$$
n_0 \approx 0.16 \pm 0.02 \hspace{1cm} \mathrm{fm}^{-3},
$$

with a corresponding binding energy $E_0$
for symmetric nuclear matter (SNM) at saturation density of

$$
E_0 = B/A=-15.6\pm 0.2 \hspace{1cm} \mathrm{MeV}.
$$

## The infinite neutron gas as a homogenous system

This is a homogeneous system and the one-particle wave functions are given by plane wave functions normalized to a volume $\Omega$
for a box with length $L$ (the limit $L\rightarrow \infty$ is to be taken after we have computed various expectation values)

$$
\psi_{\mathbf{k}\sigma}(\mathbf{r})= \frac{1}{\sqrt{\Omega}}\exp{(i\mathbf{kr})}\xi_{\sigma}
$$

where $\mathbf{k}$ is the wave number and  $\xi_{\sigma}$ is a spin function for either spin up or down

$$
\xi_{\sigma=+1/2}=\left(\begin{array}{c} 1 \\ 0 \end{array}\right) \hspace{0.5cm}
\xi_{\sigma=-1/2}=\left(\begin{array}{c} 0 \\ 1 \end{array}\right).
$$

## Periodic boundary conditions and single-particle states

When using periodic boundary conditions, the
discrete-momentum single-particle basis functions

$$
\phi_{\mathbf{k}}(\mathbf{r}) =
e^{i\mathbf{k}\cdot \mathbf{r}}/L^{d/2}
$$

are associated with
the single-particle energy

$$
\begin{equation}
  \varepsilon_{n_{x}, n_{y}} = \frac{\hbar^{2}}{2m} \left( \frac{2\pi }{L}\right)^{2}\left( n_{x}^{2} + n_{y}^{2}\right)
\label{_auto1} \tag{5}
\end{equation}
$$

for two-dimensional sytems and

$$
\begin{equation}
  \varepsilon_{n_{x}, n_{y}, n_{z}} = \frac{\hbar^{2}}{2m}
  \left( \frac{2\pi }{L}\right)^{2}
  \left( n_{x}^{2} + n_{y}^{2} + n_{z}^{2}\right)
\label{_auto2} \tag{6}
\end{equation}
$$

for three-dimensional systems.

## More on periodic boundary conditions and single-particle states

The table on the next slide illustrates  how single-particle energies
    fill energy shells in a two-dimensional neutron box.
  Here $n_{x}$ and $n_{y}$ are the momentum quantum numbers,
  $n_{x}^{2} + n_{y}^{2}$ determines the single-particle
  energy level, $N_{\uparrow \downarrow }$ represents the
  cumulated number of spin-orbitals in an unpolarized spin
  phase, and $N_{\uparrow \uparrow }$ stands for the
  cumulated number of spin-orbitals in a spin-polarized
  system.

## Magic numbers for the two-dimensional neutron (or electron)  gas

$n_{x}^{2}+n_{y}^{2}$	 $n_{x}$	 $n_{y}$	 $N_{\uparrow \downarrow }$	 $N_{\uparrow \uparrow }$

   0                        	    0          	    0          	    2                             	    1

   1                        	    -1         	    0

                            	    1          	    0

                            	    0          	    -1

                            	    0          	    1          	    10                            	    5

   2                        	    -1         	    -1

                            	    -1         	    1

                            	    1          	    -1

                            	    1          	    1          	    18                            	    9

   4                        	    -2         	    0

                            	    2          	    0

                            	    0          	    -2

                            	    0          	    2          	    26                            	    13

   5                        	    -2         	    -1

                            	    2          	    -1

                            	    -2         	    1

                            	    2          	    1

                            	    -1         	    -2

                            	    -1         	    2

                            	    1          	    -2

                            	    1          	    2          	    42                            	    21

## Three-dimensional neutron gas
Using the same approach as made with the two-dimensional electron gas with the single-particle kinetic energy defined as

$$
\frac{\hbar^2}{2m}\left(k_{n_x}^2+k_{n_y}^2k_{n_z}^2\right),
$$

and

$$
k_{n_i}=\frac{2\pi n_i}{L} \hspace{0.1cm} n_i = 0, \pm 1, \pm 2, \dots,
$$

we can set up a similar table and obtain (assuming identical particles one and including spin up and spin down solutions)  for energies less than or equal to $n_{x}^{2}+n_{y}^{2}+n_{z}^{2}\le 3$

## Single-particle states for the three-dimensional neutron gas

$n_{x}^{2}+n_{y}^{2}+n_{z}^{2}$	 $n_{x}$	 $n_{y}$	 $n_{z}$	 $N_{\uparrow \downarrow }$

   0                                  	    0          	    0          	    0          	    2

   1                                  	    -1         	    0          	    0

   1                                  	    1          	    0          	    0

   1                                  	    0          	    -1         	    0

   1                                  	    0          	    1          	    0

   1                                  	    0          	    0          	    -1

   1                                  	    0          	    0          	    1          	    14

   2                                  	    -1         	    -1         	    0

   2                                  	    -1         	    1          	    0

   2                                  	    1          	    -1         	    0

   2                                  	    1          	    1          	    0

   2                                  	    -1         	    0          	    -1

   2                                  	    -1         	    0          	    1

   2                                  	    1          	    0          	    -1

   2                                  	    1          	    0          	    1

   2                                  	    0          	    -1         	    -1

   2                                  	    0          	    -1         	    1

   2                                  	    0          	    1          	    -1

   2                                  	    0          	    1          	    1          	    38

   3                                  	    -1         	    -1         	    -1

   3                                  	    -1         	    -1         	    1

   3                                  	    -1         	    1          	    -1

   3                                  	    -1         	    1          	    1

   3                                  	    1          	    -1         	    -1

   3                                  	    1          	    -1         	    1

   3                                  	    1          	    1          	    -1

   3                                  	    1          	    1          	    1          	    54

Continuing in this way we get for $n_{x}^{2}+n_{y}^{2}+n_{z}^{2}=4$  a total of 12 additional states, resulting in $?$ as a new magic number. We can continue like this by adding more shells.

When performing calculations based on many-body perturbation theory, Coupled cluster theory or other many-body methods, we need then to add states above the Fermi level in order to sum over single-particle states which are not occupied.

## Input parameters
Every number of particles for filled shells defines also the number of particles to be used in a given calculation. We use the number of particles to define  the density of the system

$$
\rho = g \frac{k_F^3}{6\pi^2},
$$

where you need to define $k_F$ and the degeneracy $g$, which is two for one type of spin-$1/2$ particles and four for symmetric nuclear matter.

With  the density we can define the length $L$ of the box used with periodic boundary contributions, that is use the relation

$$
V= L^3= \frac{A}{\rho}.
$$

Finally we can use $L$ to define the spacing to set up the spacing between varipus $k$-values, that is

$$
\Delta k = \frac{2\pi}{L}.
$$

Here, $A$ can be the number of nucleons.

## Potential model employed in code development

The interaction we will use for these calculations is a semirealistic
nucleon-nucleon potential known as the Minnesota potential

$$
V_{\alpha}\left(r\right)=V_{\alpha}\exp{(-\alpha r^{2})}.
$$

The spin and isospin
dependence of the Minnesota potential is given by

$$
\begin{equation}
V\left( r\right)=\frac{1}{2}\left( V_{R}+\frac{1}{2}\left(1+P_{12}^{\sigma}\right) V_{T}+\frac{1}{2}\left(1-P_{12}^{\sigma}\right) V_{S}\right)\left(1-P_{12}^{\sigma}P_{12}^{\tau}\right),
\label{_auto3} \tag{7}
\end{equation}
$$

where

$$
P_{12}^{\sigma}=\frac{1}{2}\left(1+\sigma_{1}\cdot\sigma_{2}\right),
$$

and

$$
P_{12}^{\tau}=\frac{1}{2}\left( 1+\tau_{1}\cdot\tau_{2}\right)
$$

are
the spin and isospin exchange operators, respectively.

## Fourier transform
A Fourier
transform to momentum space of the radial part

$$
V_{\alpha}\left(r\right)
$$

is rather simple since
the radial depends only on the magnitude of the relative distance and
thereby the relative momentum

$$
\mathbf{q}=\frac{1}{2}\left(\mathbf{k}_{p}-\mathbf{k}_{q}-\mathbf{k}_{r}+\mathbf{k}_{s}\right)
$$

Omitting spin and isospin dependencies, the momentum space version of the
interaction reads

$$
\begin{equation}
\langle \mathbf{k}_p \mathbf{k}_q \vert V_{\alpha}\vert\mathbf{k}_r\mathbf{k}_s\rangle=
\frac{V_{\alpha}}{L^{3}}\left(\frac{\pi}{\alpha}\right)^{3/2}\exp{(\frac{-q^{2}}{4\alpha})}\delta_{\mathbf{k}_{p}+\mathbf{k}_{q},\mathbf{k}_{r}+\mathbf{k}_{s}}
\label{_auto4} \tag{8}
\end{equation}
$$

## Developing a program for infinite matter

* Structure a code in terms of functions.

* Modularize your codes.

* Be able to  read input data flexibly.

* Write unit tests (test functions) and let your code undergo heavy testing.

* Refactor code in terms of classes (instead of functions only).

* Conduct and automate large-scale numerical experiments.

* New code is added in a modular fashion to a library (modules).

* Programs are run through convenient user interfaces.

* Use scripts in order to automatize  tedious manual work.

* Make sure your scientific investigations are reproducible and document properly your results.

* Use version control software like for example **git**

## Codes and reading material

* The codes are all available from the coming Lectures Notes in Physics volume [An advanced course in computational nuclear physics: Bridging the scales from quarks to neutron stars, M. Hjorth-Jensen, M. P. Lombardo, U. van Kolck, Editors](https://github.com/ManyBodyPhysics/LectureNotesPhysics/tree/master/doc/src/Chapter8-programs)

* For more reading on CCD see chapter 8 of the coming Lectures Notes in Physics volume [An advanced course in computational nuclear physics: Bridging the scales from quarks to neutron stars, M. Hjorth-Jensen, M. P. Lombardo, U. van Kolck, Editors](https://github.com/ManyBodyPhysics/LectureNotesPhysics/tree/master/doc/src/lnp.pdf)

## The CCD equation
The CCD equations can be written as

$$
\left(\epsilon_i+\epsilon_j-\epsilon_a-\epsilon_b\right)t_{ij}^{ab}
  = \langle ab \vert \hat{v} \vert ij \rangle  \nonumber
$$

$$
+\frac{1}{2}\sum_{cd} \langle ab \vert \hat{v} \vert cd \rangle
  t_{ij}^{cd}+\frac{1}{2}\sum_{kl} \langle kl \vert \hat{v} \vert ij
  \rangle t_{kl}^{ab}+\hat{P}(ij\vert ab)\sum_{kc} \langle kb \vert
  \hat{v} \vert cj \rangle t_{ik}^{ac}  \nonumber
$$

$$
+\frac{1}{4}\sum_{klcd} \langle kl \vert \hat{v} \vert cd \rangle
  t_{ij}^{cd}t_{kl}^{ab}+\hat{P}(ij)\sum_{klcd} \langle kl \vert
  \hat{v} \vert cd \rangle t_{ik}^{ac}t_{jl}^{bd} \nonumber
$$

$$
\begin{equation}  -\frac{1}{2}\hat{P}(ij)\sum_{klcd} \langle kl \vert \hat{v} \vert
  cd \rangle t_{ik}^{dc}t_{lj}^{ab}-\frac{1}{2}\hat{P}(ab)\sum_{klcd}
  \langle kl \vert \hat{v} \vert cd \rangle t_{lk}^{ac}t_{ij}^{db},
\label{_auto5} \tag{9}
\end{equation}
$$

for all $i < j$ and all $a < b$, using the standard notation that
  $a,b,...$ are particle states and $i,j,...$ are hole states. With
  the CCD correlation energy given by

$$
\begin{equation}
  \Delta E_{CCD} = \frac{1}{4} \sum_{ijab}
  \langle ij \vert\hat{v}\vert ab\rangle t^{ab}_{ij}.
\label{eq:ccdcorr} \tag{10}
  \end{equation}
$$

## Solving the CCD equations
  One way to solve these equations, is to write equation
  ([eq:ccd2](#eq:ccd2)) as a series of iterative nonlinear algebraic equations

$$
t_{ij}^{ab}{}^{(n+1)} = \frac{1}{\epsilon^{ab}_{ij}} \bigg(\langle
  ab \vert \hat{v} \vert ij \rangle  \nonumber
$$

$$
+\frac{1}{2}\sum_{cd} \langle ab \vert \hat{v} \vert cd \rangle
  t_{ij}^{cd}{}^{(n)}+\frac{1}{2}\sum_{kl} \langle kl \vert \hat{v}
  \vert ij \rangle t_{kl}^{ab}{}^{(n)}+\hat{P}(ij\vert ab)\sum_{kc}
  \langle kb \vert \hat{v} \vert cj \rangle t_{ik}^{ac}{}^{(n)}
  \nonumber
$$

$$
+\frac{1}{4}\sum_{klcd} \langle kl \vert \hat{v} \vert
  cd \rangle
  t_{ij}^{cd}{}^{(n)}t_{kl}^{ab}{}^{(n)}+\hat{P}(ij)\sum_{klcd}
  \langle kl \vert \hat{v} \vert cd \rangle
  t_{ik}^{ac}{}^{(n)}t_{jl}^{bd}{}^{(n)} \nonumber
$$

$$
\begin{equation}  -\frac{1}{2}\hat{P}(ij)\sum_{klcd} \langle kl \vert \hat{v} \vert
  cd \rangle
  t_{ik}^{dc}{}^{(n)}t_{lj}^{ab}{}^{(n)}-\frac{1}{2}\hat{P}(ab)\sum_{klcd}
  \langle kl \vert \hat{v} \vert cd \rangle
  t_{lk}^{ac}{}^{(n)}t_{ij}^{db}{}^{(n)} \bigg),
\label{eq:ccd3} \tag{11}
  \end{equation}
$$

for all $i < j$ and all $a < b$, where $\epsilon^{ab}_{ij} =\left(\epsilon_i+\epsilon_j-\epsilon_a-\epsilon_b\right)$, and
  $t_{ij}^{ab}{}^{(n)}$ is the $t$ amplitude for the nth iteration of
  the series. This way, given some starting guess
  $t_{ij}^{ab}{}^{(0)}$, we can generate subsequent $t$ amplitudes
  that converges to some value.

## Memory considerations

  Care should thus be placed into how we store these objects. These are
  objects with four indices and a  sensible first implementation
  of the CCD equations would be to create two four-dimensional arrays to store the
  objects. However, it is often more convenient to work with simple
  one-dimensional arrays instead.

   The goal of our code is to calculate the correlation energy,
   $\Delta E_{CCD}$, meaning that after each iteration of our equations, we use
   our newest $t$ amplitudes to update the correlation energy

$$
\begin{equation}
  \Delta E_{CCD}^{(n)} = \frac{1}{4} \sum_{ijab}
  \langle ij\vert\hat{v}\vert ab\rangle t^{ab}_{ij}{}^{(n)}.
\label{_auto6} \tag{12}
\end{equation}
$$

We check that our result is converged by testing whether the
  most recent iteration has changed the correlation energy by less
  than some tolerance threshold $\eta$,

$$
\begin{equation}
  \eta > | \Delta E_{CCD}^{(n+1)} - \Delta E_{CCD}^{(n)} |.
\label{_auto7} \tag{13}
\end{equation}
$$

## More on memory

  One limitation that will be ran into while trying to do realistic
  CCD calculations is that of memory. The four-indexed two-body matrix elements (TBMEs) and
  $t$-amplitudes have to store a lot of elements, and the size of these
  arrays can quickly exceed the available memory on
  a machine. If a calculation wants to use 500 single-particle basis states, then
  a structure like $\langle pq\vert v\vert rs\rangle $ will need a  length of 500 for each of
  its four indices, which means it will have $500^4 = 625\times 10^8$
  elements. To get a handle on how much memory is used, consider the
  elements as double-precision floating point type. One double takes
  up 8 bytes of memory. This specific array would take up $8\times 625\times 10^8$ bytes
  = $5000 \times 10^8$ bytes = $500$ Gbytes of memory.

## Using symmetries

Most personal
  computers in 2016 have 4-8 Gbytes of RAM, meaning that this calculation would
  be way out of reach. There are supercomputers that can handle
  applications using 500 Gbytes of memory, but we can quickly reduce
  the total memory required by applying some physical arguments. In
  addition to vanishing elements with repeated indices, mentioned
  above, elements that do not obey certain symmetries are also
  zero. Almost all realistic two-body forces preserve some quantities
  due to symmetries in the interaction. For example, an interaction
  with rotational symmetry will conserve angular momentum. This means
  that a two-body ket state $\vert rs\rangle $, which has some set of quantum
  numbers, will retain quantum numbers corresponding to the
  interaction symmetries after being acted on by $\hat{v}$. This state
  is then projected onto $\vert pq\rangle $ with its own set of quantum
  numbers. Thus $\langle pq|v|rs\rangle $ is only non-zero if $\vert pq\rangle$ and
  $\vert rs\rangle $ share the same quantum numbers that are preserved by
  $\hat{v}$. In addition, because the cluster operators represent
  excitations due to the interaction, $t_{ij}^{ab}$ is only non-zero
  if $\vert ij\rangle $ has the same relevant quantum numbers as $\vert ab\rangle $.

## Using symmetries II

  To take advantage of this, these two-body ket states can be
  organized into "channels" of shared quantum numbers. In the case
  of the pairing model, the interaction preserves the total spin
  projection of a two-body state, $S_{z}=s_{z1}+s_{z2}$. The single
  particle states can have spin of +1/2 or -1/2, so there can be three
  two-body channels with $S_{z}=-1,0,+1$. These channels can then be
  indexed with a unique label in a similar way to the single particle
  index scheme. In more complicated systems, there will be many more
  channels involving multiple symmetries, so it is useful to create a
  data structure that stores the relevant two-body quantum numbers to
  keep track of the labeling scheme.

## Using symmetries III

  It is more efficient to use two-dimensional array data
  structures, where the first index refers to the channel number and
  the second refers to the element within that channel. So to access
  matrix elements or $t$ amplitudes, you can loop over the channels
  first, then the indices within that channel. To get an idea of the
  savings using this block diagonal structure, let's look at a case
  with a plane wave basis, with three momentum and one spin quantum
  numbers, with an interaction that conserves linear momentum in all
  three dimensions, as well as the total spin projection. Using 502
  basis states, the TBME's require about 0.23 Gb of memory in block
  diagonal form, which is an enormous saving from the 500 Gb mentioned
  earlier in the na\"ive storage scheme.

## Using intermediates

  Since the calculation of all  zeros can now be avoided,
  improvements in speed and memory will now follow. To get a handle on
  how these CCD calculations are implemented we need only to look at the
  most expensive sum in equation ([eq:ccd2](#eq:ccd2)). This corresponds to
  the sum over $klcd$. Since this sum is repeated for all $i < j$ and
  $a < b$, it means that these equations will scale as
  $\mathcal{O}(n_{p}^{4} n_{h}^{4})$. However,
  they can be rewritten using intermediates as

$$
0 = \langle ab|\hat{v}|ij \rangle + \hat{P}(ab) \sum_{c} \langle b| \chi
    |c\rangle \langle ac| t |ij\rangle - \hat{P}(ij) \sum_{k} \langle k| \chi |j\rangle
  \langle ab| t |ik\rangle  \nonumber
$$

$$
\begin{equation}  + \frac{1}{2}\sum_{cd} \langle ab|
    \chi |cd\rangle \langle cd| t |ij\rangle + \frac{1}{2} \sum_{kl} \langle ab| t
    |kl\rangle \langle kl| \chi |ij\rangle
\label{_auto8} \tag{14}
\end{equation}
$$

$$
+ \hat{P}(ij)\hat{P}(ab) \sum_{kc}
  \langle ac| t |ik\rangle\langle kb| \chi |cj\rangle  \nonumber
$$

for all $i,j,a,b$.

## Defining intermediates

The intermediates $\chi$ are defined as

8
1

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

8
2

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

8
3

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

8
4

<
<
<
!
!
M
A
T
H
_
B
L
O
C
K

$$
\begin{equation}
  \langle ab| \chi |cd\rangle = \langle ab|v|cd\rangle
\label{_auto12} \tag{19}
\end{equation}
$$

With the introduction of the above intermediates, the CCD equations scale now as $\mathcal{O}(n_{h}^{2}n_{p}^{4})$.

## Speed up

To  further speed up these computations, we see that these sums can be
  written in terms of  matrix-matrix multiplications. It is not obvious how to
  write all of these sums in such a way, but it is useful to first
  recall that the expression for the multiplication of two matrices $\hat{C} =\hat{A}\times \hat{B}$ can be written as

$$
\begin{equation}
  C_{ij} = \sum_{k} A_{ik} \times B_{kj}.
\label{_auto13} \tag{20}
\end{equation}
$$

We observe then  that equation ([eq:mtxEx](#eq:mtxEx)) can be written as
  \[
  \langle K| \chi |I\rangle = \langle K|v|I\rangle  + \frac{1}{2} \sum_{C}
  \langle K|v|C\rangle \langle C|t|I\rangle
  \]
  by mapping the two index pairs $kl \to K, ij \to I, cd \to C$. The sum looks now
like a matrix-matrix multiplication. This is
  useful because there are packages like BLAS (Basic Linear Algebra
  Subprograms) which have extremely fast implementations of
  matrix-matrix multiplication.

## Testing brute force against block structures for MBPT(2)

MBPT2 contribution to the correlation for pure neutron matter with $N=14$ neutrons and periodic boundary conditions. Up to approximately 1600 single-particle states have been included in the sums over intermediate states.

## Convergence properties for pure neutron matter I

Energy per particle of pure neutron matter computed in the CCD approximation with the Minnesota potential for different numbers of particles with $\mathrm{N_{max}=20}$.

## Convergence properties for pure neutron matter II

Energy per particle of pure neutron matter computed in the CCD approximation with the Minnesota potential for different model space sizes with $\mathrm{A=114}$.

## Comparing CCD with Monte Carlo

CCD, Reference energy and Diffusion Monte Carlo results for pure neutron matter with $66$ neutrons and $\mathrm{N_{max}=36}$.


# ***InfoSelect*** - Mutual Information Based Feature Selection in Python

### *Felipe Maia Polo (felipemaiapolo), Felipe Leno da Silva (f-leno)*

## Contents
1. [ Introduction ](#1)
2. [ Installing *InfoSelect*  ](#2)
3. [ Main functionalities of *InfoSelect* ](#3)
4. [ Examples of *InfoSelect* use ](#4)
5. [ References ](#5)

--------------

## 1\. Introduction

In this package we implement the ideas proposed by [1, 2] in order to make variable/feature selection prior to regression and classification tasks using Gaussian Mixture Models (GMMs) to estimate the Mutual Information between labels and features. This is an efficient and well-performing alternative and was used in a recent work [3] by one of us.

If you use our package in your research, you can cite it as follows:

    @misc{polo2020infoselect,
      title={InfoSelect - Mutual Information Based Feature Selection in Python},
      author={Polo, Felipe Maia and Da Silva, Felipe Leno},
      journal={GitHub: github.com/felipemaiapolo/infoselect},
      year={2020}
    }

--------------

## 2\. Installing *InfoSelect*

You can install the package from
[PyPI](https://pypi.org/project/infoselect/)

```python
!pip install infoselect
```

    Requirement already satisfied: infoselect in /usr/local/lib/python3.6/dist-packages (1.0.0)
    Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from infoselect) (1.1.4)
    Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from infoselect) (3.2.2)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from infoselect) (1.4.1)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from infoselect) (1.18.5)
    Requirement already satisfied: sklearn in /usr/local/lib/python3.6/dist-packages (from infoselect) (0.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->infoselect) (2018.9)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas->infoselect) (2.8.1)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->infoselect) (0.10.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->infoselect) (2.4.7)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->infoselect) (1.3.1)
    Requirement already satisfied: scikit-learn in /usr/local/lib/python3.6/dist-packages (from sklearn->infoselect) (0.22.2.post1)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.7.3->pandas->infoselect) (1.15.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sklearn->infoselect) (0.17.0)

## 3\. Main functionalities of *InfoSelect*

### 3.1\. Main Class `SelectVars`

This class is used to order features/variables according to their importance and making the selection itself. Next we detail its methods:

1. `__init__(self, gmm, selection_mode = 'forward')`
    - **gmm**:
        - If  is *non*-categorical: a [Scikit-Learn GMM](https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html) fitted in (y,X) - y should always be in the first column;
        - If  is categorical: a Python dictionary containing one [Scikit-Learn GMM](https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html) fitted in X conditional on each category - something like X[y==c,:]. Format `{0:gmm0, 1:gmm1, ..., C:gmmC}`;
        - PS: the GMMs must be `covariance_type='full'` at the current *InfoSelect* version.
    - **selection_mode**: `forward`/`backward` algorithms.
        - `forward` selection: we start with an empty set of features and then select the feature that has the largest estimated mutual information with the target variable and. At each subsequent step, we select the feature that marginally maximizes the estimated mutual information of the target and all the chosen features so far. We stop when we have selected/ordered all the features;
        - `backward` elimination: we start with the full set of features and then at each step, we eliminate the feature that marginally maximizes the estimated mutual information of the target and all the remaining features. We stop when we have no more features to eliminate;

2. `fit(self, X, y, verbose=True, eps=0)`
    - **X**: numpy array of features;
    - **y**: numpy array of labels;
    - **verbose**: print or not to print!?
    - **eps**: small value so we can avoid taking log of zero in some cases .

3. `get_info(self)`:
    - This function creates and outputs a Pandas DataFrame with the history of feature selection/elimination. The `mi_mean` column gives the estimated Mutual Information while `mi_error` gives the standard error of that estimate. On the other hand, the `delta` column gives us the percentual information loss/gain in that round, relatively to the latter;

4. `plot_delta(self)`:
    - This function plots the history of percentual changes in the mutual information.

5. `plot_mi(self)`:
    - This function plots the history of the mutual information.

6. `transform(self, X, rd)`:
    - This function takes **X** and transforms it in **X_new**, maintaining the features of Round `rd`;

### 3.2\. Auxiliary Function `get_gmm`

1. `get_gmm(X, y, y_cat=False, num_comps=[2,5,10,15,20], val_size=0.33, reg_covar=1e-06, random_state=42)`:

    - Firstly, this function validate the number of GMM components, for each model it will train, in a holdout set using the mean log likelihood of samples in that set. If Y is non-categorical, it returns a [Scikit-Learn GMM](https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html) fitted in (y,X) model (in this order). On the other hand, if Y is categorical it returns a Python dictionary containing one [Scikit-Learn GMM](https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html) fitted in X conditional on each category - something like X[y==c,:]. Format `{0:gmm0, 1:gmm1, ..., C:gmmC}`.

        - **X**: numpy array of features;
        - **y**: numpy array of labels;
        - **y_cat**: if we should consider Y as categorical;
        - **num_comps**: numbers of GMM components to be validated;
        - **val_size**: size of holdout set used to validate the GMMs numbers of components;
        - **reg_covar**: non-negative regularization added to the diagonal of covariance. Ensures the covariance matrices are non-singular.
        - **random_state**: seed.

--------------------

## 4\. Examples of *InfoSelect* use

Loading Packages:

```python
import infoselect as inf
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
```

### 1.1\. Dataset

We generate a dataset $D$ sampled from $\mathcal{D}=\{(X_{0,i},...,X_{6,i},Y_i)\}_{i=1}^{n}$ similar to the one in [here](https://www.cs.toronto.edu/~delve/data/add10/desc.html), in which $Y_i$ is given by

\begin{align}
Y_i &= f(X_{0,i},...,X_{6,i}) + \epsilon_i \\[.5em]
&=10\cdot \sin(\pi X_{0,i}  X_{1,i}) + 20 (X_{2,i}-0.5)^2 + 10 X_{3,i} + 5 X_{4,i} + \epsilon_i
\end{align}

Where $X_{0,i},...,X_{6,i} \overset{iid}{\sim} U[0,1]$ and $\epsilon_i \sim N(0,1)$ independent from all the other random variables for all $i\in [n]$. In the following we set $n=20000$:

```python
def f(X,e): return 10*np.sin(np.pi*X[:,0]*X[:,1]) + 20*(X[:,2]-.5)**2 + 10*X[:,3] + 5*X[:,4] + e
```

```python
n=20000
d=7

X = np.random.uniform(0,1,d*n).reshape((n,d))
e = np.random.normal(0,1,n)
y = f(X,e)

X.shape, y.shape
```

    ((20000, 7), (20000,))

### 1.2\. Selecting Features for a Regression Task

Training (and validating) GMM:

```python
%%time

gmm = inf.get_gmm(X, y)
```

    CPU times: user 27.5 s, sys: 18.2 s, total: 45.7 s
    Wall time: 23.4 s

```python
gmm
```

    GaussianMixture(covariance_type='full', init_params='kmeans', max_iter=100,
                    means_init=None, n_components=20, n_init=1,
                    precisions_init=None, random_state=42, reg_covar=1e-06,
                    tol=0.001, verbose=0, verbose_interval=10, warm_start=False,
                    weights_init=None)

Ordering features by their importances using the *Backward Elimination* algorithm:

```python
select = inf.SelectVars(gmm, selection_mode = 'backward')
select.fit(X, y, verbose=True)
```

    Let's start...

    Round =   0   |   Î =  1.48   |   Δ%Î =  0.00   |   Features=[0, 1, 2, 3, 4, 5, 6]
    Round =   1   |   Î =  1.48   |   Δ%Î = -0.00   |   Features=[0, 1, 2, 3, 4, 5]
    Round =   2   |   Î =  1.48   |   Δ%Î = -0.00   |   Features=[0, 1, 2, 3, 4]
    Round =   3   |   Î =  1.00   |   Δ%Î = -0.32   |   Features=[0, 1, 3, 4]
    Round =   4   |   Î =  0.75   |   Δ%Î = -0.25   |   Features=[0, 1, 3]
    Round =   5   |   Î =  0.39   |   Δ%Î = -0.48   |   Features=[1, 3]
    Round =   6   |   Î =  0.21   |   Δ%Î = -0.46   |   Features=[3]

Checking history:

```python
select.get_info()
```

      rounds
      mi_mean
      mi_error
      delta
      num_feat
      features

      0
      0
      1.483969
      0.006147
      0.000000
      7
      [0, 1, 2, 3, 4, 5, 6]

      1
      1
      1.483101
      0.006139
      -0.000585
      6
      [0, 1, 2, 3, 4, 5]

      2
      2
      1.481667
      0.006135
      -0.000967
      5
      [0, 1, 2, 3, 4]

      3
      3
      1.001292
      0.005613
      -0.324212
      4
      [0, 1, 3, 4]

      4
      4
      0.751952
      0.005343
      -0.249018
      3
      [0, 1, 3]

      5
      5
      0.390138
      0.005015
      -0.481166
      2
      [1, 3]

      6
      6
      0.212257
      0.003765
      -0.455946
      1
      [3]

It is possible to see that the estimated mutual information is untouched until Round 2, when it varies around -$30\%$.

Since there is a 'break' in Round 2, we should choose to stop the algorithm at theta round. This will be clear in the Mutual Information history plot that follows:

```python
select.plot_mi()
```

Plotting the percentual variations of the mutual information between rounds:

```python
select.plot_delta()
```

Making the selection choosing to stop at Round 2:

```python
X_new = select.transform(X, rd=2)

X_new.shape
```

    (20000, 5)

### 1.3\. Selecting Features for a Classification Task

Categorizing $Y$:

```python
ind0 = (y<np.percentile(y, 33))
ind1 = (np.percentile(y, 33)<=y) & (y<np.percentile(y, 66))
ind2 = (np.percentile(y, 66)<=y)

y[ind0] = 0
y[ind1] = 1
y[ind2] = 2

y=y.astype(int)
```

```python
y[:15]
```

    array([2, 2, 2, 1, 0, 1, 2, 0, 0, 2, 0, 0, 1, 1, 1])

Training (and validating) GMMs:

```python
%%time

gmm=inf.get_gmm(X, y, y_cat=True)
```

    CPU times: user 17.4 s, sys: 8.57 s, total: 26 s
    Wall time: 15.9 s

```python
gmm
```

    {0: GaussianMixture(covariance_type='full', init_params='kmeans', max_iter=100,
                     means_init=None, n_components=20, n_init=1,
                     precisions_init=None, random_state=42, reg_covar=1e-06,
                     tol=0.001, verbose=0, verbose_interval=10, warm_start=False,
                     weights_init=None),
     1: GaussianMixture(covariance_type='full', init_params='kmeans', max_iter=100,
                     means_init=None, n_components=20, n_init=1,
                     precisions_init=None, random_state=42, reg_covar=1e-06,
                     tol=0.001, verbose=0, verbose_interval=10, warm_start=False,
                     weights_init=None),
     2: GaussianMixture(covariance_type='full', init_params='kmeans', max_iter=100,
                     means_init=None, n_components=20, n_init=1,
                     precisions_init=None, random_state=42, reg_covar=1e-06,
                     tol=0.001, verbose=0, verbose_interval=10, warm_start=False,
                     weights_init=None)}

Ordering features by their importances using the *Forward Selection* algorithm:

```python
select=inf.SelectVars(gmm, selection_mode='forward')
select.fit(X, y, verbose=True)
```

    Let's start...

    Round =   0   |   Î =  0.00   |   Δ%Î =  0.00   |   Features=[]
    Round =   1   |   Î =  0.14   |   Δ%Î =  0.00   |   Features=[3]
    Round =   2   |   Î =  0.28   |   Δ%Î =  0.94   |   Features=[3, 1]
    Round =   3   |   Î =  0.52   |   Δ%Î =  0.87   |   Features=[3, 1, 0]
    Round =   4   |   Î =  0.64   |   Δ%Î =  0.21   |   Features=[3, 1, 0, 4]
    Round =   5   |   Î =  0.80   |   Δ%Î =  0.25   |   Features=[3, 1, 0, 4, 2]
    Round =   6   |   Î =  0.79   |   Δ%Î = -0.00   |   Features=[3, 1, 0, 4, 2, 6]
    Round =   7   |   Î =  0.79   |   Δ%Î = -0.00   |   Features=[3, 1, 0, 4, 2, 6, 5]

Checking history:

```python
select.get_info()
```

      rounds
      mi_mean
      mi_error
      delta
      num_feat
      features

      0
      0
      0.000000
      0.000000
      0.000000
      0
      []

      1
      1
      0.144904
      0.003717
      0.000000
      1
      [3]

      2
      2
      0.280840
      0.004493
      0.938110
      2
      [3, 1]

      3
      3
      0.524522
      0.004559
      0.867691
      3
      [3, 1, 0]

      4
      4
      0.636269
      0.004315
      0.213045
      4
      [3, 1, 0, 4]

      5
      5
      0.795123
      0.003400
      0.249667
      5
      [3, 1, 0, 4, 2]

      6
      6
      0.792673
      0.003550
      -0.003082
      6
      [3, 1, 0, 4, 2, 6]

      7
      7
      0.791315
      0.003708
      -0.001712
      7
      [3, 1, 0, 4, 2, 6, 5]

It is possible to see that the estimated mutual information is untouched from Round 6 onwards.

Since there is a 'break' in Round 5, we should choose to stop the algorithm at theta round. This will be clear in the Mutual Information history plot that follows:

```python
select.plot_mi()
```

Plotting the percentual variations of the mutual information between rounds:

```python
select.plot_delta()

```

Making the selection choosing to stop at Round 5:

```python
X_new = select.transform(X, rd=5)

X_new.shape
```

    (20000, 5)

## 5\. References

[1] Eirola, E., Lendasse, A., & Karhunen, J. (2014, July). Variable selection for regression problems using Gaussian mixture models to estimate mutual information. In 2014 International Joint Conference on Neural Networks (IJCNN) (pp. 1606-1613). IEEE.

[2] Lan, T., Erdogmus, D., Ozertem, U., & Huang, Y. (2006, July). Estimating mutual information using gaussian mixture model for feature ranking and selection. In The 2006 IEEE International Joint Conference on Neural Network Proceedings (pp. 5034-5039). IEEE.

[3] Polo, F. M., & Vicente, R. (2020). Covariate Shift Adaptation in High-Dimensional and Divergent Distributions. arXiv preprint arXiv:2010.01184.


### An Auto correct system is an application that changes mispelled words into the correct ones.

```python
# In this notebook I'll show how to implement an Auto Correct System that its very usefull.
# This auto correct system only search for spelling erros, not contextual errors.
```

*The implementation can be divided into 4 steps:*

[1]. **Identity a mispelled word.**

[2]. **Find strings n Edit Distance away**

[3]. **Filter Candidates** (*as Real Words that are spelled correct*)

[4]. **Calculate Word Probabilities.** (*Choose the most likely cadidate to be the replacement*)

### 1. Identity a mispelled Word

*To identify if a word was mispelled, you can check if the word is in the dictionary / vocabulary.*

```python
vocab = ['dean','deer','dear','fries','and','coke', 'congratulations', 'my']

word_test = 'Congratulations my deah'
word_test = word_test.lower()
word_test = word_test.split()

for word in word_test:
    if word in vocab:
        print(f'The word: {word} is in the vocab')
    else:
        print(f"The word: {word} isn't in the vocabulary")

```

    The word: congratulations is in the vocab
    The word: my is in the vocab
    The word: deah isn't in the vocabulary

### 2. Find strings n Edit Distance Away

*Edit is a operation performed on a string to change into another string. Edit distance count the number of these operations*

*So **n Edit Distance** tells you how many operations away one string is from another.*

*For this application we'll use the Levenshtein Distance value's cost, where this edit value are:*

* **Insert** - Operation where you insert a letter, the cost is equal to 1.

* **Delete** - Operation where you delete a letter, the cost is equal to 1.

* **Replace** - Operation where you replace one letter to another, the cost is equal to 2.

* **Switch** - Operation where you swap  2 **adjacent** letters

*Also we'll use the Minimum Edit Distance which is the minimum number of edits needed to transform 1 string into the other, for that we are using n = 2 and the Dynamic Programming algorithm. ( will be explained when it is implemented ) for evaluate our model*

```python
# To implement this operations we need to split the word into 2 parts in all possible ways

word = 'dear'

split_word = [[word[:i], word[i:]] for i in range(len(word) + 1)]
for i in split_word:
    print(i)
```

    ['', 'dear']
    ['d', 'ear']
    ['de', 'ar']
    ['dea', 'r']
    ['dear', '']

```python
# The delete operation need to delete each possible letter from the original word.

delete_operation = [[L + R[1:]] for L, R in split_word if R ]

for i in delete_operation:
    print(i)
```

    ['ear']
    ['dar']
    ['der']
    ['dea']

```python
# The same way the insert operation need to add each possible letter from the vocab to the original word

letters = 'abcdefghijklmnopqrstuvwxyz'
insert_operation = [L + s + R for L, R in split_word for s in letters]

c = 0
print('the first insert operations: ')
print()
for i in insert_operation:
    print(i)
    c += 1
    if c == 4:
        break
c = 0
print('the last insert operations:')
print()
for i in insert_operation:
    c += 1
    if c > 126:
        print(i)

```

    the first insert operations:

    adear
    bdear
    cdear
    ddear
    the last insert operations:

    dearw
    dearx
    deary
    dearz

```python
# Switch Operation

switch_operation = [[L[:-1] + R[0] + L[-1] + R[1:]] for L, R in split_word if R and L]

for i in switch_operation:
    print(i)

```

    ['edar']
    ['daer']
    ['dera']

```python
# Replace Operation

letters = 'abcdefghijklmnopqrstuvwxyz'
replace_operation = [L + s + (R[1:] if len(R) > 1 else '') for L, R in split_word if R for s in letters ]

c = 0
print('the first replace operations: ')
print()
for i in replace_operation:
    print(i)
    c += 1
    if c == 4:
        break

c = 0
print('the last replace operations:')
print()
for i in replace_operation:
    c += 1
    if c > 100:
        print(i)

# Remember that at the end we need to remove the word it self
replace_operation = set(replace_operation)
replace_operation.discard('dear')
```

    the first replace operations:

    aear
    bear
    cear
    dear
    the last replace operations:

    deaw
    deax
    deay
    deaz

### 3. Filter Candidates

*We only want to consider real and correctly spelled words form the candidate lists, so we need to compare to a know dictionary.*

*If the string does not appears in the dict, remove from the candidates, this way resulting in a list of actual words only*

```python
vocab = ['dean','deer','dear','fries','and','coke', 'congratulations', 'my']

# for example we can use the replace operations words to filter in our vocab

filtered_words = [word for word in replace_operation if word in vocab]
print(filtered_words)
```

    ['deer', 'dean']

### 4. Calculate the words probabilities

*We need to find the most likely word from the cadidate list, to calculate the probability of a word in the
sentence we need to first calculate the word frequencies, also we want to count the total number of word in the body of texts
or corpus.*

*So we compute the probability that each word will appear if randomly selected from the corpus of words.*

$$P(w_i) = \frac{C(w_i)}{M} \tag{Eq 01}$$
*where*

$C(w_i)$ *is the total number of times $w_i$ appears in the corpus.*

$M$ *is the total number of words in the corpus.*

*For example, the probability of the word 'am' in the sentence **'I am happy because I am learning'** is:*

$$P(am) = \frac{C(w_i)}{M} = \frac {2}{7} \tag{Eq 02}.$$

### Now the we know the four steps of the Auto Correct System, we can start to implement it

```python
# import libraries
import re
from collections import Counter
import numpy as np
import pandas as pd
```

*The first thing to do is the data pre processing, for this example we'll use the file called **'shakespeare.txt'** this file can be found in the directory.*

```python
def process_data(filename):
    """
    Input:
        A file_name which is found in the current directory. We just have to read it in.
    Output:
        words: a list containing all the words in the corpus (text file you read) in lower case.
    """

    words = []
    with open(filename, 'r') as f:
        text = f.read()

    words = re.findall(r'\w+', text)
    words = [word.lower() for word in words]

    return words
```

```python
words = process_data('shakespeare.txt')
vocab = set(words) # eliminate duplicates

print(f'The vocabulary has {len(vocab)} unique words.')
```

    The vocabulary has 6116 unique words.

*The second step, we need to count the frequency of every word in the dictionary to later calculate the probabilities*

```python
def get_count(word):
    '''
    Input:
        word_l: a set of words representing the corpus.
    Output:
        word_count_dict: The wordcount dictionary where key is the word and value is its frequency.
    '''
    word_count_dict = {}

    word_count_dict = Counter(word)

    return word_count_dict

word_count_dict = get_count(words)
print(f'There are {len(word_count_dict)} key par values')
print(f"The count for the word 'thee' is {word_count_dict.get('thee',0)}")
```

    There are 6116 key par values
    The count for the word 'thee' is 240

*Now we must calculate the probability that each word appears using the (eq 01):*

```python

def get_probs(word_count_dict):
    '''
    Input:
        word_count_dict: The wordcount dictionary where key is the word and value is its frequency.
    Output:
        probs: A dictionary where keys are the words and the values are the probability that a word will occur.
    '''

    probs = {}
    total_words = 0

    for word, value in word_count_dict.items():
        total_words += value  # we add the quantity of each word appears

    for word, value in word_count_dict.items():
        probs[word] = value / total_words

    return probs

probs = get_probs(word_count_dict)
print(f"Length of probs is {len(probs)}")
print(f"P('thee') is {probs['thee']:.4f}")
```

    Length of probs is 6116
    P('thee') is 0.0045

*Now, that we have computed $P(w_i)$ for all the words in the corpus, we'll write the functions such as delete, insert, switch and replace to manipulate strings so that we can edit the erroneous strings and return the right spellings of the words.*

```python
def delete_letter(word, verbose = False):
    '''
    Input:
        word: the string/word for which you will generate all possible words
                in the vocabulary which have 1 missing character
    Output:
        delete_l: a list of all possible strings obtained by deleting 1 character from word
    '''

    delete = []
    split_word = []

    split_word = [[word[:i], word[i:]] for i in range(len(word))]

    delete = [L + R[1:] for L, R in split_word if R]

    if verbose: print(f"input word {word}, \nsplit_word = {split_word}, \ndelete_word = {delete}")

    return delete

delete_word = delete_letter(word="cans",
                        verbose=True)
```

    input word cans,
    split_word = [['', 'cans'], ['c', 'ans'], ['ca', 'ns'], ['can', 's']],
    delete_word = ['ans', 'cns', 'cas', 'can']

```python
def switch_letter(word, verbose = False):
    '''
    Input:
        word: input string
     Output:
        switches: a list of all possible strings with one adjacent charater switched
    '''

    switch = []
    split_word = []

    split_word = [[word[:i], word[i:]] for i in range(len(word))]

    switch = [L[:-1] + R[0] + L[-1] + R[1:] for L, R in split_word if L and R]

    if verbose: print(f"Input word = {word} \nsplit = {split_word} \nswitch = {switch}")

    return switch

switch_word_l = switch_letter(word="eta",
                         verbose=True)
```

    Input word = eta
    split = [['', 'eta'], ['e', 'ta'], ['et', 'a']]
    switch = ['tea', 'eat']

```python
def replace_letter(word, verbose=False):
    '''
    Input:
        word: the input string/word
    Output:
        replaces: a list of all possible strings where we replaced one letter from the original word.
    '''

    letters = 'abcdefghijklmnopqrstuvwxyz'
    replace = []
    split_word = []

    split_word = [(word[:i], word[i:]) for i in range(len(word))]

    replace = [L + s + (R[1:] if len(R) > 1 else '') for L, R in split_word if R for s in letters ]

    # we need to remove the actual word from the list
    replace = set(replace)
    replace.discard(word)

    replace = sorted(list(replace)) # turn the set back into a list and sort it, for easier viewing

    if verbose: print(f"Input word = {word} \nsplit = {split_word} \nreplace {replace}")

    return replace

replace_l = replace_letter(word='can',
                              verbose=True)
```

    Input word = can
    split = [('', 'can'), ('c', 'an'), ('ca', 'n')]
    replace ['aan', 'ban', 'caa', 'cab', 'cac', 'cad', 'cae', 'caf', 'cag', 'cah', 'cai', 'caj', 'cak', 'cal', 'cam', 'cao', 'cap', 'caq', 'car', 'cas', 'cat', 'cau', 'cav', 'caw', 'cax', 'cay', 'caz', 'cbn', 'ccn', 'cdn', 'cen', 'cfn', 'cgn', 'chn', 'cin', 'cjn', 'ckn', 'cln', 'cmn', 'cnn', 'con', 'cpn', 'cqn', 'crn', 'csn', 'ctn', 'cun', 'cvn', 'cwn', 'cxn', 'cyn', 'czn', 'dan', 'ean', 'fan', 'gan', 'han', 'ian', 'jan', 'kan', 'lan', 'man', 'nan', 'oan', 'pan', 'qan', 'ran', 'san', 'tan', 'uan', 'van', 'wan', 'xan', 'yan', 'zan']

```python

def insert_letter(word, verbose=False):
    '''
    Input:
        word: the input string/word
    Output:
        inserts: a set of all possible strings with one new letter inserted at every offset
    '''
    letters = 'abcdefghijklmnopqrstuvwxyz'
    insert = []
    split_word = []

    split_word = [(word[:i], word[i:]) for i in range(len(word) + 1 )]
    insert = [L + s + R for L, R in split_word  for s in letters]

    if verbose: print(f"Input word {word} \nsplit = {split_word} \ninsert = {insert}")

    return insert

insert = insert_letter('at', True)
print(f"Number of strings output by insert_letter('at') is {len(insert)}")
```

    Input word at
    split = [('', 'at'), ('a', 't'), ('at', '')]
    insert = ['aat', 'bat', 'cat', 'dat', 'eat', 'fat', 'gat', 'hat', 'iat', 'jat', 'kat', 'lat', 'mat', 'nat', 'oat', 'pat', 'qat', 'rat', 'sat', 'tat', 'uat', 'vat', 'wat', 'xat', 'yat', 'zat', 'aat', 'abt', 'act', 'adt', 'aet', 'aft', 'agt', 'aht', 'ait', 'ajt', 'akt', 'alt', 'amt', 'ant', 'aot', 'apt', 'aqt', 'art', 'ast', 'att', 'aut', 'avt', 'awt', 'axt', 'ayt', 'azt', 'ata', 'atb', 'atc', 'atd', 'ate', 'atf', 'atg', 'ath', 'ati', 'atj', 'atk', 'atl', 'atm', 'atn', 'ato', 'atp', 'atq', 'atr', 'ats', 'att', 'atu', 'atv', 'atw', 'atx', 'aty', 'atz']
    Number of strings output by insert_letter('at') is 78

*Now that we have implemented the string manipulations, we'll create two functions that, given a string, will return all the possible single and double edits on that string. These will be `edit_one_letter()` and `edit_two_letters()`.*

```python
def edit_one_letter(word, allow_switches = True): # The 'switch' function is a less common edit function,
                                                # so  will be selected by an "allow_switches" input argument.
    """
    Input:
        word: the string/word for which we will generate all possible wordsthat are one edit away.
    Output:
        edit_one_set: a set of words with one possible edit. Please return a set. and not a list.
    """

    edit_one_set = set()
    all_word, words = [] , []

    words.append(insert_letter(word))
    words.append(delete_letter(word))
    words.append(replace_letter(word))
    if allow_switches == True:
        words.append(switch_letter(word))

    for i in words:
        for each_word in i:
            if each_word == word: # we exclude the word it self
                continue
            all_word.append(each_word)

    edit_one_set = set(all_word)

    return edit_one_set

tmp_word = "at"
tmp_edit_one_set = edit_one_letter(tmp_word)
# turn this into a list to sort it, in order to view it
tmp_edit_one = sorted(list(tmp_edit_one_set))

print(f"input word: {tmp_word} \nedit_one \n{tmp_edit_one}\n")
print(f"The type of the returned object should be a set {type(tmp_edit_one_set)}")
print(f"Number of outputs from edit_one_letter('at') is {len(edit_one_letter('at'))}")
```

    input word: at
    edit_one
    ['a', 'aa', 'aat', 'ab', 'abt', 'ac', 'act', 'ad', 'adt', 'ae', 'aet', 'af', 'aft', 'ag', 'agt', 'ah', 'aht', 'ai', 'ait', 'aj', 'ajt', 'ak', 'akt', 'al', 'alt', 'am', 'amt', 'an', 'ant', 'ao', 'aot', 'ap', 'apt', 'aq', 'aqt', 'ar', 'art', 'as', 'ast', 'ata', 'atb', 'atc', 'atd', 'ate', 'atf', 'atg', 'ath', 'ati', 'atj', 'atk', 'atl', 'atm', 'atn', 'ato', 'atp', 'atq', 'atr', 'ats', 'att', 'atu', 'atv', 'atw', 'atx', 'aty', 'atz', 'au', 'aut', 'av', 'avt', 'aw', 'awt', 'ax', 'axt', 'ay', 'ayt', 'az', 'azt', 'bat', 'bt', 'cat', 'ct', 'dat', 'dt', 'eat', 'et', 'fat', 'ft', 'gat', 'gt', 'hat', 'ht', 'iat', 'it', 'jat', 'jt', 'kat', 'kt', 'lat', 'lt', 'mat', 'mt', 'nat', 'nt', 'oat', 'ot', 'pat', 'pt', 'qat', 'qt', 'rat', 'rt', 'sat', 'st', 't', 'ta', 'tat', 'tt', 'uat', 'ut', 'vat', 'vt', 'wat', 'wt', 'xat', 'xt', 'yat', 'yt', 'zat', 'zt']

    The type of the returned object should be a set <class 'set'>
    Number of outputs from edit_one_letter('at') is 129

```python
def edit_two_letters(word, allow_switches = True):
    '''
    Input:
        word: the input string/word
    Output:
        edit_two_set: a set of strings with all possible two edits
    '''

    edit_two_set = set()

    if allow_switches == True:
        first_edit = edit_one_letter(word)

    else:
        first_edit = edit_one_letter(word, allow_switches = False)

    first_edit = set(first_edit)
    second_edit = []
    final_edit = []

    if allow_switches == True:
        for each_word in first_edit:
            second_edit.append(edit_one_letter(each_word))
        for i in second_edit:
               for each_word in i:
                    final_edit.append(each_word)
        edit_two_set = set(final_edit)

    else:
        for each_word in first_edit:
            second_edit.append(edit_one_letter(each_word, allow_switches = False))
        for i in second_edit:
               for each_word in i:
                    final_edit.append(each_word)
        edit_two_set = set(final_edit)

    return edit_two_set

```

```python
tmp_edit_two_set = edit_two_letters("a")
tmp_edit_two_l = sorted(list(tmp_edit_two_set))
print(f"Number of strings with edit distance of two: {len(tmp_edit_two_l)}")
print(f"First 10 strings {tmp_edit_two_l[:10]}")
print(f"Last 10 strings {tmp_edit_two_l[-10:]}")
print(f"The data type of the returned object should be a set {type(tmp_edit_two_set)}")
print(f"Number of strings that are 2 edit distances from 'at' is {len(edit_two_letters('at'))}")
```

    Number of strings with edit distance of two: 2654
    First 10 strings ['', 'a', 'aa', 'aaa', 'aab', 'aac', 'aad', 'aae', 'aaf', 'aag']
    Last 10 strings ['zv', 'zva', 'zw', 'zwa', 'zx', 'zxa', 'zy', 'zya', 'zz', 'zza']
    The data type of the returned object should be a set <class 'set'>
    Number of strings that are 2 edit distances from 'at' is 7154

*Now we will use the `edit_two_letters` function to get a set of all the possible 2 edits on our word. We will then use those strings to get the most probable word we meant to substitute our word typing suggestion.*

```python
def get_corrections(word, probs, vocab, n=2, verbose = False):
    '''
    Input:
        word: a user entered string to check for suggestions
        probs: a dictionary that maps each word to its probability in the corpus
        vocab: a set containing all the vocabulary
        n: number of possible word corrections you want returned in the dictionary
    Output:
        n_best: a list of tuples with the most probable n corrected words and their probabilities.
    '''

    suggestions = []
    n_best = []

    # look if the word exist in the vocab, if doesn't, the edit_one_letter fuction its used, if any of the letter created
    # exists in the vocab, take the two letter edit function, if any of this situations are in the vocab, take the input word
    suggestions = list((word in vocab) or (edit_one_letter(word).intersection(vocab)) or (edit_two_letter(word).intersection(vocab)) or word)

    n_best= [[word, probs[word]] for word in (suggestions)]  # make a list with the possible word and probability.

    if verbose: print("entered word = ", word, "\nsuggestions = ", set(suggestions))

    return n_best

```

```python
my_word = 'dys'
tmp_corrections = get_corrections(my_word, probs, vocab, 2, verbose=True) # keep verbose=True
for i, word_prob in enumerate(tmp_corrections):
    print(f"word {i}: {word_prob[0]}, probability {word_prob[1]:.6f}")

print(f'The highest score for all the candidates is the word {tmp_corrections[np.argmax(word_prob)][0]}')

```

    entered word =  dys
    suggestions =  {'days', 'dye'}
    word 0: days, probability 0.000410
    word 1: dye, probability 0.000019
    The highest score for all the candidates is the word days

*Now that we have implemented the auto-correct system, how do you evaluate the similarity between two strings? For example: 'waht' and 'what'.*

*Also how do you efficiently find the shortest path to go from the word, 'waht' to the word 'what'?*

*We will implement a dynamic programming system that will tell you the minimum number of edits required to convert a string into another string.*

### Dynamic Programming

*Dynamic Programming breaks a problem down into subproblems which can be combined to form the final solution. Here, given a string source[0..i] and a string target[0..j], we will compute all the combinations of substrings[i, j] and calculate their edit distance. To do this efficiently, we will use a table to maintain the previously computed substrings and use those to calculate larger substrings.*

*You have to create a matrix and update each element in the matrix as follows:*

$$\text{Initialization}$$

\begin{align}
D[0,0] &= 0 \\
D[i,0] &= D[i-1,0] + del\_cost(source[i]) \tag{eq 03}\\
D[0,j] &= D[0,j-1] + ins\_cost(target[j]) \\
\end{align}

*So converting the source word **play** to the target word **stay**, using an insert cost of one, a delete cost of 1, and replace cost of 2 would give you the following table:*
<table style="width:20%">

  <tr>
    <td> <b> </b>  </td>
    <td> <b># </b>  </td>
    <td> <b>s </b>  </td>
    <td> <b>t </b> </td>
    <td> <b>a </b> </td>
    <td> <b>y </b> </td>
  </tr>
   <tr>
    <td> <b>  #  </b></td>
    <td> 0</td>
    <td> 1</td>
    <td> 2</td>
    <td> 3</td>
    <td> 4</td>

  </tr>
  <tr>
    <td> <b>  p  </b></td>
    <td> 1</td>
 <td> 2</td>
    <td> 3</td>
    <td> 4</td>
   <td> 5</td>
  </tr>

  <tr>
    <td> <b> l </b></td>
    <td>2</td>
    <td>3</td>
    <td>4</td>
    <td>5</td>
    <td>6</td>
  </tr>

  <tr>
    <td> <b> a </b></td>
    <td>3</td>
     <td>4</td>
     <td>5</td>
     <td>4</td>
     <td>5</td>
  </tr>

   <tr>
    <td> <b> y </b></td>
    <td>4</td>
      <td>5</td>
     <td>6</td>
     <td>5</td>
     <td>4</td>
  </tr>

</table>

*The operations used in this algorithm are 'insert', 'delete', and 'replace'. These correspond to the functions that we defined earlier: insert_letter(), delete_letter() and replace_letter(). switch_letter() is not used here.*

*The diagram below describes how to initialize the table. Each entry in D[i,j] represents the minimum cost of converting string source[0:i] to string target[0:j]. The first column is initialized to represent the cumulative cost of deleting the source characters to convert string "EER" to "". The first row is initialized to represent the cumulative cost of inserting the target characters to convert from "" to "NEAR".*

<div style="width:image width px; font-size:100%; text-align:center;"> Figure 1 Initializing Distance Matrix</div>

*Note that the formula for $D[i,j]$ shown in the image is equivalent to:*

\begin{align}
 \\
D[i,j] =min
\begin{cases}
D[i-1,j] + del\_cost\\
D[i,j-1] + ins\_cost\\
D[i-1,j-1] + \left\{\begin{matrix}
rep\_cost; & if src[i]\neq tar[j]\\
0 ; & if src[i]=tar[j]
\end{matrix}\right.
\end{cases}
\tag{5}
\end{align}

*The variable `sub_cost` (for substitution cost) is the same as `rep_cost`; replacement cost.  We will stick with the term "replace" whenever possible.*

<div style="width:image width px; font-size:100%; text-align:center;"> Figure 2 Examples Distance Matrix</div>

```python
def min_edit_distance(source, target, ins_cost = 1, del_cost = 1, rep_cost = 2):
    '''
    Input:
        source: a string corresponding to the string you are starting with
        target: a string corresponding to the string you want to end with
        ins_cost: an integer setting the insert cost
        del_cost: an integer setting the delete cost
        rep_cost: an integer setting the replace cost
    Output:
        D: a matrix of len(source)+1 by len(target)+1 containing minimum edit distances
        med: the minimum edit distance (med) required to convert the source string to the target
    '''

    m = len(source)
    n = len(target)

    # initialize cost matrix with zeros and dimensions (m+1, n+1)
    D = np.zeros((m+1, n+1), dtype = int)

    # Fill in column 0, from row 1 to row m, both inclusive
    for row in range(1, m+1): # Replace None with the proper range
        D[row, 0] = D[row -1, 0] + del_cost

    # Fill in row 0, for all columns from 1 to n, both inclusive
    for column in range(1, n+1):
        D[0, column] = D[0, column - 1] + ins_cost

    # Loop through row 1 to row m, both inclusive
    for row in range(1, m+1):

        # Loop through column 1 to column n, both inclusive
        for column in range(1, n+1):

            # initialize r_cost to the 'replace' cost that is passed into this function
            r_cost = rep_cost

            # check to see if source character at the previous row
            # matches the target haracter at the previous column
            if source[row - 1] == target[column - 1]:
                # Update the replacement cost to 0 if source and
                # target are equal
                r_cost = 0

            # Update the cost atow, col based on previous entries in the cost matrix
            # Refer to the equation calculate for D[i,j] (the mininum of the three calculated)
            D[row, column] = min([D[row-1, column] + del_cost, D[row, column-1] + ins_cost, D[row-1, column-1] + r_cost])

    # Set the minimum edit distance with the cost found at row m, column n

    med = D[m, n]
    return D, med
```

```python
# testing your implementation
source =  'play'
target = 'stay'
matrix, min_edits = min_edit_distance(source, target)
print("minimum edits: ",min_edits, "\n")
idx = list('#' + source)
cols = list('#' + target)
df = pd.DataFrame(matrix, index=idx, columns= cols)
print(df)
```

    minimum edits:  4

       #  s  t  a  y
    #  0  1  2  3  4
    p  1  2  3  4  5
    l  2  3  4  5  6
    a  3  4  5  4  5
    y  4  5  6  5  4

```python
# testing your implementation
source =  'eer'
target = 'near'
matrix, min_edits = min_edit_distance(source, target)
print("minimum edits: ",min_edits, "\n")
idx = list(source)
idx.insert(0, '#')
cols = list(target)
cols.insert(0, '#')
df = pd.DataFrame(matrix, index=idx, columns= cols)
print(df)
```

    minimum edits:  3

       #  n  e  a  r
    #  0  1  2  3  4
    e  1  2  1  2  3
    e  2  3  2  3  4
    r  3  4  3  4  3

Auto Correct System/.ipynb_checkpoints/Auto Correct System-checkpoint.ipynb

Auto Correct System/Auto Correct System.ipynb