```R
# Part of Bootstraping
## loading the data
library(boot)
Penn <- as.data.frame(read.table("../data/penn_jae.dat", header=T ))
n <- dim(Penn)[1]
p_1 <- dim(Penn)[2]
Penn<- subset(Penn, tg== 4| tg==0)
attach(Penn)
```

    The following objects are masked from Penn (pos = 3):

        abdt, agegt54, agelt35, black, dep, durable, female, hispanic,
        husd, inuidur1, inuidur2, lusd, muld, nondurable, othrace, q1, q2,
        q3, q4, q5, q6, recall, tg

    The following objects are masked from Penn (pos = 5):

        abdt, agegt54, agelt35, black, dep, durable, female, hispanic,
        husd, inuidur1, inuidur2, lusd, muld, nondurable, othrace, q1, q2,
        q3, q4, q5, q6, recall, tg

    The following objects are masked from Penn (pos = 6):

        abdt, agegt54, agelt35, black, dep, durable, female, hispanic,
        husd, inuidur1, inuidur2, lusd, muld, nondurable, othrace, q1, q2,
        q3, q4, q5, q6, recall, tg

    The following objects are masked from Penn (pos = 7):

        abdt, agegt54, agelt35, black, dep, durable, female, hispanic,
        husd, inuidur1, inuidur2, lusd, muld, nondurable, othrace, q1, q2,
        q3, q4, q5, q6, recall, tg

```R
library(ISLR2)
set.seed(1)
library(boot)
```

```R
T4<- (tg==4)|
summary(T4)
```

```R
# Here we define the structure of our regression:
regresion <- log(inuidur1)~T4+ (female+black+othrace+factor(dep)+q2+q3+q4+q5+q6+agelt35+agegt54+durable+lusd+husd

```

```R
boot.fn <- function(data, index) {
  coef(lm(log(inuidur1)~T4+ (female+black+othrace+factor(dep)+q2+q3+q4+q5+q6+agelt35+agegt54+durable+lusd+husd), data = data, subset = index)) }
```

```R
# Finally, here we define our bootstapping proccess:
example <- boot(Penn, boot.fn , 1000)
example

```

    ORDINARY NONPARAMETRIC BOOTSTRAP

    Call:
    boot(data = Penn, statistic = boot.fn, R = 1000)

    Bootstrap Statistics :
             original        bias    std. error
    t1*   2.178462326  0.0052223099  0.15609830
    t2*  -0.071692484  0.0002443722  0.03633901
    t3*   0.126368328 -0.0005970869  0.03548195
    t4*  -0.293767980 -0.0023950938  0.05902701
    t5*  -0.472445058 -0.0009669991  0.23662611
    t6*   0.029866899 -0.0015803274  0.05316184
    t7*   0.096186517  0.0001454218  0.04751720
    t8*   0.073678072 -0.0047203080  0.15280807
    t9*  -0.038506537 -0.0029706659  0.15249683
    t10* -0.054949195 -0.0046900884  0.15342405
    t11* -0.144177912 -0.0042572281  0.15104799
    t12*  0.003361318 -0.0024182498  0.16165529
    t13* -0.162772168 -0.0000218298  0.03768461
    t14*  0.229666708 -0.0001082586  0.05800139
    t15*  0.126557359 -0.0024061924  0.04917096
    t16* -0.175352572 -0.0007373578  0.04123540
    t17* -0.105224727 -0.0003037563  0.04271973

```R
# We do this to know which is the orden of our coefficents.
summary(lm(name, data = Penn))$coef
```

```R
plot(example)
```

# HTE I: Binary treatment

## Analysis

```R
install.packages("devtools")  # if you don't have this installed yet.

```

```R
devtools::install_github('susanathey/causalTree')
```

```R
# The causalTree package is not in CRAN, the most common R repository.
# To install it, uncomment the next lines as appropriate.
install.packages("devtools")  # if you don't have this installed yet.
devtools::install_github('susanathey/causalTree')
library(causalTree)

# use e.g., install.packages("grf") to install any of the following packages.
library(grf)
library(rpart)
library(glmnet)
library(splines)
library(MASS)
library(lmtest)
library(sandwich)
library(ggplot2)
```

As with other chapters in this tutorial, the code below should still work by replacing the next snippet of code with a different dataset, provided that you update the key variables `treatment`, `outcome`, and `covariates` below. Also, please make sure to read the comments as they may be subtle differences depending on whether your dataset was created in a randomized or observational setting.

```R
# Read in data
data <- as.data.frame(read.table("../data/penn_jae.dat", header=T ))
n <- nrow(data)
data<- subset(data, tg== 4| tg==0)
attach(data)
T4<- (tg==4)
summary(T4)
treatment <- "tg"

outcome <- "inuidur1"

# Additional covariates
covariates <- c("female","black","othrace","factor","dep","q2","q3","q4","q5","q6","agelt35","agegt54","durable","lusd","husd")
```

## Pre-specified hypotheses

We will begin by learning how to test pre-specified null hypotheses of the form
\begin{equation}
\label{eq:1}
H_{0}: E[Y(1) - Y(0) | G_i = 1] = E[Y(1) - Y(0) | G_i = 0]
\end{equation}

That is, that the treatment effect is the same regardless of membership to some group
$G_i$. Importantly, for now we’ll assume that the group $G_i$ was **pre-specified** -- it was decided _before_ looking at the data.

In a randomized setting, if the both the treatment  $W_i$ and group membership $G_i$ are binary, we can write
\begin{equation}
  E[Y_i(W_i)|G_i] = E[Y_i|W_i, G_i] = \beta_0 + \beta_w W_i + \beta_g G_i + \beta_{wg} W_i G_i
\end{equation}

When $W_i$ and $G_i$ are binary, this decomposition is true without loss of generality. Why?

This allows us to write the average effects of $W_i$ and $G_i$ on $Y_i$ as
\begin{equation}
  \begin{aligned}
    E[Y(1) | G_i=1] &= \beta_0 + \beta_w W_i + \beta_g G_i + \beta_{wg} W_i G_i, \\
    E[Y(1) | G_i=0] &= \beta_0 + \beta_w W_i,  \\
    E[Y(0) | G_i=1] &= \beta_0 + \beta_g G_i,  \\
    E[Y(0) | G_i=0] &= \beta_0.
  \end{aligned}
\end{equation}

Rewriting the null hypothesis \ref{eq:1} in terms of the decomposition \@ref(eq:decomp), we see that it boils down to a test about the coefficient in the interaction: $\beta_{xw} = 0$

```R
# Only valid in randomized settings

# Suppose this his group was defined prior to collecting the data
# Recall from last chapter -- this is equivalent to running a t-test
fmla <- log(inuidur1)~T4+ (female+black+othrace+factor(dep)+q2+q3+q4+q5+q6+agelt35+agegt54+durable+lusd+husd)
ols <- lm(fmla, data=data)
coeftest(ols, vcov=vcovHC(ols, type='HC2'))
```

## Data-driven hypotheses

Pre-specifying hypotheses prior to looking at the data is in general good practice to avoid "p-hacking" (e.g., slicing the data into different subgroups until a significant result is found). However, valid tests can also be attained if by **sample splitting**: we can use a subset of the sample to find promising subgroups, then test hypotheses about these subgroups in the remaining sample. This kind of sample splitting for hypothesis testing is called **honesty**.

### Via causal trees

**Causal trees** [(Athey and Imbens)](PNAS, 2016)](https://www.pnas.org/content/pnas/113/27/7353.full.pdf) are an intuitive algorithm that is available in the randomized setting to discover subgroups with different treatment effects.

At a high level, the idea is to divide the sample into three subsets (not necessarily of equal size). The `splitting` subset is used to fit a decision tree whose objective is modified to maximize heterogeneity in treatment effect estimates across leaves. The `estimation` subset is then used to produce a valid estimate of the treatment effect at each leaf of the fitted tree. Finally, a `test` subset can be used to validate the tree estimates.

The next snippet uses `honest.causalTree` function from the [`causalTree`](https://github.com/susanathey/causalTree) package. For more details, see the [causalTree documentation](https://github.com/susanathey/causalTree/blob/master/briefintro.pdf).

```R
# Only valid for randomized data!
fmla <- paste(outcome, " ~", paste(covariates, collapse = " + "))
fmla
```

```R
# Dividing data into three subsets
indices <- split(seq(nrow(data)), sort(seq(nrow(data)) %% 3))
names(indices) <- c('split', 'est', 'test')
```

```R
# Fitting the forest
ct.unpruned <- honest.causalTree(
  formula=fmla,            # Define the model
  data=data[indices$split,],
  treatment=data[indices$split, treatment],
  est_data=data[indices$est,],
  est_treatment=data[indices$est, treatment],
  minsize=1,                 # Min. number of treatment and control cases in each leaf
  HonestSampleSize=length(indices$est), #  Num obs used in estimation after splitting

  # We recommend not changing the parameters below
  split.Rule="CT",            # Define the splitting option
  cv.option="TOT",            # Cross validation options
  cp=0,                       # Complexity parameter
  split.Honest=TRUE,          # Use honesty when splitting
  cv.Honest=TRUE              # Use honesty when performing cross-validation
)
```

```R
# Table of cross-validated values by tuning parameter.
ct.cptable <- as.data.frame(ct.unpruned$cptable)
```

```R
# Obtain optimal complexity parameter to prune tree.
cp.selected <- which.min(ct.cptable$xerror)
```

```R
cp.optimal <- ct.cptable[cp.selected, "CP"]

# Prune the tree at optimal complexity parameter.
ct.pruned <- prune(tree=ct.unpruned, cp=cp.optimal)
```

```R
# Predict point estimates (on estimation sample)
tau.hat.est <- predict(ct.pruned, newdata=data[indices$est,])

# Create a factor column 'leaf' indicating leaf assignment in the estimation set
num.leaves <- length(unique(tau.hat.est))
leaf <- factor(tau.hat.est, levels=sort(unique(tau.hat.est)), labels = seq(num.leaves))
```

Note: if your tree is not splitting at all, try decreasing the parameter `minsize` that controls the minimum size of each leaf. The next snippet plots the learned tree. The values in the cell are the estimated treatment effect and an estimate of the fraction of the population that falls within each leaf. Both are estimated using the `estimation` sample.

```R
rpart.plot(
  x=ct.pruned,        # Pruned tree
  type=3,             # Draw separate split labels for the left and right directions
  fallen=TRUE,        # Position the leaf nodes at the bottom of the graph
  leaf.round=1,       # Rounding of the corners of the leaf node boxes
  extra=100,          # Display the percentage of observations in the node
  branch=.1,          # Shape of the branch lines
  box.palette="RdBu") # Palette for coloring the node
```

Interpret the heatmap above. What describes the subgroups with strongest and weakest estimated treatment effect?

### Via grf

The function `causal_forest` from the package `grf` allows us to get estimates of the CATE \@ref(eq:cate).

```R
# Get predictions from forest fitted above.
tau.hat <- predict(forest.tau)$predictions  # tau(X) estimates
```

Having fit a non-parametric method such as a causal forest, a researcher may (incorrectly) start by looking at the distribution of its predictions of the treatment effect. One might be tempted to think: "if the histogram is concentrated at a point, then there is no heterogeneity; if the histogram is spread out, then our estimator has found interesting heterogeneity." However, this may be false.

```R
# Do not use this for assessing heterogeneity. See text above.
hist(tau.hat, main="CATE estimates", freq=F)
```

If the histogram is concentrated at a point, we may simply be underpowered: our method was not able to detect any heterogeneity, but maybe it would detect it if we had more data. If the histogram is spread out, we may be overfitting: our model is producing very noisy estimates $\widehat{\tau}(x)$, but in fact the true  $\tau(x)$ can be much smoother as a function of $x$.

The `grf` package also produces a measure of variable importance that indicates how often a variable was used in a tree split. Again, much like the histogram above, this can be a rough diagnostic, but it should not be interpreted as indicating that, for example, variable with low importance is not related to heterogeneity. The reasoning is the same as the one presented in the causal trees section: if two covariates are highly correlated, the trees might split on one covariate but not the other, even though both (or maybe neither) are relevant in the true data-generating process.

```R
var_imp <- c(variable_importance(forest.tau))
names(var_imp) <- covariates
sorted_var_imp <- sort(var_imp, decreasing = TRUE)
sorted_var_imp[1:5]  # showing only first few
```

#### Data-driven subgroups

Just as with causal trees, we can use causal forests to divide our observations into subgroups. In place of leaves, we'll rank observation into (say) quintiles according to their estimated CATE prediction; see, e.g., [Chernozhukov, Demirer, Duflo, Fernández-Val (2020)](https://arxiv.org/abs/1712.04802) for similar ideas.

There's a subtle but important point that needs to be addressed here. As we have mentioned before, when predicting the conditional average treatment effect $\tau(X_i)$ for observation $i$ we should in general avoid using a model that was fitted using observation $i$. This sort of sample splitting (which we called **honesty** above) is one of the required ingredients to get unbiased estimates of the CATE using the methods described here. However, when ranking estimates of two observations $i$ and $j$, we need something a little stronger: we must ensure that the model was not fit using _either_ $i$ _or_ $j$'s data.

One way of overcoming this obstacle is simple. First, divide the data into $K$ folds (subsets). Then, cycle through the folds, fitting a CATE model on $K-1$ folds. Next, for each held-out fold, _separately_ rank the unseen observations into $Q$ groups based on their prediction  (i.e., if $Q=5$, then we rank observations by estimated CATE into "top quintile", "second quintile", and so on). After concatenating the independent rankings together, we can study the differences in observations in each rank-group.

[This gist](https://gist.github.com/halflearned/bea4e5137c0c81fd18a75f682da466c8) computes the above for `grf`, and it should not be hard to modify it so as to replace forests by any other non-parametric method. However, for `grf` specifically, there's a small trick that allows us to obtain a valid ranking: we can pass a vector of fold indices to the argument `clusters` and rank observations within each fold. This works because estimates for each fold ("cluster")   trees are computed using trees that were not fit using observations from that fold. Here's how to do it.

```R
# Valid randomized data and observational data with unconfoundedness+overlap.
# Note: read the comments below carefully.
# In randomized settings, do not estimate forest.e and e.hat; use known assignment probs.

# Prepare dataset
fmla <- formula(paste0("~ 0 + ", paste0(covariates, collapse="+")))
X <- model.matrix(fmla, data)
W <- data[,treatment]
Y <- data[,outcome]

# Number of rankings that the predictions will be ranking on
# (e.g., 2 for above/below median estimated CATE, 5 for estimated CATE quintiles, etc.)
num.rankings <- 5

# Prepare for data.splitting
# Assign a fold number to each observation.
# The argument 'clusters' in the next step will mimick K-fold cross-fitting.
num.folds <- 10
folds <- sort(seq(n) %% num.folds) + 1

# Comment or uncomment depending on your setting.
# Observational setting with unconfoundedness+overlap (unknown assignment probs):
# forest <- causal_forest(X, Y, W, clusters = folds)
# Randomized settings with fixed and known probabilities (here: 0.5).
forest <- causal_forest(X, Y, W, W.hat=.5, clusters = folds)

# Retrieve out-of-bag predictions.
# Predictions for observation in fold k will be computed using
# trees that were not trained using observations for that fold.
tau.hat <- predict(forest)$predictions

# Rank observations *within each fold* into quintiles according to their CATE predictions.
ranking <- rep(NA, n)
for (fold in seq(num.folds)) {
  tau.hat.quantiles <- quantile(tau.hat[folds == fold], probs = seq(0, 1, by=1/num.rankings))
  ranking[folds == fold] <- cut(tau.hat[folds == fold], tau.hat.quantiles, include.lowest=TRUE,labels=seq(num.rankings))
}
```

The next snippet computes the average treatment effect within each group defined above, i.e., $\E[Y_i(1) - Y_i(0)|G_i = g]$. This can done in two ways. First, by computing a simple difference-in-means estimate of the ATE based on observations within each group. This is valid only in randomized settings.

```R
# Valid only in randomized settings.
# Average difference-in-means within each ranking

# Formula y ~ 0 + ranking + ranking:w
fmla <- paste0(outcome, " ~ 0 + ranking + ranking:", treatment)
ols.ate <- lm(fmla, data=transform(data, ranking=factor(ranking)))
ols.ate <- coeftest(ols.ate, vcov=vcovHC(ols.ate, type='HC2'))
interact <- which(grepl(":", rownames(ols.ate)))
ols.ate <- data.frame("ols", paste0("Q", seq(num.rankings)), ols.ate[interact, 1:2])
rownames(ols.ate) <- NULL # just for display
colnames(ols.ate) <- c("method", "ranking", "estimate", "std.err")
ols.ate
```

Another option is to average the AIPW scores within each group. This valid for both randomized settings and observational settings with unconfoundedness and overlap. Moreover, AIPW-based estimators should produce estimates with tighter confidence intervals in large samples.

```R

```

```R
# HTE I: Binary treatment

## Analysis

install.packages("devtools")  # if you don't have this installed yet.

devtools::install_github('susanathey/causalTree')

# The causalTree package is not in CRAN, the most common R repository.
# To install it, uncomment the next lines as appropriate.
install.packages("devtools")  # if you don't have this installed yet.
devtools::install_github('susanathey/causalTree')
library(causalTree)

# use e.g., install.packages("grf") to install any of the following packages.
library(grf)
library(rpart)
library(glmnet)
library(splines)
library(MASS)
library(lmtest)
library(sandwich)
library(ggplot2)

As with other chapters in this tutorial, the code below should still work by replacing the next snippet of code with a different dataset, provided that you update the key variables `treatment`, `outcome`, and `covariates` below. Also, please make sure to read the comments as they may be subtle differences depending on whether your dataset was created in a randomized or observational setting.

# Read in data
data <- as.data.frame(read.table("../data/penn_jae.dat", header=T ))
n <- nrow(data)
data<- subset(data, tg== 4| tg==0)
attach(data)
T4<- (tg==4)
summary(T4)
treatment <- "tg"

outcome <- "inuidur1"

# Additional covariates
covariates <- c("female","black","othrace","factor","dep","q2","q3","q4","q5","q6","agelt35","agegt54","durable","lusd","husd")

## Pre-specified hypotheses

We will begin by learning how to test pre-specified null hypotheses of the form
\begin{equation}
\label{eq:1}
H_{0}: E[Y(1) - Y(0) | G_i = 1] = E[Y(1) - Y(0) | G_i = 0]
\end{equation}

That is, that the treatment effect is the same regardless of membership to some group
$G_i$. Importantly, for now we’ll assume that the group $G_i$ was **pre-specified** -- it was decided _before_ looking at the data.

In a randomized setting, if the both the treatment  $W_i$ and group membership $G_i$ are binary, we can write
\begin{equation}
  E[Y_i(W_i)|G_i] = E[Y_i|W_i, G_i] = \beta_0 + \beta_w W_i + \beta_g G_i + \beta_{wg} W_i G_i
\end{equation}

<font size=1>
When $W_i$ and $G_i$ are binary, this decomposition is true without loss of generality. Why?
</font>

This allows us to write the average effects of $W_i$ and $G_i$ on $Y_i$ as
\begin{equation}
  \begin{aligned}
    E[Y(1) | G_i=1] &= \beta_0 + \beta_w W_i + \beta_g G_i + \beta_{wg} W_i G_i, \\
    E[Y(1) | G_i=0] &= \beta_0 + \beta_w W_i,  \\
    E[Y(0) | G_i=1] &= \beta_0 + \beta_g G_i,  \\
    E[Y(0) | G_i=0] &= \beta_0.
  \end{aligned}
\end{equation}

Rewriting the null hypothesis \ref{eq:1} in terms of the decomposition \@ref(eq:decomp), we see that it boils down to a test about the coefficient in the interaction: $\beta_{xw} = 0$

# Only valid in randomized settings

# Suppose this his group was defined prior to collecting the data
# Recall from last chapter -- this is equivalent to running a t-test
fmla <- log(inuidur1)~T4+ (female+black+othrace+factor(dep)+q2+q3+q4+q5+q6+agelt35+agegt54+durable+lusd+husd)
ols <- lm(fmla, data=data)
coeftest(ols, vcov=vcovHC(ols, type='HC2'))

## Data-driven hypotheses

Pre-specifying hypotheses prior to looking at the data is in general good practice to avoid "p-hacking" (e.g., slicing the data into different subgroups until a significant result is found). However, valid tests can also be attained if by **sample splitting**: we can use a subset of the sample to find promising subgroups, then test hypotheses about these subgroups in the remaining sample. This kind of sample splitting for hypothesis testing is called **honesty**.

### Via causal trees

**Causal trees** [(Athey and Imbens)](PNAS, 2016)](https://www.pnas.org/content/pnas/113/27/7353.full.pdf) are an intuitive algorithm that is available in the randomized setting to discover subgroups with different treatment effects.

At a high level, the idea is to divide the sample into three subsets (not necessarily of equal size). The `splitting` subset is used to fit a decision tree whose objective is modified to maximize heterogeneity in treatment effect estimates across leaves. The `estimation` subset is then used to produce a valid estimate of the treatment effect at each leaf of the fitted tree. Finally, a `test` subset can be used to validate the tree estimates.

The next snippet uses `honest.causalTree` function from the [`causalTree`](https://github.com/susanathey/causalTree) package. For more details, see the [causalTree documentation](https://github.com/susanathey/causalTree/blob/master/briefintro.pdf).

# Only valid for randomized data!
fmla <- paste(outcome, " ~", paste(covariates, collapse = " + "))
fmla

# Dividing data into three subsets
indices <- split(seq(nrow(data)), sort(seq(nrow(data)) %% 3))
names(indices) <- c('split', 'est', 'test')

# Fitting the forest
ct.unpruned <- honest.causalTree(
  formula=fmla,            # Define the model
  data=data[indices$split,],
  treatment=data[indices$split, treatment],
  est_data=data[indices$est,],
  est_treatment=data[indices$est, treatment],
  minsize=1,                 # Min. number of treatment and control cases in each leaf
  HonestSampleSize=length(indices$est), #  Num obs used in estimation after splitting

  # We recommend not changing the parameters below
  split.Rule="CT",            # Define the splitting option
  cv.option="TOT",            # Cross validation options
  cp=0,                       # Complexity parameter
  split.Honest=TRUE,          # Use honesty when splitting
  cv.Honest=TRUE              # Use honesty when performing cross-validation
)

# Table of cross-validated values by tuning parameter.
ct.cptable <- as.data.frame(ct.unpruned$cptable)

# Obtain optimal complexity parameter to prune tree.
cp.selected <- which.min(ct.cptable$xerror)

cp.optimal <- ct.cptable[cp.selected, "CP"]

# Prune the tree at optimal complexity parameter.
ct.pruned <- prune(tree=ct.unpruned, cp=cp.optimal)

# Predict point estimates (on estimation sample)
tau.hat.est <- predict(ct.pruned, newdata=data[indices$est,])

# Create a factor column 'leaf' indicating leaf assignment in the estimation set
num.leaves <- length(unique(tau.hat.est))
leaf <- factor(tau.hat.est, levels=sort(unique(tau.hat.est)), labels = seq(num.leaves))

Note: if your tree is not splitting at all, try decreasing the parameter `minsize` that controls the minimum size of each leaf. The next snippet plots the learned tree. The values in the cell are the estimated treatment effect and an estimate of the fraction of the population that falls within each leaf. Both are estimated using the `estimation` sample.

rpart.plot(
  x=ct.pruned,        # Pruned tree
  type=3,             # Draw separate split labels for the left and right directions
  fallen=TRUE,        # Position the leaf nodes at the bottom of the graph
  leaf.round=1,       # Rounding of the corners of the leaf node boxes
  extra=100,          # Display the percentage of observations in the node
  branch=.1,          # Shape of the branch lines
  box.palette="RdBu") # Palette for coloring the node

<font size=1>
Interpret the heatmap above. What describes the subgroups with strongest and weakest estimated treatment effect?
</font>

### Via grf

The function `causal_forest` from the package `grf` allows us to get estimates of the CATE \@ref(eq:cate).

# Get predictions from forest fitted above.
tau.hat <- predict(forest.tau)$predictions  # tau(X) estimates

Having fit a non-parametric method such as a causal forest, a researcher may (incorrectly) start by looking at the distribution of its predictions of the treatment effect. One might be tempted to think: "if the histogram is concentrated at a point, then there is no heterogeneity; if the histogram is spread out, then our estimator has found interesting heterogeneity." However, this may be false.

# Do not use this for assessing heterogeneity. See text above.
hist(tau.hat, main="CATE estimates", freq=F)

If the histogram is concentrated at a point, we may simply be underpowered: our method was not able to detect any heterogeneity, but maybe it would detect it if we had more data. If the histogram is spread out, we may be overfitting: our model is producing very noisy estimates $\widehat{\tau}(x)$, but in fact the true  $\tau(x)$ can be much smoother as a function of $x$.

The `grf` package also produces a measure of variable importance that indicates how often a variable was used in a tree split. Again, much like the histogram above, this can be a rough diagnostic, but it should not be interpreted as indicating that, for example, variable with low importance is not related to heterogeneity. The reasoning is the same as the one presented in the causal trees section: if two covariates are highly correlated, the trees might split on one covariate but not the other, even though both (or maybe neither) are relevant in the true data-generating process.

var_imp <- c(variable_importance(forest.tau))
names(var_imp) <- covariates
sorted_var_imp <- sort(var_imp, decreasing = TRUE)
sorted_var_imp[1:5]  # showing only first few

#### Data-driven subgroups

Just as with causal trees, we can use causal forests to divide our observations into subgroups. In place of leaves, we'll rank observation into (say) quintiles according to their estimated CATE prediction; see, e.g., [Chernozhukov, Demirer, Duflo, Fernández-Val (2020)](https://arxiv.org/abs/1712.04802) for similar ideas.

There's a subtle but important point that needs to be addressed here. As we have mentioned before, when predicting the conditional average treatment effect $\tau(X_i)$ for observation $i$ we should in general avoid using a model that was fitted using observation $i$. This sort of sample splitting (which we called **honesty** above) is one of the required ingredients to get unbiased estimates of the CATE using the methods described here. However, when ranking estimates of two observations $i$ and $j$, we need something a little stronger: we must ensure that the model was not fit using _either_ $i$ _or_ $j$'s data.

One way of overcoming this obstacle is simple. First, divide the data into $K$ folds (subsets). Then, cycle through the folds, fitting a CATE model on $K-1$ folds. Next, for each held-out fold, _separately_ rank the unseen observations into $Q$ groups based on their prediction  (i.e., if $Q=5$, then we rank observations by estimated CATE into "top quintile", "second quintile", and so on). After concatenating the independent rankings together, we can study the differences in observations in each rank-group.

[This gist](https://gist.github.com/halflearned/bea4e5137c0c81fd18a75f682da466c8) computes the above for `grf`, and it should not be hard to modify it so as to replace forests by any other non-parametric method. However, for `grf` specifically, there's a small trick that allows us to obtain a valid ranking: we can pass a vector of fold indices to the argument `clusters` and rank observations within each fold. This works because estimates for each fold ("cluster")   trees are computed using trees that were not fit using observations from that fold. Here's how to do it.

# Valid randomized data and observational data with unconfoundedness+overlap.
# Note: read the comments below carefully.
# In randomized settings, do not estimate forest.e and e.hat; use known assignment probs.

# Prepare dataset
fmla <- formula(paste0("~ 0 + ", paste0(covariates, collapse="+")))
X <- model.matrix(fmla, data)
W <- data[,treatment]
Y <- data[,outcome]

# Number of rankings that the predictions will be ranking on
# (e.g., 2 for above/below median estimated CATE, 5 for estimated CATE quintiles, etc.)
num.rankings <- 5

# Prepare for data.splitting
# Assign a fold number to each observation.
# The argument 'clusters' in the next step will mimick K-fold cross-fitting.
num.folds <- 10
folds <- sort(seq(n) %% num.folds) + 1

# Comment or uncomment depending on your setting.
# Observational setting with unconfoundedness+overlap (unknown assignment probs):
# forest <- causal_forest(X, Y, W, clusters = folds)
# Randomized settings with fixed and known probabilities (here: 0.5).
forest <- causal_forest(X, Y, W, W.hat=.5, clusters = folds)

# Retrieve out-of-bag predictions.
# Predictions for observation in fold k will be computed using
# trees that were not trained using observations for that fold.
tau.hat <- predict(forest)$predictions

# Rank observations *within each fold* into quintiles according to their CATE predictions.
ranking <- rep(NA, n)
for (fold in seq(num.folds)) {
  tau.hat.quantiles <- quantile(tau.hat[folds == fold], probs = seq(0, 1, by=1/num.rankings))
  ranking[folds == fold] <- cut(tau.hat[folds == fold], tau.hat.quantiles, include.lowest=TRUE,labels=seq(num.rankings))
}

The next snippet computes the average treatment effect within each group defined above, i.e., $\E[Y_i(1) - Y_i(0)|G_i = g]$. This can done in two ways. First, by computing a simple difference-in-means estimate of the ATE based on observations within each group. This is valid only in randomized settings.

# Valid only in randomized settings.
# Average difference-in-means within each ranking

# Formula y ~ 0 + ranking + ranking:w
fmla <- paste0(outcome, " ~ 0 + ranking + ranking:", treatment)
ols.ate <- lm(fmla, data=transform(data, ranking=factor(ranking)))
ols.ate <- coeftest(ols.ate, vcov=vcovHC(ols.ate, type='HC2'))
interact <- which(grepl(":", rownames(ols.ate)))
ols.ate <- data.frame("ols", paste0("Q", seq(num.rankings)), ols.ate[interact, 1:2])
rownames(ols.ate) <- NULL # just for display
colnames(ols.ate) <- c("method", "ranking", "estimate", "std.err")
ols.ate

Another option is to average the AIPW scores within each group. This valid for both randomized settings and observational settings with unconfoundedness and overlap. Moreover, AIPW-based estimators should produce estimates with tighter confidence intervals in large samples.

```


## Subsurface Data Analytics

## Introduction to Spacial Bias: Cell Size's Effect on Declustering Biased Spacial Data

#### Jesse Riedel, Undergraduate Student, The University of Texas at Austin

##### [LinkedIn](https://www.linkedin.com/in/jesse-riedel-12b311163/)

#### Supervised by:

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

### PGE 383 Exercise: Cell Size Manipulation for Declustering Biased Data

Here's a simple workflow that demonstrates first, the concept of declustering in an effort to mitigate the potential bias of spacial data, and second, the effect that the choice of cell size can have on this concept of declustering.

#### Real Life Application

The concept of declustering can be very applicable in the oil and gas industry, or in general, any time there is a bias in gathered data over space. Some examples of use cases for declustering are:

* 3-D declustering data for horizontal or highly deviated wells
* Performing areal 2-D declustering for wells that are vertical or near vertical
* Declustering unconventional well data considering the fact that representativity can be an issue even with large datasets and relatively good coverage

#### Workflow Goals

Learn the basics of declustering and how cell size effects declustering. This includes:

* Loading and visualizing sample data
* Developing a basic understanding of declustering as a strategy to mitigate spatial bias
* Understanding the importance of cell size on declustered weights of spatial data

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - sample_data_biased.csv at https://git.io/fh0CW

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

```python
import geostatspy.GSLIB as GSLIB          # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats    # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda.

```python
import numpy as np                        # ndarrys for gridded data
import pandas as pd                       # DataFrames for tabular data
import os                                 # set working directory, run executables
import matplotlib.pyplot as plt           # for plotting
from scipy import stats                   # summary statistics
```

Lastly, we need to import some packages that will allow us to create interactive functionality.

```python
import ipywidgets as widgets
from IPython.display import display
from ipywidgets import interact, interactive, fixed, interact_manual
```

#### Loading Tabular Data

Here's the command to load our comma delimited data file in to a Pandas' DataFrame object.

```python
df = pd.read_csv('sample_data_biased.csv')     # load our data table (wrong name!)
```

No error now! It worked, we loaded our file into our DataFrame called 'df'. But how do you really know that it worked? Visualizing the DataFrame would be useful and we already leard about these methods in this demo (https://git.io/fNgRW).

We can preview the DataFrame by printing a slice or by utilizing the 'head' DataFrame member function (with a nice and clean format, see below). With the slice we could look at any subset of the data table and with the head command, add parameter 'n=13' to see the first 13 rows of the dataset.

```python
print(df.iloc[0:5,:])                   # display first 4 samples in the table as a preview
df.head(n=13)                           # we could also use this command for a table preview
```

         X    Y  Facies  Porosity       Perm
    0  100  900       1  0.115359   5.736104
    1  100  800       1  0.136425  17.211462
    2  100  600       1  0.135810  43.724752
    3  100  500       0  0.094414   1.609942
    4  100  100       0  0.113049  10.886001

      X
      Y
      Facies
      Porosity
      Perm

      0
      100
      900
      1
      0.115359
      5.736104

      1
      100
      800
      1
      0.136425
      17.211462

      2
      100
      600
      1
      0.135810
      43.724752

      3
      100
      500
      0
      0.094414
      1.609942

      4
      100
      100
      0
      0.113049
      10.886001

      5
      200
      800
      1
      0.154648
      106.491795

      6
      200
      700
      1
      0.153113
      140.976324

      7
      200
      500
      1
      0.126167
      12.548074

      8
      200
      400
      0
      0.094750
      1.208561

      9
      200
      100
      1
      0.150961
      44.687430

      10
      300
      800
      1
      0.199227
      1079.709291

      11
      300
      700
      1
      0.154220
      179.491695

      12
      300
      500
      1
      0.137502
      38.164911

#### Summary Statistics for Tabular Data

The table includes X and Y coordinates (meters), Facies 1 and 2 (1 is sandstone and 0 interbedded sand and mudstone), Porosity (fraction), and permeability as Perm (mDarcy).

There are a lot of efficient methods to calculate summary statistics from tabular data in DataFrames. The describe command provides count, mean, minimum, maximum, and quartiles all in a nice data table. We use transpose just to flip the table so that features are on the rows and the statistics are on the columns.

```python
df.describe().transpose()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      X
      289.0
      475.813149
      254.277530
      0.000000
      300.000000
      430.000000
      670.000000
      990.000000

      Y
      289.0
      529.692042
      300.895374
      9.000000
      269.000000
      549.000000
      819.000000
      999.000000

      Facies
      289.0
      0.813149
      0.390468
      0.000000
      1.000000
      1.000000
      1.000000
      1.000000

      Porosity
      289.0
      0.134744
      0.037745
      0.058548
      0.106318
      0.126167
      0.154220
      0.228790

      Perm
      289.0
      207.832368
      559.359350
      0.075819
      3.634086
      14.908970
      71.454424
      5308.842566

#### Specify the Area of Interest

It is natural to set the x and y coordinate and feature ranges manually. e.g. do you want your color bar to go from 0.05887 to 0.24230 exactly? Also, let's pick a color map for display. I heard that plasma is known to be friendly to the color blind as the color and intensity vary together (hope I got that right, it was an interesting Twitter conversation started by Matt Hall from Agile if I recall correctly). We will assume a study area of 0 to 1,000m in x and y and omit any data outside this area.

```python
xmin = 0.0; xmax = 1000.0               # range of x values
ymin = 0.0; ymax = 1000.0               # range of y values
pormin = 0.05; pormax = 0.25;             # range of porosity values
cmap = plt.cm.plasma
```

#### Declustering Biased Data

Declustering involves assigning a weight to each datum based on closeness to surrounding data. The first step of this process is to divide the volume of interest into a grid of cells, count the occupied cells and the number of datum in each cell. This can be illustrated through the following equation:

## \begin{equation}
w(u_j) = \frac{1}{n_l} \frac{n}{L_0}
\end{equation}

Where:

w = weight

L0 = number of occupied cells

n = total number of data points

nl = number of data in each cell

The following cell contains an augmented version of the declus (declustering) function from the geostats module of GeostatsPy

```python
def declus(df, xcol, ycol, vcol, iminmax, noff, ncell, cmin, cmax):
    """GSLIB's DECLUS program (Deutsch and Journel, 1998) converted from the
    original Fortran to Python by Michael Pyrcz, the University of Texas at
    Austin (Jan, 2019).
    Note this was simplified to 2D only.
    :param df: pandas DataFrame with the spatial data
    :param xcol: name of the x coordinate column
    :param ycol: name of the y coordinate column
    :param vcol: name of the property column
    :param iminmax: 1 / True: for use cell size with max decluster mean
                    0 / False: for declustered mean minimizing cell size
    :param noff: number of offsets
    :param ncell: number of cell sizes
    :param cmin: min cell size
    :param cmax: max cell size
    :return: TODO
    """
    # Load data and set up arrays
    #print(ncell)
    nd = len(df)
    x = df[xcol].values
    y = df[ycol].values
    v = df[vcol].values
    wt = np.zeros(nd)
    wtopt = np.ones(nd)
    index = np.zeros(nd, np.int32)
    xcs_mat = np.zeros(ncell + 2)  # we use 1,...,n for this array
    vrcr_mat = np.zeros(ncell + 2)  # we use 1,...,n for this array
    anisy = 1.0  # hard code the cells to 2D isotropic
    roff = float(noff)

    # Calculate extents
    xmin = np.min(x)
    xmax = np.max(x)
    ymin = np.min(y)
    ymax = np.max(y)

    # Calculate summary statistics
    vmean = np.mean(v)
    vstdev = np.std(v)
    vmin = np.min(v)
    vmax = np.max(v)
    xcs_mat[0] = 0.0
    vrcr_mat[0] = vmean
    vrop = vmean  # include the naive case

    print(f"There are {nd} data with:")
    print(f"   mean of      {vmean} ")
    print(f"   min and max  {vmin} and {vmax}")
    print(f"   standard dev {vstdev} ")

    # Define a "lower" origin to use for the cell sizes
    xo1 = xmin - 0.01
    yo1 = ymin - 0.01

    # Define the increment for the cell size
    xinc = (cmax - cmin) / ncell
    yinc = xinc

    # Loop over "ncell+1" cell sizes in the grid network
    ncellx = int((xmax - (xo1 - cmin)) / cmin) + 1
    ncelly = int((ymax - (yo1 - cmin * anisy)) / cmin) + 1
    #print(ncellx,ncelly)
    ncellt = ncellx * ncelly
    cellwt = np.zeros(ncellt)
    xcs = cmin - xinc
    ycs = (cmin * anisy) - yinc

    # Main loop over cell sizes
    # 0 index is the 0.0 cell, note n + 1 in Fortran
    add_cell = 2
#     if ncell > 1:
#         add_cell = 2

    for lp in range(1, ncell + add_cell):
        xcs = xcs + xinc
        ycs = ycs + yinc

#        print(xcs,ycs)

        # Initialize the weights to zero
        wt.fill(0.0)

        # Determine the maximum number of grid cells in the network
        ncellx = int((xmax - (xo1 - xcs)) / xcs) + 1
        ncelly = int((ymax - (yo1 - ycs)) / ycs) + 1
        ncellt = float(ncellx * ncelly)  # TODO: not used

        # Loop over all the origin offsets selected
        xfac = min((xcs / roff), (0.5 * (xmax - xmin)))
        yfac = min((ycs / roff), (0.5 * (ymax - ymin)))
        for kp in range(1, noff + 1):
            xo = xo1 - (float(kp) - 1.0) * xfac
            yo = yo1 - (float(kp) - 1.0) * yfac

            # Initialize the cumulative weight indicators
            cellwt.fill(0.0)

            # Determine which cell each datum is in
            for i in range(0, nd):
                icellx = int((x[i] - xo) / xcs) + 1
                icelly = int((y[i] - yo) / ycs) + 1
                icell = icellx + (icelly - 1) * ncellx - 1 # -1 for fortran to python
                index[i] = icell
                #print(i,x[i],y[i],icell)
                cellwt[icell] = cellwt[icell] + 1.0

            # The weight assigned to each datum is inversely proportional to the
            # number of data in the cell. We first need to get the sum of
            # weights so that we can normalize the weights to sum to one
            sumw = 0.0
            for i in range(0, nd):
                ipoint = index[i]
                sumw = sumw + (1.0 / cellwt[ipoint])
            sumw = 1.0 / sumw

            # Accumulate the array of weights (that now sum to one)
            for i in range(0, nd):
                ipoint = index[i]
                wt[i] = wt[i] + (1.0 / cellwt[ipoint]) * sumw

        # End loop over all offsets

        # Compute the weighted average for this cell size
        sumw = 0.0
        sumwg = 0.0
        for i in range(0, nd):
            sumw = sumw + wt[i]
            sumwg = sumwg + wt[i] * v[i]
        vrcr = sumwg / sumw
        vrcr_mat[lp] = vrcr
        xcs_mat[lp] = xcs

        # See if this weighting is optimal
        if iminmax and vrcr < vrop or not iminmax and vrcr > vrop or ncell == 1:
            best = xcs  # TODO: not used
            vrop = vrcr
            wtopt = wt.copy()  # deep copy

    # End main loop over all cell sizes

    # Get the optimal weights
    sumw = 0.0
    for i in range(0, nd):
        sumw = sumw + wtopt[i]
    wtmin = np.min(wtopt)  # TODO: not used
    wtmax = np.max(wtopt)  # TODO: not used
    facto = float(nd) / sumw
    wtopt = wtopt * facto
    return wtopt, xcs_mat, vrcr_mat
```

#### Changing Cell Size in Declustering

Now that the we have an implementation for declustering our biased data, we want to see the effect that the cell size parameter has on declustering the data.

In order to do this, we will be using ipywidgets. Ipywidgets offers interactive user interface elements like sliders, dropdown menus, and more. For this interactive workflow, we will be using a slider to change the cell size parameter of the declus (declustering) function.

Lastly, the @interact tag that is given to the interactive_cell_size function below is what allows the cell size slider to change the cell size parameter in the declus function call. This can be also be seen through the parameter of the interactive function being the instance of the slider itself.

The rest of the function below after the declus function call involves creating two plots, the left colored by the actual porosity data, and the right covered by the declustered weights, and how they are adjusted when the user changes the cell size.

```python
@interact
def interactive_cell_size(cell_size = widgets.IntSlider(min = 10, max = 2000, step = 10, description='Cell Size:')):
    wts, cell_sizes, dmeans = declus(df,'X','Y','Porosity',iminmax = 1, noff= 10, ncell=1,cmin=cell_size,cmax=cell_size)

    df['Wts'] = wts

    f = plt.figure()

    f.set_figwidth(16)
    f.set_figheight(6)

    plt.subplot(1,2,1)
    GSLIB.locmap_st(df,'X','Y','Porosity',xmin,xmax,ymin,ymax,pormin,pormax,'Well Data - Porosity','X(m)','Y(m)','Porosity (fraction)',cmap)
    plt.subplot(1,2,2)
    GSLIB.locmap_st(df,'X','Y','Wts',xmin,xmax,ymin,ymax,0.5,2.5,'Well Data Weights','X(m)','Y(m)','Weights',cmap)

    plt.tight_layout()
```

    interactive(children=(IntSlider(value=10, description='Cell Size:', max=2000, min=10, step=10), Output()), _do…

As can be seen by the Well Data Weights plot on the right, when the cell size is adjusted, the weights of data points, illustrated by the color gradient shown on the right axis, change heavily.

#### Declustered Porosity Mean vs. Cell Size

The following portion of the workflow will give another perspective of the effect of declustering spatial data.

We will first call the declus function again, but this time we will set the cell sizes to vary between a wide range from 10m to 2,000m. This will allow the geostats.declus function to take the cell size that minimizes the declustered mean, denoted by the 'iminmax' parameter being set to 1 (setting this parameter will maximize the declustered mean).

```python
df = pd.read_csv('sample_data_biased.csv')     # load our data table (wrong name!)
df['Wts'] = wts                            # add weights to the sample data DataFrame
wts, cell_sizes, dmeans = geostats.declus(df,'X','Y','Porosity',iminmax = 1, noff= 10, ncell=100,cmin=10,cmax=2000)
```

    There are 289 data with:
       mean of      0.13474387540138408
       min and max  0.058547873 and 0.228790002
       standard dev 0.03767982164385207

We can now see the porosity mean pre-declustering and post-declustering, as well as the summary statistics as a result of the declustering:

```python
por_mean = np.average(df['Porosity'].values)
por_dmean = np.average(df['Porosity'].values,weights=df['Wts'].values)
print('Porosity naive mean is ' + str(round(por_mean,3))+'.')
print('Porosity declustered mean is ' + str(round(por_dmean,3))+'.')
cor = (por_mean-por_dmean)/por_mean
print('Correction of ' + str(round(cor,4)) +'.')

print('\nSummary statistics of the declsutering weights:')
print(stats.describe(wts))
```

    Porosity naive mean is 0.135.
    Porosity declustered mean is 0.121.
    Correction of 0.1002.

    Summary statistics of the declsutering weights:
    DescribeResult(nobs=289, minmax=(0.2819756671865989, 3.984325446814365), mean=0.9999999999999996, variance=0.40927076480664726, skewness=1.9395015754905447, kurtosis=4.287460624778004)

Lastly, we can look at a plot of the declustered porosity mean vs. the declustering cell size. It can be noted that at very large cell sizes the declustered mean is the naive mean.

```python
plt.subplot(111)
plt.scatter(cell_sizes,dmeans, s=30, alpha = 0.2, edgecolors = "black", facecolors = 'red')
plt.xlabel('Cell Size (m)')
plt.ylabel('Declustered Porosity Mean (fraction)')
plt.title('Declustered Porosity Mean vs. Cell Size')
plt.plot([0,2000],[por_mean,por_mean],color = 'black')
plt.plot([200,200],[0.10,0.16],color = 'black',linestyle='dashed')
plt.text(300., 0.136, r'Naive Porosity Mean')
plt.text(500., 0.118, r'Declustered Porosity Mean')
plt.text(230., 0.154, r'Minimizing')
plt.text(230., 0.150, r'Cell Size')
plt.ylim(0.10,0.16)
plt.xlim(0,2000)
plt.subplots_adjust(left=0.0, bottom=0.0, right=1.2, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

#### Comments

This was an interactive workflow covering the basics cell size's effect on declustering spatially biased data.

The Texas Center for Geostatistics has many other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, data transformations, trend modeling and many other workflows available [here](https://github.com/GeostatsGuy/PythonNumericalDemos), along with a package for geostatistics in Python called [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy).

We hope this was helpful,

*Jesse* and *Michael*

***

#### More on Michael Pyrcz and the Texas Center for Data Analytics and Geostatistics:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


```
##### Copyright 2021 The Cirq Developers
```

```
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Toric Code Ground State

    View on QuantumAI

    Run in Google Colab

    View source on GitHub

    Download notebook

Before beginning, we will import the necessary modules into the colab.

```
try:
    import recirq
except ImportError:
    !pip install --quiet git+https://github.com/quantumlib/ReCirq
    import recirq

try:
    import qsimcirq
except ImportError:
    !pip install qsimcirq --quiet
    import qsimcirq

import cirq
import matplotlib.pyplot as plt

import recirq.toric_code.toric_code_plaquettes as tcp
import recirq.toric_code.toric_code_plotter as tcplot
import recirq.toric_code.toric_code_rectangle as tcr
import recirq.toric_code.toric_code_state_prep as tcsp

plt.rcParams['figure.dpi'] = 144
```

### Toric code Hamiltonian

The toric code Hamiltonian

\begin{equation}
H = -\sum_s A_s - \sum_p B_p
\end{equation}

involves local four-qubit parity operators, where each qubit lives on an edge in a square lattice. Here, the "star" operators $A_s$ are products of Pauli $Z$ operators around a vertex, while the "plaquette" operators $B_p$ are products of $X$ operators around a square, for example,

\begin{equation}
A_s = Z_i \otimes Z_j \otimes Z_k \otimes Z_l
\end{equation}
\begin{equation}
B_p = X_a \otimes X_b \otimes X_c \otimes X_d.
\end{equation}

These local parity operators all commute with each other: all $A_s$ commute, all $B_p$ commute, and $A_s$ and $B_p$ commute with each other because they overlap on an even number of qubits. They can thus all be simultaneously diagonalized, and those shared eigenstates are also the eigenstates of $H$.

In our paper, we mostly work with the 31-qubit lattice above. With these boundary conditions, there is a _unique_ ground state that has a $+1$ eigenvalue for all $A_s$ and $B_p$. Note for different boundary conditions, we can have _degeneracies_ that are locally-indistinguishable (for example on a torus, or with the "surface code" logical qubits we explore in Figure 4 of our paper).

In this module, we will primarily work with the smaller 22-qubit to avoid time and memory constraints associated with the larger rectangle.

### Understanding the ground state

In this example, we focus on reproducing our first figure, where we create this unique ground state $|G\rangle$ using a shallow unitary circuit. The general idea is to start out with $|0\rangle^{\otimes 22}$, so all $\langle A_s \rangle = +1$. We then apply projection operators $\mathbb{I} + B_p$ which project the state into a $+1$ eigenstate of $B_p$, after which all the local parities are $+1$:

\begin{equation}
|G\rangle \propto \prod_p (\mathbb{I} + B_p)|0\rangle^{\otimes 22}.
\end{equation}

To create this state, we assign a "team captain" qubit to each plaquette $B_p$. Starting from $|0\rangle^{\otimes n}$, we perform a Hadamard on each team captain, and then each team captain is responsible for performing a CNOT to each of its team mates. We have to be careful with the ordering to keep things efficient and avoid the captains stepping on each other's toes. This is easier to visualize for a smaller system, for example the 12-qubit version in Figure S2, reproduced below. Note the superposition of $2^4$ states, as there are four plaquettes $B_p$.

### Creating $|G\rangle$ with ReCirq

#### Basics: 22-qubit circuit

First, we can create a example 22-qubit grid by instantiate it using a `ToricCodeRectangle` object and then plot a visualization using a `ToricCodePlotter` object, both found in the ReCirq repository.

```
short_rectangle = tcr.ToricCodeRectangle(
    origin_qubit=cirq.GridQubit(3, 0), row_vector=(1, 1), rows=2, cols=4
)
plotter = tcplot.ToricCodePlotter()
plotter.plot_code(short_rectangle)
```

We can also see the full circuit of how to create this code (using CNOT gates) using these objects as well.  By printing out the circuit moment by moment, we can see the gates lined up in a visual manner.

```
full_circuit = tcsp.toric_code_cnot_circuit(short_rectangle)
for idx, moment in enumerate(full_circuit):
    print(f'moment {idx}\n{moment}\n')
```

#### Simulating the parities

For a given circuit, we can determine all the parity expectation values $\langle A_s\rangle$ by sampling 22-qubit bitstrings and then computing each expectation value. We do the same thing with for $\langle B_p \rangle$, but we include a layer of Hadamards before measurement to effectively "measure in $X$ basis."

```
def partial_circuit(
    n_moments_to_include: int, *, x_basis: bool
) -> cirq.Circuit:
    """Create the first N moments of a toric in Z or X basis.

    Args:
        n_moments_to_include: number of moments to include
        x_basis: If True, add Hadamards to effectively measure in the X basis.
            If False, measure in the computational (Z) basis.

    Returns: First N moments of a toric code circuit plus an optional
        layer of Hadamard gates to effectively measure in the X basis.
        This circuit also includes measurement gates.
    """
    sliced_circuit = full_circuit[:n_moments_to_include]
    qubits = sorted(short_rectangle.qubits)
    if x_basis:
        sliced_circuit += cirq.Moment(cirq.H.on_each(*qubits))
    return sliced_circuit + cirq.measure(*qubits)

def get_plaquettes(
    n_moment_to_include: int, repetitions: int = 1000,
    sampler: cirq.Sampler = qsimcirq.QSimSimulator()
) -> tcp.ToricCodePlaquettes:
    """Simulates the results in both bases and determine plaquette values.

    Args:
            n_moments_to_include: number of moments to include
            repetitions: number of repetitions (shots) to sample
            sampler: Sampler (simulator) to execute circuits.  Defaults to qsim.
    """
    x_data = sampler.run(
        partial_circuit(n_moment_to_include, x_basis=True), repetitions=repetitions
    )
    z_data = sampler.run(
        partial_circuit(n_moment_to_include, x_basis=False), repetitions=repetitions
    )
    return tcp.ToricCodePlaquettes.from_global_measurements(
        short_rectangle, x_data.data, z_data.data
    )
```

We can step through the circuit one moment at a time to see how the parities $A_s$ and $B_p$ evolve through the circuit. This is similar to Figure 1B in paper (but simulating instead of using experimental data). We begin with $|0\rangle^{\otimes 22}$, which corresponds to `n_moments_to_include=0`. There, all $\langle A_s \rangle = +1$ but $\langle B_p \rangle = 0$ (see colorbars below). The subsequent moments apply Hadamard and CNOT gates to stitch entanglement across the device and create $|G\rangle$.

```
for n in range(len(tcsp.toric_code_cnot_circuit(short_rectangle)) + 1):
    p = get_plaquettes(n)
    ax = plotter.plot_expectation_values(p)
    ax.set_title(f'n_moments_to_include={n}')
    plt.pause(0.001)
```

After the final step, all the parities are $+1$ (see colorbars below), indicating we have successfully created $|G\rangle$.

```
ax_z = plotter.make_colorbar(x_basis=False, orientation='horizontal')
ax_z.set_label(r'Z parity, $\langle A\rangle$')

ax_x = plotter.make_colorbar(x_basis=True, orientation='horizontal')
ax_x.set_label(r'X parity, $\langle B\rangle$')
```


# KVLCC2 prediction woth Simplified ikedas method

# Purpose
State the purpose of the notebook.

# Methodology
Quickly describe assumptions and processing steps.

# WIP - improvements
(WORK IN PROGRESS)
Use this section only if the notebook is not final.

Notable TODOs:
* todo 1
* todo 2
* todo 3

## Results
Describe and comment the most important results.

# Suggested next steps
State suggested next steps, based on results obtained in this notebook.

# Setup

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
#plt.style.use('paper')

#import data
import copy
from mdldb.run import Run

from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import rolldecayestimators.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
from src.data import database
from mdldb import tables

```

    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 461 ('figure.figsize   : 5, 3   ## figure size in inches')
    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 462 ('figure.dpi       : 100        ## figure dots per inch')

```python
from rolldecayestimators.simplified_ikeda_class import SimplifiedIkeda
import rolldecayestimators
```

```python
db = database.get_db()
```

```python
sql = """
SELECT * from run
WHERE model_number='M5057-01-A' and test_type='roll decay' and project_number=40178362;
"""
df_rolldecays = pd.read_sql(sql=sql, con=db.engine, index_col='id')

df_rolldecays['ship_speed'].fillna(0, inplace=True)

```

```python
df_rolldecays[['date','ship_speed']]
```

      date
      ship_speed

      id

      21337
      2018-04-03
      0.0

      21338
      2018-04-03
      0.0

      21339
      2018-11-28
      0.0

      21340
      2018-04-04
      15.5

```python
df_rolldecays=df_rolldecays.loc[[21337,21338]].copy()
```

```python
lowpass_filter = LowpassFilterDerivatorTransformer(cutoff=1, minimum_score=0.999)
cutter = CutTransformer(phi_max=np.deg2rad(15), phi_min=np.deg2rad(2), phi1d_start_tolerance=0.015)
offset_transformer = OffsetTransformer()
```

```python
steps = [
    ('filter',lowpass_filter),
    ('cutter', cutter),
#    ('offset_transformer',offset_transformer),
]

preprosessor = Pipeline(steps=steps)
```

```python
def prepare_inputs(run):

    data = {}
    scale_factor=run.model.scale_factor
    rho=1000
    g=9.81
    data['rho']=rho
    data['g']=g

    data['lpp']=run.ship.lpp/scale_factor
    data['TA']=run.loading_condition.TA/scale_factor
    data['TF']=run.loading_condition.TF/scale_factor
    data['beam']=run.ship.beam/scale_factor

    if not run.ship.BKL:
        data['BKL']=0
    else:
        data['BKL']=run.ship.BKL/scale_factor

    if not run.ship.BKB:
        data['BKB']=0
    else:
        data['BKB']=run.ship.BKB/scale_factor

    data['kg']=run.loading_condition.kg/scale_factor
    data['Volume']=run.loading_condition.Volume/scale_factor**3
    data['gm']=run.loading_condition.gm/scale_factor

    if pd.isnull(run.ship_speed):
        data['V']=0
    else:
        data['V']=run.ship_speed*1.852/3.6/np.sqrt(scale_factor)  #[m/s]

    data['KXX']=run.loading_condition.KXX/scale_factor
    data['A0']=run.loading_condition.A0

    return data

```

```python
models = OrderedDict()

for id,row in df_rolldecays.iterrows():
    run = db.session.query(Run).get(int(row.name))
    run = database.load_run(run, save_as_example=False, prefer_hdf5=True)

    data = prepare_inputs(run=run)

    estimator_ikeda_quadratic = IkedaQuadraticEstimator(**data,
                                        verify_input=False, limit_inputs=False)

    preprosessor.fit(X=run.df[['phi']])
    X = preprosessor.transform(run.df[['phi']])

    estimator_ikeda_quadratic.fit(X=X)
    models[id] = estimator_ikeda_quadratic

```

    c:\dev\evaluation\signal_lab\mdl_to_evaluation.py:106: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
      df_.units = units
    c:\dev\evaluation\signal_lab\mdl_to_evaluation.py:106: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
      df_.units = units

```python
for id,model in models.items():
    model.plot_fit()
```

```python
fig,ax=plt.subplots()

for id,model in models.items():

    model.plot_damping(ax=ax)
```

```python
df_models = pd.DataFrame()

for id,model in models.items():

    s = pd.Series(data=model.result_for_database(),name=id)
    s['score']=model.score()
    df_models = df_models.append(s)

```

```python
df_models
```

      B_1
      B_2
      B_44_1
      B_44_2
      B_BK_1
      B_BK_2
      B_E_1
      B_E_2
      B_F_1
      B_F_2
      B_L_1
      B_L_2
      B_W_1
      B_W_2
      B_e
      d
      omega0
      omega0_fft
      phi_start
      phi_stop
      score
      zeta

      21337
      3.623144
      18.662776
      6.783642
      9.944139
      0.0
      0.0
      3.160498
      6.320995
      1.025137
      1.025137
      0.0
      0.0
      2.598007
      2.598007
      9.944139
      0.139148
      2.475575
      2.475575
      0.161181
      0.063006
      0.916215
      0.005456

      21338
      3.562922
      18.662776
      6.983176
      10.403430
      0.0
      0.0
      3.420254
      6.840508
      1.022532
      1.022532
      0.0
      0.0
      2.540390
      2.540390
      10.403430
      0.137739
      2.463009
      2.463009
      0.175318
      0.034732
      0.828774
      0.005338

```python
interesting = ['B_1','B_2','omega0','score']
df_models[interesting].transpose().plot(kind='bar')
```

```python
phi_a = np.deg2rad(10)

B_e = lambdas.B_e_lambda(B_1=df_models['B_1'],
                         B_2=df_models['B_2'],
                         omega0=df_models['omega0'],
                         phi_a = phi_a)
```

```python
g=9.81
rho=1000
scale_factor=run.model.scale_factor
Volume = run.loading_condition.Volume/(scale_factor**3)
beam = run.ship.beam/scale_factor
lambdas.B_hat_lambda(B=B_e, Disp=Volume, beam=beam,g=g, rho=rho)
```

    21337    0.003017
    21338    0.002990
    dtype: float64

```python

```


# Variational Quantum Eigensolver

<em> Copyright (c) 2021 Institute for Quantum Computing, Baidu Inc. All Rights Reserved. </em>

## Overview

It is widely believed that one of the most promising applications of quantum computing in the near future is solving quantum chemistry problems [1-2]. **Variational Quantum Eigensolver** (VQE) is a strong proof to this possibility of studying quantum chemistry with **Noisy Intermediate-Scale Quantum** (NISQ) devices [1-4]. The core task is to solve the ground state of any molecular Hamiltonian $\hat{H}$ by preparing a parametrized wave function ansatz $|\Psi(\boldsymbol\theta)\rangle$ on a quantum computer and adopt classical optimization methods (e.g. gradient descent) to adjust the parameters $\boldsymbol\theta$ to minimize the expectation value $\langle \Psi(\boldsymbol\theta)|\hat{H}|\Psi(\boldsymbol\theta)\rangle$. This approach is based on the **Rayleigh-Ritz variational principle**.

$$
E_0 = \min_{\boldsymbol\theta} \langle \Psi(\boldsymbol\theta)|\hat{H}|\Psi(\boldsymbol\theta)\rangle.
\tag{1}
$$

where $E_0$ denotes the ground state energy. Numerically, it can be understood as finding the smallest eigenvalue $\lambda_{\min}$ of a **discretized** Hamiltonian $H$ (hermitian matrix) and its corresponding eigenvector $|\Psi_0\rangle$. How such a discretization can be done on a classical computer belongs to the art of quantum chemistry and is far beyond the scope of this tutorial. We will discuss this part with a few words in the background section. In general, such a Hamiltonian $H$ is expressed as a weighted sum of Pauli spin operators $\{X,Y,Z\}$ (native to quantum devices) such that this information can be processed on a quantum computer.

$$
H = \sum_k c_k ~ \bigg( \bigotimes_{j=0}^{M-1} \sigma_j^{(k)} \bigg),
\tag{2}
$$

where $\sigma_j^{(k)} \in \{I,X,Y,Z\}$ and $M$ stands for qubit number. We refer this form of Hamiltonian as **Pauli strings**. For example,

$$
H= 0.12~Y_0 \otimes I_1-0.04~X_0\otimes Z_1.
\tag{3}
$$

In the next section, we will provide a brief review on the electronic structure problem which essentially tells us where does the Hamiltonian $H$ come from. For those who are already familiar with this topic or only interested in how to implement VQE on Paddle Quantum, please skip this part and jump into the illustrative example of hydrogen molecule $H_2$.

## Background: the electronic structure problem

In this section, we focus on one of the fundamental problems in quantum chemistry --  **the electronic structure problem**. To be more specific, we are interested in the low lying energy eigenstates of any given molecule. These knowledge could help predict reaction rates and location of stable structures [5]. Suppose a molecule consists of $N_n$ nuclei and $N_e$ electrons, the first quantized (canonical quantization) Hamiltonian operator $\hat{H}_{mol}$ describing the total energy of this molecular system can be written as

$$
\begin{align}
\hat{H}_{\text{mol}} & = -\sum_{i}\frac{\nabla_{R_i}^2}{2M_i} - \sum_{i} \frac{\nabla_{r_i}^2}{2} -\sum_{i,j}\frac{Z_i}{\lvert R_i - r_j\lvert} + \sum_{i,j>i}\frac{Z_iZ_j}{\lvert R_i - R_j\lvert} + \sum_{i, j>i}\frac{1}{\lvert r_i - r_j\lvert},
\tag{4}
\end{align}
$$

where $R_i, M_i,$ and $Z_i$ denote the position, mass and atomic number (the number of protons) of the $i^{th}$ nucleus, and the positions of electrons are $r_i$. The first two sums describe the kinetic energy of nuclei and electrons, respectively. The third sum describes the attractive Coulomb interaction between the positively charged nuclei and the negatively charged electrons. The last two terms represent the repulsive nuclei-nuclei and electron-electron interactions.  Here, the molecular Hamiltonian $\hat{H}_\text{mol}$ is already in atomic units of energy, **Hartree**. 1 Hartree is $[\hbar^2/(m_ee^2a_0^2)] = 27.2$ eV or 630 kcal/mol, where $m_e, e,$ and $a_0$ stand for the mass of electron, charge of electron, and Bohr radius.

**Note:** The spin-orbit interaction and hyperfine interaction are not considered in this picture. They can be treated as perturbations if necessary.

### Born-Oppenheimer approximation

Since the nuclei are much heavier than electrons, the electrons will move much faster than the nuclei. It is reasonable to treat the positions of nuclei as fixed, $R_i =$ constants. This is known as the Born-Oppenheimer approximation by decoupling the behavior of nuclei and electrons in time scale. Consequently, the kinetic energy term of nuclei will disappear and the nuclei-nuclei repulsive interaction term can be viewed as an energy shift (independent of electron positions $r_i$). We could derive the electronic Hamiltonian $\hat{H}_{\text{electron}}$ as

$$
\begin{align}
\hat{H}_{\text{electron}} & =  - \sum_{i} \frac{\nabla_{r_i}^2}{2} -\sum_{i,j}\frac{Z_i}{\lvert R_i - r_j\lvert} + \sum_{i, j>i}\frac{1}{\lvert r_i - r_j\lvert}
\tag{5},
\end{align}
$$

The energy levels of the electrons in the molecule can be found by solving the time independent Schrödinger equation

$$
\hat{H}_{\text{electron}} |\Psi_n \rangle = E_n |\Psi_n \rangle,
\tag{6}
$$

where $n$ stands for the energy level. Notice the electron repulsion terms scale as $N_e(N_e-1)/2$ which means for the Oxygen molecule $O_2$ carrying 16 electrons there will be 120 electron repulsion terms in total! In general, this problem cannot be solved analytically. As Dirac concluded in [Quantum mechanics of many-electron systems](https://royalsocietypublishing.org/doi/10.1098/rspa.1929.0094) [6],

> *The underlying physical laws necessary for the mathematical theory of a large part of physics and the whole of chemistry are thus completely known, and the difficulty is only that the exact application of these laws leads to equations much too complicated to be soluble.*
>
> 																			-- Paul Dirac (1929)

A straightforward numerical approach is discretizing the infinite-dimensional Hilbert space into equidistant grid points where linear algebra guides the whole calculation. Suppose each axis of space is discretized into $k$ points, the $N$-electron (drop the subscript e for simplicity) wave function can be written as [2]

$$
|\Psi \rangle = \sum_{\mathbf{x_1}, \ldots, \mathbf{x_N}} \psi(\mathbf{x_1}, \ldots, \mathbf{x_N}) \mathcal{A}(|\mathbf{x_1}, \ldots, \mathbf{x_N}\rangle).
\tag{7}
$$

where coordinate $|\mathbf{x_j}\rangle = |r_j\rangle |\sigma_j\rangle$ records the spatial location and spin of the $j^{th}$ electron, $|r_j\rangle  = |x_j,y_j,z_j\rangle$ for $j\in \{1,2,\cdots,N\}$, $x_j,y_j,z_j \in \{0,1,\cdots,k-1\}$ and  $\sigma_j \in \{\downarrow,\uparrow\}$ for spin down or up.  There will be $k^{3N}\times 2^{N}$ complex amplitudes in total. Here, $\mathcal{A}$ denotes anti-symmetrization and a consequence of the Pauli exclusion principle (electrons are fermion), and $\psi(\mathbf{x_1}, \mathbf{x_2}, \ldots, \mathbf{x_N})=\langle\mathbf{x_1}, \mathbf{x_2}, \ldots, \mathbf{x_N}|\Psi\rangle$. One can see that storing such a wave function already requires **exponentially growing memory** with respect to the number of electrons $N$. This would make classical simulation methods based on this naive numerical approach intractable for systems size larger than few tens of electrons. Now, the question becomes can we prepare such a wave function $|\Psi\rangle$ directly on a quantum computer and measure the expectation value $E_0$? In the next section, let's take the simplest molecular system -- hydrogen molecule $H_2$ as a concrete example.

**Note:** A detailed review on quantum chemistry and existing classical computational methods are far beyond the scope of this tutorial, we refer the enthusiastic readers to the standard textbooks *'Molecular Electronic-Structure Theory'* [5] by Helgaker and *'Modern Quantum Chemistry: Introduction to Advanced Electronic Structure Theory'* [7] by Szabo & Ostlund. To bridge to knowledge gap between quantum chemistry and quantum computing, please check the following review papers [Quantum computational chemistry](https://journals.aps.org/rmp/abstract/10.1103/RevModPhys.92.015003) [2] and [Quantum Chemistry in the Age of Quantum Computing](https://pubs.acs.org/doi/10.1021/acs.chemrev.8b00803) [1].

**Note:** For energy calculation, it is desired to reach the **chemical accuracy** of $1.6\times10^{-3}$ Hartree or 1 kcal/mol .

## Ground state of the hydrogen molecule $H_2$

### Building electronic Hamiltonian

First of all, let us import the necessary libraries and packages.

```python
import os
import platform
import matplotlib.pyplot as plt
from IPython.display import clear_output

import numpy
from numpy import concatenate
from numpy import pi as PI
from numpy import savez, zeros

import paddle
from paddle_quantum.circuit import UAnsatz
from paddle_quantum.utils import pauli_str_to_matrix
from paddle_quantum.VQE.chemistrysub import H2_generator
```

To analyze specific molecules, we need several key information such as **geometry**, **basis set** (such as STO-3G), **multiplicity**, and **charge** to obtain the discretized Hamiltonian $H$. Specifically, through our built-in quantum chemistry toolkit, fermion-to-qubit mapping technology can be used to output the qubit Hamiltonian of hydrogen molecule $H_2$,

```python
Hamiltonian, N = H2_generator()
```

For more advanced users, we provide a simple tutorial on how to generate such a Hamiltonian. Install the following two packages first (**only available for Mac/Linux users, not available to Windows users temporarily**):

```python
!pip install openfermion
clear_output()
```

```python
!pip install openfermionpyscf
clear_output()
```

```python
# Operating system information
sysStr = platform.system()

# Decide which operating system the user is using
if sysStr in ('Linux', 'Darwin'):

    import openfermion
    import openfermionpyscf

    # Please check whether the geometric configuration file of h2 is downloaded correctly
    geometry = 'h2.xyz'
    # geometry = [('H', (0.0, 0.0, 0.0)), ('H', (0.0, 0.0, 0.74))]
    basis = 'sto-3g'
    charge = 0
    multiplicity = 1

    # Generate Hamiltonian
    molecular_hamiltonian = openfermionpyscf.generate_molecular_hamiltonian(geometry, basis, multiplicity, charge)
    qubit_op = openfermion.transforms.jordan_wigner(molecular_hamiltonian)

    # Print result
    print("The generated h2 Hamiltonian is \n", qubit_op)
```

    The generated h2 Hamiltonian is
     -0.042078976477822494 [] +
    -0.04475014401535163 [X0 X1 Y2 Y3] +
    0.04475014401535163 [X0 Y1 Y2 X3] +
    0.04475014401535163 [Y0 X1 X2 Y3] +
    -0.04475014401535163 [Y0 Y1 X2 X3] +
    0.17771287465139946 [Z0] +
    0.17059738328801055 [Z0 Z1] +
    0.12293305056183797 [Z0 Z2] +
    0.1676831945771896 [Z0 Z3] +
    0.17771287465139946 [Z1] +
    0.1676831945771896 [Z1 Z2] +
    0.12293305056183797 [Z1 Z3] +
    -0.24274280513140462 [Z2] +
    0.1762764080431959 [Z2 Z3] +
    -0.24274280513140462 [Z3]

**Note:** This Hamiltonian is generated with an interatomic distance of $d = 74$ pm.

In addition to hydrogen molecule $H_2$, we also provide the geometric configuration file of hydrogen fluoride (HF) molecule `hf.xyz`. If you need to test the geometric configuration of more molecules, please check out this [database](http://smart.sns.it/molecules/index.html). In addition, we also need to convert the Hamiltonian into the Pauli string format supported by Paddle Quantum. Here we provide this interface.

```python
def Hamiltonian_str_convert(qubit_op):
    '''
    Convert the Hamiltonian information provided above into Pauli strings supported by Paddle Quantum

    H = [[1.0, "z0,x1"], [-1.0, "y0,z1"], ...]
    '''
    info_dic = qubit_op.terms

    def process_tuple(tup):
        if len(tup) == 0:
            return 'i0'
        else:
            res = ''
            for ele in tup:
                res += ele[1].lower()
                res += str(ele[0])
                res += ','
            return res[:-1]
    H_info = []

    for key, value in qubit_op.terms.items():
        H_info.append([value.real, process_tuple(key)])

    return H_info

if sysStr in ('Linux', 'Darwin'):
    Hamiltonian = Hamiltonian_str_convert(qubit_op)
```

### Building QNN and trial wave function

To implement VQE, we firstly need to design a quantum neural network QNN to prepare the wave function ansatz $|\Psi(\boldsymbol\theta)\rangle$. Here, we provide a 4-qubit quantum circuit template with a depth of $D$ blocks. The dotted frame in the figure below denotes a single block:

Next, we use the `UAnsatz` class and the built-in `real_entangled_layer(theta, D)` circuit template in Paddle Quantum to realize this QNN.

```python
def U_theta(theta, Hamiltonian, N, D):
    """
    Quantum Neural Network
    """

    # Initialize the quantum neural network according to the number of qubits N
    cir = UAnsatz(N)

    # Built-in {R_y + CNOT} circuit template
    cir.real_entangled_layer(theta[:D], D)

    # Lay R_y gates in the last row
    for i in range(N):
        cir.ry(theta=theta[D][i][0], which_qubit=i)

    # The quantum neural network acts on the default initial state |0000>
    cir.run_state_vector()

    # Calculate the expected value of a given Hamiltonian
    expectation_val = cir.expecval(Hamiltonian)

    return expectation_val, cir
```

### Setting up the loss function and model

Now that we have the target Hamiltonian and QNN, we will further define the training model and loss function. By applying the QNN $U(\theta)$ on the initial state $|0..0\rangle$, we get the output state $|\psi(\boldsymbol\theta)\rangle $. Then, the loss function to be minimized is the expectation value,

$$
\min_{\boldsymbol\theta}  \mathcal{L}(\boldsymbol \theta) = \min_{\boldsymbol\theta} \langle \Psi(\boldsymbol\theta)|H |\Psi(\boldsymbol\theta)\rangle
= \min_{\boldsymbol\theta} \sum_k c_k~\langle \Psi(\boldsymbol\theta)| \bigotimes_j \sigma_j^{(k)}|\Psi(\boldsymbol\theta)\rangle.
\tag{8}
$$

```python
class StateNet(paddle.nn.Layer):

    def __init__(self, shape, dtype="float64"):
        super(StateNet, self).__init__()

        # Initialize the theta parameter list and fill the initial value with a uniform distribution of [0, 2*pi]
        self.theta = self.create_parameter(shape=shape,
                                           default_initializer=paddle.nn.initializer.Uniform(low=0.0, high=2*PI),
                                           dtype=dtype, is_bias=False)

    # Define loss function and forward propagation mechanism
    def forward(self, N, D):

        # Calculate the loss function/expected value
        loss, cir = U_theta(self.theta, Hamiltonian, N, D)

        return loss, cir
```

### Hyper-parameters

Before training the QNN, we also need to set some training hyper-parameters, mainly the learning rate (LR), the number of iterations (ITR), and the depth (D) of repeated blocks.

```python
ITR = 80  # Set the number of optimization iterations
LR = 0.4   # Set the learning rate
D = 2      # Set the depth of the repetitive calculation module in QNN
```

### Training

After all the training model parameters are set, we convert the data into Tensor in the Paddle, and then train the quantum neural network. The results of the training process is stored in the summary_data file.

```python
# Determine the parameter dimension of the network
net = StateNet(shape=[D + 1, N, 1])

# Generally speaking, we use Adam optimizer to obtain relatively good convergence,
# You can change it to SGD or RMS prop.
opt = paddle.optimizer.Adam(learning_rate=LR, parameters=net.parameters())

# Record optimization results
summary_iter, summary_loss = [], []

# Optimization loop
for itr in range(1, ITR + 1):

    # Forward propagation to calculate loss function
    loss, cir = net(N, D)

    # Use back propagation to minimize the loss function
    loss.backward()
    opt.minimize(loss)
    opt.clear_grad()

    # Record optimization results
    summary_loss.append(loss.numpy())
    summary_iter.append(itr)

    # Print result
    if itr % 20 == 0:
        print("iter:", itr, "loss:", "%.4f" % loss.numpy())
        print("iter:", itr, "Ground state energy:", "%.4f Ha"
                                            % loss.numpy())
    if itr == ITR:
        print("\nThe trained circuit:")
        print(cir)

# Save the training results to the output folder
os.makedirs("output", exist_ok=True)
savez("./output/summary_data", iter = summary_iter,
                               energy=summary_loss)
```

    iter: 20 loss: -1.0687
    iter: 20 Ground state energy: -1.0687 Ha
    iter: 40 loss: -1.1313
    iter: 40 Ground state energy: -1.1313 Ha
    iter: 60 loss: -1.1355
    iter: 60 Ground state energy: -1.1355 Ha
    iter: 80 loss: -1.1360
    iter: 80 Ground state energy: -1.1360 Ha

    The trained circuit:
    --Ry(1.557)----*--------------X----Ry(1.578)----*--------------X----Ry(3.142)--
                   |              |                 |              |
    --Ry(1.571)----X----*---------|----Ry(4.916)----X----*---------|----Ry(1.390)--
                        |         |                      |         |
    --Ry(2.972)---------X----*----|----Ry(7.847)---------X----*----|----Ry(3.104)--
                             |    |                           |    |
    --Ry(4.663)--------------X----*----Ry(1.577)--------------X----*----Ry(3.147)--

### Benchmarking
We have now completed the training of the quantum neural network, and the estimated value of the ground state energy obtained is $E_0 \approx -1.1361$ Hartree, we compare it with the theoretical value $E_0 = -1.13618$ to benchmark our model. The estimation obtained with VQE agree with a full configuration-interaction (FCI) calculation within chemical accuracy $\varepsilon = 1.6 \times 10^{-3}$ Hartree.

```python
result = numpy.load('./output/summary_data.npz')

eig_val, eig_state = numpy.linalg.eig(
                     pauli_str_to_matrix(Hamiltonian, N))
min_eig_H = numpy.min(eig_val.real)
min_loss = numpy.ones([len(result['iter'])]) * min_eig_H

plt.figure(1)
func1, = plt.plot(result['iter'], result['energy'],
                  alpha=0.7, marker='', linestyle="-", color='r')
func_min, = plt.plot(result['iter'], min_loss,
                  alpha=0.7, marker='', linestyle=":", color='b')
plt.xlabel('Number of iteration')
plt.ylabel('Energy (Ha)')

plt.legend(handles=[
    func1,
    func_min
],
    labels=[
        r'$\left\langle {\psi \left( {\theta } \right)} '
        r'\right|H\left| {\psi \left( {\theta } \right)} \right\rangle $',
        'Ground-state energy',
    ], loc='best')

#plt.savefig("vqe.png", bbox_inches='tight', dpi=300)
plt.show()
```

## Determining the interatomic distance

Recall the above calculation is done with an interatomic distance $d = 74$ pm between two hydrogen atoms. Another interesting aspect we can try with VQE is determining the true interatomic distance by modifying the `h2.xyz` file. The results are summarize in figure below,

The lowest value is found around $d = 74$ pm (1 pm = $1\times 10^{-12}$m), which is consistent with the [experimental data](https://cccbdb.nist.gov/exp2x.asp?casno=1333740&charge=0) $d_{exp} (H_2) = 74.14$ pm.

_______

## References

[1] Cao, Yudong, et al. Quantum Chemistry in the Age of Quantum Computing. [Chemical reviews 119.19 (2019): 10856-10915.](https://pubs.acs.org/doi/10.1021/acs.chemrev.8b00803)

[2] McArdle, Sam, et al. Quantum computational chemistry. [Reviews of Modern Physics 92.1 (2020): 015003.](https://journals.aps.org/rmp/abstract/10.1103/RevModPhys.92.015003)

[3] Peruzzo, A. et al. A variational eigenvalue solver on a photonic quantum processor. [Nat. Commun. 5, 4213 (2014).](https://www.nature.com/articles/ncomms5213)

[4] Moll, Nikolaj, et al. Quantum optimization using variational algorithms on near-term quantum devices. [Quantum Science and Technology 3.3 (2018): 030503.](https://iopscience.iop.org/article/10.1088/2058-9565/aab822)

[5] Helgaker, Trygve, Poul Jorgensen, and Jeppe Olsen. Molecular electronic-structure theory. John Wiley & Sons, 2014.

[6] Dirac, Paul Adrien Maurice. Quantum mechanics of many-electron systems. [Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 123.792 (1929): 714-733.](https://royalsocietypublishing.org/doi/10.1098/rspa.1929.0094)

[7] Szabo, Attila, and Neil S. Ostlund. Modern quantum chemistry: introduction to advanced electronic structure theory. Courier Corporation, 2012.