# Longitudinal Data Analysis

```stata

```

# Panel Data Analysis III

Table of Contents

1  Quick reminder

2  Defining our statistical model

2.1  Decomposition implications

2.2  A word of caution

3  Fixed Effects Model

3.1  Conceptualising the Fixed Effects model

3.2  Estimation

3.3  Interpretation

3.4  Post-estimation

3.5  Benefits of Fixed Effects

3.6  Limitations of Fixed Effects

3.7  Summarising the Fixed Effects model

4  Summary

In this section we estimate a statistical model that leverages some of the main advantages of using panel data: **Fixed Effects**. We show some examples of how to estimate and interpret this model, and reflect on the conditions under which the model is appropriate.

## Quick reminder

Let's briefly recap some essential concepts regarding panel data:

Two sources of variation ([Gould, n.d.](https://www.stata.com/support/faqs/statistics/between-estimator/)):
1. Cross-section information on differences between units
2. Time series information on differences over time within units

So far our panel data models — Pooled OLS and Between Effects — only allow us to examine differences between units.

Two main issues with estimating statistical models:
1. Interdependence of errors
2. Improper model specification

The first can lead to inefficient estimates: under-estimated standard errors and false positive tests of statistical significance.

The second to biased coefficients and incorrect inferences regarding magnitude and direction of effect of explanatory variables.

Therefore we need a statistical model that allows us to **examine change over time** and/or **control for omitted variable bias**.

## Defining our statistical model

Before estimating Fixed Effects and Random Effects models separately, it is worth identifying the key commonality between their respective statistical models.

Let's take a simplified version of our charity income statistical model, this time with only one explanatory variable (*age*) - typically it looks as follows:

\begin{equation} \text{y}_{it} = \beta_0 + \beta_1x_{1it} + \epsilon_{it} \tag{1.2} \end{equation}

However it is possible to **decompose** the residual variation (error term) into two separate terms:

\begin{equation} \text{y}_{it} = \beta_0 + \beta_1x_{1it} + \mu_{i} + \text{e}_{it} \tag{1.3} \end{equation}

In equation 1.3. we have introduced a **unit-specific** term to represent some of the residual variation in the outcome that is unexplained by the explanatory variables.

### Decomposition implications

This term ($\mu_{i}$) captures the effect of *residual heterogeneity* on the outcome i.e., unobserved or immeasurable characteristics of the units that are associated with the outcome (and possibly the explanatory variables), and vary across units.

In our charity data example, these charity-specific effects could be organisational culture, informal connections to government etc. In theory these characteristics could be measured but it's often wildly impractical.

It also controls for the effect of other omitted variables on the outcome (and possibly the explanatory variables).

In our charity data example, we do not include explanatory variables capturing the amount a charity spends on fundraising, how well known it is etc.

### A word of caution

Note the lack of a time subscript *t* in the new term $\mu_{i}$.

The implication is that the unobserved unit-specific effect is **constant** over time (i.e., within units).

Therefore Fixed Effects and Random Effects models only control for omitted variables that do not change within units (e.g., race, sex at birth, natural ability).

## Fixed Effects Model

### Conceptualising the Fixed Effects model

1. The Fixed Effects model focuses on how changes in explanatory variables are associated with changes in the outcome **within units**.

2. It assumes the observed explanatory variables and unobserved unit-specific effect are correlated (i.e., omitted variable bias is an issue).

Mehmetoglu and Jakobsen (2016, p. 241):
> "In other words, we use fixed effects whenever we are only interested in the impact of variables that vary over time. This estimator helps us explore the relationship between the dependent and the explanatory variables within a unit (person, company, country, etc.) Each unit has its own individual characteristics that may or may not influence the predictor variables."

The Fixed Effects model is specified as follows:

\begin{equation} \text{y}_{it} = \beta_0 + \lambda_{i} + \beta_1x_{1it} +...+ \beta_kx_{kit} + \text{e}_{it} \tag{1.4} \end{equation}

Where:

$\lambda_{i}$ represents the unit-specific effect on the outcome.

The value of $\lambda_{i}$ captures the effect of **all** of the unobserved time-invariant explanatory variables that are missing from the model.

As a result, while the value of $\lambda_{i}$ is calculated, it is not of much interest in and of itself.

It's main role is to allow for a more robust (i.e., unbiased) estimation of the effects of the explanatory variables in the model.

In essence the Fixed Effects model produces a unit-specific intercept, which is the sum of the overall constant and the unit-specific effect:

\begin{equation} \text{y}_{it} = \alpha_{i} + \beta_1x_{1it} +...+ \beta_kx_{kit} + \text{e}_{it} \tag{1.5} \end{equation}

Where:

$\alpha_{i} = \beta_0 + \lambda_{i}$

The unit-specific effect shifts the overall intercept up or down the y axis by the value of $\lambda$.

**QUESTION**

Why is the unobserved unit-specific effect incorporated into the constant?

Two reasons:
* It is not of interest in and of itself (who cares what the effect is of being you?)
* It doesn't vary within a unit: remember we are only modelling within-unit variation and the unit-specific effect does not vary within a unit, therefore it does not make sense to interpret the unit-specific term like an explanatory variable.

#### Final thoughts on conceptualisation

Consider it a standard cross-sectional regression model with the addition of a dummy variable being included for every unit in the panel except for one (i.e., *n - 1* dummy variables are added to the model).

### Estimation

```stata
use "../data/charity-panel-analysis-2020-09-10.dta", clear
```

    (Contains annual accounts of charities in E&W for financial years 2006-2017)

```stata
xtreg linc orgage localc west genchar nsources govern_share, fe
```

    note: localc omitted because of collinearity
    note: west omitted because of collinearity
    note: genchar omitted because of collinearity

    Fixed-effects (within) regression               Number of obs     =     23,826
    Group variable: regno                           Number of groups  =      2,166

    R-sq:                                           Obs per group:
         within  = 0.0140                                         min =         11
         between = 0.0425                                         avg =       11.0
         overall = 0.0403                                         max =         11

                                                    F(3,21657)        =     102.28
    corr(u_i, Xb)  = -0.1002                        Prob > F          =     0.0000

    ------------------------------------------------------------------------------
            linc |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
    -------------+----------------------------------------------------------------
          orgage |   .0069072   .0005802    11.90   0.000       .00577    .0080444
          localc |          0  (omitted)
            west |          0  (omitted)
         genchar |          0  (omitted)
        nsources |   .0289886   .0027931    10.38   0.000     .0235139    .0344633
    govern_share |   .0010325   .0001225     8.43   0.000     .0007923    .0012727
           _cons |   14.71504    .026082   564.18   0.000     14.66392    14.76616
    -------------+----------------------------------------------------------------
         sigma_u |  .94534636
         sigma_e |   .2821005
             rho |  .91823289   (fraction of variance due to u_i)
    ------------------------------------------------------------------------------
    F test that all u_i=0: F(2165, 21657) = 120.80               Prob > F = 0.0000

**QUESTION TIME**

1. How much of the variation in the outcome is accounted for by the model? Is this a lot?
2. Why were three of the observed explanatory variables excluded in the estimation of the model?
3. What does the $\text{rho}$ statistic tell us?
4. Is there evidence of correlation between the unit-specific effects and observed explanatory variables?

### Interpretation

The effect of the observed explanatory variables is **net of** the effect of the unit-specific term. That is, we've controlled away the correlation between *X* and $\mu_{i}$.

$\text{_cons}$ is the intercept and represents the average value of the fixed effects + the overall constant.

$\text{orgage}$ is the predicted change in the outcome for a one-unit increase in organisational age.

$\text{rho}$ is the proportion of unexplained variance in the outcome explained by unobserved differences between charities (the unit-specific effects), rather than changes within them.

If $\text{rho}$ > .5 then most of the residual variation in the outcome is due to differences between units, if $\text{rho}$ < .5 then most of the residual variation is accounted for by differences within units (i.e., the effects of the explanatory variables).

$\text{corr(u_i, Xb)}$ is the correlation between the unit-specific effect and the observed explanatory variables in the model.

* $\text{sigma_u}$ (or $\sigma_u$) is the standard deviation of residuals within units.
* $\text{sigma_e}$ (or $\sigma_e$) is the standard deviation of residuals ei.
*
* $\text{R-sq: within}$ is the proportion of variance explained by the observed explanatory variables (i.e., excluding the unit-specific effect).

### Post-estimation

Though it's very rarely of substantive interest, we can recover the unit-specific effects (and other parameter estimates) after estimating a Fixed Effects model:

```stata
capture predict fixed, u
capture predict y_hat, xb
capture predict ei, e
capture predict residuals, ue
capture egen pickone = tag(regno)
```

```stata
l regno fin_year fixed if pickone in 1/100
```

         +-------------------------------+
         |  regno   fin_year       fixed |
         |-------------------------------|
      1. | 200048    2006-07   -.9911593 |
     12. | 200051    2006-07    1.341765 |
     23. | 200069    2006-07   -.4608771 |
     34. | 200222    2006-07   -.1173254 |
     45. | 200424    2006-07   -.4077182 |
         |-------------------------------|
     56. | 200431    2006-07   -.5248324 |
     67. | 200500    2006-07   -.0236679 |
     78. | 201081    2006-07    .0432656 |
     89. | 201321    2006-07   -1.400211 |
    100. | 201911    2006-07   -.7076412 |
         +-------------------------------+

```stata
l regno fin_year linc y_hat residuals fixed ei in 1/11
```

         +-----------------------------------------------------------------+
      1. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2006-07 | 14.00189 | 15.17772 | -1.175828 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                            -.1846687                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      2. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2007-08 | 14.17788 | 15.18462 | -1.006747 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                            -.0155877                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      3. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2008-09 |  14.1851 | 15.22075 |  -1.03565 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                            -.0444904                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      4. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2009-10 |  14.2326 | 15.19844 | -.9658405 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                             .0253188                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      5. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2010-11 |  14.1709 | 15.20695 | -1.036052 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                            -.0448926                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      6. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2011-12 | 14.14801 | 15.21225 |  -1.06424 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                            -.0730808                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      7. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2012-13 |   14.376 | 15.25097 | -.8749701 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                             .1161892                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      8. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2013-14 | 14.29996 | 15.22607 | -.9261075 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                             .0650518                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
      9. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2014-15 | 14.26031 |  15.2623 | -1.001989 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                            -.0108296                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
     10. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2015-16 | 14.30113 | 15.23988 | -.9387508 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                             .0524085                            |
         +-----------------------------------------------------------------+

         +-----------------------------------------------------------------+
     11. |  regno | fin_year |     linc |    y_hat | residuals |     fixed |
         | 200048 |  2016-17 | 14.37021 | 15.24679 | -.8765777 | -.9911593 |
         |-----------------------------------------------------------------|
         |                                   ei                            |
         |                             .1145816                            |
         +-----------------------------------------------------------------+

```stata
di -.9911593 + -.1846687
```

    -1.175828

```stata
tabstat fixed ei, s(mean sd) format(%5.4f)
```

       stats |     fixed        ei
    ---------+--------------------
        mean |   -0.0000   -0.0000
          sd |    0.9451    0.2690
    ------------------------------

### Benefits of Fixed Effects

Analyse change over time [Substantive]

Control for residual heterogeneity [Methodological]

(Mehmetoglu and Jakobsen, 2016)

Coefficient estimates are **consistent** if the key assumption is true.

That is, because we have controlled for the effect of unobserved time-invariant explanatory variables, our coefficients are more robust, which means increasing the sample size increases the likelihood the estimates are converging on their true values.

### Limitations of Fixed Effects

Ignores differences between units [Substantive]

Coefficient estimates are **inefficient**, especially when compared to those from a Random Effects model. As a result, standard errors tend to be larger.

Put simply, the estimates of the coefficients are based on only one source of variation (within) and thus are more uncertain.

Cannot include observed time-invariant explanatory variables [Methodological]

This is due to a very simple reason: if a value does not vary, how can it be associated with variation in the value of another variable?

It is not well suited for variables that rarely change within units.

Think carefully about variables that change little over time - how might these influence the outcome? For example, few individuals in your panel might switch from non-graduate to graduate (let's say you have a sample of older individuals). In a fixed effects model, your estimation of the effect of switching between non-graduate and graduate will be based on a small number of occurrences and care is due in interpreting the coefficient.

Cannot control for unobserved residual heterogeneity that varies over time [Methodological]

Educational ability? Natural resilience?

The last point is worth expanding on: if units differ in an unobserved way that varies over time, this will not be controlled for in the Fixed Effects model.

### Summarising the Fixed Effects model

Focuses on change over time within a unit of analysis.

Can control for the effect of unobserved time-invariant explanatory variables (residual heterogeneity).

Provides robust estimates of observed explanatory variables when said variables are correlated with unobserved effects.

However cannot include observed explanatory variables that do not vary within units.

## Summary

Both the Pooled OLS and Between Effects models provide useful information on the association between an outcome *Y* and a set of explanatory variables *X*.

Fixed Effects provide potentially different information on the association between an outcome *Y* and a set of explanatory variables *X.

Is there a way to combine the *within* and *between* perspectives?


Here are some points to remember:

• Objects share operations according to their category; for instance, strings, lists, and tuples all share sequence operations such as concatenation, length, and
indexing.

• Only mutable objects (lists, dictionaries, and sets) may be changed in-place; you
cannot change numbers, strings, or tuples in-place.

• Files export only methods, so mutability doesn’t really apply to them—their state
may be changed when they are processed, but this isn’t quite the same as Python core type mutability constraints.

• “Numbers” in Table 9-3 includes all number types: integer (and the distinct long
integer in 2.6), floating-point, complex, decimal, and fraction.

• “Strings” in Table 9-3 includes str, as well as bytes in 3.0 and unicode in 2.6; the byte array string type in 3.0 is mutable.

• Sets are something like the keys of a valueless dictionary, but they don’t map to
values and are not ordered, so sets are neither a mapping nor a sequence type; frozenset is an immutable variant of set.

• In addition to type category operations, as of Python 2.6 and 3.0 all the types in Table 9-3 have callable methods, which are generally specific to their type

```python
X = [1,2,3]
L = ['a',X,'b']  # Embed references to X's object
D = {'x':X,'y':2}
```

```python
print(X,L,D)
```

    [1, 2, 3] ['a', [1, 2, 3], 'b'] {'x': [1, 2, 3], 'y': 2}

```python
X[1] = 'surprise'  # Changes all three references!
```

```python
L, D
```

    (['a', [1, 'surprise', 3], 'b'], {'x': [1, 'surprise', 3], 'y': 2})

```python
from wordcloud import WordCloud
import PIL.Image as image

with open("output.txt",'r',encoding='utf-8') as fp:
          text=fp.read()
          #print(text)
          #将文本放入WordCoud容器对象中并分析
          WordCloud = WordCloud(width=1000,height=800).generate(text)
          image_produce = WordCloud.to_image()
          image_produce.show()
```

```python
from sympy import plot_implicit, sin, symbols, pi
x, y = symbols('x y')
my_plot = plot_implicit(x**2 + y**2 <= 1 + 0.5 * sin((x**3 + y**3) * 2 * pi))
```

```python
from __future__ import division
a,b,c = symbols('a b c')
plot2 = plot_implicit( a**2 + b**2 <= 0)
```

```python
(a**2+b**2)/(2*c) + (b**2+c**2)/(2*a) + (c**2+a**2)/(2*b)
```

$\displaystyle \frac{a^{2} + b^{2}}{2 c} + \frac{a^{2} + c^{2}}{2 b} + \frac{b^{2} + c^{2}}{2 a}$

```python
a + b + c
```

$\displaystyle a + b + c$

```python
L = [1,2,3]
D = {'a': 1, 'b': 2}
```

```python
A = L[:] # Instead of A = L (or list(L))
B = D.copy() # Instead of B = D (ditto for sets)
```

```python
A[1] = "Ni"
B['c'] = 'spam'
L, D
```

    ([1, 2, 3], {'a': 1, 'b': 2})

```python
A, B
```

    ([1, 'Ni', 3], {'a': 1, 'b': 2, 'c': 'spam'})

```python
L = [1,2,3]
D = {'a': 1, 'b': 2}
A = L
B = D
A[1] = "Ni"
B['c'] = 'spam'
print(L,D)
print(A,B)
```

    [1, 'Ni', 3] {'a': 1, 'b': 2, 'c': 'spam'}
    [1, 'Ni', 3] {'a': 1, 'b': 2, 'c': 'spam'}

```python
# Case 1
X = [1,2,3]
L = ['a',X[:],'b'] # Embed copies of X's object
D = {'x': X[:],'y':2}
X[1] = 'surprise' # It doesn't change all three references!
```

```python
print(X,L,D)
```

    [1, 'surprise', 3] ['a', [1, 2, 3], 'b'] {'x': [1, 2, 3], 'y': 2}

```python
# case 2
X = [1,2,3]
L = ['a',X,'b']
D = {'x': X, 'y': 2}
X[1] = 'surprise'
```

```python
print(X,L,D)
```

    [1, 2, 3] ['a', [1, 2, 3], 'b'] {'x': [1, 2, 3], 'y': 2}

• Slice expressions with empty limits (L[:]) copy sequences.

• The dictionary and set copy method (X.copy()) copies a dictionary or set.

• Some built-in functions, such as list, make copies (list(L)).

• The copy standard library module makes full copies.

Empty-limit slices and the dictionary copy method only make top-level copies; that is, they do not copy nested data structures, if any are present. If you need a complete, fully independent copy of a deeply nested data structure, use the standard copy module: include an import copy statement and say X = copy.deepcopy(Y) to fully copy an arbitrarily nested object Y. This call recursively traverses objects
to copy all their parts.

```python
E = {'x': X, 'y': {str(X): X}}
```

```python
E
```

    {'x': [1, 2, 3], 'y': {'[1, 2, 3]': [1, 2, 3]}}

```python
F = E.copy()
```

```python
F
```

    {'x': [1, 2, 3], 'y': {'[1, 2, 3]': [1, 2, 3]}}

```python
from sympy import symbols
import sympy
n = symbols('n')
```

```python
from itertools import product
T = product(k,(k,1,1000))
```

```python
from sympy import plot_implicit, sin, symbols, pi
import numpy

x, y = symbols('x y')
my_plot1 = plot_implicit(3*x**2 <= 6-2*y**2)
my_plot2 = plot_implicit(abs(2*x+y)<=numpy.sqrt(11))
```

```python
from chempy import Substance
ferricyanide = Substance.from_formula('Fe(CN)6-3')
ferricyanide.composition == {0: -3, 26: 1, 6: 6, 7: 6}  # 0 for charge
```

    True

```python
print(ferricyanide.unicode_name)
print(ferricyanide.latex_name + ", " + ferricyanide.html_name)
print('%.3f' % ferricyanide.mass)
```

    Fe(CN)₆³⁻
    Fe(CN)_{6}^{3-}, Fe(CN)63-
    211.955

```python
from chempy import balance_stoichiometry  # Main reaction in NASA's booster rockets:
reac, prod = balance_stoichiometry({'NH4ClO4', 'Al'}, {'Al2O3', 'HCl', 'H2O', 'N2'})
from pprint import pprint
pprint(dict(reac))
pprint(dict(prod))
from chempy import mass_fractions
for fractions in map(mass_fractions, [reac, prod]):
    pprint({k: '{0:.3g} wt%'.format(v*100) for k, v in fractions.items()})
```

    {'Al': 10, 'NH4ClO4': 6}
    {'Al2O3': 5, 'H2O': 9, 'HCl': 6, 'N2': 3}
    {'Al': '27.7 wt%', 'NH4ClO4': '72.3 wt%'}
    {'Al2O3': '52.3 wt%', 'H2O': '16.6 wt%', 'HCl': '22.4 wt%', 'N2': '8.62 wt%'}

```python
substances = {s.name: s for s in [
    Substance('pancake', composition=dict(eggs=1, spoons_of_flour=2, cups_of_milk=1)),
    Substance('eggs_6pack', composition=dict(eggs=6)),
    Substance('milk_carton', composition=dict(cups_of_milk=4)),
    Substance('flour_bag', composition=dict(spoons_of_flour=60))
]}
pprint([dict(_) for _ in balance_stoichiometry({'eggs_6pack', 'milk_carton', 'flour_bag'},
                                               {'pancake'}, substances=substances)])
```

    [{'eggs_6pack': 10, 'flour_bag': 2, 'milk_carton': 15}, {'pancake': 60}]

```python
pprint([dict(_) for _ in balance_stoichiometry({'C', 'O2'}, {'CO2', 'CO'})])  # doctest: +SKIP
```

    [{'C': x1 + 1, 'O2': x1 + 1/2}, {'CO': 1, 'CO2': x1}]

```python
pprint([dict(_) for _ in balance_stoichiometry({'C', 'O2'}, {'CO2', 'CO'}, underdetermined=None)])
```

    [{'C': 3, 'O2': 2}, {'CO': 2, 'CO2': 1}]

```python
from chempy import Equilibrium
from sympy import symbols
K1, K2, Kw = symbols('K1 K2 Kw')
e1 = Equilibrium({'MnO4-': 1, 'H+': 8, 'e-': 5}, {'Mn+2': 1, 'H2O': 4}, K1)
e2 = Equilibrium({'O2': 1, 'H2O': 2, 'e-': 4}, {'OH-': 4}, K2)
coeff = Equilibrium.eliminate([e1, e2], 'e-')
print(coeff)
redox = e1*coeff[0] + e2*coeff[1]
print(redox)
autoprot = Equilibrium({'H2O': 1}, {'H+': 1, 'OH-': 1}, Kw)
n = redox.cancel(autoprot)
print(n)
redox2 = redox + n*autoprot
print(redox2)
```

    [4, -5]
    32 H+ + 4 MnO4- + 20 OH- = 26 H2O + 4 Mn+2 + 5 O2; K1**4/K2**5
    20
    12 H+ + 4 MnO4- = 6 H2O + 4 Mn+2 + 5 O2; K1**4*Kw**20/K2**5

```python
from collections import defaultdict
>>> from chempy.equilibria import EqSystem
>>> eqsys = EqSystem.from_string("""HCO3- = H+ + CO3-2; 10**-10.3
... H2CO3 = H+ + HCO3-; 10**-6.3
... H2O = H+ + OH-; 10**-14/55.4
... """)  # pKa1(H2CO3) = 6.3 (implicitly incl. CO2(aq)), pKa2=10.3 & pKw=14
>>> arr, info, sane = eqsys.root(defaultdict(float, {'H2O': 55.4, 'HCO3-': 1e-2}))
>>> conc = dict(zip(eqsys.substances, arr))
>>> from math import log10
>>> print("pH: %.2f" % -log10(conc['H+']))
```

    pH: 8.30

```python
>>> from chempy import Equilibrium
>>> from chempy.chemistry import Species
>>> water_autop = Equilibrium({'H2O'}, {'H+', 'OH-'}, 10**-14)  # unit "molar" assumed
>>> ammonia_prot = Equilibrium({'NH4+'}, {'NH3', 'H+'}, 10**-9.24)  # same here
>>> substances = [Species.from_formula(f) for f in 'H2O OH- H+ NH3 NH4+'.split()]
>>> eqsys = EqSystem([water_autop, ammonia_prot], substances)
>>> print('\n'.join(map(str, eqsys.rxns)))  # "rxns" short for "reactions"

>>> init_conc = defaultdict(float, {'H2O': 1, 'NH3': 0.1})
>>> x, sol, sane = eqsys.root(init_conc)
>>> assert sol['success'] and sane
>>> print(', '.join('%.2g' % v for v in x))
```

    H2O = H+ + OH-; 1e-14
    NH4+ = H+ + NH3; 5.75e-10
    1, 0.0013, 7.6e-12, 0.099, 0.0013

```python
>>> from chempy.electrolytes import ionic_strength
>>> ionic_strength({'Fe+3': 0.050, 'ClO4-': 0.150}) == .3
```

    True

```python
>>> from chempy.henry import Henry
>>> kH_O2 = Henry(1.2e-3, 1800, ref='carpenter_1966')
>>> print('%.1e' % kH_O2(298.15))
```

    1.2e-03

```python
>>> from chempy import ReactionSystem  # The rate constants below are arbitrary
>>> rsys = ReactionSystem.from_string("""2 Fe+2 + H2O2 -> 2 Fe+3 + 2 OH-; 42
...     2 Fe+3 + H2O2 -> 2 Fe+2 + O2 + 2 H+; 17
...     H+ + OH- -> H2O; 1e10
...     H2O -> H+ + OH-; 1e-4""")  # "[H2O]" = 1.0 (actually 55.4 at RT)
>>> from chempy.kinetics.ode import get_odesys
>>> odesys, extra = get_odesys(rsys)
>>> from collections import defaultdict
>>> import numpy as np
>>> tout = sorted(np.concatenate((np.linspace(0, 23), np.logspace(-8, 1))))
>>> c0 = defaultdict(float, {'Fe+2': 0.05, 'H2O2': 0.1, 'H2O': 1.0, 'H+': 1e-2, 'OH-': 1e-12})
>>> result = odesys.integrate(tout, c0, atol=1e-12, rtol=1e-14)
>>> import matplotlib.pyplot as plt
>>> fig, axes = plt.subplots(1, 2, figsize=(12, 5))
>>> for ax in axes:
...     _ = result.plot(names=[k for k in rsys.substances if k != 'H2O'], ax=ax)
...     _ = ax.legend(loc='best', prop={'size': 9})
...     _ = ax.set_xlabel('Time')
...     _ = ax.set_ylabel('Concentration')
>>> _ = axes[1].set_ylim([1e-13, 1e-1])
>>> _ = axes[1].set_xscale('log')
>>> _ = axes[1].set_yscale('log')
>>> _ = fig.tight_layout()
>>> _ = fig.savefig('kinetics.png', dpi=500)
```

```python
>>> from chempy import Substance
>>> from chempy.properties.water_density_tanaka_2001 import water_density as rho
>>> from chempy.units import to_unitless, default_units as u
>>> water = Substance.from_formula('H2O')
>>> for T_C in (15, 25, 35):
...     concentration_H2O = rho(T=(273.15 + T_C)*u.kelvin, units=u)/water.molar_mass(units=u)
...     print('[H2O] = %.2f M (at %d °C)' % (to_unitless(concentration_H2O, u.molar), T_C))
...
```

    [H2O] = 55.46 M (at 15 °C)
    [H2O] = 55.35 M (at 25 °C)
    [H2O] = 55.18 M (at 35 °C)

```python
import pandas as pd
from pandas import DataFrame
import matplotlib.pyplot as plot
target_url = ("https://archive.ics.uci.edu/ml/machine-learning-"
"databases/undocumented/connectionist-bench/sonar/sonar.all-data")
#read rocks versus mines data into pandas data frame
rocksVMines = pd.read_csv(target_url,header=None, prefix="V")
#print head and tail of data frame
print(rocksVMines.head())
print(rocksVMines.tail())
#print summary of data frame
summary = rocksVMines.describe()
print(summary)
```

           V0      V1      V2      V3      V4      V5      V6      V7      V8  \
    0  0.0200  0.0371  0.0428  0.0207  0.0954  0.0986  0.1539  0.1601  0.3109
    1  0.0453  0.0523  0.0843  0.0689  0.1183  0.2583  0.2156  0.3481  0.3337
    2  0.0262  0.0582  0.1099  0.1083  0.0974  0.2280  0.2431  0.3771  0.5598
    3  0.0100  0.0171  0.0623  0.0205  0.0205  0.0368  0.1098  0.1276  0.0598
    4  0.0762  0.0666  0.0481  0.0394  0.0590  0.0649  0.1209  0.2467  0.3564

           V9  ...     V51     V52     V53     V54     V55     V56     V57  \
    0  0.2111  ...  0.0027  0.0065  0.0159  0.0072  0.0167  0.0180  0.0084
    1  0.2872  ...  0.0084  0.0089  0.0048  0.0094  0.0191  0.0140  0.0049
    2  0.6194  ...  0.0232  0.0166  0.0095  0.0180  0.0244  0.0316  0.0164
    3  0.1264  ...  0.0121  0.0036  0.0150  0.0085  0.0073  0.0050  0.0044
    4  0.4459  ...  0.0031  0.0054  0.0105  0.0110  0.0015  0.0072  0.0048

          V58     V59  V60
    0  0.0090  0.0032    R
    1  0.0052  0.0044    R
    2  0.0095  0.0078    R
    3  0.0040  0.0117    R
    4  0.0107  0.0094    R

    [5 rows x 61 columns]
             V0      V1      V2      V3      V4      V5      V6      V7      V8  \
    203  0.0187  0.0346  0.0168  0.0177  0.0393  0.1630  0.2028  0.1694  0.2328
    204  0.0323  0.0101  0.0298  0.0564  0.0760  0.0958  0.0990  0.1018  0.1030
    205  0.0522  0.0437  0.0180  0.0292  0.0351  0.1171  0.1257  0.1178  0.1258
    206  0.0303  0.0353  0.0490  0.0608  0.0167  0.1354  0.1465  0.1123  0.1945
    207  0.0260  0.0363  0.0136  0.0272  0.0214  0.0338  0.0655  0.1400  0.1843

             V9  ...     V51     V52     V53     V54     V55     V56     V57  \
    203  0.2684  ...  0.0116  0.0098  0.0199  0.0033  0.0101  0.0065  0.0115
    204  0.2154  ...  0.0061  0.0093  0.0135  0.0063  0.0063  0.0034  0.0032
    205  0.2529  ...  0.0160  0.0029  0.0051  0.0062  0.0089  0.0140  0.0138
    206  0.2354  ...  0.0086  0.0046  0.0126  0.0036  0.0035  0.0034  0.0079
    207  0.2354  ...  0.0146  0.0129  0.0047  0.0039  0.0061  0.0040  0.0036

            V58     V59  V60
    203  0.0193  0.0157    M
    204  0.0062  0.0067    M
    205  0.0077  0.0031    M
    206  0.0036  0.0048    M
    207  0.0061  0.0115    M

    [5 rows x 61 columns]
                   V0          V1          V2          V3          V4          V5  \
    count  208.000000  208.000000  208.000000  208.000000  208.000000  208.000000
    mean     0.029164    0.038437    0.043832    0.053892    0.075202    0.104570
    std      0.022991    0.032960    0.038428    0.046528    0.055552    0.059105
    min      0.001500    0.000600    0.001500    0.005800    0.006700    0.010200
    25%      0.013350    0.016450    0.018950    0.024375    0.038050    0.067025
    50%      0.022800    0.030800    0.034300    0.044050    0.062500    0.092150
    75%      0.035550    0.047950    0.057950    0.064500    0.100275    0.134125
    max      0.137100    0.233900    0.305900    0.426400    0.401000    0.382300

                   V6          V7          V8          V9  ...         V50  \
    count  208.000000  208.000000  208.000000  208.000000  ...  208.000000
    mean     0.121747    0.134799    0.178003    0.208259  ...    0.016069
    std      0.061788    0.085152    0.118387    0.134416  ...    0.012008
    min      0.003300    0.005500    0.007500    0.011300  ...    0.000000
    25%      0.080900    0.080425    0.097025    0.111275  ...    0.008425
    50%      0.106950    0.112100    0.152250    0.182400  ...    0.013900
    75%      0.154000    0.169600    0.233425    0.268700  ...    0.020825
    max      0.372900    0.459000    0.682800    0.710600  ...    0.100400

                  V51         V52         V53         V54         V55         V56  \
    count  208.000000  208.000000  208.000000  208.000000  208.000000  208.000000
    mean     0.013420    0.010709    0.010941    0.009290    0.008222    0.007820
    std      0.009634    0.007060    0.007301    0.007088    0.005736    0.005785
    min      0.000800    0.000500    0.001000    0.000600    0.000400    0.000300
    25%      0.007275    0.005075    0.005375    0.004150    0.004400    0.003700
    50%      0.011400    0.009550    0.009300    0.007500    0.006850    0.005950
    75%      0.016725    0.014900    0.014500    0.012100    0.010575    0.010425
    max      0.070900    0.039000    0.035200    0.044700    0.039400    0.035500

                  V57         V58         V59
    count  208.000000  208.000000  208.000000
    mean     0.007949    0.007941    0.006507
    std      0.006470    0.006181    0.005031
    min      0.000300    0.000100    0.000600
    25%      0.003600    0.003675    0.003100
    50%      0.005800    0.006400    0.005300
    75%      0.010350    0.010325    0.008525
    max      0.044000    0.036400    0.043900

    [8 rows x 60 columns]

```python
import requests
url = "https://archive.ics.uci.edu/ml/machine-learning-"
"databases/undocumented/connectionist-bench/sonar/sonar.all-data"
r = requests.get(url)
with open('2.csv','wb') as f:
    f.write(r.content)
    f.close()
```

```python
import pandas as pd
from pandas import DataFrame
import matplotlib.pyplot as plot
target_url = ("https://archive.ics.uci.edu/ml/machine-learning-"
"databases/undocumented/connectionist-bench/sonar/sonar.all-data")
#read rocks versus mines data into pandas data frame
rocksVMines = pd.read_csv(target_url,header=None, prefix="V")
for i in range(208):
    #assign color based on "M" or "R" labels
    if rocksVMines.iat[i,60] == "M":
        pcolor = "red"
    else:
        pcolor = "blue"
        #plot rows of data as if they were series data
        dataRow = rocksVMines.iloc[i,0:60]
        dataRow.plot(color=pcolor)
plot.xlabel("Attribute Index")
plot.ylabel(("Attribute Values"))
plot.show()
```

```python
url1 = 'https://haier.ceping.com/Login/Elink?elink=RopfZpaoU63RmcBFDcPl1iIJ4jIYxZou39p1886LYXPfE0rnyFCU/2lWmTWBv5Zidl8EGSLHQjU=&v=1'
url2 = 'https://haier.ceping.com/Login/Elink?elink=RopfZpaoU63RmcBFDcPl1iIJ4jIYxZou39p1886LYXPfE0rnyFCU/2lWmTWBv5Zidl8EGSLHQjU=&v=1'
```

```python
url1 == url2
```

    True

```python
# 导入扩展库
import re # 正则表达式库
import collections # 词频统计库
import numpy as np # numpy数据处理库
import jieba # 结巴分词
import wordcloud # 词云展示库
from PIL import Image # 图像处理库
import matplotlib.pyplot as plt # 图像展示库

# 读取文件
fn = open('1.txt',encoding='utf-8') # 打开文件
string_data = fn.read() # 读出整个文件
fn.close() # 关闭文件

# 文本预处理
pattern = re.compile(u'\t|\n|\.|-|:|;|\)|\(|\?|"') # 定义正则表达式匹配模式
string_data = re.sub(pattern, '', string_data) # 将符合模式的字符去除

# 文本分词
seg_list_exact = jieba.cut(string_data, cut_all = False) # 精确模式分词
object_list = []
remove_words = [u'的', u'，',u'和', u'是', u'随着', u'对于', u'对',u'等',u'能',u'都',u'。',u' ',u'、',u'中',u'在',u'了',
                u'通常',u'如果',u'我们',u'需要'] # 自定义去除词库

for word in seg_list_exact: # 循环读出每个分词
    if word not in remove_words: # 如果不在去除词库中
        object_list.append(word) # 分词追加到列表

# 词频统计
word_counts = collections.Counter(object_list) # 对分词做词频统计
word_counts_top10 = word_counts.most_common(10) # 获取前10最高频的词
print (word_counts_top10) # 输出检查
backgroud_Image = plt.imread('2.jpg')

# 词频展示
mask = np.array(Image.open('2.jpg')) # 定义词频背景
wc = wordcloud.WordCloud(
    font_path='C:/Windows/Fonts/simhei.ttf', # 设置字体格式
    background_color='white',# 设置背景颜色
    mask=backgroud_Image,# 设置背景图片
    max_words=1000, # 最多显示词数
    max_font_size=100, # 字体最大值
    width = 10000,
    height = 10000
)

wc.generate_from_frequencies(word_counts) # 从字典生成词云
image_colors = wordcloud.ImageColorGenerator(mask) # 从背景图建立颜色方案
wc.recolor(color_func=image_colors) # 将词云颜色设置为背景图方案
plt.imshow(wc) # 显示词云
plt.axis('off') # 关闭坐标轴
plt.show() # 显示图像
```

```python
import pickle
from os import path
import jieba
import matplotlib.pyplot as plt
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator
text = ''
with open('1.txt', 'r', encoding='utf-8') as fin:
    for line in fin.readlines():
        line = line.strip('\n')
        # sep’.join（seq）以sep作为分隔符，将seq所有的元素合并成一个新的字符串
        text += ' '.join(jieba.cut(line))
backgroud_Image = plt.imread('2.jpg')
print('加载图片成功！')
'''设置词云样式'''
wc = WordCloud(
    background_color='white',# 设置背景颜色
    mask=backgroud_Image,# 设置背景图片
    font_path='C:\Windows\Fonts\STZHONGS.TTF',  # 若是有中文的话，这句代码必须添加，不然会出现方框，不出现汉字
    max_words=2000, # 设置最大现实的字数
    stopwords=STOPWORDS,# 设置停用词
    max_font_size=75,# 设置字体最大值
    random_state=30,# 设置有多少种随机生成状态，即有多少种配色方案
    width = 10000,
    height = 10000
)
wc.generate_from_text(text)
print('开始加载文本')
#改变字体颜色
img_colors = ImageColorGenerator(backgroud_Image)
#字体颜色为背景图片的颜色
wc.recolor(color_func=img_colors)
# 显示词云图
plt.imshow(wc)
# 是否显示x轴、y轴下标
plt.axis('off')
plt.show()
# 获得模块所在的路径的
print('生成词云成功!')
```

    Building prefix dict from the default dictionary ...
    Dumping model to file cache C:\Users\NickC\AppData\Local\Temp\jieba.cache
    Loading model cost 0.898 seconds.
    Prefix dict has been built successfully.

    加载图片成功！
    开始加载文本

    生成词云成功!

# Comparisons, Equality and Truth

```python
L1 = [1,('a',3)] # Same value, unique objects
L2 = [1,('a',3)]
L1 == L2, L1 is L2 # Equivalent? Same object ?
```

    (True, False)

```python
L = ['grail']
L.append(L)
print(L)
```

    ['grail', [...]]

#### The == operator tests value equivalence. Python performs an equivalence test, comparing all nested objects recursively.

#### The is operator tests object identity. Python tests whether the two are really the same object (i.e., live at the same address in memory).

```python
S1 = 'spam'
S2 = 'spam'
```

```python
S1 == S2, S1 is S2
```

    (True, True)

Here, we should again have two distinct objects that happen to have the same value:
== should be true, and is should be false. But because Python internally caches and
reuses some strings as an optimization, there really is just a single string 'spam' in
memory, shared by S1 and S2; hence, the is identity test reports a true result. To trigger
the normal behavior, we need to use longer strings:

```python
S1 = 'a longer string'
S2 = 'a longer string'
S1 == S2, S1 is S2
```

    (True, False)

```python
L1 = [1, ('a', 3)]
L2 = [1,('a', 2)]
L1 < L2, L1 == L2, L1 > L2
```

    (False, False, True)

In general, Python compares types as follows:

• Numbers are compared by relative magnitude.

• Strings are compared lexicographically, character by character ("abc" < "ac").

• Lists and tuples are compared by comparing each component from left to right.

• Dictionaries compare as equal if their sorted (key, value) lists are equal. Relative magnitude comparisons are not supported for dictionaries in Python 3.0, but they work in 2.6 and earlier as though comparing sorted (key, value) lists.

• Nonnumeric mixed-type comparisons (e.g., 1 < 'spam') are errors in Python 3.0.

They are allowed in Python 2.6, but use a fixed but arbitrary ordering rule. By proxy, this also applies to sorts, which use comparisons internally: nonnumeric mixed-type collections cannot be sorted in 3.0.

# Python 3.0 Dictionary Comparisons

```python
D1 = {'a': 1, 'b': 2}
D2 = {'a': 1, 'b': 3}
print(D1 == D2)
# print( D1 < D2 )
```

    False

```python
# D1 < D2 # In Python 3.0, magnitude comparisons for dictionaries are removed because they incur too much overhead when equality is desired (equality uses an optimized scheme in 3.0 that doesn’t literally compare sorted key/value lists). The alternative in 3.0 is to either write loops to compare values by key or compare the sorted key/value lists manually— the items dictionary methods and sorted built-in suffice
# '<' not supported between instances of 'dict' and 'dict'
```

```python
list(D1.items())
sorted(D1.items())
print(sorted(D1.items()) < sorted(D2.items()))
print(sorted(D1.items()) < sorted(D2.items()))
```

    True
    True

# The None object

```python
L = [None]*100
```

```python
print(L,end='')
```

    [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]

Keep in mind that None does not mean “undefined.” That is, None is something, not
nothing (despite its name!)—it is a real object and piece of memory, given a built-in
name by Python. Watch for other uses of this special object later in the book; it is also
the default return value of functions, as we’ll see in Part IV

# The bool type

```python
bool(1)
bool('spam')
bool({})
```

    False

Python also provides a bool builtin function that can be used to test the Boolean value of an object (i.e., whether it is
True—that is, nonzero or nonempty)

# Type Objects

```python
type([1]) == type([]) # Type of another list
```

    True

```python
type([1]) == list # list type name
```

    True

```python
isinstance([1],list) # List or customization thereof
```

    True

```python
import types
def f(): pass
type(f) == types.FunctionType
```

    True

# Other Types in Python

```python
L = [1,2,3]
M = ['X',L,'Y']
M
L[1] = 0 # Changes M too
M
```

    ['X', [1, 0, 3], 'Y']

```python
L = [1,2,3]
M = ['X',L[:],'Y'] # Embed a copy of L
L[1] = 0 # Changes only L, not M
print(L)
print(M)
```

    [1, 0, 3]
    ['X', [1, 2, 3], 'Y']

# Repetition Adds One Level Deep

```python
L = [4,5,6]
X = L * 4 # Like [4,5,6] + [4,5,6] + ...
```

```python
Y = [L] * 4
```

```python
(X,Y)
```

    ([4, 5, 6, 4, 5, 6, 4, 5, 6, 4, 5, 6],
     [[4, 5, 6], [4, 5, 6], [4, 5, 6], [4, 5, 6]])

```python
L[1] = 0 # Impacts Y but not X
print(X, Y)
```

    [4, 5, 6, 4, 5, 6, 4, 5, 6, 4, 5, 6] [[4, 0, 6], [4, 0, 6], [4, 0, 6], [4, 0, 6]]

# Beware of Cyclic Data Structures

```python
L = ['grail'] # Append references to same object
L.append(L) # Generates cycle in object: [...]
print(L)
```

    ['grail', [...]]

# Immutable Types Can’t Be Changed In-Place

```python
T = (1,2,3)
# T[2] = 4 # Error！
T = T[:2] + (4,) # Ok: (1,2,4)
```

```python
T
```

    (1, 2, 4)

# Chapter Summary

This chapter explored the last two major core object types—the tuple and the file. We
learned that tuples support all the usual sequence operations, have just a few methods,
and do not allow any in-place changes because they are immutable. We also learned
that files are returned by the built-in open function and provide methods for reading
and writing data. We explored how to translate Python objects to and from strings for
storing in files, and we looked at the pickle and struct modules for advanced roles
(object serialization and binary data). Finally, we wrapped up by reviewing some properties common to all object types (e.g., shared references) and went through a list of
common mistakes (“gotchas”) in the object type domain.
In the next part, we’ll shift gears, turning to the topic of statement syntax in Python—
we’ll explore all of Python’s basic procedural statements in the chapters that follow.
The next chapter kicks off that part of the book with an introduction to Python’s general
syntax model, which is applicable to all statement types. Before moving on, though,
take the chapter quiz, and then work through the end-of-part lab exercises to review
type concepts. Statements largely just create and process objects, so make sure you’ve
mastered this domain by working through all the exercises before reading on.

# Test Your Knowledge: Quiz

```python
T = (4,5,6)
```

```python
len(T)
```

    3

```python
L = list(T)
```

```python
L[0]=1
```

```python
T = tuple()
```

```python

```


# [cknowledge.org](http://cknowledge.org): Community-driven benchmarking and optimization of computing systems - from classical to quantum

[Quantum Computing](https://github.com/ctuning/ck-quantum/wiki)
* [CK-QISKit](https://github.com/ctuning/ck-qiskit) (IBM)
* [CK-Rigetti](https://github.com/ctuning/ck-rigetti) ([Rigetti Computing](https://rigetti.com/))
* [CK-ProjectQ](https://github.com/ctuning/ck-projectq) ([ProjectQ](https://projectq.ch/))

[Artificial Intelligence and Machine Learning](http://cknowledge.org/ai)
* [Reproducible Quality-Efficient Systems Tournaments](http://cknowledge.org/request) ([ReQuEST initiative](http://cknowledge.org/request.html#organizers))
* [AI artifacts](http://cknowledge.org/ai-artifacts) (cTuning foundation)
* [Android app](https://play.google.com/store/apps/details?id=openscience.crowdsource.video.experiments) (dividiti)
* [Desktop app](https://github.com/dividiti/ck-crowdsource-dnn-optimization) (dividiti)
* [CK-Caffe](https://github.com/dividiti/ck-caffe) (Berkeley)
* [CK-Caffe2](https://github.com/ctuning/ck-caffe2) (Facebook)
* [CK-CNTK](https://github.com/ctuning/ck-cntk) (Microsoft)
* [CK-KaNN](https://github.com/ctuning/ck-kann) (Kalray)
* [CK-MVNC](https://github.com/ctuning/ck-mvnc) (Movidius / Intel)
* [CK-MXNet](https://github.com/ctuning/ck-mxnet) (Apache)
* [CK-NNTest](https://github.com/ctuning/ck-nntest) (cTuning foundation)
* [CK-TensorFlow](https://github.com/ctuning/ck-tensorflow) (Google)
* [CK-TensorRT](https://github.com/ctuning/ck-tensorrt) (NVIDIA)
* etc.

# Variational Quantum Eigensolver (VQE) on Rigetti machines
[Quantum Collective Knowledge Hackaton](https://www.eventbrite.co.uk/e/quantum-computing-hackathon-tickets-46441126660#), [Centre for Mathematical Science - University of Cambridge](http://www.cms.cam.ac.uk/), 15 June 2018

## Table of Contents

1. [Organisers](#organisers)
1. [References](#references)
1. [Time-to-solution metric](#metric)
1. [Setting up](#settingup)
1. [Running experiments](#running)
1. [Experimental data](#data)
1. [Data wrangling code](#code) (for developers)
1. [Analysis](#analysis)

## Organisers

- [Rigetti Computing](https://rigetti.com): access to [Quantum Virtual Machine](http://pyquil.readthedocs.io/en/latest/qvm.html) (QVM) and [Quantum Processing Unit](http://pyquil.readthedocs.io/en/latest/qpu.html) (QPU).
- [River Lane Research](https://riverlane.io/): Steve Brierley, Oscar Higgott, Daochen Wang, Amy Flower
- [dividiti](http://dividiti.com/): Anton Lokhmotov, Leo Gordon, Flavio Vella, Grigori Fursin

## References

- [Rigetti's docs](http://grove-docs.readthedocs.io/en/latest/vqe.html)
- ["A variational eigenvalue solver on a quantum processor"](https://arxiv.org/abs/1304.3061) (2013)
- ["The theory of variational hybrid quantum-classical algorithms"](https://arxiv.org/abs/1509.04279) (2015)
- ["Quantum optimization using variational algorithms
on near-term quantum devices"](https://arxiv.org/abs/1710.01022) [2017]

## Time-to-solution metric

To compare solutions of participants we use a **time-to-solution** $T$ metric defined as follows.

### Definition

Let's assume that to find the ground state of a given molecule (e.g. [Helium](https://en.wikipedia.org/wiki/Helium)) a participant makes $N$ runs of their implementation (e.g. $N=3$). A run is considered _successful_ if the ground state found in this run is equal to the ground state known for the molecule to given precision $\delta$ (e.g. $\delta=0.1$). Assume that a single run takes $t$ samples (calls to the quantum computer) on average.

Let $s$ be the probability of success of the participant's VQE implementation (i.e. the number of successful runs divided by the total number of runs $N$).

Let $R$ be the number of runs required to find the ground state with given probability $p$ (e.g. $p=0.6$):
\begin{equation}
R = {\frac{\log(1-p)}{\log(1-s)}}.
\end{equation}

The **time-to-solution** $T$, defined as the total number of samples used throughout the whole optimisation procedure of VQE, is then calculated as:
\begin{equation}
T = R \times t.
\end{equation}

### Derivation

If the probability of success is $s$, then the probability of _failing to find_ the ground state after $R$ runs is $(1-s)^R$. Therefore, the probability of finding the ground state at least once after $R$ runs is $p =1 - (1-s)^R$. Therefore, the number of runs $R$ required to find the ground state at least once with probability $p$ can be found by solving $p=1-(1-s)^R$.

### Uncertainties

We can also calculate the standard error associated with the calculated time-to-solution $T$.

From the [binomial distribution](Binomial_distribution), the uncertainty $\sigma_s$ in the success probability $s$ is:
\begin{equation}
  \sigma_s=\sqrt{\frac{s(1-s)}{N}}
\end{equation}
where $N$ is the number of runs used to determine $s$.

The uncertainty in the time taken per run $t$ is:
\begin{equation}
  \sigma_t=\frac{\mathrm{std}}{\sqrt{N}}
\end{equation}
where $\mathrm{std}$ is the standard deviation of the times taken by all $N$ runs.

The uncertainty in total time taken is:
\begin{equation}
\sigma_T=\sqrt{0.25 \cdot (T(t+\sigma_t, s) - T(t-\sigma_t,s))^2 + (T(t, s + \sigma_s) - T(t,s))^2}
\end{equation}

## Setting up

Please follow instructions [here](https://github.com/ctuning/ck-quantum/blob/master/README.md).

## Running experiments

```
$ ck benchmark program:rigetti-vqe \
  --env.RIGETTI_QUANTUM_DEVICE=<platform> \
  --env.VQE_MINIMIZER_METHOD=<minimizer_method> \
  --env.VQE_SAMPLE_SIZE=<sample_number> \
  --env.VQE_MAX_ITERATIONS=<max_iterations> \
  --record --record_repo=local --record_uoa=<email>-<plaform> \
  --tags=qck,hackathon-2018_06_15,<email>,<platform>,<minimizer_method> \
  --repetitions=<repetitions>
```
where:
- `platform`: `8Q-Agave` or `QVM`;
- `minimizer_method`: `my_melder_nead` or `my_cobyla` or `my_minimizer` (as defined in [optimizers.py](https://github.com/ctuning/ck-quantum/blob/master/package/tool-hackathon/hackathon-src/hackathon/optimizers.py) installed under e.g. `$CK_TOOLS/hackathon-1.0-linux-64/lib/hackathon`);
- `sample_size`: e.g. `100` (or another resolution);
- `max_iterations`: e.g. `80` (or another cut-off point);
- `email`: a valid email address (later to be replaced with a team id e.g. `team-01`);
- `repetitions`: how many times to run the experiment with the given parameters: e.g. `3`.

## Get sample experimental data

The sample experimental data can be downloaded and registered with CK as follows:

```
$ wget https://www.dropbox.com/s/a1odux4asze9zpd/ck-quantum-hackathon-20180615.zip
$ ck add repo --zip=ck-quantum-hackathon-20180615.zip
```

```python
repo_uoa = 'ck-quantum-hackathon-20180615'
!ck list $repo_uoa:experiment:* --print_full | sort
```

## Data wrangling code

**NB:** Please ignore this section if you are not interested in re-running or modifying this notebook.

### Includes

#### Standard

```python
import os
import sys
import json
import re
```

#### Scientific

If some of the scientific packages are missing, please install them using:
```
# pip install jupyter pandas numpy matplotlib
```

```python
import IPython as ip
import pandas as pd
import numpy as np
import matplotlib as mp
```

```python
print ('IPython version: %s' % ip.__version__)
print ('Pandas version: %s' % pd.__version__)
print ('NumPy version: %s' % np.__version__)
print ('Matplotlib version: %s' % mp.__version__)
```

```python
from IPython.display import Image, display
def display_in_full(df):
    pd.options.display.max_columns = len(df.columns)
    pd.options.display.max_rows = len(df.index)
    display(df)
```

```python
import matplotlib.pyplot as plt
from matplotlib import cm
%matplotlib inline
```

```python
default_colormap = cm.autumn
default_fontsize = 16
default_barwidth = 0.8
default_figwidth = 16
default_figheight = 8
default_figdpi = 200
default_figsize = [default_figwidth, default_figheight]
```

```python
if mp.__version__[0]=='2': mp.style.use('classic')
mp.rcParams['figure.max_open_warning'] = 200
mp.rcParams['figure.dpi'] = default_figdpi
mp.rcParams['font.size'] = default_fontsize
mp.rcParams['legend.fontsize'] = 'medium'
```

#### Collective Knowledge

If CK is not installed, please install it using:
```
# pip install ck
```

```python
import ck.kernel as ck
print ('CK version: %s' % ck.__version__)
```

```python
# NB: Make sure the quantum hackathon tool is installed. (It should be if you have run any experiments.)
# $ ck install package --tags=ck-quantum,tool,hackathon,v1
r=ck.access({'action':'show', 'module_uoa':'env', 'tags':'tool,hackathon'})
if r['return']>0:
    print ("Error: %s" % r['error'])
    exit(1)

# Get the path to the first returned environment entry.
tool_hackathon_dir=r['lst'][0]['meta']['env']['CK_ENV_LIB_HACKATHON_LIB']
sys.path.append(tool_hackathon_dir)
from hackathon.utils import *
```

### Access experimental data

```python
def get_experimental_results(repo_uoa, tags='qck', module_uoa='experiment'):
    r = ck.access({'action':'search', 'repo_uoa':repo_uoa, 'module_uoa':module_uoa, 'tags':tags})
    if r['return']>0:
        print('Error: %s' % r['error'])
        exit(1)
    experiments = r['lst']

    dfs = []
    for experiment in experiments:
        data_uoa = experiment['data_uoa']
        r = ck.access({'action':'list_points', 'repo_uoa':repo_uoa, 'module_uoa':module_uoa, 'data_uoa':data_uoa})
        if r['return']>0:
            print('Error: %s' % r['error'])
            exit(1)
        tags = r['dict']['tags']

        skip = False
        # Get team name (final data) or email (submission data).
        team_tags = [ tag for tag in tags if tag.startswith('team-') ]
        email_tags = [ tag for tag in tags if tag.find('@')!=-1 ]
        if len(team_tags) > 0:
            team = team_tags[0][0:7]
        elif len(email_tags) > 0:
            team = email_tags[0]
        else:
            print('[Warning] Cannot determine team name for experiment in: \'%s\'' % r['path'])
            team = 'team-default'

        if skip:
            print('[Warning] Skipping experiment with bad tags:')
            print(tags)
            continue

        # For each point.
        for point in r['points']:
            point_file_path = os.path.join(r['path'], 'ckp-%s.0001.json' % point)
            with open(point_file_path) as point_file:
                point_data_raw = json.load(point_file)
            characteristics_list = point_data_raw['characteristics_list']
            num_repetitions = len(characteristics_list)
            data = [
                {
                    # features
                    'platform': characteristics['run'].get('vqe_input', {}).get('q_device_name', 'unknown').lower(),
                    # choices
                    'minimizer_method': characteristics['run'].get('vqe_input', {}).get('minimizer_method', 'n/a'),
                    'minimizer_options': characteristics['run'].get('vqe_input', {}).get('minimizer_options', {'maxfev':-1}),
                    'minimizer_src': characteristics['run'].get('vqe_input', {}).get('minimizer_src', ''),
                    'sample_number': characteristics['run'].get('vqe_input', {}).get('sample_number','n/a'),
                    # statistical repetition
                    'repetition_id': repetition_id,
                    # runtime characteristics
                    'run': characteristics['run'],
                    'report': characteristics['run'].get('report', {}),
                    'vqe_output': characteristics['run'].get('vqe_output', {}),
                }
                for (repetition_id, characteristics) in zip(range(num_repetitions), characteristics_list)
                if len(characteristics['run']) > 0
            ]

            for datum in data:
                datum['team'] = team
                datum['point'] = point
                datum['success'] = datum.get('vqe_output',{}).get('success',False)
                datum['nfev'] = np.int64(datum.get('vqe_output',{}).get('nfev',-1))
                datum['nit'] = np.int64(datum.get('vqe_output',{}).get('nit',-1))
                datum['fun'] = np.float64(datum.get('vqe_output',{}).get('fun',0))
                datum['fun_validated'] = np.float64(datum.get('vqe_output',{}).get('fun_validated',0))
                datum['fun_exact'] = np.float64(datum.get('vqe_output',{}).get('fun_exact',0))
                datum['total_seconds'] = np.float64(datum.get('report',{}).get('total_seconds',0))
                datum['total_q_seconds'] = np.float64(datum.get('report',{}).get('total_q_seconds',0))
                datum['total_q_shots'] = np.int64(datum.get('report',{}).get('total_q_shots',0))
                tmp_max_iterations = list(datum.get('minimizer_options',{'maxfev':-1}).values())
                datum['max_iterations'] = tmp_max_iterations[0] if len(tmp_max_iterations)>0 else -1
            index = [
                'platform', 'team', 'minimizer_method', 'sample_number', 'max_iterations', 'point', 'repetition_id'
            ]
            # Construct a DataFrame.
            df = pd.DataFrame(data)
            df = df.set_index(index)
            # Append to the list of similarly constructed DataFrames.
            dfs.append(df)
    if dfs:
        # Concatenate all thus constructed DataFrames (i.e. stack on top of each other).
        result = pd.concat(dfs)
        result.sort_index(ascending=True, inplace=True)
    else:
        # Construct a dummy DataFrame the success status of which can be safely checked.
        result = pd.DataFrame(columns=['success'])
    return result
```

```python
# Merge experimental results from the same team with the same parameters
# (minimizer_method, sample_number, max_iterations) and minimizer source.
def merge_experimental_results(df):
    dfs = []
    df_prev = None
    for index, row in df.iterrows():
        # Construct a DataFrame.
        df_curr = pd.DataFrame(row).T
        # Check if this row is similar to the previous row.
        if df_prev is not None: # if not the very first row
            if df_prev.index.levels[:-2]==df_curr.index.levels[:-2]: # if the indices match for all but the last two levels
                if df_prev.index.levels[-2]!=df_curr.index.levels[-2]: # if the experiments are different
                    if df_prev['minimizer_src'].values==df_curr['minimizer_src'].values: # if the minimizer source is the same
                        print('[Info] Merging experiment:')
                        print(df_curr.index.levels)
                        print('[Info] into:')
                        print(df_prev.index.levels)
                        print('[Info] as:')
    #                     df_curr.index = df_prev.index.copy() # TODO: increment repetition_id
                        df_curr.index = pd.MultiIndex.from_tuples([(x[0],x[1],x[2],x[3],x[4],x[5],x[6]+1) for x in df_prev.index])
                        print(df_curr.index.levels)
                        print
                    else:
                        print('[Warning] Cannot merge experiments as the minimizer source is different:')
    #                     print('------------------------------------------------------------------------')
                        print(df_prev.index.levels)
    #                     print(df_prev['minimizer_src'].values[0])
    #                     print
    #                     print('------------------------------------------------------------------------')
                        print(df_curr.index.levels)
    #                     print(df_curr['minimizer_src'].values[0])
                        print
    #             else:
    #                 print('[Info] Keeping experiments separate:')
    #                 print(df_prev.index.levels)
    #                 print(df_curr.index.levels)
    #                 print
        # Append to the list of similarly constructed DataFrames.
        dfs.append(df_curr)
        # Prepare for next iteration.
        df_prev = df_curr

    # Concatenate all thus constructed DataFrames (i.e. stack on top of each other).
    result = pd.concat(dfs)
    result.index.names = df.index.names
    result.sort_index(ascending=True, inplace=True)

    return result
```

```python
def get_metrics(df, delta=0.1, prob=0.5, which_fun_key='fun_exact', which_time_key='total_q_shots'):
    dfs = []
    names_no_repetitions = df.index.names[:-1]
    for index, group in df.groupby(level=names_no_repetitions):
        # Compute metrics.
        classical_energy, minimizer_method, minimizer_src, n_succ, T_ave, T_err, t_ave, t_err, s, s_err = \
            benchmark_list_of_runs(group['run'], verbose=False, delta=delta, prob=prob,
                                   which_fun_key=which_fun_key, which_time_key=which_time_key)
        # Construct a DataFrame from the metrics.
        data = {
            # Time to solution.
            'T_ave' : T_ave,
            'T_err' : T_err,
            # Time metric (seconds or shots).
            't_ave' : t_ave,
            't_err' : t_err,
            # Tries metric.
            's' : s,
            's_err' : s_err
        }
        data.update({ k : v for (k, v) in zip(names_no_repetitions, index) })
        data['num_repetitions'] = len(group)
        # NB: index must be something.
        df_ = pd.DataFrame(data=data, index=[0])
        df_ = df_.set_index(names_no_repetitions)
        # Append to the list of similarly constructed DataFrames.
        dfs.append(df_)
    if dfs:
        # Concatenate all thus constructed DataFrames (i.e. stack on top of each other).
        result = pd.concat(dfs).dropna()
        result.sort_index(ascending=True, inplace=True)
    return result
```

### Plot experimental data

```python
def plot(df, platform_set=None, minimizer_method_set=None, sample_number_set=None, max_iterations_set=None,
         markersize_divisor=20,
         xmin=0.0, xmax=85.01, xstep=5.00,
         ymin=-3.0, ymax=-0.49, ystep=0.25,
         figsize=(18,9), dpi=200, legend_loc='lower right'):

    platform_set = platform_set or df.index.get_level_values(level='platform').unique()
    minimizer_method_set = minimizer_method_set or df.index.get_level_values(level='minimizer_method').unique()
    sample_number_set = sample_number_set or df.index.get_level_values(level='sample_number').unique()
    max_iterations_set = max_iterations_set or df.index.get_level_values(level='max_iterations').unique()

    # Options.
    minimizer_method_to_color = {
        'my_cobyla' : 'orange',
        'my_nelder_mead' : 'green',
        'my_minimizer' : 'blue'
    }
    platform_to_marker = {
        '8q-agave' : '8', # octagon
        'qvm' : 's',      # square
        'local_qasm_simulator' : 'p' # pentagon
    }

    last_marker_size = 10
    last_marker_color = 'black'
    last_marker_success_true = '^'
    last_marker_success_false = 'v'

    fig = plt.figure(figsize=figsize, dpi=dpi)
    ax = fig.gca()
    for index, row in df.iterrows():
        (platform, team, minimizer_method, sample_number, max_iterations, point, repetition_id) = index
        if platform not in platform_set: continue
        if sample_number not in sample_number_set: continue
        if minimizer_method not in minimizer_method_set: continue
        # NB: This uses 'fun', not 'fun_exact' or 'fun_validated'.
        energies = [ iteration['energy'] for iteration in row['report']['iterations'] ]
        marker=platform_to_marker[platform]
        markersize=sample_number/markersize_divisor
        color=minimizer_method_to_color.get(minimizer_method, 'red')
        markerfacecolor=color
        linestyle='-'
        ax.plot(range(len(energies)), energies, marker=marker, color=color, linestyle=linestyle,
                markerfacecolor=markerfacecolor, markersize=markersize)
        # Mark last function evaluation.
        last_energy = energies[-1]
        last_fev = row['nfev']-1 if minimizer_method=='my_cobyla' or 'my_nelder_mead' else row['nfev']
        last_marker = last_marker_success_true if row['success'] else last_marker_success_false
        ax.plot(last_fev, last_energy, color=last_marker_color, marker=last_marker, markersize=last_marker_size)

    # Horizontal line for the known ground state.
    plt.axhline(y=-2.80778395754, color='red', linestyle='--')
    # Vertical lines for max_iterations.
    for max_iterations in max_iterations_set:
        plt.axvline(x=max_iterations, color='black')
    # Grid.
    plt.grid()
    # Title.
    title = 'Variational Quantum Eigensolver (VQE)'
    ax.set_title(title)
    # X axis.
    xlabel='Function evaluation'
    ax.set_xlabel(xlabel)
    ax.set_xlim(xmin, xmax)
    ax.set_xticks(np.arange(xmin, xmax, xstep))
    # Y axis.
    ylabel='Energy'
    ax.set_ylabel(ylabel)
    ax.set_ylim(ymin, ymax)
    ax.set_yticks(np.arange(ymin, ymax, ystep))
    # Legend. https://matplotlib.org/users/legend_guide.html
    handles = [
        mp.lines.Line2D([], [], label='platform="%s",minimizer_method="%s"' % (p,m), color=minimizer_method_to_color.get(m, 'red'),
                        marker=platform_to_marker[p], markersize=last_marker_size)
        for p in sorted(platform_set)
        for m in sorted(minimizer_method_set)
    ]
    handles.append(mp.lines.Line2D([],[], label='ground state', color='red', linestyle='--'))
    plt.legend(handles=handles, title='platform,minimizer_method', loc=legend_loc)
    # Save figure.
#    plt.savefig('vqe.energy.png')
```

```python
def plot_metric(df, metric='total_q_seconds'):
    df.columns.name='metric'
    # "df.index.names[:-1]" means reduce along 'repetition_id' (statistical variation).
    df_mean = df[[metric]].groupby(level=df.index.names[:-1]).mean().unstack('platform')
    df_std = df[[metric]].groupby(level=df.index.names[:-1]).std().unstack('platform')
    ax = df_mean.plot(kind='bar', yerr=df_std, grid=True, legend=True, rot=45,
                      fontsize=default_fontsize, figsize=default_figsize, colormap=default_colormap)
```

## Analysis

### All experimental data

```python
df = get_experimental_results(repo_uoa=repo_uoa)
display_in_full(df)
```

### Merge experimental results from different runs with the same parameters

```python
df = merge_experimental_results(df)
display_in_full(df)
```

### Compute the time-to-solution metric etc.

```python
df_metrics = get_metrics(df, delta=0.1, prob=0.5, which_fun_key='fun_exact', which_time_key='total_q_shots')
df_metrics
```

### The (unexpected) winner

Somewhat unexpectedly, the winner obtained the exact answer (!) with `sample_number=1` (!!) on real hardware (!!!). Please see below an explanation why this was the case.

```python
idxmin1 = df_metrics['T_ave'].idxmin()
idxmin1
```

```python
df_metrics.loc[[idxmin1]]
```

```python
df.loc[idxmin1]
```

```python
# Platform, Team, minimizer Function, number of Samples, number of function eValuations, Experiment, Repetition
(p,t,f,s,v,e) = idxmin1
# Plot the winner.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[idxmin1]))]],
     xmax=7, xstep=1, ymin=-3.00, ymax=0.00+0.01, legend_loc='upper center')
```

```python
# Exclude the winner.
df_metrics = df_metrics.drop(idxmin1)
```

#### Explanation

The [Hamiltonian](https://en.wikipedia.org/wiki/Hamiltonian_(quantum_mechanics)) of [Helium](https://en.wikipedia.org/wiki/Helium) in the [STO-3G](https://en.wikipedia.org/wiki/STO-nG_basis_sets) basis is given by:

\begin{equation}
    H = -1.6678202144537553 + 0.7019459893849936 \cdot Z_0 + 0.7019459893849936 \cdot Z_1 + 0.263928235683768058 \cdot Z_0 \cdot Z_1
\end{equation}

Since the Hamiltonian consists of a sum of commuting operators ($Z_0$, $Z_1$, $Z_0\cdot Z1$), there exists a simultaneous eigenbasis (including the ground state) on which the value of each of the operators is $+1$ or $-1$.

When `sample_number=1`, measuring an operator once in any state also results in $+1$ or $-1$. This implies that one get "lucky" with _any_ mininizer method even on noisy hardware. (Indeed, we have been able to reproduce this result with `my_nelder_mead`.)

For more complex molecules, the Hamiltonian is unlikely to consist of a sum of commuting operators, hence picking up a good optimizer and its parameters will be crucial for success.

### The (conditional) runner-up

The runner-up also used `sample_number=1` but only a single run (hence, it's a "conditional" runner-up, as we can determine the error only from multiple runs).

```python
idxmin2 = df_metrics['T_ave'].idxmin()
idxmin2
```

```python
df_metrics.loc[[idxmin2]]
```

```python
df.loc[idxmin2]
```

```python
# Platform, Team, minimizer Function, number of Samples, number of function eValuations, Experiment, Repetition
(p,t,f,s,v,e) = idxmin2
# Plot the winner.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[idxmin2]))]],
     xmax=7, xstep=1, ymin=-3.00, ymax=0.00+0.01, legend_loc='upper center')
```

```python
# Print the minimizer source.
print(df.loc[(p,t,f,s,v,e,0)]['minimizer_src'])
```

```python
# Exclude the conditional runner-up.
df_metrics = df_metrics.drop(idxmin2)
```

### The QVM runner-up

```python
platform_qvm = 'qvm'
df_metrics_qvm = df_metrics.loc[platform_qvm]
idxmin_qvm = df_metrics_qvm['T_ave'].idxmin()
idxmin_qvm
```

```python
df_metrics.loc[platform_qvm].loc[[idxmin_qvm]]
```

```python
df.loc[platform_qvm].loc[idxmin_qvm]
```

```python
# Platform, Team, minimizer Function, number of Samples, number of function eValuations, Experiment, Repetition
p = platform_qvm
(t,f,s,v,e) = idxmin_qvm
# Plot the runner up.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[(p,t,f,s,v,e)]))]],
     xmax=9, xstep=1, ymin=-3.00, ymax=0.00+0.01, legend_loc='upper right')
```

```python
# Print the minimizer source.
print(df.loc[(p,t,f,s,v,e,0)]['minimizer_src'])
```

```python
# Exclude the QVM runner-up.
df_metrics = df_metrics.drop((p,t,f,s,v,e))
```

### The QPU runner-up

```python
platform_qpu = '8q-agave'
df_metrics_qpu = df_metrics.loc[platform_qpu]
idxmin_qpu = df_metrics_qpu['T_ave'].idxmin()
idxmin_qpu
```

```python
df_metrics_qpu.loc[[idxmin_qpu]]
```

```python
df.loc[platform_qpu].loc[idxmin_qpu]
```

```python
# Platform, Team, Experiment, minimizer Function, number of Samples, number of function eValuations, Repetition
p = platform_qpu
(t,f,s,v,e) = idxmin_qpu
# Plot the QPU runner-up.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[(p,t,f,s,v,e)]))]],
     xmax=7, xstep=1, ymin=-3.00, ymax=-0.00+0.01, legend_loc='upper right')
```

```python
# Print the minimizer source.
print(df.loc[(p,t,f,s,v,e,0)]['minimizer_src'])
```

```python
# Exclude the QPU runner-up.
df_metrics = df_metrics.drop((p,t,f,s,v,e))
```

### The best entry with 100% convergence (prob=0.999)

```python
df_metrics_prob100 = get_metrics(df, delta=0.1, prob=0.999, which_fun_key='fun_exact', which_time_key='total_q_shots')
df_metrics_prob100 = df_metrics_prob100[(df_metrics_prob100['s']==1) & (df_metrics_prob100['num_repetitions']>1)]
```

```python
idxmin_prob100 = df_metrics_prob100['T_ave'].idxmin()
idxmin_prob100
```

```python
df_metrics_prob100.loc[[idxmin_prob100]]
```

```python
df.loc[idxmin_prob100]
```

```python
# Platform, Team, Experiment, minimizer Function, number of Samples, number of function eValuations, Repetition
(p,t,f,s,v,e) = idxmin_prob100
# Plot.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[(p,t,f,s,v,e)]))]],
     xmax=30, xstep=1, ymin=-3.00, ymax=-0.00+0.01, legend_loc='upper right')
```

### The worst entry with 100% convergence (prob=0.999)

```python
idxmax_prob100 = df_metrics_prob100['T_ave'].idxmax()
idxmax_prob100
```

```python
df_metrics_prob100.loc[[idxmax_prob100]]
```

```python
df.loc[idxmax_prob100]
```

```python
# Platform, Team, Experiment, minimizer Function, number of Samples, number of function eValuations, Repetition
(p,t,f,s,v,e) = idxmax_prob100
# Plot.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[(p,t,f,s,v,e)]))]],
     xmax=31, xstep=1, ymin=-3.00, ymax=-0.00+0.01, legend_loc='upper right')
```

```python
# The ratio of the worst of the best with 100% convergence.
df_metrics_prob100.loc[idxmax_prob100]['T_ave'] / df_metrics_prob100.loc[idxmin_prob100]['T_ave']
```

```python
# Exclude the best entry with 100% convergence.
df_metrics_prob100 = df_metrics_prob100.drop(idxmin_prob100)
```

### The most accurate entry (also the runner-up with 100% convergence!)

```python
df_metrics_delta0 = get_metrics(df, delta=0.01, prob=0.999, which_fun_key='fun_exact', which_time_key='total_q_shots')
df_metrics_delta0 = df_metrics_delta0[(df_metrics_delta0['num_repetitions']>1)]
```

```python
idxmin_delta0 = df_metrics_delta0['T_ave'].idxmin()
idxmin_delta0
```

```python
# Also, the runner-up entry with 100% convergence!
idxmin_prob100 = df_metrics_prob100['T_ave'].idxmin()
idxmin_prob100
```

```python
df_metrics_delta0.loc[[idxmin_delta0]]
```

```python
df.loc[idxmin_delta0]
```

```python
# Platform, Team, Experiment, minimizer Function, number of Samples, number of function eValuations, Repetition
(p,t,f,s,v,e) = idxmin_delta0
# Plot.
plot(df.loc[[(p,t,f,s,v,e,k) for k in range(len(df.loc[(p,t,f,s,v,e)]))]],
     xmax=9, xstep=1, ymin=-3.00, ymax=-0.00+0.01, legend_loc='upper right')
```

```python
# Print the minimizer source.
print(df.loc[(p,t,f,s,v,e,0)]['minimizer_src'])
```

### Plot convergence

```python
# # Plot all.
# plot(df, legend_loc='center')
```

```python
# Plot QPU only.
plot(df, platform_set=[platform_qpu], markersize_divisor=10,
     xmin=0, xmax=34+0.01, xstep=1, ymin=-2.80, ymax=-1.80-0.01, ystep=0.05, legend_loc='lower left')
```

```python
# Plot COBYLA only.
plot(df, minimizer_method_set=['my_cobyla'], sample_number_set=[50], markersize_divisor=5,
     xmin=5, xmax=32+0.01, xstep=1, ymin=-2.89, ymax=-1.79+0.01, ystep=0.05, legend_loc='upper right')
```

### Plot execution metrics

```python
# plot_metric(df)
```

```python
# plot_metric(df, metric='total_q_shots')
```


```python
%load_ext watermark
%watermark -a 'cs224' -u -d -v -p numpy,pandas,matplotlib,sklearn,h5py,pytest
```

    Author: cs224

    Last updated: 2021-09-10

    Python implementation: CPython
    Python version       : 3.8.10
    IPython version      : 7.22.0

    numpy     : 1.20.2
    pandas    : 1.2.5
    matplotlib: 3.3.4
    sklearn   : 0.24.2
    h5py      : 2.10.0
    pytest    : 6.2.4

```python
%matplotlib inline
import numpy as np, scipy, scipy.stats as stats, pandas as pd, matplotlib.pyplot as plt, seaborn as sns
import sklearn, sklearn.pipeline, sklearn.model_selection, sklearn.preprocessing, sklearn.linear_model

pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
# pd.set_option('display.float_format', lambda x: '%.2f' % x)
np.set_printoptions(edgeitems=10)
np.set_printoptions(linewidth=1000)
np.set_printoptions(suppress=True)
np.core.arrayprint._line_width = 180

SEED = 42
np.random.seed(SEED)

sns.set()
```

```python
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:60% !important; }</style>"))
```

<style>.container { width:60% !important; }</style>

```python
# import os,sys
# sys.path.append(os.path.realpath(os.path.abspath('') + '/../../lib'))
```

```python
# %load_ext autoreload
# %autoreload 1
# %aimport somemodule
```

```python
from IPython.display import display, HTML

from IPython.display import display_html
def display_side_by_side(*args):
    html_str=''
    for df in args:
        if type(df) == np.ndarray:
            df = pd.DataFrame(df)
        html_str+=df.to_html()
    html_str = html_str.replace('table','table style="display:inline"')
    # print(html_str)
    display_html(html_str,raw=True)

CSS = """
.output {
    flex-direction: row;
}
"""

def display_graphs_side_by_side(*args):
    html_str='<table><tr>'
    for g in args:
        html_str += '<td>'
        html_str += g._repr_svg_()
        html_str += '</td>'
    html_str += '</tr></table>'
    display_html(html_str,raw=True)

display(HTML("<style>.container { width:70% !important; }</style>"))
```

<style>.container { width:70% !important; }</style>

```python
# import importlib, logging
# importlib.reload(logging)
# logging.basicConfig(level=logging.DEBUG)
# logging.info('test')
```

```python
# import warnings
# with warnings.catch_warnings():
#     warnings.simplefilter("ignore")
#     warnings.filterwarnings("ignore", module='xxx_module_xxx')
#     warnings.filterwarnings('error', category=RuntimeWarning, module='empyrical.stats')

# with warnings.catch_warnings(record=True) as w:
#     # Cause all warnings to always be triggered.
#     warnings.simplefilter("always")

# warnings.filterwarnings('error, message='', category=Warning, module='', lineno=0, append=False)
# warnings.filterwarnings('error', module='pandas')
# warnings.filterwarnings('ignore', module='numba_scipy')
# warnings.filterwarnings('ignore', module='zipline.assets')
```

```python
N_subjects = 1000

X = stats.norm(loc=0, scale=10).rvs(size=(N_subjects,1), random_state=np.random.RandomState(43))
X[:5,:]
```

    array([[ 2.57399925],
           [-9.08481433],
           [-3.78503106],
           [-5.34915599],
           [ 8.58073346]])

```python
a = 0.5
b = 1
y = a * X + b
y = y + stats.norm(loc=0, scale=0.5).rvs(size=(N_subjects,1), random_state=np.random.RandomState(44))
y[:5,:]
```

    array([[ 1.91169227],
           [-2.8842285 ],
           [-0.26944552],
           [-2.47703586],
           [ 4.55629489]])

```python
x_min = np.min(X)
x_max = np.max(X)
x_lin = np.linspace(x_min, x_max, 100)
y_lin = a * x_lin + b
```

```python
lr_poly = sklearn.linear_model.LinearRegression()
lr_poly.fit(X, y)
lr_poly.intercept_, lr_poly.coef_
```

    (array([0.99580863]), array([[0.49856981]]))

```python
plt.figure(figsize=(15, 8), dpi=80, facecolor='w', edgecolor='k')
ax = plt.subplot(1, 1, 1)
ax.scatter(X,y)
ax.plot(x_lin,y_lin)
```

# LaTeX

Here is a `newcommand` (see in the source):
$$\newcommand{gpvec}[1]{{\bf #1}}$$
$$\newcommand{bfvec}[1]{{\pmb{#1}}}$$
$$p(f_*|x_*, X, y) = \mathcal{N}\left(\frac{1}{\sigma_n^2}\gpvec{x_*}^TA^{-1}X\gpvec{y},\; \gpvec{x_*}^TA^{-1}\bfvec{x_*}\right)$$
$$A=\frac{1}{\sigma_n^2}XX^T+\Sigma_p^{-1}$$

$\displaystyle
\begin{array}{rcl}
\text{μ} & \sim & a\\
\text{σ} & \sim & b\\
\end{array}
$

$$
\begin{align}
s_{t+1}=\begin{pmatrix}\beta_{t+1}\\\alpha_{t+1}\end{pmatrix}&=&\begin{pmatrix}\beta_t\\\alpha_t\end{pmatrix}+\eta_t&,\qquad& \eta_t \sim N(0, I_2\sigma_\eta^2)\\
y_t &=& Z_t\cdot s_t + \varepsilon_t = \begin{pmatrix}x_t&1\end{pmatrix}\cdot\begin{pmatrix}\beta_t\\\alpha_t\end{pmatrix} + \varepsilon_t =\beta_tx_t+\alpha_t \
&,\qquad&\varepsilon_t \sim N(0, \sigma_\varepsilon^2)
\end{align}
$$

Given a joint distribution:
$$\left[\begin{matrix}x\\ y\end{matrix}\right]\sim\mathcal{N}\left(
\left[\begin{matrix}\mu_x\\\mu_y\end{matrix}\right],
\left[\begin{matrix}A&C\\ C^T&B\end{matrix}\right]
\right)$$

We get the conditional distribution analytically as follows:

$$p(x\,|\, y)\sim\mathcal{N}(\mu_x+CB^{-1}(y-\mu_y), A-CB^{-1}C^T)$$

And as we set the mean to $0$ this reduces to:

$$p(x\,|\, y)\sim\mathcal{N}(CB^{-1}y, A-CB^{-1}C^T)$$

# SymPy

```python
import sympy
sympy.init_printing()
```

```python
def myfn(x):
    return (x+1)*(x-2)*(x-2.5)*(x-4)

symx = sympy.symbols('x')
sympy.expand(myfn(symx))
```

```python
sx = sympy.symbols('x')
sfx = \
      -0.0000009957371   * sx**16 + \
       0.000056100257831 * sx**15 + \
      -0.0014371151322   * sx**14 + \
       0.022152182049259 * sx**13 + \
      -0.229167949577895 * sx**12 + \
       1.680307250977133 * sx**11 + \
      -8.98984498449258  * sx**10 + \
      35.59846747504916  * sx** 9 + \
    -104.74198119349185  * sx**8  + \
     227.94559624952703  * sx**7  + \
    -362.2994941897943   * sx**6  + \
     411.6650485438511   * sx**5  + \
    -323.9001750849299   * sx**4  + \
     168.63970442305256  * sx**3  + \
     -54.90472890518606  * sx**2  + \
      11.515498193577798 * sx
sfx2 = sfx**2
isfx2 = sympy.Integral(sfx2, sx)
isfx2
```

```python
isfx2 = sympy.integrate(sfx2, (sx,0,6.56))
isfx2*sympy.pi
```

```python
precision=50
sympy.N(isfx2*sympy.pi, precision)
```

```python
sympy.plotting.plot(sfx, (sx, 0, 6.56))
```

```python
sa, sb, sc, sd = sympy.symbols(['a', 'b', 'c', 'd'])
se, sf, sg, sh = sympy.symbols('e f g h')
```

```python
sm = sympy.Matrix([[sa, sb], [sc, sd]])
sm, sm.inv()
```

```python
sympy.S("{:6.2f}".format(1.23))
```

```python
solution = sympy.solveset(sympy.Eq(sx**2 - 1, 0), sx)
solution
```

```python
list(solution)[0]
```