```python
import pandas as pd
import ms3
from  ms3.utils import *
import os
from ms3 import Score
import matplotlib.pyplot as plt
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
from scipy import stats
# include directory
home_dir = '/home/nulpe/Desktop/Tresillo/'
```

# Towards a Rhythmical Definition of the Tressilio beat and a Tracing of it in Popular Music

## 1) Introduction & Research Question

### 1.1) Research question:
"Can we compute to which extent a defined rhythm, which we refer to as Tresillo rhythm, is used in a given pop song and if so can we measure the intensity of Tresillo rhythm use in top-20-billboard songs over the past twenty years?"

**Discussion:**
In our project we would like to discuss the use of a rhythm, which we refer to as 'Tresillo rhythm', in the popular music of the last 20 years. We define this rhythm in our project, given secondary literature, and thus obtain a precise notation and formulization of the Tresillo rhythm. Given this definition we can then compute the similarity between the Tresillo rhythm and the rhythm of a given pop song. Thus, we hope to obtain a similarity coefficient which measures how similar the rythm of a given pop song is to our self defined Tresillo rythm. Given the computed similarity coefficients, we hope to measure the use of the Tresillo rhythm in the top 20 billboard songs of the past 20 years (1999-2019).

### 1.2) Assumptions

- We assume that for most pop songs present in the billboaard charts, one can identify one dominant rythm per song.This assumption is supported by [insert bibliographie or use our rythm space and a time trajectory over the bars of the songs and show that a vector of a song is not changing a lot over the bars of the song].
- For our presntation of songs in rythm space, we also assume that the majority of the songs in the billboard charts is in the time signature of 4/4. The following data exploration justifies this assumption.(There are 420 songs with 4/4, 6 with 3/4 and 16 with changing time signature).
- We assume that a time sample longer than [40] seconds is suitable to present the main rythm of a pop song. We need to discuss which treshold we want to set. The lower the more songswe have ,the higher the time treshold the weaker our assumption.

```python
#Some numbers regarding our song length assumption, not adressed in the former notebooks
list_midis = os.listdir(r"C:/Users/Florian/Documents/GitHub/Tresillo/dataset/project_midi/billboard")
length_midis = []
for el in list_midis:
    midi_data = pretty_midi.PrettyMIDI("C:/Users/Florian/Documents/GitHub/Tresillo/dataset/project_midi/billboard" +'/'+el)
    length_midis.append(midi_data.get_end_time())

average_midi_length = (sum(length_midis)/len(length_midis))
plt.figure(1)
plt.hist(length_midis, 20)
midi_length_standart_dev = statistics.stdev(length_midis)
counter_30 = 0
counter_40 = 0
counter_30 = sum (i < 30 for i in length_midis)
counter_40 = sum (i < 40 for i in length_midis)

print("Samples shorter than 30 sec: " + counter)
print("Samples shorter 40 sec: "  + counter_40)
print ("Average sample length" + average_midi_length)
print("Standart deveiation sample length" + midi_length_standart_dev)
```

### 1.3) Data Representation
Initially our data is represented in the MIDI file format. The representation of music in the MIDI format has the advantage, that often several voices of different instruments are represented in such files. In contrast to musescore, where often only the voice of one instrument (mostly piano) is notated.
However, to obtain a list of onsets of every musical event in a given song, we have to convert our MIDI (.midi) files to Musescore (.mscx) files.
To convert and further analyze our files, we will use the [ms3](https://pypi.org/project/ms3/) python library.
To convert a directory of .midi files to .mscx files we use following command:

```python
path_midi = '/home/nulpe/Desktop/Tresillo/dataset/project_midi/tresillo/'
target = '/home/nulpe/Desktop/Tresillo/dataset/project_mscx/mscx_tresillos_billboard/'

dir_list = os.listdir(path_midi)

for el in dir_list:
    convert(path_midi+el, target+el[:-4]+'.mscx', MS='musescore3')
```

**TA instructions**: Precision of Research Question

- State the final version of your research question as you understand it now.

-  Make all assumptions and hypotheses explicit.

- In case Milestone 2 did not include the final data representation that you are analyzing, present the representation you finally used and the necessary steps to get to it.

**Thoughts Aurel:**
 - Reasearch Question: Move to a fuzzy definition of Tresillio-ness: E.g.:
    Are pop songs increasingly using a rythm pattern which is similar to a rythm pattern which we reffer to as 'Tresillio rythm'?

 - Assumptions are very important for them. Here we have to note our definition(s) of the Tresillio rythm pattern and how we derive them (incl sources). Furthermore, we have to discuss cases where there are Rythms which are similar to the Tresillio rythm but not equivalent (e.g.: reggaeton) and how we deal with them computationally.

 - Here we have to discuss the conversion of our MIDI files to the musescore3 file format. Furthermore, we have to discuss what the musescore3 format offers us, and why it is the better choice for our analysis.

  **Toughts Florian:**
    -Assumptions:

    -30 second piece of pop song is suitable to identfy the main rythm

    -Main rythm can be identified by counting the onsets

    -the great majority of pop songs comes in 4/4 (not shure if we really need that assumption)

## 2) Methods

### 2.1) Definition of the Tresillo rhythm
We notate a clean version of a rythm wich we from now on use as our defintion for the clean tresillo rythm, in the following context also called vanilla tresillo.

The rhythm pattern consists of a dotted eighth note, followed by a sixteenth note, an eighth rest and an eighth note and is repeated two times in a 4/4 bar. The rytm pattern is beeing used as a own rythm or as a rythmic part of a more complex rythm, for example the "clave" pattern or the ragaetton rythm.
By notating the rythm in MuseScore 3 and saving it as a .mscx file, we can use it our data processing pipeline described in the following sections.

### 2.2) Rhythm histograms and vectors

To be able to measure the similarity between rhythm we must have a clear definition and thus following representation of rhythm. In general, one can define rhythm as "a series of onsets and durations of musical events.” (Rohrmeier, 2020). In our specific case however, we are interested in the dominat and repeating rhythm of a given song. Therefore we prefer a narrower definition of rhythm as “repeated recurrences in alternate heavy and light beats” (Chin and Wu, 1992). To furthermore simplify our data, we assume that the main rhythm of a song can be defined by the onsets  of its musical events (notes).
To obtain a representation of the dominate rhythm of a song, we preceed to aggregate all musical onset to one bar. Collapsing all musical onsets to one bar and thus obtaining onset 'histograms' is a common pratice and has been used beside others to analyze western classical music (Palmer and Krumhansl, 1990) and american folk music (Huron and Ommen, 2006).
With the onset histogram of a song we can compile a n dimensional vector for each song, which we refer to as a 'rhythm vector'. Given that the number of songs with meters other than 4/4 is negible, we only consider songs with a 4/4 meter in our analysis.  Given that we only consider songs with 4/4 meters, we obtain for each song a 16 dimensional vector.

### 2.3) Naive approach: Rhythm simularity measured with cosine simularity
Given the 16 dimensional rhythm vectors we obtain following the methode described above, we can now compute simple similarity metrics.
In rhythm analysis a common similarity metric which is used to calculate the similarity between two rhythm vectors is the cosine distance (see: Panteli et al., 2014; Parry and Essa, 2003).  The cosine similarity metric is scale invariante, which as such is interesting for rhythm similarity given that thus only relative frequencies of onsets are important and not absolute frequencies.
The cosine distance between two vectors A and B is defined as following:
\begin{equation}
\cos ({\bf A},{\bf B})= {{\bf A}*{\bf B} \over \|{\bf A}\| \|{\bf B}\|} = \frac{ \sum_{i=1}^{n}{{\bf A}_i{\bf B}_i} }{ \sqrt{\sum_{i=1}^{n}{({\bf A}_i)^2}} \sqrt{\sum_{i=1}^{n}{({\bf B}_i)^2}} }
\end{equation}

Given the definition of the cosine similarity we can now compute the similarity between our self defined Tresillo rhythm and the billboard songs.
First, however we will validate this similarity metric by testing it on our self compiled list of songs which do comprise a Tresillo rhythm and songs which do not comprise a Tresillo rhythm. We then compute the mean 'Tresillo-ness' (similarity to Tresillo rhythm) of both samples. By employing the Bootstrapping method we can also obtain a measurement of uncertainty, as provided by 2.5% and 97.5% confidence intervals.

### 2.4) Reducing noise: (pushkar)

**TA instructions**:
- How did you deal with the problems you mentioned in Milestone 2?

- Which methods did you use to obtain the final results? Provide self-contained explanations and make sure to cite relevant literature where appropriate.

- Explain your core calculations using equations.

- Do not describe all the methods you tried out but only those that lead to the results: the final analysis is not an exploratory analysis anymore.

- Specify any adjustments you made to pre-existing methods

**Thoughts Aurel**:
 -  Talk about how we got to the bar representation of our music. Furthermore, also discuss how we get to a 'perfect Tresillio Histogram'

 - Here I propose we try out several things and compare the results of several methods:

  a) A first big topics is how we define the perfect Tresillo:
   1. Given predefined rythm patterns (by Florian)
   2. Given songs with high Tresillio-ness
    - All instruments collapsed
    - Only key instruments
    - Certain instruments

  a) The second big question is how do we measure Tresillio-ness in the pop songs, I would suggest three approaches. All aproaches require that we first obtain 16 dimensional Rythm vectors of each song of interest:
   1. Compare our vanilla self defined Tresillio rythm vector with all vectors of our pop songs. Measure distance or simularity with some commonly used distance measure in the literature
   2. Very similar to 1) but this time use the Tresillio rythm vector as defined by our songs
   3. Prior clustering of the rythm vectors. Obtaining centroid and measuring with it Tresillio ness in the charts (method as proposed by Pushkar)

- Equations should be included in the prior part
- Discuss critically any outliers, problems and limitations of our methodology. Extra focus on the question how we deal with related but not the same rythm (e.g.: Reggaeton)

**Toughts Florian**
- Following the last paper discussion one measuurement we could use is the Cosin distance
- Not sure about an instrument selection. We can mention that a reduction seems not favorable as the tresillo rythm  is presented with different instruments throughout the dataset

## 3) Final Results

### 3.1) Onset histograms and rhythm vectors
In this first part we will use onset histograms to compute rhythm vectors.
To obtain the onset histogram of a given song, we use the notes representation provided by the [ms3](https://pypi.org/project/ms3/) libary and colapse all musical onsets to one bar. In the example below we will compile the histograms for our self defined 'Vanilla Tresillo' and for the example song 'shape of you' by Ed Sheran. Then we will proceed to compute the rhythm vectors for both songs

```python
#paths to both examples
shape_of_you = home_dir+'dataset/project_mscx/mscx_tresillos_billboard/Shape of you-Ed Sheran.mscx'
vanilla_tresillo = home_dir+'dataset/project_mscx/mscx_tresillos/Vanilla_Tresillo.mscx'

# get the note scores of both examples
df_shape_of_you = Score(shape_of_you).mscx.notes
df_vanilla_tresillo = Score(vanilla_tresillo).mscx.notes

#calculate quarter note position for each note
df_shape_of_you['quarter_beats'] = (df_shape_of_you.mc_onset*16).astype('int32')
df_vanilla_tresillo['quarter_beats'] = (df_vanilla_tresillo.mc_onset*16).astype('int32')

fig, ax = plt.subplots(1,2, figsize=(12,3))

ax[0].hist(df_shape_of_you['quarter_beats'], bins=16)
ax[1].hist(df_vanilla_tresillo['quarter_beats'], bins=16)
ax[0].set_xlabel('quarter_beats')
ax[1].set_xlabel('quarter_beats')
ax[0].xaxis.set_ticks(np.arange(0, 16, 1))
ax[1].xaxis.set_ticks(np.arange(0, 16, 1))
ax[0].set_ylabel('count')
ax[0].set_title('Shape of you')
ax[1].set_title('Vanilla Tresillo')

#mention somewhere that we are working with quarter notes
```

In a next step we want to compile rhythm of each song given this notion of histograms. E.g: every dimension incorporates the absolute frequency of onsets on one given quarter note. This is done as follows:

```python
rhythm_vector_shape_you = df_shape_of_you.groupby(['quarter_beats'])['mn'].agg(['count'])
rhythm_vector_shape_you  = rhythm_vector_shape_you.reindex(list(range(0,16)),fill_value=0).T
rhythm_vector_shape_you
```

      quarter_beats
      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15

      count
      206
      33
      44
      119
      72
      37
      169
      12
      200
      29
      75
      112
      75
      27
      167
      10

In the assumption that the rhythm vectors of distinct voices within a song might include information we want to preserve, we compiled one rhythm vector per instrument in a song as follows:

```python
# Define instruments
shape_of_you_score = Score(shape_of_you)
instrument_dict = {}
for key in shape_of_you_score.mscx.metadata['parts']:
    for staff in shape_of_you_score.mscx.metadata['parts'][key].keys():
            instrument_dict[staff] = key

#staff to voice/instruments
df_shape_of_you['instrument'] = [instrument_dict[el] if el in instrument_dict else 'na' for el in df_shape_of_you.staff]

#compute rhythm vectors per voice
rhythm_vector_shape_you_instruments = df_shape_of_you.groupby(['instrument','quarter_beats'])['mn'].agg(['count'])
rhythm_vector_shape_you_instruments = rhythm_vector_shape_you_instruments.groupby(level=0).apply(lambda x: x.reset_index(level = 0).drop(['instrument'],axis=1).reindex(list(range(0,16)),fill_value=0).T)
rhythm_vector_shape_you_instruments = rhythm_vector_shape_you_instruments.reset_index()
rhythm_vector_shape_you_instruments
```

      quarter_beats
      instrument
      level_1
      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15

      0
      Grand Piano
      count
      66
      0
      0
      0
      0
      0
      0
      0
      66
      0
      0
      0
      0
      0
      0
      0

      1
      Marimba, untitled
      count
      66
      0
      0
      53
      14
      0
      63
      0
      68
      0
      0
      57
      17
      0
      63
      0

      2
      Melodic Drum
      count
      0
      0
      33
      36
      13
      19
      39
      0
      0
      0
      50
      28
      10
      19
      38
      0

      3
      Overdrive Gtr
      count
      6
      2
      1
      1
      4
      4
      6
      2
      6
      1
      6
      2
      6
      1
      6
      0

      4
      Percussion
      count
      24
      23
      0
      2
      19
      2
      22
      2
      21
      21
      1
      2
      19
      2
      23
      2

      5
      Tenor Sax
      count
      20
      8
      10
      8
      16
      12
      16
      8
      16
      7
      18
      6
      15
      5
      14
      8

      6
      Woodblock
      count
      2
      0
      0
      2
      0
      0
      2
      0
      2
      0
      0
      1
      2
      0
      2
      0

      7
      Xylophone
      count
      22
      0
      0
      17
      6
      0
      21
      0
      21
      0
      0
      16
      6
      0
      21
      0

If we want to compile rhythm vectors (per voice) for all mscx files in one directory we can use following loop:

```python
def rythm_vectors(in_dir, out_dir):
    list_sheet_music = os.listdir(in_dir)
    df_rythm_vectors =[]

    for idx, el in enumerate(list_sheet_music):
        if el[-4:] == 'mscx':

            #Get notes with onsets
            s = Score(dir_sheet_music+el)
            df = s.mscx.notes

            # Define instruments
            instrument_dict = {}
            for key in s.mscx.metadata['parts']:
                for staff in s.mscx.metadata['parts'][key].keys():
                    instrument_dict[staff] = key

            #staff to instruments
            df['instrument'] = [instrument_dict[el] if el in instrument_dict else 'na' for el in df.staff]

            # define quarter beat
            df['quarter_beats'] = (df.mc_onset*16).astype('int32')

            #make rythm matrix & data frame
            df_histogram = df.groupby(['instrument','quarter_beats'])['mn'].agg(['count'])
            df_histogram = df_histogram.groupby(level=0).apply(lambda x: x.reset_index(level = 0).drop(['instrument'],axis=1).reindex(list(range(0,16)),fill_value=0).T)
            df_histogram = df_histogram.reset_index()

            df_histogram.insert(loc=0, column='song_artist', value=el[:-5])

            #concat to big rythm vector df
            if len(df_rythm_vectors) == 0: df_rythm_vectors = df_histogram

            df_rythm_vectors = pd.concat([df_rythm_vectors,df_histogram], axis=0)

    df_rythm_vectors.to_csv(out_dir, index = False)

dir_sheet_music = home_dir + '/dataset/project_mscx/mscx_billboard/'
out_dir = home_dir + '/dataset/rythm_vectors/rythm_vectors_billboard.csv'
rythm_vectors(dir_sheet_music, out_dir)
```

### 3.2) Tresilio-ness with Cosine Similarity
In a first naive analysis we will employ the cosine similarity measurement to assess if the vector of a given song is similar to our defined Tresillo rhythm. We will need some helper functions.
Following two helper functions will help us to collapse all rhythms of all instruments to one rhythm vector per song.
The second function calculates the cosine similarity between a pandas data frame of rhythm vectors and one single rhythm vector.
The third function calculates the 2.5% and the 97.5% confidence intervals of the distribution of a mean of a given data set. This function will allow us to assess, how big the uncerntaingty is in our data set and if the means of two distributions are indeed significantly different.

```python
def collapse_normalize_vectors(df_rythm_vectors):
    rythm_vector_collaped = df_rythm_vectors.groupby(['song_artist']).agg(['sum'])
    rythm_vector_collaped.columns = rythm_vector_collaped.columns.droplevel(1)
    rythm_vector_collaped = rythm_vector_collaped.drop(['instrument', 'level_1'],axis=1)
    rythm_vector_collaped["sum"] = rythm_vector_collaped.sum(axis=1)
    rythm_vector_collaped = rythm_vector_collaped.loc[:,"0":"15"].div(rythm_vector_collaped["sum"], axis=0)
    return rythm_vector_collaped

def calc_cosine_sim(rythm_vectors, tresillo_vector):
    rythm_vectors['cosine_sim_tresillo'] = cosine_similarity(rythm_vectors.loc[:,"0":"15"],tresillo_vector)
    return rythm_vectors

def bootstrap_CI(data, nbr_draws):
    means = np.zeros(nbr_draws)
    data = np.array(data)

    for n in range(nbr_draws):
        indices = np.random.randint(0, len(data), len(data))
        data_tmp = data[indices]
        means[n] = np.nanmean(data_tmp)
    return [np.nanpercentile(means, 2.5),np.nanpercentile(means, 97.5)]
```

Now lets read in the rhythm vector of our vanilla Tresillo.

```python
df_synt_tresillo = pd.read_csv(home_dir + '/dataset/rythm_vectors/rythm_vectors_tresillio.csv') # read in all defined 'tresillos' and variations
df_vanilla_tresillo = df_synt_tresillo[df_synt_tresillo['song_artist']=='Vanilla_Tresillo'].loc[: ,"0":"15"] # only use the 16d vector of our vanilla_tresillo
vector_vanilla_tresillo = np.asarray(df_vanilla_tresillo)
vector_vanilla_tresillo
```

    array([[20,  0,  0, 20,  0,  0, 20,  0, 20,  0,  0, 20,  0,  0, 20,  0]])

We will now calculate the cosine similarity between our vanilla Tresillo vector and between a hand selected set of Tresillo songs.
This set of songs has been selected by us ourself and none of those 'validation' songs  are included in the billboard data sets.

```python

tresillo_test_set_vectors = pd.read_csv(home_dir + '/dataset/rythm_vectors/rythm_vectors_tresillo_not_billboard.csv')
#tresillo_test_set_vectors = pd.read_csv(home_dir + '/dataset/rythm_vectors/rythm_vectors_tresillos_billboard.csv')
tresillo_test_set_vectors = collapse_normalize_vectors(tresillo_test_set_vectors)
tresillo_test_set_vectors = calc_cosine_sim(tresillo_test_set_vectors, vector_vanilla_tresillo)
print(tresillo_test_set_vectors['cosine_sim_tresillo'])
```

    song_artist
    Attention Charlie Puth                     0.613390
    Chandelier Sia                             0.605698
    Cold Water Major Lazer                     0.896750
    Hips Don't Lie Shakira                     0.819003
    I Don't Care Ed Sheeran & Justin Bieber    0.876542
    One Dance - Drake & Wizkid, Kyla Reid      0.717454
    Rockabye Baby Clean Bandit                 0.880677
    Sorry Justin Bieber                        0.853424
    Titanium David Guetta                      0.652046
    Name: cosine_sim_tresillo, dtype: float64

Let's calculate the mean Tresillo-ness and also the 2.5% and 97.5% confidence intervals of the mean as obtained by bootstrapping.

```python
print('mean Tresillo-ness in the test set: ', tresillo_test_set_vectors['cosine_sim_tresillo'].mean())
lower_tresillo_ci, upper_tresillo_ci = bootstrap_CI(tresillo_test_set_vectors['cosine_sim_tresillo'], 100)
print('tresillo upper and lower ci on 100 draws: ', lower_tresillo_ci, upper_tresillo_ci)
```

    mean Tresillo-ness in the test set:  0.7683315382006799
    tresillo upper and lower ci on 100 draws:  0.6808770780048731 0.8481138158323536

Let's do exact same thing, but with a comparison data set of songs which we kno to not include any Tresillo songs.
Also calculating the mean and the confidence intervals of the not Tresillo songs.

```python
non_tresillio_vectors =  pd.read_csv(home_dir + '/dataset/rythm_vectors/rythm_vectors_not_tresillo_validation.csv')
#non_tresillio_vectors = pd.read_csv(home_dir + '/dataset/rythm_vectors/rythm_vectors_no_tresillos_4_4.csv')

non_tresillio_vectors = collapse_normalize_vectors(non_tresillio_vectors)
non_tresillio_vectors = calc_cosine_sim(non_tresillio_vectors, vector_vanilla_tresillo)
print(non_tresillio_vectors['cosine_sim_tresillo'])

print('mean Tresillo-ness in the test set: ', non_tresillio_vectors['cosine_sim_tresillo'].mean())
lower_non_tresillo_ci, upper_non_tresillo_ci = bootstrap_CI(non_tresillio_vectors['cosine_sim_tresillo'], 100)
print('tresillo upper and lower ci on 100 draws: ', lower_non_tresillo_ci, upper_non_tresillo_ci)
```

    song_artist
    Dress You Up           0.666450
    Fear of the Dark       0.689443
    Folsom Prison Blues    0.657568
    Immigrant Song         0.584275
    Jailhouse Rock         0.593940
    One Step Closer        0.523399
    Sweet Child O'Mine     0.724700
    Sweet Home Alabama     0.534338
    Name: cosine_sim_tresillo, dtype: float64
    mean Tresillo-ness in the test set:  0.6217641548092555
    tresillo upper and lower ci on 100 draws:  0.5792063648094466 0.6717705312519476

Already looking at the means of the two samples and their confidence intervalls, they seem to be significantly different.
However let's also calculate the t-test statistics to ensure that the distributions are actually different.

```python
ttest_pvalue = stats.ttest_ind(tresillo_test_set_vectors['cosine_sim_tresillo'], non_tresillio_vectors['cosine_sim_tresillo']).pvalue
print('p value that the means are the same: ', ttest_pvalue)
```

    p value that the means are the same:  0.009618772484088895

**Discussion:**
Looking at the the Tresillio cosine similarity metrics of both data sets and comparing the distributions of the means whit each others (confidence intervals and t-test), we see that their mean Tresillo-ness is indeed significantly diffferent.
However, we also see that 1) this Tresillo-ness measurement seems to be quite noisy 2) The distribution of the two samples are not as far apart as we would like them to be.
The noisiness of this metric, can be especially seen, if one looks at the individual cosine Tresillo-ness values of the songs. Here we see that there is great variance in Tresillo-ness in the hand selected Tresillo data set. Values can be as small as 0.6, which corresponds to the value of a none Tresillo song.
Furthermore, it seems that also songs wich we classified to have no Tresillo, have a cosine Tresillo-ness up to 0.72.
In general we can state that the cosine similarity methods seems to identify a ceratin Tresillo-ness, however it is questionable how robust this method is.

### 3.3) Tresilio-ness over Time

Now that we explored several ways to compute Tresillo-ness, we can calculate the Tresilo-ness of ou billboard data set.
First let us load the rhythm vectors of our billboard songs and let's also merge it to the metadata of the billboard songs.
The metadata of the billboard songs includes time signatures with which we can Tresillo-ness over time.

```python
df_billboard = pd.read_csv(home_dir + '/dataset/rythm_vectors/rythm_vectors_billboard.csv')
df_billboard_meta = pd.read_csv(home_dir + '/dataset/billboard_data_sets/billboard_1999-2019_unique_top_20_selection.csv')

#calculate cosine simularity
df_billboard = collapse_normalize_vectors(df_billboard)
df_billboard_sim = calc_cosine_sim(df_billboard, vector_vanilla_tresillo)

#prepare for merge with meta data
df_billboard_sim['song_artist'] = df_billboard_sim.index
df_billboard_sim['song'] = df_billboard_sim.song_artist.apply(lambda x: x.split('_')[0])
df_billboard_sim['artist'] = df_billboard_sim.song_artist.apply(lambda x: x.split('_')[1][:-1])
df_billboard_sim = df_billboard_sim.drop(['song_artist'], axis=1)

#merge data frames
df_billboard_merged = df_billboard_sim.merge(df_billboard_meta, left_on=['song','artist'], right_on=['Name', 'Artists'], how='left')
df_billboard_reduced = df_bilboard_merged[['Name', 'Artists', 'Peak.position', 'Week', 'Genre', 'cosine_sim_tresillo']]
#let's look at some songs with very high tresillo-ness
df_billboard_reduced[df_billboard_reduced['cosine_sim_tresillo']>0.85]

```

      Name
      Artists
      Peak.position
      Week
      Genre
      cosine_sim_tresillo

      89
      Cheap Thrills
      Sia
      19.0
      2016-06-18
      Jamaica,Remix,Australia,Rap,Synth-Pop,Pop
      0.908945

      293
      Rather Be
      Clean Bandit
      19.0
      2014-08-08
      Electro-Pop,Deep House,Eurodance,Dance-Pop,Cla...
      0.861075

      319
      Shape Of You
      Ed Sheeran
      1.0
      2017-01-28
      Tropical House,Dancehall,Pop,UK
      0.915565

      387
      Treat You Better
      Shawn Mendes
      20.0
      2016-07-30
      Canada,Pop
      0.883772

      389
      Turn Me On
      Kevin Lyttle
      20.0
      2004-06-23
      Pop
      0.937243

      423
      Where Have You Been
      Rihanna
      18.0
      2012-05-25
      House,Electro-Pop,Pop
      0.947448

Let's plot weekly cosine Tresillo-ness

```python
df_bilboard_merged_weekly = df_bilboard_merged.groupby(['Week'])['cosine_sim_tresillo'].agg(['mean']).reset_index()
plt.plot(pd.to_datetime(df_bilboard_merged_weekly.Week), df_bilboard_merged_weekly['mean'])
plt.title('Weekly Cosine Tresillo-ness in the Top 20 Billboards')
plt.xlabel('Time')
plt.ylabel('Cosine Tresillo-ness')
```

The time trend above is very noisy and hard to read. By calculating a 4 weeks moveing average we might get smoother results.

```python
df_bilboard_merged_weekly['rolling_mean'] = df_bilboard_merged_weekly.iloc[:,1].rolling(window=4).mean()

plt.plot(pd.to_datetime(df_bilboard_merged_weekly.Week), df_bilboard_merged_weekly['rolling_mean'])
plt.title('4 Weeks Moving Average Cosine Tresillo-ness in the Top 20 Billboards')
plt.xlabel('Time')
plt.ylabel('Cosine Tresillo-ness')
```

```python

```

```python

```

**TA instructions**:
- Present your results in relation to your research question.
- Present them in a logical order that does not have to be the order in which you achieved them.

**Thoughts Aurel:**

 4. Discussion of the clustering method, either k-mean clustering or something we dont have to set the cluster number
 5. Finding Tresillio-ness in the pop charts with all three methods a) Vanilla Tresillio-ness b) Tresillio songs vector c)

## 4) Outlook on final interpretation

Points to discuss as stated by TAs:
- Interpreting your results is the final step that you will do in preparing Milestone 4 (your presentations). Please end your submission by giving a first,preliminary outlook on this final step: what aspects of your results do you find interesting with respect to your hypotheses and previous literature? What do you think might be the main points to elaborate upon in the discussion?

```python

```

```python

```

## References

- Chin,  F.  and  Wu,  S.  (1992).   An  efficient  algorithm  for  rhythm-finding.Computer MusicJournal, 16(2):35–44.
- Dixon,  S.,  Gouyon,  F.,  Widmer,  G.,  et  al.  (2004).   Towards  characterisation  of  music  viarhythmic patterns.  InISMIR.
- Huron, D. and Ommen, A. (2006).  An empirical study of syncopation in american popularmusic, 1890–1939.Music Theory Spectrum, 28(2):211–231.
- Palmer, C. and Krumhansl, C. L. (1990).  Mental representations for musical meter.Journalof Experimental Psychology: Human Perception and Performance, 16(4):728.
- Panteli,  M.,  Bogaards,  N.,  Honingh,  A.  K.,  et  al.  (2014).   Modeling  rhythm  similarity  forelectronic dance music.  InISMIR, pages 537–542.
- Parry, M. and Essa, I. (2003).  Rhythmic similarity through elaboration.
- Pohle,  T.,  Schnitzer,  D.,  Schedl,  M.,  Knees,  P.,  and  Widmer,  G.  (2009).   On  rhythm  andgeneral music similarity.  InISMIR, pages 525–530.
- Rohrmeier, M. (2020).  Towards a formalization of musical rhythm.  InProc. of the 21st Int.Society for Music Information Retrieval Conf
- Floyd, Samuel A. "Black music in the circum-Caribbean." American Music (1999): 1-38.

```python

```


# Tutorial-IllinoisGRMHD: inlined_functions.C

## Authors: Leo Werneck & Zach Etienne

**This module is currently under development**

## In this tutorial module we explain a series of inline functions that are used by major functions within IllinoisGRMHD.

### Required and recommended citations:

* **(Required)** Etienne, Z. B., Paschalidis, V., Haas R., Mösta P., and Shapiro, S. L. IllinoisGRMHD: an open-source, user-friendly GRMHD code for dynamical spacetimes. Class. Quantum Grav. 32 (2015) 175009. ([arxiv:1501.07276](http://arxiv.org/abs/1501.07276)).
* **(Required)** Noble, S. C., Gammie, C. F., McKinney, J. C., Del Zanna, L. Primitive Variable Solvers for Conservative General Relativistic Magnetohydrodynamics. Astrophysical Journal, 641, 626 (2006) ([astro-ph/0512420](https://arxiv.org/abs/astro-ph/0512420)).
* **(Recommended)** Del Zanna, L., Bucciantini N., Londrillo, P. An efficient shock-capturing central-type scheme for multidimensional relativistic flows - II. Magnetohydrodynamics. A&A 400 (2) 397-413 (2003). DOI: 10.1051/0004-6361:20021641 ([astro-ph/0210618](https://arxiv.org/abs/astro-ph/0210618)).

# Table of Contents
$$\label{toc}$$

This module is organized as follows

0. [Step 0](#src_dir): **Source directory creation**
1. [Step 1](#introduction): **Introduction**
1. [Step 2](#pow): **`pow`**
1. [Step 3](#find_cp_cm): **`find_cp_cm`**
1. [Step 4](#compute_v02): **`compute_v02`**
1. [Step 5](#ppeos__c_code): **Polytropic Equations of State**
    1. [Step 5.a](#ppeos__c_code__prelim): *Preliminary treatment of the input*
        1. [Step 5.a.i](#ppeos__c_code__prelim__computing_ktab): Determining $\left\{K_{1},K_{2},\ldots,K_{\rm neos}\right\}$
        1. [Step 5.a.ii](#ppeos__c_code__prelim__computing_eps_integ_consts): Determining $\left\{C_{0},C_{1},C_{2},\ldots,C_{\rm neos}\right\}$
    1. [Step 5.b](#ppeos__c_code__eos_struct_setup) *Setting up the `eos_struct`*
    1. [Step 5.c](#ppeos__c_code__find_polytropic_k_and_gamma_index) *The `find_polytropic_K_and_Gamma_index()` function*
    1. [Step 5.d](#ppeos__c_code__compute_P_cold__eps_cold__dPcold_drho__eps_th__h__Gamma_cold): *The new `compute_P_cold__eps_cold__dPcold_drho__eps_th__h__Gamma_cold()` function*
        1. [Step 5.d.i](#ppeos__c_code__compute_P_cold__eps_cold__dPcold_drho__eps_th__h__Gamma_cold__case1__rhob_equal_zero): Case 1:  $\rho_{b} = 0$
        1. [Step 5.d.ii](#ppeos__c_code__compute_P_cold__eps_cold__dPcold_drho__eps_th__h__Gamma_cold__case2__single_polytropic_eos): Case 2: Polytropic EOSs
    1. [Step 5.e](#font_fix__rhob_loop): The `font_fix__rhob_loop()` function
1. [Step 6](#lower_4vector_output_spatial_part): **`lower_4vector_output_spatial_part`**
1. [Step 7](#impose_speed_limit_output_u0): **`impose_speed_limit_output_u0`**
1. [Step 8](#enforce_pressure_floor_ceiling): **`enforce_pressure_floor_ceiling`**
1. [Step 9](#compute_smallba_b2_and_u_i_over_u0_psi4): **`compute_smallba_b2_and_u_i_over_u0_psi4`**
1. [Step 11](#code_validation): **Code validation**
1. [Step 12](#latex_pdf_output): **Output this notebook to $\LaTeX$-formatted PDF file**

# Step 0: Source directory creation \[Back to [top](#toc)\]
$$\label{src_dir}$$

We will now use the [cmdline_helper.py NRPy+ module](Tutorial-Tutorial-cmdline_helper.ipynb) to create the source directory within the `IllinoisGRMHD` NRPy+ directory, if it does not exist yet.

```python
# Step 0: Creation of the IllinoisGRMHD source directory
# Step 0a: Add NRPy's directory to the path
# https://stackoverflow.com/questions/16780014/import-file-from-parent-directory
import os,sys
nrpy_dir_path = os.path.join("..","..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)

# Step 0b: Load up cmdline_helper and create the directory
import cmdline_helper as cmd
IGM_src_dir_path = os.path.join("..","src")
cmd.mkdir(IGM_src_dir_path)

# Step 0c: Create the output file path
outfile_path__inlined_functions__C = os.path.join(IGM_src_dir_path,"inlined_functions.C")
```

# Step 1: Introduction \[Back to [top](#toc)\]
$$\label{introduction}$$

In this tutorial notebook we explain functions of `IllinoisGRMHD` which are called for various purposes. This means that this notebook does not have a specific "theme". We will cover functions whose purposes vary from a simple optimization when squaring numbers to computing minimum and maximum characteristic speeds at cell interfaces.

We have tried our best to keep this tutorial module as independent from the others as possible. When new concepts appear, we offer useful references. The mathematical requirements of each function are also covered in great detailed.

# Step 2: `pow` \[Back to [top](#toc)\]
$$\label{pow}$$

This is an extremely simple function which simply checks whether or not we are trying to square a number before calling C's `pow()` function. This is because in C it is computationally quicker to do `x*x` than to use the function call `pow(x,2)`. Notice that we also use the "function" `SQR()`, which is declared in `IllinoisGRMHD_headers.h`, which is defined as

```c
#define SQR(x) ( (x) * (x) )
```

# Step 3: `find_cp_cm` \[Back to [top](#toc)\]
$$\label{find_cp_cm}$$

We will now explain the inlined function `find_cp_cm`. Keep in mind that this function depend on the function `compute_v02`, [which is implemented below](#compute_v02). This function is called with the objective of computing the minimum ($-$) and maximum ($+$) characteristic speeds at each cell interface, $c_{\pm}^{r,l}$.

We approximate the general GRMHD dispersion relation (eq. 27 of [Gammie & McKinney (2003)](https://arxiv.org/pdf/astro-ph/0301509.pdf)) by the simpler expression

$$
\omega_{\rm cm}^{2} = \left[v_{\rm A}^{2} + c_{\rm s}^{2}\left(1-v_{\rm A}^{2}\right)\right]k_{\rm cm}^{2}\ ,
$$

where $\omega_{\rm cm}=-k_{\mu}u^{\mu}$ is the frequency and $k_{\rm cm}^{2} = K_{\mu}K^{\mu}$ the wavenumber of an MHD wave mode in the frame comoving with the fluid, where $K_{\mu}$ is defined as the projection of the wave vector $k^{\nu}$ onto the direction normal to $u^{\nu}$: $K_{\mu} = \left(g_{\mu\nu}+u_{\mu}u_{\nu}\right)k^{\nu}$. $c_{\rm s}$ is the sound speed, and $v_{\rm A}$ is the Alfvén speed, given by

$$
v_{\rm A} = \sqrt{\frac{b^{2}}{\rho_{b}h + b^{2}}}\ .
$$

With these definitions, we may then solve the approximate dispersion relation above along direction $i$, noting that in the comoving frame $k_{\mu} = \left(-\omega,k_{j}\delta^{j}_{\ i}\right)$ and the wave (phase) velocity is $c_{\pm} = \left.\omega\middle/\left(k_{j}\delta^{j}_{\ i}\right)\right.$. The dispersion can then be written as a quadratic equation for $c_{\pm}$:

$$
ac_{\pm}^{2} + bc_{\pm} + c = 0\ ,
$$

with

$$
\boxed{
\begin{align}
a &= \left(1-v_{0}^{2}\right)\left(u^{0}\right)^{2} - v_{0}^{2}g^{00}\ ,\\
b &= 2v_{0}^{2}g^{i0} - 2u^{i}u^{0}\left(1-v^{2}_{0}\right)\ ,\\
c &= \left(1-v_{0}^{2}\right)\left(u^{i}\right)^{2} - v_{0}^{2}g^{ii}\ ,\\
v_{0}^{2} &= v_{\rm A}^{2} + c_{\rm s}^{2}\left(1-v_{\rm A}^{2}\right)\ ,\\
c_{\rm s} &= \left.\left[\frac{dP_{\rm cold}}{d\rho_{b}} + \Gamma_{\rm th}\left(\Gamma_{\rm th}-1\right)\epsilon_{\rm th}\right]\middle/h\right.\ ,\\
c_{+} &= \max\left(\frac{-b \pm \sqrt{b^{2}-4ac}}{2a}\right)\ ,\\
c_{-} &= \min\left(\frac{-b \pm \sqrt{b^{2}-4ac}}{2a}\right)\ .
\end{align}
}
$$

For the implementation of $v_{0}^{2}$, please see [Step 4 below](#compute_v02).

```python
%%writefile $outfile_path__inlined_functions__C

static inline void find_cp_cm(CCTK_REAL &cplus,CCTK_REAL &cminus,CCTK_REAL v02,CCTK_REAL u0,
                       CCTK_REAL vi,CCTK_REAL ONE_OVER_LAPSE_SQUARED,CCTK_REAL shifti,CCTK_REAL psim4,CCTK_REAL gupii) {
  // This computes phase speeds in the direction given by flux_dirn.
  //  Note that we replace the full dispersion relation with a simpler
  //     one, which overestimates the max. speeds by a factor of ~2.
  //     See full discussion around Eqs. 49 and 50 in
  //     http://arxiv.org/pdf/astro-ph/0503420.pdf .
  //  What follows is a complete derivation of the quadratic we solve.
  // wcm = (-k_0 u0 - k_x ux)
  // kcm^2 = K_{\mu} K^{\mu},
  // K_{\mu} K^{\mu} = (g_{\mu a} + u_{\mu} u_a) k^a * g^{\mu b} [ (g_{c b} + u_c u_b) k^c ]
  // --> g^{\mu b} (g_{c b} + u_{c} u_{b}) k^c = (\delta^{\mu}_c + u_c u^{\mu} ) k^c
  //                 = (g_{\mu a} + u_{\mu} u_a) k^a * (\delta^{\mu}_c + u_c u^{\mu} ) k^c
  //                 =[(g_{\mu a} + u_{\mu} u_a) \delta^{\mu}_c + (g_{\mu a} + u_{\mu} u_a) u_c u^{\mu} ] k^c k^a
  //                 =[(g_{c a} + u_c u_a) + (u_c u_a -  u_a u_c] k^c k^a
  //                 =(g_{c a} + u_c u_a) k^c k^a
  //                 = k_a k^a + u^c u^a k_c k_a
  // k^a = g^{\mu a} k_{\mu} = g^{0 a} k_0 + g^{x a} k_x
  // k_a k^a = k_0 g^{0 0} k_0 + k_x k_0 g^{0 x} + g^{x 0} k_0 k_x + g^{x x} k_x k_x
  //         = g^{00} (k_0)^2 + 2 g^{x0} k_0 k_x + g^{xx} (k_x)^2
  // u^c u^a k_c k_a = (u^0 k_0 + u^x k_x) (u^0 k_0 + u^x k_x) = (u^0 k_0)^2 + 2 u^x k_x u^0 k_0 + (u^x k_x)^2
  // (k_0 u0)^2  + 2 k_x ux k_0 u0 + (k_x ux)^2 = v02 [ (u^0 k_0)^2 + 2 u^x k_x u^0 k_0 + (u^x k_x)^2 + g^{00} (k_0)^2 + 2 g^{x0} k_0 k_x + g^{xx} (k_x)^2]
  // (1-v02) (u^0 k_0 + u^x k_x)^2 = v02 (g^{00} (k_0)^2 + 2 g^{x0} k_0 k_x + g^{xx} (k_x)^2)
  // (1-v02) (u^0 k_0/k_x + u^x)^2 = v02 (g^{00} (k_0/k_x)^2 + 2 g^{x0} k_0/k_x + g^{xx})
  // (1-v02) (u^0 X + u^x)^2 = v02 (g^{00} X^2 + 2 g^{x0} X + g^{xx})
  // (1-v02) (u0^2 X^2 + 2 ux u0 X + ux^2) = v02 (g^{00} X^2 + 2 g^{x0} X + g^{xx})
  // X^2 ( (1-v02) u0^2 - v02 g^{00}) + X (2 ux u0 (1-v02) - 2 v02 g^{x0}) + (1-v02) ux^2 - v02 g^{xx}
  // a = (1-v02) u0^2 - v02 g^{00} = (1-v02) u0^2 + v02/lapse^2 <-- VERIFIED
  // b = 2 ux u0 (1-v02) - 2 v02 shiftx/lapse^2 <-- VERIFIED, X->-X, because X = -w/k_1, and we are solving for -X.
  // c = (1-v02) ux^2 - v02 (gupxx*psim4 - (shiftx/lapse)^2) <-- VERIFIED
  // v02 = v_A^2 + c_s^2 (1 - v_A^2)
  CCTK_REAL u0_SQUARED=SQR(u0);
```

    Writing ../src/inlined_functions.C

We start by setting

$$
\boxed{
\begin{align}
a &= \left(1-v_{0}^{2}\right)\left(u^{0}\right)^{2} - v_{0}^{2}g^{00}\\
b &= 2v_{0}^{2}g^{i0} - 2u^{i}u^{0}\left(1-v^{2}_{0}\right)\\
c &= \left(1-v_{0}^{2}\right)\left(u^{i}\right)^{2} - v_{0}^{2}g^{ii}
\end{align}
}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  //Find cplus, cminus:
  CCTK_REAL a = u0_SQUARED * (1.0-v02) + v02*ONE_OVER_LAPSE_SQUARED;
  CCTK_REAL b = 2.0* ( shifti*ONE_OVER_LAPSE_SQUARED * v02 - u0_SQUARED * vi * (1.0-v02) );
  CCTK_REAL c = u0_SQUARED*SQR(vi) * (1.0-v02) - v02 * ( psim4*gupii -
                                                         SQR(shifti)*ONE_OVER_LAPSE_SQUARED);
```

    Appending to ../src/inlined_functions.C

Then we find the minimum ($-$) and maximum ($+$) characteristic speeds

$$
\boxed{
\begin{align}
c_{+} &= \max\left(\frac{-b \pm \sqrt{b^{2}-4ac}}{2a}\right)\ ,\\
c_{-} &= \min\left(\frac{-b \pm \sqrt{b^{2}-4ac}}{2a}\right)\ .
\end{align}
}
$$

```python
%%writefile -a $IGM_src_dir_path/inlined_functions.C

  CCTK_REAL detm = b*b - 4.0*a*c;
  //ORIGINAL LINE OF CODE:
  //if(detm < 0.0) detm = 0.0;
  //New line of code (without the if() statement) has the same effect:
  detm = sqrt(0.5*(detm + fabs(detm))); /* Based on very nice suggestion from Roland Haas */

  cplus = 0.5*(detm-b)/a;
  cminus = -0.5*(detm+b)/a;
  if (cplus < cminus) {
    CCTK_REAL cp = cminus;
    cminus = cplus;
    cplus = cp;
  }
}
```

    Appending to ../src/inlined_functions.C

# Step 4: `compute_v02` \[Back to [top](#toc)\]
$$\label{compute_v02}$$

This function is used to evaluate $v_{0}^{2}$, a quantity necessary for the computation of the minimum and maximum characteristic speeds at each cell interface, $c_{\pm}^{r,l}$. For more information on this procedure, please see the [implementation of the `find_cp_cm` function in Step 3](#find_cp_cm).

We start with the sound speed:

$$
\boxed{
c_{\rm s} = \left.\left[\frac{dP_{\rm cold}}{d\rho_{b}} + \Gamma_{\rm th}\left(\Gamma_{\rm th}-1\right)\epsilon_{\rm th}\right]\middle/h\right.
}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

static inline void compute_v02(CCTK_REAL dPcold_drho,CCTK_REAL Gamma_th,CCTK_REAL eps_th,CCTK_REAL h,CCTK_REAL *smallb,CCTK_REAL *U,  CCTK_REAL &v02L) {

  if(U[RHOB]<=0) { v02L=1.0; return; }

  /* c_s = sound speed = (dP_c/drho + \Gamma(\Gamma-1) \epsilon_th)/h */
  CCTK_REAL c_s_squared  = (dPcold_drho + Gamma_th*(Gamma_th-1.0)*eps_th)/(h);
```

    Appending to ../src/inlined_functions.C

Next we compute the square of the Alfén speed, $v_{\rm A}$, which is given by

$$
\boxed{v_{\rm A}^{2} = \frac{b^{2}}{\rho_{b}h + b^{2}}}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  /* v_A = Alfven speed = sqrt( b^2/(rho0 h + b^2) ) */
  CCTK_REAL v_A_squared = smallb[SMALLB2]/(smallb[SMALLB2] + U[RHOB]*(h));
```

    Appending to ../src/inlined_functions.C

Finally, $v_{0}$ is related to the sound speed and the Alfén speed via

$$
\boxed{v_{0}^{2} = v_{\rm A}^{2} + c_{\rm s}^{2}\left(1-v_{\rm A}^{2}\right)}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  v02L = v_A_squared + c_s_squared*(1.0-v_A_squared);
}
```

    Appending to ../src/inlined_functions.C

## Step 5.e: The `font_fix__rhob_loop()` function \[Back to [top](#toc)\]
$$\label{font_fix__rhob_loop}$$

```python
%%writefile -a $outfile_path__inlined_functions__C

/* Function    : font_fix__rhob_loop()
 * Authors     : Leo Werneck
 * Description : Determines rhob using the font fix prescription
 * Dependencies: find_polytropic_K_and_Gamma_index()
 *             : compute_P_cold__eps_cold()
 * Reference   : Etienne et al. (2011) [https://arxiv.org/pdf/1112.0568.pdf]
 *
 * Inputs      : maxits          - maximum number of iterations allowed
 *             : tol             - font fix tolerance
 *             : W               - See eq. (A26)
 *             : Sf2             - S_{fluid}^{2}, see eq. (A24)
 *             : Psim6           - This is equal to sqrt(\gamma)
 *             : sdots           - \tilde{S}_{\mu}\tilde{S}^{\mu}
 *             : BbardotS2       - (\bar{B}^{\mu}S_{\mu})^{2},
 *             : B2bar           - \bar{B}^{2}, see eq. (A28)
 *             : CONSERVS        - Array of conservative variables
 *             : eos             - Struct of EOS parameters
 *             : rhob_in         - Initial value of rhob
 *             : rhob_out        - Output variable
 *
 * Outputs     : rhob_out        - Updated value of rhob
 *             : return value: 0 - Font fix worked
 *             : return value: 1 - Font fix failed
 */
inline int font_fix__rhob_loop( int maxits, CCTK_REAL tol,
                                CCTK_REAL W, CCTK_REAL Sf2, CCTK_REAL Psim6, CCTK_REAL sdots, CCTK_REAL BbardotS2, CCTK_REAL B2bar,
                                CCTK_REAL *CONSERVS,
                                eos_struct eos, CCTK_REAL rhob_in, CCTK_REAL &rhob_out ) {

  /* Declare basic variables */
  bool fontcheck=true;
  int itcount = 0, j0, j1;
  CCTK_REAL W0, Sf20, rhob0, rhob1, h, P_cold, eps_cold;

  //////////////////////
  // OUTER LOOP START //
  //////////////////////
  while(fontcheck && itcount < maxits) {

    /* Set variables to their input values */
    itcount++;
    W0    = W;
    Sf20  = Sf2;
    rhob1 = rhob_in;

    /* Based on rhob_in (i.e. rhob1), determine the
     * polytropic index j1
     */
    j1 = find_polytropic_K_and_Gamma_index(eos,rhob1);

    //////////////////////
    // INNER LOOP START //
    //////////////////////
    do {

      /* Set rhob0/j0 to be equal to the rhob/j used
       * in the previous iteration, i.e. rhob1/j1.
       */
      rhob0 = rhob1;
      j0    = j1;

      /* Compute h using h_cold and our polytropic EOS
       * .------------------------------------------.
       * | h = h_cold = 1 + eps_cold + P_cold/rhob. |
       * .------------------------------------------.
       */
      compute_P_cold__eps_cold(eos,rhob0, P_cold, eps_cold);
      h = 1.0 + eps_cold + P_cold/rhob0;

      /* Update rhob using eq. (A62) in Etienne et al. (2011)
       *          https://arxiv.org/pdf/1112.0568.pdf
       * .---------------------------------------------------------------------------.
       * | rhob = rho_star * Psi^{-6} / sqrt( 1 + S_fluid^{2}/( (rho_star*h)^{2} ) ) |
       * .---------------------------------------------------------------------------.
       */
      rhob1 = CONSERVS[RHOSTAR]*Psim6/sqrt(1.0+Sf20/SQR(CONSERVS[RHOSTAR]*h));

      /* Update j1 */
      j1 = find_polytropic_K_and_Gamma_index(eos,rhob1);

    }  while( fabs(rhob1-rhob0) > rhob1*tol || j1 != j0);
    //////////////////////
    //  INNER LOOP END  //
    //////////////////////

    /* Output the last value of rhob */
    rhob_out = rhob1;

    /* Perform physical checks on the variables
     * and output the last value of h obtained
     */
    compute_P_cold__eps_cold(eos,rhob_out, P_cold, eps_cold);
    h = 1.0 + eps_cold + P_cold/rhob_out;

    /* Set W based on eq. (A60) in Etienne et al. (2011)
     *       https://arxiv.org/pdf/1112.0568.pdf
     * .-------------------------------------------------------.
     * | W = psi^{-6} * sqrt( S_fluid^{2} + (rho_star*h)^{2} ) |
     * .-------------------------------------------------------.
     */
    W = sqrt( Sf20 + SQR(CONSERVS[RHOSTAR]*h))*Psim6;

    /* Then update S_{fluid}^{2} using eq. (A61) in Etienne et al. (2011)
     *           https://arxiv.org/pdf/1112.0568.pdf
     * .---------------------------------------------------------------------------.
     * | S_fluid^{2} = ( W^{2}*S^{2} + (B.S)^2*(B^{2} + 2W) )/( ( W + B^{2} )^{2} )|
     * .---------------------------------------------------------------------------.
     */
    Sf2 = (SQR(W)*sdots + BbardotS2*(B2bar + 2.0*W))/SQR(W+B2bar);

    if ( fabs(W-W0) < W*tol && fabs(Sf20-Sf2) < Sf2*tol) fontcheck=false;

  }
  //////////////////////
  //  OUTER LOOP END  //
  //////////////////////

  /* If the code converged before the max
   * number of iterations were exceeded,
   * return 0, otherwise return 1.
   */
  if(fontcheck || itcount >= maxits) {
    return 1;
  }
  else {
    return 0;
  }
}
```

    Appending to ../src/inlined_functions.C

# Step 6: `lower_4vector_output_spatial_part` \[Back to [top](#toc)\]
$$\label{lower_4vector_output_spatial_part}$$

This function is used to lower the indices of the spatial components of 4-vectors, $b^{\mu}$. Consider

$$
\begin{align}
b_{i} &= g_{i\mu}b^{\mu}                                       \\
      &= g_{i0}b^{0} + g_{ij}b^{j}                             \\
      &= \left(\gamma_{ij}\beta^{j}\right)b^{0} + \gamma_{ij}b^{j} \\
      &= \gamma_{ij}\left(b^{j} + \beta^{j}b^{0}\right)\ ,
\end{align}
$$

or, using the conformal metric and each component seperately

$$
\boxed{
\begin{align}
b_{x} &= \psi^{4}\left[\bar{\gamma}_{xx}\left(b^{x} + \beta^{x}b^{0}\right)+\bar{\gamma}_{xy}\left(b^{y} + \beta^{y}b^{0}\right)+\bar{\gamma}_{xz}\left(b^{z} + \beta^{z}b^{0}\right)\right]\\
b_{y} &= \psi^{4}\left[\bar{\gamma}_{yx}\left(b^{x} + \beta^{x}b^{0}\right)+\bar{\gamma}_{yy}\left(b^{y} + \beta^{y}b^{0}\right)+\bar{\gamma}_{yz}\left(b^{z} + \beta^{z}b^{0}\right)\right]\\
b_{z} &= \psi^{4}\left[\bar{\gamma}_{zx}\left(b^{x} + \beta^{x}b^{0}\right)+\bar{\gamma}_{zy}\left(b^{y} + \beta^{y}b^{0}\right)+\bar{\gamma}_{zz}\left(b^{z} + \beta^{z}b^{0}\right)\right]
\end{align}
}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

// b_x = g_{\mu x} b^{\mu}
//     = g_{t x} b^t + g_{i x} b^i
//     = b^t gamma_{xj} beta^j + gamma_{ix} b^i
//     = gamma_{xj} (b^j + beta^j b^t)
static inline void lower_4vector_output_spatial_part(CCTK_REAL psi4,CCTK_REAL *METRIC,CCTK_REAL *smallb, CCTK_REAL *smallb_lower) {
  smallb_lower[SMALLBX] = psi4*( METRIC[GXX]*(smallb[SMALLBX]+smallb[SMALLBT]*METRIC[SHIFTX]) + METRIC[GXY]*(smallb[SMALLBY]+smallb[SMALLBT]*METRIC[SHIFTY]) +
                                 METRIC[GXZ]*(smallb[SMALLBZ]+smallb[SMALLBT]*METRIC[SHIFTZ]) );
  smallb_lower[SMALLBY] = psi4*( METRIC[GXY]*(smallb[SMALLBX]+smallb[SMALLBT]*METRIC[SHIFTX]) + METRIC[GYY]*(smallb[SMALLBY]+smallb[SMALLBT]*METRIC[SHIFTY]) +
                                 METRIC[GYZ]*(smallb[SMALLBZ]+smallb[SMALLBT]*METRIC[SHIFTZ]) );
  smallb_lower[SMALLBZ] = psi4*( METRIC[GXZ]*(smallb[SMALLBX]+smallb[SMALLBT]*METRIC[SHIFTX]) + METRIC[GYZ]*(smallb[SMALLBY]+smallb[SMALLBT]*METRIC[SHIFTY]) +
                                 METRIC[GZZ]*(smallb[SMALLBZ]+smallb[SMALLBT]*METRIC[SHIFTZ]) );
}
```

    Appending to ../src/inlined_functions.C

# Step 7: `impose_speed_limit_output_u0` \[Back to [top](#toc)\]
$$\label{impose_speed_limit_output_u0}$$

We now call upon the `impose_speed_limit_output_u0()` function inside the `inlined_functions.C` code file of `IllinoisGRMHD`. The basic algorithm performed by this function is summarized here. We start by evaluating the quantity

$$
\begin{align}
{\rm one\_minus\_one\_over\_alpha\_u0\_squared} \equiv A
&= \gamma_{ij}\left(\frac{v^{i}+\beta^{i}}{\alpha}\right)\left(\frac{v^{j}+\beta^{j}}{\alpha}\right)\\
&= \frac{\gamma_{ij}}{\alpha^{2}}\left[\frac{\gamma^{ik}u_{k}}{u^{0}} - \beta^{i} + \beta^{i}\right]\left[\frac{\gamma^{j\ell}u_{\ell}}{u^{0}} - \beta^{j} + \beta^{j}\right]\\
&=\frac{\gamma_{ij}u^{i}u^{j}}{\left(\alpha u^{0}\right)^{2}}\\
&=\frac{\left(\alpha u^{0}\right)^{2}-1}{\left(\alpha u^{0}\right)^{2}}\\
&=1 - \frac{1}{\left(\alpha u^{0}\right)^{2}}\ \\
\implies \boxed{A = 1 - \frac{1}{\left(\alpha u^{0}\right)^{2}}}\ ,
\end{align}
$$

where when going from line 1 to 2 and from line 3 to 4 we have used eqs. (53) and (56) from [Duez *et al.*](https://arxiv.org/pdf/astro-ph/0503420.pdf), respectively. Keep in mind that the equation we are going to implement below is

$$
\boxed{{\rm one\_minus\_one\_over\_alpha\_u0\_squared} = \gamma_{ij}\left(\frac{v^{i}+\beta^{i}}{\alpha}\right)\left(\frac{v^{j}+\beta^{j}}{\alpha}\right)}\ ,
$$

but it is important to know that this equation also equals $A$ above.

```python
%%writefile -a $outfile_path__inlined_functions__C

static inline void impose_speed_limit_output_u0(CCTK_REAL *METRIC,CCTK_REAL *U,CCTK_REAL psi4,CCTK_REAL ONE_OVER_LAPSE,output_stats &stats, CCTK_REAL &u0_out) {

#ifndef ENABLE_STANDALONE_IGM_C2P_SOLVER
  DECLARE_CCTK_PARAMETERS;
#endif

  // Derivation of first equation:
  // \gamma_{ij} (v^i + \beta^i)(v^j + \beta^j)/(\alpha)^2
  //   = \gamma_{ij} 1/(u^0)^2 ( \gamma^{ik} u_k \gamma^{jl} u_l /(\alpha)^2 <- Using Eq. 53 of arXiv:astro-ph/0503420
  //   = 1/(u^0 \alpha)^2 u_j u_l \gamma^{jl}  <- Since \gamma_{ij} \gamma^{ik} = \delta^k_j
  //   = 1/(u^0 \alpha)^2 ( (u^0 \alpha)^2 - 1 ) <- Using Eq. 56 of arXiv:astro-ph/0503420
  //   = 1 - 1/(u^0 \alpha)^2 <= 1
  CCTK_REAL one_minus_one_over_alpha_u0_squared = psi4*(METRIC[GXX]* SQR(U[VX] + METRIC[SHIFTX]) +
                                                        2.0*METRIC[GXY]*(U[VX] + METRIC[SHIFTX])*(U[VY] + METRIC[SHIFTY]) +
                                                        2.0*METRIC[GXZ]*(U[VX] + METRIC[SHIFTX])*(U[VZ] + METRIC[SHIFTZ]) +
                                                        METRIC[GYY]* SQR(U[VY] + METRIC[SHIFTY]) +
                                                        2.0*METRIC[GYZ]*(U[VY] + METRIC[SHIFTY])*(U[VZ] + METRIC[SHIFTZ]) +
                                                        METRIC[GZZ]* SQR(U[VZ] + METRIC[SHIFTZ]) )*SQR(ONE_OVER_LAPSE);
```

    Appending to ../src/inlined_functions.C

Then we construct the "speed limit quantity"

$$
{\rm ONE\_MINUS\_ONE\_OVER\_GAMMA\_SPEED\_LIMIT\_SQUARED} \equiv B = 1-\frac{1}{\gamma^{2}_{\rm speed\ limit}}\ .
$$

If $A > B$, then we construct the correction factor $C\equiv A / B$, and adjust the velocities using

$$
\boxed{v^{i} \to \left(v^{i}+\beta^{i}\right)C - \beta^{i}}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  /*** Limit velocity to GAMMA_SPEED_LIMIT ***/
  const CCTK_REAL ONE_MINUS_ONE_OVER_GAMMA_SPEED_LIMIT_SQUARED = 1.0-1.0/SQR(GAMMA_SPEED_LIMIT);
  if(one_minus_one_over_alpha_u0_squared > ONE_MINUS_ONE_OVER_GAMMA_SPEED_LIMIT_SQUARED) {
    CCTK_REAL correction_fac = sqrt(ONE_MINUS_ONE_OVER_GAMMA_SPEED_LIMIT_SQUARED/one_minus_one_over_alpha_u0_squared);
    U[VX] = (U[VX] + METRIC[SHIFTX])*correction_fac-METRIC[SHIFTX];
    U[VY] = (U[VY] + METRIC[SHIFTY])*correction_fac-METRIC[SHIFTY];
    U[VZ] = (U[VZ] + METRIC[SHIFTZ])*correction_fac-METRIC[SHIFTZ];
    one_minus_one_over_alpha_u0_squared=ONE_MINUS_ONE_OVER_GAMMA_SPEED_LIMIT_SQUARED;
    stats.failure_checker+=1000;
  }
```

    Appending to ../src/inlined_functions.C

Finally, since $A$ is evaluated using the first line above, namely

$$
\gamma_{ij}\left(\frac{v^{i}+\beta^{i}}{\alpha}\right)\left(\frac{v^{j}+\beta^{j}}{\alpha}\right) = A = 1 - \frac{1}{\left(\alpha u^{0}\right)^{2}}\ ,
$$

we can then compute $u_{0}$ by simply doing

$$
\boxed{u^{0} = \frac{1}{\alpha\sqrt{1-A}}}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  // A = 1.0-one_minus_one_over_alpha_u0_squared = 1-(1-1/(al u0)^2) = 1/(al u0)^2
  // 1/sqrt(A) = al u0
  //CCTK_REAL alpha_u0_minus_one = 1.0/sqrt(1.0-one_minus_one_over_alpha_u0_squared)-1.0;
  //u0_out          = (alpha_u0_minus_one + 1.0)*ONE_OVER_LAPSE;
  CCTK_REAL alpha_u0 = 1.0/sqrt(1.0-one_minus_one_over_alpha_u0_squared);
  if(std::isnan(alpha_u0*ONE_OVER_LAPSE)) printf("BAD FOUND NAN U0 CALC: %.15e %.15e %.15e | %.15e %.15e\n",alpha_u0,ONE_OVER_LAPSE,one_minus_one_over_alpha_u0_squared,psi4, U[VX]);
  u0_out = alpha_u0*ONE_OVER_LAPSE;
}

// The two lines of code below are written to reduce roundoff error and were in the above function. I don't think they reduce error.
// one_over_alpha_u0  = sqrt(1.0-one_minus_one_over_alpha_u0_squared);
/* Proof of following line: */
/*   [        1-1/(alphau0)^2         ] / [ 1/(alphau0) (1 + 1/(alphau0)) ] */
/* = [ (alphau0)^2 - 1)/((alphau0)^2) ] / [ 1/(alphau0) + 1/(alphau0)^2   ] */
/* = [ (alphau0)^2 - 1)/((alphau0)^2) ] / [  (alphau0 + 1)/(alphau0)^2    ] */
/* = [        (alphau0)^2 - 1)        ] / [        (alphau0 + 1)          ] */
/*   [   (alphau0 + 1) (alphau0 - 1)  ] / [        (alphau0 + 1)          ] */
/* =                              alphau0 - 1                               */
//alpha_u0_minus_one = one_minus_one_over_alpha_u0_squared/one_over_alpha_u0/(1.0+one_over_alpha_u0);
//u0_out = (alpha_u0_minus_one+1.0)*ONE_OVER_LAPSE;
```

    Appending to ../src/inlined_functions.C

# Step 8: `enforce_pressure_floor_ceiling` \[Back to [top](#toc)\]
$$\label{enforce_pressure_floor_ceiling}$$

After the Newton-Raphson solver has successfully found a set of primitives, the primitives are checked for physicality, and if they are not in the physical range, they are minimally modified until they return to the physical range. First,if the velocity is found to be superluminal, the speed is reduced to `IllinoisGRMHD`’s default Lorentz factor limit, a procedure which we already explained above when we discussed the `impose_speed_limit_output_u0` function.

Next, `IllinoisGRMHD` does not include any cooling mechanism, which means that for evolutions adopting a $\Gamma$-law equation of state, the pressure should not physically drop below $P_{\rm cold}$. So a pressure floor of $0.9P_{\rm cold}$ is imposed. Increasing this floor to $P_{\rm cold}$ exactly results in large central density drifts in TOV star evolutions.

**NOTE**: Please keep in mind that the floor and ceiling values presented here were found ***empirically***.

```python
%%writefile -a $outfile_path__inlined_functions__C

static inline void enforce_pressure_floor_ceiling(output_stats &stats,CCTK_REAL kpoly,CCTK_REAL P_cold,CCTK_REAL Psi6,const CCTK_REAL Psi6threshold,CCTK_REAL rho_b,const CCTK_REAL rhobatm,  CCTK_REAL &P) {
  CCTK_REAL P_min=0.9*P_cold;
  if(P<P_min) {
    stats.failure_checker+=10;
    P=P_min;
  }
  //MAX(P,P_min);
  //if(P < P_min) P=1.0*P_cold;

  /* OLD: Discarded because lower limit is unphysical.
     if(P <= 0.5*kpoly*P_cold) {
     P=0.5*kpoly*P_cold;
     }
  */
```

    Appending to ../src/inlined_functions.C

Simulations can crash in the other extreme, if $P/P_{\rm cold}$ becomes too large. This typically only happens in very low density regions or inside black holes.  So at densities $\rho_{b}<100\rho_{\rm atm}$ or deep inside black hole horizons, a ceiling on $P$ of $100P_{\rm cold}$ is enforced (see Appendix A of [Etienne *et al.* (2012)](https://arxiv.org/abs/1112.0568) for more details).

We also introduce a parameter, $\psi^{6}_{\rm threshold}$, which determines whether the region under consideration is deep inside the BH horizon or not. For regions deep inside the BH horizon, defined by $\sqrt{\gamma} = \psi^{6} > \psi^{6}_{\rm threshold}$, the primary goal is to keep the evolution stable and prevent inaccurate data from leaking out of the BH horizon. It was determined that in this situation, a better ceiling on $P$ is $10^{5}P_{\rm cold}$.

```python
%%writefile -a $outfile_path__inlined_functions__C

  //CCTK_REAL P_max = 10.0*P_cold;
  CCTK_REAL P_max = 100.0*P_cold;
  if(Psi6 > Psi6threshold) P_max = 1e5*P_cold; // <-- better than 10.

  if((rho_b < 100.0*rhobatm || Psi6 > Psi6threshold) && P>P_max) {
    P=P_max;
    stats.failure_checker+=100;
  }

  /*
    CCTK_REAL rho_horiz_cap = 1000.0*rhobatm;

    //New density damping mechanism inside the horizon
    if(Psi6 > Psi6threshold && rho_b>rho_horiz_cap) {
    CCTK_REAL six_phi=log(Psi6);
    CCTK_REAL six_phithreshold=log(Psi6threshold);
    CCTK_REAL Psi6max_approx=350000;
    rho_b = rho_horiz_cap+(rho_b-rho_horiz_cap)*exp(-200.0*SQR((six_phi-six_phithreshold)/log(Psi6max_approx)));
    }
  */
}
```

    Appending to ../src/inlined_functions.C

# Step 9: `compute_smallba_b2_and_u_i_over_u0_psi4` \[Back to [top](#toc)\]
$$\label{compute_smallba_b2_and_u_i_over_u0_psi4}$$

In this inlined function we will compute quantities related to the magnetic field measured in the comoving fluid frame, $b^{\mu}$.

We will need the following identities

$$
\begin{align}
v^{i}       &= \frac{u^{i}}{u^{0}}\ ,\\
B^{0}_{(u)} &= \frac{u_{i}B^{i}}{\alpha}\ ,\\
B^{i}_{(u)} &= \frac{1}{u^{0}}\left(\frac{B^{i}}{\alpha} + u^{i}B^{0}_{(u)}\right)\ ,\\
b^{\mu}     &= \frac{B^{\mu}_{(u)}}{\sqrt{4\pi}}\ .
\end{align}
$$

We start by setting the relation

$$
b^{0} = \frac{u_{i}B^{i}}{\alpha\sqrt{4\pi}} \implies \boxed{\alpha\sqrt{4\pi}b^{0} = u_{i}B^{i}}\ .
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

static inline void compute_smallba_b2_and_u_i_over_u0_psi4(CCTK_REAL *METRIC,CCTK_REAL *METRIC_LAP_PSI4,CCTK_REAL *U,CCTK_REAL u0L,CCTK_REAL ONE_OVER_LAPSE_SQRT_4PI,
                                                    CCTK_REAL &u_x_over_u0_psi4,CCTK_REAL &u_y_over_u0_psi4,CCTK_REAL &u_z_over_u0_psi4,CCTK_REAL *smallb) {

  // NOW COMPUTE b^{\mu} and b^2 = b^{\mu} b^{\nu} g_{\mu \nu}
  CCTK_REAL ONE_OVER_U0 = 1.0/u0L;
  CCTK_REAL shiftx_plus_vx = (METRIC[SHIFTX]+U[VX]);
  CCTK_REAL shifty_plus_vy = (METRIC[SHIFTY]+U[VY]);
  CCTK_REAL shiftz_plus_vz = (METRIC[SHIFTZ]+U[VZ]);

  // Eq. 56 in http://arxiv.org/pdf/astro-ph/0503420.pdf:
  //  u_i = gamma_{ij} u^0 (v^j + beta^j), gamma_{ij} is the physical metric, and gamma_{ij} = Psi4 * METRIC[Gij], since METRIC[Gij] is the conformal metric.
  u_x_over_u0_psi4 =  METRIC[GXX]*shiftx_plus_vx + METRIC[GXY]*shifty_plus_vy + METRIC[GXZ]*shiftz_plus_vz;
  u_y_over_u0_psi4 =  METRIC[GXY]*shiftx_plus_vx + METRIC[GYY]*shifty_plus_vy + METRIC[GYZ]*shiftz_plus_vz;
  u_z_over_u0_psi4 =  METRIC[GXZ]*shiftx_plus_vx + METRIC[GYZ]*shifty_plus_vy + METRIC[GZZ]*shiftz_plus_vz;

  // Eqs. 23 and 31 in http://arxiv.org/pdf/astro-ph/0503420.pdf:
  //   Compute alpha sqrt(4 pi) b^t = u_i B^i
  CCTK_REAL alpha_sqrt_4pi_bt = ( u_x_over_u0_psi4*U[BX_CENTER] + u_y_over_u0_psi4*U[BY_CENTER] + u_z_over_u0_psi4*U[BZ_CENTER] ) * METRIC_LAP_PSI4[PSI4]*u0L;
```

    Appending to ../src/inlined_functions.C

Then we compute

$$
\begin{align}
b^{i} &= \frac{B^{i}_{(u)}}{\sqrt{4\pi}}\\
      &= \frac{1}{u^{0}\sqrt{4\pi}}\left(\frac{B^{i}}{\alpha} + B^{0}_{(u)}u^{i}\right)\\
      &= \frac{1}{u^{0}\sqrt{4\pi}}\left(\frac{B^{i}}{\alpha} + \sqrt{4\pi}b^{0}u^{i}\right)\\
      &= \frac{1}{\alpha\sqrt{4\pi}}\left(\frac{B^{i}}{u^{0}} + \alpha\sqrt{4\pi}b^{0}\frac{u^{i}}{u^{0}}\right)\\
\implies &\boxed{b^{i} = \frac{1}{\alpha\sqrt{4\pi}}\left(\frac{B^{i}}{u^{0}} + \alpha\sqrt{4\pi}b^{0}v^{i}\right)}\ .
\end{align}
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  // Eq. 24 in http://arxiv.org/pdf/astro-ph/0503420.pdf:
  // b^i = B^i_u / sqrt(4 pi)
  // b^i = ( B^i/alpha + B^0_u u^i ) / ( u^0 sqrt(4 pi) )
  // b^i = ( B^i/alpha +  sqrt(4 pi) b^t u^i ) / ( u^0 sqrt(4 pi) )
  // b^i = ( B^i +  alpha sqrt(4 pi) b^t u^i ) / ( alpha u^0 sqrt(4 pi) )
  // b^i = ( B^i/u^0 +  alpha sqrt(4 pi) b^t u^i/u^0 ) / ( alpha sqrt(4 pi) )
  // b^i = ( B^i/u^0 +  alpha sqrt(4 pi) b^t v^i ) / ( alpha sqrt(4 pi) )
  smallb[SMALLBX] = (U[BX_CENTER]*ONE_OVER_U0 + U[VX]*alpha_sqrt_4pi_bt)*ONE_OVER_LAPSE_SQRT_4PI;
  smallb[SMALLBY] = (U[BY_CENTER]*ONE_OVER_U0 + U[VY]*alpha_sqrt_4pi_bt)*ONE_OVER_LAPSE_SQRT_4PI;
  smallb[SMALLBZ] = (U[BZ_CENTER]*ONE_OVER_U0 + U[VZ]*alpha_sqrt_4pi_bt)*ONE_OVER_LAPSE_SQRT_4PI;
  // Eq. 23 in http://arxiv.org/pdf/astro-ph/0503420.pdf, with alpha sqrt (4 pi) b^2 = u_i B^i already computed above
  smallb[SMALLBT] = alpha_sqrt_4pi_bt * ONE_OVER_LAPSE_SQRT_4PI;
```

    Appending to ../src/inlined_functions.C

Finally, we compute

$$
\begin{align}
b^{2} &= g_{\mu\nu}b^{\mu}b^{\nu}\\
      &= g_{00}\left(b^{0}\right)^{2} + g_{ij}b^{i}b^{j} + 2g_{0i}b^{0}b^{i}\\
      &= \left(-\alpha^{2} + \gamma_{ij}\beta^{i}\beta^{j}\right)\left(b^{0}\right)^{2} + \gamma_{ij}b^{i}b^{j} + 2b^{0}\gamma_{ij}\beta^{j}b^{i}\\
      &= -\left(\alpha b^{0}\right)^{2} + \gamma_{ij}\left[b^{i}b^{j} + 2b^{0}b^{i}\beta^{j} + \left(b^{0}\right)^{2}\beta^{i}\beta^{j}\right]\\
\implies &\boxed{b^{2} = -\left(\alpha b^{0}\right)^{2} + \gamma_{ij}\left(b^{i} + b^{0}\beta^{i}\right)\left(b^{j} + b^{0}\beta^{j}\right)}
\end{align}
$$

```python
%%writefile -a $outfile_path__inlined_functions__C

  // b^2 = g_{\mu \nu} b^{\mu} b^{\nu}
  //     = gtt bt^2 + gxx bx^2 + gyy by^2 + gzz bz^2 + 2 (gtx bt bx + gty bt by + gtz bt bz + gxy bx by + gxz bx bz + gyz by bz)
  //     = (-al^2 + gamma_{ij} betai betaj) bt^2 + b^i b^j gamma_{ij} + 2 g_{t i} b^t b^i
  //     = - (alpha b^t)^2 + (b^t)^2 gamma_{ij} beta^i beta^j + b^i b^j gamma_{ij} + 2 b^t g_{t i} b^i
  //     = - (alpha b^t)^2 + (b^t)^2 gamma_{ij} beta^i beta^j + b^i b^j gamma_{ij} + 2 b^t (gamma_{ij} beta^j) b^i
  //     = - (alpha b^t)^2 + gamma_{ij} ((b^t)^2 beta^i beta^j + b^i b^j + 2 b^t beta^j b^i)
  //     = - (alpha b^t)^2 + gamma_{ij} ((b^t)^2 beta^i beta^j + 2 b^t beta^j b^i + b^i b^j)
  //     = - (alpha b^t)^2 + gamma_{ij} (b^i + b^t beta^i) (b^j + b^t beta^j)
  CCTK_REAL bx_plus_shiftx_bt = smallb[SMALLBX]+METRIC[SHIFTX]*smallb[SMALLBT];
  CCTK_REAL by_plus_shifty_bt = smallb[SMALLBY]+METRIC[SHIFTY]*smallb[SMALLBT];
  CCTK_REAL bz_plus_shiftz_bt = smallb[SMALLBZ]+METRIC[SHIFTZ]*smallb[SMALLBT];
  smallb[SMALLB2] = -SQR(METRIC_LAP_PSI4[LAPSE]*smallb[SMALLBT]) +
    (  METRIC[GXX]*SQR(bx_plus_shiftx_bt) + METRIC[GYY]*SQR(by_plus_shifty_bt) + METRIC[GZZ]*SQR(bz_plus_shiftz_bt) +
       2.0*( METRIC[GXY]*(bx_plus_shiftx_bt)*(by_plus_shifty_bt) +
             METRIC[GXZ]*(bx_plus_shiftx_bt)*(bz_plus_shiftz_bt) +
             METRIC[GYZ]*(by_plus_shifty_bt)*(bz_plus_shiftz_bt) )  ) * METRIC_LAP_PSI4[PSI4]; // mult by psi4 because METRIC[GIJ] is the conformal metric.
  /***********************************************************/
}

```

    Appending to ../src/inlined_functions.C

# Step 10: Code validation \[Back to [top](#toc)\]
$$\label{code_validation}$$

First we download the original `IllinoisGRMHD` source code and then compare it to the source code generated by this tutorial notebook.

```python
# Verify if the code generated by this tutorial module
# matches the original IllinoisGRMHD source code

# First download the original IllinoisGRMHD source code
import urllib
from os import path

original_IGM_file_url  = "https://bitbucket.org/zach_etienne/wvuthorns/raw/5611b2f0b17135538c9d9d17c7da062abe0401b6/IllinoisGRMHD/src/inlined_functions.C"
original_IGM_file_name = "inlined_functions-original.C"
original_IGM_file_path = os.path.join(IGM_src_dir_path,original_IGM_file_name)

# Then download the original IllinoisGRMHD source code
# We try it here in a couple of ways in an attempt to keep
# the code more portable
try:
    original_IGM_file_code = urllib.request.urlopen(original_IGM_file_url).read().decode("utf-8")
    # Write down the file the original IllinoisGRMHD source code
    with open(original_IGM_file_path,"w") as file:
        file.write(original_IGM_file_code)
except:
    try:
        original_IGM_file_code = urllib.urlopen(original_IGM_file_url).read().decode("utf-8")
        # Write down the file the original IllinoisGRMHD source code
        with open(original_IGM_file_path,"w") as file:
            file.write(original_IGM_file_code)
    except:
        # If all else fails, hope wget does the job
        !wget -O $original_IGM_file_path $original_IGM_file_url

# Perform validation
Validation__inlined_functions__C  = !diff $original_IGM_file_path $outfile_path__inlined_functions__C

if Validation__inlined_functions__C == []:
    # If the validation passes, we do not need to store the original IGM source code file
    !rm $original_IGM_file_path
    print("Validation test for inlined_functions.C: PASSED!")
else:
    # If the validation fails, we keep the original IGM source code file
    print("Validation test for inlined_functions.C: FAILED!")
    # We also print out the difference between the code generated
    # in this tutorial module and the original IGM source code
    print("Diff:")
    for diff_line in Validation__inlined_functions__C:
        print(diff_line)
```

    Validation test for inlined_functions.C: FAILED!
    Diff:
    1,4c1
    < static inline CCTK_REAL fasterpow_ppm_reconstruct(CCTK_REAL inputvar,CCTK_REAL inputpow) {
    <   if(inputpow==2.0) return SQR(inputvar);
    <   return pow(inputvar,inputpow);
    < }
    ---
    >
    10c7
    <   //     one, which overestimates the max. speeds by a factor of ~2.
    ---
    >   //     one, which overestimates the max. speeds by a factor of ~2.
    15c12
    <   // kcm^2 = K_{\mu} K^{\mu},
    ---
    >   // kcm^2 = K_{\mu} K^{\mu},
    38a36
    >
    43a42
    >
    49c48
    <
    ---
    >
    59c58,59
    < static inline void compute_v02(CCTK_REAL dPcold_drho,CCTK_REAL gamma_th,CCTK_REAL eps_th,CCTK_REAL h,CCTK_REAL *smallb,CCTK_REAL *U,  CCTK_REAL &v02L) {
    ---
    >
    > static inline void compute_v02(CCTK_REAL dPcold_drho,CCTK_REAL Gamma_th,CCTK_REAL eps_th,CCTK_REAL h,CCTK_REAL *smallb,CCTK_REAL *U,  CCTK_REAL &v02L) {
    64c64,65
    <   CCTK_REAL c_s_squared  = (dPcold_drho + gamma_th*(gamma_th-1.0)*eps_th)/(h);
    ---
    >   CCTK_REAL c_s_squared  = (dPcold_drho + Gamma_th*(Gamma_th-1.0)*eps_th)/(h);
    >
    66a68
    >
    70,84c72,180
    < static inline void compute_P_cold__eps_cold__dPcold_drho__eps_th__h__gamma_cold(CCTK_REAL *U, eos_struct &eos,
    <                                                                          CCTK_REAL &P_cold,CCTK_REAL &eps_cold,CCTK_REAL &dPcold_drho,CCTK_REAL &eps_th,CCTK_REAL &h,
    <                                                                          CCTK_REAL &gamma_cold) {
    <   // This code handles equations of state of the form defined
    <   // in Eqs 13-16 in http://arxiv.org/pdf/0802.0200.pdf
    <
    <   if(U[RHOB]==0) {
    <     P_cold = 0.0;
    <     eps_cold = 0.0;
    <     dPcold_drho = 0.0;
    <     eps_th = 0.0;
    <     h = 0.0;
    <     gamma_cold = eos.gamma_tab[0];
    <     return;
    <   }
    ---
    > /* Function    : font_fix__rhob_loop()
    >  * Authors     : Leo Werneck
    >  * Description : Determines rhob using the font fix prescription
    >  * Dependencies: find_polytropic_K_and_Gamma_index()
    >  *             : compute_P_cold__eps_cold()
    >  * Reference   : Etienne et al. (2011) [https://arxiv.org/pdf/1112.0568.pdf]
    >  *
    >  * Inputs      : maxits          - maximum number of iterations allowed
    >  *             : tol             - font fix tolerance
    >  *             : W               - See eq. (A26)
    >  *             : Sf2             - S_{fluid}^{2}, see eq. (A24)
    >  *             : Psim6           - This is equal to sqrt(\gamma)
    >  *             : sdots           - \tilde{S}_{\mu}\tilde{S}^{\mu}
    >  *             : BbardotS2       - (\bar{B}^{\mu}S_{\mu})^{2},
    >  *             : B2bar           - \bar{B}^{2}, see eq. (A28)
    >  *             : CONSERVS        - Array of conservative variables
    >  *             : eos             - Struct of EOS parameters
    >  *             : rhob_in         - Initial value of rhob
    >  *             : rhob_out        - Output variable
    >  *
    >  * Outputs     : rhob_out        - Updated value of rhob
    >  *             : return value: 0 - Font fix worked
    >  *             : return value: 1 - Font fix failed
    >  */
    > inline int font_fix__rhob_loop( int maxits, CCTK_REAL tol,
    >                                 CCTK_REAL W, CCTK_REAL Sf2, CCTK_REAL Psim6, CCTK_REAL sdots, CCTK_REAL BbardotS2, CCTK_REAL B2bar,
    >                                 CCTK_REAL *CONSERVS,
    >                                 eos_struct eos, CCTK_REAL rhob_in, CCTK_REAL &rhob_out ) {
    >
    >   /* Declare basic variables */
    >   bool fontcheck=true;
    >   int itcount = 0, j0, j1;
    >   CCTK_REAL W0, Sf20, rhob0, rhob1, h, P_cold, eps_cold;
    >
    >   //////////////////////
    >   // OUTER LOOP START //
    >   //////////////////////
    >   while(fontcheck && itcount < maxits) {
    >
    >     /* Set variables to their input values */
    >     itcount++;
    >     W0    = W;
    >     Sf20  = Sf2;
    >     rhob1 = rhob_in;
    >
    >     /* Based on rhob_in (i.e. rhob1), determine the
    >      * polytropic index j1
    >      */
    >     j1 = find_polytropic_K_and_Gamma_index(eos,rhob1);
    >
    >     //////////////////////
    >     // INNER LOOP START //
    >     //////////////////////
    >     do {
    >
    >       /* Set rhob0/j0 to be equal to the rhob/j used
    >        * in the previous iteration, i.e. rhob1/j1.
    >        */
    >       rhob0 = rhob1;
    >       j0    = j1;
    >
    >       /* Compute h using h_cold and our polytropic EOS
    >        * .------------------------------------------.
    >        * | h = h_cold = 1 + eps_cold + P_cold/rhob. |
    >        * .------------------------------------------.
    >        */
    >       compute_P_cold__eps_cold(eos,rhob0, P_cold, eps_cold);
    >       h = 1.0 + eps_cold + P_cold/rhob0;
    >
    >       /* Update rhob using eq. (A62) in Etienne et al. (2011)
    >        *          https://arxiv.org/pdf/1112.0568.pdf
    >        * .---------------------------------------------------------------------------.
    >        * | rhob = rho_star * Psi^{-6} / sqrt( 1 + S_fluid^{2}/( (rho_star*h)^{2} ) ) |
    >        * .---------------------------------------------------------------------------.
    >        */
    >       rhob1 = CONSERVS[RHOSTAR]*Psim6/sqrt(1.0+Sf20/SQR(CONSERVS[RHOSTAR]*h));
    >
    >       /* Update j1 */
    >       j1 = find_polytropic_K_and_Gamma_index(eos,rhob1);
    >
    >     }  while( fabs(rhob1-rhob0) > rhob1*tol || j1 != j0);
    >     //////////////////////
    >     //  INNER LOOP END  //
    >     //////////////////////
    >
    >     /* Output the last value of rhob */
    >     rhob_out = rhob1;
    >
    >     /* Perform physical checks on the variables
    >      * and output the last value of h obtained
    >      */
    >     compute_P_cold__eps_cold(eos,rhob_out, P_cold, eps_cold);
    >     h = 1.0 + eps_cold + P_cold/rhob_out;
    >
    >     /* Set W based on eq. (A60) in Etienne et al. (2011)
    >      *       https://arxiv.org/pdf/1112.0568.pdf
    >      * .-------------------------------------------------------.
    >      * | W = psi^{-6} * sqrt( S_fluid^{2} + (rho_star*h)^{2} ) |
    >      * .-------------------------------------------------------.
    >      */
    >     W = sqrt( Sf20 + SQR(CONSERVS[RHOSTAR]*h))*Psim6;
    >
    >     /* Then update S_{fluid}^{2} using eq. (A61) in Etienne et al. (2011)
    >      *           https://arxiv.org/pdf/1112.0568.pdf
    >      * .---------------------------------------------------------------------------.
    >      * | S_fluid^{2} = ( W^{2}*S^{2} + (B.S)^2*(B^{2} + 2W) )/( ( W + B^{2} )^{2} )|
    >      * .---------------------------------------------------------------------------.
    >      */
    >     Sf2 = (SQR(W)*sdots + BbardotS2*(B2bar + 2.0*W))/SQR(W+B2bar);
    86c182
    <   CCTK_REAL U_RHOB_inv = 1.0/U[RHOB];
    ---
    >     if ( fabs(W-W0) < W*tol && fabs(Sf20-Sf2) < Sf2*tol) fontcheck=false;
    88,111d183
    <   if(eos.neos==1) {
    <     // Eq. 14 of http://arxiv.org/pdf/0802.0200.pdf :
    <     // P_{cold} = K_i rho_i^{\Gamma_i}
    <     P_cold = eos.k_tab[0]*fasterpow_ppm_reconstruct(U[RHOB],eos.gamma_tab[0]);
    <     // Eq. 16 of http://arxiv.org/pdf/0802.0200.pdf :
    <     // \epsilon_{cold} = \int ( P_{cold}(rho) / rho^2 ) drho
    <     //                 = \int ( K_0 \rho^{\Gamma_0 - 2} ) drho
    <     //                 = ( K_0 \rho^{\Gamma_0 - 1} ) / (\Gamma_0 - 1)
    <     //                 = ( P_{cold} / rho          ) / (\Gamma_0 - 1)
    <     eps_cold = P_cold*U_RHOB_inv/(eos.gamma_tab[0]-1.0);
    <     // dPcold/drho = K_i \Gamma_i rho_i^{\Gamma_i-1} = \Gamma_i P_{cold} / rho
    <     dPcold_drho = eos.gamma_tab[0]*P_cold*U_RHOB_inv;
    <     // Eq. 15 of http://arxiv.org/pdf/0802.0200.pdf :
    <     // P_{th} = (\Gamma_{th} - 1) \rho_0 \epsilon_{th},
    <     // Eq. 13 of http://arxiv.org/pdf/0802.0200.pdf :
    <     // P_{th} = P - P_{cold}
    <     // -> P - P_{cold} = (\Gamma_{th} - 1) \rho_0 \epsilon_{th}
    <     // -> \epsilon_{th} = ( P - P_{cold} ) / [ (\Gamma_{th} - 1) \rho_0 ]
    <     eps_th = (U[PRESSURE] - P_cold)/(eos.gamma_th-1.0)*U_RHOB_inv;
    <     // Just below Eq. 16 in http://arxiv.org/pdf/astro-ph/0503420.pdf :
    <     // h = 1 + \epsilon + P/rho
    <     h = 1.0 + eps_cold + eps_th + U[PRESSURE]*U_RHOB_inv;
    <     gamma_cold = eos.gamma_tab[0];
    <     return;
    113,125c185,194
    <
    <   // See comments above for the eos.neos==1 case for relevant
    <   // equations & references; the extension to arbitrary "nn"
    <   // is straightforward.
    <   for(int nn=1;nn<eos.neos;nn++) {
    <     if (U[RHOB] <= eos.rho_tab[nn]  &&  U[RHOB] > eos.rho_tab[nn-1]) {
    <       P_cold = eos.k_tab[nn]*fasterpow_ppm_reconstruct(U[RHOB],eos.gamma_tab[nn]);
    <       eps_cold = eos.eps_tab[nn-1] + (P_cold*U_RHOB_inv - eos.P_tab[nn-1]/eos.rho_tab[nn-1])/(eos.gamma_tab[nn]-1.0);
    <       dPcold_drho = eos.gamma_tab[nn]*P_cold*U_RHOB_inv;
    <       eps_th = (U[PRESSURE] - P_cold)/(eos.gamma_th-1.0)*U_RHOB_inv;
    <       h = 1.0 + eps_cold + eps_th + U[PRESSURE]*U_RHOB_inv;
    <       gamma_cold = eos.gamma_tab[nn];
    <     }
    ---
    >   //////////////////////
    >   //  OUTER LOOP END  //
    >   //////////////////////
    >
    >   /* If the code converged before the max
    >    * number of iterations were exceeded,
    >    * return 0, otherwise return 1.
    >    */
    >   if(fontcheck || itcount >= maxits) {
    >     return 1;
    127,133c196,197
    <   if (U[RHOB] > eos.rho_tab[eos.neos-1]) {
    <     P_cold = eos.k_tab[eos.neos]*fasterpow_ppm_reconstruct(U[RHOB],eos.gamma_tab[eos.neos]);
    <     eps_cold = eos.eps_tab[eos.neos-1] + (P_cold*U_RHOB_inv - eos.P_tab[eos.neos-1]/eos.rho_tab[eos.neos-1])/(eos.gamma_tab[eos.neos]-1.0);
    <     dPcold_drho = eos.gamma_tab[eos.neos]*P_cold*U_RHOB_inv;
    <     eps_th = (U[PRESSURE] - P_cold)/(eos.gamma_th-1.0)*U_RHOB_inv;
    <     h = 1.0 + eps_cold + eps_th + U[PRESSURE]*U_RHOB_inv;
    <     gamma_cold = eos.gamma_tab[eos.neos];
    ---
    >   else {
    >     return 0;
    136a201
    >
    149a215
    >
    150a217,218
    >
    > #ifndef ENABLE_STANDALONE_IGM_C2P_SOLVER
    151a220,221
    > #endif
    >
    154c224
    <   // \gamma_{ij} (v^i + \beta^i)(v^j + \beta^j)/(\alpha)^2
    ---
    >   // \gamma_{ij} (v^i + \beta^i)(v^j + \beta^j)/(\alpha)^2
    165a236
    >
    176a248
    >
    187,195c259,267
    < // one_over_alpha_u0  = sqrt(1.0-one_minus_one_over_alpha_u0_squared);
    < /* Proof of following line: */
    < /*   [        1-1/(alphau0)^2         ] / [ 1/(alphau0) (1 + 1/(alphau0)) ] */
    < /* = [ (alphau0)^2 - 1)/((alphau0)^2) ] / [ 1/(alphau0) + 1/(alphau0)^2   ] */
    < /* = [ (alphau0)^2 - 1)/((alphau0)^2) ] / [  (alphau0 + 1)/(alphau0)^2    ] */
    < /* = [        (alphau0)^2 - 1)        ] / [        (alphau0 + 1)          ] */
    < /*   [   (alphau0 + 1) (alphau0 - 1)  ] / [        (alphau0 + 1)          ] */
    < /* =                              alphau0 - 1                               */
    < //alpha_u0_minus_one = one_minus_one_over_alpha_u0_squared/one_over_alpha_u0/(1.0+one_over_alpha_u0);
    ---
    > // one_over_alpha_u0  = sqrt(1.0-one_minus_one_over_alpha_u0_squared);
    > /* Proof of following line: */
    > /*   [        1-1/(alphau0)^2         ] / [ 1/(alphau0) (1 + 1/(alphau0)) ] */
    > /* = [ (alphau0)^2 - 1)/((alphau0)^2) ] / [ 1/(alphau0) + 1/(alphau0)^2   ] */
    > /* = [ (alphau0)^2 - 1)/((alphau0)^2) ] / [  (alphau0 + 1)/(alphau0)^2    ] */
    > /* = [        (alphau0)^2 - 1)        ] / [        (alphau0 + 1)          ] */
    > /*   [   (alphau0 + 1) (alphau0 - 1)  ] / [        (alphau0 + 1)          ] */
    > /* =                              alphau0 - 1                               */
    > //alpha_u0_minus_one = one_minus_one_over_alpha_u0_squared/one_over_alpha_u0/(1.0+one_over_alpha_u0);
    197a270
    >
    212a286
    >
    224c298
    <
    ---
    >
    235c309,310
    < static inline void compute_smallba_b2_and_u_i_over_u0_psi4(CCTK_REAL *METRIC,CCTK_REAL *METRIC_LAP_PSI4,CCTK_REAL *U,CCTK_REAL u0L,CCTK_REAL ONE_OVER_LAPSE_SQRT_4PI,
    ---
    >
    > static inline void compute_smallba_b2_and_u_i_over_u0_psi4(CCTK_REAL *METRIC,CCTK_REAL *METRIC_LAP_PSI4,CCTK_REAL *U,CCTK_REAL u0L,CCTK_REAL ONE_OVER_LAPSE_SQRT_4PI,
    252a328
    >
    265a342
    >
    276c353
    <   CCTK_REAL bz_plus_shiftz_bt = smallb[SMALLBZ]+METRIC[SHIFTZ]*smallb[SMALLBT];
    ---
    >   CCTK_REAL bz_plus_shiftz_bt = smallb[SMALLBZ]+METRIC[SHIFTZ]*smallb[SMALLBT];
    283a361
    >

# Step 11: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-IllinoisGRMHD__inlined_functions.pdf](Tutorial-IllinoisGRMHD__inlined_functions.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means).

```python
latex_nrpy_style_path = os.path.join(nrpy_dir_path,"latex_nrpy_style.tplx")
#!jupyter nbconvert --to latex --template $latex_nrpy_style_path --log-level='WARN' Tutorial-IllinoisGRMHD__inlined_functions.ipynb
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__inlined_functions.tex
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__inlined_functions.tex
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__inlined_functions.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```