Instruction:
Explain what the following Python 3 code does
def single_run_arrays(spanning_cluster=True, **kwargs):
    r'''
    Generate statistics for a single run

    This is a stand-alone helper function to evolve a single sample state
    (realization) and return the cluster statistics.

    Parameters
    ----------
    spanning_cluster : bool, optional
        Whether to detect a spanning cluster or not.
        Defaults to ``True``.

    kwargs : keyword arguments
        Piped through to :func:`sample_states`

    Returns
    -------

    ret : dict
        Cluster statistics

    ret['N'] : int
        Total number of sites

    ret['M'] : int
        Total number of bonds

    ret['max_cluster_size'] : 1-D :py:class:`numpy.ndarray` of int, size ``ret['M'] + 1``
        Array of the sizes of the largest cluster (absolute number of sites) at
        the respective occupation number.

    ret['has_spanning_cluster'] : 1-D :py:class:`numpy.ndarray` of bool, size ``ret['M'] + 1``
        Array of booleans for each occupation number.
        The respective entry is ``True`` if there is a spanning cluster,
        ``False`` otherwise.
        Only exists if `spanning_cluster` argument is set to ``True``.

    ret['moments'] : 2-D :py:class:`numpy.ndarray` of int
        Array of shape ``(5, ret['M'] + 1)``.
        The ``(k, m)``-th entry is the ``k``-th raw moment of the (absolute)
        cluster size distribution, with ``k`` ranging from ``0`` to ``4``, at
        occupation number ``m``.

    See Also
    --------

    sample_states

    '''

    # initial iteration
    # we do not need a copy of the result dictionary since we copy the values
    # anyway
    kwargs['copy_result'] = False
    ret = dict()

    for n, state in enumerate(sample_states(
        spanning_cluster=spanning_cluster, **kwargs
    )):

        # merge cluster statistics
        if 'N' in ret:
            assert ret['N'] == state['N']
        else:
            ret['N'] = state['N']

        if 'M' in ret:
            assert ret['M'] == state['M']
        else:
            ret['M'] = state['M']
            number_of_states = state['M'] + 1
            max_cluster_size = np.empty(number_of_states)
            if spanning_cluster:
                has_spanning_cluster = np.empty(number_of_states, dtype=np.bool)
            moments = np.empty((5, number_of_states))

        max_cluster_size[n] = state['max_cluster_size']
        for k in range(5):
            moments[k, n] = state['moments'][k]
        if spanning_cluster:
            has_spanning_cluster[n] = state['has_spanning_cluster']

    ret['max_cluster_size'] = max_cluster_size
    ret['moments'] = moments
    if spanning_cluster:
        ret['has_spanning_cluster'] = has_spanning_cluster

    return ret

Response: r Generate statistics for a single run of a single state tree.


Instruction:
Can you tell what is the following Python 3 function doing
def _microcanonical_average_spanning_cluster(has_spanning_cluster, alpha):
    r'''
    Compute the average number of runs that have a spanning cluster

    Helper function for :func:`microcanonical_averages`

    Parameters
    ----------

    has_spanning_cluster : 1-D :py:class:`numpy.ndarray` of bool
        Each entry is the ``has_spanning_cluster`` field of the output of
        :func:`sample_states`:
        An entry is ``True`` if there is a spanning cluster in that respective
        run, and ``False`` otherwise.

    alpha : float
        Significance level.

    Returns
    -------

    ret : dict
        Spanning cluster statistics

    ret['spanning_cluster'] : float
        The average relative number (Binomial proportion) of runs that have a
        spanning cluster.
        This is the Bayesian point estimate of the posterior mean, with a
        uniform prior.

    ret['spanning_cluster_ci'] : 1-D :py:class:`numpy.ndarray` of float, size 2
        The lower and upper bounds of the Binomial proportion confidence
        interval with uniform prior.

    See Also
    --------

    sample_states : spanning cluster detection

    microcanonical_averages : spanning cluster statistics

    Notes
    -----

    Averages and confidence intervals for Binomial proportions

    As Cameron [8]_ puts it, the normal approximation to the confidence
    interval for a Binomial proportion :math:`p` "suffers a *systematic*
    decline in performance (...) towards extreme values of :math:`p` near
    :math:`0` and :math:`1`, generating binomial [confidence intervals]
    with effective coverage far below the desired level." (see also
    References [6]_ and [7]_).

    A different approach to quantifying uncertainty is Bayesian inference.
    [5]_
    For :math:`n` independent Bernoulli trails with common success
    probability :math:`p`, the *likelihood* to have :math:`k` successes
    given :math:`p` is the binomial distribution

    .. math::

        P(k|p) = \binom{n}{k} p^k (1-p)^{n-k} \equiv B(a,b),

    where :math:`B(a, b)` is the *Beta distribution* with parameters
    :math:`a = k + 1` and :math:`b = n - k + 1`.
    Assuming a uniform prior :math:`P(p) = 1`, the *posterior* is [5]_

    .. math::

        P(p|k) = P(k|p)=B(a,b).

    A point estimate is the posterior mean

    .. math::

        \bar{p} = \frac{k+1}{n+2}

    with the :math:`1 - \alpha` credible interval :math:`(p_l, p_u)` given
    by

    .. math::

        \int_0^{p_l} dp B(a,b) = \int_{p_u}^1 dp B(a,b) = \frac{\alpha}{2}.

    References
    ----------

    .. [5] Wasserman, L. All of Statistics (Springer New York, 2004),
       `doi:10.1007/978-0-387-21736-9 <http://dx.doi.org/10.1007/978-0-387-21736-9>`_.

    .. [6] DasGupta, A., Cai, T. T. & Brown, L. D. Interval Estimation for a
       Binomial Proportion. Statistical Science 16, 101-133 (2001).
       `doi:10.1214/ss/1009213286 <http://dx.doi.org/10.1214/ss/1009213286>`_.

    .. [7] Agresti, A. & Coull, B. A. Approximate is Better than "Exact" for
       Interval Estimation of Binomial Proportions. The American Statistician
       52, 119-126 (1998),
       `doi:10.2307/2685469 <http://dx.doi.org/10.2307/2685469>`_.

    .. [8] Cameron, E. On the Estimation of Confidence Intervals for Binomial
       Population Proportions in Astronomy: The Simplicity and Superiority of
       the Bayesian Approach. Publications of the Astronomical Society of
       Australia 28, 128-139 (2011),
       `doi:10.1071/as10046 <http://dx.doi.org/10.1071/as10046>`_.

    '''

    ret = dict()
    runs = has_spanning_cluster.size

    # Bayesian posterior mean for Binomial proportion (uniform prior)
    k = has_spanning_cluster.sum(dtype=np.float)
    ret['spanning_cluster'] = (
        (k + 1) / (runs + 2)
    )

    # Bayesian credible interval for Binomial proportion (uniform
    # prior)
    ret['spanning_cluster_ci'] = scipy.stats.beta.ppf(
        [alpha / 2, 1 - alpha / 2], k + 1, runs - k + 1
    )

    return ret

Response: r Returns the microcanonical average of the spanning cluster of the given base.


Instruction:
Here you have a function in Python 3, explain what it does
def _microcanonical_average_max_cluster_size(max_cluster_size, alpha):

    ret = dict()
    runs = max_cluster_size.size
    sqrt_n = np.sqrt(runs)

    max_cluster_size_sample_mean = max_cluster_size.mean()
    ret['max_cluster_size'] = max_cluster_size_sample_mean

    max_cluster_size_sample_std = max_cluster_size.std(ddof=1)
    if max_cluster_size_sample_std:
        old_settings = np.seterr(all='raise')
        ret['max_cluster_size_ci'] = scipy.stats.t.interval(
            1 - alpha,
            df=runs - 1,
            loc=max_cluster_size_sample_mean,
            scale=max_cluster_size_sample_std / sqrt_n
        )
        np.seterr(**old_settings)
    else:
        ret['max_cluster_size_ci'] = (
            max_cluster_size_sample_mean * np.ones(2)
        )

    return ret

Response: Compute the average size of the largest cluster.


Instruction:
Can you implement a function in Python 3 that
computes the microcanonical average moments of the cluster size distributions.

Response:
def _microcanonical_average_moments(moments, alpha):
    """
    Compute the average moments of the cluster size distributions

    Helper function for :func:`microcanonical_averages`

    Parameters
    ----------

    moments : 2-D :py:class:`numpy.ndarray` of int
        ``moments.shape[1] == 5`.
        Each array ``moments[r, :]`` is the ``moments`` field of the output of
        :func:`sample_states`:
        The ``k``-th entry is the ``k``-th raw moment of the (absolute) cluster
        size distribution.

    alpha: float
        Significance level.

    Returns
    -------

    ret : dict
        Moment statistics

    ret['moments'] : 1-D :py:class:`numpy.ndarray` of float, size 5
        The ``k``-th entry is the average ``k``-th raw moment of the (absolute)
        cluster size distribution, with ``k`` ranging from ``0`` to ``4``.

    ret['moments_ci'] : 2-D :py:class:`numpy.ndarray` of float, shape (5,2)
        ``ret['moments_ci'][k]`` are the lower and upper bounds of the normal
        confidence interval of the average ``k``-th raw moment of the
        (absolute) cluster size distribution, with ``k`` ranging from ``0`` to
        ``4``.

    See Also
    --------

    sample_states : computation of moments

    microcanonical_averages : moment statistics
    """

    ret = dict()
    runs = moments.shape[0]
    sqrt_n = np.sqrt(runs)

    moments_sample_mean = moments.mean(axis=0)
    ret['moments'] = moments_sample_mean

    moments_sample_std = moments.std(axis=0, ddof=1)
    ret['moments_ci'] = np.empty((5, 2))
    for k in range(5):
        if moments_sample_std[k]:
            old_settings = np.seterr(all='raise')
            ret['moments_ci'][k] = scipy.stats.t.interval(
                1 - alpha,
                df=runs - 1,
                loc=moments_sample_mean[k],
                scale=moments_sample_std[k] / sqrt_n
            )
            np.seterr(**old_settings)
        else:
            ret['moments_ci'][k] = (
                moments_sample_mean[k] * np.ones(2)
            )

    return ret


Instruction:
Can you generate the documentation for the following Python 3 function
def microcanonical_averages(
    graph, runs=40, spanning_cluster=True, model='bond', alpha=alpha_1sigma,
    copy_result=True
):
    r'''
    Generate successive microcanonical percolation ensemble averages

    This is a :ref:`generator function <python:tut-generators>` to successively
    add one edge at a time from the graph to the percolation model for a number
    of independent runs in parallel.
    At each iteration, it calculates and returns the averaged cluster
    statistics.

    Parameters
    ----------
    graph : networkx.Graph
        The substrate graph on which percolation is to take place

    runs : int, optional
        Number of independent runs.
        Defaults to ``40``.

    spanning_cluster : bool, optional
        Defaults to ``True``.

    model : str, optional
        The percolation model (either ``'bond'`` or ``'site'``).
        Defaults to ``'bond'``.

        .. note:: Other models than ``'bond'`` are not supported yet.

    alpha: float, optional
        Significance level.
        Defaults to 1 sigma of the normal distribution.
        ``1 - alpha`` is the confidence level.

    copy_result : bool, optional
        Whether to return a copy or a reference to the result dictionary.
        Defaults to ``True``.

    Yields
    ------
    ret : dict
        Cluster statistics

    ret['n'] : int
        Number of occupied bonds

    ret['N'] : int
        Total number of sites

    ret['M'] : int
        Total number of bonds

    ret['spanning_cluster'] : float
        The average number (Binomial proportion) of runs that have a spanning
        cluster.
        This is the Bayesian point estimate of the posterior mean, with a
        uniform prior.
        Only exists if `spanning_cluster` is set to ``True``.

    ret['spanning_cluster_ci'] : 1-D :py:class:`numpy.ndarray` of float, size 2
        The lower and upper bounds of the Binomial proportion confidence
        interval with uniform prior.
        Only exists if `spanning_cluster` is set to ``True``.

    ret['max_cluster_size'] : float
        Average size of the largest cluster (absolute number of sites)

    ret['max_cluster_size_ci'] : 1-D :py:class:`numpy.ndarray` of float, size 2
        Lower and upper bounds of the normal confidence interval of the average
        size of the largest cluster (absolute number of sites)

    ret['moments'] : 1-D :py:class:`numpy.ndarray` of float, size 5
        The ``k``-th entry is the average ``k``-th raw moment of the (absolute)
        cluster size distribution, with ``k`` ranging from ``0`` to ``4``.

    ret['moments_ci'] : 2-D :py:class:`numpy.ndarray` of float, shape (5,2)
        ``ret['moments_ci'][k]`` are the lower and upper bounds of the normal
        confidence interval of the average ``k``-th raw moment of the
        (absolute) cluster size distribution, with ``k`` ranging from ``0`` to
        ``4``.

    Raises
    ------
    ValueError
        If `runs` is not a positive integer

    ValueError
        If `alpha` is not a float in the interval (0, 1)

    See also
    --------

    sample_states

    percolate.percolate._microcanonical_average_spanning_cluster

    percolate.percolate._microcanonical_average_max_cluster_size

    Notes
    -----
    Iterating through this generator corresponds to several parallel runs of
    the Newman-Ziff algorithm.
    Each iteration yields a microcanonical percolation ensemble for the number
    :math:`n` of occupied bonds. [9]_
    The first iteration yields the trivial microcanonical percolation ensemble
    with :math:`n = 0` occupied bonds.

    Spanning cluster

        .. seealso:: :py:func:`sample_states`

    Raw moments of the cluster size distribution

        .. seealso:: :py:func:`sample_states`

    References
    ----------
    .. [9] Newman, M. E. J. & Ziff, R. M. Fast monte carlo algorithm for site
        or bond percolation. Physical Review E 64, 016706+ (2001),
        `doi:10.1103/physreve.64.016706 <http://dx.doi.org/10.1103/physreve.64.016706>`_.

    '''

    try:
        runs = int(runs)
    except:
        raise ValueError("runs needs to be a positive integer")

    if runs <= 0:
        raise ValueError("runs needs to be a positive integer")

    try:
        alpha = float(alpha)
    except:
        raise ValueError("alpha needs to be a float in the interval (0, 1)")

    if alpha <= 0.0 or alpha >= 1.0:
        raise ValueError("alpha needs to be a float in the interval (0, 1)")

    # initial iteration
    # we do not need a copy of the result dictionary since we copy the values
    # anyway
    run_iterators = [
        sample_states(
            graph, spanning_cluster=spanning_cluster, model=model,
            copy_result=False
        )
        for _ in range(runs)
    ]

    ret = dict()
    for microcanonical_ensemble in zip(*run_iterators):
        # merge cluster statistics
        ret['n'] = microcanonical_ensemble[0]['n']
        ret['N'] = microcanonical_ensemble[0]['N']
        ret['M'] = microcanonical_ensemble[0]['M']

        max_cluster_size = np.empty(runs)
        moments = np.empty((runs, 5))
        if spanning_cluster:
            has_spanning_cluster = np.empty(runs)

        for r, state in enumerate(microcanonical_ensemble):
            assert state['n'] == ret['n']
            assert state['N'] == ret['N']
            assert state['M'] == ret['M']
            max_cluster_size[r] = state['max_cluster_size']
            moments[r] = state['moments']
            if spanning_cluster:
                has_spanning_cluster[r] = state['has_spanning_cluster']

        ret.update(_microcanonical_average_max_cluster_size(
            max_cluster_size, alpha
        ))

        ret.update(_microcanonical_average_moments(moments, alpha))

        if spanning_cluster:
            ret.update(_microcanonical_average_spanning_cluster(
                has_spanning_cluster, alpha
            ))

        if copy_result:
            yield copy.deepcopy(ret)
        else:
            yield ret

Response: r Returns a dictionary of microcanonical percolation ensemble averages for the given number of independent runs.


Instruction:
Implement a Python 3 function for
generating a linear chain with auxiliary nodes for spanning cluster detection.

Response:
def spanning_1d_chain(length):
    """
    Generate a linear chain with auxiliary nodes for spanning cluster detection

    Parameters
    ----------

    length : int
       Number of nodes in the chain, excluding the auxiliary nodes.

    Returns
    -------

    networkx.Graph
       A linear chain graph with auxiliary nodes for spanning cluster detection

    See Also
    --------

    sample_states : spanning cluster detection

    """
    ret = nx.grid_graph(dim=[int(length + 2)])

    ret.node[0]['span'] = 0
    ret[0][1]['span'] = 0
    ret.node[length + 1]['span'] = 1
    ret[length][length + 1]['span'] = 1

    return ret


Instruction:
How would you explain what the following Python 3 function does
def spanning_2d_grid(length):
    ret = nx.grid_2d_graph(length + 2, length)

    for i in range(length):
        # side 0
        ret.node[(0, i)]['span'] = 0
        ret[(0, i)][(1, i)]['span'] = 0

        # side 1
        ret.node[(length + 1, i)]['span'] = 1
        ret[(length + 1, i)][(length, i)]['span'] = 1

    return ret

Response: Generate a 2D grid of the given number of nodes in one dimension excluding the auxiliary nodes.


Instruction:
Can you tell what is the following Python 3 function doing
def microcanonical_averages_arrays(microcanonical_averages):

    ret = dict()

    for n, microcanonical_average in enumerate(microcanonical_averages):
        assert n == microcanonical_average['n']
        if n == 0:
            num_edges = microcanonical_average['M']
            num_sites = microcanonical_average['N']
            spanning_cluster = ('spanning_cluster' in microcanonical_average)
            ret['max_cluster_size'] = np.empty(num_edges + 1)
            ret['max_cluster_size_ci'] = np.empty((num_edges + 1, 2))

            if spanning_cluster:
                ret['spanning_cluster'] = np.empty(num_edges + 1)
                ret['spanning_cluster_ci'] = np.empty((num_edges + 1, 2))

            ret['moments'] = np.empty((5, num_edges + 1))
            ret['moments_ci'] = np.empty((5, num_edges + 1, 2))

        ret['max_cluster_size'][n] = microcanonical_average['max_cluster_size']
        ret['max_cluster_size_ci'][n] = (
            microcanonical_average['max_cluster_size_ci']
        )

        if spanning_cluster:
            ret['spanning_cluster'][n] = (
                microcanonical_average['spanning_cluster']
            )
            ret['spanning_cluster_ci'][n] = (
                microcanonical_average['spanning_cluster_ci']
            )

        ret['moments'][:, n] = microcanonical_average['moments']
        ret['moments_ci'][:, n] = microcanonical_average['moments_ci']

    # normalize by number of sites
    for key in ret:
        if 'spanning_cluster' in key:
            continue
        ret[key] /= num_sites

    ret['M'] = num_edges
    ret['N'] = num_sites
    return ret

Response: Returns a dictionary that contains the microcanonical averages over all iteration steps into a single array.


Instruction:
Make a summary of the following Python 3 code
def _binomial_pmf(n, p):

    n = int(n)
    ret = np.empty(n + 1)

    nmax = int(np.round(p * n))

    ret[nmax] = 1.0

    old_settings = np.seterr(under='ignore')  # seterr to known value

    for i in range(nmax + 1, n + 1):
        ret[i] = ret[i - 1] * (n - i + 1.0) / i * p / (1.0 - p)

    for i in range(nmax - 1, -1, -1):
        ret[i] = ret[i + 1] * (i + 1.0) / (n - i) * (1.0 - p) / p

    np.seterr(**old_settings)  # reset to default

    return ret / ret.sum()

Response: Compute the binomial PMF according to Newman and Ziff.


Instruction:
Write a Python 3 function for
computing the canonical cluster statistics from microcanonical statistics This is according to Newman and Ziff, Equation (2). Note that we also simply average the bounds of the confidence intervals according to this formula. Parameters ---------- ps : iterable of float Each entry is a probability for which to form the canonical ensemble and compute the weighted statistics from the microcanonical statistics microcanonical_averages_arrays Typically the output of :func:`microcanonical_averages_arrays` Returns ------- ret : dict Canonical ensemble cluster statistics ret['ps'] : iterable of float The parameter `ps` ret['N'] : int Total number of sites ret['M'] : int Total number of bonds ret['spanning_cluster'] : 1-D :py:class:`numpy.ndarray` of float The percolation probability: The normalized average number of runs that have a spanning cluster. ret['spanning_cluster_ci'] : 2-D :py:class:`numpy.ndarray` of float, size 2 The lower and upper bounds of the percolation probability. ret['max_cluster_size'] : 1-D :py:class:`numpy.ndarray` of float The percolation strength: Average relative size of the largest cluster ret['max_cluster_size_ci'] : 2-D :py:class:`numpy.ndarray` of float Lower and upper bounds of the normal confidence interval of the percolation strength. ret['moments'] : 2-D :py:class:`numpy.ndarray` of float, shape (5, M + 1) Average raw moments of the (relative) cluster size distribution. ret['moments_ci'] : 3-D :py:class:`numpy.ndarray` of float, shape (5, M + 1, 2) Lower and upper bounds of the normal confidence interval of the raw moments of the (relative) cluster size distribution. See Also -------- microcanonical_averages microcanonical_averages_arrays

Response:
def canonical_averages(ps, microcanonical_averages_arrays):
    """
    Compute the canonical cluster statistics from microcanonical statistics

    This is according to Newman and Ziff, Equation (2).
    Note that we also simply average the bounds of the confidence intervals
    according to this formula.

    Parameters
    ----------

    ps : iterable of float
       Each entry is a probability for which to form the canonical ensemble
       and compute the weighted statistics from the microcanonical statistics

    microcanonical_averages_arrays
       Typically the output of :func:`microcanonical_averages_arrays`

    Returns
    -------

    ret : dict
       Canonical ensemble cluster statistics

    ret['ps'] : iterable of float
        The parameter `ps`

    ret['N'] : int
        Total number of sites

    ret['M'] : int
        Total number of bonds

    ret['spanning_cluster'] : 1-D :py:class:`numpy.ndarray` of float
        The percolation probability:
        The normalized average number of runs that have a spanning cluster.

    ret['spanning_cluster_ci'] : 2-D :py:class:`numpy.ndarray` of float, size 2
        The lower and upper bounds of the percolation probability.

    ret['max_cluster_size'] : 1-D :py:class:`numpy.ndarray` of float
        The percolation strength:
        Average relative size of the largest cluster

    ret['max_cluster_size_ci'] : 2-D :py:class:`numpy.ndarray` of float
        Lower and upper bounds of the normal confidence interval of the
        percolation strength.

    ret['moments'] : 2-D :py:class:`numpy.ndarray` of float, shape (5, M + 1)
        Average raw moments of the (relative) cluster size distribution.

    ret['moments_ci'] : 3-D :py:class:`numpy.ndarray` of float, shape (5, M + 1, 2)
        Lower and upper bounds of the normal confidence interval of the raw
        moments of the (relative) cluster size distribution.

    See Also
    --------

    microcanonical_averages

    microcanonical_averages_arrays

    """

    num_sites = microcanonical_averages_arrays['N']
    num_edges = microcanonical_averages_arrays['M']
    spanning_cluster = ('spanning_cluster' in microcanonical_averages_arrays)

    ret = dict()
    ret['ps'] = ps
    ret['N'] = num_sites
    ret['M'] = num_edges

    ret['max_cluster_size'] = np.empty(ps.size)
    ret['max_cluster_size_ci'] = np.empty((ps.size, 2))

    if spanning_cluster:
        ret['spanning_cluster'] = np.empty(ps.size)
        ret['spanning_cluster_ci'] = np.empty((ps.size, 2))

    ret['moments'] = np.empty((5, ps.size))
    ret['moments_ci'] = np.empty((5, ps.size, 2))

    for p_index, p in enumerate(ps):
        binomials = _binomial_pmf(n=num_edges, p=p)

        for key, value in microcanonical_averages_arrays.items():
            if len(key) <= 1:
                continue

            if key in ['max_cluster_size', 'spanning_cluster']:
                ret[key][p_index] = np.sum(binomials * value)
            elif key in ['max_cluster_size_ci', 'spanning_cluster_ci']:
                ret[key][p_index] = np.sum(
                    np.tile(binomials, (2, 1)).T * value, axis=0
                )
            elif key == 'moments':
                ret[key][:, p_index] = np.sum(
                    np.tile(binomials, (5, 1)) * value, axis=1
                )
            elif key == 'moments_ci':
                ret[key][:, p_index] = np.sum(
                    np.rollaxis(np.tile(binomials, (5, 2, 1)), 2, 1) * value,
                    axis=1
                )
            else:
                raise NotImplementedError(
                    '{}-dimensional array'.format(value.ndim)
                )

    return ret


Instruction:
How would you explain what the following Python 3 function does
def statistics(
    graph, ps, spanning_cluster=True, model='bond', alpha=alpha_1sigma, runs=40
):
    """
    Helper function to compute percolation statistics

    See Also
    --------

    canonical_averages

    microcanonical_averages

    sample_states

    """

    my_microcanonical_averages = microcanonical_averages(
        graph=graph, runs=runs, spanning_cluster=spanning_cluster, model=model,
        alpha=alpha
    )

    my_microcanonical_averages_arrays = microcanonical_averages_arrays(
        my_microcanonical_averages
    )

    return canonical_averages(ps, my_microcanonical_averages_arrays)

Response: Helper function to compute percolation statistics for a given cluster.


Instruction:
Write a Python 3 function for
rendering the graph as an HTML string.

Response:
def to_html(graph: BELGraph) -> str:
    """Render the graph as an HTML string.

    Common usage may involve writing to a file like:

    >>> from pybel.examples import sialic_acid_graph
    >>> with open('html_output.html', 'w') as file:
    ...     print(to_html(sialic_acid_graph), file=file)
    """
    context = get_network_summary_dict(graph)
    summary_dict = graph.summary_dict()

    citation_years = context['citation_years']
    function_count = context['function_count']
    relation_count = context['relation_count']
    error_count = context['error_count']
    transformations_count = context['modifications_count']
    hub_data = context['hub_data']
    disease_data = context['disease_data']
    authors_count = context['authors_count']
    variants_count = context['variants_count']
    namespaces_count = context['namespaces_count']
    confidence_count = context['confidence_count']
    confidence_data = [
        (label, confidence_count.get(label, 0))
        for label in ('None', 'Very Low', 'Low', 'Medium', 'High', 'Very High')
    ]

    template = environment.get_template('index.html')
    return template.render(
        graph=graph,
        # Node Charts
        chart_1_data=prepare_c3(function_count, 'Node Count'),
        chart_6_data=prepare_c3(namespaces_count, 'Node Count'),
        chart_5_data=prepare_c3(variants_count, 'Node Count'),
        number_variants=sum(variants_count.values()),
        number_namespaces=len(namespaces_count),
        # Edge Charts
        chart_2_data=prepare_c3(relation_count, 'Edge Count'),
        chart_4_data=prepare_c3(transformations_count, 'Edge Count') if transformations_count else None,
        number_transformations=sum(transformations_count.values()),
        # Error Charts
        chart_3_data=prepare_c3(error_count, 'Error Type') if error_count else None,
        # Topology Charts
        chart_7_data=prepare_c3(hub_data, 'Degree'),
        chart_9_data=prepare_c3(disease_data, 'Degree') if disease_data else None,
        # Bibliometrics Charts
        chart_authors_count=prepare_c3(authors_count, 'Edges Contributed'),
        chart_10_data=prepare_c3_time_series(citation_years, 'Number of Articles') if citation_years else None,
        chart_confidence_count=prepare_c3(confidence_data, 'Edge Count'),
        summary_dict=summary_dict,
        # Everything else :)
        **context
    )


Instruction:
Can you generate the documentation for the following Python 3 function
def get_network_summary_dict(graph: BELGraph) -> Mapping:
    return dict(
        # Counters
        function_count=count_functions(graph),
        modifications_count=get_modifications_count(graph),
        relation_count=count_relations(graph),
        authors_count=count_authors(graph).most_common(15),
        variants_count=count_variants(graph),
        namespaces_count=count_namespaces(graph),
        hub_data={
            (
                node.name or node.identifier
                if NAME in node or IDENTIFIER in node else
                str(node)
            ): degree
            for node, degree in get_top_hubs(graph, n=15)
        },
        disease_data={
            (
                node.name or node.identifier
                if NAME in node or IDENTIFIER in node else
                str(node)
            ): count
            for node, count in get_top_pathologies(graph, n=15)
        },
        # BioGrammar
        regulatory_pairs=[
            get_pair_tuple(u, v)
            for u, v in get_regulatory_pairs(graph)
        ],
        unstable_pairs=list(itt.chain(
            (get_pair_tuple(u, v) + ('Chaotic',) for u, v, in get_chaotic_pairs(graph)),
            (get_pair_tuple(u, v) + ('Dampened',) for u, v, in get_dampened_pairs(graph)),
        )),
        contradictory_pairs=[
            get_pair_tuple(u, v) + (relation,)
            for u, v, relation in get_contradiction_summary(graph)
        ],
        contradictory_triplets=list(itt.chain(
            (get_triplet_tuple(a, b, c) + ('Separate',) for a, b, c in
             get_separate_unstable_correlation_triples(graph)),
            (get_triplet_tuple(a, b, c) + ('Mutual',) for a, b, c in get_mutually_unstable_correlation_triples(graph)),
            (get_triplet_tuple(a, b, c) + ('Jens',) for a, b, c in get_jens_unstable(graph)),
            (get_triplet_tuple(a, b, c) + ('Increase Mismatch',) for a, b, c in get_increase_mismatch_triplets(graph)),
            (get_triplet_tuple(a, b, c) + ('Decrease Mismatch',) for a, b, c in get_decrease_mismatch_triplets(graph)),
        )),
        unstable_triplets=list(itt.chain(
            (get_triplet_tuple(a, b, c) + ('Chaotic',) for a, b, c in get_chaotic_triplets(graph)),
            (get_triplet_tuple(a, b, c) + ('Dampened',) for a, b, c in get_dampened_triplets(graph)),
        )),
        causal_pathologies=sorted({
            get_pair_tuple(u, v) + (graph[u][v][k][RELATION],)
            for u, v, k in filter_edges(graph, has_pathology_causal)
        }),
        # Misc.
        undefined_namespaces=get_undefined_namespaces(graph),
        undefined_annotations=get_undefined_annotations(graph),
        namespaces_with_incorrect_names=get_namespaces_with_incorrect_names(graph),
        unused_namespaces=get_unused_namespaces(graph),
        unused_annotations=get_unused_annotations(graph),
        unused_list_annotation_values=get_unused_list_annotation_values(graph),
        naked_names=get_naked_names(graph),
        error_count=count_error_types(graph),
        # Errors
        error_groups=get_most_common_errors(graph),
        syntax_errors=get_syntax_errors(graph),
        # Bibliometrics
        citation_years=get_citation_years(graph),
        confidence_count=count_confidences(graph),
    )

Response: Create a summary dictionary.


Instruction:
Write a Python 3 function for
getting the pair as a tuple of BEL and hashes.

Response:
def get_pair_tuple(a: BaseEntity, b: BaseEntity) -> Tuple[str, str, str, str]:
    """Get the pair as a tuple of BEL/hashes."""
    return a.as_bel(), a.sha512, b.as_bel(), b.sha512


Instruction:
Create a Python 3 function to
get the triple as a tuple of BEL / hashes.

Response:
def get_triplet_tuple(a: BaseEntity, b: BaseEntity, c: BaseEntity) -> Tuple[str, str, str, str, str, str]:
    """Get the triple as a tuple of BEL/hashes."""
    return a.as_bel(), a.sha512, b.as_bel(), b.sha512, c.as_bel(), c.sha512


Instruction:
Can you generate the documentation for the following Python 3 function
def rank_causalr_hypothesis(graph, node_to_regulation, regulator_node):
    upregulation_hypothesis = {
        'correct': 0,
        'incorrect': 0,
        'ambiguous': 0
    }
    downregulation_hypothesis = {
        'correct': 0,
        'incorrect': 0,
        'ambiguous': 0
    }

    targets = [
        node
        for node in node_to_regulation
        if node != regulator_node
    ]

    predicted_regulations = run_cna(graph, regulator_node, targets)  # + signed hypothesis

    for _, target_node, predicted_regulation in predicted_regulations:

        if (predicted_regulation is Effect.inhibition or predicted_regulation is Effect.activation) and (
                predicted_regulation.value == node_to_regulation[target_node]):
            upregulation_hypothesis['correct'] += 1
            downregulation_hypothesis['incorrect'] += 1

        elif predicted_regulation is Effect.ambiguous:
            upregulation_hypothesis['ambiguous'] += 1
            downregulation_hypothesis['ambiguous'] += 1

        elif predicted_regulation is Effect.no_effect:
            continue

        else:
            downregulation_hypothesis['correct'] += 1
            upregulation_hypothesis['incorrect'] += 1

    upregulation_hypothesis['score'] = upregulation_hypothesis['correct'] - upregulation_hypothesis['incorrect']
    downregulation_hypothesis['score'] = downregulation_hypothesis['correct'] - downregulation_hypothesis['incorrect']

    return upregulation_hypothesis, downregulation_hypothesis

Response: This method tests the regulator hypothesis of the given node on the input data using the causal r algorithm.


Instruction:
Create a Python 3 function for
running the CNA algorithm.

Response:
def run_cna(graph, root, targets, relationship_dict=None):
    """ Returns the effect from the root to the target nodes represented as {-1,1}

    :param pybel.BELGraph graph: A BEL graph
    :param BaseEntity root: The root node
    :param iter targets: The targets nodes
    :param dict relationship_dict: dictionary with relationship effects
    :return list[tuple]:
    """
    causal_effects = []

    relationship_dict = causal_effect_dict if relationship_dict is None else relationship_dict

    for target in targets:
        try:
            shortest_paths = nx.all_shortest_paths(graph, source=root, target=target)

            effects_in_path = set()

            for shortest_path in shortest_paths:
                effects_in_path.add(get_path_effect(graph, shortest_path, relationship_dict))

            if len(effects_in_path) == 1:
                causal_effects.append((root, target, next(iter(effects_in_path))))  # Append the only predicted effect

            elif Effect.activation in effects_in_path and Effect.inhibition in effects_in_path:
                causal_effects.append((root, target, Effect.ambiguous))

            elif Effect.activation in effects_in_path and Effect.inhibition not in effects_in_path:
                causal_effects.append((root, target, Effect.activation))

            elif Effect.inhibition in effects_in_path and Effect.activation not in effects_in_path:
                causal_effects.append((root, target, Effect.inhibition))

            else:
                log.warning('Exception in set: {}.'.format(effects_in_path))

        except nx.NetworkXNoPath:
            log.warning('No shortest path between: {} and {}.'.format(root, target))

    return causal_effects


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_path_effect(graph, path, relationship_dict):
    causal_effect = []

    for predecessor, successor in pairwise(path):

        if pair_has_contradiction(graph, predecessor, successor):
            return Effect.ambiguous

        edges = graph.get_edge_data(predecessor, successor)

        edge_key, edge_relation, _ = rank_edges(edges)

        relation = graph[predecessor][successor][edge_key][RELATION]

        # Returns Effect.no_effect if there is a non causal edge in path
        if relation not in relationship_dict or relationship_dict[relation] == 0:
            return Effect.no_effect

        causal_effect.append(relationship_dict[relation])

    final_effect = reduce(lambda x, y: x * y, causal_effect)

    return Effect.activation if final_effect == 1 else Effect.inhibition

Response: Calculate the final effect of the root node to the sink node in the path.


Instruction:
Make a summary of the following Python 3 code
def rank_edges(edges, edge_ranking=None):
    edge_ranking = default_edge_ranking if edge_ranking is None else edge_ranking

    edges_scores = [
        (edge_id, edge_data[RELATION], edge_ranking[edge_data[RELATION]])
        for edge_id, edge_data in edges.items()
    ]

    return max(edges_scores, key=itemgetter(2))

Response: Return the highest ranked edge from a multiedge.


Instruction:
Write a Python 3 function for
grouping the nodes occurring in edges by the given annotation.

Response:
def group_nodes_by_annotation(graph: BELGraph, annotation: str = 'Subgraph') -> Mapping[str, Set[BaseEntity]]:
    """Group the nodes occurring in edges by the given annotation."""
    result = defaultdict(set)

    for u, v, d in graph.edges(data=True):
        if not edge_has_annotation(d, annotation):
            continue

        result[d[ANNOTATIONS][annotation]].add(u)
        result[d[ANNOTATIONS][annotation]].add(v)

    return dict(result)


Instruction:
How would you implement a function in Python 3 that
returns a mapping from subgraphs to their average value based on the average of all nodes values.

Response:
def average_node_annotation(graph: BELGraph,
                            key: str,
                            annotation: str = 'Subgraph',
                            aggregator: Optional[Callable[[Iterable[X]], X]] = None,
                            ) -> Mapping[str, X]:
    """Groups graph into subgraphs and assigns each subgraph a score based on the average of all nodes values
    for the given node key

    :param pybel.BELGraph graph: A BEL graph
    :param key: The key in the node data dictionary representing the experimental data
    :param annotation: A BEL annotation to use to group nodes
    :param aggregator: A function from list of values -> aggregate value. Defaults to taking the average of a list of
                       floats.
    :type aggregator: lambda
    """

    if aggregator is None:
        def aggregator(x):
            """Calculates the average"""
            return sum(x) / len(x)

    result = {}

    for subgraph, nodes in group_nodes_by_annotation(graph, annotation).items():
        values = [graph.nodes[node][key] for node in nodes if key in graph.nodes[node]]
        result[subgraph] = aggregator(values)

    return result


Instruction:
Can you generate the documentation for the following Python 3 function
def group_nodes_by_annotation_filtered(graph: BELGraph,
                                       node_predicates: NodePredicates = None,
                                       annotation: str = 'Subgraph',
                                       ) -> Mapping[str, Set[BaseEntity]]:
    """Group the nodes occurring in edges by the given annotation, with a node filter applied.

    :param graph: A BEL graph
    :param node_predicates: A predicate or list of predicates (graph, node) -> bool
    :param annotation: The annotation to use for grouping
    :return: A dictionary of {annotation value: set of nodes}
    """
    node_filter = concatenate_node_predicates(node_predicates)

    return {
        key: {
            node
            for node in nodes
            if node_filter(graph, node)
        }
        for key, nodes in group_nodes_by_annotation(graph, annotation).items()
    }

Response: Group the nodes occurring in edges by the given annotation.


Instruction:
Write a Python 3 function that can
return a dict with keys = nodes that match the namespace and in names and values other nodes = variants orthologous or this node.

Response:
def get_mapped_nodes(graph: BELGraph, namespace: str, names: Iterable[str]) -> Mapping[BaseEntity, Set[BaseEntity]]:
    """Return a dict with keys: nodes that match the namespace and in names and values other nodes (complexes, variants, orthologous...) or this node.

    :param graph: A BEL graph
    :param namespace: The namespace to search
    :param names: List or set of values from which we want to map nodes from
    :return: Main node to variants/groups.
    """
    parent_to_variants = defaultdict(set)
    names = set(names)

    for u, v, d in graph.edges(data=True):
        if d[RELATION] in {HAS_MEMBER, HAS_COMPONENT} and v.get(NAMESPACE) == namespace and v.get(NAME) in names:
            parent_to_variants[v].add(u)

        elif d[RELATION] == HAS_VARIANT and u.get(NAMESPACE) == namespace and u.get(NAME) in names:
            parent_to_variants[u].add(v)

        elif d[RELATION] == ORTHOLOGOUS and u.get(NAMESPACE) == namespace and u.get(NAME) in names:
            parent_to_variants[u].add(v)

    return dict(parent_to_variants)


Instruction:
Can you generate the documentation for the following Python 3 function
def build_expand_node_neighborhood_by_hash(manager: Manager) -> Callable[[BELGraph, BELGraph, str], None]:

    @uni_in_place_transformation
    def expand_node_neighborhood_by_hash(universe: BELGraph, graph: BELGraph, node_hash: str) -> None:
        """Expand around the neighborhoods of a node by identifier."""
        node = manager.get_dsl_by_hash(node_hash)
        return expand_node_neighborhood(universe, graph, node)

    return expand_node_neighborhood_by_hash

Response: Builds an expand function that expands around the neighborhoods of a node by its hash.


Instruction:
How would you implement a function in Python 3 that
builds a delete function that can be used to remove a node by its hash.

Response:
def build_delete_node_by_hash(manager: Manager) -> Callable[[BELGraph, str], None]:
    """Make a delete function that's bound to the manager."""

    @in_place_transformation
    def delete_node_by_hash(graph: BELGraph, node_hash: str) -> None:
        """Remove a node by identifier."""
        node = manager.get_dsl_by_hash(node_hash)
        graph.remove_node(node)

    return delete_node_by_hash


Instruction:
Here you have a function in Python 3, explain what it does
def bel_to_spia_matrices(graph: BELGraph) -> Mapping[str, pd.DataFrame]:
    index_nodes = get_matrix_index(graph)
    spia_matrices = build_spia_matrices(index_nodes)

    for u, v, edge_data in graph.edges(data=True):
        # Both nodes are CentralDogma abundances
        if isinstance(u, CentralDogma) and isinstance(v, CentralDogma):
            # Update matrix dict
            update_spia_matrices(spia_matrices, u, v, edge_data)

        # Subject is CentralDogmaAbundance and node is ListAbundance
        elif isinstance(u, CentralDogma) and isinstance(v, ListAbundance):
            # Add a relationship from subject to each of the members in the object
            for node in v.members:
                # Skip if the member is not in CentralDogma
                if not isinstance(node, CentralDogma):
                    continue

                update_spia_matrices(spia_matrices, u, node, edge_data)

        # Subject is ListAbundance and node is CentralDogmaAbundance
        elif isinstance(u, ListAbundance) and isinstance(v, CentralDogma):
            # Add a relationship from each of the members of the subject to the object
            for node in u.members:
                # Skip if the member is not in CentralDogma
                if not isinstance(node, CentralDogma):
                    continue

                update_spia_matrices(spia_matrices, node, v, edge_data)

        # Both nodes are ListAbundance
        elif isinstance(u, ListAbundance) and isinstance(v, ListAbundance):
            for sub_member, obj_member in product(u.members, v.members):
                # Update matrix if both are CentralDogma
                if isinstance(sub_member, CentralDogma) and isinstance(obj_member, CentralDogma):
                    update_spia_matrices(spia_matrices, sub_member, obj_member, edge_data)

        # else Not valid edge

    return spia_matrices

Response: Convert a BEL graph to a list of SPIA matrices.


Instruction:
Implement a function in Python 3 to
return set of HGNC names from Proteins / Rnas / Genes / miRNA nodes that can be used by SPIA.

Response:
def get_matrix_index(graph: BELGraph) -> Set[str]:
    """Return set of HGNC names from Proteins/Rnas/Genes/miRNA, nodes that can be used by SPIA."""
    # TODO: Using HGNC Symbols for now
    return {
        node.name
        for node in graph
        if isinstance(node, CentralDogma) and node.namespace.upper() == 'HGNC'
    }


Instruction:
Here you have a function in Python 3, explain what it does
def build_spia_matrices(nodes: Set[str]) -> Dict[str, pd.DataFrame]:
    nodes = list(sorted(nodes))

    # Create sheets of the excel in the given order
    matrices = OrderedDict()
    for relation in KEGG_RELATIONS:
        matrices[relation] = pd.DataFrame(0, index=nodes, columns=nodes)

    return matrices

Response: Build an adjacency matrix for each KEGG relationship and return in a dictionary.


Instruction:
Explain what the following Python 3 code does
def update_spia_matrices(spia_matrices: Dict[str, pd.DataFrame],
                         u: CentralDogma,
                         v: CentralDogma,
                         edge_data: EdgeData,
                         ) -> None:
    """Populate the adjacency matrix."""
    if u.namespace.upper() != 'HGNC' or v.namespace.upper() != 'HGNC':
        return

    u_name = u.name
    v_name = v.name
    relation = edge_data[RELATION]

    if relation in CAUSAL_INCREASE_RELATIONS:
        # If it has pmod check which one and add it to the corresponding matrix
        if v.variants and any(isinstance(variant, ProteinModification) for variant in v.variants):
            for variant in v.variants:
                if not isinstance(variant, ProteinModification):
                    continue
                if variant[IDENTIFIER][NAME] == "Ub":
                    spia_matrices["activation_ubiquination"][u_name][v_name] = 1
                elif variant[IDENTIFIER][NAME] == "Ph":
                    spia_matrices["activation_phosphorylation"][u_name][v_name] = 1
        elif isinstance(v, (Gene, Rna)):  # Normal increase, add activation
            spia_matrices['expression'][u_name][v_name] = 1
        else:
            spia_matrices['activation'][u_name][v_name] = 1

    elif relation in CAUSAL_DECREASE_RELATIONS:
        # If it has pmod check which one and add it to the corresponding matrix
        if v.variants and any(isinstance(variant, ProteinModification) for variant in v.variants):
            for variant in v.variants:
                if not isinstance(variant, ProteinModification):
                    continue
                if variant[IDENTIFIER][NAME] == "Ub":
                    spia_matrices['inhibition_ubiquination'][u_name][v_name] = 1
                elif variant[IDENTIFIER][NAME] == "Ph":
                    spia_matrices["inhibition_phosphorylation"][u_name][v_name] = 1
        elif isinstance(v, (Gene, Rna)):  # Normal decrease, check which matrix
            spia_matrices["repression"][u_name][v_name] = 1
        else:
            spia_matrices["inhibition"][u_name][v_name] = 1

    elif relation == ASSOCIATION:
        spia_matrices["binding_association"][u_name][v_name] = 1

Response: Populate the adjacency matrix.


Instruction:
Implement a function in Python 3 to
export a SPIA data dictionary into an Excel sheet at the given path.

Response:
def spia_matrices_to_excel(spia_matrices: Mapping[str, pd.DataFrame], path: str) -> None:
    """Export a SPIA data dictionary into an Excel sheet at the given path.

    .. note::

        # The R import should add the values:
        # ["nodes"] from the columns
        # ["title"] from the name of the file
        # ["NumberOfReactions"] set to "0"
    """
    writer = pd.ExcelWriter(path, engine='xlsxwriter')

    for relation, df in spia_matrices.items():
        df.to_excel(writer, sheet_name=relation, index=False)

    # Save excel
    writer.save()


Instruction:
Given the following Python 3 function, write the documentation
def spia_matrices_to_tsvs(spia_matrices: Mapping[str, pd.DataFrame], directory: str) -> None:
    os.makedirs(directory, exist_ok=True)
    for relation, df in spia_matrices.items():
        df.to_csv(os.path.join(directory, f'{relation}.tsv'), index=True)

Response: Export a SPIA data dictionary into a directory as several TSV documents.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def main(graph: BELGraph, xlsx: str, tsvs: str):
    if not xlsx and not tsvs:
        click.secho('Specify at least one option --xlsx or --tsvs', fg='red')
        sys.exit(1)

    spia_matrices = bel_to_spia_matrices(graph)

    if xlsx:
        spia_matrices_to_excel(spia_matrices, xlsx)

    if tsvs:
        spia_matrices_to_tsvs(spia_matrices, tsvs)

Response: Export the graph to a SPIA Excel sheet.


Instruction:
Can you create a Python 3 function that
overlays tabular data on the network.

Response:
def overlay_data(graph: BELGraph,
                 data: Mapping[BaseEntity, Any],
                 label: Optional[str] = None,
                 overwrite: bool = False,
                 ) -> None:
    """Overlays tabular data on the network

    :param graph: A BEL Graph
    :param data: A dictionary of {tuple node: data for that node}
    :param label: The annotation label to put in the node dictionary
    :param overwrite: Should old annotations be overwritten?
    """
    if label is None:
        label = WEIGHT

    for node, value in data.items():
        if node not in graph:
            log.debug('%s not in graph', node)
            continue

        if label in graph.nodes[node] and not overwrite:
            log.debug('%s already on %s', label, node)
            continue

        graph.nodes[node][label] = value


Instruction:
Here you have a function in Python 3, explain what it does
def overlay_type_data(graph: BELGraph,
                      data: Mapping[str, float],
                      func: str,
                      namespace: str,
                      label: Optional[str] = None,
                      overwrite: bool = False,
                      impute: Optional[float] = None,
                      ) -> None:
    """Overlay tabular data on the network for data that comes from an data set with identifiers that lack
    namespaces.

    For example, if you want to overlay differential gene expression data from a table, that table
    probably has HGNC identifiers, but no specific annotations that they are in the HGNC namespace or
    that the entities to which they refer are RNA.

    :param graph: A BEL Graph
    :param dict data: A dictionary of {name: data}
    :param func: The function of the keys in the data dictionary
    :param namespace: The namespace of the keys in the data dictionary
    :param label: The annotation label to put in the node dictionary
    :param overwrite: Should old annotations be overwritten?
    :param impute: The value to use for missing data
    """
    new_data = {
        node: data.get(node[NAME], impute)
        for node in filter_nodes(graph, function_namespace_inclusion_builder(func, namespace))
    }

    overlay_data(graph, new_data, label=label, overwrite=overwrite)

Response: Overlay tabular data on the network for data that comes from a table with identifiers that lack the HGNC identifiers.


Instruction:
How would you implement a function in Python 3 that
loads and pre - process a differential gene expression data.

Response:
def load_differential_gene_expression(path: str,
                                      gene_symbol_column: str = 'Gene.symbol',
                                      logfc_column: str = 'logFC',
                                      aggregator: Optional[Callable[[List[float]], float]] = None,
                                      ) -> Mapping[str, float]:
    """Load and pre-process a differential gene expression data.

    :param path: The path to the CSV
    :param gene_symbol_column: The header of the gene symbol column in the data frame
    :param logfc_column: The header of the log-fold-change column in the data frame
    :param aggregator: A function that aggregates a list of differential gene expression values. Defaults to
                       :func:`numpy.median`. Could also use: :func:`numpy.mean`, :func:`numpy.average`,
                       :func:`numpy.min`, or :func:`numpy.max`
    :return: A dictionary of {gene symbol: log fold change}
    """
    if aggregator is None:
        aggregator = np.median

    # Load the data frame
    df = pd.read_csv(path)

    # Check the columns exist in the data frame
    assert gene_symbol_column in df.columns
    assert logfc_column in df.columns

    # throw away columns that don't have gene symbols - these represent control sequences
    df = df.loc[df[gene_symbol_column].notnull(), [gene_symbol_column, logfc_column]]

    values = defaultdict(list)

    for _, gene_symbol, log_fold_change in df.itertuples():
        values[gene_symbol].append(log_fold_change)

    return {
        gene_symbol: aggregator(log_fold_changes)
        for gene_symbol, log_fold_changes in values.items()
    }


Instruction:
Can you implement a function in Python 3 that
loads many namespaces and merges their names.

Response:
def get_merged_namespace_names(locations, check_keywords=True):
    """Loads many namespaces and combines their names.

    :param iter[str] locations: An iterable of URLs or file paths pointing to BEL namespaces.
    :param bool check_keywords: Should all the keywords be the same? Defaults to ``True``
    :return: A dictionary of {names: labels}
    :rtype: dict[str, str]

    Example Usage

    >>> from pybel.resources import write_namespace
    >>> from pybel_tools.definition_utils import export_namespace, get_merged_namespace_names
    >>> graph = ...
    >>> original_ns_url = ...
    >>> export_namespace(graph, 'MBS') # Outputs in current directory to MBS.belns
    >>> value_dict = get_merged_namespace_names([original_ns_url, 'MBS.belns'])
    >>> with open('merged_namespace.belns', 'w') as f:
    >>> ...  write_namespace('MyBrokenNamespace', 'MBS', 'Other', 'Charles Hoyt', 'PyBEL Citation', value_dict, file=f)
    """
    resources = {location: get_bel_resource(location) for location in locations}

    if check_keywords:
        resource_keywords = set(config['Namespace']['Keyword'] for config in resources.values())
        if 1 != len(resource_keywords):
            raise ValueError('Tried merging namespaces with different keywords: {}'.format(resource_keywords))

    result = {}
    for resource in resources:
        result.update(resource['Values'])
    return result


Instruction:
Can you generate the documentation for the following Python 3 function
def merge_namespaces(input_locations, output_path, namespace_name, namespace_keyword, namespace_domain, author_name,
                     citation_name, namespace_description=None, namespace_species=None, namespace_version=None,
                     namespace_query_url=None, namespace_created=None, author_contact=None, author_copyright=None,
                     citation_description=None, citation_url=None, citation_version=None, citation_date=None,
                     case_sensitive=True, delimiter='|', cacheable=True, functions=None, value_prefix='',
                     sort_key=None, check_keywords=True):
    """Merges namespaces from multiple locations to one.

    :param iter input_locations: An iterable of URLs or file paths pointing to BEL namespaces.
    :param str output_path: The path to the file to write the merged namespace
    :param str namespace_name: The namespace name
    :param str namespace_keyword: Preferred BEL Keyword, maximum length of 8
    :param str namespace_domain: One of: :data:`pybel.constants.NAMESPACE_DOMAIN_BIOPROCESS`,
                            :data:`pybel.constants.NAMESPACE_DOMAIN_CHEMICAL`,
                            :data:`pybel.constants.NAMESPACE_DOMAIN_GENE`, or
                            :data:`pybel.constants.NAMESPACE_DOMAIN_OTHER`
    :param str author_name: The namespace's authors
    :param str citation_name: The name of the citation
    :param str namespace_query_url: HTTP URL to query for details on namespace values (must be valid URL)
    :param str namespace_description: Namespace description
    :param str namespace_species: Comma-separated list of species taxonomy id's
    :param str namespace_version: Namespace version
    :param str namespace_created: Namespace public timestamp, ISO 8601 datetime
    :param str author_contact: Namespace author's contact info/email address
    :param str author_copyright: Namespace's copyright/license information
    :param str citation_description: Citation description
    :param str citation_url: URL to more citation information
    :param str citation_version: Citation version
    :param str citation_date: Citation publish timestamp, ISO 8601 Date
    :param bool case_sensitive: Should this config file be interpreted as case-sensitive?
    :param str delimiter: The delimiter between names and labels in this config file
    :param bool cacheable: Should this config file be cached?
    :param functions: The encoding for the elements in this namespace
    :type functions: iterable of characters
    :param str value_prefix: a prefix for each name
    :param sort_key: A function to sort the values with :func:`sorted`
    :param bool check_keywords: Should all the keywords be the same? Defaults to ``True``
    """
    results = get_merged_namespace_names(input_locations, check_keywords=check_keywords)

    with open(output_path, 'w') as file:
        write_namespace(
            namespace_name=namespace_name,
            namespace_keyword=namespace_keyword,
            namespace_domain=namespace_domain,
            author_name=author_name,
            citation_name=citation_name,
            values=results,
            namespace_species=namespace_species,
            namespace_description=namespace_description,
            namespace_query_url=namespace_query_url,
            namespace_version=namespace_version,
            namespace_created=namespace_created,
            author_contact=author_contact,
            author_copyright=author_copyright,
            citation_description=citation_description,
            citation_url=citation_url,
            citation_version=citation_version,
            citation_date=citation_date,
            case_sensitive=case_sensitive,
            delimiter=delimiter,
            cacheable=cacheable,
            functions=functions,
            value_prefix=value_prefix,
            sort_key=sort_key,
            file=file
        )

Response: Merges multiple BEL namespaces into one.


Instruction:
Write a Python 3 script for
running the reverse causal reasoning algorithm on a graph.

Response:
def run_rcr(graph, tag='dgxp'):
    """Run the reverse causal reasoning algorithm on a graph.

    Steps:

    1. Get all downstream controlled things into map (that have at least 4 downstream things)
    2. calculate population of all things that are downstream controlled

    .. note:: Assumes all nodes have been pre-tagged with data

    :param pybel.BELGraph graph:
    :param str tag: The key for the nodes' data dictionaries that corresponds to the integer value for its differential
                    expression.
    """

    # Step 1: Calculate the hypothesis subnetworks (just simple star graphs)
    hypotheses = defaultdict(set)
    increases = defaultdict(set)
    decreases = defaultdict(set)

    for u, v, d in graph.edges(data=True):
        hypotheses[u].add(v)

        if d[RELATION] in CAUSAL_INCREASE_RELATIONS:
            increases[u].add(v)

        elif d[RELATION] in CAUSAL_DECREASE_RELATIONS:
            decreases[u].add(v)

    # Step 2: Calculate the matching of the data points to the causal relationships

    #: A dictionary from {tuple controller node: int count of correctly matching observations}
    correct = defaultdict(int)
    #: A dictionary from {tuple controller node: int count of incorrectly matching observations}
    contra = defaultdict(int)
    #: A dictionary from {tuple controller node: int count of ambiguous observations}
    ambiguous = defaultdict(int)
    #: A dictionary from {tuple controller node: int count of missing obvservations}
    missing = defaultdict(int)

    for controller, downstream_nodes in hypotheses.items():
        if len(downstream_nodes) < 4:
            continue  # need enough data to make reasonable calculations!

        for node in downstream_nodes:

            if node in increases[controller] and node in decreases[controller]:
                ambiguous[controller] += 1

            elif node in increases[controller]:
                if graph.node[node][tag] == 1:
                    correct[controller] += 1
                elif graph.node[node][tag] == -1:
                    contra[controller] += 1

            elif node in decreases[controller]:
                if graph.node[node][tag] == 1:
                    contra[controller] += 1
                elif graph.node[node][tag] == -1:
                    correct[controller] += 1

            else:
                missing[controller] += 1

    # Step 3: Keep only controller nodes who have 4 or more downstream nodes
    controllers = {
        controller
        for controller, downstream_nodes in hypotheses.items()
        if 4 <= len(downstream_nodes)
    }

    # Step 4: Calculate concordance scores
    concordance_scores = {
        controller: scipy.stats.beta(0.5, correct[controller], contra[controller])
        for controller in controllers
    }

    # Step 5: Calculate richness scores
    # TODO

    # Calculate the population as the union of all downstream nodes for all controllers
    population = {
        node
        for controller in controllers
        for node in hypotheses[controller]
    }
    population_size = len(population)

    # Step 6: Export

    return pandas.DataFrame({
        'contra': contra,
        'correct': correct,
        'concordance': concordance_scores
    })


Instruction:
Here you have a function in Python 3, explain what it does
def export_namespace(graph, namespace, directory=None, cacheable=False):
    directory = os.getcwd() if directory is None else directory
    path = os.path.join(directory, '{}.belns'.format(namespace))

    with open(path, 'w') as file:
        log.info('Outputting to %s', path)
        right_names = get_names_by_namespace(graph, namespace)
        log.info('Graph has %d correct names in %s', len(right_names), namespace)
        wrong_names = get_incorrect_names_by_namespace(graph, namespace)
        log.info('Graph has %d incorrect names in %s', len(right_names), namespace)
        undefined_ns_names = get_undefined_namespace_names(graph, namespace)
        log.info('Graph has %d names in missing namespace %s', len(right_names), namespace)

        names = (right_names | wrong_names | undefined_ns_names)

        if 0 == len(names):
            log.warning('%s is empty', namespace)

        write_namespace(
            namespace_name=namespace,
            namespace_keyword=namespace,
            namespace_domain='Other',
            author_name=graph.authors,
            author_contact=graph.contact,
            citation_name=graph.name,
            values=names,
            cacheable=cacheable,
            file=file
        )

Response: Exports all names and missing names from the given namespace to its own BEL Namespace files in the given directory.


Instruction:
Can you create a Python 3 function that
helps remove extraneous whitespace from the lines of a file

Response:
def lint_file(in_file, out_file=None):
    """Helps remove extraneous whitespace from the lines of a file

    :param file in_file: A readable file or file-like
    :param file out_file: A writable file or file-like
    """
    for line in in_file:
        print(line.strip(), file=out_file)


Instruction:
Create a Python 3 function to
add a linted version of each document in the source directory to the target directory ArcGIS does not have a. bel extension.

Response:
def lint_directory(source, target):
    """Adds a linted version of each document in the source directory to the target directory

    :param str source: Path to directory to lint
    :param str target: Path to directory to output
    """
    for path in os.listdir(source):
        if not path.endswith('.bel'):
            continue

        log.info('linting: %s', path)
        with open(os.path.join(source, path)) as i, open(os.path.join(target, path), 'w') as o:
            lint_file(i, o)


Instruction:
How would you explain what the following Python 3 function does
def make_pubmed_abstract_group(pmids: Iterable[Union[str, int]]) -> Iterable[str]:
    for pmid in set(pmids):
        yield ''

        res = requests.get(title_url_fmt.format(pmid))
        title = res.content.decode('utf-8').strip()

        yield 'SET Citation = {{"{}", "{}"}}'.format(title, pmid)

        res = requests.get(abstract_url_fmt.format(pmid))
        abstract = res.content.decode('utf-8').strip()

        yield 'SET Evidence = "{}"'.format(abstract)
        yield '\nUNSET Evidence\nUNSET Citation'

Response: Build a skeleton for the citations and evidence groups.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_entrez_gene_data(entrez_ids: Iterable[Union[str, int]]):
    url = PUBMED_GENE_QUERY_URL.format(','.join(str(x).strip() for x in entrez_ids))
    response = requests.get(url)
    tree = ElementTree.fromstring(response.content)

    return {
        element.attrib['uid']: {
            'summary': _sanitize(element.find('Summary').text),
            'description': element.find('Description').text
        }
        for element in tree.findall('./DocumentSummarySet/DocumentSummary')
    }

Response: Get gene info from Entrez.


Instruction:
Implement a function in Python 3 to
build a skeleton for the PubMed gene summaries

Response:
def make_pubmed_gene_group(entrez_ids: Iterable[Union[str, int]]) -> Iterable[str]:
    """Builds a skeleton for gene summaries

    :param entrez_ids: A list of Entrez Gene identifiers to query the PubMed service
    :return: An iterator over statement lines for NCBI Entrez Gene summaries
    """
    url = PUBMED_GENE_QUERY_URL.format(','.join(str(x).strip() for x in entrez_ids))
    response = requests.get(url)
    tree = ElementTree.fromstring(response.content)

    for x in tree.findall('./DocumentSummarySet/DocumentSummary'):
        yield '\n# {}'.format(x.find('Description').text)
        yield 'SET Citation = {{"Other", "PubMed Gene", "{}"}}'.format(x.attrib['uid'])
        yield 'SET Evidence = "{}"'.format(x.find('Summary').text.strip().replace('\n', ''))
        yield '\nUNSET Evidence\nUNSET Citation'


Instruction:
How would you implement a function in Python 3 that
writes a BEL document to a file.

Response:
def write_boilerplate(name: str,
                      version: Optional[str] = None,
                      description: Optional[str] = None,
                      authors: Optional[str] = None,
                      contact: Optional[str] = None,
                      copyright: Optional[str] = None,
                      licenses: Optional[str] = None,
                      disclaimer: Optional[str] = None,
                      namespace_url: Optional[Mapping[str, str]] = None,
                      namespace_patterns: Optional[Mapping[str, str]] = None,
                      annotation_url: Optional[Mapping[str, str]] = None,
                      annotation_patterns: Optional[Mapping[str, str]] = None,
                      annotation_list: Optional[Mapping[str, Set[str]]] = None,
                      pmids: Optional[Iterable[Union[str, int]]] = None,
                      entrez_ids: Optional[Iterable[Union[str, int]]] = None,
                      file: Optional[TextIO] = None,
                      ) -> None:
    """Write a boilerplate BEL document, with standard document metadata, definitions.

    :param name: The unique name for this BEL document
    :param contact: The email address of the maintainer
    :param description: A description of the contents of this document
    :param authors: The authors of this document
    :param version: The version. Defaults to current date in format ``YYYYMMDD``.
    :param copyright: Copyright information about this document
    :param licenses: The license applied to this document
    :param disclaimer: The disclaimer for this document
    :param namespace_url: an optional dictionary of {str name: str URL} of namespaces
    :param namespace_patterns: An optional dictionary of {str name: str regex} namespaces
    :param annotation_url: An optional dictionary of {str name: str URL} of annotations
    :param annotation_patterns: An optional dictionary of {str name: str regex} of regex annotations
    :param annotation_list: An optional dictionary of {str name: set of names} of list annotations
    :param pmids: A list of PubMed identifiers to auto-populate with citation and abstract
    :param entrez_ids: A list of Entrez identifiers to autopopulate the gene summary as evidence
    :param file: A writable file or file-like. If None, defaults to :data:`sys.stdout`
    """
    lines = make_knowledge_header(
        name=name,
        version=version or '1.0.0',
        description=description,
        authors=authors,
        contact=contact,
        copyright=copyright,
        licenses=licenses,
        disclaimer=disclaimer,
        namespace_url=namespace_url,
        namespace_patterns=namespace_patterns,
        annotation_url=annotation_url,
        annotation_patterns=annotation_patterns,
        annotation_list=annotation_list,
    )

    for line in lines:
        print(line, file=file)

    if pmids is not None:
        for line in make_pubmed_abstract_group(pmids):
            print(line, file=file)

    if entrez_ids is not None:
        for line in make_pubmed_gene_group(entrez_ids):
            print(line, file=file)


Instruction:
Can you implement a function in Python 3 that
induces a sub - graph on the nodes that pass the given predicates.

Response:
def get_subgraph_by_node_filter(graph: BELGraph, node_predicates: NodePredicates) -> BELGraph:
    """Induce a sub-graph on the nodes that pass the given predicate(s)."""
    return get_subgraph_by_induction(graph, filter_nodes(graph, node_predicates))


Instruction:
Make a summary of the following Python 3 code
def get_subgraph_by_node_search(graph: BELGraph, query: Strings) -> BELGraph:
    nodes = search_node_names(graph, query)
    return get_subgraph_by_induction(graph, nodes)

Response: Get a sub - graph induced over all nodes matching the query string.


Instruction:
Can you create a Python 3 function that
gets the largest giant component of a graph.

Response:
def get_largest_component(graph: BELGraph) -> BELGraph:
    """Get the giant component of a graph."""
    biggest_component_nodes = max(nx.weakly_connected_components(graph), key=len)
    return subgraph(graph, biggest_component_nodes)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def random_by_nodes(graph: BELGraph, percentage: Optional[float] = None) -> BELGraph:
    percentage = percentage or 0.9

    assert 0 < percentage <= 1

    nodes = graph.nodes()
    n = int(len(nodes) * percentage)

    subnodes = random.sample(nodes, n)

    result = graph.subgraph(subnodes)

    update_node_helper(graph, result)

    return result

Response: Get a random graph by inducing over a percentage of the original nodes.


Instruction:
Can you tell what is the following Python 3 function doing
def random_by_edges(graph: BELGraph, percentage: Optional[float] = None) -> BELGraph:
    percentage = percentage or 0.9
    assert 0 < percentage <= 1

    edges = graph.edges(keys=True)
    n = int(graph.number_of_edges() * percentage)

    subedges = random.sample(edges, n)

    rv = graph.fresh_copy()

    for u, v, k in subedges:
        safe_add_edge(rv, u, v, k, graph[u][v][k])

    update_node_helper(graph, rv)

    return rv

Response: Get a random graph by keeping a certain percentage of original edges.


Instruction:
Can you write a function in Python 3 where it
shuffles the node s data dictionary.

Response:
def shuffle_node_data(graph: BELGraph, key: str, percentage: Optional[float] = None) -> BELGraph:
    """Shuffle the node's data.

    Useful for permutation testing.

    :param graph: A BEL graph
    :param key: The node data dictionary key
    :param percentage: What percentage of possible swaps to make
    """
    percentage = percentage or 0.3
    assert 0 < percentage <= 1

    n = graph.number_of_nodes()
    swaps = int(percentage * n * (n - 1) / 2)

    result: BELGraph = graph.copy()

    for _ in range(swaps):
        s, t = random.sample(result.node, 2)
        result.nodes[s][key], result.nodes[t][key] = result.nodes[t][key], result.nodes[s][key]

    return result


Instruction:
Can you tell what is the following Python 3 function doing
def shuffle_relations(graph: BELGraph, percentage: Optional[str] = None) -> BELGraph:
    percentage = percentage or 0.3
    assert 0 < percentage <= 1

    n = graph.number_of_edges()
    swaps = int(percentage * n * (n - 1) / 2)

    result: BELGraph = graph.copy()

    edges = result.edges(keys=True)

    for _ in range(swaps):
        (s1, t1, k1), (s2, t2, k2) = random.sample(edges, 2)
        result[s1][t1][k1], result[s2][t2][k2] = result[s2][t2][k2], result[s1][t1][k1]

    return result

Response:
Shuffle the relations.

    Useful for permutation testing.

    :param graph: A BEL graph
    :param percentage: What percentage of possible swaps to make


Instruction:
Given the following Python 3 function, write the documentation
def is_edge_consistent(graph, u, v):
    if not graph.has_edge(u, v):
        raise ValueError('{} does not contain an edge ({}, {})'.format(graph, u, v))

    return 0 == len(set(d[RELATION] for d in graph.edge[u][v].values()))

Response: Check if all edges between two nodes have the same relation.


Instruction:
Given the following Python 3 function, write the documentation
def all_edges_consistent(graph):
    return all(
        is_edge_consistent(graph, u, v)
        for u, v in graph.edges()
    )

Response: Return True if all edges are consistent in a BEL graph.


Instruction:
Can you write a function in Python 3 where it
rewires a graph s edges target nodes.

Response:
def rewire_targets(graph, rewiring_probability):
    """Rewire a graph's edges' target nodes.

    - For BEL graphs, assumes edge consistency (all edges between two given nodes are have the same relation)
    - Doesn't make self-edges

    :param pybel.BELGraph graph: A BEL graph
    :param float rewiring_probability: The probability of rewiring (between 0 and 1)
    :return: A rewired BEL graph
    """
    if not all_edges_consistent(graph):
        raise ValueError('{} is not consistent'.format(graph))

    result = graph.copy()
    nodes = result.nodes()

    for u, v in result.edges():
        if random.random() < rewiring_probability:
            continue

        w = random.choice(nodes)

        while w == u or result.has_edge(u, w):
            w = random.choice(nodes)

        result.add_edge(w, v)
        result.remove_edge(u, v)

    return result


Instruction:
Can you tell what is the following Python 3 function doing
def self_edge_filter(_: BELGraph, source: BaseEntity, target: BaseEntity, __: str) -> bool:
    return source == target

Response: Check if the source and target nodes are the same.


Instruction:
Implement a Python 3 function for
checking if a protein modification of source causes activity of target.

Response:
def has_protein_modification_increases_activity(graph: BELGraph,
                                                source: BaseEntity,
                                                target: BaseEntity,
                                                key: str,
                                                ) -> bool:
    """Check if pmod of source causes activity of target."""
    edge_data = graph[source][target][key]
    return has_protein_modification(graph, source) and part_has_modifier(edge_data, OBJECT, ACTIVITY)


Instruction:
Write a Python 3 function that can
check if the degradation of source causes activity of target.

Response:
def has_degradation_increases_activity(data: Dict) -> bool:
    """Check if the degradation of source causes activity of target."""
    return part_has_modifier(data, SUBJECT, DEGRADATION) and part_has_modifier(data, OBJECT, ACTIVITY)


Instruction:
Explain what the following Python 3 code does
def has_translocation_increases_activity(data: Dict) -> bool:
    return part_has_modifier(data, SUBJECT, TRANSLOCATION) and part_has_modifier(data, OBJECT, ACTIVITY)

Response: Check if the translocation of source causes activity of target.


Instruction:
Create a Python 3 function to
do the given complex contain the member?

Response:
def complex_has_member(graph: BELGraph, complex_node: ComplexAbundance, member_node: BaseEntity) -> bool:
    """Does the given complex contain the member?"""
    return any(  # TODO can't you look in the members of the complex object (if it's enumerated)
        v == member_node
        for _, v, data in graph.out_edges(complex_node, data=True)
        if data[RELATION] == HAS_COMPONENT
    )


Instruction:
Write a Python 3 script to
return True if the formation of a complex with u increases the activity of v.

Response:
def complex_increases_activity(graph: BELGraph, u: BaseEntity, v: BaseEntity, key: str) -> bool:
    """Return if the formation of a complex with u increases the activity of v."""
    return (
        isinstance(u, (ComplexAbundance, NamedComplexAbundance)) and
        complex_has_member(graph, u, v) and
        part_has_modifier(graph[u][v][key], OBJECT, ACTIVITY)
    )


Instruction:
How would you implement a function in Python 3 that
finds edges that are A - A meaning that some conditions in the edge best describe the interaction.

Response:
def find_activations(graph: BELGraph):
    """Find edges that are A - A, meaning that some conditions in the edge best describe the interaction."""
    for u, v, key, data in graph.edges(keys=True, data=True):
        if u != v:
            continue

        bel = graph.edge_to_bel(u, v, data)

        line = data.get(LINE)

        if line is None:
            continue  # this was inferred, so need to investigate another way

        elif has_protein_modification_increases_activity(graph, u, v, key):
            print(line, '- pmod changes -', bel)
            find_related(graph, v, data)

        elif has_degradation_increases_activity(data):
            print(line, '- degradation changes -', bel)
            find_related(graph, v, data)

        elif has_translocation_increases_activity(data):
            print(line, '- translocation changes -', bel)
            find_related(graph, v, data)

        elif complex_increases_activity(graph, u, v, key):
            print(line, '- complex changes - ', bel)
            find_related(graph, v, data)

        elif has_same_subject_object(graph, u, v, key):
            print(line, '- same sub/obj -', bel)

        else:
            print(line, '- *** - ', bel)


Instruction:
Here you have a function in Python 3, explain what it does
def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = itt.tee(iterable)
    next(b, None)
    return zip(a, b)

Response: s -> s0 s1 s2 s3...


Instruction:
How would you implement a function in Python 3 that
takes in a path and calculates a score for the edge that is ranked.

Response:
def rank_path(graph, path, edge_ranking=None):
    """Takes in a path (a list of nodes in the graph) and calculates a score

    :param pybel.BELGraph graph: A BEL graph
    :param list[tuple] path: A list of nodes in the path (includes terminal nodes)
    :param dict edge_ranking: A dictionary of {relationship: score}
    :return: The score for the edge
    :rtype: int
    """
    edge_ranking = default_edge_ranking if edge_ranking is None else edge_ranking

    return sum(max(edge_ranking[d[RELATION]] for d in graph.edge[u][v].values()) for u, v in pairwise(path))


Instruction:
Can you generate a brief explanation for the following Python 3 code
def find_root_in_path(graph, path_nodes):
    path_graph = graph.subgraph(path_nodes)

    # node_in_degree_tuple: list of tuples with (node,in_degree_of_node) in ascending order
    node_in_degree_tuple = sorted([(n, d) for n, d in path_graph.in_degree().items()], key=itemgetter(1))
    # node_out_degree_tuple: ordered list of tuples with (node,in_degree_of_node) in descending order
    node_out_degree_tuple = sorted([(n, d) for n, d in path_graph.out_degree().items()], key=itemgetter(1),
                                   reverse=True)

    # In case all have the same in degree it needs to be reference before
    tied_root_index = 0

    # Get index where the min in_degree stops (in case they are duplicates)
    for i in range(0, (len(node_in_degree_tuple) - 1)):
        if node_in_degree_tuple[i][1] < node_in_degree_tuple[i + 1][1]:
            tied_root_index = i
            break

    # If there are multiple nodes with minimum in_degree take the one with max out degree
    # (in case multiple have the same out degree pick one random)
    if tied_root_index != 0:
        root_tuple = max(node_out_degree_tuple[:tied_root_index], key=itemgetter(1))
    else:
        root_tuple = node_in_degree_tuple[0]

    return path_graph, root_tuple[0]

Response: Find the root node of the path in the graph.


Instruction:
Can you generate the documentation for the following Python 3 function
def summarize_edge_filter(graph: BELGraph, edge_predicates: EdgePredicates) -> None:
    passed = count_passed_edge_filter(graph, edge_predicates)
    print('{}/{} edges passed {}'.format(
        passed, graph.number_of_edges(),
        (
            ', '.join(edge_filter.__name__ for edge_filter in edge_predicates)
            if isinstance(edge_predicates, Iterable) else
            edge_predicates.__name__
        )
    ))

Response: Prints a summary of the number of edges passing a given set of filters.


Instruction:
Given the following Python 3 function, write the documentation
def build_edge_data_filter(annotations: Mapping, partial_match: bool = True) -> EdgePredicate: # noqa: D202

    @edge_predicate
    def annotation_dict_filter(data: EdgeData) -> bool:
        """A filter that matches edges with the given dictionary as a sub-dictionary."""
        return subdict_matches(data, annotations, partial_match=partial_match)

    return annotation_dict_filter

Response: Build a filter that keeps edges whose data dictionaries are super - dictionaries to the given dictionary.


Instruction:
Can you implement a function in Python 3 that
builds an edge predicate that returns True if the given PubMed identifiers are not contained in the given PubMed identifiers.

Response:
def build_pmid_exclusion_filter(pmids: Strings) -> EdgePredicate:
    """Fail for edges with citations whose references are one of the given PubMed identifiers.

    :param pmids: A PubMed identifier or list of PubMed identifiers to filter against
    """
    if isinstance(pmids, str):
        @edge_predicate
        def pmid_exclusion_filter(data: EdgeData) -> bool:
            """Fail for edges with PubMed citations matching the contained PubMed identifier.

            :return: If the edge has a PubMed citation with the contained PubMed identifier
            """
            return has_pubmed(data) and data[CITATION][CITATION_REFERENCE] != pmids

    elif isinstance(pmids, Iterable):
        pmids = set(pmids)

        @edge_predicate
        def pmid_exclusion_filter(data: EdgeData) -> bool:
            """Pass for edges with PubMed citations matching one of the contained PubMed identifiers.

            :return: If the edge has a PubMed citation with one of the contained PubMed identifiers
            """
            return has_pubmed(data) and data[CITATION][CITATION_REFERENCE] not in pmids

    else:
        raise TypeError

    return pmid_exclusion_filter


Instruction:
Here you have a function in Python 3, explain what it does
def node_has_namespace(node: BaseEntity, namespace: str) -> bool:
    ns = node.get(NAMESPACE)
    return ns is not None and ns == namespace

Response: Pass for nodes that have the given namespace.


Instruction:
Implement a Python 3 function for
passing for nodes that have one of the given namespaces.

Response:
def node_has_namespaces(node: BaseEntity, namespaces: Set[str]) -> bool:
    """Pass for nodes that have one of the given namespaces."""
    ns = node.get(NAMESPACE)
    return ns is not None and ns in namespaces


Instruction:
Can you tell what is the following Python 3 function doing
def build_source_namespace_filter(namespaces: Strings) -> EdgePredicate:
    if isinstance(namespaces, str):
        def source_namespace_filter(_, u: BaseEntity, __, ___) -> bool:
            return node_has_namespace(u, namespaces)

    elif isinstance(namespaces, Iterable):
        namespaces = set(namespaces)

        def source_namespace_filter(_, u: BaseEntity, __, ___) -> bool:
            return node_has_namespaces(u, namespaces)

    else:
        raise TypeError

    return source_namespace_filter

Response: Build a filter function that returns true if the source nodes have the given namespace or one of the given namespaces.


Instruction:
Here you have a function in Python 3, explain what it does
def build_target_namespace_filter(namespaces: Strings) -> EdgePredicate:
    if isinstance(namespaces, str):
        def target_namespace_filter(_, __, v: BaseEntity, ___) -> bool:
            return node_has_namespace(v, namespaces)

    elif isinstance(namespaces, Iterable):
        namespaces = set(namespaces)

        def target_namespace_filter(_, __, v: BaseEntity, ___) -> bool:
            return node_has_namespaces(v, namespaces)

    else:
        raise TypeError

    return target_namespace_filter

Response: Build a filter that only passes for edges whose target nodes have the given namespace or one of the given namespaces.


Instruction:
How would you explain what the following Python 3 function does
def search_node_namespace_names(graph, query, namespace):
    node_predicates = [
        namespace_inclusion_builder(namespace),
        build_node_name_search(query)
    ]

    return filter_nodes(graph, node_predicates)

Response: Search for nodes with the given namespace and whose names containing a given string ( s ).


Instruction:
Write a Python 3 script for
assigning if a value is greater than or less than a cutoff.

Response:
def get_cutoff(value: float, cutoff: Optional[float] = None) -> int:
    """Assign if a value is greater than or less than a cutoff."""
    cutoff = cutoff if cutoff is not None else 0

    if value > cutoff:
        return 1

    if value < (-1 * cutoff):
        return - 1

    return 0


Instruction:
Write a Python 3 script for
helping calculate the concordance tree for a given node data dictionary key

Response:
def calculate_concordance_helper(graph: BELGraph,
                                 key: str,
                                 cutoff: Optional[float] = None,
                                 ) -> Tuple[int, int, int, int]:
    """Help calculate network-wide concordance

    Assumes data already annotated with given key

    :param graph: A BEL graph
    :param key: The node data dictionary key storing the logFC
    :param cutoff: The optional logFC cutoff for significance
    """
    scores = defaultdict(int)

    for u, v, k, d in graph.edges(keys=True, data=True):
        c = edge_concords(graph, u, v, k, d, key, cutoff=cutoff)
        scores[c] += 1

    return (
        scores[Concordance.correct],
        scores[Concordance.incorrect],
        scores[Concordance.ambiguous],
        scores[Concordance.unassigned],
    )


Instruction:
Implement a function in Python 3 to
calculate the concordance of a node in a network.

Response:
def calculate_concordance(graph: BELGraph, key: str, cutoff: Optional[float] = None,
                          use_ambiguous: bool = False) -> float:
    """Calculates network-wide concordance.

    Assumes data already annotated with given key

    :param graph: A BEL graph
    :param key: The node data dictionary key storing the logFC
    :param cutoff: The optional logFC cutoff for significance
    :param use_ambiguous: Compare to ambiguous edges as well
    """
    correct, incorrect, ambiguous, _ = calculate_concordance_helper(graph, key, cutoff=cutoff)

    try:
        return correct / (correct + incorrect + (ambiguous if use_ambiguous else 0))
    except ZeroDivisionError:
        return -1.0


Instruction:
Write a Python 3 function that can
calculate the one - sided probability of getting a value more extreme than the distribution.

Response:
def one_sided(value: float, distribution: List[float]) -> float:
    """Calculate the one-sided probability of getting a value more extreme than the distribution."""
    assert distribution
    return sum(value < element for element in distribution) / len(distribution)


Instruction:
Can you tell what is the following Python 3 function doing
def calculate_concordance_probability(graph: BELGraph,
                                      key: str,
                                      cutoff: Optional[float] = None,
                                      permutations: Optional[int] = None,
                                      percentage: Optional[float] = None,
                                      use_ambiguous: bool = False,
                                      permute_type: str = 'shuffle_node_data',
                                      ) -> Tuple[float, List[float], float]:
    """Calculates a graph's concordance as well as its statistical probability.

    :param graph: A BEL graph
    :param str key: The node data dictionary key storing the logFC
    :param float cutoff: The optional logFC cutoff for significance
    :param int permutations: The number of random permutations to test. Defaults to 500
    :param float percentage: The percentage of the graph's edges to maintain. Defaults to 0.9
    :param bool use_ambiguous: Compare to ambiguous edges as well
    :returns: A triple of the concordance score, the null distribution, and the p-value.
    """
    if permute_type == 'random_by_edges':
        permute_func = partial(random_by_edges, percentage=percentage)
    elif permute_type == 'shuffle_node_data':
        permute_func = partial(shuffle_node_data, key=key, percentage=percentage)
    elif permute_type == 'shuffle_relations':
        permute_func = partial(shuffle_relations, percentage=percentage)
    else:
        raise ValueError('Invalid permute_type: {}'.format(permute_type))

    graph: BELGraph = graph.copy()
    collapse_to_genes(graph)
    collapse_all_variants(graph)

    score = calculate_concordance(graph, key, cutoff=cutoff)

    distribution = []

    for _ in range(permutations or 500):
        permuted_graph = permute_func(graph)
        permuted_graph_scores = calculate_concordance(permuted_graph, key, cutoff=cutoff, use_ambiguous=use_ambiguous)
        distribution.append(permuted_graph_scores)

    return score, distribution, one_sided(score, distribution)

Response: Calculates the concordance probability of a graph.


Instruction:
Explain what the following Python 3 code does
def calculate_concordance_by_annotation(graph, annotation, key, cutoff=None):
    return {
        value: calculate_concordance(subgraph, key, cutoff=cutoff)
        for value, subgraph in get_subgraphs_by_annotation(graph, annotation).items()
    }

Response: Calculates the concordance scores for each stratified graph based on the given annotation and returns the concordance scores for each stratified graph.


Instruction:
Write a Python 3 script for
calculating the concordance probability of each node in the graph with the given annotation.

Response:
def calculate_concordance_probability_by_annotation(graph, annotation, key, cutoff=None, permutations=None,
                                                    percentage=None,
                                                    use_ambiguous=False):
    """Returns the results of concordance analysis on each subgraph, stratified by the given annotation.

    :param pybel.BELGraph graph: A BEL graph
    :param str annotation: The annotation to group by.
    :param str key: The node data dictionary key storing the logFC
    :param float cutoff: The optional logFC cutoff for significance
    :param int permutations: The number of random permutations to test. Defaults to 500
    :param float percentage: The percentage of the graph's edges to maintain. Defaults to 0.9
    :param bool use_ambiguous: Compare to ambiguous edges as well
    :rtype: dict[str,tuple]
    """
    result = [
        (value, calculate_concordance_probability(
            subgraph,
            key,
            cutoff=cutoff,
            permutations=permutations,
            percentage=percentage,
            use_ambiguous=use_ambiguous,
        ))
        for value, subgraph in get_subgraphs_by_annotation(graph, annotation).items()
    ]

    return dict(result)


Instruction:
Make a summary of the following Python 3 code
def _get_drug_target_interactions(manager: Optional['bio2bel_drugbank.manager'] = None) -> Mapping[str, List[str]]:
    if manager is None:
        import bio2bel_drugbank
        manager = bio2bel_drugbank.Manager()

    if not manager.is_populated():
        manager.populate()

    return manager.get_drug_to_hgnc_symbols()

Response: Get a mapping from drugs to their list of gene names.


Instruction:
How would you code a function in Python 3 to
run EpiCom analysis on many graphs.

Response:
def multi_run_epicom(graphs: Iterable[BELGraph], path: Union[None, str, TextIO]) -> None:
    """Run EpiCom analysis on many graphs."""
    if isinstance(path, str):
        with open(path, 'w') as file:
            _multi_run_helper_file_wrapper(graphs, file)

    else:
        _multi_run_helper_file_wrapper(graphs, path)


Instruction:
Can you implement a function in Python 3 that
converts the Alzheimer s and Parkinson s disease NeuroMMSig excel sheets to BEL.

Response:
def main():
    """Convert the Alzheimer's and Parkinson's disease NeuroMMSig excel sheets to BEL."""
    logging.basicConfig(level=logging.INFO)
    log.setLevel(logging.INFO)

    bms_base = get_bms_base()
    neurommsig_base = get_neurommsig_base()
    neurommsig_excel_dir = os.path.join(neurommsig_base, 'resources', 'excels', 'neurommsig')

    nift_values = get_nift_values()

    log.info('Starting Alzheimers')

    ad_path = os.path.join(neurommsig_excel_dir, 'alzheimers', 'alzheimers.xlsx')
    ad_df = preprocess(ad_path)
    with open(os.path.join(bms_base, 'aetionomy', 'alzheimers', 'neurommsigdb_ad.bel'), 'w') as ad_file:
        write_neurommsig_bel(ad_file, ad_df, mesh_alzheimer, nift_values)

    log.info('Starting Parkinsons')

    pd_path = os.path.join(neurommsig_excel_dir, 'parkinsons', 'parkinsons.xlsx')
    pd_df = preprocess(pd_path)
    with open(os.path.join(bms_base, 'aetionomy', 'parkinsons', 'neurommsigdb_pd.bel'), 'w') as pd_file:
        write_neurommsig_bel(pd_file, pd_df, mesh_parkinson, nift_values)


Instruction:
Given the following Python 3 function, write the documentation
def remove_inconsistent_edges(graph: BELGraph) -> None:
    for u, v in get_inconsistent_edges(graph):
        edges = [(u, v, k) for k in graph[u][v]]
        graph.remove_edges_from(edges)

Response: Remove all edges between node pairs with inconsistent edges.


Instruction:
How would you explain what the following Python 3 function does
def get_walks_exhaustive(graph, node, length):
    if 0 == length:
        return (node,),

    return tuple(
        (node, key) + path
        for neighbor in graph.edge[node]
        for path in get_walks_exhaustive(graph, neighbor, length - 1)
        if node not in path
        for key in graph.edge[node][neighbor]
    )

Response: Returns a list of paths that are not in the same node as the given node.


Instruction:
Explain what the following Python 3 code does
def match_simple_metapath(graph, node, simple_metapath):
    if 0 == len(simple_metapath):
        yield node,

    else:
        for neighbor in graph.edges[node]:
            if graph.nodes[neighbor][FUNCTION] == simple_metapath[0]:
                for path in match_simple_metapath(graph, neighbor, simple_metapath[1:]):
                    if node not in path:
                        yield (node,) + path

Response: Matches a simple metapath starting at the given node and returns an iterable over the nodes that match the given simple metapath.


Instruction:
How would you explain what the following Python 3 function does
def build_database(manager: pybel.Manager, annotation_url: Optional[str] = None) -> None:
    annotation_url = annotation_url or NEUROMMSIG_DEFAULT_URL

    annotation = manager.get_namespace_by_url(annotation_url)

    if annotation is None:
        raise RuntimeError('no graphs in database with given annotation')

    networks = get_networks_using_annotation(manager, annotation)

    dtis = ...

    for network in networks:
        graph = network.as_bel()

        scores = epicom_on_graph(graph, dtis)

        for (drug_name, subgraph_name), score in scores.items():
            drug_model = get_drug_model(manager, drug_name)
            subgraph_model = manager.get_annotation_entry(annotation_url, subgraph_name)

            score_model = Score(
                network=network,
                annotation=subgraph_model,
                drug=drug_model,
                score=score
            )

            manager.session.add(score_model)

    t = time.time()
    logger.info('committing scores')
    manager.session.commit()
    logger.info('committed scores in %.2f seconds', time.time() - t)

Response: Build a database of scores for NeuroMMSig annotated graphs.