Instruction:
How would you implement a function in Python 3 that
merges timeseries into a new : class:`~. TimeSeries instance.

Response:
def ts_merge(series):
    '''Merge timeseries into a new :class:`~.TimeSeries` instance.

    :parameter series: an iterable over :class:`~.TimeSeries`.
    '''
    series = iter(series)
    ts = next(series)
    return ts.merge(series)


Instruction:
How would you implement a function in Python 3 that
returns the algorithm for the given operation and name.

Response:
def getalgo(self, operation, name):
        '''Return the algorithm for *operation* named *name*'''
        if operation not in self._algorithms:
            raise NotAvailable('{0} not registered.'.format(operation))
        oper = self._algorithms[operation]
        try:
            return oper[name]
        except KeyError:
            raise NotAvailable('{0} algorithm {1} not registered.'
                               .format(operation, name))


Instruction:
Can you generate the documentation for the following Python 3 function
def dates(self, desc=None):
        '''Returns an iterable over ``datetime.date`` instances
in the timeseries.'''
        c = self.dateinverse
        for key in self.keys(desc=desc):
            yield c(key)

Response:
Returns an iterable over datetime. date instances
in the timeseries.


Instruction:
Can you tell what is the following Python 3 function doing
def items(self, desc=None, start_value=None, shift_by=None):
        '''Returns a python ``generator`` which can be used to iterate over
        :func:`dynts.TimeSeries.dates` and :func:`dynts.TimeSeries.values`
        returning a two dimensional
        tuple ``(date,value)`` in each iteration.
        Similar to the python dictionary items
        function.

        :parameter desc: if ``True`` the iteratioon starts from the more
            recent data and proceeds backwards.
        :parameter shift_by: optional parallel shift in values.
        :parameter start_value: optional start value of timeseries.
        '''
        if self:
            if shift_by is None and start_value is not None:
                for cross in self.values():
                    missings = 0
                    if shift_by is None:
                        shift_by = []
                        for v in cross:
                            shift_by.append(start_value - v)
                            if v != v:
                                missings += 1
                    else:
                        for j in range(len(shift_by)):
                            s = shift_by[j]
                            v = cross[j]
                            if s != s:
                                if v == v:
                                    shift_by[j] = start_value - v
                                else:
                                    missings += 1
                    if not missings:
                        break
            if shift_by:
                for d, v in zip(self.dates(desc=desc), self.values(desc=desc)):
                    yield d, v + shift_by
            else:
                for d, v in zip(self.dates(desc=desc), self.values(desc=desc)):
                    yield d, v

Response: Returns a generator which can be used to iterate over the items of the time series.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def named_series(self, ordering=None):
        '''Generator of tuples with name and serie data.'''
        series = self.series()
        if ordering:
            series = list(series)
            todo = dict(((n, idx) for idx, n in enumerate(self.names())))
            for name in ordering:
                if name in todo:
                    idx = todo.pop(name)
                    yield name, series[idx]
            for name in todo:
                idx = todo[name]
                yield name, series[idx]
        else:
            for name_serie in zip(self.names(), series):
                yield name_serie

Response: Generator of tuples with name and serie data.


Instruction:
Implement a function in Python 3 to
create a clone of timeseries

Response:
def clone(self, date=None, data=None, name=None):
        '''Create a clone of timeseries'''
        name = name or self.name
        data = data if data is not None else self.values()
        ts = self.__class__(name)
        ts._dtype = self._dtype
        if date is None:
            # dates not provided
            ts.make(self.keys(), data, raw=True)
        else:
            ts.make(date, data)
        return ts


Instruction:
Given the following Python 3 function, write the documentation
def reduce(self, size, method='simple', **kwargs):
        '''Trim :class:`Timeseries` to a new *size* using the algorithm
*method*. If *size* is greater or equal than len(self) it does nothing.'''
        if size >= len(self):
            return self
        return self.getalgo('reduce', method)(self, size, **kwargs)

Response:
Trim the time series to a new size using the algorithm
 * method*.


Instruction:
Make a summary of the following Python 3 code
def clean(self, algorithm=None):
        '''Create a new :class:`TimeSeries` with missing data removed or
replaced by the *algorithm* provided'''
        # all dates
        original_dates = list(self.dates())
        series = []
        all_dates = set()
        for serie in self.series():
            dstart, dend, vend = None, None, None
            new_dates = []
            new_values = []
            missings = []
            values = {}
            for d, v in zip(original_dates, serie):
                if v == v:
                    if dstart is None:
                        dstart = d
                    if missings:
                        for dx, vx in algorithm(dend, vend, d, v, missings):
                            new_dates.append(dx)
                            new_values.append(vx)
                        missings = []
                    dend = d
                    vend = v
                    values[d] = v
                elif dstart is not None and algorithm:
                    missings.append((dt, v))
            if missings:
                for dx, vx in algorithm(dend, vend, None, None, missings):
                    new_dates.append(dx)
                    new_values.append(vx)
                    dend = dx
            series.append((dstart, dend, values))
            all_dates = all_dates.union(values)
        cdate = []
        cdata = []
        for dt in sorted(all_dates):
            cross = []
            for start, end, values in series:
                if start is None or (dt >= start and dt <= end):
                    value = values.get(dt)
                    if value is None:
                        cross = None
                        break
                else:
                    value = nan
                cross.append(value)
            if cross:
                cdate.append(dt)
                cdata.append(cross)
        return self.clone(date=cdate, data=cdata)

Response: Create a new : class:`TimeSeries` with missing data removed or replaced by the algorithm provided.


Instruction:
Can you implement a function in Python 3 that
checks if the timeseries is consistent

Response:
def isconsistent(self):
        '''Check if the timeseries is consistent'''
        for dt1, dt0 in laggeddates(self):
            if dt1 <= dt0:
                return False
        return True


Instruction:
Write a Python 3 function that can
calculate variance of timeseries. Return a vector containing the variance of each series in the timeseries. Default 0.

Response:
def var(self, ddof=0):
        '''Calculate variance of timeseries. Return a vector containing
the variances of each series in the timeseries.

:parameter ddof: delta degree of freedom, the divisor used in the calculation
                 is given by ``N - ddof`` where ``N`` represents the length
                 of timeseries. Default ``0``.

.. math::

    var = \\frac{\\sum_i^N (x - \\mu)^2}{N-ddof}
    '''
        N = len(self)
        if N:
            v = self.values()
            mu = sum(v)
            return (sum(v*v) - mu*mu/N)/(N-ddof)
        else:
            return None


Instruction:
Here you have a function in Python 3, explain what it does
def sd(self):
        '''Calculate standard deviation of timeseries'''
        v = self.var()
        if len(v):
            return np.sqrt(v)
        else:
            return None

Response: Calculate standard deviation of timeseries


Instruction:
How would you code a function in Python 3 to
apply function func to the timeseries.

Response:
def apply(self, func, window=None, bycolumn=True, align=None, **kwargs):
        '''Apply function ``func`` to the timeseries.

        :keyword func: string indicating function to apply
        :keyword window: Rolling window, If not defined ``func`` is applied on
            the whole dataset. Default ``None``.
        :keyword bycolumn: If ``True``, function ``func`` is applied on
            each column separately. Default ``True``.
        :keyword align: string specifying whether the index of the result
            should be ``left`` or ``right`` (default) or ``centered``
            aligned compared to the rolling window of observations.
        :keyword kwargs: dictionary of auxiliary parameters used by
            function ``func``.
        '''
        N = len(self)
        window = window or N
        self.precondition(window <= N and window > 0, OutOfBound)
        return self._rollapply(func,
                               window=window,
                               align=align or self.default_align,
                               bycolumn=bycolumn,
                               **kwargs)


Instruction:
Can you generate the documentation for the following Python 3 function
def unwind(self, values, backend, **kwargs):
        '''Unwind expression by applying *values* to the abstract nodes.

        The ``kwargs`` dictionary can contain data which can be used
        to override values
        '''
        if not hasattr(self, "_unwind_value"):
            self._unwind_value = self._unwind(values, backend, **kwargs)
        return self._unwind_value

Response: Unwind expression by applying values * to the abstract nodes.


Instruction:
Write a Python 3 script to
loop over children a remove duplicate entries. is a list of duplicate entries. Returns a list of removed entries.

Response:
def removeduplicates(self, entries = None):
        '''
        Loop over children a remove duplicate entries.
        @return - a list of removed entries
        '''
        removed = []
        if entries == None:
            entries = {}
        new_children = []
        for c in self.children:
            cs = str(c)
            cp = entries.get(cs,None)
            if cp:
                new_children.append(cp)
                removed.append(c)
            else:
                dups = c.removeduplicates(entries)
                if dups:
                    removed.extend(dups)
                entries[cs] = c
                new_children.append(c)
        self.children = new_children
        return removed


Instruction:
Can you write a function in Python 3 where it
converts a string or html file to a Markdown table string.

Response:
def html2md(html_string):
    """
    Convert a string or html file to a markdown table string.

    Parameters
    ----------
    html_string : str
        Either the html string, or the filepath to the html

    Returns
    -------
    str
        The html table converted to a Markdown table

    Notes
    -----
    This function requires BeautifulSoup_ to work.

    Example
    -------
    >>> html_text = '''
    ...

    ...

    ...
    ...             Header 1
    ...
    ...
    ...             Header 2
    ...
    ...
    ...             Header 3
    ...
    ...

    ...
    ...
This is a paragraph

    ...
    ...
    ...             Just text
    ...
    ...
    ...             Hot dog
    ...
    ...

    ...

    ... '''
    >>> import dashtable
    >>> print(dashtable.html2md(html_text))
    |      Header 1       | Header 2  | Header 3 |
    |---------------------|-----------|----------|
    | This is a paragraph | Just text | Hot dog  |

    .. _BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/
    """

    if os.path.isfile(html_string):
        file = open(html_string, 'r', encoding='utf-8')
        lines = file.readlines()
        file.close()
        html_string = ''.join(lines)

    table_data, spans, use_headers = html2data(html_string)
    if table_data == '':
        return ''

    return data2md(table_data)


Instruction:
Can you tell what is the following Python 3 function doing
def table_cells_2_spans(table, spans):
    new_spans = []
    for row in range(len(table)):
        for column in range(len(table[row])):
            span = get_span(spans, row, column)

            if not span:
                new_spans.append([[row, column]])

    new_spans.extend(spans)
    new_spans = list(sorted(new_spans))

    return new_spans

Response: Converts the table data to a list of spans for consistency.


Instruction:
Explain what the following Python 3 code does
def get_html_column_count(html_string):
    try:
        from bs4 import BeautifulSoup
    except ImportError:
        print("ERROR: You must have BeautifulSoup to use html2data")
        return

    soup = BeautifulSoup(html_string, 'html.parser')
    table = soup.find('table')
    if not table:
        return 0

    column_counts = []
    trs = table.findAll('tr')
    if len(trs) == 0:
        return 0

    for tr in range(len(trs)):
        if tr == 0:
            tds = trs[tr].findAll('th')
            if len(tds) == 0:
                tds = trs[tr].findAll('td')
        else:
            tds = trs[tr].findAll('td')

        count = 0
        for td in tds:
            if td.has_attr('colspan'):
                count += int(td['colspan'])
            else:
                count += 1

        column_counts.append(count)

    return max(column_counts)

Response: Returns the number of columns in an html table.


Instruction:
Create a Python 3 function for
adding cushions to start and end of each string in a list of lists.

Response:
def add_cushions(table):
    """
    Add space to start and end of each string in a list of lists

    Parameters
    ----------
    table : list of lists of str
        A table of rows of strings. For example::

            [
                ['dog', 'cat', 'bicycle'],
                ['mouse', trumpet', '']
            ]

    Returns
    -------
    table : list of lists of str

    Note
    ----
    Each cell in an rst grid table should to have a cushion of at least
    one space on each side of the string it contains. For example::

        +-----+-------+
        | foo | bar   |
        +-----+-------+
        | cat | steve |
        +-----+-------+

    is better than::

        +-----+---+
        |foo| bar |
        +-----+---+
        |cat|steve|
        +-----+---+
    """
    for row in range(len(table)):
        for column in range(len(table[row])):
            lines = table[row][column].split("\n")

            for i in range(len(lines)):
                if not lines[i] == "":
                    lines[i] = " " + lines[i].rstrip() + " "

            table[row][column] = "\n".join(lines)

    return table


Instruction:
Here you have a function in Python 3, explain what it does
def rollsingle(self, func, window=20, name=None, fallback=False,
               align='right', **kwargs):
    '''Efficient rolling window calculation for min, max type functions
    '''
    rname = 'roll_{0}'.format(func)
    if fallback:
        rfunc = getattr(lib.fallback, rname)
    else:
        rfunc = getattr(lib, rname, None)
        if not rfunc:
            rfunc = getattr(lib.fallback, rname)
    data = np.array([list(rfunc(serie, window)) for serie in self.series()])
    name = name or self.makename(func, window=window)
    dates = asarray(self.dates())
    desc = settings.desc
    if (align == 'right' and not desc) or desc:
        dates = dates[window-1:]
    else:
        dates = dates[:-window+1]
    return self.clone(dates, data.transpose(), name=name)

Response: Efficient rolling window calculation for min max type functions


Instruction:
Can you generate a brief explanation for the following Python 3 code
def find_ge(self, dt):
        '''Building block of all searches. Find the index
corresponding to the leftmost value greater or equal to *dt*.
If *dt* is greater than the
:func:`dynts.TimeSeries.end` a :class:`dynts.exceptions.RightOutOfBound`
exception will raise.

*dt* must be a python datetime.date instance.'''
        i = bisect_left(self.dates, dt)
        if i != len(self.dates):
            return i
        raise RightOutOfBound

Response:
Build a block of all searches. Find the index
corresponding to the leftmost value greater or equal to dt*. Raise RightOutOfBound if there are no more entries.


Instruction:
Implement a function in Python 3 to
find the index corresponding to the rightmost value less than or equal to dt*.

Response:
def find_le(self, dt):
        '''Find the index corresponding to the rightmost
value less than or equal to *dt*.
If *dt* is less than :func:`dynts.TimeSeries.end`
a :class:`dynts.exceptions.LeftOutOfBound`
exception will raise.

*dt* must be a python datetime.date instance.'''
        i = bisect_right(self.dates, dt)
        if i:
            return i-1
        raise LeftOutOfBound


Instruction:
Here you have a function in Python 3, explain what it does
def read_newick(newick, root_node=None, format=0):

    ## check newick type as a string or filepath, Toytree parses urls to str's
    if isinstance(newick, six.string_types):
        if os.path.exists(newick):
            if newick.endswith('.gz'):
                import gzip
                nw = gzip.open(newick).read()
            else:
                nw = open(newick, 'rU').read()
        else:
            nw = newick

        ## get re matcher for testing newick formats
        matcher = compile_matchers(formatcode=format)
        nw = nw.strip()
        if not nw.startswith('(') and nw.endswith(';'):
            return _read_node_data(nw[:-1], root_node, "single", matcher, format)

        elif not nw.startswith('(') or not nw.endswith(';'):
            raise NewickError('Unexisting tree file or Malformed newick tree structure.')
        else:
            return _read_newick_from_string(nw, root_node, matcher, format)
    else:
        raise NewickError("'newick' argument must be either a filename or a newick string.")

Response: Reads a newick tree from either a string or a file and returns a TreeNode object.


Instruction:
Here you have a function in Python 3, explain what it does
def _read_newick_from_string(nw, root_node, matcher, formatcode):
    if nw.count('(') != nw.count(')'):
        raise NewickError('Parentheses do not match. Broken tree structure?')

    # white spaces and separators are removed
    nw = re.sub("[\n\r\t]+", "", nw)

    current_parent = None
    # Each chunk represents the content of a parent node, and it could contain
    # leaves and closing parentheses.
    # We may find:
    # leaf, ..., leaf,
    # leaf, ..., leaf))),
    # leaf)), leaf, leaf))
    # leaf))
    # ) only if formatcode == 100

    for chunk in nw.split("(")[1:]:
        # If no node has been created so far, this is the root, so use the node.
        current_parent = root_node if current_parent is None else current_parent.add_child()

        subchunks = [ch.strip() for ch in chunk.split(",")]
        # We should expect that the chunk finished with a comma (if next chunk
        # is an internal sister node) or a subchunk containing closing parenthesis until the end of the tree.
        #[leaf, leaf, '']
        #[leaf, leaf, ')))', leaf, leaf, '']
        #[leaf, leaf, ')))', leaf, leaf, '']
        #[leaf, leaf, ')))', leaf), leaf, 'leaf);']
        if subchunks[-1] != '' and not subchunks[-1].endswith(';'):
            raise NewickError('Broken newick structure at: %s' %chunk)

        # lets process the subchunks. Every closing parenthesis will close a
        # node and go up one level.
        for i, leaf in enumerate(subchunks):
            if leaf.strip() == '' and i == len(subchunks) - 1:
                continue # "blah blah ,( blah blah"
            closing_nodes = leaf.split(")")

            # first part after splitting by ) always contain leaf info
            _read_node_data(closing_nodes[0], current_parent, "leaf", matcher, formatcode)

            # next contain closing nodes and data about the internal nodes.
            if len(closing_nodes)>1:
                for closing_internal in closing_nodes[1:]:
                    closing_internal =  closing_internal.rstrip(";")
                    # read internal node data and go up one level
                    _read_node_data(closing_internal, current_parent, "internal", matcher, formatcode)
                    current_parent = current_parent.up
    return root_node

Response: Reads a string in the New Hampshire format.


Instruction:
Make a summary of the following Python 3 code
def _parse_extra_features(node, NHX_string):
    NHX_string = NHX_string.replace("[&&NHX:", "")
    NHX_string = NHX_string.replace("]", "")
    for field in NHX_string.split(":"):
        try:
            pname, pvalue = field.split("=")
        except ValueError as e:
            raise NewickError('Invalid NHX format %s' %field)
        node.add_feature(pname, pvalue)

Response: Reads node s extra data form its NHX string.


Instruction:
Implement a Python 3 function for
compiling the newick string against format types? and makes a re. compile

Response:
def compile_matchers(formatcode):
    """
    Tests newick string against format types? and makes a re.compile
    """
    matchers = {}
    for node_type in ["leaf", "single", "internal"]:
        if node_type == "leaf" or node_type == "single":
            container1 = NW_FORMAT[formatcode][0][0]
            container2 = NW_FORMAT[formatcode][1][0]
            converterFn1 = NW_FORMAT[formatcode][0][1]
            converterFn2 = NW_FORMAT[formatcode][1][1]
            flexible1 = NW_FORMAT[formatcode][0][2]
            flexible2 = NW_FORMAT[formatcode][1][2]
        else:
            container1 = NW_FORMAT[formatcode][2][0]
            container2 = NW_FORMAT[formatcode][3][0]
            converterFn1 = NW_FORMAT[formatcode][2][1]
            converterFn2 = NW_FORMAT[formatcode][3][1]
            flexible1 = NW_FORMAT[formatcode][2][2]
            flexible2 = NW_FORMAT[formatcode][3][2]

        if converterFn1 == str:
            FIRST_MATCH = "("+_NAME_RE+")"
        elif converterFn1 == float:
            FIRST_MATCH = "("+_FLOAT_RE+")"
        elif converterFn1 is None:
            FIRST_MATCH = '()'

        if converterFn2 == str:
            SECOND_MATCH = "(:"+_NAME_RE+")"
        elif converterFn2 == float:
            SECOND_MATCH = "(:"+_FLOAT_RE+")"
        elif converterFn2 is None:
            SECOND_MATCH = '()'

        if flexible1 and node_type != 'leaf':
            FIRST_MATCH += "?"
        if flexible2:
            SECOND_MATCH += "?"

        matcher_str= '^\s*%s\s*%s\s*(%s)?\s*$' % (FIRST_MATCH, SECOND_MATCH, _NHX_RE)
        compiled_matcher = re.compile(matcher_str)
        matchers[node_type] = [container1, container2, converterFn1, converterFn2, compiled_matcher]

    return matchers


Instruction:
Can you generate the documentation for the following Python 3 function
def _read_node_data(subnw, current_node, node_type, matcher, formatcode):

    if node_type == "leaf" or node_type == "single":
        if node_type == "leaf":
            node = current_node.add_child()
        else:
            node = current_node
    else:
        node = current_node

    subnw = subnw.strip()

    if not subnw and node_type == 'leaf' and formatcode != 100:
        raise NewickError('Empty leaf node found')
    elif not subnw:
        return

    container1, container2, converterFn1, converterFn2, compiled_matcher = matcher[node_type]
    data = re.match(compiled_matcher, subnw)
    if data:
        data = data.groups()
        # This prevents ignoring errors even in flexible nodes:
        if subnw and data[0] is None and data[1] is None and data[2] is None:
            raise NewickError("Unexpected newick format '%s'" %subnw)

        if data[0] is not None and data[0] != '':
            node.add_feature(container1, converterFn1(data[0].strip()))

        if data[1] is not None and data[1] != '':
            node.add_feature(container2, converterFn2(data[1][1:].strip()))

        if data[2] is not None \
                and data[2].startswith("[&&NHX"):
            _parse_extra_features(node, data[2])
    else:
        raise NewickError("Unexpected newick format '%s' " %subnw[0:50])
    return

Response: Reads a leaf node from a subpart of the original newicktree


Instruction:
Write a Python 3 script to
write a tree structure and returns its NHX representation.

Response:
def write_newick(rootnode,
	features=None,
	format=1,
	format_root_node=True,
    is_leaf_fn=None,
    dist_formatter=None,
    support_formatter=None,
    name_formatter=None):
    """
    Iteratively export a tree structure and returns its NHX
    representation.
    """
    newick = []
    leaf = is_leaf_fn if is_leaf_fn else lambda n: not bool(n.children)
    for postorder, node in rootnode.iter_prepostorder(is_leaf_fn=is_leaf_fn):
        if postorder:
            newick.append(")")
            if node.up is not None or format_root_node:
                newick.append(format_node(node, "internal", format,
                                          dist_formatter=dist_formatter,
                                          support_formatter=support_formatter,
                                          name_formatter=name_formatter))
                newick.append(_get_features_string(node, features))
        else:
            if node is not rootnode and node != node.up.children[0]:
                newick.append(",")

            if leaf(node):
                safe_name = re.sub("["+_ILEGAL_NEWICK_CHARS+"]", "_", \
                               str(getattr(node, "name")))
                newick.append(format_node(node, "leaf", format,
                              dist_formatter=dist_formatter,
                              support_formatter=support_formatter,
                              name_formatter=name_formatter))
                newick.append(_get_features_string(node, features))
            else:
                newick.append("(")

    newick.append(";")
    return ''.join(newick)


Instruction:
Here you have a function in Python 3, explain what it does
def _get_features_string(self, features=None):
    string = ""
    if features is None:
        features = []
    elif features == []:
        features = self.features

    for pr in features:
        if hasattr(self, pr):
            raw = getattr(self, pr)
            if type(raw) in ITERABLE_TYPES:
                raw = '|'.join([str(i) for i in raw])
            elif type(raw) == dict:
                raw = '|'.join(
                    map(lambda x,y: "%s-%s" %(x, y), six.iteritems(raw)))
            elif type(raw) == str:
                pass
            else:
                raw = str(raw)

            value = re.sub("["+_ILEGAL_NEWICK_CHARS+"]", "_", \
                             raw)
            if string != "":
                string +=":"
            string +="%s=%s"  %(pr, str(value))
    if string != "":
        string = "[&&NHX:"+string+"]"

    return string

Response: Generates the extended newick string NHX with extra data about a node.


Instruction:
Explain what the following Python 3 code does
def get_column_width(column, table):
    width = 3

    for row in range(len(table)):
        cell_width = len(table[row][column])
        if cell_width > width:
            width = cell_width

    return width

Response: Get the character width of a column in a table.


Instruction:
Write a Python 3 script for
returning a simple Text Mark object

Response:
def get_text_mark(ttree):
    """ makes a simple Text Mark object"""

    if ttree._orient in ["right"]:
        angle = 0.
        ypos = ttree.verts[-1*len(ttree.tree):, 1]
        if ttree._kwargs["tip_labels_align"]:
            xpos = [ttree.verts[:, 0].max()] * len(ttree.tree)
            start = xpos
            finish = ttree.verts[-1*len(ttree.tree):, 0]
            align_edges = np.array([(i, i+len(xpos)) for i in range(len(xpos))])
            align_verts = np.array(zip(start, ypos) + zip(finish, ypos))
        else:
            xpos = ttree.verts[-1*len(ttree.tree):, 0]

    elif ttree._orient in ['down']:
        angle = -90.
        xpos = ttree.verts[-1*len(ttree.tree):, 0]
        if ttree._kwargs["tip_labels_align"]:
            ypos = [ttree.verts[:, 1].min()] * len(ttree.tree)
            start = ypos
            finish = ttree.verts[-1*len(ttree.tree):, 1]
            align_edges = np.array([(i, i+len(ypos)) for i in range(len(ypos))])
            align_verts = np.array(zip(xpos, start) + zip(xpos, finish))
        else:
            ypos = ttree.verts[-1*len(ttree.tree):, 1]

    table = toyplot.data.Table()
    table['x'] = toyplot.require.scalar_vector(xpos)
    table['y'] = toyplot.require.scalar_vector(ypos, table.shape[0])
    table['text'] = toyplot.broadcast.pyobject(ttree.get_tip_labels(), table.shape[0])
    table["angle"] = toyplot.broadcast.scalar(angle, table.shape[0])
    table["opacity"] = toyplot.broadcast.scalar(1.0, table.shape[0])
    table["title"] = toyplot.broadcast.pyobject(None, table.shape[0])
    style = toyplot.style.require(ttree._kwargs["tip_labels_style"],
                                  allowed=toyplot.style.allowed.text)
    default_color = [toyplot.color.black]
    color = toyplot.color.broadcast(
        colors=ttree._kwargs["tip_labels_color"],
        shape=(table.shape[0], 1),
        default=default_color,
        )
    table["fill"] = color[:, 0]

    text_mark = toyplot.mark.Text(
        coordinate_axes=['x', 'y'],
        table=table,
        coordinates=['x', 'y'],
        text=["text"],
        angle=["angle"],
        fill=["fill"],
        opacity=["opacity"],
        title=["title"],
        style=style,
        annotation=True,
        filename=None
        )
    return text_mark


Instruction:
Make a summary of the following Python 3 code
def get_edge_mark(ttree):

    ## tree style
    if ttree._kwargs["tree_style"] in ["c", "cladogram"]:
        a=ttree.edges
        vcoordinates=ttree.verts
    else:
        a=ttree._lines
        vcoordinates=ttree._coords

    ## fixed args
    along='x'
    vmarker='o'
    vcolor=None
    vlshow=False
    vsize=0.
    estyle=ttree._kwargs["edge_style"]

    ## get axes
    layout = toyplot.layout.graph(a, vcoordinates=vcoordinates)
    along = toyplot.require.value_in(along, ["x", "y"])
    if along == "x":
        coordinate_axes = ["x", "y"]
    elif along == "y":
        coordinate_axes = ["y", "x"]

    ## broadcast args along axes
    vlabel = layout.vids
    vmarker = toyplot.broadcast.pyobject(vmarker, layout.vcount)
    vsize = toyplot.broadcast.scalar(vsize, layout.vcount)
    estyle = toyplot.style.require(estyle, allowed=toyplot.style.allowed.line)

    ## fixed args
    vcolor = toyplot.color.broadcast(colors=None, shape=layout.vcount, default=toyplot.color.black)
    vopacity = toyplot.broadcast.scalar(1.0, layout.vcount)
    vtitle = toyplot.broadcast.pyobject(None, layout.vcount)
    vstyle = None
    vlstyle = None

    ## this could be modified in the future to allow diff color edges
    ecolor = toyplot.color.broadcast(colors=None, shape=layout.ecount, default=toyplot.color.black)
    ewidth = toyplot.broadcast.scalar(1.0, layout.ecount)
    eopacity = toyplot.broadcast.scalar(1.0, layout.ecount)
    hmarker = toyplot.broadcast.pyobject(None, layout.ecount)
    mmarker = toyplot.broadcast.pyobject(None, layout.ecount)
    mposition = toyplot.broadcast.scalar(0.5, layout.ecount)
    tmarker = toyplot.broadcast.pyobject(None, layout.ecount)

    ## tables are required if I don't want to edit the class
    vtable = toyplot.data.Table()
    vtable["id"] = layout.vids
    for axis, coordinates in zip(coordinate_axes, layout.vcoordinates.T):
        vtable[axis] = coordinates
        #_mark_exportable(vtable, axis)
    vtable["label"] = vlabel
    vtable["marker"] = vmarker
    vtable["size"] = vsize
    vtable["color"] = vcolor
    vtable["opacity"] = vopacity
    vtable["title"] = vtitle

    etable = toyplot.data.Table()
    etable["source"] = layout.edges.T[0]
    #_mark_exportable(etable, "source")
    etable["target"] = layout.edges.T[1]
    #_mark_exportable(etable, "target")
    etable["shape"] = layout.eshapes
    etable["color"] = ecolor
    etable["width"] = ewidth
    etable["opacity"] = eopacity
    etable["hmarker"] = hmarker
    etable["mmarker"] = mmarker
    etable["mposition"] = mposition
    etable["tmarker"] = tmarker

    edge_mark = toyplot.mark.Graph(
        coordinate_axes=['x', 'y'],
        ecolor=["color"],
        ecoordinates=layout.ecoordinates,
        efilename=None,
        eopacity=["opacity"],
        eshape=["shape"],
        esource=["source"],
        estyle=estyle,
        etable=etable,
        etarget=["target"],
        ewidth=["width"],
        hmarker=["hmarker"],
        mmarker=["mmarker"],
        mposition=["mposition"],
        tmarker=["tmarker"],
        vcolor=["color"],
        vcoordinates=['x', 'y'],
        vfilename=None,
        vid=["id"],
        vlabel=["label"],
        vlshow=False,
        vlstyle=None,
        vmarker=["marker"],
        vopacity=["opacity"],
        vsize=["size"],
        vstyle=None,
        vtable=vtable,
        vtitle=["title"],
        )
    return edge_mark

Response: returns a simple edge mark object


Instruction:
Can you tell what is the following Python 3 function doing
def split_styles(mark):

    markers = [mark._table[key] for key in mark._marker][0]
    nstyles = []
    for m in markers:
        ## fill and stroke are already rgb() since already in markers
        msty = toyplot.style.combine({
            "fill": m.mstyle['fill'],
            "stroke": m.mstyle['stroke'],
            "opacity": m.mstyle["fill-opacity"],
        }, m.mstyle)
        msty = _color_fixup(msty)
        nstyles.append(msty)

    ## uses 'marker.size' so we need to loop over it
    lstyles = []
    for m in markers:
        lsty = toyplot.style.combine({
        "font-family": "Helvetica",
        "-toyplot-vertical-align": "middle",
        "fill": toyplot.color.black,
        "font-size": "%rpx" % (m.size * 0.75),
        "stroke": "none",
        "text-anchor": "middle",
        }, m.lstyle)
        ## update fonts
        fonts = toyplot.font.ReportlabLibrary()
        layout = toyplot.text.layout(m.label, lsty, fonts)
        lsty = _color_fixup(layout.style)
        lstyles.append(lsty)

    nallkeys = set(itertools.chain(*[i.keys() for i in nstyles]))
    lallkeys = set(itertools.chain(*[i.keys() for i in lstyles]))
    nuniquekeys = []
    nsharedkeys = []
    for key in nallkeys:
        vals = [nstyles[i].get(key) for i in range(len(nstyles))]
        if len(set(vals)) > 1:
            nuniquekeys.append(key)
        else:
            nsharedkeys.append(key)
    luniquekeys = []
    lsharedkeys = []
    for key in lallkeys:
        vals = [lstyles[i].get(key) for i in range(len(lstyles))]
        if len(set(vals)) > 1:
            luniquekeys.append(key)
        else:
            lsharedkeys.append(key)

    ## keys shared between mark and text markers
    repeated = set(lsharedkeys).intersection(set(nsharedkeys))
    for repeat in repeated:
        ## if same then keep only one copy of it
        lidx = lsharedkeys.index(repeat)
        nidx = nsharedkeys.index(repeat)
        if lsharedkeys[lidx] == nsharedkeys[nidx]:
            lsharedkeys.remove(repeat)
        else:
            lsharedkeys.remove(repeat)
            luniquekeys.append(repeat)
            nsharedkeys.remove(repeat)
            nuniquekeys.append(repeat)

    ## check node values
    natt = ["%s:%s" % (key, nstyles[0][key]) for key in sorted(nsharedkeys)]
    latt = ["%s:%s" % (key, lstyles[0][key]) for key in sorted(lsharedkeys)]
    shared_styles = ";".join(natt+latt)
    unique_styles = {
        "node": [{k:v for k,v in nstyles[idx].items() if k in nuniquekeys} for idx in range(len(markers))],
        "text": [{k:v for k,v in lstyles[idx].items() if k in luniquekeys} for idx in range(len(markers))]
    }

    return shared_styles, unique_styles

Response: split the styles of a single mark into two lists of unique and shared styles


Instruction:
Can you write a function in Python 3 where it
centers the text within a cell s grid.

Response:
def center_cell_text(cell):
    """
    Horizontally center the text within a cell's grid

    Like this::

        +---------+     +---------+
        | foo     | --> |   foo   |
        +---------+     +---------+

    Parameters
    ----------
    cell : dashtable.data2rst.Cell

    Returns
    -------
    cell : dashtable.data2rst.Cell
    """
    lines = cell.text.split('\n')
    cell_width = len(lines[0]) - 2

    truncated_lines = ['']
    for i in range(1, len(lines) - 1):
        truncated = lines[i][2:len(lines[i]) - 2].rstrip()
        truncated_lines.append(truncated)

    truncated_lines.append('')

    max_line_length = get_longest_line_length('\n'.join(truncated_lines))
    remainder = cell_width - max_line_length

    left_width = math.floor(remainder / 2)
    left_space = left_width * ' '

    for i in range(len(truncated_lines)):
        truncated_lines[i] = left_space + truncated_lines[i]
        right_width = cell_width - len(truncated_lines[i])
        truncated_lines[i] += right_width * ' '

    for i in range(1, len(lines) - 1):
        lines[i] = ''.join([
            lines[i][0], truncated_lines[i], lines[i][-1]
        ])

    cell.text = '\n'.join(lines)

    return cell


Instruction:
Write a Python 3 function for
computing the Hamming distance between two words.

Response:
def hamming_distance(word1, word2):
    """
    Computes the Hamming distance.

    [Reference]: https://en.wikipedia.org/wiki/Hamming_distance
    [Article]: Hamming, Richard W. (1950), "Error detecting and error correcting codes",
        Bell System Technical Journal 29 (2): 147–160
    """
    from operator import ne
    if len(word1) != len(word2):
        raise WrongLengthException('The words need to be of the same length!')

    return sum(map(ne, word1, word2))


Instruction:
Can you generate a brief explanation for the following Python 3 code
def timeseries(name='', backend=None, date=None, data=None, **kwargs):
    '''Create a new :class:`dynts.TimeSeries` instance using a ``backend``
    and populating it with provided the data.

    :parameter name: optional timeseries name. For multivarate timeseries
                     the :func:`dynts.tsname` utility function can be used
                     to build it.
    :parameter backend: optional backend name.
        If not provided, numpy will be used.
    :parameter date: optional iterable over dates.
    :parameter data: optional iterable over data.
    '''
    backend = backend or settings.backend
    TS = BACKENDS.get(backend)
    if not TS:
        raise InvalidBackEnd(
            'Could not find a TimeSeries class %s' % backend
        )
    return TS(name=name, date=date, data=data, **kwargs)

Response: Create a new instance of a new TimeSeries with the given name and date and data.


Instruction:
Create a Python 3 function to
force each cell in the table to be a string

Response:
def ensure_table_strings(table):
    """
    Force each cell in the table to be a string

    Parameters
    ----------
    table : list of lists

    Returns
    -------
    table : list of lists of str
    """
    for row in range(len(table)):
        for column in range(len(table[row])):
            table[row][column] = str(table[row][column])
    return table


Instruction:
How would you implement a function in Python 3 that
returns the number of sections that touch the left side.

Response:
def left_sections(self):
        """
        The number of sections that touch the left side.

        During merging, the cell's text will grow to include other
        cells. This property keeps track of the number of sections that
        are touching the left side. For example::

                        +-----+-----+
            section --> | foo | dog | <-- section
                        +-----+-----+
            section --> | cat |
                        +-----+

        Has 2 sections on the left, but 1 on the right

        Returns
        -------
        sections : int
            The number of sections on the left
        """
        lines = self.text.split('\n')
        sections = 0

        for i in range(len(lines)):
            if lines[i].startswith('+'):
                sections += 1
        sections -= 1

        return sections


Instruction:
Create a Python 3 function to
return the number of sections that touch the right side.

Response:
def right_sections(self):
        """
        The number of sections that touch the right side.

        Returns
        -------
        sections : int
            The number of sections on the right
        """
        lines = self.text.split('\n')
        sections = 0
        for i in range(len(lines)):
            if lines[i].endswith('+'):
                sections += 1
        return sections - 1


Instruction:
Explain what the following Python 3 code does
def top_sections(self):

        top_line = self.text.split('\n')[0]
        sections = len(top_line.split('+')) - 2

        return sections

Response: Returns the number of sections that touch the top side.


Instruction:
Implement a Python 3 function for
returning the number of sections on the bottom side of the text.

Response:
def bottom_sections(self):
        """
        The number of cells that touch the bottom side.

        Returns
        -------
        sections : int
            The number of sections on the top
        """
        bottom_line = self.text.split('\n')[-1]
        sections = len(bottom_line.split('+')) - 2

        return sections


Instruction:
Can you generate a brief explanation for the following Python 3 code
def is_header(self):
        bottom_line = self.text.split('\n')[-1]

        if is_only(bottom_line, ['+', '=']):
            return True

        return False

Response: Returns True if the cell is a header cell False otherwise.


Instruction:
Write a Python 3 script to
return a numeric identifier of the latest git changeset.

Response:
def get_git_changeset(filename=None):
    """Returns a numeric identifier of the latest git changeset.

    The result is the UTC timestamp of the changeset in YYYYMMDDHHMMSS format.
    This value isn't guaranteed to be unique, but collisions are very unlikely,
    so it's sufficient for generating the development version numbers.
    """
    dirname = os.path.dirname(filename or __file__)
    git_show = sh('git show --pretty=format:%ct --quiet HEAD',
                  cwd=dirname)
    timestamp = git_show.partition('\n')[0]
    try:
        timestamp = datetime.datetime.utcfromtimestamp(int(timestamp))
    except ValueError:
        return None
    return timestamp.strftime('%Y%m%d%H%M%S')


Instruction:
How would you implement a function in Python 3 that
checks if the html table contains headers and returns True or False

Response:
def headers_present(html_string):
    """
    Checks if the html table contains headers and returns True/False

    Parameters
    ----------
    html_string : str

    Returns
    -------
    bool
    """
    try:
        from bs4 import BeautifulSoup
    except ImportError:
        print("ERROR: You must have BeautifulSoup to use html2data")
        return

    soup = BeautifulSoup(html_string, 'html.parser')
    table = soup.find('table')
    if not table:
        return False

    th = table.findAll('th')
    if len(th) > 0:
        return True
    else:
        return False


Instruction:
How would you explain what the following Python 3 function does
def extract_spans(html_string):
    try:
        from bs4 import BeautifulSoup
    except ImportError:
        print("ERROR: You must have BeautifulSoup to use html2data")
        return

    soup = BeautifulSoup(html_string, 'html.parser')
    table = soup.find('table')
    if not table:
        return []

    trs = table.findAll('tr')
    if len(trs) == 0:
        return []

    spans = []
    for tr in range(len(trs)):
        if tr == 0:
            ths = trs[tr].findAll('th')
            if len(ths) == 0:
                ths = trs[tr].findAll('td')
            tds = ths
        else:
            tds = trs[tr].findAll('td')

        column = 0
        for td in tds:
            r_span_count = 1
            c_span_count = 1
            current_column = column

            if td.has_attr('rowspan'):
                r_span_count = int(td['rowspan'])
            if td.has_attr('colspan'):
                c_span_count = int(td['colspan'])
                column += c_span_count
            else:
                column += 1

            new_span = []
            for r_index in range(tr, tr + r_span_count):
                for c_index in range(current_column, column):
                    if not get_span(spans, r_index, c_index):
                        new_span.append([r_index, c_index])

            if len(new_span) > 0:
                spans.append(new_span)

    return spans

Response: Extracts the spanned cell groups of row and column pairs from a string.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def translation(first, second):
    if len(first) != len(second):
        raise WrongLengthException('The lists are not of the same length!')
    return dict(zip(first, second))

Response: Create an index of mapped letters zip to dict


Instruction:
Write a Python 3 function for
lagging iterator over dates

Response:
def laggeddates(ts, step=1):
    '''Lagged iterator over dates'''
    if step == 1:
        dates = ts.dates()
        if not hasattr(dates, 'next'):
            dates = dates.__iter__()
        dt0 = next(dates)
        for dt1 in dates:
            yield dt1, dt0
            dt0 = dt1
    else:
        while done:
            done += 1
            lag.append(next(dates))
        for dt1 in dates:
            lag.append(dt1)
            yield dt1, lag.pop(0)


Instruction:
Here you have a function in Python 3, explain what it does
def make_skiplist(*args, use_fallback=False):
    '''Create a new skiplist'''
    sl = fallback.Skiplist if use_fallback else Skiplist
    return sl(*args)

Response: Create a new skiplist


Instruction:
How would you explain what the following Python 3 function does
def data2md(table):

    table = copy.deepcopy(table)

    table = ensure_table_strings(table)
    table = multis_2_mono(table)
    table = add_cushions(table)

    widths = []
    for column in range(len(table[0])):
        widths.append(get_column_width(column, table))

    output = '|'
    for i in range(len(table[0])):
        output = ''.join(
            [output, center_line(widths[i], table[0][i]), '|'])

    output = output + '\n|'
    for i in range(len(table[0])):
        output = ''.join([
            output, center_line(widths[i], "-" * widths[i]), '|'])
    output = output + '\n|'

    for row in range(1, len(table)):
        for column in range(len(table[row])):
            output = ''.join(
                [output, center_line(widths[column],
                 table[row][column]), '|'])
        output = output + '\n|'

    split = output.split('\n')
    split.pop()

    table_string = '\n'.join(split)

    return table_string

Response: Converts a list of strings to a markdown table.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def data2rst(table, spans=[[[0, 0]]], use_headers=True,
             center_cells=False, center_headers=False):
    """
    Convert a list of lists of str into a reStructuredText Grid Table

    Parameters
    ----------
    table : list of lists of str
    spans : list of lists of lists of int, optional
        These are [row, column] pairs of cells that are merged in the
        table. Rows and columns start in the top left of the table.For
        example::

            +--------+--------+
            | [0, 0] | [0, 1] |
            +--------+--------+
            | [1, 0] | [1, 1] |
            +--------+--------+

    use_headers : bool, optional
        Whether or not the first row of table data will become headers.
    center_cells : bool, optional
        Whether or not cells will be centered
    center_headers: bool, optional
        Whether or not headers will be centered

    Returns
    -------
    str
        The grid table string

    Example
    -------
    >>> spans = [
    ...     [ [3, 1], [4, 1] ],
    ...     [ [3, 2], [4, 2] ],
    ...     [ [2, 1], [2, 2] ],
    ... ]
    >>> table = [
    ...     ["Header 1", "Header 2", "Header 3"],
    ...     ["body row 1", "column 2", "column 3"],
    ...     ["body row 2", "Cells may span columns", ""],
    ...     ["body row 3", "Cells may span rows.", "- Cells\\n-contain\\n-blocks"],
    ...     ["body row 4", "", ""],
    ... ]
    >>> print(dashtable.data2rst(table, spans))
    +------------+------------+-----------+
    | Header 1   | Header 2   | Header 3  |
    +============+============+===========+
    | body row 1 | column 2   | column 3  |
    +------------+------------+-----------+
    | body row 2 | Cells may span columns.|
    +------------+------------+-----------+
    | body row 3 | Cells may  | - Cells   |
    +------------+ span rows. | - contain |
    | body row 4 |            | - blocks. |
    +------------+------------+-----------+
    """

    table = copy.deepcopy(table)

    table_ok = check_table(table)
    if not table_ok == "":
        return "ERROR: " + table_ok

    if not spans == [[[0, 0]]]:
        for span in spans:
            span_ok = check_span(span, table)
            if not span_ok == "":
                return "ERROR: " + span_ok

    table = ensure_table_strings(table)
    table = add_cushions(table)

    spans = table_cells_2_spans(table, spans)

    widths = get_output_column_widths(table, spans)
    heights = get_output_row_heights(table, spans)

    cells = []
    for span in spans:
        cell = make_cell(table, span, widths, heights, use_headers)
        cells.append(cell)

    cells = list(sorted(cells))

    if center_cells:
        for cell in cells:
            if not cell.is_header:
                center_cell_text(cell)
                v_center_cell_text(cell)

    if center_headers:
        for cell in cells:
            if cell.is_header:
                center_cell_text(cell)
                v_center_cell_text(cell)

    grid_table = merge_all_cells(cells)

    return grid_table

Response: Convert a list of lists of str into a reStructuredText Grid Table


Instruction:
Implement a function in Python 3 to
calculate reasonable height and width for tree given N tips

Response:
def set_dims_from_tree_size(self):
        "Calculate reasonable height and width for tree given N tips"
        tlen = len(self.treelist[0])
        if self.style.orient in ("right", "left"):
            # long tip-wise dimension
            if not self.style.height:
                self.style.height = max(275, min(1000, 18 * (tlen)))
            if not self.style.width:
                self.style.width = max(300, min(500, 18 * (tlen)))
        else:
            # long tip-wise dimension
            if not self.style.width:
                self.style.width = max(275, min(1000, 18 * (tlen)))
            if not self.style.height:
                self.style.height = max(225, min(500, 18 * (tlen)))


Instruction:
Can you create a Python 3 function that
adds tip - labels to the axes.

Response:
def add_tip_labels_to_axes(self):
        """
        Add text offset from tips of tree with correction for orientation,
        and fixed_order which is usually used in multitree plotting.
        """
        # get tip-coords and replace if using fixed_order
        if self.style.orient in ("up", "down"):
            ypos = np.zeros(self.ntips)
            xpos = np.arange(self.ntips)

        if self.style.orient in ("right", "left"):
            xpos = np.zeros(self.ntips)
            ypos = np.arange(self.ntips)

        # pop fill from color dict if using color
        if self.style.tip_labels_colors:
            self.style.tip_labels_style.pop("fill")

        # fill anchor shift if None
        # (Toytrees fill this at draw() normally when tip_labels != None)
        if self.style.tip_labels_style["-toyplot-anchor-shift"] is None:
            self.style.tip_labels_style["-toyplot-anchor-shift"] = "15px"

        # add tip names to coordinates calculated above
        self.axes.text(
            xpos,
            ypos,
            self.tip_labels,
            angle=(0 if self.style.orient in ("right", "left") else -90),
            style=self.style.tip_labels_style,
            color=self.style.tip_labels_colors,
        )
        # get stroke-width for aligned tip-label lines (optional)
        # copy stroke-width from the edge_style unless user set it
        if not self.style.edge_align_style.get("stroke-width"):
            self.style.edge_align_style['stroke-width'] = (
                self.style.edge_style['stroke-width'])


Instruction:
Can you tell what is the following Python 3 function doing
def fit_tip_labels(self):

        if not self.tip_labels:
            return

        # longest name (this will include html hacks)
        longest_name = max([len(i) for i in self.tip_labels])
        if longest_name > 10:
            multiplier = 0.85
        else:
            multiplier = 0.25

        if self.style.use_edge_lengths:
            addon = (self.treelist[0].treenode.height + (
                self.treelist[0].treenode.height * multiplier))
        else:
            addon = self.treelist[0].treenode.get_farthest_leaf(True)[1]

        # modify display for orientations
        if self.style.orient == "right":
            self.axes.x.domain.max = addon
        elif self.style.orient == "down":
            self.axes.y.domain.min = -1 * addon

Response: Fits the tip labels to the display.


Instruction:
Write a Python 3 function for
converting a p element into a new text.

Response:
def convert_p(element, text):
    """
    Adds 2 newlines to the end of text
    """
    depth = -1
    while element:
        if (not element.name == '[document]' and
                not element.parent.get('id') == '__RESTRUCTIFY_WRAPPER__'):
            depth += 1

        element = element.parent

    if text:
        text = '    ' * depth + text
    return text


Instruction:
How would you implement a function in Python 3 that
converts a simple rst table to data

Response:
def simple2data(text):
    """
    Convert a simple table to data (the kind used by DashTable)

    Parameters
    ----------
    text : str
        A valid simple rst table

    Returns
    -------
    table : list of lists of str
    spans : list of lists of lists of int
        A span is a [row, column] pair that defines a group of merged
        cells in the table. In a simple rst table, spans can only be
        colspans.
    use_headers : bool
        Whether or not this table uses headers
    headers_row : int
        The row where headers are located

    Notes
    -----
    This function requires docutils_.

    .. _docutils: http://docutils.sourceforge.net/

    Example
    -------
    >>> html_text = '''
    ... ======  =====  ======
    ...    Inputs      Output
    ... -------------  ------
    ...   A       B    A or B
    ... ======  =====  ======
    ... False   False  False
    ...  True   False   True
    ... False   True    True
    ...  True   True    True
    ... ======  =====  ======
    ... '''
    >>> from dashtable import simple2data
    >>> table, spans, use_headers, headers_row = simple2data(html_text)
    >>> from pprint import pprint
    >>> pprint(table)
    [['Inputs', 'Output', ''],
     ['A', 'B', 'A or B'],
     ['False', 'False', 'False'],
     ['True, 'False', 'True'],
     ['False', 'True', 'True'],
     ['True', 'True', 'True']]
    >>> print(spans)
    [[[0, 0], [0, 1]]]
    >>> print(use_headers)
    True
    >>> print(headers_row)
    1
    """
    try:
        import docutils.statemachine
        import docutils.parsers.rst.tableparser
    except ImportError:
        print("ERROR: You must install the docutils library to use simple2data")
        return

    lines = text.split('\n')
    lines = truncate_empty_lines(lines)
    leading_space = lines[0].replace(lines[0].lstrip(), '')
    for i in range(len(lines)):
        lines[i] = lines[i][len(leading_space)::]
    parser = docutils.parsers.rst.tableparser.SimpleTableParser()

    block = docutils.statemachine.StringList(list(lines))
    simple_data = list(parser.parse(block))

    column_widths = simple_data.pop(0)
    column_count = len(column_widths)
    headers_row = 0

    if len(simple_data[0]) > 0:
        use_headers = True
        headers_row = len(simple_data[0]) - 1
        headers = simple_data[0][0]
        row_count = len(simple_data[1]) + len(simple_data[0])
        while len(simple_data[0]) > 0:
            simple_data[1].insert(0, simple_data[0][-1])
            simple_data[0].pop(-1)
        simple_data.pop(0)
    else:
        use_headers = False
        simple_data.pop(0)
        row_count = len(simple_data[0])

    simple_data = simple_data[0]
    table = make_empty_table(row_count, column_count)
    spans = []

    for row in range(len(simple_data)):
        for column in range(len(simple_data[row])):
            try:
                text = '\n'.join(simple_data[row][column][3]).rstrip()
                table[row][column] = text
                extra_rows = simple_data[row][column][0]
                extra_columns = simple_data[row][column][1]
                span = make_span(row, column, extra_rows, extra_columns)
                span = sorted(span)
                span = list(span for span,_ in itertools.groupby(span))
                if not len(span) == 1:
                    spans.append(span)
            except TypeError:
                pass
    spans = sorted(spans)
    return table, spans, use_headers, headers_row


Instruction:
Can you write a function in Python 3 where it
gets the widths of the columns of the output table.

Response:
def get_output_column_widths(table, spans):
    """
    Gets the widths of the columns of the output table

    Parameters
    ----------
    table : list of lists of str
        The table of rows of text
    spans : list of lists of int
        The [row, column] pairs of combined cells

    Returns
    -------
    widths : list of int
        The widths of each column in the output table
    """
    widths = []
    for column in table[0]:
        widths.append(3)

    for row in range(len(table)):
        for column in range(len(table[row])):
            span = get_span(spans, row, column)
            column_count = get_span_column_count(span)

            if column_count == 1:
                text_row = span[0][0]
                text_column = span[0][1]

                text = table[text_row][text_column]

                length = get_longest_line_length(text)
                if length > widths[column]:
                    widths[column] = length

    for row in range(len(table)):
        for column in range(len(table[row])):
            span = get_span(spans, row, column)
            column_count = get_span_column_count(span)

            if column_count > 1:
                text_row = span[0][0]
                text_column = span[0][1]

                text = table[text_row][text_column]

                end_column = text_column + column_count

                available_space = sum(
                    widths[text_column:end_column])
                available_space += column_count - 1

                length = get_longest_line_length(text)

                while length > available_space:
                    for i in range(text_column, end_column):
                        widths[i] += 1

                        available_space = sum(
                            widths[text_column:end_column])

                        available_space += column_count - 1
                        if length <= available_space:
                            break
    return widths


Instruction:
Implement a function in Python 3 to
make an empty table for the given row and column count.

Response:
def make_empty_table(row_count, column_count):
    """
    Make an empty table

    Parameters
    ----------
    row_count : int
        The number of rows in the new table
    column_count : int
        The number of columns in the new table

    Returns
    -------
    table : list of lists of str
        Each cell will be an empty str ('')
    """
    table = []
    while row_count > 0:
        row = []
        for column in range(column_count):
            row.append('')
        table.append(row)
        row_count -= 1
    return table


Instruction:
Make a summary of the following Python 3 code
def beta(self):
        '''\
The linear estimation of the parameter vector :math:`\beta` given by

.. math::

    \beta = (X^T X)^-1 X^T y

'''
        t = self.X.transpose()
        XX = dot(t,self.X)
        XY = dot(t,self.y)
        return linalg.solve(XX,XY)

Response: Calculates the beta of the current object.


Instruction:
Given the following Python 3 function, write the documentation
def oftype(self, typ):
        '''Return a generator of formatters codes of type typ'''
        for key, val in self.items():
            if val.type == typ:
                yield key

Response: Return a generator of formatters codes of type typ


Instruction:
Write a Python 3 script to
list of names for series in dataset. It will always return a list or names with length given by .

Response:
def names(self, with_namespace=False):
        '''List of names for series in dataset.

        It will always return a list or names with length given by
        :class:`~.DynData.count`.
        '''
        N = self.count()
        names = self.name.split(settings.splittingnames)[:N]
        n = 0
        while len(names) < N:
            n += 1
            names.append('unnamed%s' % n)
        if with_namespace and self.namespace:
            n = self.namespace
            s = settings.field_separator
            return [n + s + f for f in names]
        else:
            return names


Instruction:
How would you implement a function in Python 3 that
dumps the timeseries using a specific format.

Response:
def dump(self, format=None, **kwargs):
        """Dump the timeseries using a specific ``format``.
        """
        formatter = Formatters.get(format, None)
        if not format:
            return self.display()
        elif not formatter:
            raise FormattingException('Formatter %s not available' % format)
        else:
            return formatter(self, **kwargs)


Instruction:
Can you create a Python 3 function that
parses binary operator and return tuple

Response:
def p_expression_binop(p):
    '''expression : expression PLUS expression
                  | expression MINUS expression
                  | expression TIMES expression
                  | expression DIVIDE expression
                  | expression EQUAL expression
                  | expression CONCAT expression
                  | expression SPLIT expression'''
    v = p[2]
    if v == '+':
        p[0] = PlusOp(p[1], p[3])
    elif v == '-':
        p[0] = MinusOp(p[1], p[3])
    elif v == '*':
        p[0] = MultiplyOp(p[1], p[3])
    elif v == '/':
        p[0] = DivideOp(p[1], p[3])
    elif v == '=':
        p[0] = EqualOp(p[1], p[3])
    elif v == settings.concat_operator:
        p[0] = ConcatenationOp(p[1], p[3])
    elif v == settings.separator_operator:
        p[0] = SplittingOp(p[1], p[3])
    elif v == settings.field_operator:
        p[0] = Symbol(p[1], field=p[3])


Instruction:
Write a Python 3 script for
parsing the expression group.

Response:
def p_expression_group(p):
    '''expression : LPAREN expression RPAREN
                  | LSQUARE expression RSQUARE'''
    v = p[1]
    if v == '(':
        p[0] = functionarguments(p[2])
    elif v == '[':
        p[0] = tsentry(p[2])


Instruction:
Here you have a function in Python 3, explain what it does
def merge_cells(cell1, cell2, direction):

    cell1_lines = cell1.text.split("\n")
    cell2_lines = cell2.text.split("\n")

    if direction == "RIGHT":
        for i in range(len(cell1_lines)):
            cell1_lines[i] = cell1_lines[i] + cell2_lines[i][1::]
        cell1.text = "\n".join(cell1_lines)
        cell1.column_count += cell2.column_count

    elif direction == "TOP":
        if cell1_lines[0].count('+') > cell2_lines[-1].count('+'):
            cell2_lines.pop(-1)
        else:
            cell1_lines.pop(0)
        cell2_lines.extend(cell1_lines)
        cell1.text = "\n".join(cell2_lines)
        cell1.row_count += cell2.row_count
        cell1.row = cell2.row
        cell1.column = cell2.column

    elif direction == "BOTTOM":
        if (cell1_lines[-1].count('+') > cell2_lines[0].count('+') or
                cell1.is_header):
            cell2_lines.pop(0)
        else:
            cell1_lines.pop(-1)
        cell1_lines.extend(cell2_lines)
        cell1.text = "\n".join(cell1_lines)
        cell1.row_count += cell2.row_count

    elif direction == "LEFT":
        for i in range(len(cell1_lines)):
            cell1_lines[i] = cell2_lines[i][0:-1] + cell1_lines[i]
        cell1.text = "\n".join(cell1_lines)
        cell1.column_count += cell2.column_count
        cell1.row = cell2.row
        cell1.column = cell2.column

Response: Merge two cells into one.


Instruction:
Write a Python 3 function that can
iterate over the class - level attributes of a class.

Response:
def iterclass(cls):
    """Iterates over (valid) attributes of a class.

    Args:
        cls (object): the class to iterate over

    Yields:
        (str, obj) tuples: the class-level attributes.
    """
    for field in dir(cls):
        if hasattr(cls, field):
            value = getattr(cls, field)
            yield field, value


Instruction:
Write a Python 3 script to
create a tcp socket to host and port.

Response:
def _mksocket(host, port, q, done, stop):
    """Returns a tcp socket to (host/port). Retries forever if connection fails"""
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(2)
    attempt = 0
    while not stop.is_set():
        try:
            s.connect((host, port))
            return s
        except Exception as ex:
            # Simple exponential backoff: sleep for 1,2,4,8,16,30,30...
            time.sleep(min(30, 2 ** attempt))
            attempt += 1


Instruction:
Can you tell what is the following Python 3 function doing
def _push(host, port, q, done, mps, stop, test_mode):
    sock = None
    retry_line = None
    while not ( stop.is_set() or ( done.is_set() and retry_line == None and q.empty()) ):
        stime = time.time()

        if sock == None and not test_mode:
            sock = _mksocket(host, port, q, done, stop)
            if sock == None:
                break

        if retry_line:
            line = retry_line
            retry_line = None
        else:
            try:
                line = q.get(True, 1)  # blocking, with 1 second timeout
            except:
                if done.is_set():  # no items in queue, and parent finished
                    break
                else:  # no items in queue, but parent might send more
                    continue

        if not test_mode:
            try:
                sock.sendall(line.encode('utf-8'))
            except:
                sock = None  # notify that we need to make a new socket at start of loop
                retry_line = line  # can't really put back in q, so remember to retry this line
                continue

        etime = time.time() - stime  #time that actually elapsed

        #Expected value of wait_time is 1/MPS_LIMIT, ie. MPS_LIMIT per second.
        if mps > 0:
            wait_time = (2.0 * random.random()) / (mps)
            if wait_time > etime:  #if we should wait
                time.sleep(wait_time - etime)  #then wait

    if sock:
        sock.close()

Response: Worker thread. Connect to host and port pull data from q until done is set.


Instruction:
How would you explain what the following Python 3 function does
def log(self, name, val, **tags):
        global _last_timestamp, _last_metrics

        # do not allow .log after closing
        assert not self.done.is_set(), "worker thread has been closed"
        # check if valid metric name
        assert all(c in _valid_metric_chars for c in name), "invalid metric name " + name

        val = float(val)  #Duck type to float/int, if possible.
        if int(val) == val:
            val = int(val)

        if self.host_tag and 'host' not in tags:
            tags['host'] = self.host_tag

        # get timestamp from system time, unless it's supplied as a tag
        timestamp = int(tags.pop('timestamp', time.time()))

        assert not self.done.is_set(), "tsdb object has been closed"
        assert tags != {}, "Need at least one tag"

        tagvals = ' '.join(['%s=%s' % (k, v) for k, v in tags.items()])

        # OpenTSDB has major problems if you insert a data point with the same
        # metric, timestamp and tags. So we keep a temporary set of what points
        # we have sent for the last timestamp value. If we encounter a duplicate,
        # it is dropped.
        unique_str = "%s, %s, %s, %s, %s" % (name, timestamp, tagvals, self.host, self.port)
        if timestamp == _last_timestamp or _last_timestamp == None:
            if unique_str in _last_metrics:
                return  # discard duplicate metrics
            else:
                _last_metrics.add(unique_str)
        else:
            _last_timestamp = timestamp
            _last_metrics.clear()

        line = "put %s %d %s %s\n" % (name, timestamp, val, tagvals)

        try:
            self.q.put(line, False)
            self.queued += 1
        except queue.Full:
            print("potsdb - Warning: dropping oldest metric because Queue is full. Size: %s" % self.q.qsize(), file=sys.stderr)
            self.q.get()  #Drop the oldest metric to make room
            self.q.put(line, False)
        return line

Response: Log a metric name with value val.


Instruction:
Can you implement a function in Python 3 that
returns a list of available serial ports

Response:
def available_ports():
        """
        Scans COM1 through COM255 for available serial ports

        returns a list of available ports
        """
        ports = []

        for i in range(256):
            try:
                p = Serial('COM%d' % i)
                p.close()
                ports.append(p)
            except SerialException:
                pass

        return ports


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_current_response(self):
        response = {'port': 0,
                    'pressed': False,
                    'key': 0,
                    'time': 0}
        if len(self.__response_structs_queue) > 0:
            # make a copy just in case any other internal members of
            # XidConnection were tracking the structure
            response = self.__response_structs_queue[0].copy()
            # we will now hand over 'response' to the calling code,
            # so remove it from the internal queue
            self.__response_structs_queue.pop(0)

        return response

Response: reads the current response data from the object and returns the current response in a dict.


Instruction:
Write a Python 3 function for
detecting xid devices on all of the com ports connected to the computer.

Response:
def detect_xid_devices(self):
        """
        For all of the com ports connected to the computer, send an
        XID command '_c1'.  If the device response with '_xid', it is
        an xid device.
        """
        self.__xid_cons = []

        for c in self.__com_ports:
            device_found = False
            for b in [115200, 19200, 9600, 57600, 38400]:
                con = XidConnection(c, b)

                try:
                    con.open()
                except SerialException:
                    continue

                con.flush_input()
                con.flush_output()
                returnval = con.send_xid_command("_c1", 5).decode('ASCII')

                if returnval.startswith('_xid'):
                    device_found = True
                    self.__xid_cons.append(con)

                    if(returnval != '_xid0'):
                        # set the device into XID mode
                        con.send_xid_command('c10')
                        con.flush_input()
                        con.flush_output()

                    # be sure to reset the timer to avoid the 4.66 hours
                    # problem. (refer to XidConnection.xid_input_found to
                    # read about the 4.66 hours)
                    con.send_xid_command('e1')
                    con.send_xid_command('e5')

                con.close()
                if device_found:
                    break


Instruction:
Explain what the following Python 3 code does
def device_at_index(self, index):
        if index >= len(self.__xid_cons):
            raise ValueError("Invalid device index")

        return self.__xid_cons[index]

Response: Returns the device at the specified index.


Instruction:
Can you generate the documentation for the following Python 3 function
def query_base_timer(self):
        (_, _, time) = unpack('<ccI', self.con.send_xid_command("e3", 6))
        return time

Response: Query the base timer for the current time


Instruction:
Can you tell what is the following Python 3 function doing
def poll_for_response(self):
        key_state = self.con.check_for_keypress()

        if key_state != NO_KEY_DETECTED:
            response = self.con.get_current_response()

            if self.keymap is not None:
                response['key'] = self.keymap[response['key']]
            else:
                response['key'] -= 1

            self.response_queue.append(response)

Response: Polls the device for user input and adds the response to the internal response_queue


Instruction:
How would you code a function in Python 3 to
set the pulse duration for events in miliseconds when activate_line is called is called

Response:
def set_pulse_duration(self, duration):
        """
        Sets the pulse duration for events in miliseconds when activate_line
        is called
        """
        if duration > 4294967295:
            raise ValueError('Duration is too long. Please choose a value '
                             'less than 4294967296.')

        big_endian = hex(duration)[2:]
        if len(big_endian) % 2 != 0:
            big_endian = '0'+big_endian

        little_endian = []

        for i in range(0, len(big_endian), 2):
            little_endian.insert(0, big_endian[i:i+2])

        for i in range(0, 4-len(little_endian)):
            little_endian.append('00')

        command = 'mp'
        for i in little_endian:
            command += chr(int(i, 16))

        self.con.send_xid_command(command, 0)


Instruction:
Here you have a function in Python 3, explain what it does
def activate_line(self, lines=None, bitmask=None,
                      leave_remaining_lines=False):
        """
        Triggers an output line on StimTracker.

        There are 8 output lines on StimTracker that can be raised in any
        combination.  To raise lines 1 and 7, for example, you pass in
        the list: activate_line(lines=[1, 7]).

        To raise a single line, pass in just an integer, or a list with a
        single element to the lines keyword argument:

            activate_line(lines=3)

            or

            activate_line(lines=[3])

        The `lines` argument must either be an Integer, list of Integers, or
        None.

        If you'd rather specify a bitmask for setting the lines, you can use
        the bitmask keyword argument.  Bitmask must be a Integer value between
        0 and 255 where 0 specifies no lines, and 255 is all lines.  For a
        mapping between lines and their bit values, see the `_lines` class
        variable.

        To use this, call the function as so to activate lines 1 and 6:

            activate_line(bitmask=33)

        leave_remaining_lines tells the function to only operate on the lines
        specified.  For example, if lines 1 and 8 are active, and you make
        the following function call:

            activate_line(lines=4, leave_remaining_lines=True)

        This will result in lines 1, 4 and 8 being active.

        If you call activate_line(lines=4) with leave_remaining_lines=False
        (the default), if lines 1 and 8 were previously active, only line 4
        will be active after the call.
        """
        if lines is None and bitmask is None:
            raise ValueError('Must set one of lines or bitmask')
        if lines is not None and bitmask is not None:
            raise ValueError('Can only set one of lines or bitmask')

        if bitmask is not None:
            if bitmask not in range(0, 256):
                raise ValueError('bitmask must be an integer between '
                                 '0 and 255')

        if lines is not None:
            if not isinstance(lines, list):
                lines = [lines]

            bitmask = 0
            for l in lines:
                if l < 1 or l > 8:
                    raise ValueError('Line numbers must be between 1 and 8 '
                                     '(inclusive)')
                bitmask |= self._lines[l]

        self.con.set_digital_output_lines(bitmask, leave_remaining_lines)

Response: Activate the specified lines of the class.


Instruction:
Write a Python 3 function that can
clear the line with the specified number of lines.

Response:
def clear_line(self, lines=None, bitmask=None,
                   leave_remaining_lines=False):
        """
        The inverse of activate_line.  If a line is active, it deactivates it.

        This has the same parameters as activate_line()
        """
        if lines is None and bitmask is None:
            raise ValueError('Must set one of lines or bitmask')
        if lines is not None and bitmask is not None:
            raise ValueError('Can only set one of lines or bitmask')

        if bitmask is not None:
            if bitmask not in range(0, 256):
                raise ValueError('bitmask must be an integer between '
                                 '0 and 255')

        if lines is not None:
            if not isinstance(lines, list):
                lines = [lines]

            bitmask = 0
            for l in lines:
                if l < 1 or l > 8:
                    raise ValueError('Line numbers must be between 1 and 8 '
                                     '(inclusive)')
                bitmask |= self._lines[l]

        self.con.clear_digital_output_lines(bitmask, leave_remaining_lines)


Instruction:
Can you create a Python 3 function that
initializes the device with the proper keymaps and name.

Response:
def init_device(self):
        """
        Initializes the device with the proper keymaps and name
        """
        try:
            product_id = int(self._send_command('_d2', 1))
        except ValueError:
            product_id = self._send_command('_d2', 1)

        if product_id == 0:
            self._impl = ResponseDevice(
                self.con,
                'Cedrus Lumina LP-400 Response Pad System',
                lumina_keymap)
        elif product_id == 1:
            self._impl = ResponseDevice(
                self.con,
                'Cedrus SV-1 Voice Key',
                None,
                'Voice Response')
        elif product_id == 2:
            model_id = int(self._send_command('_d3', 1))
            if model_id == 1:
                self._impl = ResponseDevice(
                    self.con,
                    'Cedrus RB-530',
                    rb_530_keymap)
            elif model_id == 2:
                self._impl = ResponseDevice(
                    self.con,
                    'Cedrus RB-730',
                    rb_730_keymap)
            elif model_id == 3:
                self._impl = ResponseDevice(
                    self.con,
                    'Cedrus RB-830',
                    rb_830_keymap)
            elif model_id == 4:
                self._impl = ResponseDevice(
                    self.con,
                    'Cedrus RB-834',
                    rb_834_keymap)
            else:
                raise XidError('Unknown RB Device')
        elif product_id == 4:
            self._impl = StimTracker(
                self.con,
                'Cedrus C-POD')
        elif product_id == b'S':
            self._impl = StimTracker(
                self.con,
                'Cedrus StimTracker')

        elif product_id == -99:
            raise XidError('Invalid XID device')


Instruction:
Write a Python 3 script for
sending an XID command to the device

Response:
def _send_command(self, command, expected_bytes):
        """
        Send an XID command to the device
        """
        response = self.con.send_xid_command(command, expected_bytes)

        return response


Instruction:
Write a Python 3 function that can
return a list of all Xid devices connected to your computer.

Response:
def get_xid_devices():
    """
    Returns a list of all Xid devices connected to your computer.
    """
    devices = []
    scanner = XidScanner()
    for i in range(scanner.device_count()):
        com = scanner.device_at_index(i)
        com.open()
        device = XidDevice(com)
        devices.append(device)
    return devices