Instruction:
Given the following Python 3 function, write the documentation
def ipi_base_number(name=None):

    if name is None:
        name = 'IPI Base Number Field'

    field = pp.Regex('I-[0-9]{9}-[0-9]')

    # Name
    field.setName(name)

    field_num = basic.numeric(13)
    field_num.setName(name)

    field = field | field_num

    # White spaces are not removed
    field.leaveWhitespace()

    return field.setResultsName('ipi_base_n')

Response: returns a parser for the IPI Base Number field


Instruction:
How would you explain what the following Python 3 function does
def ipi_name_number(name=None):

    if name is None:
        name = 'IPI Name Number Field'

    field = basic.numeric(11)

    field.setName(name)

    return field.setResultsName('ipi_name_n')

Response: A function to create a parser for the IPI Name Number field.


Instruction:
Can you generate the documentation for the following Python 3 function
def iswc(name=None):

    if name is None:
        name = 'ISWC Field'

    # T followed by 10 numbers
    field = pp.Regex('T[0-9]{10}')

    # Name
    field.setName(name)

    # White spaces are not removed
    field.leaveWhitespace()

    return field.setResultsName('iswc')

Response: A function to create a parser for the ISWC code field.


Instruction:
Explain what the following Python 3 code does
def percentage(columns, maximum=100, name=None):

    if name is None:
        name = 'Percentage Field'

    if columns < 3:
        message = 'The values can not be lower than 3'
        raise pp.ParseException(message)

    field = basic.numeric_float(columns, 3)

    field.addParseAction(lambda v: _assert_is_percentage(v[0], maximum))

    field.setName(name)

    return field

Response: Returns a grammar for a numeric field storing a percentage field.


Instruction:
Create a Python 3 function for
ensuring that the received value is a percentage.

Response:
def _assert_is_percentage(value, maximum=100):
    """
    Makes sure the received value is a percentage. Otherwise an exception is
    thrown.

    :param value: the value to check
    """

    if value < 0 or value > maximum:
        message = 'The value on a percentage field should be between 0 and %s' \
                  % maximum
        raise pp.ParseException(message)


Instruction:
Can you implement a function in Python 3 that
returns the grammar for an EAN 13 code.

Response:
def ean_13(name=None):
    """
    Creates the grammar for an EAN 13 code.

    These are the codes on thirteen digits barcodes.

    :param name: name for the field
    :return: grammar for an EAN 13 field
    """

    if name is None:
        name = 'EAN 13 Field'

    field = basic.numeric(13)

    field = field.setName(name)

    return field.setResultsName('ean_13')


Instruction:
Write a Python 3 function that can
return a grammar for an ISRC code.

Response:
def isrc(name=None):
    """
    Creates the grammar for an ISRC code.

    ISRC stands for International Standard Recording Code, which is the
    standard ISO 3901. This stores information identifying a particular
    recording.

    :param name: name for the field
    :return: grammar for an ISRC field
    """

    if name is None:
        name = 'ISRC Field'

    field = _isrc_short(name) | _isrc_long(name)

    field.setName(name)

    return field.setResultsName('isrc')


Instruction:
Can you tell what is the following Python 3 function doing
def _isrc_long(name=None):

    config = CWRTables()

    if name is None:
        name = 'ISRC Field'

    country = config.get_data('isrc_country_code')
    # registrant = basic.alphanum(3)
    # year = pp.Regex('[0-9]{2}')
    # work_id = pp.Regex('[0-9]{5}')

    country_regex = ''
    for c in country:
        if len(country_regex) > 0:
            country_regex += '|'
        country_regex += c
    country_regex = '(' + country_regex + ')'

    field = pp.Regex(country_regex + '.{3}[0-9]{2}[0-9]{5}')

    # country.setName('ISO-2 Country Code')
    # registrant.setName('Registrant')
    # year.setName('Year')
    # work_id.setName('Work ID')

    field.setName(name)

    return field.setResultsName('isrc')

Response: Returns a grammar for a short ISRC code.


Instruction:
Write a Python 3 function that can
return a grammar for a V - ISAN code.

Response:
def visan(name=None):
    """
    Creates the grammar for a V-ISAN code.

    This is a variation on the ISAN (International Standard Audiovisual Number)

    :param name: name for the field
    :return: grammar for an ISRC field
    """

    if name is None:
        name = 'V-ISAN Field'

    field = pp.Regex('[0-9]{25}')

    field.setName(name)

    return field.setResultsName('visan')


Instruction:
Can you generate the documentation for the following Python 3 function
def audio_visual_key(name=None):

    if name is None:
        name = 'AVI Field'

    society_code = basic.numeric(3)
    society_code = society_code.setName('Society Code') \
        .setResultsName('society_code')

    av_number = basic.alphanum(15, extended=True, isLast=True)
    field_empty = pp.Regex('[ ]{15}')
    field_empty.setParseAction(pp.replaceWith(''))
    av_number = av_number | field_empty
    av_number = av_number.setName('Audio-Visual Number') \
        .setResultsName('av_number')

    field = pp.Group(society_code + pp.Optional(av_number))

    field.setParseAction(lambda v: _to_avi(v[0]))

    field = field.setName(name)

    return field.setResultsName('audio_visual_key')

Response: Creates the grammar for an audio visual key code.


Instruction:
How would you implement a function in Python 3 that
returns a grammar for a date and time field.

Response:
def date_time(name=None):
    """
    Creates the grammar for a date and time field, which is a combination of
    the Date (D) and Time or Duration field (T).

    This field requires first a Date, and then a Time, without any space in
    between.

    :param name: name for the field
    :return: grammar for a Date and Time field
    """
    if name is None:
        name = 'Date and Time Field'

    date = basic.date('Date')
    time = basic.time('Time')

    date = date.setResultsName('date')
    time = time.setResultsName('time')

    field = pp.Group(date + time)

    field.setParseAction(lambda d: _combine_date_time(d[0]))

    field.setName(name)

    return field.setResultsName('date_time')


Instruction:
Can you create a Python 3 function that
returns a Lookup field which transforms the result into an integer.

Response:
def lookup_int(values, name=None):
    """
    Lookup field which transforms the result into an integer.

    :param values: values allowed
    :param name: name for the field
    :return: grammar for the lookup field
    """
    field = basic.lookup(values, name)

    field.addParseAction(lambda l: int(l[0]))

    return field


Instruction:
Here you have a function in Python 3, explain what it does
def extract_function_argument(text, f_name, f_argn, f_argt=asttypes.String):

    tree = parse(text)
    return list(filter_function_argument(tree, f_name, f_argn, f_argt))

Response: Extract a specific argument from a specific function name.


Instruction:
Make a summary of the following Python 3 code
def yield_amd_require_string_arguments(
        node, pos,
        reserved_module=reserved_module, wrapped=define_wrapped):
    """
    This yields only strings within the lists provided in the argument
    list at the specified position from a function call.

    Originally, this was implemented for yield a list of module names to
    be imported as represented by this given node, which must be of the
    FunctionCall type.
    """

    for i, child in enumerate(node.args.items[pos]):
        if isinstance(child, asttypes.String):
            result = to_str(child)
            if ((result not in reserved_module) and (
                    result != define_wrapped.get(i))):
                yield result

Response: Yields only strings within the list of module names that are not defined in the function call.


Instruction:
Given the following Python 3 function, write the documentation
def yield_string_argument(node, pos):

    if not isinstance(node.args.items[pos], asttypes.String):
        return
    yield to_str(node.args.items[pos])

Response: Yield just a string argument from the given position.


Instruction:
Here you have a function in Python 3, explain what it does
def yield_module_imports(root, checks=string_imports()):

    if not isinstance(root, asttypes.Node):
        raise TypeError('provided root must be a node')

    for child in yield_function(root, deep_filter):
        for f, condition in checks:
            if condition(child):
                for name in f(child):
                    yield name
                continue

Response: Yields all module names from the source tree root.


Instruction:
Make a summary of the following Python 3 code
def yield_module_imports_nodes(root, checks=import_nodes()):

    if not isinstance(root, asttypes.Node):
        raise TypeError('provided root must be a node')

    for child in yield_function(root, deep_filter):
        for f, condition in checks:
            if condition(child):
                for name in f(child):
                    yield name
                continue

Response: Yield all nodes that provide an import


Instruction:
Can you generate a brief explanation for the following Python 3 code
def open_fasta_index(self):
        index = self.fasta_index
        try:
            handle = open(index, 'rb')
        except (IOError, TypeError):
            sys.stderr.write('index not found, creating it\n')
            try:
                self.build_fasta_index()
                return
            except IOError:
                raise IOError("Index File "+self.fasta_index+"can't be found nor created, check file permissions")
        self.sequence_index = {}
        _seq_dict = self.sequence_index
        for row in handle:
            entry = row.decode('utf-8').strip().split('\t')
            #stored as: {header: length, # of chars to end of this header, length of fasta lines, length of each line including breakchar}
            _seq_dict[entry[0]] = (entry[1], entry[2], entry[3], entry[4])

Response: open the index file and return the object


Instruction:
How would you explain what the following Python 3 function does
def get_sequence(self, chrom, start, end, strand='+', indexing=(-1, 0)):
        try:
            divisor = int(self.sequence_index[chrom][2])
        except KeyError:
            self.open_fasta_index()
            try:
                divisor = int(self.sequence_index[chrom][2])
            except KeyError:
                sys.stderr.write("%s cannot be found within the fasta index file.\n" % chrom)
                return ""
        start+=indexing[0]
        end+=indexing[1]
        #is it a valid position?
        if ( start < 0 or end > int(self.sequence_index[chrom][0] )):
            raise ValueError("The range %d-%d is invalid. Valid range for this feature is 1-%d." % (start-indexing[0], end-indexing[1],
                                                                                int(self.sequence_index[chrom][0])))
        #go to start of chromosome
        seekpos = int(self.sequence_index[chrom][1])
        #find how many newlines we have
        seekpos += start+start/divisor
        slen = end-start
        endpos = int(slen + (slen/divisor) + 1) #a hack of sorts but it works and is easy
        self.fasta_file.seek(seekpos, 0)
        output = self.fasta_file.read(endpos)
        output = output.replace('\n', '')
        out = output[:slen]
        if strand == '+' or strand == 1:
            return out
        if strand == '-' or strand == -1:
            return _reverse_complement(out)

Response: Get the sequence from the file.


Instruction:
Explain what the following Python 3 code does
def resolve_child_module_registries_lineage(registry):

    children = [registry]
    while isinstance(registry, BaseChildModuleRegistry):
        if registry.parent in children:
            # this should never normally occur under normal usage where
            # classes have been properly subclassed with methods defined
            # to specificiation and with standard entry point usage, but
            # non-standard definitions/usage can definitely trigger this
            # self-referential loop.
            raise TypeError(
                "registry '%s' was already recorded in the lineage, "
                "indicating that it may be some (grand)child of itself, which "
                "is an illegal reference in the registry system; previously "
                "resolved lineage is: %r" % (registry.parent.registry_name, [
                    r.registry_name for r in reversed(children)
                ])
            )

        pl = len(registry.parent.registry_name)
        if len(registry.parent.registry_name) > len(registry.registry_name):
            logger.warning(
                "the parent registry '%s' somehow has a longer name than its "
                "child registry '%s'; the underlying registry class may be "
                "constructed in an invalid manner",
                registry.parent.registry_name,
                registry.registry_name,
            )
        elif registry.registry_name[:pl] != registry.parent.registry_name:
            logger.warning(
                "child registry '%s' does not share the same common prefix as "
                "its parent registry '%s'; there may be errors with how the "
                "related registries are set up or constructed",
                registry.registry_name,
                registry.parent.registry_name,
            )
        children.append(registry.parent)
        registry = registry.parent
    # the lineage down from parent to child.
    return iter(reversed(children))

Response: Given a child module registry attempt to resolve the lineage.


Instruction:
Given the following Python 3 function, write the documentation
def addModification(self, aa,position, modMass, modType):
        #clean up xtandem
        if not modType:
            #try to figure out what it is
            tmass = abs(modMass)
            smass = str(tmass)
            prec = len(str(tmass-int(tmass)))-2
            precFormat = '%'+'0.%df'%prec
            # modType = ""
            # masses = config.MODIFICATION_MASSES
            # for i in masses:
            #     if tmass in masses[i] or smass == precFormat%masses[i][0]:
            #         #found it
            #         modType = i
            # if not modType:
            #     sys.stderr.write('mod not found %s\n'%modMass)
        self.mods.add((aa,str(position),str(modMass),str(modType)))

Response: Add a modification to the list of modifications.


Instruction:
Write a Python 3 function for
attempting to resolve the ArcGIS module name and a distribution.

Response:
def resource_filename_mod_dist(module_name, dist):
    """
    Given a module name and a distribution, attempt to resolve the
    actual path to the module.
    """

    try:
        return pkg_resources.resource_filename(
            dist.as_requirement(), join(*module_name.split('.')))
    except pkg_resources.DistributionNotFound:
        logger.warning(
            "distribution '%s' not found, falling back to resolution using "
            "module_name '%s'", dist, module_name,
        )
        return pkg_resources.resource_filename(module_name, '')


Instruction:
Can you implement a function in Python 3 that
returns the full path to the resource file for the given module and entry point.

Response:
def resource_filename_mod_entry_point(module_name, entry_point):
    """
    If a given package declares a namespace and also provide submodules
    nested at that namespace level, and for whatever reason that module
    is needed, Python's import mechanism will not have a path associated
    with that module.  However, if given an entry_point, this path can
    be resolved through its distribution.  That said, the default
    resource_filename function does not accept an entry_point, and so we
    have to chain that back together manually.
    """

    if entry_point.dist is None:
        # distribution missing is typically caused by mocked entry
        # points from tests; silently falling back to basic lookup
        result = pkg_resources.resource_filename(module_name, '')
    else:
        result = resource_filename_mod_dist(module_name, entry_point.dist)

    if not result:
        logger.warning(
            "resource path cannot be found for module '%s' and entry_point "
            "'%s'", module_name, entry_point
        )
        return None
    if not exists(result):
        logger.warning(
            "resource path found at '%s' for module '%s' and entry_point "
            "'%s', but it does not exist", result, module_name, entry_point,
        )
        return None
    return result


Instruction:
How would you code a function in Python 3 to
generate a JavaScript styled module location listing for the given entry point.

Response:
def modgen(
        module, entry_point,
        modpath='pkg_resources', globber='root', fext=JS_EXT,
        registry=_utils):
    """
    JavaScript styled module location listing generator.

    Arguments:

    module
        The Python module to start fetching from.

    entry_point
        This is the original entry point that has a distribution
        reference such that the resource_filename API call may be used
        to locate the actual resources.

    Optional Arguments:

    modpath
        The name to the registered modpath function that will fetch the
        paths belonging to the module.  Defaults to 'pkg_resources'.

    globber
        The name to the registered file globbing function.  Defaults to
        one that will only glob the local path.

    fext
        The filename extension to match.  Defaults to `.js`.

    registry
        The "registry" to extract the functions from

    Yields 3-tuples of

    - raw list of module name fragments
    - the source base path to the python module (equivalent to module)
    - the relative path to the actual module

    For each of the module basepath and source files the globber finds.
    """

    globber_f = globber if callable(globber) else registry['globber'][globber]
    modpath_f = modpath if callable(modpath) else registry['modpath'][modpath]

    logger.debug(
        'modgen generating file listing for module %s',
        module.__name__,
    )

    module_frags = module.__name__.split('.')
    module_base_paths = modpath_f(module, entry_point)

    for module_base_path in module_base_paths:
        logger.debug('searching for *%s files in %s', fext, module_base_path)
        for path in globber_f(module_base_path, '*' + fext):
            mod_path = (relpath(path, module_base_path))
            yield (
                module_frags + mod_path[:-len(fext)].split(sep),
                module_base_path,
                mod_path,
            )


Instruction:
How would you explain what the following Python 3 function does
def modpath_all(module, entry_point):

    module_paths = getattr(module, '__path__', [])
    if not module_paths:
        logger.warning(
            "module '%s' does not appear to be a namespace module or does not "
            "export available paths onto the filesystem; JavaScript source "
            "files cannot be extracted from this module.", module.__name__
        )
    return module_paths

Response: Provides the raw __path__ of the module.


Instruction:
Can you create a Python 3 function that
returns the raw path of the last module in the list.

Response:
def modpath_last(module, entry_point):
    """
    Provides the raw __path__.  Incompatible with PEP 302-based import
    hooks and incompatible with zip_safe packages.

    Deprecated.  Will be removed by calmjs-4.0.
    """

    module_paths = modpath_all(module, entry_point)
    if len(module_paths) > 1:
        logger.info(
            "module '%s' has multiple paths, default selecting '%s' as base.",
            module.__name__, module_paths[-1],
        )
    return module_paths[-1:]


Instruction:
Implement a Python 3 function for
going through pkg_resources for compliance with various PEPs.

Response:
def modpath_pkg_resources(module, entry_point):
    """
    Goes through pkg_resources for compliance with various PEPs.

    This one accepts a module as argument.
    """

    result = []
    try:
        path = resource_filename_mod_entry_point(module.__name__, entry_point)
    except ImportError:
        logger.warning("module '%s' could not be imported", module.__name__)
    except Exception:
        logger.warning("%r does not appear to be a valid module", module)
    else:
        if path:
            result.append(path)
    return result


Instruction:
How would you explain what the following Python 3 function does
def mapper(module, entry_point,
           modpath='pkg_resources', globber='root', modname='es6',
           fext=JS_EXT, registry=_utils):
    """
    General mapper

    Loads components from the micro registry.
    """

    modname_f = modname if callable(modname) else _utils['modname'][modname]

    return {
        modname_f(modname_fragments): join(base, subpath)
        for modname_fragments, base, subpath in modgen(
            module, entry_point=entry_point,
            modpath=modpath, globber=globber,
            fext=fext, registry=_utils)
    }

Response: General mapper for the micro registry.


Instruction:
Can you generate the documentation for the following Python 3 function
def mapper_python(module, entry_point, globber='root', fext=JS_EXT):

    return mapper(
        module, entry_point=entry_point, modpath='pkg_resources',
        globber=globber, modname='python', fext=fext)

Response: Default mapper using python style globber


Instruction:
Create a Python 3 function to
return the ID code in a printable form filling with zeros if needed.

Response:
def _printable_id_code(self):
        """
        Returns the code in a printable form, filling with zeros if needed.

        :return: the ID code in a printable form
        """
        code = str(self.id_code)
        while len(code) < self._code_size:
            code = '0' + code

        return code


Instruction:
Write a Python 3 function for
returning the code in a printable form separating it into groups of three characters using a point between them.

Response:
def _printable_id_code(self):
        """
        Returns the code in a printable form, separating it into groups of
        three characters using a point between them.

        :return: the ID code in a printable form
        """
        code = super(ISWCCode, self)._printable_id_code()

        code1 = code[:3]
        code2 = code[3:6]
        code3 = code[-3:]

        return '%s.%s.%s' % (code1, code2, code3)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def discard(self, s):

        lines = s.splitlines(True)
        for line in lines:
            if line[-1] not in '\r\n':
                if not self.warn:
                    logger.warning(
                        'partial line discard UNSUPPORTED; source map '
                        'generated will not match at the column level'
                    )
                    self.warn = True
            else:
                # simply increment row
                self.row += 1

Response: Discard from original file.


Instruction:
Create a Python 3 function to
write a string that is not part of the original file.

Response:
def write_padding(self, s):
        """
        Write string that are not part of the original file.
        """

        lines = s.splitlines(True)
        for line in lines:
            self.stream.write(line)
            if line[-1] in '\r\n':
                self._newline()
            else:
                # this is the last line
                self.generated_col += len(line)


Instruction:
Make a summary of the following Python 3 code
def format_currency(number, currency, format, locale=babel.numbers.LC_NUMERIC,
                    force_frac=None, format_type='standard'):
    """Same as ``babel.numbers.format_currency``, but has ``force_frac``
    argument instead of ``currency_digits``.

    If the ``force_frac`` argument is given, the argument is passed down to
    ``pattern.apply``.
    """
    locale = babel.core.Locale.parse(locale)
    if format:
        pattern = babel.numbers.parse_pattern(format)
    else:
        try:
            pattern = locale.currency_formats[format_type]
        except KeyError:
            raise babel.numbers.UnknownCurrencyFormatError(
                "%r is not a known currency format type" % format_type)
    if force_frac is None:
        fractions = babel.core.get_global('currency_fractions')
        try:
            digits = fractions[currency][0]
        except KeyError:
            digits = fractions['DEFAULT'][0]
        frac = (digits, digits)
    else:
        frac = force_frac
    return pattern.apply(number, locale, currency=currency, force_frac=frac)

Response: Same as babel. numbers. format_currency but has a force_frac argument instead of currency_digits.


Instruction:
Can you generate the documentation for the following Python 3 function
def run(unihan_options={}):
    print("This example prints variant character data.")

    c = Cihai()
    if not c.unihan.is_bootstrapped:  # download and install Unihan to db
        c.unihan.bootstrap(unihan_options)

    c.unihan.add_plugin(
        'cihai.data.unihan.dataset.UnihanVariants', namespace='variants'
    )

    print("## ZVariants")
    variant_list(c.unihan, "kZVariant")

    print("## kSemanticVariant")
    variant_list(c.unihan, "kSemanticVariant")

    print("## kSpecializedSemanticVariant")
    variant_list(c.unihan, "kSpecializedSemanticVariant")

Response: This example prints the variant character data.


Instruction:
Implement a function in Python 3 to
reflect the database info.

Response:
def reflect_db(self):
        """
        No-op to reflect db info.

        This is available as a method so the database can be reflected
        outside initialization (such bootstrapping unihan during CLI usage).
        """
        self.metadata.reflect(views=True, extend_existing=True)
        self.base = automap_base(metadata=self.metadata)
        self.base.prepare()


Instruction:
Make a summary of the following Python 3 code
def parse(s, element=Element, atomicstring=lambda s: s):

    global Element_, AtomicString_

    Element_ = element
    AtomicString_ = atomicstring

    s, nodes = parse_exprs(s)
    remove_invisible(nodes)
    nodes = map(remove_private, nodes)

    return El('math', El('mstyle', *nodes))

Response: Parses a string representation of a single element into a tree of elements.


Instruction:
Explain what the following Python 3 code does
def trace_parser(p):

    def nodes_to_string(n):
        if isinstance(n, list):
            result = '[ '
            for m in map(nodes_to_string, n):
                result += m
                result += ' '
            result += ']'

            return result
        else:
            try:
                return tostring(remove_private(copy(n)))
            except Exception as e:
                return n

    def print_trace(*args):
        import sys

        sys.stderr.write("    " * tracing_level)
        for arg in args:
            sys.stderr.write(str(arg))
            sys.stderr.write(' ')
        sys.stderr.write('\n')
        sys.stderr.flush()

    def wrapped(s, *args, **kwargs):
        global tracing_level

        print_trace(p.__name__, repr(s))

        tracing_level += 1
        s, n = p(s, *args, **kwargs)
        tracing_level -= 1

        print_trace("-> ", repr(s), nodes_to_string(n))

        return s, n

    return wrapped

Response: Decorator for tracing the parser.


Instruction:
Given the following Python 3 function, write the documentation
def remove_symbol_from_dist(dist, index):
    '''
    prob is a ndarray representing a probability distribution.
    index is a number between 0 and and the number of symbols ( len(prob)-1 )
    return the probability distribution if the element at 'index' was no longer available
    '''
    if type(dist) is not Distribution:
        raise TypeError("remove_symbol_from_dist got an object ot type {0}".format(type(dist)))

    new_prob = dist.prob.copy()
    new_prob[index]=0
    new_prob /= sum(new_prob)
    return Distribution(new_prob)

Response: Removes a symbol from a probability distribution.


Instruction:
How would you implement a function in Python 3 that
changes every response in x that matches index by randomly sampling from prob

Response:
def change_response(x, prob, index):
    '''
    change every response in x that matches 'index' by randomly sampling from prob
    '''
    #pdb.set_trace()
    N = (x==index).sum()
    #x[x==index]=9
    x[x==index] = dist.sample(N)


Instruction:
How would you code a function in Python 3 to
make a toy example where x is uniformly distributed with N bits and y is uniformly distributed with N bits and y follows x and with symbol dependent noise.

Response:
def toy_example():
    """
    Make a toy example where x is uniformly distributed with N bits and y
    follows x but with symbol dependent noise.
    x=0 -> y=0
    x=1 -> y=1 + e
    x=2 -> y=2 + 2*e
    ...
    x=n -> y=n + n*e
    where by n*e I am saying that the noise grows
    """
    #pdb.set_trace()
    N=4
    m = 100
    x = np.zeros(m*(2**N))
    y = np.zeros(m*(2**N))

    for i in range(1, 2**N):
        x[i*m:(i+1)*m] = i
        y[i*m:(i+1)*m] = i + np.random.randint(0, 2*i, m)

    diff = differentiate_mi(x,y)
    return x, y, diff


Instruction:
Can you write a function in Python 3 where it
computes the mi of the given x and y

Response:
def differentiate_mi(x, y):
    '''
    for each symbol in x, change x such that there are no more of such symbols
    (replacing by a random distribution with the same proba of all other symbols)
    and compute mi(new_x, y)
    '''
    #pdb.set_trace()
    dist = Distribution(discrete.symbols_to_prob(x))

    diff = np.zeros(len(dist.prob))

    for i in range(len(dist.prob)):
        i = int(i)
        dist = Distribution(remove_symbol_from_dist(dist, i).prob)

        new_x = change_response(x, dist, i)

        diff[i] = discrete.mi(x,y)

    return diff


Instruction:
Write a Python 3 function for
generating a random number in [ 0 1 ) and return the index into self. prob where the random number is greater than random_number.

Response:
def sample(self, *args):
        '''
        generate a random number in [0,1) and return the index into self.prob
        such that self.prob[index] <= random_number but self.prob[index+1] > random_number

        implementation note: the problem is identical to finding the index into self.cumsum
        where the random number should be inserted to keep the array sorted. This is exactly
        what searchsorted does.

        usage:
            myDist = Distribution(array(0.5, .25, .25))
            x = myDist.sample()         # generates 1 sample
            x = myDist.sample(100)      # generates 100 samples
            x = myDist.sample(10,10)    # generates a 10x10 ndarray
        '''
        return self.cumsum.searchsorted(np.random.rand(*args))


Instruction:
Can you implement a function in Python 3 that
returns the default log template for the given log record.

Response:
def default_log_template(self, record):
    """Return the prefix for the log message. Template for Formatter.

    :param: record: :py:class:`logging.LogRecord` object. this is passed in
    from inside the :py:meth:`logging.Formatter.format` record.

    """

    reset = Style.RESET_ALL
    levelname = [
        LEVEL_COLORS.get(record.levelname),
        Style.BRIGHT,
        '(%(levelname)s)',
        Style.RESET_ALL,
        ' ',
    ]
    asctime = [
        '[',
        Fore.BLACK,
        Style.DIM,
        Style.BRIGHT,
        '%(asctime)s',
        Fore.RESET,
        Style.RESET_ALL,
        ']',
    ]
    name = [
        ' ',
        Fore.WHITE,
        Style.DIM,
        Style.BRIGHT,
        '%(name)s',
        Fore.RESET,
        Style.RESET_ALL,
        ' ',
    ]

    tpl = "".join(reset + levelname + asctime + name + reset)

    return tpl


Instruction:
How would you explain what the following Python 3 function does
def merge_dict(base, additional):
    if base is None:
        return additional

    if additional is None:
        return base

    if not (
        isinstance(base, collections.Mapping)
        and isinstance(additional, collections.Mapping)
    ):
        return additional

    merged = base
    for key, value in additional.items():
        if isinstance(value, collections.Mapping):
            merged[key] = merge_dict(merged.get(key), value)
        else:
            merged[key] = value

    return merged

Response: Combine two dictionary - like objects.


Instruction:
Write a Python 3 script to
expand configuration XDG variables environmental variables and tildes.

Response:
def expand_config(d, dirs):
    """
    Expand configuration XDG variables, environmental variables, and tildes.

    Parameters
    ----------
    d : dict
        config information
    dirs : appdirs.AppDirs
        XDG application mapping

    Notes
    -----
    *Environmentable variables* are expanded via :py:func:`os.path.expandvars`.
    So ``${PWD}`` would be replaced by the current PWD in the shell,
    ``${USER}`` would be the user running the app.

    *XDG variables* are expanded via :py:meth:`str.format`. These do not have a
    dollar sign. They are:

    - ``{user_cache_dir}``
    - ``{user_config_dir}``
    - ``{user_data_dir}``
    - ``{user_log_dir}``
    - ``{site_config_dir}``
    - ``{site_data_dir}``

    See Also
    --------
    os.path.expanduser, os.path.expandvars :
        Standard library functions for expanding variables. Same concept, used inside.
    """
    context = {
        'user_cache_dir': dirs.user_cache_dir,
        'user_config_dir': dirs.user_config_dir,
        'user_data_dir': dirs.user_data_dir,
        'user_log_dir': dirs.user_log_dir,
        'site_config_dir': dirs.site_config_dir,
        'site_data_dir': dirs.site_data_dir,
    }

    for k, v in d.items():
        if isinstance(v, dict):
            expand_config(v, dirs)
        if isinstance(v, string_types):
            d[k] = os.path.expanduser(os.path.expandvars(d[k]))
            d[k] = d[k].format(**context)


Instruction:
How would you explain what the following Python 3 function does
def bootstrap_unihan(metadata, options={}):
    options = merge_dict(UNIHAN_ETL_DEFAULT_OPTIONS.copy(), options)

    p = unihan.Packager(options)
    p.download()
    data = p.export()
    table = create_unihan_table(UNIHAN_FIELDS, metadata)
    metadata.create_all()
    metadata.bind.execute(table.insert(), data)

Response: Download extract and import unihan to database.


Instruction:
Can you generate the documentation for the following Python 3 function
def is_bootstrapped(metadata):
    fields = UNIHAN_FIELDS + DEFAULT_COLUMNS
    if TABLE_NAME in metadata.tables.keys():
        table = metadata.tables[TABLE_NAME]

        if set(fields) == set(c.name for c in table.columns):
            return True
        else:
            return False
    else:
        return False

Response: Return True if cihai is correctly bootstrapped.


Instruction:
Can you implement a function in Python 3 that
creates a new table with columns and index.

Response:
def create_unihan_table(columns, metadata):
    """Create table and return  :class:`sqlalchemy.Table`.

    Parameters
    ----------
    columns : list
        columns for table, e.g. ``['kDefinition', 'kCantonese']``
    metadata : :class:`sqlalchemy.schema.MetaData`
        Instance of sqlalchemy metadata

    Returns
    -------
    :class:`sqlalchemy.schema.Table` :
        Newly created table with columns and index.
    """

    if TABLE_NAME not in metadata.tables:
        table = Table(TABLE_NAME, metadata)

        table.append_column(Column('char', String(12), primary_key=True))
        table.append_column(Column('ucn', String(12), primary_key=True))

        for column_name in columns:
            col = Column(column_name, String(256), nullable=True)
            table.append_column(col)

        return table
    else:
        return Table(TABLE_NAME, metadata)


Instruction:
Given the following Python 3 function, write the documentation
def get_address(pk, main_net=True, prefix=None):
    if isinstance(pk, str):
        pk = unhexlify(pk.encode())
    assert len(pk) == 32, 'PK is 32bytes {}'.format(len(pk))
    k = keccak_256(pk).digest()
    ripe = RIPEMD160.new(k).digest()
    if prefix is None:
        body = (b"\x68" if main_net else b"\x98") + ripe
    else:
        assert isinstance(prefix, bytes), 'Set prefix 1 bytes'
        body = prefix + ripe
    checksum = keccak_256(body).digest()[0:4]
    return b32encode(body + checksum).decode()

Response: compute the nem - py address from the public one


Instruction:
How would you explain what the following Python 3 function does
def lookup_char(self, char):
        Unihan = self.sql.base.classes.Unihan
        return self.sql.session.query(Unihan).filter_by(char=char)

Response: Return character information from datasets.


Instruction:
Implement a function in Python 3 to
return QuerySet of objects from SQLAlchemy of results.

Response:
def reverse_char(self, hints):
        """Return QuerySet of objects from SQLAlchemy of results.

        Parameters
        ----------
        hints: list of str
            strings to lookup

        Returns
        -------
        :class:`sqlalchemy.orm.query.Query` :
            reverse matches
        """
        if isinstance(hints, string_types):
            hints = [hints]

        Unihan = self.sql.base.classes.Unihan
        columns = Unihan.__table__.columns
        return self.sql.session.query(Unihan).filter(
            or_(*[column.contains(hint) for column in columns for hint in hints])
        )


Instruction:
How would you explain what the following Python 3 function does
def with_fields(self, *fields):
        Unihan = self.sql.base.classes.Unihan
        query = self.sql.session.query(Unihan)
        for field in fields:
            query = query.filter(Column(field).isnot(None))
        return query

Response: Returns list of characters with information for certain fields.


Instruction:
Can you tell what is the following Python 3 function doing
def entropy(data=None, prob=None, method='nearest-neighbors', bins=None, errorVal=1e-5, units='bits'):
    '''
    given a probability distribution (prob) or an interable of symbols (data) compute and
    return its continuous entropy.

    inputs:
    ------
        data:       samples by dimensions ndarray

        prob:       iterable with probabilities

        method:     'nearest-neighbors', 'gaussian', or 'bin'

        bins:       either a list of num_bins, or a list of lists containing
                    the bin edges

        errorVal:   if prob is given, 'entropy' checks that the sum is about 1.
                    It raises an error if abs(sum(prob)-1) >= errorVal

        units:      either 'bits' or 'nats'

        Different Methods:

        'nearest-neighbors' computes the binless entropy (bits) of a random vector
        using average nearest neighbors distance (Kozachenko and Leonenko, 1987).
        For a review see Beirlant et al., 2001 or Chandler & Field, 2007.

        'gaussian' computes the binless entropy based on estimating the covariance
        matrix and assuming the data is normally distributed.

        'bin' discretizes the data and computes the discrete entropy.

    '''

    if prob is None and data is None:
        raise ValueError("%s.entropy requires either 'prob' or 'data' to be defined" % __name__)

    if prob is not None and data is not None:
        raise ValueError("%s.entropy requires only 'prob' or 'data to be given but not both" % __name__)

    if prob is not None and not isinstance(prob, np.ndarray):
        raise TypeError("'entropy' in '%s' needs 'prob' to be an ndarray" % __name__)

    if prob is not None and abs(prob.sum()-1) > errorVal:
        raise ValueError("parameter 'prob' in '%s.entropy' should sum to 1" % __name__)

    if data is not None:
        num_samples = data.shape[0]
        if len(data.shape) == 1:
            num_dimensions = 1
        else:
            num_dimensions = data.shape[1]

    if method == 'nearest-neighbors':
        from sklearn.neighbors import NearestNeighbors
        from scipy.special import gamma

        if data is None:
            raise ValueError('Nearest neighbors entropy requires original data')

        if len(data.shape) > 1:
            k = num_dimensions
        else:
            k = 1

        nbrs = NearestNeighbors(n_neighbors=2, algorithm='auto').fit(data)
        distances, indices = nbrs.kneighbors(data)
        rho = distances[:,1] # take nearest-neighbor distance (first column is always zero)
        Ak  = (k*np.pi**(float(k)/float(2)))/gamma(float(k)/float(2)+1)

        if units is 'bits':
            # 0.577215... is the Euler-Mascheroni constant (np.euler_gamma)
            return k*np.mean(np.log2(rho)) + np.log2(num_samples*Ak/k) + np.log2(np.exp(1))*np.euler_gamma
        elif units is 'nats':
            # 0.577215... is the Euler-Mascheroni constant (np.euler_gamma)
            return k*np.mean(np.log(rho)) + np.log(num_samples*Ak/k) + np.log(np.exp(1))*np.euler_gamma
        else:
            print('Units not recognized: {}'.format(units))

    elif method == 'gaussian':
        from numpy.linalg import det

        if data is None:
            raise ValueError('Nearest neighbors entropy requires original data')

        detCov = det(np.dot(data.transpose(), data)/num_samples)
        normalization = (2*np.pi*np.exp(1))**num_dimensions

        if detCov == 0:
            return -np.inf
        else:
            if units is 'bits':
                return 0.5*np.log2(normalization*detCov)
            elif units is 'nats':
                return 0.5*np.log(normalization*detCov)
            else:
                print('Units not recognized: {}'.format(units))

    elif method == 'bin':
        if prob is None and bins is None:
            raise ValueError('Either prob or bins must be specified.')

        if data is not None:
            prob = symbols_to_prob(data, bins=bins)

        if units is 'bits':
            # compute the log2 of the probability and change any -inf by 0s
            logProb = np.log2(prob)
            logProb[logProb == -np.inf] = 0
        elif units is 'nats':
            # compute the log2 of the probability and change any -inf by 0s
            logProb = np.log(prob)
            logProb[logProb == -np.inf] = 0
        else:
            print('Units not recognized: {}'.format(units))

        # return sum of product of logProb and prob
        # (not using np.dot here because prob, logprob are nd arrays)
        return -float(np.sum(prob * logProb))

Response: Compute and return continuous entropy of a random vector.


Instruction:
Can you tell what is the following Python 3 function doing
def mi(x, y, bins_x=None, bins_y=None, bins_xy=None, method='nearest-neighbors', units='bits'):
    '''
    compute and return the mutual information between x and y

    inputs:
    -------
        x, y:       numpy arrays of shape samples x dimension
        method:     'nearest-neighbors', 'gaussian', or 'bin'
        units:      'bits' or 'nats'

    output:
    -------
        mi:     float

    Notes:
    ------
        if you are trying to mix several symbols together as in mi(x, (y0,y1,...)), try

        info[p] = _info.mi(x, info.combine_symbols(y0, y1, ...) )
    '''
    # dict.values() returns a view object that has to be converted to a list before being
    # converted to an array
    # the following lines will execute properly in python3, but not python2 because there
    # is no zip object
    try:
        if isinstance(x, zip):
            x = list(x)
        if isinstance(y, zip):
            y = list(y)
    except:
        pass

    # wrapped in try bracket because x, y might have no .shape attribute
    try:
        # handling for 1d np arrays
        if len(x.shape) == 1:
            x = np.expand_dims(x, 1)
        if len(y.shape) == 1:
            y = np.expand_dims(y, 1)
    except:
        pass

    HX  = entropy(data=x, bins=bins_x, method=method, units=units)
    HY  = entropy(data=y, bins=bins_y, method=method, units=units)
    HXY = entropy(data=np.concatenate([x, y], axis=1), bins=bins_xy, method=method, units=units)

    return HX + HY - HXY

Response: compute the mutual information between x and y


Instruction:
Explain what the following Python 3 code does
def cond_entropy(x, y, bins_y=None, bins_xy=None, method='nearest-neighbors', units='bits'):
    '''
    compute the conditional entropy H(X|Y).

    method:     'nearest-neighbors', 'gaussian', or 'bin'
                if 'bin' need to provide bins_y, and bins_xy
    units:      'bits' or 'nats'
    '''
    HXY = entropy(data=np.concatenate([x, y], axis=1), bins=bins_xy, method=method, units=units)
    HY  = entropy(data=y, bins=bins_y, method=method, units=units)

    return HXY - HY

Response: compute the conditional entropy of two arrays x and y


Instruction:
Implement a function in Python 3 to
compute and return the entropy of a single object

Response:
def entropy(data=None, prob=None, tol=1e-5):
    '''
    given a probability distribution (prob) or an interable of symbols (data) compute and
    return its entropy

    inputs:
    ------
        data:       iterable of symbols

        prob:       iterable with probabilities

        tol:        if prob is given, 'entropy' checks that the sum is about 1.
                    It raises an error if abs(sum(prob)-1) >= tol
    '''

    if prob is None and data is None:
        raise ValueError("%s.entropy requires either 'prob' or 'data' to be defined" % __name__)

    if prob is not None and data is not None:
        raise ValueError("%s.entropy requires only 'prob' or 'data to be given but not both" % __name__)

    if prob is not None and not isinstance(prob, np.ndarray):
        raise TypeError("'entropy' in '%s' needs 'prob' to be an ndarray" % __name__)

    if prob is not None and abs(prob.sum()-1) > tol:
        raise ValueError("parameter 'prob' in '%s.entropy' should sum to 1" % __name__)

    if data is not None:
        prob = symbols_to_prob(data).prob()

    # compute the log2 of the probability and change any -inf by 0s
    logProb = np.log2(prob)
    logProb[logProb == -np.inf] = 0

    # return dot product of logProb and prob
    return -float(np.dot(prob, logProb))


Instruction:
Can you generate the documentation for the following Python 3 function
def symbols_to_prob(symbols):
    '''
    Return a dict mapping symbols to  probability.

    input:
    -----
        symbols:     iterable of hashable items
                     works well if symbols is a zip of iterables
    '''
    myCounter = Counter(symbols)

    N = float(len(list(symbols)))   # symbols might be a zip object in python 3

    for k in myCounter:
        myCounter[k] /= N

    return myCounter

Response: Returns a dict mapping symbols to probability.


Instruction:
Write a Python 3 script to
combine different symbols into a super - symbol.

Response:
def combine_symbols(*args):
    '''
    Combine different symbols into a 'super'-symbol

    args can be an iterable of iterables that support hashing

    see example for 2D ndarray input

    usage:
        1) combine two symbols, each a number into just one symbol
        x = numpy.random.randint(0,4,1000)
        y = numpy.random.randint(0,2,1000)
        z = combine_symbols(x,y)

        2) combine a letter and a number
        s = 'abcd'
        x = numpy.random.randint(0,4,1000)
        y = [s[randint(4)] for i in range(1000)]
        z = combine_symbols(x,y)

        3) suppose you are running an experiment and for each sample, you measure 3 different
        properties and you put the data into a 2d ndarray such that:
            samples_N, properties_N = data.shape

        and you want to combine all 3 different properties into just 1 symbol
        In this case you have to find a way to impute each property as an independent array

            combined_symbol = combine_symbols(*data.T)

        4) if data from 3) is such that:
            properties_N, samples_N  = data.shape

        then run:

            combined_symbol = combine_symbols(*data)

    '''
    for arg in args:
        if len(arg)!=len(args[0]):
            raise ValueError("combine_symbols got inputs with different sizes")

    return tuple(zip(*args))


Instruction:
How would you explain what the following Python 3 function does
def mi(x, y):
    '''
    compute and return the mutual information between x and y

    inputs:
    -------
        x, y:   iterables of hashable items

    output:
    -------
        mi:     float

    Notes:
    ------
        if you are trying to mix several symbols together as in mi(x, (y0,y1,...)), try

        info[p] = _info.mi(x, info.combine_symbols(y0, y1, ...) )
    '''
    # dict.values() returns a view object that has to be converted to a list before being
    # converted to an array
    # the following lines will execute properly in python3, but not python2 because there
    # is no zip object
    try:
        if isinstance(x, zip):
            x = list(x)
        if isinstance(y, zip):
            y = list(y)
    except:
        pass

    probX = symbols_to_prob(x).prob()
    probY = symbols_to_prob(y).prob()
    probXY = symbols_to_prob(combine_symbols(x, y)).prob()

    return entropy(prob=probX) + entropy(prob=probY) - entropy(prob=probXY)

Response: compute and return the mutual information between x and y


Instruction:
Can you tell what is the following Python 3 function doing
def cond_mi(x, y, z):
    '''
    compute and return the mutual information between x and y given z, I(x, y | z)

    inputs:
    -------
        x, y, z:   iterables with discrete symbols

    output:
    -------
        mi:     float

    implementation notes:
    ---------------------
        I(x, y | z) = H(x | z) - H(x | y, z)
                    = H(x, z) - H(z) - ( H(x, y, z) - H(y,z) )
                    = H(x, z) + H(y, z) - H(z) - H(x, y, z)
    '''
    # dict.values() returns a view object that has to be converted to a list before being converted to an array
    probXZ = symbols_to_prob(combine_symbols(x, z)).prob()
    probYZ = symbols_to_prob(combine_symbols(y, z)).prob()
    probXYZ =symbols_to_prob(combine_symbols(x, y, z)).prob()
    probZ = symbols_to_prob(z).prob()

    return entropy(prob=probXZ) + entropy(prob=probYZ) - entropy(prob=probXYZ) - entropy(prob=probZ)

Response: compute and return the mutual information between x y and z given z


Instruction:
Make a summary of the following Python 3 code
def mi_chain_rule(X, y):
    '''
    Decompose the information between all X and y according to the chain rule and return all the terms in the chain rule.

    Inputs:
    -------
        X:          iterable of iterables. You should be able to compute [mi(x, y) for x in X]

        y:          iterable of symbols

    output:
    -------
        ndarray:    terms of chaing rule

    Implemenation notes:
        I(X; y) = I(x0, x1, ..., xn; y)
                = I(x0; y) + I(x1;y | x0) + I(x2; y | x0, x1) + ... + I(xn; y | x0, x1, ..., xn-1)
    '''

    # allocate ndarray output
    chain = np.zeros(len(X))

    # first term in the expansion is not a conditional information, but the information between the first x and y
    chain[0] = mi(X[0], y)

    for i in range(1, len(X)):
        chain[i] = cond_mi(X[i], y, X[:i])

    return chain

Response: Decompose the information between all X and y according to the chain rule and return all the terms in the chain rule.


Instruction:
Write a Python 3 script to
compute the KL divergence between distributions P and Q where P and Q should be dictionaries linking symbols to probabilities.

Response:
def KL_divergence(P,Q):
    '''
    Compute the KL divergence between distributions P and Q

    P and Q should be dictionaries linking symbols to probabilities.
    the keys to P and Q should be the same.
    '''
    assert(P.keys()==Q.keys())

    distance = 0
    for k in P.keys():
        distance += P[k] * log(P[k]/Q[k])

    return distance


Instruction:
Can you generate the documentation for the following Python 3 function
def bin(x, bins, maxX=None, minX=None):
    '''
    bin signal x using 'binsN' bin. If minX, maxX are None, they default to the full
    range of the signal. If they are not None, everything above maxX gets assigned to
    binsN-1 and everything below minX gets assigned to 0, this is effectively the same
    as clipping x before passing it to 'bin'

    input:
    -----
        x:      signal to be binned, some sort of iterable

        bins:   int, number of bins
                iterable, bin edges

        maxX:   clips data above maxX

        minX:   clips data below maxX

    output:
    ------
        binnedX:    x after being binned

        bins:       bins used for binning.
                    if input 'bins' is already an iterable it just returns the
                    same iterable

    example:
        # make 10 bins of equal length spanning from x.min() to x.max()
        bin(x, 10)

        # use predefined bins such that each bin has the same number of points (maximize
        entropy)
        binsN = 10
        percentiles = list(np.arange(0, 100.1, 100/binsN))
        bins = np.percentile(x, percentiles)
        bin(x, bins)
    '''
    if maxX is None:
        maxX = x.max()

    if minX is None:
        minX = x.min()

    if not np.iterable(bins):
        bins = np.linspace(minX, maxX+1e-5, bins+1)

    # digitize works on 1d array but not nd arrays.
    # So I pass the flattened version of x and then reshape back into x's original shape
    return np.digitize(x.ravel(), bins).reshape(x.shape), bins

Response: bin signal x using binsN bins


Instruction:
Implement a Python 3 function for
determining the URL corresponding to the Python object in the specified domain.

Response:
def linkcode_resolve(domain, info):  # NOQA: C901
    """
    Determine the URL corresponding to Python object

    Notes
    -----
    From https://github.com/numpy/numpy/blob/v1.15.1/doc/source/conf.py, 7c49cfa
    on Jul 31. License BSD-3. https://github.com/numpy/numpy/blob/v1.15.1/LICENSE.txt
    """
    if domain != 'py':
        return None

    modname = info['module']
    fullname = info['fullname']

    submod = sys.modules.get(modname)
    if submod is None:
        return None

    obj = submod
    for part in fullname.split('.'):
        try:
            obj = getattr(obj, part)
        except Exception:
            return None

    # strip decorators, which would resolve to the source of the decorator
    # possibly an upstream bug in getsourcefile, bpo-1764286
    try:
        unwrap = inspect.unwrap
    except AttributeError:
        pass
    else:
        obj = unwrap(obj)

    try:
        fn = inspect.getsourcefile(obj)
    except Exception:
        fn = None
    if not fn:
        return None

    try:
        source, lineno = inspect.getsourcelines(obj)
    except Exception:
        lineno = None

    if lineno:
        linespec = "#L%d-L%d" % (lineno, lineno + len(source) - 1)
    else:
        linespec = ""

    fn = relpath(fn, start=dirname(cihai.__file__))

    if 'dev' in about['__version__']:
        return "%s/blob/master/%s/%s%s" % (
            about['__github__'],
            about['__package_name__'],
            fn,
            linespec,
        )
    else:
        return "%s/blob/v%s/%s/%s%s" % (
            about['__github__'],
            about['__version__'],
            about['__package_name__'],
            fn,
            linespec,
        )


Instruction:
Make a summary of the following Python 3 code
def inv(z):
    # Adapted from curve25519_athlon.c in djb's Curve25519.
    z = qdiv(z)
    z2 = z * z % PRIME  # 2
    z9 = pow2(z2, 2) * z % PRIME  # 9
    z11 = z9 * z2 % PRIME  # 11
    z2_5_0 = (z11*z11) % PRIME * z9 % PRIME  # 31 == 2^5 - 2^0
    z2_10_0 = pow2(z2_5_0, 5) * z2_5_0 % PRIME  # 2^10 - 2^0
    z2_20_0 = pow2(z2_10_0, 10) * z2_10_0 % PRIME  # ...
    z2_40_0 = pow2(z2_20_0, 20) * z2_20_0 % PRIME
    z2_50_0 = pow2(z2_40_0, 10) * z2_10_0 % PRIME
    z2_100_0 = pow2(z2_50_0, 50) * z2_50_0 % PRIME
    z2_200_0 = pow2(z2_100_0, 100) * z2_100_0 % PRIME
    z2_250_0 = pow2(z2_200_0, 50) * z2_50_0 % PRIME  # 2^250 - 2^0
    return pow2(z2_250_0, 5) * z11 % PRIME

Response: Inverse of the curve25519 curve.


Instruction:
Can you generate the documentation for the following Python 3 function
def scalarmult_B(e):
    # scalarmult(B, l) is the identity
    e %= L
    P = IDENT
    for i in range(253):
        if e & 1:
            P = edwards_add(P=P, Q=Bpow[i])
        e //= 2
    assert e == 0, e
    return P

Response: Implements scalarmult B more efficiently.


Instruction:
Make a summary of the following Python 3 code
def recover(y):
    p = (y*y - 1) * inverse(D*y*y + 1)
    x = powmod(p, (PRIME+3) // 8, PRIME)
    if (x*x - p) % PRIME != 0:
        i = powmod(2, (PRIME-1) // 4, PRIME)
        x = (x*i) % PRIME
    if x % 2 != 0:
        x = PRIME - x
    return x

Response: given a value y recover the preimage x


Instruction:
Can you generate a brief explanation for the following Python 3 code
def __rubberband(y, sr, **kwargs):
    '''Execute rubberband

    Parameters
    ----------
    y : np.ndarray [shape=(n,) or (n, c)]
        Audio time series, either single or multichannel

    sr : int > 0
        sampling rate of y

    **kwargs
        keyword arguments to rubberband

    Returns
    -------
    y_mod : np.ndarray [shape=(n,) or (n, c)]
        `y` after rubberband transformation

    '''

    assert sr > 0

    # Get the input and output tempfile
    fd, infile = tempfile.mkstemp(suffix='.wav')
    os.close(fd)
    fd, outfile = tempfile.mkstemp(suffix='.wav')
    os.close(fd)

    # dump the audio
    sf.write(infile, y, sr)

    try:
        # Execute rubberband
        arguments = [__RUBBERBAND_UTIL, '-q']

        for key, value in six.iteritems(kwargs):
            arguments.append(str(key))
            arguments.append(str(value))

        arguments.extend([infile, outfile])

        subprocess.check_call(arguments, stdout=DEVNULL, stderr=DEVNULL)

        # Load the processed audio.
        y_out, _ = sf.read(outfile, always_2d=True)

        # make sure that output dimensions matches input
        if y.ndim == 1:
            y_out = np.squeeze(y_out)

    except OSError as exc:
        six.raise_from(RuntimeError('Failed to execute rubberband. '
                                    'Please verify that rubberband-cli '
                                    'is installed.'),
                       exc)

    finally:
        # Remove temp files
        os.unlink(infile)
        os.unlink(outfile)

    return y_out

Response: Execute rubberband on the input time series y and return the output time series y after rubberband transformation


Instruction:
Implement a Python 3 function for
applying a time stretch of rate to an audio time series.

Response:
def time_stretch(y, sr, rate, rbargs=None):
    '''Apply a time stretch of `rate` to an audio time series.

    This uses the `tempo` form for rubberband, so the
    higher the rate, the faster the playback.

    Parameters
    ----------
    y : np.ndarray [shape=(n,) or (n, c)]
        Audio time series, either single or multichannel

    sr : int > 0
        Sampling rate of `y`

    rate : float > 0
        Desired playback rate.

    rbargs
        Additional keyword parameters for rubberband

        See `rubberband -h` for details.

    Returns
    -------
    y_stretch : np.ndarray
        Time-stretched audio

    Raises
    ------
    ValueError
        if `rate <= 0`
    '''

    if rate <= 0:
        raise ValueError('rate must be strictly positive')

    if rate == 1.0:
        return y

    if rbargs is None:
        rbargs = dict()

    rbargs.setdefault('--tempo', rate)

    return __rubberband(y, sr, **rbargs)


Instruction:
Explain what the following Python 3 code does
def timemap_stretch(y, sr, time_map, rbargs=None):
    '''Apply a timemap stretch to an audio time series.

    A timemap stretch allows non-linear time-stretching by mapping source to
    target sample frame numbers for fixed time points within the audio data.

    This uses the `time` and `timemap` form for rubberband.

    Parameters
    ----------
    y : np.ndarray [shape=(n,) or (n, c)]
        Audio time series, either single or multichannel

    sr : int > 0
        Sampling rate of `y`

    time_map : list
        Each element is a tuple `t` of length 2 which corresponds to the
        source sample position and target sample position.

        If `t[1] < t[0]` the track will be sped up in this area.

        `time_map[-1]` must correspond to the lengths of the source audio and
        target audio.

    rbargs
        Additional keyword parameters for rubberband

        See `rubberband -h` for details.

    Returns
    -------
    y_stretch : np.ndarray
        Time-stretched audio

    Raises
    ------
    ValueError
        if `time_map` is not monotonic
        if `time_map` is not non-negative
        if `time_map[-1][0]` is not the input audio length
    '''

    if rbargs is None:
        rbargs = dict()

    is_positive = all(time_map[i][0] >= 0 and time_map[i][1] >= 0
                      for i in range(len(time_map)))
    is_monotonic = all(time_map[i][0] <= time_map[i+1][0] and
                       time_map[i][1] <= time_map[i+1][1]
                       for i in range(len(time_map)-1))
    if not is_positive:
        raise ValueError('time_map should be non-negative')

    if not is_monotonic:
        raise ValueError('time_map is not monotonic')

    if time_map[-1][0] != len(y):
        raise ValueError('time_map[-1] should correspond to the last sample')

    time_stretch = time_map[-1][1] * 1.0 / time_map[-1][0]
    rbargs.setdefault('--time', time_stretch)

    stretch_file = tempfile.NamedTemporaryFile(mode='w', suffix='.txt',
                                               delete=False)
    try:
        for t in time_map:
            stretch_file.write('{:0} {:1}\n'.format(t[0], t[1]))
        stretch_file.close()

        rbargs.setdefault('--timemap', stretch_file.name)
        y_stretch = __rubberband(y, sr, **rbargs)
    finally:
        # Remove temp file
        os.unlink(stretch_file.name)

    return y_stretch

Response: Apply a timemap stretch to an audio time series.


Instruction:
Write a Python 3 function that can
apply a pitch shift to an audio time series.

Response:
def pitch_shift(y, sr, n_steps, rbargs=None):
    '''Apply a pitch shift to an audio time series.

    Parameters
    ----------
    y : np.ndarray [shape=(n,) or (n, c)]
        Audio time series, either single or multichannel

    sr : int > 0
        Sampling rate of `y`

    n_steps : float
        Shift by `n_steps` semitones.

    rbargs
        Additional keyword parameters for rubberband

        See `rubberband -h` for details.

    Returns
    -------
    y_shift : np.ndarray
        Pitch-shifted audio
    '''

    if n_steps == 0:
        return y

    if rbargs is None:
        rbargs = dict()

    rbargs.setdefault('--pitch', n_steps)

    return __rubberband(y, sr, **rbargs)


Instruction:
Write a Python 3 function that can
convert a kuten string to GB2312 - 1980 hex

Response:
def kuten_to_gb2312(kuten):
    """
    Convert GB kuten / quwei form (94 zones * 94 points) to GB2312-1980 /
    ISO-2022-CN hex (internal representation)
    """
    zone, point = int(kuten[:2]), int(kuten[2:])
    hi, lo = hexd(zone + 0x20), hexd(point + 0x20)

    gb2312 = "%s%s" % (hi, lo)

    assert isinstance(gb2312, bytes)
    return gb2312


Instruction:
Here you have a function in Python 3, explain what it does
def gb2312_to_euc(gb2312hex):
    hi, lo = int(gb2312hex[:2], 16), int(gb2312hex[2:], 16)
    hi, lo = hexd(hi + 0x80), hexd(lo + 0x80)

    euc = "%s%s" % (hi, lo)
    assert isinstance(euc, bytes)
    return euc

Response: Convert a GB2312 - 1980 hex string to EUC - CN hex string


Instruction:
Given the following Python 3 function, write the documentation
def euc_to_python(hexstr):
    hi = hexstr[0:2]
    lo = hexstr[2:4]
    gb_enc = b'\\x' + hi + b'\\x' + lo
    return gb_enc.decode("gb2312")

Response: Convert a EUC - CN hex string to a Python unicode string.


Instruction:
Can you generate the documentation for the following Python 3 function
def euc_to_utf8(euchex):
    utf8 = euc_to_python(euchex).encode("utf-8")
    uf8 = utf8.decode('unicode_escape')

    uf8 = uf8.encode('latin1')

    uf8 = uf8.decode('euc-jp')
    return uf8

Response: Convert EUC hex to UTF8 hex.


Instruction:
Can you generate the documentation for the following Python 3 function
def ucn_to_unicode(ucn):
    if isinstance(ucn, string_types):
        ucn = ucn.strip("U+")
        if len(ucn) > int(4):
            char = b'\U' + format(int(ucn, 16), '08x').encode('latin1')
            char = char.decode('unicode_escape')
        else:
            char = unichr(int(ucn, 16))
    else:
        char = unichr(ucn)

    assert isinstance(char, text_type)

    return char

Response: Convert a Unicode Universal Character Number to a Python unicode character.


Instruction:
Explain what the following Python 3 code does
def euc_to_unicode(hexstr):
    hi = hexstr[0:2]
    lo = hexstr[2:4]
    # hi and lo are only 2 characters long, no risk with eval-ing them

    gb_enc = b'\\x' + hi + b'\\x' + lo
    assert isinstance(gb_enc, bytes)

    # Requires coercing back to text_type in 2.7
    gb_enc = gb_enc.decode('unicode_escape')

    gb_enc = gb_enc.encode('latin1')

    gb_enc = gb_enc.decode('gb2312')

    assert isinstance(gb_enc, text_type)
    return gb_enc

Response: Convert an EUC - CN hex string to a Python unicode string.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def python_to_ucn(uni_char, as_bytes=False):
    ucn = uni_char.encode('unicode_escape').decode('latin1')
    ucn = text_type(ucn).replace('\\', '').upper().lstrip('U')
    if len(ucn) > int(4):
        # get rid of the zeroes that Python uses to pad 32 byte UCNs
        ucn = ucn.lstrip("0")
    ucn = "U+" + ucn.upper()

    if as_bytes:
        ucn = ucn.encode('latin1')

    return ucn

Response: Converts a Python Unicode character to a UCN character.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def python_to_euc(uni_char, as_bytes=False):
    euc = repr(uni_char.encode("gb2312"))[1:-1].replace("\\x", "").strip("'")

    if as_bytes:
        euc = euc.encode('utf-8')
        assert isinstance(euc, bytes)

    return euc

Response: Converts a Python Unicode character to an EUC character.


Instruction:
Create a Python 3 function for
returning ucnstring as Unicode.

Response:
def ucnstring_to_unicode(ucn_string):
    """Return ucnstring as Unicode."""
    ucn_string = ucnstring_to_python(ucn_string).decode('utf-8')

    assert isinstance(ucn_string, text_type)
    return ucn_string


Instruction:
Can you generate the documentation for the following Python 3 function
def ucnstring_to_python(ucn_string):
    res = re.findall("U\+[0-9a-fA-F]*", ucn_string)
    for r in res:
        ucn_string = ucn_string.replace(text_type(r), text_type(ucn_to_unicode(r)))

    ucn_string = ucn_string.encode('utf-8')

    assert isinstance(ucn_string, bytes)
    return ucn_string

Response: Convert Unicode UCN to native Python Unicode Unicode


Instruction:
Here you have a function in Python 3, explain what it does
def parse_var(var):
    bits = var.split("<", 1)
    if len(bits) < 2:
        tag = None
    else:
        tag = bits[1]
    return ucn_to_unicode(bits[0]), tag

Response: Parse a UCN - style variable into a tuple consisting of a string and a tag.


Instruction:
How would you implement a function in Python 3 that
creates a Cihai instance from a JSON or YAML config file.

Response:
def from_file(cls, config_path=None, *args, **kwargs):
        """
        Create a Cihai instance from a JSON or YAML config.

        Parameters
        ----------
        config_path : str, optional
            path to custom config file

        Returns
        -------
        :class:`Cihai` :
            application object
        """

        config_reader = kaptan.Kaptan()

        config = {}

        if config_path:
            if not os.path.exists(config_path):
                raise exc.CihaiException(
                    '{0} does not exist.'.format(os.path.abspath(config_path))
                )
            if not any(
                config_path.endswith(ext) for ext in ('json', 'yml', 'yaml', 'ini')
            ):
                raise exc.CihaiException(
                    '{0} does not have a yaml,yml,json,ini extend.'.format(
                        os.path.abspath(config_path)
                    )
                )
            else:
                custom_config = config_reader.import_config(config_path).get()
                config = merge_dict(config, custom_config)

        return cls(config)


Instruction:
Can you generate the documentation for the following Python 3 function
def _process_locale(self, locale):

        if locale.lower().startswith('en'):
            return False

        return (locale in self.enabled_locales or

                self.reverse_locale_map.get(locale.lower(), None)
                in self.enabled_locales or

                locale in self.lower_locales or

                self.reverse_locale_map.get(locale.lower(), None)
                in self.lower_locales
        )

Response: Return True if this locale should be processed.


Instruction:
Write a Python 3 function for
returning the Desk - style locale for locale.

Response:
def desk_locale(self, locale):
        """Return the Desk-style locale for locale."""

        locale = locale.lower().replace('-', '_')
        return self.vendor_locale_map.get(locale, locale)


Instruction:
Explain what the following Python 3 code does
def push(self):

        tx = Tx(self.tx_project_slug)

        # asssemble the template catalog
        template = babel.messages.catalog.Catalog()
        for topic in self.desk.topics():
            if topic.show_in_portal:
                template.add(topic.name)

        # serialize the catalog as a PO file
        template_po = StringIO()
        babel.messages.pofile.write_po(template_po, template)

        # upload/update the catalog resource
        tx.create_or_update_resource(
            self.TOPIC_STRINGS_SLUG,
            DEFAULT_SOURCE_LANGUAGE,
            "Help Center Topics",
            template_po.getvalue(),
            i18n_type='PO',
            project_slug=self.tx_project_slug,
        )

Response: Push topics to Transifex.


Instruction:
Can you tell what is the following Python 3 function doing
def pull(self):

        topic_stats = txlib.api.statistics.Statistics.get(
            project_slug=self.tx_project_slug,
            resource_slug=self.TOPIC_STRINGS_SLUG,
        )

        translated = {}

        # for each language
        for locale in self.enabled_locales:

            if not self._process_locale(locale):
                continue

            locale_stats = getattr(topic_stats, locale, None)
            if locale_stats is None:
                self.log.debug('Locale %s not present when pulling topics.' %
                               (locale,))
                continue

            if locale_stats['completed'] == '100%':
                # get the resource from Tx
                translation = txlib.api.translations.Translation.get(
                    project_slug=self.tx_project_slug,
                    slug=self.TOPIC_STRINGS_SLUG,
                    lang=locale,
                )

                translated[locale] = babel.messages.pofile.read_po(
                    StringIO(translation.content.encode('utf-8'))
                )

        # now that we've pulled everything from Tx, upload to Desk
        for topic in self.desk.topics():

            for locale in translated:

                if topic.name in translated[locale]:

                    self.log.debug(
                        'Updating topic (%s) for locale (%s)' %
                        (topic.name, locale),
                    )

                    if locale in topic.translations:
                        topic.translations[locale].update(
                            name=translated[locale][topic.name].string,
                        )
                    else:
                        topic.translations.create(
                            locale=locale,
                            name=translated[locale][topic.name].string,
                        )
                else:

                    self.log.error(
                        'Topic name (%s) does not exist in locale (%s)' %
                        (topic['name'], locale),
                    )

Response: Pulls topics from Transifex and uploads them to Desk.


Instruction:
Explain what the following Python 3 code does
def make_resource_document(self, title, content, tags=[],):

        assert "<html>" not in content
        assert "<body>" not in content

        return """
        <html>
        <head><title>%(title)s</title></head>
        <body>
        %(content)s
        </body>
        """ % dict(
            title=title,
            content=content,
        )

Response: Return a single HTML document containing the title and content.


Instruction:
Write a Python 3 script for
parsing the resource document.

Response:
def parse_resource_document(self, content):
        """Return a dict with the keys title, content, tags for content."""

        content = content.strip()

        if not content.startswith('<html>'):
            # this is not a full HTML doc, probably content w/o title, tags, etc
            return dict(body=content)

        result = {}
        if '<title>' in content and '</title>' in content:
            result['subject'] = content[content.find('<title>') + 7:content.find('</title>')].strip()
        result['body'] = content[content.find('<body>') + 6:content.find('</body>')].strip()

        return result


Instruction:
Write a Python 3 function for
pushing tutorials to Transifex.

Response:
def push(self):
        """Push tutorials to Transifex."""

        tx = Tx(self.tx_project_slug)

        if self.options.resources:
            articles = [
                self.desk.articles().by_id(r.strip())
                for r in self.options.resources.split(',')
            ]
        else:
            articles = self.desk.articles()

        for a in articles:

            self.log.debug(
                'Inspecting Desk resource %s', a.api_href
            )

            for translation in a.translations.items().values():
                our_locale = self.desk_to_our_locale(translation.locale)

                self.log.debug('Checking locale %s', translation.locale)

                if not self._process_locale(translation.locale):
                    self.log.debug('Skipping locale.')
                    continue

                # make sure the project exists in Tx
                tx.get_project(our_locale)

                a_id = a.api_href.rsplit('/', 1)[1]
                if (self.options.force or
                    not tx.resource_exists(a_id, our_locale) or
                    translation.outdated
                ):
                    self.log.info('Resource %(id)s out of date in %(locale)s; updating.' %
                             {'id': a_id,
                              'locale': our_locale,
                              },
                    )

                    tx.create_or_update_resource(
                        a_id,
                        our_locale,
                        self.make_resource_title(a),
                        self.make_resource_document(a.subject, a.body),
                    )


Instruction:
Create a Python 3 function to
get or creates a Transifex project for the current project prefix and locale

Response:
def get_project(self, locale, source_language_code=DEFAULT_SOURCE_LANGUAGE, **kwargs):
        """
        Gets or creates the Transifex project for the current project prefix and locale

        :param locale: A locale to which content is to be translated
        :type locale: string
        :param source_language_code: The language of the original untranslated content (i.e. Spanish),
            defaults to DEFAULT_SOURCE_LANGUAGE, which is English
        :type source_language_code: string, optional
        :return: The Transifex project to which resources can be pushed or pulled
        :rtype: project.Project
        """

        try:
            locale_project = project.Project.get(slug=self.get_project_slug(locale))

        except NotFoundError:

            locale_project = project.Project(
                slug=self.get_project_slug(locale),
            )
            defaults = {
                'name': 'Help Center (%s)' % (locale, ),
                'description': 'Help Center pages to translate to %s' % (
                    locale,
                ),
                'source_language_code': source_language_code,
                'private': True,
            }

            valid_keys = ('name','description')
            defaults.update(
                dict((k,v) for k,v in kwargs.iteritems() if k in valid_keys)
            )

            for k,v in defaults.iteritems():
                setattr(locale_project, k, v)

            locale_project.save()

        return locale_project


Instruction:
Can you create a Python 3 function that
returns True if the translation exists for this slug.

Response:
def translation_exists(self, slug, lang):
        """Return True if the translation exists for this slug."""

        try:
            return translations.Translation.get(
                project_slug=self.get_project_slug(lang),
                slug=slug,
                lang=lang,
            )

        except (NotFoundError, RemoteServerError):
            pass

        return False


Instruction:
How would you code a function in Python 3 to
return a sequence of resources for a given lang.

Response:
def list_resources(self, lang):
        """Return a sequence of resources for a given lang.

        Each Resource is a dict containing the slug, name, i18n_type,
        source_language_code and the category.
        """

        return registry.registry.http_handler.get(
            '/api/2/project/%s/resources/' % (
                self.get_project_slug(lang),)
        )


Instruction:
Here you have a function in Python 3, explain what it does
def resources(self, lang, slug):

        resource = resources.Resource.get(
            project_slug=self.get_project_slug(lang),
            slug=slug,
        )

        return resource

Response: Generate a list of Resources in the Project.


Instruction:
Write a Python 3 script for
returning True if a Resource with the given slug exists in locale.

Response:
def resource_exists(self, slug, locale, project_slug=None):
        """Return True if a Resource with the given slug exists in locale."""

        try:
            resource = resources.Resource.get(
                project_slug=project_slug or self.get_project_slug(locale),
                slug=slug,
            )

            return resource

        except NotFoundError:
            pass

        return None


Instruction:
Implement a Python 3 function for
doing HTTP request and return response as a string.

Response:
def session_request(session, url, **kwargs):
    """Do HTTP/S request and return response as a string."""
    try:
        response = session(url, **kwargs)

        response.raise_for_status()

        return response.text

    except requests.exceptions.HTTPError as errh:
        _LOGGER.debug("%s, %s", response, errh)
        raise_error(response.status_code)

    except requests.exceptions.ConnectionError as errc:
        _LOGGER.debug("%s", errc)
        raise RequestError("Connection error: {}".format(errc))

    except requests.exceptions.Timeout as errt:
        _LOGGER.debug("%s", errt)
        raise RequestError("Timeout: {}".format(errt))

    except requests.exceptions.RequestException as err:
        _LOGGER.debug("%s", err)
        raise RequestError("Unknown error: {}".format(err))


Instruction:
Make a summary of the following Python 3 code
def get_event_list(config):
    eventinstances = session_request(
        config.session.post, device_event_url.format(
            proto=config.web_proto, host=config.host, port=config.port),
        auth=config.session.auth, headers=headers, data=request_xml)

    raw_event_list = _prepare_event(eventinstances)

    event_list = {}
    for entry in MAP + METAMAP:
        instance = raw_event_list
        try:
            for item in sum(entry[MAP_BASE].values(), []):
                instance = instance[item]
        except KeyError:
            continue
        event_list[entry[MAP_TYPE]] = instance

    return event_list

Response: Get a dict of supported events from device.


Instruction:
Implement a function in Python 3 to
convert an event instance list to a relevant dictionary.

Response:
def _prepare_event(eventinstances):
    """Converts event instances to a relevant dictionary."""
    import xml.etree.ElementTree as ET

    def parse_event(events):
        """Find all events inside of an topicset list.

        MessageInstance signals that subsequent children will
        contain source and data descriptions.
        """

        def clean_attrib(attrib={}):
            """Clean up child attributes by removing XML namespace."""
            attributes = {}
            for key, value in attrib.items():
                attributes[key.split('}')[-1]] = value
            return attributes

        description = {}
        for child in events:
            child_tag = child.tag.split('}')[-1]
            child_attrib = clean_attrib(child.attrib)
            if child_tag != 'MessageInstance':
                description[child_tag] = {
                    **child_attrib, **parse_event(child)}
            elif child_tag == 'MessageInstance':
                description = {}
                for item in child:
                    tag = item.tag.split('}')[-1]
                    description[tag] = clean_attrib(item[0].attrib)
        return description

    root = ET.fromstring(eventinstances)
    return parse_event(root[0][0][0])


Instruction:
Given the following Python 3 function, write the documentation
def url(self):
        return URL.format(http=self.web_proto, host=self.host, port=self.port)

Response: Represent device base url.


Instruction:
Write a Python 3 script for
processing the raw dict.

Response:
def process_raw(self, raw: dict) -> None:
        """Pre-process raw dict.

        Prepare parameters to work with APIItems.
        """
        raw_ports = {}

        for param in raw:
            port_index = REGEX_PORT_INDEX.search(param).group(0)

            if port_index not in raw_ports:
                raw_ports[port_index] = {}

            name = param.replace(IOPORT + '.I' + port_index + '.', '')
            raw_ports[port_index][name] = raw[param]

        super().process_raw(raw_ports)


Instruction:
Here you have a function in Python 3, explain what it does
def name(self) -> str:
        if self.direction == DIRECTION_IN:
            return self.raw.get('Input.Name', '')
        return self.raw.get('Output.Name', '')

Response: Return name relevant to direction.


Instruction:
Given the following Python 3 function, write the documentation
def initialize_params(self, preload_data=True) -> None:
        params = ''
        if preload_data:
            params = self.request('get', param_url)

        self.params = Params(params, self.request)

Response: Load device parameters and initialize parameter management.


Instruction:
Can you generate the documentation for the following Python 3 function
def initialize_ports(self) -> None:
        if not self.params:
            self.initialize_params(preload_data=False)
            self.params.update_ports()

        self.ports = Ports(self.params, self.request)

Response: Load IO port parameters for device.


Instruction:
Given the following Python 3 function, write the documentation
def initialize_users(self) -> None:
        users = self.request('get', pwdgrp_url)
        self.users = Users(users, self.request)

Response: Load device user data and initialize user management.


Instruction:
How would you explain what the following Python 3 function does
def new_event(self, event_data: str) -> None:
        event = self.parse_event_xml(event_data)

        if EVENT_OPERATION in event:
            self.manage_event(event)

Response: Process a new event.


Instruction:
Write a Python 3 function that can
receive new metadata. Operation initialized means new event, also happens if reconnecting. Operation changed updates existing events state.

Response:
def manage_event(self, event) -> None:
        """Received new metadata.

        Operation initialized means new event, also happens if reconnecting.
        Operation changed updates existing events state.
        """
        name = EVENT_NAME.format(
            topic=event[EVENT_TOPIC], source=event.get(EVENT_SOURCE_IDX))

        if event[EVENT_OPERATION] == 'Initialized' and name not in self.events:

            for event_class in EVENT_CLASSES:
                if event_class.TOPIC in event[EVENT_TOPIC]:
                    self.events[name] = event_class(event)
                    self.signal('add', name)
                    return

            _LOGGER.debug('Unsupported event %s', event[EVENT_TOPIC])

        elif event[EVENT_OPERATION] == 'Changed' and name in self.events:
            self.events[name].state = event[EVENT_VALUE]


Instruction:
Write a Python 3 script for
updating state of event.

Response:
def state(self, state: str) -> None:
        """Update state of event."""
        self._state = state
        for callback in self._callbacks:
            callback()


Instruction:
Make a summary of the following Python 3 code
def remove_callback(self, callback) -> None:
        if callback in self._callbacks:
            self._callbacks.remove(callback)

Response: Remove callback from the list of callbacks.


Instruction:
How would you explain what the following Python 3 function does
def enable_events(self, event_callback=None) -> None:
        self.event = EventManager(event_callback)
        self.stream.event = self.event

Response: Enable events for stream.


Instruction:
Explain what the following Python 3 code does
def update_brand(self) -> None:
        self.update(path=URL_GET + GROUP.format(group=BRAND))

Response: Update brand group of parameters.


Instruction:
Can you generate the documentation for the following Python 3 function
def update_ports(self) -> None:
        self.update(path=URL_GET + GROUP.format(group=INPUT))
        self.update(path=URL_GET + GROUP.format(group=IOPORT))
        self.update(path=URL_GET + GROUP.format(group=OUTPUT))

Response: Update port groups of parameters.


Instruction:
Write a Python 3 function for
creating a smaller dictionary containing all ports.

Response:
def ports(self) -> dict:
        """Create a smaller dictionary containing all ports."""
        return {
            param: self[param].raw
            for param in self
            if param.startswith(IOPORT)
        }


Instruction:
Explain what the following Python 3 code does
def update_properties(self) -> None:
        self.update(path=URL_GET + GROUP.format(group=PROPERTIES))

Response: Update properties group of parameters.


Instruction:
Can you generate the documentation for the following Python 3 function
def process_raw(self, raw: str) -> None:
        raw_params = dict(group.split('=', 1) for group in raw.splitlines())

        super().process_raw(raw_params)

Response:
Pre - process raw string.

        Prepare parameters to work with APIItems.


Instruction:
Can you create a Python 3 function that
removes user from the cache.

Response:
def delete(self, user: str) -> None:
        """Remove user."""
        data = {
            'action': 'remove',
            'user': user
        }

        self._request('post', URL, data=data)


Instruction:
Make a summary of the following Python 3 code
def process_raw(self, raw: str) -> None:
        raw_dict = dict(group.split('=') for group in raw.splitlines())

        raw_users = {
            user: {
                group: user in REGEX_STRING.findall(raw_dict[group])
                for group in [ADMIN, OPERATOR, VIEWER, PTZ]
            }
            for user in REGEX_STRING.findall(raw_dict['users'])
        }

        super().process_raw(raw_users)

Response:
Pre-process raw string.

        Prepare users to work with APIItems.
        Create booleans with user levels.


Instruction:
How would you explain what the following Python 3 function does
def init_done(self, fut):
        try:
            if fut.exception():
                fut.result()
        except OSError as err:
            _LOGGER.debug('RTSP got exception %s', err)
            self.stop()
            self.callback(SIGNAL_FAILED)

Response: Callback when the init process is complete.


Instruction:
Given the following Python 3 function, write the documentation
def connection_made(self, transport):
        self.transport = transport
        self.transport.write(self.method.message.encode())
        self.time_out_handle = self.loop.call_later(
            TIME_OUT_LIMIT, self.time_out)

Response: Connect to device is successful.


Instruction:
Make a summary of the following Python 3 code
def data_received(self, data):
        self.time_out_handle.cancel()
        self.session.update(data.decode())

        if self.session.state == STATE_STARTING:
            self.transport.write(self.method.message.encode())
            self.time_out_handle = self.loop.call_later(
                TIME_OUT_LIMIT, self.time_out)

        elif self.session.state == STATE_PLAYING:
            self.callback(SIGNAL_PLAYING)

            if self.session.session_timeout != 0:
                interval = self.session.session_timeout - 5
                self.loop.call_later(interval, self.keep_alive)

        else:
            self.stop()

Response: Called when data is received from the server.


Instruction:
How would you implement a function in Python 3 that
stops the RTSP session and send a SIGAL_FAILED signal.

Response:
def time_out(self):
        """If we don't get a response within time the RTSP request time out.

        This usually happens if device isn't available on specified IP.
        """
        _LOGGER.warning('Response timed out %s', self.session.host)
        self.stop()
        self.callback(SIGNAL_FAILED)


Instruction:
Can you write a function in Python 3 where it
returns RTSP method based on sequence number from session.

Response:
def message(self):
        """Return RTSP method based on sequence number from session."""
        message = self.message_methods[self.session.method]()
        _LOGGER.debug(message)
        return message


Instruction:
Can you create a Python 3 function that
requests options device supports.

Response:
def OPTIONS(self, authenticate=True):
        """Request options device supports."""
        message = "OPTIONS " + self.session.url + " RTSP/1.0\r\n"
        message += self.sequence
        message += self.authentication if authenticate else ''
        message += self.user_agent
        message += self.session_id
        message += '\r\n'
        return message


Instruction:
Can you generate the documentation for the following Python 3 function
def DESCRIBE(self):
        message = "DESCRIBE " + self.session.url + " RTSP/1.0\r\n"
        message += self.sequence
        message += self.authentication
        message += self.user_agent
        message += "Accept: application/sdp\r\n"
        message += '\r\n'
        return message

Response: Request description of what services RTSP server make available.


Instruction:
Can you tell what is the following Python 3 function doing
def SETUP(self):
        message = "SETUP " + self.session.control_url + " RTSP/1.0\r\n"
        message += self.sequence
        message += self.authentication
        message += self.user_agent
        message += self.transport
        message += '\r\n'
        return message

Response: Set up stream transport.


Instruction:
Given the following Python 3 function, write the documentation
def transport(self):
        transport = "Transport: RTP/AVP;unicast;client_port={}-{}\r\n"
        return transport.format(
            str(self.session.rtp_port), str(self.session.rtcp_port))

Response: Generate the transport string.


Instruction:
How would you implement a function in Python 3 that
updates the internal state of the object from the device response.

Response:
def update(self, response):
        """Update session information from device response.

        Increment sequence number when starting stream, not when playing.
        If device requires authentication resend previous message with auth.
        """
        data = response.splitlines()
        _LOGGER.debug('Received data %s from %s', data, self.host)
        while data:
            line = data.pop(0)
            if 'RTSP/1.0' in line:
                self.rtsp_version = int(line.split(' ')[0][5])
                self.status_code = int(line.split(' ')[1])
                self.status_text = line.split(' ')[2]
            elif 'CSeq' in line:
                self.sequence_ack = int(line.split(': ')[1])
            elif 'Date' in line:
                self.date = line.split(': ')[1]
            elif 'Public' in line:
                self.methods_ack = line.split(': ')[1].split(', ')
            elif "WWW-Authenticate: Basic" in line:
                self.basic = True
                self.realm = line.split('"')[1]
            elif "WWW-Authenticate: Digest" in line:
                self.digest = True
                self.realm = line.split('"')[1]
                self.nonce = line.split('"')[3]
                self.stale = (line.split('stale=')[1] == 'TRUE')
            elif 'Content-Type' in line:
                self.content_type = line.split(': ')[1]
            elif 'Content-Base' in line:
                self.content_base = line.split(': ')[1]
            elif 'Content-Length' in line:
                self.content_length = int(line.split(': ')[1])
            elif 'Session' in line:
                self.session_id = line.split(': ')[1].split(";")[0]
                if '=' in line:
                    self.session_timeout = int(line.split(': ')[1].split('=')[1])
            elif 'Transport' in line:
                self.transport_ack = line.split(': ')[1]
            elif 'Range' in line:
                self.range = line.split(': ')[1]
            elif 'RTP-Info' in line:
                self.rtp_info = line.split(': ')[1]
            elif not line:
                if data:
                    self.sdp = data
                    break
        if self.sdp:
            stream_found = False
            for param in self.sdp:
                if not stream_found and 'm=application' in param:
                    stream_found = True
                elif stream_found and 'a=control:rtsp' in param:
                    self.control_url = param.split(':', 1)[1]
                    break

        if self.status_code == 200:
            if self.state == STATE_STARTING:
                self.sequence += 1
        elif self.status_code == 401:
            # Device requires authorization, do not increment to next method
            pass
        else:
            # If device configuration is correct we should never get here
            _LOGGER.debug(
                "%s RTSP %s %s", self.host, self.status_code, self.status_text)


Instruction:
Here you have a function in Python 3, explain what it does
def generate_digest(self):
        from hashlib import md5
        ha1 = self.username + ':' + self.realm + ':' + self.password
        HA1 = md5(ha1.encode('UTF-8')).hexdigest()
        ha2 = self.method + ':' + self.url
        HA2 = md5(ha2.encode('UTF-8')).hexdigest()
        encrypt_response = HA1 + ':' + self.nonce + ':' + HA2
        response = md5(encrypt_response.encode('UTF-8')).hexdigest()

        digest_auth = 'Digest '
        digest_auth += 'username=\"' + self.username + "\", "
        digest_auth += 'realm=\"' + self.realm + "\", "
        digest_auth += "algorithm=\"MD5\", "
        digest_auth += 'nonce=\"' + self.nonce + "\", "
        digest_auth += 'uri=\"' + self.url + "\", "
        digest_auth += 'response=\"' + response + '\"'
        return digest_auth

Response: RFC 2617. 1. 1


Instruction:
Can you generate a brief explanation for the following Python 3 code
def generate_basic(self):
        from base64 import b64encode
        if not self.basic_auth:
            creds = self.username + ':' + self.password
            self.basic_auth = 'Basic '
            self.basic_auth += b64encode(creds.encode('UTF-8')).decode('UTF-8')
        return self.basic_auth

Response: RFC 2617. 1. 1 Basic Auth