Instruction:
Here you have a function in Python 3, explain what it does
def cmd():
    '''Handler for command line invocation'''

    # Try to handle any reasonable thing thrown at this.
    # Consume '-f' and '-o' as input/output, allow '-' for stdin/stdout
    # and treat any subsequent arguments as a space separated string to
    # be titlecased (so it still works if people forget quotes)
    parser = argparse.ArgumentParser()
    in_group = parser.add_mutually_exclusive_group()
    in_group.add_argument('string', nargs='*', default=[],
            help='String to titlecase')
    in_group.add_argument('-f', '--input-file',
            help='File to read from to titlecase')
    parser.add_argument('-o', '--output-file',
            help='File to write titlecased output to)')

    args = parser.parse_args()

    if args.input_file is not None:
        if args.input_file == '-':
            ifile = sys.stdin
        else:
            ifile = open(args.input_file)
    else:
        ifile = sys.stdin

    if args.output_file is not None:
        if args.output_file == '-':
            ofile = sys.stdout
        else:
            ofile = open(args.output_file, 'w')
    else:
        ofile = sys.stdout

    if len(args.string) > 0:
        in_string = ' '.join(args.string)
    else:
        with ifile:
            in_string = ifile.read()

    with ofile:
        ofile.write(titlecase(in_string))

Response: Handler for command line invocation


Instruction:
Can you tell what is the following Python 3 function doing
def add_options(cls, parser: OptionManager) -> None:
        parser.add_option(
            '--eradicate-aggressive',
            default=False,
            help=(
                'Enables aggressive mode for eradicate; '
                'this may result in false positives'
            ),
            action='store_true',
            type=None,
        )

Response: Add options to the given option parser instance.


Instruction:
Can you tell what is the following Python 3 function doing
def run(self) -> Generator[Tuple[int, int, str, type], None, None]:
        if self.filename != STDIN:
            buffer = StringIO()
            options = _Options(aggressive=self.options.eradicate_aggressive)
            fix_file(self.filename, options, buffer)
            traceback = buffer.getvalue()

            if traceback:
                yield 1, 0, self._error(traceback), type(self)

Response:
Runs the checker.

        ``fix_file()`` only mutates the buffer object.
        It is the only way to find out if some error happened.


Instruction:
Implement a Python 3 function for
yielding unique values yielded by g removing any duplicates.

Response:
def unique(g):
    """
    Yield values yielded by ``g``, removing any duplicates.

    Example
    -------
    >>> list(unique(iter([1, 3, 1, 2, 3])))
    [1, 3, 2]
    """
    yielded = set()
    for value in g:
        if value not in yielded:
            yield value
            yielded.add(value)


Instruction:
Explain what the following Python 3 code does
def static_get_type_attr(t, name):
    for type_ in t.mro():
        try:
            return vars(type_)[name]
        except KeyError:
            pass
    raise AttributeError(name)

Response: Get a type attribute statically circumventing the descriptor protocol.


Instruction:
How would you code a function in Python 3 to
format an error message for conflicting default implementations.

Response:
def _conflicting_defaults(typename, conflicts):
    """Format an error message for conflicting default implementations.

    Parameters
    ----------
    typename : str
        Name of the type for which we're producing an error.
    conflicts : dict[str -> list[Interface]]
        Map from strings to interfaces providing a default with that name.

    Returns
    -------
    message : str
        User-facing error message.
    """
    message = "\nclass {C} received conflicting default implementations:".format(
        C=typename,
    )
    for attrname, interfaces in conflicts.items():
        message += dedent(
            """

            The following interfaces provided default implementations for {attr!r}:
            {interfaces}"""
        ).format(
            attr=attrname,
            interfaces=bulleted_list(sorted(map(getname, interfaces))),
        )
    return InvalidImplementation(message)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def verify(self, type_):
        raw_missing, mistyped, mismatched = self._diff_signatures(type_)

        # See if we have defaults for missing methods.
        missing = []
        defaults_to_use = {}
        for name in raw_missing:
            try:
                defaults_to_use[name] = self._defaults[name].implementation
            except KeyError:
                missing.append(name)

        if not any((missing, mistyped, mismatched)):
            return defaults_to_use

        raise self._invalid_implementation(type_, missing, mistyped, mismatched)

Response: Verify that a type implements this class.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _invalid_implementation(self, t, missing, mistyped, mismatched):
        assert missing or mistyped or mismatched, "Implementation wasn't invalid."

        message = "\nclass {C} failed to implement interface {I}:".format(
            C=getname(t),
            I=getname(self),
        )
        if missing:
            message += dedent(
                """

                The following methods of {I} were not implemented:
                {missing_methods}"""
            ).format(
                I=getname(self),
                missing_methods=self._format_missing_methods(missing)
            )

        if mistyped:
            message += dedent(
                """

                The following methods of {I} were implemented with incorrect types:
                {mismatched_types}"""
            ).format(
                I=getname(self),
                mismatched_types=self._format_mismatched_types(mistyped),
            )

        if mismatched:
            message += dedent(
                """

                The following methods of {I} were implemented with invalid signatures:
                {mismatched_methods}"""
            ).format(
                I=getname(self),
                mismatched_methods=self._format_mismatched_methods(mismatched),
            )
        return InvalidImplementation(message)

Response: Make a TypeError explaining why t doesn t implement our interface.


Instruction:
How would you code a function in Python 3 to
create an interface from an existing class.

Response:
def from_class(cls, existing_class, subset=None, name=None):
        """Create an interface from an existing class.

        Parameters
        ----------
        existing_class : type
            The type from which to extract an interface.
        subset : list[str], optional
            List of methods that should be included in the interface.
            Default is to use all attributes not defined in an empty class.
        name : str, optional
            Name of the generated interface.
            Default is ``existing_class.__name__ + 'Interface'``.

        Returns
        -------
        interface : type
            A new interface class with stubs generated from ``existing_class``.
        """
        if name is None:
            name = existing_class.__name__ + 'Interface'

        if subset is None:
            subset = set(dir(existing_class)) - TRIVIAL_CLASS_ATTRIBUTES

        return InterfaceMeta(
            name,
            (Interface,),
            {name: static_get_type_attr(existing_class, name) for name in subset},
        )


Instruction:
Can you generate the documentation for the following Python 3 function
def compatible(impl_sig, iface_sig):
    return all([
        positionals_compatible(
            takewhile(is_positional, impl_sig.parameters.values()),
            takewhile(is_positional, iface_sig.parameters.values()),
        ),
        keywords_compatible(
            valfilter(complement(is_positional), impl_sig.parameters),
            valfilter(complement(is_positional), iface_sig.parameters),
        ),
    ])

Response: Check whether impl_sig is compatible with iface_sig.


Instruction:
Explain what the following Python 3 code does
def aggregate_group_loop(*args, **kwargs):
    func = kwargs['func']
    del kwargs['func']
    return aggregate_np(*args, func=lambda x: func(x), **kwargs)

Response: wraps func in lambda which prevents aggregate_numpy from recognising and optimising it. Instead it groups and loops.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def step_count(group_idx):
    cmp_pos = 0
    steps = 1
    if len(group_idx) < 1:
        return 0
    for i in range(len(group_idx)):
        if group_idx[cmp_pos] != group_idx[i]:
            cmp_pos = i
            steps += 1
    return steps

Response: Return the amount of index changes within group_idx.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def step_indices(group_idx):
    ilen = step_count(group_idx) + 1
    indices = np.empty(ilen, np.int64)
    indices[0] = 0
    indices[-1] = group_idx.size
    cmp_pos = 0
    ri = 1
    for i in range(len(group_idx)):
        if group_idx[cmp_pos] != group_idx[i]:
            cmp_pos = i
            indices[ri] = i
            ri += 1
    return indices

Response: Return the edges of areas within group_idx which are filled with the same value.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def callable(cls, nans=False, reverse=False, scalar=False):
        _valgetter = cls._valgetter_scalar if scalar else cls._valgetter
        valgetter = nb.njit(_valgetter)
        outersetter = nb.njit(cls._outersetter)

        _cls_inner = nb.njit(cls._inner)
        if nans:
            def _inner(ri, val, ret, counter, mean):
                if not np.isnan(val):
                    _cls_inner(ri, val, ret, counter, mean)
            inner = nb.njit(_inner)
        else:
            inner = _cls_inner

        def _loop(group_idx, a, ret, counter, mean, outer, fill_value, ddof):
            # fill_value and ddof need to be present for being exchangeable with loop_2pass
            size = len(ret)
            rng = range(len(group_idx) - 1, -1 , -1) if reverse else range(len(group_idx))
            for i in rng:
                ri = group_idx[i]
                if ri < 0:
                    raise ValueError("negative indices not supported")
                if ri >= size:
                    raise ValueError("one or more indices in group_idx are too large")
                val = valgetter(a, i)
                inner(ri, val, ret, counter, mean)
                outersetter(outer, i, ret[ri])
        return nb.njit(_loop, nogil=True)

Response: Compile a jitted function for the class.


Instruction:
How would you code a function in Python 3 to
compile a jitted function and loop over the sorted data.

Response:
def callable(self, nans=False):
        """Compile a jitted function and loop it over the sorted data."""
        jitfunc = nb.njit(self.func, nogil=True)

        def _loop(sortidx, group_idx, a, ret):
            size = len(ret)
            group_idx_srt = group_idx[sortidx]
            a_srt = a[sortidx]

            indices = step_indices(group_idx_srt)
            for i in range(len(indices) - 1):
                start_idx, stop_idx =  indices[i], indices[i + 1]
                ri = group_idx_srt[start_idx]
                if ri < 0:
                    raise ValueError("negative indices not supported")
                if ri >= size:
                    raise ValueError("one or more indices in group_idx are too large")
                ret[ri] = jitfunc(a_srt[start_idx:stop_idx])
        return nb.njit(_loop, nogil=True)


Instruction:
Can you generate the documentation for the following Python 3 function
def get_aliasing(*extra):
    alias = dict((k, k) for k in funcs_common)
    alias.update(_alias_str)
    alias.update((fn, fn) for fn in _alias_builtin.values())
    alias.update(_alias_builtin)
    for d in extra:
        alias.update(d)
    alias.update((k, k) for k in set(alias.values()))
    # Treat nan-functions as firstclass member and add them directly
    for key in set(alias.values()):
        if key not in funcs_no_separate_nan:
            key = 'nan' + key
            alias[key] = key
    return alias

Response: Returns the dict mapping strings and functions to the list of available functions.


Instruction:
Write a Python 3 script to
return the key of a found implementation or the func itself

Response:
def get_func(func, aliasing, implementations):
    """ Return the key of a found implementation or the func itself """
    try:
        func_str = aliasing[func]
    except KeyError:
        if callable(func):
            return func
    else:
        if func_str in implementations:
            return func_str
        if func_str.startswith('nan') and \
                func_str[3:] in funcs_no_separate_nan:
            raise ValueError("%s does not have a nan-version".format(func_str[3:]))
        else:
            raise NotImplementedError("No such function available")
    raise ValueError("func %s is neither a valid function string nor a "
                     "callable object".format(func))


Instruction:
Can you generate a brief explanation for the following Python 3 code
def minimum_dtype(x, dtype=np.bool_):

    def check_type(x, dtype):
        try:
            converted = dtype.type(x)
        except (ValueError, OverflowError):
            return False
        # False if some overflow has happened
        return converted == x or np.isnan(x)

    def type_loop(x, dtype, dtype_dict, default=None):
        while True:
            try:
                dtype = np.dtype(dtype_dict[dtype.name])
                if check_type(x, dtype):
                    return np.dtype(dtype)
            except KeyError:
                if default is not None:
                    return np.dtype(default)
                raise ValueError("Can not determine dtype of %r" % x)

    dtype = np.dtype(dtype)
    if check_type(x, dtype):
        return dtype

    if np.issubdtype(dtype, np.inexact):
        return type_loop(x, dtype, _next_float_dtype)
    else:
        return type_loop(x, dtype, _next_int_dtype, default=np.float32)

Response: returns the most basic dtype which represents x properly which is the same value range as the specified dtype.


Instruction:
Write a Python 3 script to
perform basic input validation on the group_idx and a.

Response:
def input_validation(group_idx, a, size=None, order='C', axis=None,
                     ravel_group_idx=True, check_bounds=True):
    """ Do some fairly extensive checking of group_idx and a, trying to
    give the user as much help as possible with what is wrong. Also,
    convert ndim-indexing to 1d indexing.
    """
    if not isinstance(a, (int, float, complex)):
        a = np.asanyarray(a)
    group_idx = np.asanyarray(group_idx)

    if not np.issubdtype(group_idx.dtype, np.integer):
        raise TypeError("group_idx must be of integer type")

    # This check works for multidimensional indexing as well
    if check_bounds and np.any(group_idx < 0):
        raise ValueError("negative indices not supported")

    ndim_idx = np.ndim(group_idx)
    ndim_a = np.ndim(a)

    # Deal with the axis arg: if present, then turn 1d indexing into
    # multi-dimensional indexing along the specified axis.
    if axis is None:
        if ndim_a > 1:
            raise ValueError("a must be scalar or 1 dimensional, use .ravel to"
                             " flatten. Alternatively specify axis.")
    elif axis >= ndim_a or axis < -ndim_a:
        raise ValueError("axis arg too large for np.ndim(a)")
    else:
        axis = axis if axis >= 0 else ndim_a + axis  # negative indexing
        if ndim_idx > 1:
            # TODO: we could support a sequence of axis values for multiple
            # dimensions of group_idx.
            raise NotImplementedError("only 1d indexing currently"
                                      "supported with axis arg.")
        elif a.shape[axis] != len(group_idx):
            raise ValueError("a.shape[axis] doesn't match length of group_idx.")
        elif size is not None and not np.isscalar(size):
            raise NotImplementedError("when using axis arg, size must be"
                                      "None or scalar.")
        else:
            # Create the broadcast-ready multidimensional indexing.
            # Note the user could do this themselves, so this is
            # very much just a convenience.
            size_in = np.max(group_idx) + 1 if size is None else size
            group_idx_in = group_idx
            group_idx = []
            size = []
            for ii, s in enumerate(a.shape):
                ii_idx = group_idx_in if ii == axis else np.arange(s)
                ii_shape = [1] * ndim_a
                ii_shape[ii] = s
                group_idx.append(ii_idx.reshape(ii_shape))
                size.append(size_in if ii == axis else s)
            # Use the indexing, and return. It's a bit simpler than
            # using trying to keep all the logic below happy
            group_idx = np.ravel_multi_index(group_idx, size, order=order,
                                             mode='raise')
            flat_size = np.prod(size)
            ndim_idx = ndim_a
            return group_idx.ravel(), a.ravel(), flat_size, ndim_idx, size

    if ndim_idx == 1:
        if size is None:
            size = np.max(group_idx) + 1
        else:
            if not np.isscalar(size):
                raise ValueError("output size must be scalar or None")
            if check_bounds and np.any(group_idx > size - 1):
                raise ValueError("one or more indices are too large for "
                                 "size %d" % size)
        flat_size = size
    else:
        if size is None:
            size = np.max(group_idx, axis=1) + 1
        elif np.isscalar(size):
            raise ValueError("output size must be of length %d"
                             % len(group_idx))
        elif len(size) != len(group_idx):
            raise ValueError("%d sizes given, but %d output dimensions "
                             "specified in index" % (len(size),
                                                     len(group_idx)))
        if ravel_group_idx:
            group_idx = np.ravel_multi_index(group_idx, size, order=order,
                                             mode='raise')
        flat_size = np.prod(size)

    if not (np.ndim(a) == 0 or len(a) == group_idx.size):
        raise ValueError("group_idx and a must be of the same length, or a"
                         " can be scalar")

    return group_idx, a, flat_size, ndim_idx, size


Instruction:
How would you explain what the following Python 3 function does
def multi_arange(n):
    if n.ndim != 1:
        raise ValueError("n is supposed to be 1d array.")

    n_mask = n.astype(bool)
    n_cumsum = np.cumsum(n)
    ret = np.ones(n_cumsum[-1] + 1, dtype=int)
    ret[n_cumsum[n_mask]] -= n[n_mask]
    ret[0] -= 1
    return np.cumsum(ret)[:-1]

Response: This function returns a 1d array that contains the range of n.


Instruction:
How would you explain what the following Python 3 function does
def label_contiguous_1d(X):

    if X.ndim != 1:
        raise ValueError("this is for 1d masks only.")

    is_start = np.empty(len(X), dtype=bool)
    is_start[0] = X[0]  # True if X[0] is True or non-zero

    if X.dtype.kind == 'b':
        is_start[1:] = ~X[:-1] & X[1:]
        M = X
    else:
        M = X.astype(bool)
        is_start[1:] = X[:-1] != X[1:]
        is_start[~M] = False

    L = np.cumsum(is_start)
    L[~M] = 0
    return L

Response: This function labels contiguous 1 - dimensional arrays X with a label.


Instruction:
How would you code a function in Python 3 to
relabel unique groups in a node.

Response:
def relabel_groups_unique(group_idx):
    """
    See also ``relabel_groups_masked``.

    keep_group:  [0 3 3 3 0 2 5 2 0 1 1 0 3 5 5]
    ret:         [0 3 3 3 0 2 4 2 0 1 1 0 3 4 4]

    Description of above: unique groups in input was ``1,2,3,5``, i.e.
    ``4`` was missing, so group 5 was relabled to be ``4``.
    Relabeling maintains order, just "compressing" the higher numbers
    to fill gaps.
    """

    keep_group = np.zeros(np.max(group_idx) + 1, dtype=bool)
    keep_group[0] = True
    keep_group[group_idx] = True
    return relabel_groups_masked(group_idx, keep_group)


Instruction:
Create a Python 3 function to
relabel groups in a group_idx array to be filled with gaps.

Response:
def relabel_groups_masked(group_idx, keep_group):
    """
    group_idx: [0 3 3 3 0 2 5 2 0 1 1 0 3 5 5]

                 0 1 2 3 4 5
    keep_group: [0 1 0 1 1 1]

    ret:       [0 2 2 2 0 0 4 0 0 1 1 0 2 4 4]

    Description of above in words: remove group 2, and relabel group 3,4, and 5
    to be 2, 3 and 4 respecitvely, in order to fill the gap.  Note that group 4 was never used
    in the input group_idx, but the user supplied mask said to keep group 4, so group
    5 is only moved up by one place to fill the gap created by removing group 2.

    That is, the mask describes which groups to remove,
    the remaining groups are relabled to remove the gaps created by the falsy
    elements in ``keep_group``.  Note that ``keep_group[0]`` has no particular meaning because it refers
    to the zero group which cannot be "removed".

    ``keep_group`` should be bool and ``group_idx`` int.
    Values in ``group_idx`` can be any order, and
    """

    keep_group = keep_group.astype(bool, copy=not keep_group[0])
    if not keep_group[0]:  # ensuring keep_group[0] is True makes life easier
        keep_group[0] = True

    relabel = np.zeros(keep_group.size, dtype=group_idx.dtype)
    relabel[keep_group] = np.arange(np.count_nonzero(keep_group))
    return relabel[group_idx]


Instruction:
Here you have a function in Python 3, explain what it does
def _array(group_idx, a, size, fill_value, dtype=None):
    if fill_value is not None and not (np.isscalar(fill_value) or
                                       len(fill_value) == 0):
        raise ValueError("fill_value must be None, a scalar or an empty "
                         "sequence")
    order_group_idx = np.argsort(group_idx, kind='mergesort')
    counts = np.bincount(group_idx, minlength=size)
    ret = np.split(a[order_group_idx], np.cumsum(counts)[:-1])
    ret = np.asanyarray(ret)
    if fill_value is None or np.isscalar(fill_value):
        _fill_untouched(group_idx, ret, fill_value)
    return ret

Response: groups a into separate arrays keeping the order intact.


Instruction:
How would you explain what the following Python 3 function does
def _generic_callable(group_idx, a, size, fill_value, dtype=None,
                      func=lambda g: g, **kwargs):
    """groups a by inds, and then applies foo to each group in turn, placing
    the results in an array."""
    groups = _array(group_idx, a, size, ())
    ret = np.full(size, fill_value, dtype=dtype or np.float64)

    for i, grp in enumerate(groups):
        if np.ndim(grp) == 1 and len(grp) > 0:
            ret[i] = func(grp)
    return ret

Response:
generic function for group_idx and apply func to each group in turn placing
    the results in an array.


Instruction:
Given the following Python 3 function, write the documentation
def _fill_untouched(idx, ret, fill_value):
    untouched = np.ones_like(ret, dtype=bool)
    untouched[idx] = False
    ret[untouched] = fill_value

Response: fill_value is the value to fill_value in the elements of ret not indexed by idx.


Instruction:
How would you explain what the following Python 3 function does
def aggregate_grouploop(*args, **kwargs):
    extrafuncs = {'allnan': allnan, 'anynan': anynan,
                  'first': itemgetter(0), 'last': itemgetter(-1),
                  'nanfirst': nanfirst, 'nanlast': nanlast}
    func = kwargs.pop('func')
    func = extrafuncs.get(func, func)
    if isinstance(func, str):
        raise NotImplementedError("Grouploop needs to be called with a function")
    return aggregate_numpy.aggregate(*args, func=lambda x: func(x), **kwargs)

Response: wraps func in lambda which prevents aggregate_numpy from recognising and optimising it. Instead it groups and loops.


Instruction:
Given the following Python 3 function, write the documentation
def c_func(funcname, reverse=False, nans=False, scalar=False):
    varnames = ['group_idx', 'a', 'ret', 'counter']
    codebase = c_base_reverse if reverse else c_base
    iteration = c_iter_scalar[funcname] if scalar else c_iter[funcname]
    if scalar:
        varnames.remove('a')
    return codebase % dict(init=c_init(varnames), iter=iteration,
                           finish=c_finish.get(funcname, ''),
                           ri_redir=(c_ri_redir if nans else c_ri))

Response: Fill c_funcs with constructed code from the templates


Instruction:
Create a Python 3 function to
get the indices of the edges of areas within group_idx which are filled with the same value

Response:
def step_indices(group_idx):
    """ Get the edges of areas within group_idx, which are filled
        with the same value
    """
    ilen = step_count(group_idx) + 1
    indices = np.empty(ilen, int)
    indices[0] = 0
    indices[-1] = group_idx.size
    inline(c_step_indices, ['group_idx', 'indices'], define_macros=c_macros, extra_compile_args=c_args)
    return indices


Instruction:
Create a Python 3 function to
create a random projection matrix for the given size

Response:
def __create_proj_mat(self, size):
        """Create a random projection matrix

        [1] D. Achlioptas. Database-friendly random projections: Johnson-Lindenstrauss with binary coins.
        [2] P. Li, et al. Very sparse random projections.

        http://scikit-learn.org/stable/modules/random_projection.html#sparse-random-projection
        """

        # [1]
        # return np.random.choice([-np.sqrt(3), 0, np.sqrt(3)], size=size, p=[1 / 6, 2 / 3, 1 / 6])

        # [2]
        s = 1 / self.density
        return np.random.choice([-np.sqrt(s / self.k), 0, np.sqrt(s / self.k)],
                                size=size,
                                p=[1 / (2 * s), 1 - 1 / s, 1 / (2 * s)])


Instruction:
Can you generate a brief explanation for the following Python 3 code
def load_movies(data_home, size):
    all_genres = ['Action',
                  'Adventure',
                  'Animation',
                  "Children's",
                  'Comedy',
                  'Crime',
                  'Documentary',
                  'Drama',
                  'Fantasy',
                  'Film-Noir',
                  'Horror',
                  'Musical',
                  'Mystery',
                  'Romance',
                  'Sci-Fi',
                  'Thriller',
                  'War',
                  'Western']
    n_genre = len(all_genres)

    movies = {}

    if size == '100k':
        with open(os.path.join(data_home, 'u.item'), encoding='ISO-8859-1') as f:
            lines = list(map(lambda l: l.rstrip().split('|'), f.readlines()))

        for line in lines:
            movie_vec = np.zeros(n_genre)
            for i, flg_chr in enumerate(line[-n_genre:]):
                if flg_chr == '1':
                    movie_vec[i] = 1.
            movie_id = int(line[0])
            movies[movie_id] = movie_vec
    elif size == '1m':
        with open(os.path.join(data_home, 'movies.dat'), encoding='ISO-8859-1') as f:
            lines = list(map(lambda l: l.rstrip().split('::'), f.readlines()))

        for item_id_str, title, genres in lines:
            movie_vec = np.zeros(n_genre)
            for genre in genres.split('|'):
                i = all_genres.index(genre)
                movie_vec[i] = 1.
            item_id = int(item_id_str)
            movies[item_id] = movie_vec

    return movies

Response: Load movie genres as a context.


Instruction:
Implement a function in Python 3 to
load user demographics as contexts.

Response:
def load_users(data_home, size):
    """Load user demographics as contexts.User ID -> {sex (M/F), age (7 groupd), occupation(0-20; 21)}
    Returns:
        dict of user vectors: user_id -> numpy array (1+1+21,); (sex_flg + age_group + n_occupation, )
    """
    ages = [1, 18, 25, 35, 45, 50, 56, 999]

    users = {}

    if size == '100k':
        all_occupations = ['administrator',
                           'artist',
                           'doctor',
                           'educator',
                           'engineer',
                           'entertainment',
                           'executive',
                           'healthcare',
                           'homemaker',
                           'lawyer',
                           'librarian',
                           'marketing',
                           'none',
                           'other',
                           'programmer',
                           'retired',
                           'salesman',
                           'scientist',
                           'student',
                           'technician',
                           'writer']

        with open(os.path.join(data_home, 'u.user'), encoding='ISO-8859-1') as f:
            lines = list(map(lambda l: l.rstrip().split('|'), f.readlines()))

        for user_id_str, age_str, sex_str, occupation_str, zip_code in lines:
                user_vec = np.zeros(1 + 1 + 21)  # 1 categorical, 1 value, 21 categorical
                user_vec[0] = 0 if sex_str == 'M' else 1  # sex

                # age (ML1M is "age group", but 100k has actual "age")
                age = int(age_str)
                for i in range(7):
                    if age >= ages[i] and age < ages[i + 1]:
                        user_vec[1] = i
                        break

                user_vec[2 + all_occupations.index(occupation_str)] = 1  # occupation (1-of-21)
                users[int(user_id_str)] = user_vec
    elif size == '1m':
        with open(os.path.join(data_home, 'users.dat'), encoding='ISO-8859-1') as f:
            lines = list(map(lambda l: l.rstrip().split('::'), f.readlines()))

        for user_id_str, sex_str, age_str, occupation_str, zip_code in lines:
                user_vec = np.zeros(1 + 1 + 21)  # 1 categorical, 1 value, 21 categorical
                user_vec[0] = 0 if sex_str == 'M' else 1  # sex
                user_vec[1] = ages.index(int(age_str))  # age group (1, 18, ...)
                user_vec[2 + int(occupation_str)] = 1  # occupation (1-of-21)
                users[int(user_id_str)] = user_vec

    return users


Instruction:
Given the following Python 3 function, write the documentation
def load_ratings(data_home, size):

    if size == '100k':
        with open(os.path.join(data_home, 'u.data'), encoding='ISO-8859-1') as f:
            lines = list(map(lambda l: list(map(int, l.rstrip().split('\t'))), f.readlines()))
    elif size == '1m':
        with open(os.path.join(data_home, 'ratings.dat'), encoding='ISO-8859-1') as f:
            lines = list(map(lambda l: list(map(int, l.rstrip().split('::'))), f.readlines()))

    ratings = []

    for l in lines:
        # Since we consider positive-only feedback setting, ratings < 5 will be excluded.
        if l[2] == 5:
            ratings.append(l)

    ratings = np.asarray(ratings)

    # sorted by timestamp
    return ratings[np.argsort(ratings[:, 3])]

Response: Load all samples in the dataset.


Instruction:
Write a Python 3 script to
compute difference between given 2 dates in month / day.

Response:
def delta(d1, d2, opt='d'):
    """Compute difference between given 2 dates in month/day.
    """
    delta = 0

    if opt == 'm':
        while True:
            mdays = monthrange(d1.year, d1.month)[1]
            d1 += timedelta(days=mdays)
            if d1 <= d2:
                delta += 1
            else:
                break
    else:
        delta = (d2 - d1).days

    return delta


Instruction:
Explain what the following Python 3 code does
def feature_hash(feature, dim, seed=123):
    vec = np.zeros(dim)
    i = mmh3.hash(feature, seed) % dim
    vec[i] = 1
    return vec

Response:
Feature hashing.

    Args:
        feature (str): Target feature represented as string.
        dim (int): Number of dimensions for a hash value.
        seed (float): Seed of a MurmurHash3 hash function.

    Returns:
        numpy 1d array: one-hot-encoded feature vector for `s`.


Instruction:
Explain what the following Python 3 code does
def count_true_positive(truth, recommend):
    tp = 0
    for r in recommend:
        if r in truth:
            tp += 1
    return tp

Response: Counts the number of true positives from given set of samples.


Instruction:
Write a Python 3 script to
return the recall of the current node in the tree.

Response:
def recall(truth, recommend, k=None):
    """Recall@k.

    Args:
        truth (numpy 1d array): Set of truth samples.
        recommend (numpy 1d array): Ordered set of recommended samples.
        k (int): Top-k items in `recommend` will be recommended.

    Returns:
        float: Recall@k.

    """
    if len(truth) == 0:
        if len(recommend) == 0:
            return 1.
        return 0.

    if k is None:
        k = len(recommend)
    return count_true_positive(truth, recommend[:k]) / float(truth.size)


Instruction:
Can you tell what is the following Python 3 function doing
def precision(truth, recommend, k=None):
    if len(recommend) == 0:
        if len(truth) == 0:
            return 1.
        return 0.

    if k is None:
        k = len(recommend)
    return count_true_positive(truth, recommend[:k]) / float(k)

Response: Returns the precision of the truth and recommendations.


Instruction:
Explain what the following Python 3 code does
def average_precision(truth, recommend):
    if len(truth) == 0:
        if len(recommend) == 0:
            return 1.
        return 0.

    tp = accum = 0.
    for n in range(recommend.size):
        if recommend[n] in truth:
            tp += 1.
            accum += (tp / (n + 1.))
    return accum / truth.size

Response: Average Precision of a set of truth and recommendations.


Instruction:
Implement a Python 3 function for
calculating the AUC of the given set of truth and recommendations.

Response:
def auc(truth, recommend):
    """Area under the ROC curve (AUC).

    Args:
        truth (numpy 1d array): Set of truth samples.
        recommend (numpy 1d array): Ordered set of recommended samples.

    Returns:
        float: AUC.

    """
    tp = correct = 0.
    for r in recommend:
        if r in truth:
            # keep track number of true positives placed before
            tp += 1.
        else:
            correct += tp
    # number of all possible tp-fp pairs
    pairs = tp * (recommend.size - tp)

    # if there is no TP (or no FP), it's meaningless for this metric (i.e., AUC=0.5)
    if pairs == 0:
        return 0.5

    return correct / pairs


Instruction:
Implement a Python 3 function for
returning the reciprocal rank of the set of truth and recommend.

Response:
def reciprocal_rank(truth, recommend):
    """Reciprocal Rank (RR).

    Args:
        truth (numpy 1d array): Set of truth samples.
        recommend (numpy 1d array): Ordered set of recommended samples.

    Returns:
        float: RR.

    """
    for n in range(recommend.size):
        if recommend[n] in truth:
            return 1. / (n + 1)
    return 0.


Instruction:
Can you write a function in Python 3 where it
means Percentile Rank of a set of truth and recommendations.

Response:
def mpr(truth, recommend):
    """Mean Percentile Rank (MPR).

    Args:
        truth (numpy 1d array): Set of truth samples.
        recommend (numpy 1d array): Ordered set of recommended samples.

    Returns:
        float: MPR.

    """
    if len(recommend) == 0 and len(truth) == 0:
        return 0.  # best
    elif len(truth) == 0 or len(truth) == 0:
        return 100.  # worst

    accum = 0.
    n_recommend = recommend.size
    for t in truth:
        r = np.where(recommend == t)[0][0] / float(n_recommend)
        accum += r
    return accum * 100. / truth.size


Instruction:
Can you tell what is the following Python 3 function doing
def ndcg(truth, recommend, k=None):
    if k is None:
        k = len(recommend)

    def idcg(n_possible_truth):
        res = 0.
        for n in range(n_possible_truth):
            res += 1. / np.log2(n + 2)
        return res

    dcg = 0.
    for n, r in enumerate(recommend[:k]):
        if r not in truth:
            continue
        dcg += 1. / np.log2(n + 2)

    res_idcg = idcg(np.min([truth.size, k]))
    if res_idcg == 0.:
        return 0.
    return dcg / res_idcg

Response: Returns the NDCG of the given set of truth and recommendations.


Instruction:
Can you create a Python 3 function that
initializes a recommender by resetting stored users and items.

Response:
def initialize(self, *args):
        """Initialize a recommender by resetting stored users and items.
        """
        # number of observed users
        self.n_user = 0

        # store user data
        self.users = {}

        # number of observed items
        self.n_item = 0

        # store item data
        self.items = {}


Instruction:
Write a Python 3 script for
registering a user into the dictionary.

Response:
def register_user(self, user):
        """For new users, append their information into the dictionaries.

        Args:
            user (User): User.

        """
        self.users[user.index] = {'known_items': set()}
        self.n_user += 1


Instruction:
Can you generate the documentation for the following Python 3 function
def scores2recos(self, scores, candidates, rev=False):
        sorted_indices = np.argsort(scores)

        if rev:
            sorted_indices = sorted_indices[::-1]

        return candidates[sorted_indices], scores[sorted_indices]

Response: Return a list of items for a user u_index based on scores.


Instruction:
Create a Python 3 function to
train a model using the first 30% positive events to avoid cold - start.

Response:
def fit(self, train_events, test_events, n_epoch=1):
        """Train a model using the first 30% positive events to avoid cold-start.

        Evaluation of this batch training is done by using the next 20% positive events.
        After the batch SGD training, the models are incrementally updated by using the 20% test events.

        Args:
            train_events (list of Event): Positive training events (0-30%).
            test_events (list of Event): Test events (30-50%).
            n_epoch (int): Number of epochs for the batch training.

        """
        # make initial status for batch training
        for e in train_events:
            self.__validate(e)
            self.rec.users[e.user.index]['known_items'].add(e.item.index)
            self.item_buffer.append(e.item.index)

        # for batch evaluation, temporarily save new users info
        for e in test_events:
            self.__validate(e)
            self.item_buffer.append(e.item.index)

        self.__batch_update(train_events, test_events, n_epoch)

        # batch test events are considered as a new observations;
        # the model is incrementally updated based on them before the incremental evaluation step
        for e in test_events:
            self.rec.users[e.user.index]['known_items'].add(e.item.index)
            self.rec.update(e)


Instruction:
Can you implement a function in Python 3 that
iterates recommend and update procedure and compute incremental recall.

Response:
def evaluate(self, test_events):
        """Iterate recommend/update procedure and compute incremental recall.

        Args:
            test_events (list of Event): Positive test events.

        Returns:
            list of tuples: (rank, recommend time, update time)

        """
        for i, e in enumerate(test_events):
            self.__validate(e)

            # target items (all or unobserved depending on a detaset)
            unobserved = set(self.item_buffer)
            if not self.repeat:
                unobserved -= self.rec.users[e.user.index]['known_items']

            # item i interacted by user u must be in the recommendation candidate
            # even if it is a new item
            unobserved.add(e.item.index)

            candidates = np.asarray(list(unobserved))

            # make top-{at} recommendation for the 1001 items
            start = time.clock()
            recos, scores = self.__recommend(e, candidates)
            recommend_time = (time.clock() - start)

            rank = np.where(recos == e.item.index)[0][0]

            # Step 2: update the model with the observed event
            self.rec.users[e.user.index]['known_items'].add(e.item.index)
            start = time.clock()
            self.rec.update(e)
            update_time = (time.clock() - start)

            self.item_buffer.append(e.item.index)

            # (top-1 score, where the correct item is ranked, rec time, update time)
            yield scores[0], rank, recommend_time, update_time


Instruction:
How would you code a function in Python 3 to
batch update called by the fitting method.

Response:
def __batch_update(self, train_events, test_events, n_epoch):
        """Batch update called by the fitting method.

        Args:
            train_events (list of Event): Positive training events.
            test_events (list of Event): Test events.
            n_epoch (int): Number of epochs for the batch training.

        """
        for epoch in range(n_epoch):
            # SGD requires us to shuffle events in each iteration
            # * if n_epoch == 1
            #   => shuffle is not required because it is a deterministic training (i.e. matrix sketching)
            if n_epoch != 1:
                np.random.shuffle(train_events)

            # train
            for e in train_events:
                self.rec.update(e, batch_train=True)

            # test
            MPR = self.__batch_evaluate(test_events)
            if self.debug:
                logger.debug('epoch %2d: MPR = %f' % (epoch + 1, MPR))


Instruction:
Create a Python 3 function for
evaluating the current model by using the given test events.

Response:
def __batch_evaluate(self, test_events):
        """Evaluate the current model by using the given test events.

        Args:
            test_events (list of Event): Current model is evaluated by these events.

        Returns:
            float: Mean Percentile Rank for the test set.

        """
        percentiles = np.zeros(len(test_events))

        all_items = set(self.item_buffer)
        for i, e in enumerate(test_events):

            # check if the data allows users to interact the same items repeatedly
            unobserved = all_items
            if not self.repeat:
                # make recommendation for all unobserved items
                unobserved -= self.rec.users[e.user.index]['known_items']
                # true item itself must be in the recommendation candidates
                unobserved.add(e.item.index)

            candidates = np.asarray(list(unobserved))
            recos, scores = self.__recommend(e, candidates)

            pos = np.where(recos == e.item.index)[0][0]
            percentiles[i] = pos / (len(recos) - 1) * 100

        return np.mean(percentiles)


Instruction:
Implement a function in Python 3 to
scale X values to new width

Response:
def _scale_x_values(self, values, max_width):
        '''Scale X values to new width'''

        if type(values) == dict:
            values = self._scale_x_values_timestamps(values=values, max_width=max_width)

        adjusted_values = list(values)
        if len(adjusted_values) > max_width:

            def get_position(current_pos):
                return len(adjusted_values) * current_pos // max_width

            adjusted_values = [statistics.mean(adjusted_values[get_position(i):get_position(i + 1)]) for i in range(max_width)]

        return adjusted_values


Instruction:
Given the following Python 3 function, write the documentation
def _scale_x_values_timestamps(self, values, max_width):
        '''Scale X values to new width based on timestamps'''
        first_timestamp = float(values[0][0])
        last_timestamp = float(values[-1][0])
        step_size = (last_timestamp - first_timestamp) / max_width

        values_by_column = [[] for i in range(max_width)]
        for timestamp, value in values:
            if value is None:
                continue
            timestamp = float(timestamp)
            column = (timestamp - first_timestamp) // step_size
            column = int(min(column, max_width - 1))  # Don't go beyond the last column
            values_by_column[column].append(value)

        adjusted_values = [statistics.mean(values) if values else 0 for values in values_by_column]  # Average each column, 0 if no values

        return adjusted_values

Response: Scale X values based on timestamps


Instruction:
Can you generate the documentation for the following Python 3 function
def _scale_y_values(self, values, new_min, new_max, scale_old_from_zero=True):
        '''
        Take values and transmute them into a new range
        '''
        # Scale Y values - Create a scaled list of values to use for the visual graph
        scaled_values = []
        y_min_value = min(values)
        if scale_old_from_zero:
            y_min_value = 0
        y_max_value = max(values)
        new_min = 0
        OldRange = (y_max_value - y_min_value) or 1  # Prevents division by zero if all values are the same
        NewRange = (new_max - new_min)  # max_height is new_max
        for old_value in values:
            new_value = (((old_value - y_min_value) * NewRange) / OldRange) + new_min
            scaled_values.append(new_value)
        return scaled_values

Response: Scale the Y values of the entries in the list of entries.


Instruction:
Write a Python 3 script to
create a representation of an ascii graph using two lists of values.

Response:
def _get_ascii_field(self, values):
        '''Create a representation of an ascii graph using two lists in this format: field[x][y] = "char"'''

        empty_space = ' '

        # This formats as field[x][y]
        field = [[empty_space for y in range(max(values) + 1)] for x in range(len(values))]

        # Draw graph into field
        for x in range(len(values)):
            y = values[x]
            y_prev = values[x - 1] if x - 1 in range(len(values)) else y
            y_next = values[x + 1] if x + 1 in range(len(values)) else y
            # Fill empty space
            if abs(y_prev - y) > 1:
                # Fill space between y and y_prev
                step = 1 if y_prev - y > 0 else -1

                # We don't want the first item to be inclusive, so we use step instead of y+1 since step can be negative
                for h in range(y + step, y_prev, step):
                    if field[x][h] is empty_space:
                        field[x][h] = '|'

            # Assign the character to be placed into the graph
            char = self._assign_ascii_character(y_prev, y, y_next)

            field[x][y] = char
        return field


Instruction:
Can you create a Python 3 function that
assigns the character to be placed into the graph

Response:
def _assign_ascii_character(self, y_prev, y, y_next):  # noqa for complexity
            '''Assign the character to be placed into the graph'''
            char = '?'
            if y_next > y and y_prev > y:
                char = '-'
            elif y_next < y and y_prev < y:
                char = '-'
            elif y_prev < y and y == y_next:
                char = '-'
            elif y_prev == y and y_next < y:
                char = '-'
            elif y_next > y:
                char = '/'
            elif y_next < y:
                char = '\\'
            elif y_prev < y:
                char = '/'
            elif y_prev > y:
                char = '\\'
            elif y_next == y:
                char = '-'
            elif y == y_prev:
                char = '-'
            return char


Instruction:
Make a summary of the following Python 3 code
def _draw_ascii_graph(self, field):
        '''Draw graph from field double nested list, format field[x][y] = char'''
        row_strings = []
        for y in range(len(field[0])):
            row = ''
            for x in range(len(field)):
                row += field[x][y]
            row_strings.insert(0, row)
        graph_string = '\n'.join(row_strings)
        return graph_string

Response: Draw graph from field double nested list format field [ x ] = char


Instruction:
Given the following Python 3 function, write the documentation
def asciigraph(self, values=None, max_height=None, max_width=None, label=False):
        '''
        Accepts a list of y values and returns an ascii graph
        Optionally values can also be a dictionary with a key of timestamp, and a value of value. InGraphs returns data in this format for example.
        '''
        result = ''
        border_fill_char = '*'
        start_ctime = None
        end_ctime = None

        if not max_width:
            max_width = 180

        # If this is a dict of timestamp -> value, sort the data, store the start/end time, and convert values to a list of values
        if isinstance(values, dict):
            time_series_sorted = sorted(list(values.items()), key=lambda x: x[0])  # Sort timestamp/value dict by the timestamps

            start_timestamp = time_series_sorted[0][0]
            end_timestamp = time_series_sorted[-1][0]

            start_ctime = datetime.fromtimestamp(float(start_timestamp)).ctime()
            end_ctime = datetime.fromtimestamp(float(end_timestamp)).ctime()
            values = self._scale_x_values_timestamps(values=time_series_sorted, max_width=max_width)
        values = [value for value in values if value is not None]

        if not max_height:
            max_height = min(20, max(values))

        stdev = statistics.stdev(values)
        mean = statistics.mean(values)

        # Do value adjustments
        adjusted_values = list(values)
        adjusted_values = self._scale_x_values(values=values, max_width=max_width)
        upper_value = max(adjusted_values)  # Getting upper/lower after scaling x values so we don't label a spike we can't see
        lower_value = min(adjusted_values)
        adjusted_values = self._scale_y_values(values=adjusted_values, new_min=0, new_max=max_height, scale_old_from_zero=False)
        adjusted_values = self._round_floats_to_ints(values=adjusted_values)

        # Obtain Ascii Graph String
        field = self._get_ascii_field(adjusted_values)
        graph_string = self._draw_ascii_graph(field=field)

        # Label the graph
        if label:
            top_label = 'Upper value: {upper_value:.2f} '.format(upper_value=upper_value).ljust(max_width, border_fill_char)
            result += top_label + '\n'
        result += '{graph_string}\n'.format(graph_string=graph_string)
        if label:
            lower = f'Lower value: {lower_value:.2f} '
            stats = f' Mean: {mean:.2f} *** Std Dev: {stdev:.2f} ******'
            fill_length = max_width - len(lower) - len(stats)
            stat_label = f'{lower}{"*" * fill_length}{stats}\n'
            result += stat_label

            if start_ctime and end_ctime:
                fill_length = max_width - len(start_ctime) - len(end_ctime)
                result += f'{start_ctime}{" " * fill_length}{end_ctime}\n'

        return result

Response: This function returns an ascii graph of the ingraph data for the given list of y values.


Instruction:
Create a Python 3 function for
replacing variables in the given expression using the given substitution.

Response:
def substitute(expression: Union[Expression, Pattern], substitution: Substitution) -> Replacement:
    """Replaces variables in the given *expression* using the given *substitution*.

    >>> print(substitute(f(x_), {'x': a}))
    f(a)

    If nothing was substituted, the original expression is returned:

    >>> expression = f(x_)
    >>> result = substitute(expression, {'y': a})
    >>> print(result)
    f(x_)
    >>> expression is result
    True

    Note that this function returns a list of expressions iff the expression is a variable and its substitution
    is a list of expressions. In other cases were a substitution is a list of expressions, the expressions will
    be integrated as operands in the surrounding operation:

    >>> print(substitute(f(x_, c), {'x': [a, b]}))
    f(a, b, c)

    If you substitute with a `Multiset` of values, they will be sorted:

    >>> replacement = Multiset([b, a, b])
    >>> print(substitute(f(x_, c), {'x': replacement}))
    f(a, b, b, c)

    Parameters:
        expression:
            An expression in which variables are substituted.
        substitution:
            A substitution dictionary. The key is the name of the variable,
            the value either an expression or a list of expression to use as a replacement for
            the variable.

    Returns:
        The expression resulting from applying the substitution.
    """
    if isinstance(expression, Pattern):
        expression = expression.expression
    return _substitute(expression, substitution)[0]


Instruction:
Make a summary of the following Python 3 code
def replace(expression: Expression, position: Sequence[int], replacement: Replacement) -> Replacement:
    if len(position) == 0:
        return replacement
    if not isinstance(expression, Operation):
        raise IndexError("Invalid position {!r} for expression {!s}".format(position, expression))
    if position[0] >= op_len(expression):
        raise IndexError("Position {!r} out of range for expression {!s}".format(position, expression))
    pos = position[0]
    operands = list(op_iter(expression))
    subexpr = replace(operands[pos], position[1:], replacement)
    if isinstance(subexpr, Sequence):
        new_operands = tuple(operands[:pos]) + tuple(subexpr) + tuple(operands[pos + 1:])
        return create_operation_expression(expression, new_operands)
    operands[pos] = subexpr
    return create_operation_expression(expression, operands)

Response: r Replaces the subexpression of expression at the given position with the given replacement.


Instruction:
Here you have a function in Python 3, explain what it does
def replace_many(expression: Expression, replacements: Sequence[Tuple[Sequence[int], Replacement]]) -> Replacement:
    if len(replacements) == 0:
        return expression
    replacements = sorted(replacements)
    if len(replacements[0][0]) == 0:
        if len(replacements) > 1:
            raise IndexError(
                "Cannot replace child positions for expression {}, got {!r}".format(expression, replacements[1:])
            )
        return replacements[0][1]
    if len(replacements) == 1:
        return replace(expression, replacements[0][0], replacements[0][1])
    if not isinstance(expression, Operation):
        raise IndexError("Invalid replacements {!r} for expression {!s}".format(replacements, expression))
    operands = list(op_iter(expression))
    new_operands = []
    last_index = 0
    for index, group in itertools.groupby(replacements, lambda r: r[0][0]):
        new_operands.extend(operands[last_index:index])
        replacements = [(pos[1:], r) for pos, r in group]
        if len(replacements) == 1:
            replacement = replace(operands[index], replacements[0][0], replacements[0][1])
        else:
            replacement = replace_many(operands[index], replacements)
        if isinstance(replacement, (list, tuple, Multiset)):
            new_operands.extend(replacement)
        else:
            new_operands.append(replacement)
        last_index = index + 1
    new_operands.extend(operands[last_index:len(operands)])
    return create_operation_expression(expression, new_operands)

Response: r Replaces the subexpressions of expression at the given positions with the given replacements.


Instruction:
Write a Python 3 script to
replace all occurrences of the patterns in the given expression with the given set of replacement rules.

Response:
def replace_all(expression: Expression, rules: Iterable[ReplacementRule], max_count: int=math.inf) \
        -> Union[Expression, Sequence[Expression]]:
    """Replace all occurrences of the patterns according to the replacement rules.

    A replacement rule consists of a *pattern*, that is matched against any subexpression
    of the expression. If a match is found, the *replacement* callback of the rule is called with
    the variables from the match substitution. Whatever the callback returns is used as a replacement for the
    matched subexpression. This can either be a single expression or a sequence of expressions, which is then
    integrated into the surrounding operation in place of the subexpression.

    Note that the pattern can therefore not be a single sequence variable/wildcard, because only single expressions
    will be matched.

    Args:
        expression:
            The expression to which the replacement rules are applied.
        rules:
            A collection of replacement rules that are applied to the expression.
        max_count:
            If given, at most *max_count* applications of the rules are performed. Otherwise, the rules
            are applied until there is no more match. If the set of replacement rules is not confluent,
            the replacement might not terminate without a *max_count* set.

    Returns:
        The resulting expression after the application of the replacement rules. This can also be a sequence of
        expressions, if the root expression is replaced with a sequence of expressions by a rule.
    """
    rules = [ReplacementRule(pattern, replacement) for pattern, replacement in rules]
    expression = expression
    replaced = True
    replace_count = 0
    while replaced and replace_count < max_count:
        replaced = False
        for subexpr, pos in preorder_iter_with_position(expression):
            for pattern, replacement in rules:
                try:
                    subst = next(match(subexpr, pattern))
                    result = replacement(**subst)
                    expression = replace(expression, pos, result)
                    replaced = True
                    break
                except StopIteration:
                    pass
            if replaced:
                break
        replace_count += 1

    return expression


Instruction:
How would you explain what the following Python 3 function does
def replace_all_post_order(expression: Expression, rules: Iterable[ReplacementRule]) \
        -> Union[Expression, Sequence[Expression]]:
    """Replace all occurrences of the patterns according to the replacement rules.

    A replacement rule consists of a *pattern*, that is matched against any subexpression
    of the expression. If a match is found, the *replacement* callback of the rule is called with
    the variables from the match substitution. Whatever the callback returns is used as a replacement for the
    matched subexpression. This can either be a single expression or a sequence of expressions, which is then
    integrated into the surrounding operation in place of the subexpression.

    Note that the pattern can therefore not be a single sequence variable/wildcard, because only single expressions
    will be matched.

    Args:
        expression:
            The expression to which the replacement rules are applied.
        rules:
            A collection of replacement rules that are applied to the expression.
        max_count:
            If given, at most *max_count* applications of the rules are performed. Otherwise, the rules
            are applied until there is no more match. If the set of replacement rules is not confluent,
            the replacement might not terminate without a *max_count* set.

    Returns:
        The resulting expression after the application of the replacement rules. This can also be a sequence of
        expressions, if the root expression is replaced with a sequence of expressions by a rule.
    """
    return _replace_all_post_order(expression, rules)[0]

Response: Replaces all occurrences of the patterns according to the replacement rules.


Instruction:
How would you code a function in Python 3 to
check whether the given subject matches given pattern.

Response:
def is_match(subject: Expression, pattern: Expression) -> bool:
    """
    Check whether the given *subject* matches given *pattern*.

    Args:
        subject:
            The subject.
        pattern:
            The pattern.

    Returns:
        True iff the subject matches the pattern.
    """
    return any(True for _ in match(subject, pattern))


Instruction:
Given the following Python 3 function, write the documentation
def as_graph(self) -> Graph:  # pragma: no cover
        if Graph is None:
            raise ImportError('The graphviz package is required to draw the graph.')
        graph = Graph()
        nodes_left = {}  # type: Dict[TLeft, str]
        nodes_right = {}  # type: Dict[TRight, str]
        node_id = 0
        for (left, right), value in self._edges.items():
            if left not in nodes_left:
                name = 'node{:d}'.format(node_id)
                nodes_left[left] = name
                graph.node(name, label=str(left))
                node_id += 1
            if right not in nodes_right:
                name = 'node{:d}'.format(node_id)
                nodes_right[right] = name
                graph.node(name, label=str(right))
                node_id += 1
            edge_label = value is not True and str(value) or ''
            graph.edge(nodes_left[left], nodes_right[right], edge_label)
        return graph

Response: Returns a graphviz. Graph representation of this bipartite graph.


Instruction:
Explain what the following Python 3 code does
def find_matching(self) -> Dict[TLeft, TRight]:
        # The directed graph is represented as a dictionary of edges
        # The key is the tail of all edges which are represented by the value
        # The value is a set of heads for the all edges originating from the tail (key)
        # In addition, the graph stores which part of the bipartite graph a node originated from
        # to avoid problems when a value exists in both halfs.
        # Only one direction of the undirected edge is needed for the HopcroftKarp class
        directed_graph = {}  # type: Dict[Tuple[int, TLeft], Set[Tuple[int, TRight]]]

        for (left, right) in self._edges:
            tail = (LEFT, left)
            head = (RIGHT, right)
            if tail not in directed_graph:
                directed_graph[tail] = {head}
            else:
                directed_graph[tail].add(head)

        matching = HopcroftKarp(directed_graph).maximum_matching()

        # Filter out the partitions (LEFT and RIGHT) and only return the matching edges
        # that go from LEFT to RIGHT
        return dict((tail[1], head[1]) for tail, head in matching.items() if tail[0] == LEFT)

Response: Finds a matching in the bipartite graph.


Instruction:
Explain what the following Python 3 code does
def without_nodes(self, edge: Edge) -> 'BipartiteGraph[TLeft, TRight, TEdgeValue]':
        return BipartiteGraph(((n1, n2), v) for (n1, n2), v in self._edges.items() if n1 != edge[0] and n2 != edge[1])

Response: Returns a copy of this bipartite graph with the given edge and its adjacent nodes removed.


Instruction:
Explain what the following Python 3 code does
def without_edge(self, edge: Edge) -> 'BipartiteGraph[TLeft, TRight, TEdgeValue]':
        return BipartiteGraph((e2, v) for e2, v in self._edges.items() if edge != e2)

Response: Returns a copy of this bipartite graph with the given edge removed.


Instruction:
Can you implement a function in Python 3 that
returns a new subgraph where only the nodes from the given sets are included.

Response:
def limited_to(self, left: Set[TLeft], right: Set[TRight]) -> 'BipartiteGraph[TLeft, TRight, TEdgeValue]':
        """Returns the induced subgraph where only the nodes from the given sets are included."""
        return BipartiteGraph(((n1, n2), v) for (n1, n2), v in self._edges.items() if n1 in left and n2 in right)


Instruction:
Write a Python 3 script to
return a graphviz. Digraph representation of this directed match graph.

Response:
def as_graph(self) -> Digraph:  # pragma: no cover
        """Returns a :class:`graphviz.Digraph` representation of this directed match graph."""
        if Digraph is None:
            raise ImportError('The graphviz package is required to draw the graph.')
        graph = Digraph()

        subgraphs = [Digraph(graph_attr={'rank': 'same'}), Digraph(graph_attr={'rank': 'same'})]
        nodes = [{}, {}]  # type: List[Dict[Union[TLeft, TRight], str]]
        edges = []  # type: List [Tuple[str, str]]
        node_id = 0
        for (tail_part, tail), head_set in self.items():
            if tail not in nodes[tail_part]:
                name = 'node{:d}'.format(node_id)
                nodes[tail_part][tail] = name
                subgraphs[tail_part].node(name, label=str(tail))
                node_id += 1
            for head_part, head in head_set:
                if head not in nodes[head_part]:
                    name = 'node{:d}'.format(node_id)
                    nodes[head_part][head] = name
                    subgraphs[head_part].node(name, label=str(head))
                    node_id += 1
                edges.append((nodes[tail_part][tail], nodes[head_part][head]))
        graph.subgraph(subgraphs[0])
        graph.subgraph(subgraphs[1])
        for tail_node, head_node in edges:
            graph.edge(tail_node, head_node)
        return graph


Instruction:
How would you explain what the following Python 3 function does
def is_constant(expression):
    if isinstance(expression, Wildcard):
        return False
    if isinstance(expression, Expression):
        return expression.is_constant
    if isinstance(expression, Operation):
        return all(is_constant(o) for o in op_iter(expression))
    return True

Response: Check if the given expression is constant i. e. it does not contain Wildcards.


Instruction:
Write a Python 3 function that can
check if the given expression is syntactic.

Response:
def is_syntactic(expression):
    """
    Check if the given expression is syntactic, i.e. it does not contain sequence wildcards or
    associative/commutative operations.
    """
    if isinstance(expression, Wildcard):
        return expression.fixed_size
    if isinstance(expression, Expression):
        return expression.is_syntactic
    if isinstance(expression, (AssociativeOperation, CommutativeOperation)):
        return False
    if isinstance(expression, Operation):
        return all(is_syntactic(o) for o in op_iter(expression))
    return True


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_head(expression):
    if isinstance(expression, Wildcard):
        if isinstance(expression, SymbolWildcard):
            return expression.symbol_type
        return None
    return type(expression)

Response: Returns the given expression s head.


Instruction:
Can you generate the documentation for the following Python 3 function
def match_head(subject, pattern):
    if isinstance(pattern, Pattern):
        pattern = pattern.expression
    pattern_head = get_head(pattern)
    if pattern_head is None:
        return True
    if issubclass(pattern_head, OneIdentityOperation):
        return True
    subject_head = get_head(subject)
    assert subject_head is not None
    return issubclass(subject_head, pattern_head)

Response: Checks if the head of subject matches the pattern s head.


Instruction:
Write a Python 3 script to
iterate over the expression in preorder.

Response:
def preorder_iter(expression):
    """Iterate over the expression in preorder."""
    yield expression
    if isinstance(expression, Operation):
        for operand in op_iter(expression):
            yield from preorder_iter(operand)


Instruction:
Write a Python 3 function that can
iterate over the expression in preorder. Also yields the position of each subexpression.

Response:
def preorder_iter_with_position(expression):
    """Iterate over the expression in preorder.

    Also yields the position of each subexpression.
    """
    yield expression, ()
    if isinstance(expression, Operation):
        for i, operand in enumerate(op_iter(expression)):
            for child, pos in preorder_iter_with_position(operand):
                yield child, (i, ) + pos


Instruction:
Can you tell what is the following Python 3 function doing
def is_anonymous(expression):
    if hasattr(expression, 'variable_name') and expression.variable_name:
        return False
    if isinstance(expression, Operation):
        return all(is_anonymous(o) for o in op_iter(expression))
    return True

Response: Returns True iff the expression contains any variables.


Instruction:
How would you code a function in Python 3 to
return True iff the expression contains any of the variables from the given set.

Response:
def contains_variables_from_set(expression, variables):
    """Returns True iff the expression contains any of the variables from the given set."""
    if hasattr(expression, 'variable_name') and expression.variable_name in variables:
        return True
    if isinstance(expression, Operation):
        return any(contains_variables_from_set(o, variables) for o in op_iter(expression))
    return False


Instruction:
Can you implement a function in Python 3 that
returns the set of variable names in the given expression.

Response:
def get_variables(expression, variables=None):
    """Returns the set of variable names in the given expression."""
    if variables is None:
        variables = set()
    if hasattr(expression, 'variable_name') and expression.variable_name is not None:
        variables.add(expression.variable_name)
    if isinstance(expression, Operation):
        for operand in op_iter(expression):
            get_variables(operand, variables)
    return variables


Instruction:
Write a Python 3 script for
returning a new expression with renamed variables.

Response:
def rename_variables(expression: Expression, renaming: Dict[str, str]) -> Expression:
    """Rename the variables in the expression according to the given dictionary.

    Args:
        expression:
            The expression in which the variables are renamed.
        renaming:
            The renaming dictionary. Maps old variable names to new ones.
            Variable names not occuring in the dictionary are left unchanged.

    Returns:
        The expression with renamed variables.
    """
    if isinstance(expression, Operation):
        if hasattr(expression, 'variable_name'):
            variable_name = renaming.get(expression.variable_name, expression.variable_name)
            return create_operation_expression(
                expression, [rename_variables(o, renaming) for o in op_iter(expression)], variable_name=variable_name
            )
        operands = [rename_variables(o, renaming) for o in op_iter(expression)]
        return create_operation_expression(expression, operands)
    elif isinstance(expression, Expression):
        expression = expression.__copy__()
        expression.variable_name = renaming.get(expression.variable_name, expression.variable_name)
    return expression


Instruction:
Create a Python 3 function to
return an iterator over the integer vectors that are not larger than or equal to max_vector.

Response:
def fixed_integer_vector_iter(max_vector: Tuple[int, ...], vector_sum: int) -> Iterator[Tuple[int, ...]]:
    """
    Return an iterator over the integer vectors which

    - are componentwise less than or equal to *max_vector*, and
    - are non-negative, and where
    - the sum of their components is exactly *vector_sum*.

    The iterator yields the vectors in lexicographical order.

    Examples:

        List all vectors that are between ``(0, 0)`` and ``(2, 2)`` componentwise, where the sum of components is 2:

        >>> vectors = list(fixed_integer_vector_iter([2, 2], 2))
        >>> vectors
        [(0, 2), (1, 1), (2, 0)]
        >>> list(map(sum, vectors))
        [2, 2, 2]

    Args:
        max_vector:
            Maximum vector for the iteration. Every yielded result will be less than or equal to this componentwise.
        vector_sum:
            Every iterated vector will have a component sum equal to this value.

    Yields:
        All non-negative vectors that have the given sum and are not larger than the given maximum.

    Raises:
        ValueError:
            If *vector_sum* is negative.
    """
    if vector_sum < 0:
        raise ValueError("Vector sum must not be negative")
    if len(max_vector) == 0:
        if vector_sum == 0:
            yield tuple()
        return
    total = sum(max_vector)
    if vector_sum <= total:
        start = max(max_vector[0] + vector_sum - total, 0)
        end = min(max_vector[0], vector_sum)
        for j in range(start, end + 1):
            for vec in fixed_integer_vector_iter(max_vector[1:], vector_sum - j):
                yield (j, ) + vec


Instruction:
Write a Python 3 script for
yielding all weak compositions of integer n into num_parts parts.

Response:
def weak_composition_iter(n: int, num_parts: int) -> Iterator[Tuple[int, ...]]:
    """Yield all weak compositions of integer *n* into *num_parts* parts.

    Each composition is yielded as a tuple. The generated partitions are order-dependant and not unique when
    ignoring the order of the components. The partitions are yielded in lexicographical order.

    Example:

        >>> compositions = list(weak_composition_iter(5, 2))
        >>> compositions
        [(0, 5), (1, 4), (2, 3), (3, 2), (4, 1), (5, 0)]

        We can easily verify that all compositions are indeed valid:

        >>> list(map(sum, compositions))
        [5, 5, 5, 5, 5, 5]

    The algorithm was adapted from an answer to this `Stackoverflow question`_.

    Args:
        n:
            The integer to partition.
        num_parts:
            The number of parts for the combination.

    Yields:
        All non-negative tuples that have the given sum and size.

    Raises:
        ValueError:
            If *n* or *num_parts* are negative.

    .. _Stackoverflow question: http://stackoverflow.com/questions/40538923/40540014#40540014
    """
    if n < 0:
        raise ValueError("Total must not be negative")
    if num_parts < 0:
        raise ValueError("Number of num_parts must not be negative")
    if num_parts == 0:
        if n == 0:
            yield tuple()
        return
    m = n + num_parts - 1
    last = (m, )
    first = (-1, )
    for t in itertools.combinations(range(m), num_parts - 1):
        yield tuple(v - u - 1 for u, v in zip(first + t, t + last))


Instruction:
Write a Python 3 function for
yielding all possible variable substitutions for given values and variables.

Response:
def commutative_sequence_variable_partition_iter(values: Multiset, variables: List[VariableWithCount]
                                                ) -> Iterator[Dict[str, Multiset]]:
    """Yield all possible variable substitutions for given values and variables.

    .. note::

        The results are not yielded in any particular order because the algorithm uses dictionaries. Dictionaries until
        Python 3.6 do not keep track of the insertion order.

    Example:

        For a subject like ``fc(a, a, a, b, b, c)`` and a pattern like ``f(x__, y___, y___)`` one can define the
        following input parameters for the partitioning:

        >>> x = VariableWithCount(name='x', count=1, minimum=1, default=None)
        >>> y = VariableWithCount(name='y', count=2, minimum=0, default=None)
        >>> values = Multiset('aaabbc')

        Then the solutions are found (and sorted to get a unique output):

        >>> substitutions = commutative_sequence_variable_partition_iter(values, [x, y])
        >>> as_strings = list(str(Substitution(substitution)) for substitution in substitutions)
        >>> for substitution in sorted(as_strings):
        ...     print(substitution)
        {x ↦ {a, a, a, b, b, c}, y ↦ {}}
        {x ↦ {a, a, a, c}, y ↦ {b}}
        {x ↦ {a, b, b, c}, y ↦ {a}}
        {x ↦ {a, c}, y ↦ {a, b}}

    Args:
        values:
            The multiset of values which are partitioned and distributed among the variables.
        variables:
            A list of the variables to distribute the values among. Each variable has a name, a count of how many times
            it occurs and a minimum number of values it needs.

    Yields:
        Each possible substitutions that is a valid partitioning of the values among the variables.
    """
    if len(variables) == 1:
        yield from _commutative_single_variable_partiton_iter(values, variables[0])
        return

    generators = []
    for value, count in values.items():
        generators.append(_make_variable_generator_factory(value, count, variables))

    initial = dict((var.name, Multiset()) for var in variables)  # type: Dict[str, 'Multiset[T]']
    for subst in generator_chain(initial, *generators):
        valid = True
        for var in variables:
            if var.default is not None and len(subst[var.name]) == 0:
                subst[var.name] = var.default
            elif len(subst[var.name]) < var.minimum:
                valid = False
                break
        if valid:
            if None in subst:
                del subst[None]
            yield subst


Instruction:
Can you tell what is the following Python 3 function doing
def get_short_lambda_source(lambda_func: LambdaType) -> Optional[str]:
    try:
        all_source_lines, lnum = inspect.findsource(lambda_func)
        source_lines, _ = inspect.getsourcelines(lambda_func)
    except (IOError, TypeError):
        return None
    all_source_lines = [l.rstrip('\r\n') for l in all_source_lines]
    block_end = lnum + len(source_lines)
    source_ast = None
    for i in range(lnum, -1, -1):
        try:
            block = all_source_lines[i:block_end]
            if block[0].startswith(' ') or block[0].startswith('\t'):
                block.insert(0, 'with 0:')
            source_ast = ast.parse(os.linesep.join(block))
        except (SyntaxError, tokenize.TokenError):
            pass
        else:
            break
    nv = LambdaNodeVisitor(block)
    nv.visit(source_ast)
    lambda_code = lambda_func.__code__
    for candidate_code, lambda_text in nv.lambdas:
        candidate_code = candidate_code.co_consts[0]
        # We don't check for direct equivalence since the flags can be different
        if (candidate_code.co_code == lambda_code.co_code and
            candidate_code.co_consts == lambda_code.co_consts and
            candidate_code.co_names == lambda_code.co_names and
            candidate_code.co_varnames == lambda_code.co_varnames and
            candidate_code.co_cellvars == lambda_code.co_cellvars and
            candidate_code.co_freevars == lambda_code.co_freevars):
            return lambda_text[lambda_text.index(':')+1:].strip()

    return None

Response: Return the source of a short lambda function.


Instruction:
How would you explain what the following Python 3 function does
def extended_euclid(a: int, b: int) -> Tuple[int, int, int]:
    if b == 0:
        return (1, 0, a)

    x0, y0, d = extended_euclid(b, a % b)
    x, y = y0, x0 - (a // b) * y0

    return (x, y, d)

Response: Extended Euclidean algorithm that computes the Bézout coefficients and GCD of a and b.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def generator_chain(initial_data: T, *factories: Callable[[T], Iterator[T]]) -> Iterator[T]:
    generator_count = len(factories)
    if generator_count == 0:
        yield initial_data
        return
    generators = [None] * generator_count  # type: List[Optional[Iterator[T]]]
    next_data = initial_data
    generator_index = 0
    while True:
        try:
            while generator_index < generator_count:
                if generators[generator_index] is None:
                    generators[generator_index] = factories[generator_index](next_data)
                next_data = next(generators[generator_index])
                generator_index += 1
            yield next_data
            generator_index -= 1
        except StopIteration:
            generators[generator_index] = None
            generator_index -= 1
            if generator_index < 0:
                break

Response: A generator that yields data from one of the given generators.


Instruction:
Implement a function in Python 3 to
try to add a variable with its replacement to the substitution.

Response:
def try_add_variable(self, variable_name: str, replacement: VariableReplacement) -> None:
        """Try to add the variable with its replacement to the substitution.

        This considers an existing replacement and will only succeed if the new replacement
        can be merged with the old replacement. Merging can occur if either the two replacements
        are equivalent. Replacements can also be merged if the old replacement for the variable_name was
        unordered (i.e. a :class:`~.Multiset`) and the new one is an equivalent ordered version of it:

        >>> subst = Substitution({'x': Multiset(['a', 'b'])})
        >>> subst.try_add_variable('x', ('a', 'b'))
        >>> print(subst)
        {x ↦ (a, b)}

        Args:
            variable:
                The name of the variable to add.
            replacement:
                The replacement for the variable.

        Raises:
            ValueError:
                if the variable cannot be merged because it conflicts with the existing
                substitution for the variable_name.
        """
        if variable_name not in self:
            self[variable_name] = replacement.copy() if isinstance(replacement, Multiset) else replacement
        else:
            existing_value = self[variable_name]

            if isinstance(existing_value, tuple):
                if isinstance(replacement, Multiset):
                    if Multiset(existing_value) != replacement:
                        raise ValueError
                elif replacement != existing_value:
                    raise ValueError
            elif isinstance(existing_value, Multiset):
                if not isinstance(replacement, (tuple, list, Multiset)):
                    raise ValueError
                compare_value = Multiset(replacement)
                if existing_value == compare_value:
                    if not isinstance(replacement, Multiset):
                        self[variable_name] = replacement
                else:
                    raise ValueError
            elif replacement != existing_value:
                raise ValueError


Instruction:
Can you create a Python 3 function that
creates a new substitution with the given variable added.

Response:
def union_with_variable(self, variable: str, replacement: VariableReplacement) -> 'Substitution':
        """Try to create a new substitution with the given variable added.

        See :meth:`try_add_variable` for a version of this method that modifies the substitution
        in place.

        Args:
            variable_name:
                The name of the variable to add.
            replacement:
                The substitution for the variable.

        Returns:
            The new substitution with the variable_name added or merged.

        Raises:
            ValueError:
                if the variable cannot be merged because it conflicts with the existing
                substitution for the variable.
        """
        new_subst = Substitution(self)
        new_subst.try_add_variable(variable, replacement)
        return new_subst


Instruction:
Can you create a Python 3 function that
extracts the variable substitution for the given pattern and subject. This assumes that subject and pattern already match when being considered as linear. Also, they both must be :term:`syntactic`, as sequence variables cannot be handled here. All that this method does is checking whether all the substitutions for the variables can be unified. So, in case it returns ``False``, the substitution is invalid for the match. ..warning:: This method mutates the substitution and will even do so in case the extraction fails. Create a copy before using this method if you need to preserve the original substitution. Example: With an empty initial substitution and a linear pattern, the extraction will always succeed: >>> subst = Substitution() >>> subst.extract_substitution(f(a, b), f(x_, y_)) True >>> print(subst) {x ↦ a, y ↦ b} Clashing values for existing variables will fail: >>> subst.extract_substitution(b, x_) False For non-linear patterns, the extraction can also fail with an empty substitution: >>> subst = Substitution() >>> subst.extract_substitution(f(a, b), f(x_, x_)) False >>> print(subst) {x ↦ a} Note that the initial substitution got mutated even though the extraction failed! Args: subject: A :term:`syntactic` subject that matches the pattern. pattern: A :term:`syntactic` pattern that matches the subject. Returns: ``True`` iff the substitution could be extracted successfully.

Response:
def extract_substitution(self, subject: 'expressions.Expression', pattern: 'expressions.Expression') -> bool:
        """Extract the variable substitution for the given pattern and subject.

        This assumes that subject and pattern already match when being considered as linear.
        Also, they both must be :term:`syntactic`, as sequence variables cannot be handled here.
        All that this method does is checking whether all the substitutions for the variables can be unified.
        So, in case it returns ``False``, the substitution is invalid for the match.

        ..warning::

            This method mutates the substitution and will even do so in case the extraction fails.

            Create a copy before using this method if you need to preserve the original substitution.

        Example:

            With an empty initial substitution and a linear pattern, the extraction will always succeed:

            >>> subst = Substitution()
            >>> subst.extract_substitution(f(a, b), f(x_, y_))
            True
            >>> print(subst)
            {x ↦ a, y ↦ b}

            Clashing values for existing variables will fail:

            >>> subst.extract_substitution(b, x_)
            False

            For non-linear patterns, the extraction can also fail with an empty substitution:

            >>> subst = Substitution()
            >>> subst.extract_substitution(f(a, b), f(x_, x_))
            False
            >>> print(subst)
            {x ↦ a}

            Note that the initial substitution got mutated even though the extraction failed!

        Args:
            subject:
                A :term:`syntactic` subject that matches the pattern.
            pattern:
                A :term:`syntactic` pattern that matches the subject.

        Returns:
            ``True`` iff the substitution could be extracted successfully.
        """
        if getattr(pattern, 'variable_name', False):
            try:
                self.try_add_variable(pattern.variable_name, subject)
            except ValueError:
                return False
            return True
        elif isinstance(pattern, expressions.Operation):
            assert isinstance(subject, type(pattern))
            assert op_len(subject) == op_len(pattern)
            op_expression = cast(expressions.Operation, subject)
            for subj, patt in zip(op_iter(op_expression), op_iter(pattern)):
                if not self.extract_substitution(subj, patt):
                    return False
        return True


Instruction:
How would you implement a function in Python 3 that
returns a new substitution that is the union of this and the others.

Response:
def union(self, *others: 'Substitution') -> 'Substitution':
        """Try to merge the substitutions.

        If a variable occurs in multiple substitutions, try to merge the replacements.
        See :meth:`union_with_variable` to see how replacements are merged.

        Does not modify any of the original substitutions.

        Example:

        >>> subst1 = Substitution({'x': Multiset(['a', 'b']), 'z': a})
        >>> subst2 = Substitution({'x': ('a', 'b'), 'y': ('c', )})
        >>> print(subst1.union(subst2))
        {x ↦ (a, b), y ↦ (c), z ↦ a}

        Args:
            others:
                The other substitutions to merge with this one.

        Returns:
            The new substitution with the other substitutions merged.

        Raises:
            ValueError:
                if a variable occurs in multiple substitutions but cannot be merged because the
                substitutions conflict.
        """
        new_subst = Substitution(self)
        for other in others:
            for variable_name, replacement in other.items():
                new_subst.try_add_variable(variable_name, replacement)
        return new_subst


Instruction:
Here you have a function in Python 3, explain what it does
def rename(self, renaming: Dict[str, str]) -> 'Substitution':
        return Substitution((renaming.get(name, name), value) for name, value in self.items())

Response: Returns a copy of the substitution with renamed variables.


Instruction:
Can you tell what is the following Python 3 function doing
def is_operation(term: Any) -> bool:
    return isinstance(term, type) and issubclass(term, Operation)

Response: Return True iff the given term is a subclass of Operation.


Instruction:
Explain what the following Python 3 code does
def is_symbol_wildcard(term: Any) -> bool:
    return isinstance(term, type) and issubclass(term, Symbol)

Response: Return True iff the given term is a subclass of Symbol.


Instruction:
Can you tell what is the following Python 3 function doing
def _get_symbol_wildcard_label(state: '_State', symbol: Symbol) -> Type[Symbol]:
    return next((t for t in state.keys() if is_symbol_wildcard(t) and isinstance(symbol, t)), None)

Response: Return the transition target for the given symbol type from the given state or None if it does not exist.


Instruction:
Can you generate the documentation for the following Python 3 function
def _term_str(term: TermAtom) -> str:  # pragma: no cover
    if is_operation(term):
        return term.name + '('
    elif is_symbol_wildcard(term):
        return '*{!s}'.format(term.__name__)
    elif isinstance(term, Wildcard):
        return '*{!s}{!s}'.format(term.min_count, (not term.fixed_size) and '+' or '')
    elif term == Wildcard:
        return '*'
    else:
        return str(term)

Response: Return a string representation of a term atom.


Instruction:
Here you have a function in Python 3, explain what it does
def is_syntactic(self):
        for term in self._terms:
            if isinstance(term, Wildcard) and not term.fixed_size:
                return False
            if is_operation(term) and issubclass(term, (AssociativeOperation, CommutativeOperation)):
                return False
        return True

Response: True iff the flatterm is syntactic.


Instruction:
Here you have a function in Python 3, explain what it does
def merged(cls, *flatterms: 'FlatTerm') -> 'FlatTerm':
        return cls(cls._combined_wildcards_iter(sum(flatterms, cls.empty())))

Response: Concatenate the given flatterms into a single flatterm.


Instruction:
Make a summary of the following Python 3 code
def _flatterm_iter(cls, expression: Expression) -> Iterator[TermAtom]:
        if isinstance(expression, Operation):
            yield type(expression)
            for operand in op_iter(expression):
                yield from cls._flatterm_iter(operand)
            yield OPERATION_END
        elif isinstance(expression, SymbolWildcard):
            yield expression.symbol_type
        elif isinstance(expression, (Symbol, Wildcard)):
            yield expression
        else:
            assert False, "Unreachable unless a new unsupported expression type is added."

Response: Generator that yields the atoms of the expressions in prefix notation with operation end markers.


Instruction:
Here you have a function in Python 3, explain what it does
def _combined_wildcards_iter(flatterm: Iterator[TermAtom]) -> Iterator[TermAtom]:
        last_wildcard = None  # type: Optional[Wildcard]
        for term in flatterm:
            if isinstance(term, Wildcard) and not isinstance(term, SymbolWildcard):
                if last_wildcard is not None:
                    new_min_count = last_wildcard.min_count + term.min_count
                    new_fixed_size = last_wildcard.fixed_size and term.fixed_size
                    last_wildcard = Wildcard(new_min_count, new_fixed_size)
                else:
                    last_wildcard = Wildcard(term.min_count, term.fixed_size)
            else:
                if last_wildcard is not None:
                    yield last_wildcard
                    last_wildcard = None
                yield term
        if last_wildcard is not None:
            yield last_wildcard

Response: Combine consecutive wildcards in a flatterm into a single one.


Instruction:
Make a summary of the following Python 3 code
def labels(self) -> Set[TransitionLabel]:
        labels = set()  # type: Set[TransitionLabel]
        if self.state1 is not None and self.fixed != 1:
            labels.update(self.state1.keys())
        if self.state2 is not None and self.fixed != 2:
            labels.update(self.state2.keys())
        if self.fixed != 0:
            if self.fixed == 1 and self.state2 is None:
                labels.add(OPERATION_END)
            elif self.fixed == 2 and self.state1 is None:
                labels.add(OPERATION_END)
            labels.add(Wildcard)
        return labels

Response: Return the set of transition labels to examine for this queue state.


Instruction:
Write a Python 3 function that can
add a pattern to the discrimination net.

Response:
def add(self, pattern: Union[Pattern, FlatTerm], final_label: T=None) -> int:
        """Add a pattern to the discrimination net.

        Args:
            pattern:
                The pattern which is added to the DiscriminationNet. If an expression is given, it will be converted to
                a `FlatTerm` for internal processing. You can also pass a `FlatTerm` directly.
            final_label:
                A label that is returned if the pattern matches when using :meth:`match`. This will default to the
                pattern itself.

        Returns:
            The index of the newly added pattern. This is used internally to later to get the pattern and its final
            label once a match is found.
        """
        index = len(self._patterns)
        self._patterns.append((pattern, final_label))
        flatterm = FlatTerm(pattern.expression) if not isinstance(pattern, FlatTerm) else pattern
        if flatterm.is_syntactic or len(flatterm) == 1:
            net = self._generate_syntactic_net(flatterm, index)
        else:
            net = self._generate_net(flatterm, index)

        if self._root:
            self._root = self._product_net(self._root, net)
        else:
            self._root = net
        return index