Instruction:
Can you create a Python 3 function that
performs an asof merge between two DataFrames.

Response:
def merge_asof(left, right, on=None,
               left_on=None, right_on=None,
               left_index=False, right_index=False,
               by=None, left_by=None, right_by=None,
               suffixes=('_x', '_y'),
               tolerance=None,
               allow_exact_matches=True,
               direction='backward'):
    """Perform an asof merge. This is similar to a left-join except that we
    match on nearest key rather than equal keys.

    Both DataFrames must be sorted by the key.

    For each row in the left DataFrame:

      - A "backward" search selects the last row in the right DataFrame whose
        'on' key is less than or equal to the left's key.

      - A "forward" search selects the first row in the right DataFrame whose
        'on' key is greater than or equal to the left's key.

      - A "nearest" search selects the row in the right DataFrame whose 'on'
        key is closest in absolute distance to the left's key.

    The default is "backward" and is compatible in versions below 0.20.0.
    The direction parameter was added in version 0.20.0 and introduces
    "forward" and "nearest".

    Optionally match on equivalent keys with 'by' before searching with 'on'.

    .. versionadded:: 0.19.0

    Parameters
    ----------
    left : DataFrame
    right : DataFrame
    on : label
        Field name to join on. Must be found in both DataFrames.
        The data MUST be ordered. Furthermore this must be a numeric column,
        such as datetimelike, integer, or float. On or left_on/right_on
        must be given.
    left_on : label
        Field name to join on in left DataFrame.
    right_on : label
        Field name to join on in right DataFrame.
    left_index : boolean
        Use the index of the left DataFrame as the join key.

        .. versionadded:: 0.19.2

    right_index : boolean
        Use the index of the right DataFrame as the join key.

        .. versionadded:: 0.19.2

    by : column name or list of column names
        Match on these columns before performing merge operation.
    left_by : column name
        Field names to match on in the left DataFrame.

        .. versionadded:: 0.19.2

    right_by : column name
        Field names to match on in the right DataFrame.

        .. versionadded:: 0.19.2

    suffixes : 2-length sequence (tuple, list, ...)
        Suffix to apply to overlapping column names in the left and right
        side, respectively.
    tolerance : integer or Timedelta, optional, default None
        Select asof tolerance within this range; must be compatible
        with the merge index.
    allow_exact_matches : boolean, default True

        - If True, allow matching with the same 'on' value
          (i.e. less-than-or-equal-to / greater-than-or-equal-to)
        - If False, don't match the same 'on' value
          (i.e., strictly less-than / strictly greater-than)

    direction : 'backward' (default), 'forward', or 'nearest'
        Whether to search for prior, subsequent, or closest matches.

        .. versionadded:: 0.20.0

    Returns
    -------
    merged : DataFrame

    See Also
    --------
    merge
    merge_ordered

    Examples
    --------
    >>> left = pd.DataFrame({'a': [1, 5, 10], 'left_val': ['a', 'b', 'c']})
    >>> left
        a left_val
    0   1        a
    1   5        b
    2  10        c

    >>> right = pd.DataFrame({'a': [1, 2, 3, 6, 7],
    ...                       'right_val': [1, 2, 3, 6, 7]})
    >>> right
       a  right_val
    0  1          1
    1  2          2
    2  3          3
    3  6          6
    4  7          7

    >>> pd.merge_asof(left, right, on='a')
        a left_val  right_val
    0   1        a          1
    1   5        b          3
    2  10        c          7

    >>> pd.merge_asof(left, right, on='a', allow_exact_matches=False)
        a left_val  right_val
    0   1        a        NaN
    1   5        b        3.0
    2  10        c        7.0

    >>> pd.merge_asof(left, right, on='a', direction='forward')
        a left_val  right_val
    0   1        a        1.0
    1   5        b        6.0
    2  10        c        NaN

    >>> pd.merge_asof(left, right, on='a', direction='nearest')
        a left_val  right_val
    0   1        a          1
    1   5        b          6
    2  10        c          7

    We can use indexed DataFrames as well.

    >>> left = pd.DataFrame({'left_val': ['a', 'b', 'c']}, index=[1, 5, 10])
    >>> left
       left_val
    1         a
    5         b
    10        c

    >>> right = pd.DataFrame({'right_val': [1, 2, 3, 6, 7]},
    ...                      index=[1, 2, 3, 6, 7])
    >>> right
       right_val
    1          1
    2          2
    3          3
    6          6
    7          7

    >>> pd.merge_asof(left, right, left_index=True, right_index=True)
       left_val  right_val
    1         a          1
    5         b          3
    10        c          7

    Here is a real-world times-series example

    >>> quotes
                         time ticker     bid     ask
    0 2016-05-25 13:30:00.023   GOOG  720.50  720.93
    1 2016-05-25 13:30:00.023   MSFT   51.95   51.96
    2 2016-05-25 13:30:00.030   MSFT   51.97   51.98
    3 2016-05-25 13:30:00.041   MSFT   51.99   52.00
    4 2016-05-25 13:30:00.048   GOOG  720.50  720.93
    5 2016-05-25 13:30:00.049   AAPL   97.99   98.01
    6 2016-05-25 13:30:00.072   GOOG  720.50  720.88
    7 2016-05-25 13:30:00.075   MSFT   52.01   52.03

    >>> trades
                         time ticker   price  quantity
    0 2016-05-25 13:30:00.023   MSFT   51.95        75
    1 2016-05-25 13:30:00.038   MSFT   51.95       155
    2 2016-05-25 13:30:00.048   GOOG  720.77       100
    3 2016-05-25 13:30:00.048   GOOG  720.92       100
    4 2016-05-25 13:30:00.048   AAPL   98.00       100

    By default we are taking the asof of the quotes

    >>> pd.merge_asof(trades, quotes,
    ...                       on='time',
    ...                       by='ticker')
                         time ticker   price  quantity     bid     ask
    0 2016-05-25 13:30:00.023   MSFT   51.95        75   51.95   51.96
    1 2016-05-25 13:30:00.038   MSFT   51.95       155   51.97   51.98
    2 2016-05-25 13:30:00.048   GOOG  720.77       100  720.50  720.93
    3 2016-05-25 13:30:00.048   GOOG  720.92       100  720.50  720.93
    4 2016-05-25 13:30:00.048   AAPL   98.00       100     NaN     NaN

    We only asof within 2ms between the quote time and the trade time

    >>> pd.merge_asof(trades, quotes,
    ...                       on='time',
    ...                       by='ticker',
    ...                       tolerance=pd.Timedelta('2ms'))
                         time ticker   price  quantity     bid     ask
    0 2016-05-25 13:30:00.023   MSFT   51.95        75   51.95   51.96
    1 2016-05-25 13:30:00.038   MSFT   51.95       155     NaN     NaN
    2 2016-05-25 13:30:00.048   GOOG  720.77       100  720.50  720.93
    3 2016-05-25 13:30:00.048   GOOG  720.92       100  720.50  720.93
    4 2016-05-25 13:30:00.048   AAPL   98.00       100     NaN     NaN

    We only asof within 10ms between the quote time and the trade time
    and we exclude exact matches on time. However *prior* data will
    propagate forward

    >>> pd.merge_asof(trades, quotes,
    ...                       on='time',
    ...                       by='ticker',
    ...                       tolerance=pd.Timedelta('10ms'),
    ...                       allow_exact_matches=False)
                         time ticker   price  quantity     bid     ask
    0 2016-05-25 13:30:00.023   MSFT   51.95        75     NaN     NaN
    1 2016-05-25 13:30:00.038   MSFT   51.95       155   51.97   51.98
    2 2016-05-25 13:30:00.048   GOOG  720.77       100     NaN     NaN
    3 2016-05-25 13:30:00.048   GOOG  720.92       100     NaN     NaN
    4 2016-05-25 13:30:00.048   AAPL   98.00       100     NaN     NaN
    """
    op = _AsOfMerge(left, right,
                    on=on, left_on=left_on, right_on=right_on,
                    left_index=left_index, right_index=right_index,
                    by=by, left_by=left_by, right_by=right_by,
                    suffixes=suffixes,
                    how='asof', tolerance=tolerance,
                    allow_exact_matches=allow_exact_matches,
                    direction=direction)
    return op.get_result()


Instruction:
Here you have a function in Python 3, explain what it does
def _restore_dropped_levels_multijoin(left, right, dropped_level_names,
                                      join_index, lindexer, rindexer):
    """
    *this is an internal non-public method*

    Returns the levels, labels and names of a multi-index to multi-index join.
    Depending on the type of join, this method restores the appropriate
    dropped levels of the joined multi-index.
    The method relies on lidx, rindexer which hold the index positions of
    left and right, where a join was feasible

    Parameters
    ----------
    left : MultiIndex
        left index
    right : MultiIndex
        right index
    dropped_level_names : str array
        list of non-common level names
    join_index : MultiIndex
        the index of the join between the
        common levels of left and right
    lindexer : intp array
        left indexer
    rindexer : intp array
        right indexer

    Returns
    -------
    levels : list of Index
        levels of combined multiindexes
    labels : intp array
        labels of combined multiindexes
    names : str array
        names of combined multiindexes

    """

    def _convert_to_mulitindex(index):
        if isinstance(index, MultiIndex):
            return index
        else:
            return MultiIndex.from_arrays([index.values],
                                          names=[index.name])

    # For multi-multi joins with one overlapping level,
    # the returned index if of type Index
    # Assure that join_index is of type MultiIndex
    # so that dropped levels can be appended
    join_index = _convert_to_mulitindex(join_index)

    join_levels = join_index.levels
    join_codes = join_index.codes
    join_names = join_index.names

    # lindexer and rindexer hold the indexes where the join occurred
    # for left and right respectively. If left/right is None then
    # the join occurred on all indices of left/right
    if lindexer is None:
        lindexer = range(left.size)

    if rindexer is None:
        rindexer = range(right.size)

    # Iterate through the levels that must be restored
    for dropped_level_name in dropped_level_names:
        if dropped_level_name in left.names:
            idx = left
            indexer = lindexer
        else:
            idx = right
            indexer = rindexer

        # The index of the level name to be restored
        name_idx = idx.names.index(dropped_level_name)

        restore_levels = idx.levels[name_idx]
        # Inject -1 in the codes list where a join was not possible
        # IOW indexer[i]=-1
        codes = idx.codes[name_idx]
        restore_codes = algos.take_nd(codes, indexer, fill_value=-1)

        join_levels = join_levels + [restore_levels]
        join_codes = join_codes + [restore_codes]
        join_names = join_names + [dropped_level_name]

    return join_levels, join_codes, join_names

Response: This method restores the dropped levels of a multi - index to the appropriate MultiIndex.


Instruction:
Create a Python 3 function to
restore index levels specified as on parameters.

Response:
def _maybe_restore_index_levels(self, result):
        """
        Restore index levels specified as `on` parameters

        Here we check for cases where `self.left_on` and `self.right_on` pairs
        each reference an index level in their respective DataFrames. The
        joined columns corresponding to these pairs are then restored to the
        index of `result`.

        **Note:** This method has side effects. It modifies `result` in-place

        Parameters
        ----------
        result: DataFrame
            merge result

        Returns
        -------
        None
        """
        names_to_restore = []
        for name, left_key, right_key in zip(self.join_names,
                                             self.left_on,
                                             self.right_on):
            if (self.orig_left._is_level_reference(left_key) and
                    self.orig_right._is_level_reference(right_key) and
                    name not in result.index.names):

                names_to_restore.append(name)

        if names_to_restore:
            result.set_index(names_to_restore, inplace=True)


Instruction:
Write a Python 3 script to
return the join indexers

Response:
def _get_join_indexers(self):
        """ return the join indexers """
        return _get_join_indexers(self.left_join_keys,
                                  self.right_join_keys,
                                  sort=self.sort,
                                  how=self.how)


Instruction:
Implement a Python 3 function for
creating a join index by rearranging one index to match another index.

Response:
def _create_join_index(self, index, other_index, indexer,
                           other_indexer, how='left'):
        """
        Create a join index by rearranging one index to match another

        Parameters
        ----------
        index: Index being rearranged
        other_index: Index used to supply values not found in index
        indexer: how to rearrange index
        how: replacement is only necessary if indexer based on other_index

        Returns
        -------
        join_index
        """
        join_index = index.take(indexer)
        if (self.how in (how, 'outer') and
                not isinstance(other_index, MultiIndex)):
            # if final index requires values in other_index but not target
            # index, indexer may hold missing (-1) values, causing Index.take
            # to take the final value in target index
            mask = indexer == -1
            if np.any(mask):
                # if values missing (-1) from target index,
                # take from other_index instead
                join_list = join_index.to_numpy()
                other_list = other_index.take(other_indexer).to_numpy()
                join_list[mask] = other_list[mask]
                join_index = Index(join_list, dtype=join_index.dtype,
                                   name=join_index.name)
        return join_index


Instruction:
Write a Python 3 script for
getting the keys that should be used to merge the two entries.

Response:
def _get_merge_keys(self):
        """
        Note: has side effects (copy/delete key columns)

        Parameters
        ----------
        left
        right
        on

        Returns
        -------
        left_keys, right_keys
        """
        left_keys = []
        right_keys = []
        join_names = []
        right_drop = []
        left_drop = []

        left, right = self.left, self.right

        is_lkey = lambda x: is_array_like(x) and len(x) == len(left)
        is_rkey = lambda x: is_array_like(x) and len(x) == len(right)

        # Note that pd.merge_asof() has separate 'on' and 'by' parameters. A
        # user could, for example, request 'left_index' and 'left_by'. In a
        # regular pd.merge(), users cannot specify both 'left_index' and
        # 'left_on'. (Instead, users have a MultiIndex). That means the
        # self.left_on in this function is always empty in a pd.merge(), but
        # a pd.merge_asof(left_index=True, left_by=...) will result in a
        # self.left_on array with a None in the middle of it. This requires
        # a work-around as designated in the code below.
        # See _validate_specification() for where this happens.

        # ugh, spaghetti re #733
        if _any(self.left_on) and _any(self.right_on):
            for lk, rk in zip(self.left_on, self.right_on):
                if is_lkey(lk):
                    left_keys.append(lk)
                    if is_rkey(rk):
                        right_keys.append(rk)
                        join_names.append(None)  # what to do?
                    else:
                        if rk is not None:
                            right_keys.append(
                                right._get_label_or_level_values(rk))
                            join_names.append(rk)
                        else:
                            # work-around for merge_asof(right_index=True)
                            right_keys.append(right.index)
                            join_names.append(right.index.name)
                else:
                    if not is_rkey(rk):
                        if rk is not None:
                            right_keys.append(
                                right._get_label_or_level_values(rk))
                        else:
                            # work-around for merge_asof(right_index=True)
                            right_keys.append(right.index)
                        if lk is not None and lk == rk:
                            # avoid key upcast in corner case (length-0)
                            if len(left) > 0:
                                right_drop.append(rk)
                            else:
                                left_drop.append(lk)
                    else:
                        right_keys.append(rk)
                    if lk is not None:
                        left_keys.append(left._get_label_or_level_values(lk))
                        join_names.append(lk)
                    else:
                        # work-around for merge_asof(left_index=True)
                        left_keys.append(left.index)
                        join_names.append(left.index.name)
        elif _any(self.left_on):
            for k in self.left_on:
                if is_lkey(k):
                    left_keys.append(k)
                    join_names.append(None)
                else:
                    left_keys.append(left._get_label_or_level_values(k))
                    join_names.append(k)
            if isinstance(self.right.index, MultiIndex):
                right_keys = [lev._values.take(lev_codes) for lev, lev_codes
                              in zip(self.right.index.levels,
                                     self.right.index.codes)]
            else:
                right_keys = [self.right.index._values]
        elif _any(self.right_on):
            for k in self.right_on:
                if is_rkey(k):
                    right_keys.append(k)
                    join_names.append(None)
                else:
                    right_keys.append(right._get_label_or_level_values(k))
                    join_names.append(k)
            if isinstance(self.left.index, MultiIndex):
                left_keys = [lev._values.take(lev_codes) for lev, lev_codes
                             in zip(self.left.index.levels,
                                    self.left.index.codes)]
            else:
                left_keys = [self.left.index.values]

        if left_drop:
            self.left = self.left._drop_labels_or_levels(left_drop)

        if right_drop:
            self.right = self.right._drop_labels_or_levels(right_drop)

        return left_keys, right_keys, join_names


Instruction:
Can you create a Python 3 function that
returns the indexers for the join keys

Response:
def _get_join_indexers(self):
        """ return the join indexers """

        def flip(xs):
            """ unlike np.transpose, this returns an array of tuples """
            labels = list(string.ascii_lowercase[:len(xs)])
            dtypes = [x.dtype for x in xs]
            labeled_dtypes = list(zip(labels, dtypes))
            return np.array(lzip(*xs), labeled_dtypes)

        # values to compare
        left_values = (self.left.index.values if self.left_index else
                       self.left_join_keys[-1])
        right_values = (self.right.index.values if self.right_index else
                        self.right_join_keys[-1])
        tolerance = self.tolerance

        # we require sortedness and non-null values in the join keys
        msg_sorted = "{side} keys must be sorted"
        msg_missings = "Merge keys contain null values on {side} side"

        if not Index(left_values).is_monotonic:
            if isnull(left_values).any():
                raise ValueError(msg_missings.format(side='left'))
            else:
                raise ValueError(msg_sorted.format(side='left'))

        if not Index(right_values).is_monotonic:
            if isnull(right_values).any():
                raise ValueError(msg_missings.format(side='right'))
            else:
                raise ValueError(msg_sorted.format(side='right'))

        # initial type conversion as needed
        if needs_i8_conversion(left_values):
            left_values = left_values.view('i8')
            right_values = right_values.view('i8')
            if tolerance is not None:
                tolerance = tolerance.value

        # a "by" parameter requires special handling
        if self.left_by is not None:
            # remove 'on' parameter from values if one existed
            if self.left_index and self.right_index:
                left_by_values = self.left_join_keys
                right_by_values = self.right_join_keys
            else:
                left_by_values = self.left_join_keys[0:-1]
                right_by_values = self.right_join_keys[0:-1]

            # get tuple representation of values if more than one
            if len(left_by_values) == 1:
                left_by_values = left_by_values[0]
                right_by_values = right_by_values[0]
            else:
                left_by_values = flip(left_by_values)
                right_by_values = flip(right_by_values)

            # upcast 'by' parameter because HashTable is limited
            by_type = _get_cython_type_upcast(left_by_values.dtype)
            by_type_caster = _type_casters[by_type]
            left_by_values = by_type_caster(left_by_values)
            right_by_values = by_type_caster(right_by_values)

            # choose appropriate function by type
            func = _asof_by_function(self.direction)
            return func(left_values,
                        right_values,
                        left_by_values,
                        right_by_values,
                        self.allow_exact_matches,
                        tolerance)
        else:
            # choose appropriate function by type
            func = _asof_function(self.direction)
            return func(left_values,
                        right_values,
                        self.allow_exact_matches,
                        tolerance)


Instruction:
Explain what the following Python 3 code does
def is_dtype(cls, dtype):
        dtype = getattr(dtype, 'dtype', dtype)

        if isinstance(dtype, (ABCSeries, ABCIndexClass,
                              ABCDataFrame, np.dtype)):
            # https://github.com/pandas-dev/pandas/issues/22960
            # avoid passing data to `construct_from_string`. This could
            # cause a FutureWarning from numpy about failing elementwise
            # comparison from, e.g., comparing DataFrame == 'category'.
            return False
        elif dtype is None:
            return False
        elif isinstance(dtype, cls):
            return True
        try:
            return cls.construct_from_string(dtype) is not None
        except TypeError:
            return False

Response: Check if we are in the specified dtype.


Instruction:
Given the following Python 3 function, write the documentation
def cat_core(list_of_columns, sep):
    list_with_sep = [sep] * (2 * len(list_of_columns) - 1)
    list_with_sep[::2] = list_of_columns
    return np.sum(list_with_sep, axis=0)

Response: A utility function for concatenating the columns of a tree with a separator string


Instruction:
Can you tell what is the following Python 3 function doing
def str_count(arr, pat, flags=0):
    regex = re.compile(pat, flags=flags)
    f = lambda x: len(regex.findall(x))
    return _na_map(f, arr, dtype=int)

Response: Count occurrences of pattern in each string of the Series or Index.


Instruction:
Can you implement a function in Python 3 that
tests if a given pattern or regex is contained within a given string.

Response:
def str_contains(arr, pat, case=True, flags=0, na=np.nan, regex=True):
    """
    Test if pattern or regex is contained within a string of a Series or Index.

    Return boolean Series or Index based on whether a given pattern or regex is
    contained within a string of a Series or Index.

    Parameters
    ----------
    pat : str
        Character sequence or regular expression.
    case : bool, default True
        If True, case sensitive.
    flags : int, default 0 (no flags)
        Flags to pass through to the re module, e.g. re.IGNORECASE.
    na : default NaN
        Fill value for missing values.
    regex : bool, default True
        If True, assumes the pat is a regular expression.

        If False, treats the pat as a literal string.

    Returns
    -------
    Series or Index of boolean values
        A Series or Index of boolean values indicating whether the
        given pattern is contained within the string of each element
        of the Series or Index.

    See Also
    --------
    match : Analogous, but stricter, relying on re.match instead of re.search.
    Series.str.startswith : Test if the start of each string element matches a
        pattern.
    Series.str.endswith : Same as startswith, but tests the end of string.

    Examples
    --------

    Returning a Series of booleans using only a literal pattern.

    >>> s1 = pd.Series(['Mouse', 'dog', 'house and parrot', '23', np.NaN])
    >>> s1.str.contains('og', regex=False)
    0    False
    1     True
    2    False
    3    False
    4      NaN
    dtype: object

    Returning an Index of booleans using only a literal pattern.

    >>> ind = pd.Index(['Mouse', 'dog', 'house and parrot', '23.0', np.NaN])
    >>> ind.str.contains('23', regex=False)
    Index([False, False, False, True, nan], dtype='object')

    Specifying case sensitivity using `case`.

    >>> s1.str.contains('oG', case=True, regex=True)
    0    False
    1    False
    2    False
    3    False
    4      NaN
    dtype: object

    Specifying `na` to be `False` instead of `NaN` replaces NaN values
    with `False`. If Series or Index does not contain NaN values
    the resultant dtype will be `bool`, otherwise, an `object` dtype.

    >>> s1.str.contains('og', na=False, regex=True)
    0    False
    1     True
    2    False
    3    False
    4    False
    dtype: bool

    Returning 'house' or 'dog' when either expression occurs in a string.

    >>> s1.str.contains('house|dog', regex=True)
    0    False
    1     True
    2     True
    3    False
    4      NaN
    dtype: object

    Ignoring case sensitivity using `flags` with regex.

    >>> import re
    >>> s1.str.contains('PARROT', flags=re.IGNORECASE, regex=True)
    0    False
    1    False
    2     True
    3    False
    4      NaN
    dtype: object

    Returning any digit using regular expression.

    >>> s1.str.contains('\\d', regex=True)
    0    False
    1    False
    2    False
    3     True
    4      NaN
    dtype: object

    Ensure `pat` is a not a literal pattern when `regex` is set to True.
    Note in the following example one might expect only `s2[1]` and `s2[3]` to
    return `True`. However, '.0' as a regex matches any character
    followed by a 0.

    >>> s2 = pd.Series(['40', '40.0', '41', '41.0', '35'])
    >>> s2.str.contains('.0', regex=True)
    0     True
    1     True
    2    False
    3     True
    4    False
    dtype: bool
    """
    if regex:
        if not case:
            flags |= re.IGNORECASE

        regex = re.compile(pat, flags=flags)

        if regex.groups > 0:
            warnings.warn("This pattern has match groups. To actually get the"
                          " groups, use str.extract.", UserWarning,
                          stacklevel=3)

        f = lambda x: bool(regex.search(x))
    else:
        if case:
            f = lambda x: pat in x
        else:
            upper_pat = pat.upper()
            f = lambda x: upper_pat in x
            uppered = _na_map(lambda x: x.upper(), arr)
            return _na_map(f, uppered, na, dtype=bool)
    return _na_map(f, arr, na, dtype=bool)


Instruction:
How would you code a function in Python 3 to
return a Series of booleans indicating whether each string element in the array starts with pat.

Response:
def str_startswith(arr, pat, na=np.nan):
    """
    Test if the start of each string element matches a pattern.

    Equivalent to :meth:`str.startswith`.

    Parameters
    ----------
    pat : str
        Character sequence. Regular expressions are not accepted.
    na : object, default NaN
        Object shown if element tested is not a string.

    Returns
    -------
    Series or Index of bool
        A Series of booleans indicating whether the given pattern matches
        the start of each string element.

    See Also
    --------
    str.startswith : Python standard library string method.
    Series.str.endswith : Same as startswith, but tests the end of string.
    Series.str.contains : Tests if string element contains a pattern.

    Examples
    --------
    >>> s = pd.Series(['bat', 'Bear', 'cat', np.nan])
    >>> s
    0     bat
    1    Bear
    2     cat
    3     NaN
    dtype: object

    >>> s.str.startswith('b')
    0     True
    1    False
    2    False
    3      NaN
    dtype: object

    Specifying `na` to be `False` instead of `NaN`.

    >>> s.str.startswith('b', na=False)
    0     True
    1    False
    2    False
    3    False
    dtype: bool
    """
    f = lambda x: x.startswith(pat)
    return _na_map(f, arr, na, dtype=bool)


Instruction:
Can you generate the documentation for the following Python 3 function
def str_endswith(arr, pat, na=np.nan):
    f = lambda x: x.endswith(pat)
    return _na_map(f, arr, na, dtype=bool)

Response: Returns a Series of booleans indicating whether the end of each string element matches pat.


Instruction:
Implement a function in Python 3 to
return a Series or Index of repeated strings in the Series or Index.

Response:
def str_repeat(arr, repeats):
    """
    Duplicate each string in the Series or Index.

    Parameters
    ----------
    repeats : int or sequence of int
        Same value for all (int) or different value per (sequence).

    Returns
    -------
    Series or Index of object
        Series or Index of repeated string objects specified by
        input parameter repeats.

    Examples
    --------
    >>> s = pd.Series(['a', 'b', 'c'])
    >>> s
    0    a
    1    b
    2    c
    dtype: object

    Single int repeats string in Series

    >>> s.str.repeat(repeats=2)
    0    aa
    1    bb
    2    cc
    dtype: object

    Sequence of int repeats corresponding string in Series

    >>> s.str.repeat(repeats=[1, 2, 3])
    0      a
    1     bb
    2    ccc
    dtype: object
    """
    if is_scalar(repeats):
        def scalar_rep(x):
            try:
                return bytes.__mul__(x, repeats)
            except TypeError:
                return str.__mul__(x, repeats)

        return _na_map(scalar_rep, arr)
    else:

        def rep(x, r):
            try:
                return bytes.__mul__(x, r)
            except TypeError:
                return str.__mul__(x, r)

        repeats = np.asarray(repeats, dtype=object)
        result = libops.vec_binop(com.values_from_object(arr), repeats, rep)
        return result


Instruction:
Write a Python 3 function that can
determine if each string matches a regular expression.

Response:
def str_match(arr, pat, case=True, flags=0, na=np.nan):
    """
    Determine if each string matches a regular expression.

    Parameters
    ----------
    pat : str
        Character sequence or regular expression.
    case : bool, default True
        If True, case sensitive.
    flags : int, default 0 (no flags)
        re module flags, e.g. re.IGNORECASE.
    na : default NaN
        Fill value for missing values.

    Returns
    -------
    Series/array of boolean values

    See Also
    --------
    contains : Analogous, but less strict, relying on re.search instead of
        re.match.
    extract : Extract matched groups.
    """
    if not case:
        flags |= re.IGNORECASE

    regex = re.compile(pat, flags=flags)

    dtype = bool
    f = lambda x: bool(regex.match(x))

    return _na_map(f, arr, na, dtype=dtype)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _groups_or_na_fun(regex):
    if regex.groups == 0:
        raise ValueError("pattern contains no capture groups")
    empty_row = [np.nan] * regex.groups

    def f(x):
        if not isinstance(x, str):
            return empty_row
        m = regex.search(x)
        if m:
            return [np.nan if item is None else item for item in m.groups()]
        else:
            return empty_row
    return f

Response: Used in both extract_noexpand and extract_frame


Instruction:
Explain what the following Python 3 code does
def _str_extract_noexpand(arr, pat, flags=0):
    from pandas import DataFrame, Index

    regex = re.compile(pat, flags=flags)
    groups_or_na = _groups_or_na_fun(regex)

    if regex.groups == 1:
        result = np.array([groups_or_na(val)[0] for val in arr], dtype=object)
        name = _get_single_group_name(regex)
    else:
        if isinstance(arr, Index):
            raise ValueError("only one regex group is supported with Index")
        name = None
        names = dict(zip(regex.groupindex.values(), regex.groupindex.keys()))
        columns = [names.get(1 + i, i) for i in range(regex.groups)]
        if arr.empty:
            result = DataFrame(columns=columns, dtype=object)
        else:
            result = DataFrame(
                [groups_or_na(val) for val in arr],
                columns=columns,
                index=arr.index,
                dtype=object)
    return result, name

Response: This function is called from str_extract and can return Series DataFrame or Index.


Instruction:
Here you have a function in Python 3, explain what it does
def _str_extract_frame(arr, pat, flags=0):
    from pandas import DataFrame

    regex = re.compile(pat, flags=flags)
    groups_or_na = _groups_or_na_fun(regex)
    names = dict(zip(regex.groupindex.values(), regex.groupindex.keys()))
    columns = [names.get(1 + i, i) for i in range(regex.groups)]

    if len(arr) == 0:
        return DataFrame(columns=columns, dtype=object)
    try:
        result_index = arr.index
    except AttributeError:
        result_index = None
    return DataFrame(
        [groups_or_na(val) for val in arr],
        columns=columns,
        index=result_index,
        dtype=object)

Response:
This function extracts groups from the
    first match of regular expression pat and returns a DataFrame.


Instruction:
Here you have a function in Python 3, explain what it does
def str_extractall(arr, pat, flags=0):

    regex = re.compile(pat, flags=flags)
    # the regex must contain capture groups.
    if regex.groups == 0:
        raise ValueError("pattern contains no capture groups")

    if isinstance(arr, ABCIndexClass):
        arr = arr.to_series().reset_index(drop=True)

    names = dict(zip(regex.groupindex.values(), regex.groupindex.keys()))
    columns = [names.get(1 + i, i) for i in range(regex.groups)]
    match_list = []
    index_list = []
    is_mi = arr.index.nlevels > 1

    for subject_key, subject in arr.iteritems():
        if isinstance(subject, str):

            if not is_mi:
                subject_key = (subject_key, )

            for match_i, match_tuple in enumerate(regex.findall(subject)):
                if isinstance(match_tuple, str):
                    match_tuple = (match_tuple,)
                na_tuple = [np.NaN if group == "" else group
                            for group in match_tuple]
                match_list.append(na_tuple)
                result_key = tuple(subject_key + (match_i, ))
                index_list.append(result_key)

    from pandas import MultiIndex
    index = MultiIndex.from_tuples(
        index_list, names=arr.index.names + ["match"])

    result = arr._constructor_expanddim(match_list, index=index,
                                        columns=columns)
    return result

Response: r Extracts groups from all the subject strings in the Series that match the regular expression pat.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def str_get_dummies(arr, sep='|'):
    arr = arr.fillna('')
    try:
        arr = sep + arr + sep
    except TypeError:
        arr = sep + arr.astype(str) + sep

    tags = set()
    for ts in arr.str.split(sep):
        tags.update(ts)
    tags = sorted(tags - {""})

    dummies = np.empty((len(arr), len(tags)), dtype=np.int64)

    for i, t in enumerate(tags):
        pat = sep + t + sep
        dummies[:, i] = lib.map_infer(arr.values, lambda x: pat in x)
    return dummies, tags

Response:
Split each string in the Series by sep and return a DataFrame
    Dummy variables corresponding to values of the Series.


Instruction:
Write a Python 3 script to
find all occurrences of pattern or regular expression in the Series/Index. Equivalent to applying :func:`re.findall` to all the elements in the Series/Index. Parameters ---------- pat : str Pattern or regular expression. flags : int, default 0 Flags from ``re`` module, e.g. `re.IGNORECASE` (default is 0, which means no flags). Returns ------- Series/Index of lists of strings All non-overlapping matches of pattern or regular expression in each string of this Series/Index. See Also -------- count : Count occurrences of pattern or regular expression in each string of the Series/Index. extractall : For each string in the Series, extract groups from all matches of regular expression and return a DataFrame with one row for each match and one column for each group. re.findall : The equivalent ``re`` function to all non-overlapping matches of pattern or regular expression in string, as a list of strings. Examples -------- >>> s = pd.Series(['Lion', 'Monkey', 'Rabbit']) The search for the pattern 'Monkey' returns one match: >>> s.str.findall('Monkey') 0 [] 1 [Monkey] 2 [] dtype: object On the other hand, the search for the pattern 'MONKEY' doesn't return any match: >>> s.str.findall('MONKEY') 0 [] 1 [] 2 [] dtype: object Flags can be added to the pattern or regular expression. For instance, to find the pattern 'MONKEY' ignoring the case: >>> import re >>> s.str.findall('MONKEY', flags=re.IGNORECASE) 0 [] 1 [Monkey] 2 [] dtype: object When the pattern matches more than one string in the Series, all matches are returned: >>> s.str.findall('on') 0 [on] 1 [on] 2 [] dtype: object Regular expressions are supported too. For instance, the search for all the strings ending with the word 'on' is shown next: >>> s.str.findall('on$') 0 [on] 1 [] 2 [] dtype: object If the pattern is found more than once in the same string, then a list of multiple strings is returned: >>> s.str.findall('b') 0 [] 1 [] 2 [b, b] dtype: object

Response:
def str_findall(arr, pat, flags=0):
    """
    Find all occurrences of pattern or regular expression in the Series/Index.

    Equivalent to applying :func:`re.findall` to all the elements in the
    Series/Index.

    Parameters
    ----------
    pat : str
        Pattern or regular expression.
    flags : int, default 0
        Flags from ``re`` module, e.g. `re.IGNORECASE` (default is 0, which
        means no flags).

    Returns
    -------
    Series/Index of lists of strings
        All non-overlapping matches of pattern or regular expression in each
        string of this Series/Index.

    See Also
    --------
    count : Count occurrences of pattern or regular expression in each string
        of the Series/Index.
    extractall : For each string in the Series, extract groups from all matches
        of regular expression and return a DataFrame with one row for each
        match and one column for each group.
    re.findall : The equivalent ``re`` function to all non-overlapping matches
        of pattern or regular expression in string, as a list of strings.

    Examples
    --------

    >>> s = pd.Series(['Lion', 'Monkey', 'Rabbit'])

    The search for the pattern 'Monkey' returns one match:

    >>> s.str.findall('Monkey')
    0          []
    1    [Monkey]
    2          []
    dtype: object

    On the other hand, the search for the pattern 'MONKEY' doesn't return any
    match:

    >>> s.str.findall('MONKEY')
    0    []
    1    []
    2    []
    dtype: object

    Flags can be added to the pattern or regular expression. For instance,
    to find the pattern 'MONKEY' ignoring the case:

    >>> import re
    >>> s.str.findall('MONKEY', flags=re.IGNORECASE)
    0          []
    1    [Monkey]
    2          []
    dtype: object

    When the pattern matches more than one string in the Series, all matches
    are returned:

    >>> s.str.findall('on')
    0    [on]
    1    [on]
    2      []
    dtype: object

    Regular expressions are supported too. For instance, the search for all the
    strings ending with the word 'on' is shown next:

    >>> s.str.findall('on$')
    0    [on]
    1      []
    2      []
    dtype: object

    If the pattern is found more than once in the same string, then a list of
    multiple strings is returned:

    >>> s.str.findall('b')
    0        []
    1        []
    2    [b, b]
    dtype: object
    """
    regex = re.compile(pat, flags=flags)
    return _na_map(regex.findall, arr)


Instruction:
Given the following Python 3 function, write the documentation
def str_find(arr, sub, start=0, end=None, side='left'):

    if not isinstance(sub, str):
        msg = 'expected a string object, not {0}'
        raise TypeError(msg.format(type(sub).__name__))

    if side == 'left':
        method = 'find'
    elif side == 'right':
        method = 'rfind'
    else:  # pragma: no cover
        raise ValueError('Invalid side')

    if end is None:
        f = lambda x: getattr(x, method)(sub, start)
    else:
        f = lambda x: getattr(x, method)(sub, start, end)

    return _na_map(f, arr, dtype=int)

Response: Find the indexes in each string in the Series or Index where the substring is fully contained between [ start end ).


Instruction:
How would you explain what the following Python 3 function does
def str_pad(arr, width, side='left', fillchar=' '):
    if not isinstance(fillchar, str):
        msg = 'fillchar must be a character, not {0}'
        raise TypeError(msg.format(type(fillchar).__name__))

    if len(fillchar) != 1:
        raise TypeError('fillchar must be a character, not str')

    if not is_integer(width):
        msg = 'width must be of integer type, not {0}'
        raise TypeError(msg.format(type(width).__name__))

    if side == 'left':
        f = lambda x: x.rjust(width, fillchar)
    elif side == 'right':
        f = lambda x: x.ljust(width, fillchar)
    elif side == 'both':
        f = lambda x: x.center(width, fillchar)
    else:  # pragma: no cover
        raise ValueError('Invalid side')

    return _na_map(f, arr)

Response: Pads the string of the object with the specified width.


Instruction:
Make a summary of the following Python 3 code
def str_slice(arr, start=None, stop=None, step=None):
    obj = slice(start, stop, step)
    f = lambda x: x[obj]
    return _na_map(f, arr)

Response: Slice substrings from each element in the Series or Index.


Instruction:
Can you implement a function in Python 3 that
replaces a positional slice of a string with another value. Parameters ---------- start : int, optional Left index position to use for the slice. If not specified (None), the slice is unbounded on the left, i.e. slice from the start of the string. stop : int, optional Right index position to use for the slice. If not specified (None), the slice is unbounded on the right, i.e. slice until the end of the string. repl : str, optional String for replacement. If not specified (None), the sliced region is replaced with an empty string. Returns ------- Series or Index Same type as the original object. See Also -------- Series.str.slice : Just slicing without replacement. Examples -------- >>> s = pd.Series(['a', 'ab', 'abc', 'abdc', 'abcde']) >>> s 0 a 1 ab 2 abc 3 abdc 4 abcde dtype: object Specify just `start`, meaning replace `start` until the end of the string with `repl`. >>> s.str.slice_replace(1, repl='X') 0 aX 1 aX 2 aX 3 aX 4 aX dtype: object Specify just `stop`, meaning the start of the string to `stop` is replaced with `repl`, and the rest of the string is included. >>> s.str.slice_replace(stop=2, repl='X') 0 X 1 X 2 Xc 3 Xdc 4 Xcde dtype: object Specify `start` and `stop`, meaning the slice from `start` to `stop` is replaced with `repl`. Everything before or after `start` and `stop` is included as is. >>> s.str.slice_replace(start=1, stop=3, repl='X') 0 aX 1 aX 2 aX 3 aXc 4 aXde dtype: object

Response:
def str_slice_replace(arr, start=None, stop=None, repl=None):
    """
    Replace a positional slice of a string with another value.

    Parameters
    ----------
    start : int, optional
        Left index position to use for the slice. If not specified (None),
        the slice is unbounded on the left, i.e. slice from the start
        of the string.
    stop : int, optional
        Right index position to use for the slice. If not specified (None),
        the slice is unbounded on the right, i.e. slice until the
        end of the string.
    repl : str, optional
        String for replacement. If not specified (None), the sliced region
        is replaced with an empty string.

    Returns
    -------
    Series or Index
        Same type as the original object.

    See Also
    --------
    Series.str.slice : Just slicing without replacement.

    Examples
    --------
    >>> s = pd.Series(['a', 'ab', 'abc', 'abdc', 'abcde'])
    >>> s
    0        a
    1       ab
    2      abc
    3     abdc
    4    abcde
    dtype: object

    Specify just `start`, meaning replace `start` until the end of the
    string with `repl`.

    >>> s.str.slice_replace(1, repl='X')
    0    aX
    1    aX
    2    aX
    3    aX
    4    aX
    dtype: object

    Specify just `stop`, meaning the start of the string to `stop` is replaced
    with `repl`, and the rest of the string is included.

    >>> s.str.slice_replace(stop=2, repl='X')
    0       X
    1       X
    2      Xc
    3     Xdc
    4    Xcde
    dtype: object

    Specify `start` and `stop`, meaning the slice from `start` to `stop` is
    replaced with `repl`. Everything before or after `start` and `stop` is
    included as is.

    >>> s.str.slice_replace(start=1, stop=3, repl='X')
    0      aX
    1      aX
    2      aX
    3     aXc
    4    aXde
    dtype: object
    """
    if repl is None:
        repl = ''

    def f(x):
        if x[start:stop] == '':
            local_stop = start
        else:
            local_stop = stop
        y = ''
        if start is not None:
            y += x[:start]
        y += repl
        if stop is not None:
            y += x[local_stop:]
        return y

    return _na_map(f, arr)


Instruction:
Can you implement a function in Python 3 that
strips whitespace (including newlines) from each string in the Series/Index. Parameters ---------- to_strip : str or unicode side : {'left', 'right', 'both'}, default 'both' Returns ------- Series or Index

Response:
def str_strip(arr, to_strip=None, side='both'):
    """
    Strip whitespace (including newlines) from each string in the
    Series/Index.

    Parameters
    ----------
    to_strip : str or unicode
    side : {'left', 'right', 'both'}, default 'both'

    Returns
    -------
    Series or Index
    """
    if side == 'both':
        f = lambda x: x.strip(to_strip)
    elif side == 'left':
        f = lambda x: x.lstrip(to_strip)
    elif side == 'right':
        f = lambda x: x.rstrip(to_strip)
    else:  # pragma: no cover
        raise ValueError('Invalid side')
    return _na_map(f, arr)


Instruction:
Given the following Python 3 function, write the documentation
def str_wrap(arr, width, **kwargs):
    kwargs['width'] = width

    tw = textwrap.TextWrapper(**kwargs)

    return _na_map(lambda s: '\n'.join(tw.wrap(s)), arr)

Response: r Wrap long strings in a Series or Index.


Instruction:
Can you create a Python 3 function that
returns the element from each component in the sequence at specified position.

Response:
def str_get(arr, i):
    """
    Extract element from each component at specified position.

    Extract element from lists, tuples, or strings in each element in the
    Series/Index.

    Parameters
    ----------
    i : int
        Position of element to extract.

    Returns
    -------
    Series or Index

    Examples
    --------
    >>> s = pd.Series(["String",
    ...               (1, 2, 3),
    ...               ["a", "b", "c"],
    ...               123,
    ...               -456,
    ...               {1: "Hello", "2": "World"}])
    >>> s
    0                        String
    1                     (1, 2, 3)
    2                     [a, b, c]
    3                           123
    4                          -456
    5    {1: 'Hello', '2': 'World'}
    dtype: object

    >>> s.str.get(1)
    0        t
    1        2
    2        b
    3      NaN
    4      NaN
    5    Hello
    dtype: object

    >>> s.str.get(-1)
    0      g
    1      3
    2      c
    3    NaN
    4    NaN
    5    None
    dtype: object
    """
    def f(x):
        if isinstance(x, dict):
            return x.get(i)
        elif len(x) > i >= -len(x):
            return x[i]
        return np.nan
    return _na_map(f, arr)


Instruction:
Make a summary of the following Python 3 code
def str_decode(arr, encoding, errors="strict"):
    if encoding in _cpython_optimized_decoders:
        # CPython optimized implementation
        f = lambda x: x.decode(encoding, errors)
    else:
        decoder = codecs.getdecoder(encoding)
        f = lambda x: decoder(x, errors)[0]
    return _na_map(f, arr)

Response: Decode character string in the Series or Index using specified encoding.


Instruction:
Can you generate the documentation for the following Python 3 function
def str_encode(arr, encoding, errors="strict"):
    if encoding in _cpython_optimized_encoders:
        # CPython optimized implementation
        f = lambda x: x.encode(encoding, errors)
    else:
        encoder = codecs.getencoder(encoding)
        f = lambda x: encoder(x, errors)[0]
    return _na_map(f, arr)

Response: Encode character string in the Series / Index using specified encoding.


Instruction:
Explain what the following Python 3 code does
def copy(source):
    "Copy a docstring from another source function (if present)"

    def do_copy(target):
        if source.__doc__:
            target.__doc__ = source.__doc__
        return target

    return do_copy

Response: Copy a docstring from another source function ( if present )


Instruction:
Can you generate the documentation for the following Python 3 function
def _get_series_list(self, others, ignore_index=False):

        # Once str.cat defaults to alignment, this function can be simplified;
        # will not need `ignore_index` and the second boolean output anymore

        from pandas import Index, Series, DataFrame

        # self._orig is either Series or Index
        idx = self._orig if isinstance(self._orig, Index) else self._orig.index

        err_msg = ('others must be Series, Index, DataFrame, np.ndarrary or '
                   'list-like (either containing only strings or containing '
                   'only objects of type Series/Index/list-like/np.ndarray)')

        # Generally speaking, all objects without an index inherit the index
        # `idx` of the calling Series/Index - i.e. must have matching length.
        # Objects with an index (i.e. Series/Index/DataFrame) keep their own
        # index, *unless* ignore_index is set to True.
        if isinstance(others, Series):
            warn = not others.index.equals(idx)
            # only reconstruct Series when absolutely necessary
            los = [Series(others.values, index=idx)
                   if ignore_index and warn else others]
            return (los, warn)
        elif isinstance(others, Index):
            warn = not others.equals(idx)
            los = [Series(others.values,
                          index=(idx if ignore_index else others))]
            return (los, warn)
        elif isinstance(others, DataFrame):
            warn = not others.index.equals(idx)
            if ignore_index and warn:
                # without copy, this could change "others"
                # that was passed to str.cat
                others = others.copy()
                others.index = idx
            return ([others[x] for x in others], warn)
        elif isinstance(others, np.ndarray) and others.ndim == 2:
            others = DataFrame(others, index=idx)
            return ([others[x] for x in others], False)
        elif is_list_like(others, allow_sets=False):
            others = list(others)  # ensure iterators do not get read twice etc

            # in case of list-like `others`, all elements must be
            # either one-dimensional list-likes or scalars
            if all(is_list_like(x, allow_sets=False) for x in others):
                los = []
                join_warn = False
                depr_warn = False
                # iterate through list and append list of series for each
                # element (which we check to be one-dimensional and non-nested)
                while others:
                    nxt = others.pop(0)  # nxt is guaranteed list-like by above

                    # GH 21950 - DeprecationWarning
                    # only allowing Series/Index/np.ndarray[1-dim] will greatly
                    # simply this function post-deprecation.
                    if not (isinstance(nxt, (Series, Index)) or
                            (isinstance(nxt, np.ndarray) and nxt.ndim == 1)):
                        depr_warn = True

                    if not isinstance(nxt, (DataFrame, Series,
                                            Index, np.ndarray)):
                        # safety for non-persistent list-likes (e.g. iterators)
                        # do not map indexed/typed objects; info needed below
                        nxt = list(nxt)

                    # known types for which we can avoid deep inspection
                    no_deep = ((isinstance(nxt, np.ndarray) and nxt.ndim == 1)
                               or isinstance(nxt, (Series, Index)))
                    # nested list-likes are forbidden:
                    # -> elements of nxt must not be list-like
                    is_legal = ((no_deep and nxt.dtype == object)
                                or all(not is_list_like(x) for x in nxt))

                    # DataFrame is false positive of is_legal
                    # because "x in df" returns column names
                    if not is_legal or isinstance(nxt, DataFrame):
                        raise TypeError(err_msg)

                    nxt, wnx = self._get_series_list(nxt,
                                                     ignore_index=ignore_index)
                    los = los + nxt
                    join_warn = join_warn or wnx

                if depr_warn:
                    warnings.warn('list-likes other than Series, Index, or '
                                  'np.ndarray WITHIN another list-like are '
                                  'deprecated and will be removed in a future '
                                  'version.', FutureWarning, stacklevel=3)
                return (los, join_warn)
            elif all(not is_list_like(x) for x in others):
                return ([Series(others, index=idx)], False)
        raise TypeError(err_msg)

Response: A method to convert a list of Series or Index objects into a list of Series.


Instruction:
Make a summary of the following Python 3 code
def cat(self, others=None, sep=None, na_rep=None, join=None):
        from pandas import Index, Series, concat

        if isinstance(others, str):
            raise ValueError("Did you mean to supply a `sep` keyword?")
        if sep is None:
            sep = ''

        if isinstance(self._orig, Index):
            data = Series(self._orig, index=self._orig)
        else:  # Series
            data = self._orig

        # concatenate Series/Index with itself if no "others"
        if others is None:
            data = ensure_object(data)
            na_mask = isna(data)
            if na_rep is None and na_mask.any():
                data = data[~na_mask]
            elif na_rep is not None and na_mask.any():
                data = np.where(na_mask, na_rep, data)
            return sep.join(data)

        try:
            # turn anything in "others" into lists of Series
            others, warn = self._get_series_list(others,
                                                 ignore_index=(join is None))
        except ValueError:  # do not catch TypeError raised by _get_series_list
            if join is None:
                raise ValueError('All arrays must be same length, except '
                                 'those having an index if `join` is not None')
            else:
                raise ValueError('If `others` contains arrays or lists (or '
                                 'other list-likes without an index), these '
                                 'must all be of the same length as the '
                                 'calling Series/Index.')

        if join is None and warn:
            warnings.warn("A future version of pandas will perform index "
                          "alignment when `others` is a Series/Index/"
                          "DataFrame (or a list-like containing one). To "
                          "disable alignment (the behavior before v.0.23) and "
                          "silence this warning, use `.values` on any Series/"
                          "Index/DataFrame in `others`. To enable alignment "
                          "and silence this warning, pass `join='left'|"
                          "'outer'|'inner'|'right'`. The future default will "
                          "be `join='left'`.", FutureWarning, stacklevel=2)

        # if join is None, _get_series_list already force-aligned indexes
        join = 'left' if join is None else join

        # align if required
        if any(not data.index.equals(x.index) for x in others):
            # Need to add keys for uniqueness in case of duplicate columns
            others = concat(others, axis=1,
                            join=(join if join == 'inner' else 'outer'),
                            keys=range(len(others)), sort=False, copy=False)
            data, others = data.align(others, join=join)
            others = [others[x] for x in others]  # again list of Series

        all_cols = [ensure_object(x) for x in [data] + others]
        na_masks = np.array([isna(x) for x in all_cols])
        union_mask = np.logical_or.reduce(na_masks, axis=0)

        if na_rep is None and union_mask.any():
            # no na_rep means NaNs for all rows where any column has a NaN
            # only necessary if there are actually any NaNs
            result = np.empty(len(data), dtype=object)
            np.putmask(result, union_mask, np.nan)

            not_masked = ~union_mask
            result[not_masked] = cat_core([x[not_masked] for x in all_cols],
                                          sep)
        elif na_rep is not None and union_mask.any():
            # fill NaNs with na_rep in case there are actually any NaNs
            all_cols = [np.where(nm, na_rep, col)
                        for nm, col in zip(na_masks, all_cols)]
            result = cat_core(all_cols, sep)
        else:
            # no NaNs - can just concatenate
            result = cat_core(all_cols, sep)

        if isinstance(self._orig, Index):
            # add dtype for case that result is all-NA
            result = Index(result, dtype=object, name=self._orig.name)
        else:  # Series
            result = Series(result, dtype=object, index=data.index,
                            name=self._orig.name)
        return result

Response: Returns a string that is the concatenation of the strings in the Series or Index of the others.


Instruction:
Write a Python 3 function that can
return a new string with zeros at the left and length of the string.

Response:
def zfill(self, width):
        """
        Pad strings in the Series/Index by prepending '0' characters.

        Strings in the Series/Index are padded with '0' characters on the
        left of the string to reach a total string length  `width`. Strings
        in the Series/Index with length greater or equal to `width` are
        unchanged.

        Parameters
        ----------
        width : int
            Minimum length of resulting string; strings with length less
            than `width` be prepended with '0' characters.

        Returns
        -------
        Series/Index of objects

        See Also
        --------
        Series.str.rjust : Fills the left side of strings with an arbitrary
            character.
        Series.str.ljust : Fills the right side of strings with an arbitrary
            character.
        Series.str.pad : Fills the specified sides of strings with an arbitrary
            character.
        Series.str.center : Fills boths sides of strings with an arbitrary
            character.

        Notes
        -----
        Differs from :meth:`str.zfill` which has special handling
        for '+'/'-' in the string.

        Examples
        --------
        >>> s = pd.Series(['-1', '1', '1000', 10, np.nan])
        >>> s
        0      -1
        1       1
        2    1000
        3      10
        4     NaN
        dtype: object

        Note that ``10`` and ``NaN`` are not strings, therefore they are
        converted to ``NaN``. The minus sign in ``'-1'`` is treated as a
        regular character and the zero is added to the left of it
        (:meth:`str.zfill` would have moved it to the left). ``1000``
        remains unchanged as it is longer than `width`.

        >>> s.str.zfill(3)
        0     0-1
        1     001
        2    1000
        3     NaN
        4     NaN
        dtype: object
        """
        result = str_pad(self._parent, width, side='left', fillchar='0')
        return self._wrap_result(result)


Instruction:
Create a Python 3 function to
return the Unicode normal form for the strings in the Series / Index.

Response:
def normalize(self, form):
        """
        Return the Unicode normal form for the strings in the Series/Index.
        For more information on the forms, see the
        :func:`unicodedata.normalize`.

        Parameters
        ----------
        form : {'NFC', 'NFKC', 'NFD', 'NFKD'}
            Unicode form

        Returns
        -------
        normalized : Series/Index of objects
        """
        import unicodedata
        f = lambda x: unicodedata.normalize(form, x)
        result = _na_map(f, self._parent)
        return self._wrap_result(result)


Instruction:
Can you generate the documentation for the following Python 3 function
def get_sys_info():
    "Returns system information as a dict"

    blob = []

    # get full commit hash
    commit = None
    if os.path.isdir(".git") and os.path.isdir("pandas"):
        try:
            pipe = subprocess.Popen('git log --format="%H" -n 1'.split(" "),
                                    stdout=subprocess.PIPE,
                                    stderr=subprocess.PIPE)
            so, serr = pipe.communicate()
        except (OSError, ValueError):
            pass
        else:
            if pipe.returncode == 0:
                commit = so
                try:
                    commit = so.decode('utf-8')
                except ValueError:
                    pass
                commit = commit.strip().strip('"')

    blob.append(('commit', commit))

    try:
        (sysname, nodename, release,
         version, machine, processor) = platform.uname()
        blob.extend([
            ("python", '.'.join(map(str, sys.version_info))),
            ("python-bits", struct.calcsize("P") * 8),
            ("OS", "{sysname}".format(sysname=sysname)),
            ("OS-release", "{release}".format(release=release)),
            # ("Version", "{version}".format(version=version)),
            ("machine", "{machine}".format(machine=machine)),
            ("processor", "{processor}".format(processor=processor)),
            ("byteorder", "{byteorder}".format(byteorder=sys.byteorder)),
            ("LC_ALL", "{lc}".format(lc=os.environ.get('LC_ALL', "None"))),
            ("LANG", "{lang}".format(lang=os.environ.get('LANG', "None"))),
            ("LOCALE", '.'.join(map(str, locale.getlocale()))),
        ])
    except (KeyError, ValueError):
        pass

    return blob

Response: Returns system information as a dict


Instruction:
How would you explain what the following Python 3 function does
def whitelist_method_generator(base, klass, whitelist):

    method_wrapper_template = \
        """def %(name)s(%(sig)s) :
    \"""
    %(doc)s
    \"""
    f = %(self)s.__getattr__('%(name)s')
    return f(%(args)s)"""
    property_wrapper_template = \
        """@property
def %(name)s(self) :
    \"""%(doc)s\"""
    return self.__getattr__('%(name)s')"""

    for name in whitelist:
        # don't override anything that was explicitly defined
        # in the base class
        if hasattr(base, name):
            continue
        # ugly, but we need the name string itself in the method.
        f = getattr(klass, name)
        doc = f.__doc__
        doc = doc if type(doc) == str else ''
        if isinstance(f, types.MethodType):
            wrapper_template = method_wrapper_template
            decl, args = make_signature(f)
            # pass args by name to f because otherwise
            # GroupBy._make_wrapper won't know whether
            # we passed in an axis parameter.
            args_by_name = ['{0}={0}'.format(arg) for arg in args[1:]]
            params = {'name': name,
                      'doc': doc,
                      'sig': ','.join(decl),
                      'self': args[0],
                      'args': ','.join(args_by_name)}
        else:
            wrapper_template = property_wrapper_template
            params = {'name': name, 'doc': doc}
        yield wrapper_template % params

Response: Generates a generator that yields all GroupBy member defs for DataFrame or Series names in whitelist.


Instruction:
How would you code a function in Python 3 to
return a sliced object.

Response:
def _gotitem(self, key, ndim, subset=None):
        """
        Sub-classes to define. Return a sliced object.

        Parameters
        ----------
        key : string / list of selections
        ndim : 1,2
            requested ndim of result
        subset : object, default None
            subset to act on
        """
        # create a new object to prevent aliasing
        if subset is None:
            subset = self.obj

        # we need to make a shallow copy of ourselves
        # with the same groupby
        kwargs = {attr: getattr(self, attr) for attr in self._attributes}

        # Try to select from a DataFrame, falling back to a Series
        try:
            groupby = self._groupby[key]
        except IndexError:
            groupby = self._groupby

        self = self.__class__(subset,
                              groupby=groupby,
                              parent=self,
                              **kwargs)
        self._reset_cache()
        if subset.ndim == 2:
            if is_scalar(key) and key in subset or is_list_like(key):
                self._selection = key
        return self


Instruction:
Given the following Python 3 function, write the documentation
def to_str(s):
    if isinstance(s, bytes):
        s = s.decode('utf-8')
    elif not isinstance(s, str):
        s = str(s)
    return s

Response: Convert bytes and non - string into Python 3 str


Instruction:
Write a Python 3 script for
binding the name and qualname attributes of the function f to the class name and class name.

Response:
def set_function_name(f, name, cls):
    """
    Bind the name/qualname attributes of the function
    """
    f.__name__ = name
    f.__qualname__ = '{klass}.{name}'.format(
        klass=cls.__name__,
        name=name)
    f.__module__ = cls.__module__
    return f


Instruction:
Can you implement a function in Python 3 that
raises exception with existing traceback.

Response:
def raise_with_traceback(exc, traceback=Ellipsis):
    """
    Raise exception with existing traceback.
    If traceback is not passed, uses sys.exc_info() to get traceback.
    """
    if traceback == Ellipsis:
        _, _, traceback = sys.exc_info()
    raise exc.with_traceback(traceback)


Instruction:
Create a Python 3 function for
converting a style dictionary to an openpyxl style object

Response:
def _convert_to_style(cls, style_dict):
        """
        converts a style_dict to an openpyxl style object
        Parameters
        ----------
        style_dict : style dictionary to convert
        """

        from openpyxl.style import Style
        xls_style = Style()
        for key, value in style_dict.items():
            for nk, nv in value.items():
                if key == "borders":
                    (xls_style.borders.__getattribute__(nk)
                     .__setattr__('border_style', nv))
                else:
                    xls_style.__getattribute__(key).__setattr__(nk, nv)

        return xls_style


Instruction:
Can you implement a function in Python 3 that
converts a style_dict to a set of kwargs suitable for initializing an openpyxl v2 style object.

Response:
def _convert_to_style_kwargs(cls, style_dict):
        """
        Convert a style_dict to a set of kwargs suitable for initializing
        or updating-on-copy an openpyxl v2 style object
        Parameters
        ----------
        style_dict : dict
            A dict with zero or more of the following keys (or their synonyms).
                'font'
                'fill'
                'border' ('borders')
                'alignment'
                'number_format'
                'protection'
        Returns
        -------
        style_kwargs : dict
            A dict with the same, normalized keys as ``style_dict`` but each
            value has been replaced with a native openpyxl style object of the
            appropriate class.
        """

        _style_key_map = {
            'borders': 'border',
        }

        style_kwargs = {}
        for k, v in style_dict.items():
            if k in _style_key_map:
                k = _style_key_map[k]
            _conv_to_x = getattr(cls, '_convert_to_{k}'.format(k=k),
                                 lambda x: None)
            new_v = _conv_to_x(v)
            if new_v:
                style_kwargs[k] = new_v

        return style_kwargs


Instruction:
Can you tell what is the following Python 3 function doing
def _convert_to_color(cls, color_spec):

        from openpyxl.styles import Color

        if isinstance(color_spec, str):
            return Color(color_spec)
        else:
            return Color(**color_spec)

Response: Convert color_spec to an openpyxl v2 Color object.


Instruction:
Can you write a function in Python 3 where it
converts font_dict to an openpyxl v2 Font object.

Response:
def _convert_to_font(cls, font_dict):
        """
        Convert ``font_dict`` to an openpyxl v2 Font object
        Parameters
        ----------
        font_dict : dict
            A dict with zero or more of the following keys (or their synonyms).
                'name'
                'size' ('sz')
                'bold' ('b')
                'italic' ('i')
                'underline' ('u')
                'strikethrough' ('strike')
                'color'
                'vertAlign' ('vertalign')
                'charset'
                'scheme'
                'family'
                'outline'
                'shadow'
                'condense'
        Returns
        -------
        font : openpyxl.styles.Font
        """

        from openpyxl.styles import Font

        _font_key_map = {
            'sz': 'size',
            'b': 'bold',
            'i': 'italic',
            'u': 'underline',
            'strike': 'strikethrough',
            'vertalign': 'vertAlign',
        }

        font_kwargs = {}
        for k, v in font_dict.items():
            if k in _font_key_map:
                k = _font_key_map[k]
            if k == 'color':
                v = cls._convert_to_color(v)
            font_kwargs[k] = v

        return Font(**font_kwargs)


Instruction:
Make a summary of the following Python 3 code
def _convert_to_fill(cls, fill_dict):

        from openpyxl.styles import PatternFill, GradientFill

        _pattern_fill_key_map = {
            'patternType': 'fill_type',
            'patterntype': 'fill_type',
            'fgColor': 'start_color',
            'fgcolor': 'start_color',
            'bgColor': 'end_color',
            'bgcolor': 'end_color',
        }

        _gradient_fill_key_map = {
            'fill_type': 'type',
        }

        pfill_kwargs = {}
        gfill_kwargs = {}
        for k, v in fill_dict.items():
            pk = gk = None
            if k in _pattern_fill_key_map:
                pk = _pattern_fill_key_map[k]
            if k in _gradient_fill_key_map:
                gk = _gradient_fill_key_map[k]
            if pk in ['start_color', 'end_color']:
                v = cls._convert_to_color(v)
            if gk == 'stop':
                v = cls._convert_to_stop(v)
            if pk:
                pfill_kwargs[pk] = v
            elif gk:
                gfill_kwargs[gk] = v
            else:
                pfill_kwargs[k] = v
                gfill_kwargs[k] = v

        try:
            return PatternFill(**pfill_kwargs)
        except TypeError:
            return GradientFill(**gfill_kwargs)

Response: Convert a dict of fill attributes to an openpyxl v2 Fill object.


Instruction:
Write a Python 3 function for
converting a dict of keys to an openpyxl v2 Side object.

Response:
def _convert_to_side(cls, side_spec):
        """
        Convert ``side_spec`` to an openpyxl v2 Side object
        Parameters
        ----------
        side_spec : str, dict
            A string specifying the border style, or a dict with zero or more
            of the following keys (or their synonyms).
                'style' ('border_style')
                'color'
        Returns
        -------
        side : openpyxl.styles.Side
        """

        from openpyxl.styles import Side

        _side_key_map = {
            'border_style': 'style',
        }

        if isinstance(side_spec, str):
            return Side(style=side_spec)

        side_kwargs = {}
        for k, v in side_spec.items():
            if k in _side_key_map:
                k = _side_key_map[k]
            if k == 'color':
                v = cls._convert_to_color(v)
            side_kwargs[k] = v

        return Side(**side_kwargs)


Instruction:
Make a summary of the following Python 3 code
def _convert_to_border(cls, border_dict):

        from openpyxl.styles import Border

        _border_key_map = {
            'diagonalup': 'diagonalUp',
            'diagonaldown': 'diagonalDown',
        }

        border_kwargs = {}
        for k, v in border_dict.items():
            if k in _border_key_map:
                k = _border_key_map[k]
            if k == 'color':
                v = cls._convert_to_color(v)
            if k in ['left', 'right', 'top', 'bottom', 'diagonal']:
                v = cls._convert_to_side(v)
            border_kwargs[k] = v

        return Border(**border_kwargs)

Response: Convert dict to openpyxl v2 Border object


Instruction:
Can you write a function in Python 3 where it
constructs and return a row or column based frame apply object

Response:
def frame_apply(obj, func, axis=0, broadcast=None,
                raw=False, reduce=None, result_type=None,
                ignore_failures=False,
                args=None, kwds=None):
    """ construct and return a row or column based frame apply object """

    axis = obj._get_axis_number(axis)
    if axis == 0:
        klass = FrameRowApply
    elif axis == 1:
        klass = FrameColumnApply

    return klass(obj, func, broadcast=broadcast,
                 raw=raw, reduce=reduce, result_type=result_type,
                 ignore_failures=ignore_failures,
                 args=args, kwds=kwds)


Instruction:
Write a Python 3 function for
computing the result of the function

Response:
def get_result(self):
        """ compute the results """

        # dispatch to agg
        if is_list_like(self.f) or is_dict_like(self.f):
            return self.obj.aggregate(self.f, axis=self.axis,
                                      *self.args, **self.kwds)

        # all empty
        if len(self.columns) == 0 and len(self.index) == 0:
            return self.apply_empty_result()

        # string dispatch
        if isinstance(self.f, str):
            # Support for `frame.transform('method')`
            # Some methods (shift, etc.) require the axis argument, others
            # don't, so inspect and insert if necessary.
            func = getattr(self.obj, self.f)
            sig = inspect.getfullargspec(func)
            if 'axis' in sig.args:
                self.kwds['axis'] = self.axis
            return func(*self.args, **self.kwds)

        # ufunc
        elif isinstance(self.f, np.ufunc):
            with np.errstate(all='ignore'):
                results = self.obj._data.apply('apply', func=self.f)
            return self.obj._constructor(data=results, index=self.index,
                                         columns=self.columns, copy=False)

        # broadcasting
        if self.result_type == 'broadcast':
            return self.apply_broadcast()

        # one axis empty
        elif not all(self.obj.shape):
            return self.apply_empty_result()

        # raw
        elif self.raw and not self.obj._is_mixed_type:
            return self.apply_raw()

        return self.apply_standard()


Instruction:
Here you have a function in Python 3, explain what it does
def apply_raw(self):

        try:
            result = reduction.reduce(self.values, self.f, axis=self.axis)
        except Exception:
            result = np.apply_along_axis(self.f, self.axis, self.values)

        # TODO: mixed type case
        if result.ndim == 2:
            return self.obj._constructor(result,
                                         index=self.index,
                                         columns=self.columns)
        else:
            return self.obj._constructor_sliced(result,
                                                index=self.agg_axis)

Response: apply to the values as a numpy array


Instruction:
Write a Python 3 script for
wrapping the results for the rows

Response:
def wrap_results_for_axis(self):
        """ return the results for the rows """

        results = self.results
        result = self.obj._constructor(data=results)

        if not isinstance(results[0], ABCSeries):
            try:
                result.index = self.res_columns
            except ValueError:
                pass

        try:
            result.columns = self.res_index
        except ValueError:
            pass

        return result


Instruction:
Can you implement a function in Python 3 that
wraps the results for the columns

Response:
def wrap_results_for_axis(self):
        """ return the results for the columns """
        results = self.results

        # we have requested to expand
        if self.result_type == 'expand':
            result = self.infer_to_same_shape()

        # we have a non-series and don't want inference
        elif not isinstance(results[0], ABCSeries):
            from pandas import Series
            result = Series(results)
            result.index = self.res_index

        # we may want to infer results
        else:
            result = self.infer_to_same_shape()

        return result


Instruction:
Write a Python 3 function that can
infer the results to the same shape as the input object

Response:
def infer_to_same_shape(self):
        """ infer the results to the same shape as the input object """
        results = self.results

        result = self.obj._constructor(data=results)
        result = result.T

        # set the index
        result.index = self.res_index

        # infer dtypes
        result = result.infer_objects()

        return result


Instruction:
Make a summary of the following Python 3 code
def cartesian_product(X):
    msg = "Input must be a list-like of list-likes"
    if not is_list_like(X):
        raise TypeError(msg)
    for x in X:
        if not is_list_like(x):
            raise TypeError(msg)

    if len(X) == 0:
        return []

    lenX = np.fromiter((len(x) for x in X), dtype=np.intp)
    cumprodX = np.cumproduct(lenX)

    a = np.roll(cumprodX, 1)
    a[0] = 1

    if cumprodX[-1] != 0:
        b = cumprodX[-1] / cumprodX
    else:
        # if any factor is empty, the cartesian product is empty
        b = np.zeros_like(cumprodX)

    return [np.tile(np.repeat(np.asarray(com.values_from_object(x)), b[i]),
                    np.product(a[i]))
            for i, x in enumerate(X)]

Response: Returns the cartesian product of the input iterables.


Instruction:
Explain what the following Python 3 code does
def _strip_schema(url):
    result = parse_url(url, allow_fragments=False)
    return result.netloc + result.path

Response: Returns the url without the s3:// part


Instruction:
Can you tell what is the following Python 3 function doing
def xception(c, k=8, n_middle=8):
    "Preview version of Xception network. Not tested yet - use at own risk. No pretrained model yet."
    layers = [
        conv(3, k*4, 3, 2),
        conv(k*4, k*8, 3),
        ConvSkip(k*8, k*16, act=False),
        ConvSkip(k*16, k*32),
        ConvSkip(k*32, k*91),
    ]
    for i in range(n_middle): layers.append(middle_flow(k*91))
    layers += [
        ConvSkip(k*91,k*128),
        sep_conv(k*128,k*192,act=False),
        sep_conv(k*192,k*256),
        nn.ReLU(),
        nn.AdaptiveAvgPool2d(1),
        Flatten(),
        nn.Linear(k*256,c)
    ]
    return nn.Sequential(*layers)

Response: Preview version of Xception network. Not tested yet - use at own risk. No pretrained model yet.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_model(self, opt_fn, emb_sz, n_hid, n_layers, **kwargs):
        m = get_language_model(self.nt, emb_sz, n_hid, n_layers, self.pad_idx, **kwargs)
        model = SingleModel(to_gpu(m))
        return RNN_Learner(self, model, opt_fn=opt_fn)

Response: Method returns an instance of the RNN_Learner class that wraps an instance of the RNN_Encoder module.


Instruction:
Make a summary of the following Python 3 code
def get_files(path:PathOrStr, extensions:Collection[str]=None, recurse:bool=False,
              include:Optional[Collection[str]]=None)->FilePathList:
    "Return list of files in `path` that have a suffix in `extensions`; optionally `recurse`."
    if recurse:
        res = []
        for i,(p,d,f) in enumerate(os.walk(path)):
            # skip hidden dirs
            if include is not None and i==0:  d[:] = [o for o in d if o in include]
            else:                             d[:] = [o for o in d if not o.startswith('.')]
            res += _get_files(path, p, f, extensions)
        return res
    else:
        f = [o.name for o in os.scandir(path) if o.is_file()]
        return _get_files(path, path, f, extensions)

Response: Return list of files in path that have a suffix in extensions ; optionally recurse.


Instruction:
How would you explain what the following Python 3 function does
def _databunch_load_empty(cls, path, fname:str='export.pkl'):
    "Load an empty `DataBunch` from the exported file in `path/fname` with optional `tfms`."
    sd = LabelLists.load_empty(path, fn=fname)
    return sd.databunch()

Response: Load an empty DataBunch from the exported file in path / fname with optional tfms.


Instruction:
Write a Python 3 script for
applying processor or self. processor to self.

Response:
def process(self, processor:PreProcessors=None):
        "Apply `processor` or `self.processor` to `self`."
        if processor is not None: self.processor = processor
        self.processor = listify(self.processor)
        for p in self.processor: p.process(self)
        return self


Instruction:
Write a Python 3 function for
applying processor or self. processor to item.

Response:
def process_one(self, item:ItemBase, processor:PreProcessors=None):
        "Apply `processor` or `self.processor` to `item`."
        if processor is not None: self.processor = processor
        self.processor = listify(self.processor)
        for p in self.processor: item = p.process_one(item)
        return item


Instruction:
Here you have a function in Python 3, explain what it does
def reconstruct(self, t:Tensor, x:Tensor=None):
        "Reconstruct one of the underlying item for its data `t`."
        return self[0].reconstruct(t,x) if has_arg(self[0].reconstruct, 'x') else self[0].reconstruct(t)

Response: Reconstruct one of the underlying item for its data t.


Instruction:
Explain what the following Python 3 code does
def new(self, items:Iterator, processor:PreProcessors=None, **kwargs)->'ItemList':
        "Create a new `ItemList` from `items`, keeping the same attributes."
        processor = ifnone(processor, self.processor)
        copy_d = {o:getattr(self,o) for o in self.copy_new}
        kwargs = {**copy_d, **kwargs}
        return self.__class__(items=items, processor=processor, **kwargs)

Response: Create a new ItemList from items keeping the same attributes.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def from_folder(cls, path:PathOrStr, extensions:Collection[str]=None, recurse:bool=True,
                    include:Optional[Collection[str]]=None, processor:PreProcessors=None, **kwargs)->'ItemList':
        """Create an `ItemList` in `path` from the filenames that have a suffix in `extensions`.
        `recurse` determines if we search subfolders."""
        path = Path(path)
        return cls(get_files(path, extensions, recurse=recurse, include=include), path=path, processor=processor, **kwargs)

Response: Create an ItemList in path from the files in path that have a suffix in extensions.


Instruction:
Can you create a Python 3 function that
creates an ItemList in path from the inputs in the cols of df.

Response:
def from_df(cls, df:DataFrame, path:PathOrStr='.', cols:IntsOrStrs=0, processor:PreProcessors=None, **kwargs)->'ItemList':
        "Create an `ItemList` in `path` from the inputs in the `cols` of `df`."
        inputs = df.iloc[:,df_names_to_idx(cols, df)]
        assert inputs.isna().sum().sum() == 0, f"You have NaN values in column(s) {cols} of your dataframe, please fix it."
        res = cls(items=_maybe_squeeze(inputs.values), path=path, inner_df=df, processor=processor, **kwargs)
        return res


Instruction:
Write a Python 3 function for
creating an ItemList in path from the inputs in csv_name.

Response:
def from_csv(cls, path:PathOrStr, csv_name:str, cols:IntsOrStrs=0, delimiter:str=None, header:str='infer',
                 processor:PreProcessors=None, **kwargs)->'ItemList':
        """Create an `ItemList` in `path` from the inputs in the `cols` of `path/csv_name`"""
        df = pd.read_csv(Path(path)/csv_name, delimiter=delimiter, header=header)
        return cls.from_df(df, path=path, cols=cols, processor=processor, **kwargs)


Instruction:
How would you explain what the following Python 3 function does
def use_partial_data(self, sample_pct:float=0.01, seed:int=None)->'ItemList':
        "Use only a sample of `sample_pct`of the full dataset and an optional `seed`."
        if seed is not None: np.random.seed(seed)
        rand_idx = np.random.permutation(range_of(self))
        cut = int(sample_pct * len(self))
        return self[rand_idx[:cut]]

Response: Use only a sample of sample_pct of the full dataset and an optional seed.


Instruction:
Write a Python 3 script for
saving self. items to fn in self. path.

Response:
def to_text(self, fn:str):
        "Save `self.items` to `fn` in `self.path`."
        with open(self.path/fn, 'w') as f: f.writelines([f'{o}\n' for o in self._relative_item_paths()])


Instruction:
Can you generate a brief explanation for the following Python 3 code
def filter_by_folder(self, include=None, exclude=None):
        "Only keep filenames in `include` folder or reject the ones in `exclude`."
        include,exclude = listify(include),listify(exclude)
        def _inner(o):
            if isinstance(o, Path): n = o.relative_to(self.path).parts[0]
            else: n = o.split(os.path.sep)[len(str(self.path).split(os.path.sep))]
            if include and not n in include: return False
            if exclude and     n in exclude: return False
            return True
        return self.filter_by_func(_inner)

Response: Only keep filenames in include folder or reject the ones in exclude.


Instruction:
Can you tell what is the following Python 3 function doing
def filter_by_rand(self, p:float, seed:int=None):
        "Keep random sample of `items` with probability `p` and an optional `seed`."
        if seed is not None: np.random.seed(seed)
        return self.filter_by_func(lambda o: rand_bool(p))

Response: Keep random sample of items with probability p and an optional seed.


Instruction:
Explain what the following Python 3 code does
def split_none(self):
        "Don't split the data and create an empty validation set."
        val = self[[]]
        val.ignore_empty = True
        return self._split(self.path, self, val)

Response: Don t split the data and create an empty validation set.


Instruction:
Can you create a Python 3 function that
splits the data between train and valid.

Response:
def split_by_list(self, train, valid):
        "Split the data between `train` and `valid`."
        return self._split(self.path, train, valid)


Instruction:
Write a Python 3 script to
split the data between train_idx and valid_idx.

Response:
def split_by_idxs(self, train_idx, valid_idx):
        "Split the data between `train_idx` and `valid_idx`."
        return self.split_by_list(self[train_idx], self[valid_idx])


Instruction:
Can you implement a function in Python 3 that
splits the data according to the indexes in valid_idx.

Response:
def split_by_idx(self, valid_idx:Collection[int])->'ItemLists':
        "Split the data according to the indexes in `valid_idx`."
        #train_idx = [i for i in range_of(self.items) if i not in valid_idx]
        train_idx = np.setdiff1d(arange_of(self.items), valid_idx)
        return self.split_by_idxs(train_idx, valid_idx)


Instruction:
Here you have a function in Python 3, explain what it does
def split_by_folder(self, train:str='train', valid:str='valid')->'ItemLists':
        "Split the data depending on the folder (`train` or `valid`) in which the filenames are."
        return self.split_by_idxs(self._get_by_folder(train), self._get_by_folder(valid))

Response: Split the data depending on the folder ( train or valid ) in which the filenames are.


Instruction:
Implement a function in Python 3 to
split the items randomly by putting valid_pct in the validation set optional seed can be passed.

Response:
def split_by_rand_pct(self, valid_pct:float=0.2, seed:int=None)->'ItemLists':
        "Split the items randomly by putting `valid_pct` in the validation set, optional `seed` can be passed."
        if valid_pct==0.: return self.split_none()
        if seed is not None: np.random.seed(seed)
        rand_idx = np.random.permutation(range_of(self))
        cut = int(valid_pct * len(self))
        return self.split_by_idx(rand_idx[:cut])


Instruction:
Can you generate a brief explanation for the following Python 3 code
def split_subsets(self, train_size:float, valid_size:float, seed=None) -> 'ItemLists':
        "Split the items into train set with size `train_size * n` and valid set with size `valid_size * n`."
        assert 0 < train_size < 1
        assert 0 < valid_size < 1
        assert train_size + valid_size <= 1.
        if seed is not None: np.random.seed(seed)
        n = len(self.items)
        rand_idx = np.random.permutation(range(n))
        train_cut, valid_cut = int(train_size * n), int(valid_size * n)
        return self.split_by_idxs(rand_idx[:train_cut], rand_idx[-valid_cut:])

Response: Split the items into train set with size train_size * n and valid set with size valid_size * n.


Instruction:
How would you code a function in Python 3 to
split the data by result of func which returns True for validation set.

Response:
def split_by_valid_func(self, func:Callable)->'ItemLists':
        "Split the data by result of `func` (which returns `True` for validation set)."
        valid_idx = [i for i,o in enumerate(self.items) if func(o)]
        return self.split_by_idx(valid_idx)


Instruction:
Can you generate the documentation for the following Python 3 function
def split_by_files(self, valid_names:'ItemList')->'ItemLists':
        "Split the data by using the names in `valid_names` for validation."
        if isinstance(self.items[0], Path): return self.split_by_valid_func(lambda o: o.name in valid_names)
        else: return self.split_by_valid_func(lambda o: os.path.basename(o) in valid_names)

Response: Split the data by using the names in valid_names for validation.


Instruction:
Given the following Python 3 function, write the documentation
def split_by_fname_file(self, fname:PathOrStr, path:PathOrStr=None)->'ItemLists':
        "Split the data by using the names in `fname` for the validation set. `path` will override `self.path`."
        path = Path(ifnone(path, self.path))
        valid_names = loadtxt_str(path/fname)
        return self.split_by_files(valid_names)

Response: Split the data by using the names in fname for the validation set. path will override self. path.


Instruction:
How would you implement a function in Python 3 that
splits the data from the col in the dataframe in self. inner_df.

Response:
def split_from_df(self, col:IntsOrStrs=2):
        "Split the data from the `col` in the dataframe in `self.inner_df`."
        valid_idx = np.where(self.inner_df.iloc[:,df_names_to_idx(col, self.inner_df)])[0]
        return self.split_by_idx(valid_idx)


Instruction:
Implement a function in Python 3 to
return label_cls or guess one from the first element of labels.

Response:
def get_label_cls(self, labels, label_cls:Callable=None, label_delim:str=None, **kwargs):
        "Return `label_cls` or guess one from the first element of `labels`."
        if label_cls is not None:               return label_cls
        if self.label_cls is not None:          return self.label_cls
        if label_delim is not None:             return MultiCategoryList
        it = index_row(labels,0)
        if isinstance(it, (float, np.float32)): return FloatList
        if isinstance(try_int(it), (str, Integral)):  return CategoryList
        if isinstance(it, Collection):          return MultiCategoryList
        return ItemList


Instruction:
Can you implement a function in Python 3 that
labels self. items with labels.

Response:
def _label_from_list(self, labels:Iterator, label_cls:Callable=None, from_item_lists:bool=False, **kwargs)->'LabelList':
        "Label `self.items` with `labels`."
        if not from_item_lists:
            raise Exception("Your data isn't split, if you don't want a validation set, please use `split_none`.")
        labels = array(labels, dtype=object)
        label_cls = self.get_label_cls(labels, label_cls=label_cls, **kwargs)
        y = label_cls(labels, path=self.path, **kwargs)
        res = self._label_list(x=self, y=y)
        return res


Instruction:
Given the following Python 3 function, write the documentation
def label_from_df(self, cols:IntsOrStrs=1, label_cls:Callable=None, **kwargs):
        "Label `self.items` from the values in `cols` in `self.inner_df`."
        labels = self.inner_df.iloc[:,df_names_to_idx(cols, self.inner_df)]
        assert labels.isna().sum().sum() == 0, f"You have NaN values in column(s) {cols} of your dataframe, please fix it."
        if is_listy(cols) and len(cols) > 1 and (label_cls is None or label_cls == MultiCategoryList):
            new_kwargs,label_cls = dict(one_hot=True, classes= cols),MultiCategoryList
            kwargs = {**new_kwargs, **kwargs}
        return self._label_from_list(_maybe_squeeze(labels), label_cls=label_cls, **kwargs)

Response: Label self. items from the values in cols in self. inner_df.


Instruction:
Write a Python 3 function that can
label every item with const.

Response:
def label_const(self, const:Any=0, label_cls:Callable=None, **kwargs)->'LabelList':
        "Label every item with `const`."
        return self.label_from_func(func=lambda o: const, label_cls=label_cls, **kwargs)


Instruction:
Here you have a function in Python 3, explain what it does
def label_empty(self, **kwargs):
        "Label every item with an `EmptyLabel`."
        kwargs['label_cls'] = EmptyLabelList
        return self.label_from_func(func=lambda o: 0., **kwargs)

Response: Label every item with an EmptyLabel.


Instruction:
Can you generate the documentation for the following Python 3 function
def label_from_func(self, func:Callable, label_cls:Callable=None, **kwargs)->'LabelList':
        "Apply `func` to every input to get its label."
        return self._label_from_list([func(o) for o in self.items], label_cls=label_cls, **kwargs)

Response: Apply func to every input to get its label.


Instruction:
Make a summary of the following Python 3 code
def label_from_folder(self, label_cls:Callable=None, **kwargs)->'LabelList':
        "Give a label to each filename depending on its folder."
        return self.label_from_func(func=lambda o: (o.parts if isinstance(o, Path) else o.split(os.path.sep))[-2],
                                    label_cls=label_cls, **kwargs)

Response: Give a label to each filename depending on its folder.