Instruction:
Can you implement a function in Python 3 that
looks up the path to the dataset specified in the data home directory, which is found using the ``get_data_home`` function. By default data home is colocated with the code, but can be modified with the YELLOWBRICK_DATA environment variable, or passing in a different directory. The file returned will be by default, the name of the dataset in compressed CSV format. Other files and extensions can be passed in to locate other data types or auxilliary files. If the dataset is not found a ``DatasetsError`` is raised by default. Parameters ---------- dataset : str The name of the dataset; should either be a folder in data home or specified in the yellowbrick.datasets.DATASETS variable. data_home : str, optional The path on disk where data is stored. If not passed in, it is looked up from YELLOWBRICK_DATA or the default returned by ``get_data_home``. fname : str, optional The filename to look up in the dataset path, by default it will be the name of the dataset. The fname must include an extension. ext : str, default: ".csv.gz" The extension of the data to look up in the dataset path, if the fname is specified then the ext parameter is ignored. If ext is None then the directory of the dataset will be returned. raises : bool, default: True If the path does not exist, raises a DatasetsError unless this flag is set to False, at which point None is returned (e.g. for checking if the path exists or not). Returns ------- path : str or None A path to the requested file, guaranteed to exist if an exception is not raised during processing of the request (unless None is returned). raises : DatasetsError If raise is True and the path does not exist, raises a DatasetsError.

Response:
def find_dataset_path(dataset, data_home=None, fname=None, ext=".csv.gz", raises=True):
    """
    Looks up the path to the dataset specified in the data home directory,
    which is found using the ``get_data_home`` function. By default data home
    is colocated with the code, but can be modified with the YELLOWBRICK_DATA
    environment variable, or passing in a different directory.

    The file returned will be by default, the name of the dataset in compressed
    CSV format. Other files and extensions can be passed in to locate other data
    types or auxilliary files.

    If the dataset is not found a ``DatasetsError`` is raised by default.

    Parameters
    ----------
    dataset : str
        The name of the dataset; should either be a folder in data home or
        specified in the yellowbrick.datasets.DATASETS variable.

    data_home : str, optional
        The path on disk where data is stored. If not passed in, it is looked
        up from YELLOWBRICK_DATA or the default returned by ``get_data_home``.

    fname : str, optional
        The filename to look up in the dataset path, by default it will be the
        name of the dataset. The fname must include an extension.

    ext : str, default: ".csv.gz"
        The extension of the data to look up in the dataset path, if the fname
        is specified then the ext parameter is ignored. If ext is None then
        the directory of the dataset will be returned.

    raises : bool, default: True
        If the path does not exist, raises a DatasetsError unless this flag is set
        to False, at which point None is returned (e.g. for checking if the
        path exists or not).

    Returns
    -------
    path : str or None
        A path to the requested file, guaranteed to exist if an exception is
        not raised during processing of the request (unless None is returned).

    raises : DatasetsError
        If raise is True and the path does not exist, raises a DatasetsError.
    """
    # Figure out the root directory of the datasets
    data_home = get_data_home(data_home)

    # Figure out the relative path to the dataset
    if fname is None:
        if ext is None:
            path = os.path.join(data_home, dataset)
        else:
            path = os.path.join(data_home, dataset, "{}{}".format(dataset, ext))
    else:
        path = os.path.join(data_home, dataset, fname)

    # Determine if the path exists
    if not os.path.exists(path):

        # Suppress exceptions if required
        if not raises:
            return None

        raise DatasetsError((
            "could not find dataset at {} - does it need to be downloaded?"
        ).format(path))

    return path


Instruction:
Explain what the following Python 3 code does
def dataset_exists(dataset, data_home=None):
    data_home = get_data_home(data_home)
    path = os.path.join(data_home, dataset)

    return os.path.exists(path) and os.path.isdir(path)

Response: Checks to see if a dataset exists in the data home directory.


Instruction:
Can you generate the documentation for the following Python 3 function
def dataset_archive(dataset, signature, data_home=None, ext=".zip"):
    data_home = get_data_home(data_home)
    path = os.path.join(data_home, dataset+ext)

    if os.path.exists(path) and os.path.isfile(path):
        return sha256sum(path) == signature

    return False

Response: Checks to see if the dataset archive file exists in the data home directory and is the latest version of the dataset.


Instruction:
Explain what the following Python 3 code does
def cleanup_dataset(dataset, data_home=None, ext=".zip"):
    removed = 0
    data_home = get_data_home(data_home)

    # Paths to remove
    datadir = os.path.join(data_home, dataset)
    archive = os.path.join(data_home, dataset+ext)

    # Remove directory and contents
    if os.path.exists(datadir):
        shutil.rmtree(datadir)
        removed += 1

    # Remove the archive file
    if os.path.exists(archive):
        os.remove(archive)
        removed += 1

    return removed

Response: Removes the dataset directory and archive file from the data home directory.


Instruction:
Explain what the following Python 3 code does
def fit_transform_poof(self, X, y=None, **kwargs):
        Xp = self.fit_transform(X, y, **kwargs)
        self.poof(**kwargs)
        return Xp

Response: Fit to data transform it then visualize it.


Instruction:
Write a Python 3 function for
fitting the object to the specified features.

Response:
def fit(self, X, y=None, **fit_params):
        """
        This method performs preliminary computations in order to set up the
        figure or perform other analyses. It can also call drawing methods in
        order to set up various non-instance related figure elements.

        This method must return self.
        """

        # Handle the feature names if they're None.
        if self.features_ is None:

            # If X is a data frame, get the columns off it.
            if is_dataframe(X):
                self.features_ = np.array(X.columns)

            # Otherwise create numeric labels for each column.
            else:
                _, ncols = X.shape
                self.features_ = np.arange(0, ncols)

        return self


Instruction:
Implement a Python 3 function for
fitting the data to the target class and store the class labels for the target class.

Response:
def fit(self, X, y=None, **kwargs):
        """
        The fit method is the primary drawing input for the
        visualization since it has both the X and y data required for the
        viz and the transform method does not.

        Parameters
        ----------
        X : ndarray or DataFrame of shape n x m
            A matrix of n instances with m features

        y : ndarray or Series of length n
            An array or series of target or class values

        kwargs : dict
            Pass generic arguments to the drawing method

        Returns
        -------
        self : instance
            Returns the instance of the transformer/visualizer
        """
        super(DataVisualizer, self).fit(X, y, **kwargs)

        # Store the classes for the legend if they're None.
        if self.classes_ is None:
            # TODO: Is this the most efficient method?
            self.classes_ = [str(label) for label in np.unique(y)]

        # Draw the instances
        self.draw(X, y, **kwargs)

        # Fit always returns self.
        return self


Instruction:
Can you implement a function in Python 3 that
returns a color palette object with color definition and handling.

Response:
def color_palette(palette=None, n_colors=None):
    """
    Return a color palette object with color definition and handling.

    Calling this function with ``palette=None`` will return the current
    matplotlib color cycle.

    This function can also be used in a ``with`` statement to temporarily
    set the color cycle for a plot or set of plots.

    Parameters
    ----------

    palette : None or str or sequence
        Name of a palette or ``None`` to return the current palette. If a
        sequence the input colors are used but possibly cycled.

        Available palette names from :py:mod:`yellowbrick.colors.palettes` are:

        .. hlist::
            :columns: 3

            * :py:const:`accent`
            * :py:const:`dark`
            * :py:const:`paired`
            * :py:const:`pastel`
            * :py:const:`bold`
            * :py:const:`muted`
            * :py:const:`colorblind`
            * :py:const:`sns_colorblind`
            * :py:const:`sns_deep`
            * :py:const:`sns_muted`
            * :py:const:`sns_pastel`
            * :py:const:`sns_bright`
            * :py:const:`sns_dark`
            * :py:const:`flatui`
            * :py:const:`neural_paint`

    n_colors : None or int
        Number of colors in the palette. If ``None``, the default will depend
        on how ``palette`` is specified. Named palettes default to 6 colors
        which allow the use of the names "bgrmyck", though others do have more
        or less colors; therefore reducing the size of the list can only be
        done by specifying this parameter. Asking for more colors than exist
        in the palette will cause it to cycle.

    Returns
    -------
    list(tuple)
        Returns a ColorPalette object, which behaves like a list, but can be
        used as a context manager and possesses functions to convert colors.

    .. seealso::

        :func:`.set_palette`
            Set the default color cycle for all plots.
        :func:`.set_color_codes`
            Reassign color codes like ``"b"``, ``"g"``, etc. to
            colors from one of the yellowbrick palettes.
        :func:`..colors.resolve_colors`
            Resolve a color map or listed sequence of colors.

    """
    if palette is None:
        palette = get_color_cycle()
        if n_colors is None:
            n_colors = len(palette)

    elif not isinstance(palette, str):
        if n_colors is None:
            n_colors = len(palette)

    else:
        if palette.lower() not in PALETTES:
            raise YellowbrickValueError(
                "'{}' is not a recognized palette!".format(palette)
            )

        palette = PALETTES[palette.lower()]
        if n_colors is None:
            n_colors = len(palette)

    # Always return as many colors as we asked for
    pal_cycle = cycle(palette)
    palette = [next(pal_cycle) for _ in range(n_colors)]

    # Always return in RGB tuple format
    try:
        palette = map(mpl.colors.colorConverter.to_rgb, palette)
        palette = ColorPalette(palette)
    except ValueError:
        raise YellowbrickValueError(
            "Could not generate a palette for %s" % str(palette)
        )

    return palette


Instruction:
Can you create a Python 3 function that
sets the color codes of the base object in the current color cycle.

Response:
def set_color_codes(palette="accent"):
    """
    Change how matplotlib color shorthands are interpreted.

    Calling this will change how shorthand codes like "b" or "g"
    are interpreted by matplotlib in subsequent plots.

    Parameters
    ----------
    palette : str
        Named yellowbrick palette to use as the source of colors.

    See Also
    --------
    set_palette : Color codes can also be set through the function that
                  sets the matplotlib color cycle.
    """

    if palette not in PALETTES:
        raise YellowbrickValueError(
            "'{}' is not a recognized palette!".format(palette)
        )

    # Fetch the colors and adapt the length
    colors = PALETTES[palette]

    if len(colors) > 7:
        # Truncate colors that are longer than 7
        colors = colors[:7]

    elif len(colors) < 7:
        # Add the key (black) color to colors that are shorter than 7
        colors = colors + [YB_KEY]

    # Set the color codes on matplotlib
    for code, color in zip("bgrmyck", colors):
        rgb = mpl.colors.colorConverter.to_rgb(color)
        mpl.colors.colorConverter.colors[code] = rgb
        mpl.colors.colorConverter.cache[code] = rgb


Instruction:
Write a Python 3 script to
return a ListedColormap object from a named color sequence palette.

Response:
def color_sequence(palette=None, n_colors=None):
    """
    Return a `ListedColormap` object from a named sequence palette. Useful
    for continuous color scheme values and color maps.

    Calling this function with ``palette=None`` will return the default
    color sequence: Color Brewer RdBu.

    Parameters
    ----------

    palette : None or str or sequence
        Name of a palette or ``None`` to return the default palette. If a
        sequence the input colors are used to create a ListedColormap.

        The currently implemented color sequences are from Color Brewer.

        Available palette names from :py:mod:`yellowbrick.colors.palettes` are:

        .. hlist::
            :columns: 3

            * :py:const: Blues
            * :py:const: BrBG
            * :py:const: BuGn
            * :py:const: BuPu
            * :py:const: GnBu
            * :py:const: Greens
            * :py:const: Greys
            * :py:const: OrRd
            * :py:const: Oranges
            * :py:const: PRGn
            * :py:const: PiYG
            * :py:const: PuBu
            * :py:const: PuBuGn
            * :py:const: PuOr
            * :py:const: PuRd
            * :py:const: Purples
            * :py:const: RdBu
            * :py:const: RdGy
            * :py:const: RdPu
            * :py:const: RdYlBu
            * :py:const: RdYlGn
            * :py:const: Reds
            * :py:const: Spectral
            * :py:const: YlGn
            * :py:const: YlGnBu
            * :py:const: YlOrBr
            * :py:const: YlOrRd
            * :py:const: ddl_heat

    n_colors : None or int
        Number of colors in the palette. If ``None``, the default will depend
        on how ``palette`` is specified - selecting the largest sequence for
        that palette name. Note that sequences have a minimum lenght of 3 - if
        a number of colors is specified that is not available for the sequence
        a ``ValueError`` is raised.

    Returns
    -------
    colormap
        Returns a ListedColormap object, an artist object from the matplotlib
        library that can be used wherever a colormap is necessary.
    """
    # Select the default colormap if None is passed in.
    palette = palette or DEFAULT_SEQUENCE

    # Create a listed color map from the sequence
    if not isinstance(palette, str):
        return mplcol.ListedColormap(palette)

    # Otherwise perform a case-insensitive lookup
    sequences = {key.lower(): key for key in SEQUENCES.keys()}
    if palette.lower() not in sequences:
        raise YellowbrickValueError(
            "'{}' is not a recognized palette!".format(palette)
        )

    # Collect the palette into the dictionary of lists.
    n_palettes = SEQUENCES[sequences[palette.lower()]]

    # If no definitive color is passed in, maximize it.
    if n_colors is None:
        n_colors = max(n_palettes.keys())

    else:
        if n_colors not in n_palettes.keys():
            raise YellowbrickValueError(
                "No {} palette of length {}".format(palette, n_colors)
            )

    # Return the color map from the sequence
    return mplcol.ListedColormap(n_palettes[n_colors], name=palette, N=n_colors)


Instruction:
Explain what the following Python 3 code does
def as_hex(self):
        hex = [mpl.colors.rgb2hex(rgb) for rgb in self]
        return ColorPalette(hex)

Response: Return a color palette with hex codes instead of RGB values.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def as_rgb(self):
        rgb = [mpl.colors.colorConverter.to_rgb(hex) for hex in self]
        return ColorPalette(rgb)

Response: Return a color palette with RGB values instead of hex codes.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def plot(self, size=1):
        n = len(self)
        fig, ax = plt.subplots(1, 1, figsize=(n * size, size))
        ax.imshow(np.arange(n).reshape(1,n),
                  cmap=mpl.colors.ListedColormap(list(self)),
                  interpolation="nearest", aspect="auto")
        ax.set_xticks(np.arange(n) - .5)
        ax.set_yticks([-.5, .5])
        ax.set_xticklabels([])
        ax.set_yticklabels([])

Response: Plot the values in the color palette as a horizontal array.


Instruction:
Here you have a function in Python 3, explain what it does
def _make_path(self, path, name):
        if not os.path.exists(path):
            os.mkdirs(path)

        if os.path.isdir(path) :
            return os.path.join(path, name)

        return path

Response: Make a path to the log file.


Instruction:
Given the following Python 3 function, write the documentation
def plot_manifold_embedding(self, algorithm="lle", path="images"):
        _, ax = plt.subplots(figsize=(9,6))
        path = self._make_path(path, "s_curve_{}_manifold.png".format(algorithm))

        oz = Manifold(
            ax=ax, manifold=algorithm,
            target='continuous', colors='nipy_spectral'
        )

        oz.fit(self.X, self.y)
        oz.poof(outpath=path)

Response: Draw the manifold embedding for the specified algorithm


Instruction:
Write a Python 3 script for
downloading the zipped data set at the given URL and saves it to the data directory specified by get_data_home.

Response:
def download_data(url, signature, data_home=None, replace=False, extract=True):
    """
    Downloads the zipped data set specified at the given URL, saving it to
    the data directory specified by ``get_data_home``. This function verifies
    the download with the given signature and extracts the archive.

    Parameters
    ----------
    url : str
        The URL of the dataset on the Internet to GET

    signature : str
        The SHA 256 hash of the dataset archive being downloaded to verify
        that the dataset has been correctly downloaded

    data_home : str, optional
        The path on disk where data is stored. If not passed in, it is looked
        up from YELLOWBRICK_DATA or the default returned by ``get_data_home``.

    replace : bool, default: False
        If the data archive already exists, replace the dataset. If this is
        False and the dataset exists, an exception is raised.

    extract : bool, default: True
        Extract the archive file after downloading it
    """
    data_home = get_data_home(data_home)

    # Get the name of the file from the URL
    basename = os.path.basename(url)
    name, _ = os.path.splitext(basename)

    # Get the archive and data directory paths
    archive = os.path.join(data_home, basename)
    datadir = os.path.join(data_home, name)

    # If the archive exists cleanup or raise override exception
    if os.path.exists(archive):
        if not replace:
            raise DatasetsError((
                "dataset already exists at {}, set replace=False to overwrite"
            ).format(archive))

        cleanup_dataset(name, data_home=data_home)

    # Create the output directory if it does not exist
    if not os.path.exists(datadir):
        os.mkdir(datadir)

    # Fetch the response in a streaming fashion and write it to disk.
    response = urlopen(url)

    with open(archive, 'wb') as f:
        while True:
            chunk = response.read(CHUNK)
            if not chunk:
                break
            f.write(chunk)

    # Compare the signature of the archive to the expected one
    if sha256sum(archive) != signature:
        raise ValueError(
            "Download signature does not match hardcoded signature!"
        )

    # If extract, extract the zipfile.
    if extract:
        zf = zipfile.ZipFile(archive)
        zf.extractall(path=data_home)


Instruction:
Given the following Python 3 function, write the documentation
def set_aesthetic(palette="yellowbrick", font="sans-serif", font_scale=1,
                  color_codes=True, rc=None):
    """
    Set aesthetic parameters in one step.

    Each set of parameters can be set directly or temporarily, see the
    referenced functions below for more information.

    Parameters
    ----------
    palette : string or sequence
        Color palette, see :func:`color_palette`
    font : string
        Font family, see matplotlib font manager.
    font_scale : float, optional
        Separate scaling factor to independently scale the size of the
        font elements.
    color_codes : bool
        If ``True`` and ``palette`` is a yellowbrick palette, remap the shorthand
        color codes (e.g. "b", "g", "r", etc.) to the colors from this palette.
    rc : dict or None
        Dictionary of rc parameter mappings to override the above.
    """
    _set_context(font_scale)
    set_style(rc={"font.family": font})
    set_palette(palette, color_codes=color_codes)
    if rc is not None:
        mpl.rcParams.update(rc)

Response: Set the parameters of the aesthetic for this base class.


Instruction:
Make a summary of the following Python 3 code
def _axes_style(style=None, rc=None):
    if isinstance(style, dict):
        style_dict = style

    else:
        # Define colors here
        dark_gray = ".15"
        light_gray = ".8"

        # Common parameters
        style_dict = {
            "figure.facecolor": "white",
            "text.color": dark_gray,
            "axes.labelcolor": dark_gray,
            "legend.frameon": False,
            "legend.numpoints": 1,
            "legend.scatterpoints": 1,
            "xtick.direction": "out",
            "ytick.direction": "out",
            "xtick.color": dark_gray,
            "ytick.color": dark_gray,
            "axes.axisbelow": True,
            "image.cmap": "Greys",
            "font.family": ["sans-serif"],
            "font.sans-serif": ["Arial", "Liberation Sans",
                                "Bitstream Vera Sans", "sans-serif"],
            "grid.linestyle": "-",
            "axes.grid": True,
            "lines.solid_capstyle": "round",
            "axes.facecolor": "white",
            "axes.edgecolor": light_gray,
            "axes.linewidth": 1.25,
            "grid.color": light_gray,
            "xtick.major.size": 0,
            "ytick.major.size": 0,
            "xtick.minor.size": 0,
            "ytick.minor.size": 0,
        }

    # Override these settings with the provided rc dictionary
    if rc is not None:
        rc = {k: v for k, v in rc.items() if k in _style_keys}
        style_dict.update(rc)

    # Wrap in an _AxesStyle object so this can be used in a with statement
    style_object = _AxesStyle(style_dict)

    return style_object

Response: Returns a parameter dictionary for the aesthetic style of the aesthetic.


Instruction:
Can you create a Python 3 function that
sets the aesthetic style of the plots.

Response:
def set_style(style=None, rc=None):
    """
    Set the aesthetic style of the plots.

    This affects things like the color of the axes, whether a grid is
    enabled by default, and other aesthetic elements.

    Parameters
    ----------
    style : dict, None, or one of {darkgrid, whitegrid, dark, white, ticks}
        A dictionary of parameters or the name of a preconfigured set.
    rc : dict, optional
        Parameter mappings to override the values in the preset seaborn
        style dictionaries. This only updates parameters that are
        considered part of the style definition.
    """
    style_object = _axes_style(style, rc)
    mpl.rcParams.update(style_object)


Instruction:
Make a summary of the following Python 3 code
def _set_context(context=None, font_scale=1, rc=None):
    context_object = _plotting_context(context, font_scale, rc)
    mpl.rcParams.update(context_object)

Response: Set the plotting context parameters for a base context.


Instruction:
How would you code a function in Python 3 to
set the matplotlib color cycle using a seaborn color palette.

Response:
def set_palette(palette, n_colors=None, color_codes=False):
    """
    Set the matplotlib color cycle using a seaborn palette.

    Parameters
    ----------
    palette : yellowbrick color palette | seaborn color palette (with ``sns_`` prepended)
        Palette definition. Should be something that :func:`color_palette`
        can process.
    n_colors : int
        Number of colors in the cycle. The default number of colors will depend
        on the format of ``palette``, see the :func:`color_palette`
        documentation for more information.
    color_codes : bool
        If ``True`` and ``palette`` is a seaborn palette, remap the shorthand
        color codes (e.g. "b", "g", "r", etc.) to the colors from this palette.
    """
    colors = color_palette(palette, n_colors)
    if mpl_ge_150:
        from cycler import cycler
        cyl = cycler('color', colors)
        mpl.rcParams['axes.prop_cycle'] = cyl
    else:
        mpl.rcParams["axes.color_cycle"] = list(colors)
    mpl.rcParams["patch.facecolor"] = colors[0]
    if color_codes:
        set_color_codes(palette)


Instruction:
Write a Python 3 function for
displaying a dispersion plot for words within a corpus.

Response:
def dispersion(words, corpus, y=None, ax=None, colors=None, colormap=None,
               labels=None, annotate_docs=False, ignore_case=False, **kwargs):
    """ Displays lexical dispersion plot for words in a corpus

    This helper function is a quick wrapper to utilize the DisperstionPlot
    Visualizer for one-off analysis

    Parameters
    ----------

    words : list
        A list of words whose dispersion will be examined within a corpus

    y : ndarray or Series of length n
        An optional array or series of target or class values for
        instances. If this is specified, then the points will be colored
        according to their class.

    corpus : list
        Should be provided as a list of documents that contain
        a list of words in the order they appear in the document.

    ax : matplotlib axes, default: None
        The axes to plot the figure on.

    labels : list of strings
        The names of the classes in the target, used to create a legend.
        Labels must match names of classes in sorted order.

    colors : list or tuple of colors
        Specify the colors for each individual class

    colormap : string or matplotlib cmap
        Qualitative colormap for discrete target

    annotate_docs : boolean, default: False
        Specify whether document boundaries will be displayed.  Vertical lines
        are positioned at the end of each document.

    ignore_case : boolean, default: False
	Specify whether input  will be case-sensitive.

    kwargs : dict
        Pass any additional keyword arguments to the super class.

    Returns
    -------
    ax: matplotlib axes
        Returns the axes that the plot was drawn on
    """

    # Instantiate the visualizer
    visualizer = DispersionPlot(
        words, ax=ax, colors=colors, colormap=colormap,
        ignore_case=ignore_case, labels=labels,
        annotate_docs=annotate_docs, **kwargs
    )

    # Fit and transform the visualizer (calls draw)
    visualizer.fit(corpus, y, **kwargs)

    # Return the axes object on the visualizer
    return visualizer.ax


Instruction:
How would you explain what the following Python 3 function does
def fit(self, X, y=None, **kwargs):

        if y is not None:
            self.classes_ = np.unique(y)
        elif y is None and self.labels is not None:
            self.classes_ = np.array([self.labels[0]])
        else:
            self.classes_ = np.array([self.NULL_CLASS])

        # Create an index (e.g. the y position) for the target words
        self.indexed_words_ = np.flip(self.target_words, axis=0)
        if self.ignore_case:
            self.indexed_words_ = np.array([w.lower() for w in self.indexed_words_])

        # Stack is used to create a 2D array from the generator
        try:
            points_target = np.stack(self._compute_dispersion(X, y))
        except ValueError:
            raise YellowbrickValueError((
                "No indexed words were found in the corpus"
                ))
        points = np.stack(zip(points_target[:,0].astype(int),
                              points_target[:,1].astype(int)))

        self.target = points_target[:,2]

        self._check_missing_words(points)

        self.draw(points, self.target)
        return self

Response:
This method is the primary drawing input for the dispersion
            visualization.


Instruction:
Write a Python 3 function that can
create the canvas and draws the plot on it.

Response:
def draw(self, points, target=None, **kwargs):
        """
        Called from the fit method, this method creates the canvas and
        draws the plot on it.
        Parameters
        ----------
        kwargs: generic keyword arguments.
        """

        # Resolve the labels with the classes
        labels = self.labels if self.labels is not None else self.classes_
        if len(labels) != len(self.classes_):
            raise YellowbrickValueError((
                "number of supplied labels ({}) does not "
                "match the number of classes ({})"
            ).format(len(labels), len(self.classes_)))

        # Create the color mapping for the labels.
        color_values = resolve_colors(
            n_colors=len(labels), colormap=self.colormap, colors=self.color)
        colors = dict(zip(labels, color_values))

        # Transform labels into a map of class to label
        labels = dict(zip(self.classes_, labels))

        # Define boundaries with a vertical line
        if self.annotate_docs:
            for xcoords in self.boundaries_:
                self.ax.axvline(x=xcoords, color='lightgray', linestyle='dashed')

        series = defaultdict(lambda: {'x':[], 'y':[]})

        if target is not None:
            for point, t in zip(points, target):
                label = labels[t]
                series[label]['x'].append(point[0])
                series[label]['y'].append(point[1])
        else:
            label = self.classes_[0]
            for x, y in points:
                series[label]['x'].append(x)
                series[label]['y'].append(y)

        for label, points in series.items():
            self.ax.scatter(points['x'], points['y'], marker='|',
                            c=colors[label], zorder=100, label=label)

        self.ax.set_yticks(list(range(len(self.indexed_words_))))
        self.ax.set_yticklabels(self.indexed_words_)


Instruction:
Can you generate the documentation for the following Python 3 function
def manifold(self, transformer):
        if not is_estimator(transformer):
            if transformer not in self.ALGORITHMS:
                raise YellowbrickValueError(
                    "could not create manifold for '%s'".format(str(transformer))
                )

            # Create a new transformer with the specified params
            self._name = MANIFOLD_NAMES[transformer]
            transformer = clone(self.ALGORITHMS[transformer])
            params = {
                "n_components": 2,
                "n_neighbors": self.n_neighbors,
                "random_state": self.random_state,
            }

            for param in list(params.keys()):
                if param not in transformer.get_params():
                    del params[param]

            transformer.set_params(**params)

        self._manifold = transformer
        if self._name is None:
            self._name = self._manifold.__class__.__name__

Response: Creates the manifold estimator if a string value is passed in validates other objects passed in.


Instruction:
Create a Python 3 function to
fit the manifold on X and transforms the data to plot it on the axes.

Response:
def fit_transform(self, X, y=None):
        """
        Fits the manifold on X and transforms the data to plot it on the axes.
        The optional y specified can be used to declare discrete colors. If
        the target is set to 'auto', this method also determines the target
        type, and therefore what colors will be used.

        Note also that fit records the amount of time it takes to fit the
        manifold and reports that information in the visualization.

        Parameters
        ----------
        X : array-like of shape (n, m)
            A matrix or data frame with n instances and m features where m > 2.

        y : array-like of shape (n,), optional
            A vector or series with target values for each instance in X. This
            vector is used to determine the color of the points in X.

        Returns
        -------
        self : Manifold
            Returns the visualizer object.
        """
        # Determine target type
        self._determine_target_color_type(y)

        # Compute classes and colors if target type is discrete
        if self._target_color_type == DISCRETE:
            self.classes_ = np.unique(y)

            color_kwargs = {'n_colors': len(self.classes_)}

            if isinstance(self.colors, str):
                color_kwargs['colormap'] = self.colors
            else:
                color_kwargs['colors'] = self.colors

            self._colors = resolve_colors(**color_kwargs)

        # Compute target range if colors are continuous
        elif self._target_color_type == CONTINUOUS:
            y = np.asarray(y)
            self.range_ = (y.min(), y.max())

        with Timer() as self.fit_time_:
            Xp = self.manifold.fit_transform(X)

        self.draw(Xp, y)
        return Xp


Instruction:
Can you generate the documentation for the following Python 3 function
def transform(self, X):
        try:
            return self.manifold.transform(X)
        except AttributeError as e:
            raise AttributeError(str(e) + " try using fit_transform instead.")

Response: Returns the transformed data points from the manifold embedding.


Instruction:
How would you explain what the following Python 3 function does
def draw(self, X, y=None):
        scatter_kwargs = {"alpha": self.alpha}

        # Determine the colors
        if self._target_color_type == SINGLE:
            scatter_kwargs["c"] = "b"

        elif self._target_color_type == DISCRETE:
            if y is None:
                raise YellowbrickValueError("y is required for discrete target")

            scatter_kwargs["c"] = [
                self._colors[np.searchsorted(self.classes_, (yi))] for yi in y
            ]

        elif self._target_color_type == CONTINUOUS:
            if y is None:
                raise YellowbrickValueError("y is required for continuous target")

            # TODO manually make colorbar so we can draw it in finalize
            scatter_kwargs["c"] = y
            scatter_kwargs["cmap"] = self.colors or palettes.DEFAULT_SEQUENCE

        else:
            # Technically this should never be raised
            raise NotFitted("could not determine target color type")

        # Draw the scatter plot with the associated colors and alpha
        self._scatter = self.ax.scatter(X[:,0], X[:,1], **scatter_kwargs)
        return self.ax

Response: Draws the points described by X and colored by the points in y.


Instruction:
Given the following Python 3 function, write the documentation
def finalize(self):
        self.set_title(
            '{} Manifold (fit in {:0.2f} seconds)'.format(
                self._name, self.fit_time_.interval
            )
        )
        self.ax.set_xticklabels([])
        self.ax.set_yticklabels([])

        if self._target_color_type == DISCRETE:
            # Add the legend
            manual_legend(self, self.classes_, self._colors, frameon=True)

        elif self._target_color_type == CONTINUOUS:
            # Add the color bar
            plt.colorbar(self._scatter, ax=self.ax)

Response: Add title and modify axes to make the image ready for display.


Instruction:
Make a summary of the following Python 3 code
def _determine_target_color_type(self, y):
        if y is None:
            self._target_color_type = SINGLE
        elif self.target == "auto":
            # NOTE: See #73 for a generalization to use when implemented
            if len(np.unique(y)) < 10:
                self._target_color_type = DISCRETE
            else:
                self._target_color_type = CONTINUOUS
        else:
            self._target_color_type = self.target

        if self._target_color_type not in {SINGLE, DISCRETE, CONTINUOUS}:
            raise YellowbrickValueError((
                "could not determine target color type "
                "from target='{}' to '{}'"
            ).format(self.target, self._target_color_type))

Response: Determines the target color type from the vector y.


Instruction:
Can you tell what is the following Python 3 function doing
def rfecv(model, X, y, ax=None, step=1, groups=None, cv=None,
          scoring=None, **kwargs):
    """
    Performs recursive feature elimination with cross-validation to determine
    an optimal number of features for a model. Visualizes the feature subsets
    with respect to the cross-validation score.

    This helper function is a quick wrapper to utilize the RFECV visualizer
    for one-off analysis.

    Parameters
    ----------
    model : a scikit-learn estimator
        An object that implements ``fit`` and provides information about the
        relative importance of features with either a ``coef_`` or
        ``feature_importances_`` attribute.

        Note that the object is cloned for each validation.

    X : array-like, shape (n_samples, n_features)
        Training vector, where n_samples is the number of samples and
        n_features is the number of features.

    y : array-like, shape (n_samples) or (n_samples, n_features), optional
        Target relative to X for classification or regression.

    ax : matplotlib.Axes object, optional
        The axes object to plot the figure on.

    step : int or float, optional (default=1)
        If greater than or equal to 1, then step corresponds to the (integer)
        number of features to remove at each iteration. If within (0.0, 1.0),
        then step corresponds to the percentage (rounded down) of features to
        remove at each iteration.

    groups : array-like, with shape (n_samples,), optional
        Group labels for the samples used while splitting the dataset into
        train/test set.

    cv : int, cross-validation generator or an iterable, optional
        Determines the cross-validation splitting strategy.
        Possible inputs for cv are:

          - None, to use the default 3-fold cross-validation,
          - integer, to specify the number of folds.
          - An object to be used as a cross-validation generator.
          - An iterable yielding train/test splits.

        see the scikit-learn
        `cross-validation guide <http://scikit-learn.org/stable/modules/cross_validation.html>`_
        for more information on the possible strategies that can be used here.

    scoring : string, callable or None, optional, default: None
        A string or scorer callable object / function with signature
        ``scorer(estimator, X, y)``. See scikit-learn model evaluation
        documentation for names of possible metrics.

    kwargs : dict
        Keyword arguments that are passed to the base class and may influence
        the visualization as defined in other Visualizers. These arguments are
        also passed to the `poof()` method, e.g. can pass a path to save the
        figure to.

    Returns
    -------
    ax : matplotlib axes
        Returns the axes that the rfecv were drawn on.
    """
    # Initialize the visualizer
    oz = RFECV(model, ax=ax, step=step, groups=groups, cv=cv, scoring=scoring)

    # Fit and poof the visualizer
    oz.fit(X, y)
    oz.poof(**kwargs)
    return oz.ax

Response: This function is used to visualize the RFECV model for one - off analysis.


Instruction:
Can you write a function in Python 3 where it
fits the RFECV with the wrapped model to the specified data and draws the RFECV curve with the optimal number of features found.

Response:
def fit(self, X, y=None):
        """
        Fits the RFECV with the wrapped model to the specified data and draws
        the rfecv curve with the optimal number of features found.

        Parameters
        ----------
        X : array-like, shape (n_samples, n_features)
            Training vector, where n_samples is the number of samples and
            n_features is the number of features.

        y : array-like, shape (n_samples) or (n_samples, n_features), optional
            Target relative to X for classification or regression.

        Returns
        -------
        self : instance
            Returns the instance of the RFECV visualizer.
        """
        X, y = check_X_y(X, y, "csr")
        n_features = X.shape[1]

        # This check is kind of unnecessary since RFE will do it, but it's
        # nice to get it out of the way ASAP and raise a meaningful error.
        if 0.0 < self.step < 1.0:
            step = int(max(1, self.step * n_features))
        else:
            step = int(self.step)

        if step < 0:
            raise YellowbrickValueError("step must be >0")

        # Create the RFE model
        rfe = RFE(self.estimator, step=step)
        self.n_feature_subsets_ = np.arange(1, n_features+step, step)

        # Create the cross validation params
        # TODO: handle random state
        cv_params = {
            key: self.get_params()[key]
            for key in ('groups', 'cv', 'scoring')
        }

        # Perform cross-validation for each feature subset
        scores = []
        for n_features_to_select in self.n_feature_subsets_:
            rfe.set_params(n_features_to_select=n_features_to_select)
            scores.append(cross_val_score(rfe, X, y, **cv_params))

        # Convert scores to array
        self.cv_scores_ = np.array(scores)

        # Find the best RFE model
        bestidx = self.cv_scores_.mean(axis=1).argmax()
        self.n_features_ = self.n_feature_subsets_[bestidx]

        # Fit the final RFE model for the number of features
        self.rfe_estimator_ = rfe
        self.rfe_estimator_.set_params(n_features_to_select=self.n_features_)
        self.rfe_estimator_.fit(X, y)

        # Rewrap the visualizer to use the rfe estimator
        self._wrapped = self.rfe_estimator_

        # Hoist the RFE params to the visualizer
        self.support_ = self.rfe_estimator_.support_
        self.ranking_ = self.rfe_estimator_.ranking_

        self.draw()
        return self


Instruction:
Here you have a function in Python 3, explain what it does
def draw(self, **kwargs):
        # Compute the curves
        x = self.n_feature_subsets_
        means = self.cv_scores_.mean(axis=1)
        sigmas = self.cv_scores_.std(axis=1)

        # Plot one standard deviation above and below the mean
        self.ax.fill_between(x, means - sigmas, means+sigmas, alpha=0.25)

        # Plot the curve
        self.ax.plot(x, means, 'o-')

        # Plot the maximum number of features
        self.ax.axvline(
            self.n_features_, c='k', ls='--',
            label="n_features = {}\nscore = {:0.3f}".format(
                self.n_features_, self.cv_scores_.mean(axis=1).max()
            )
        )

        return self.ax

Response: Renders the rfecv curve.


Instruction:
Make a summary of the following Python 3 code
def scatterviz(X,
               y=None,
               ax=None,
               features=None,
               classes=None,
               color=None,
               colormap=None,
               markers=None,
               alpha=1.0,
               **kwargs):
    """Displays a bivariate scatter plot.

    This helper function is a quick wrapper to utilize the ScatterVisualizer
    (Transformer) for one-off analysis.

    Parameters
    ----------

    X : ndarray or DataFrame of shape n x m
        A matrix of n instances with m features

    y : ndarray or Series of length n, default: None
        An array or series of target or class values

    ax : matplotlib axes, default: None
        The axes to plot the figure on.

    features : list of strings, default: None
        The names of two features or columns.
        More than that will raise an error.

    classes : list of strings, default: None
        The names of the classes in the target

    color : list or tuple of colors, default: None
        Specify the colors for each individual class

    colormap : string or matplotlib cmap, default: None
        Sequential colormap for continuous target

    markers : iterable of strings, default: ,+o*vhd
        Matplotlib style markers for points on the scatter plot points

    alpha : float, default: 1.0
        Specify a transparency where 1 is completely opaque and 0 is completely
        transparent. This property makes densely clustered points more visible.

    Returns
    -------
    ax : matplotlib axes
        Returns the axes that the parallel coordinates were drawn on.
    """
    # Instantiate the visualizer
    visualizer = ScatterVisualizer(ax=ax, features=features, classes=classes,
                                   color=color, colormap=colormap,
                                   markers=markers, alpha=alpha, **kwargs)

    # Fit and transform the visualizer (calls draw)
    visualizer.fit(X, y, **kwargs)
    visualizer.transform(X)

    # Return the axes object on the visualizer
    return visualizer.ax

Response: Displays a bivariate scatter plot for one - off analysis.


Instruction:
Create a Python 3 function for
fitting the transformer and visualizer to the target or class values of the object.

Response:
def fit(self, X, y=None, **kwargs):
        """
        The fit method is the primary drawing input for the parallel coords
        visualization since it has both the X and y data required for the
        viz and the transform method does not.

        Parameters
        ----------
        X : ndarray or DataFrame of shape n x m
            A matrix of n instances with 2 features

        y : ndarray or Series of length n
            An array or series of target or class values

        kwargs : dict
            Pass generic arguments to the drawing method

        Returns
        -------
        self : instance
            Returns the instance of the transformer/visualizer
        """
        _, ncols = X.shape

        if ncols == 2:
            X_two_cols = X
            if self.features_ is None:
                self.features_ = ["Feature One", "Feature Two"]

        # Handle the feature names if they're None.
        elif self.features_ is not None and is_dataframe(X):
            X_two_cols = X[self.features_].as_matrix()

        # handle numpy named/ structured array
        elif self.features_ is not None and is_structured_array(X):
            X_selected = X[self.features_]
            X_two_cols = X_selected.copy().view((np.float64, len(X_selected.dtype.names)))

        # handle features that are numeric columns in ndarray matrix
        elif self.features_ is not None and has_ndarray_int_columns(self.features_, X):
            f_one, f_two = self.features_
            X_two_cols = X[:, [int(f_one), int(f_two)]]

        else:
            raise YellowbrickValueError("""
                ScatterVisualizer only accepts two features, please
                explicitly set these two features in the init kwargs or
                pass a matrix/ dataframe in with only two columns.""")

        # Store the classes for the legend if they're None.
        if self.classes_ is None:
            # TODO: Is this the most efficient method?
            self.classes_ = [str(label) for label in np.unique(y)]

        # Draw the instances
        self.draw(X_two_cols, y, **kwargs)

        # Fit always returns self.
        return self


Instruction:
Write a Python 3 function that can
call from the fit method creates a scatter plot that draws each instance of the class or target colored point whose location is determined by the feature data set. This method creates a scatter plot that draws each instance of the class or target colored point whose location is determined by the feature data set.

Response:
def draw(self, X, y, **kwargs):
        """Called from the fit method, this method creates a scatter plot that
        draws each instance as a class or target colored point, whose location
        is determined by the feature data set.
        """
        # Set the axes limits
        self.ax.set_xlim([-1,1])
        self.ax.set_ylim([-1,1])

        # set the colors
        color_values = resolve_colors(
            n_colors=len(self.classes_),
            colormap=self.colormap,
            colors=self.color
        )

        colors = dict(zip(self.classes_, color_values))

        # Create a data structure to hold the scatter plot representations
        to_plot = {}
        for kls in self.classes_:
            to_plot[kls] = [[], []]

        # Add each row of the data set to to_plot for plotting
        # TODO: make this an independent function for override
        for i, row in enumerate(X):
            row_ = np.repeat(np.expand_dims(row, axis=1), 2, axis=1)
            x_, y_ = row_[0], row_[1]
            kls = self.classes_[y[i]]

            to_plot[kls][0].append(x_)
            to_plot[kls][1].append(y_)

        # Add the scatter plots from the to_plot function
        # TODO: store these plots to add more instances to later
        # TODO: make this a separate function
        for i, kls in enumerate(self.classes_):
            self.ax.scatter(
                to_plot[kls][0],
                to_plot[kls][1],
                marker=next(self.markers),
                color=colors[kls],
                label=str(kls),
                alpha=self.alpha,
                **kwargs)

        self.ax.axis('equal')


Instruction:
Can you tell what is the following Python 3 function doing
def finalize(self, **kwargs):
        # Divide out the two features
        feature_one, feature_two = self.features_

        # Set the title
        self.set_title('Scatter Plot: {0} vs {1}'.format(
            str(feature_one), str(feature_two)))
        # Add the legend
        self.ax.legend(loc='best')
        self.ax.set_xlabel(str(feature_one))
        self.ax.set_ylabel(str(feature_two))

Response: Executes any subclass - specific axes finalization steps.


Instruction:
Here you have a function in Python 3, explain what it does
def gridsearch_color_plot(model, x_param, y_param, X=None, y=None, ax=None,
                          **kwargs):
    """Quick method:
    Create a color plot showing the best grid search scores across two
    parameters.

    This helper function is a quick wrapper to utilize GridSearchColorPlot
    for one-off analysis.

    If no `X` data is passed, the model is assumed to be fit already. This
    allows quick exploration without waiting for the grid search to re-run.

    Parameters
    ----------
    model : Scikit-Learn grid search object
        Should be an instance of GridSearchCV. If not, an exception is raised.
        The model may be fit or unfit.

    x_param : string
        The name of the parameter to be visualized on the horizontal axis.

    y_param : string
        The name of the parameter to be visualized on the vertical axis.

    metric : string (default 'mean_test_score')
        The field from the grid search's `cv_results` that we want to display.

    X  : ndarray or DataFrame of shape n x m or None (default None)
        A matrix of n instances with m features. If not None, forces the
        GridSearchCV object to be fit.

    y  : ndarray or Series of length n or None (default None)
        An array or series of target or class values.

    ax : matplotlib axes
        The axes to plot the figure on.

    classes : list of strings
        The names of the classes in the target

    Returns
    -------
    ax : matplotlib axes
        Returns the axes that the classification report was drawn on.
    """
    # Instantiate the visualizer
    visualizer = GridSearchColorPlot(model, x_param, y_param, ax=ax, **kwargs)

    # Fit if necessary
    if X is not None:
        visualizer.fit(X, y)
    else:
        visualizer.draw()

    # Return the axes object on the visualizer
    return visualizer.ax

Response: Create a color plot showing the best grid search scores across two parameters.


Instruction:
Can you generate the documentation for the following Python 3 function
def filter_missing(X, y=None):
    if y is not None:
        return filter_missing_X_and_y(X, y)
    else:
        return X[~np.isnan(X).any(axis=1)]

Response: This function filters out any missing values in data and returns a tuple of the nans in the order they appear in X.


Instruction:
Explain what the following Python 3 code does
def filter_missing_X_and_y(X, y):
    y_nans = np.isnan(y)
    x_nans = np.isnan(X).any(axis=1)
    unioned_nans = np.logical_or(x_nans, y_nans)

    return X[~unioned_nans], y[~unioned_nans]

Response: Remove rows from X and y where either contains nans.


Instruction:
Given the following Python 3 function, write the documentation
def warn_if_nans_exist(X):
    null_count = count_rows_with_nans(X)
    total = len(X)
    percent = 100 * null_count / total

    if null_count > 0:
        warning_message = \
            'Warning! Found {} rows of {} ({:0.2f}%) with nan values. Only ' \
            'complete rows will be plotted.'.format(null_count, total, percent)
        warnings.warn(warning_message, DataWarning)

Response: Warn if nans exist in a numpy array.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def count_rows_with_nans(X):
    if X.ndim == 2:
        return np.where(np.isnan(X).sum(axis=1) != 0, 1, 0).sum()

Response: Count the number of rows in 2D arrays that contain any nan values.


Instruction:
Explain what the following Python 3 code does
def fit(self, X, y):

        # note that GLM takes endog (y) and then exog (X):
        # this is the reverse of sklearn's methods
        self.glm_model = self.glm_partial(y, X)
        self.glm_results = self.glm_model.fit()
        return self

Response: Fit the GLM model on the data X and y.


Instruction:
Make a summary of the following Python 3 code
def missing_dispersion(X, y=None, ax=None, classes=None, alpha=0.5, marker="|", **kwargs):
    # Instantiate the visualizer
    visualizer = MissingValuesDispersion(
        ax=ax, classes=classes, alpha=alpha, marker=marker, **kwargs
    )

    # Fit and transform the visualizer (calls draw)
    visualizer.fit(X, y)
    visualizer.poof()

    # Return the axes object on the visualizer
    return visualizer.ax

Response: Creates a Missing Values Dispersion visualizer that shows the locations of missing values in the feature dataset.


Instruction:
Explain what the following Python 3 code does
def get_nan_locs(self, **kwargs):
        if np.issubdtype(self.X.dtype, np.string_) or np.issubdtype(self.X.dtype, np.unicode_):
            mask = np.where( self.X == '' )
            nan_matrix = np.zeros(self.X.shape)
            nan_matrix[mask] = np.nan

        else:
            nan_matrix = self.X.astype(float)

        if self.y is None:
            return np.argwhere(np.isnan(nan_matrix))
        else:
            nan_locs = []
            for target_value in np.unique(self.y):
                indices = np.argwhere(self.y == target_value)
                target_matrix = nan_matrix[indices.flatten()]
                nan_target_locs = np.argwhere(np.isnan(target_matrix))
                nan_locs.append((target_value, nan_target_locs))

            return nan_locs

Response:
Gets the locations of nans in feature data and returns
            the coordinates in the matrix


Instruction:
Write a Python 3 function for
calling from the fit method this method creates a scatter plot that draws each instance of a class or target colored point whose location is determined by the feature data set.

Response:
def draw(self, X, y, **kwargs):
        """Called from the fit method, this method creates a scatter plot that
        draws each instance as a class or target colored point, whose location
        is determined by the feature data set.

        If y is not None, then it draws a scatter plot where each class is in a
        different color.
        """
        nan_locs = self.get_nan_locs()
        if y is None:
            x_, y_ = list(zip(*nan_locs))
            self.ax.scatter(x_, y_, alpha=self.alpha, marker=self.marker, label=None)
        else:
            self.draw_multi_dispersion_chart(nan_locs)


Instruction:
Can you create a Python 3 function that
draws a multi dimensional dispersion chart for each color corresponds to a different target variable.

Response:
def draw_multi_dispersion_chart(self, nan_locs):
        """Draws a multi dimensional dispersion chart, each color corresponds
        to a different target variable.
        """
        for index, nan_values in enumerate(nan_locs):
            label, nan_locations = nan_values

            # if features passed in then, label as such
            if self.classes_ is not None:
                label = self.classes_[index]

            color = self.colors[index]

            x_, y_ = list(zip(*nan_locations))
            self.ax.scatter(x_, y_, alpha=self.alpha, marker=self.marker, color=color, label=label)


Instruction:
Write a Python 3 function that can
project the grid search results onto 2 dimensions. The display value is taken as the max over the non-displayed dimensions. Parameters ---------- cv_results : dict A dictionary of results from the `GridSearchCV` object's `cv_results_` attribute. x_param : string The name of the parameter to be visualized on the horizontal axis. y_param : string The name of the parameter to be visualized on the vertical axis. metric : string (default 'mean_test_score') The field from the grid search's `cv_results` that we want to display. Returns ------- unique_x_vals : list The parameter values that will be used to label the x axis. unique_y_vals: list The parameter values that will be used to label the y axis. best_scores: 2D numpy array (n_y by n_x) Array of scores to be displayed for each parameter value pair.

Response:
def param_projection(cv_results, x_param, y_param, metric='mean_test_score'):
    """
    Projects the grid search results onto 2 dimensions.

    The display value is taken as the max over the non-displayed dimensions.

    Parameters
    ----------
    cv_results : dict
        A dictionary of results from the `GridSearchCV` object's `cv_results_`
        attribute.

    x_param : string
        The name of the parameter to be visualized on the horizontal axis.

    y_param : string
        The name of the parameter to be visualized on the vertical axis.

    metric : string (default 'mean_test_score')
        The field from the grid search's `cv_results` that we want to display.

    Returns
    -------
    unique_x_vals : list
        The parameter values that will be used to label the x axis.

    unique_y_vals: list
        The parameter values that will be used to label the y axis.

    best_scores: 2D numpy array (n_y by n_x)
        Array of scores to be displayed for each parameter value pair.
    """
    # Extract the parameter values and score corresponding to each gridsearch
    # trial.
    # These are masked arrays where the cases where each parameter is
    # non-applicable are masked.
    try:
        x_vals = cv_results['param_' + x_param]
    except KeyError:
        raise YellowbrickKeyError("Parameter '{}' does not exist in the grid "
                                  "search results".format(x_param))
    try:
        y_vals = cv_results['param_' + y_param]
    except KeyError:
        raise YellowbrickKeyError("Parameter '{}' does not exist in the grid "
                                  "search results".format(y_param))

    if metric not in cv_results:
        raise YellowbrickKeyError("Metric '{}' does not exist in the grid "
                                  "search results".format(metric))

    # Get unique, unmasked values of the two display parameters
    unique_x_vals = sorted(list(set(x_vals.compressed())))
    unique_y_vals = sorted(list(set(y_vals.compressed())))
    n_x = len(unique_x_vals)
    n_y = len(unique_y_vals)

    # Get mapping of each parameter value -> an integer index
    int_mapping_1 = {value: idx for idx, value in enumerate(unique_x_vals)}
    int_mapping_2 = {value: idx for idx, value in enumerate(unique_y_vals)}

    # Translate each gridsearch result to indices on the grid
    idx_x = [int_mapping_1[value] if value else None for value in x_vals]
    idx_y = [int_mapping_2[value] if value else None for value in y_vals]

    # Create an array of all scores for each value of the display parameters.
    # This is a n_x by n_y array of lists with `None` in place of empties
    # (my kingdom for a dataframe...)
    all_scores = [[None for _ in range(n_x)] for _ in range(n_y)]
    for x, y, score in zip(idx_x, idx_y, cv_results[metric]):
        if x is not None and y is not None:
            if all_scores[y][x] is None:
                all_scores[y][x] = []
            all_scores[y][x].append(score)

    # Get a numpy array consisting of the best scores for each parameter pair
    best_scores = np.empty((n_y, n_x))
    for x in range(n_x):
        for y in range(n_y):
            if all_scores[y][x] is None:
                best_scores[y, x] = np.nan
            else:
                try:
                    best_scores[y, x] = max(all_scores[y][x])
                except ValueError:
                    raise YellowbrickValueError(
                        "Cannot display grid search results for metric '{}': "
                        "result values may not all be numeric".format(metric)
                    )

    return unique_x_vals, unique_y_vals, best_scores


Instruction:
Here you have a function in Python 3, explain what it does
def param_projection(self, x_param, y_param, metric):
        return param_projection(self.estimator.cv_results_, x_param, y_param, metric)

Response: This method projects the grid search results onto 2 dimensions and returns a 2D numpy array that contains the max over the non - displayed dimensions.


Instruction:
Given the following Python 3 function, write the documentation
def __threshold(self, ymx_i):
        return ymx_i - (self.S * np.diff(self.xsn).mean())

Response: Calculates the difference threshold for a specific local maximum.


Instruction:
Implement a Python 3 function for
finding and returns the knee or elbow value normalized knee or None.

Response:
def find_knee(self, ):
        """
        Finds and returns the "knee"or "elbow" value, the normalized knee
        value, and the x value where the knee is located.

        """
        if not self.xmx_idx.size:
            warning_message = \
                'No "knee" or "elbow point" detected ' \
                'This could be due to bad clustering, no '\
                'actual clusters being formed etc.'
            warnings.warn(warning_message,YellowbrickWarning)
            return None, None, None

        mxmx_iter = np.arange(self.xmx_idx[0], len(self.xsn))
        xmx_idx_iter = np.append(self.xmx_idx, len(self.xsn))

        knee_, norm_knee_, knee_x = 0.0, 0.0, None
        for mxmx_i, mxmx in enumerate(xmx_idx_iter):
            # stopping criteria for exhasuting array
            if mxmx_i == len(xmx_idx_iter) - 1:
                break
            # indices between maxima/minima
            idxs = (mxmx_iter > xmx_idx_iter[mxmx_i]) * \
                (mxmx_iter < xmx_idx_iter[mxmx_i + 1])
            between_local_mx = mxmx_iter[np.where(idxs)]

            for j in between_local_mx:
                if j in self.xmn_idx:
                    # reached a minima, x indices are unique
                    # only need to check if j is a min
                    if self.yd[j + 1] > self.yd[j]:
                        self.Tmx[mxmx_i] = 0
                        knee_x = None  # reset x where yd crossed Tmx
                    elif self.yd[j + 1] <= self.yd[j]:
                        warning_message="If this is a minima, " \
                                        "how would you ever get here."
                        warnings.warn(warning_message, YellowbrickWarning)
                if self.yd[j] < self.Tmx[mxmx_i] or self.Tmx[mxmx_i] < 0:
                    # declare a knee
                    if not knee_x:
                        knee_x = j
                    knee_ = self.x[self.xmx_idx[mxmx_i]]
                    norm_knee_ = self.xsn[self.xmx_idx[mxmx_i]]
        return knee_, norm_knee_, knee_x


Instruction:
Create a Python 3 function for
plotting the normalized curve the distance curve and the knee

Response:
def plot_knee_normalized(self, ):
        """
        Plots the normalized curve, the distance curve (xd, ysn) and the
        knee, if it exists.

        """
        import matplotlib.pyplot as plt

        plt.figure(figsize=(8, 8))
        plt.plot(self.xsn, self.ysn)
        plt.plot(self.xd, self.yd, 'r')
        plt.xticks(np.arange(min(self.xsn), max(self.xsn) + 0.1, 0.1))
        plt.yticks(np.arange(min(self.xd), max(self.ysn) + 0.1, 0.1))

        plt.vlines(self.norm_knee, plt.ylim()[0], plt.ylim()[1])


Instruction:
How would you implement a function in Python 3 that
plots the curve and the knee if it exists

Response:
def plot_knee(self, ):
        """
        Plot the curve and the knee, if it exists

        """
        import matplotlib.pyplot as plt

        plt.figure(figsize=(8, 8))
        plt.plot(self.x, self.y)
        plt.vlines(self.knee, plt.ylim()[0], plt.ylim()[1])


Instruction:
Explain what the following Python 3 code does
def fit(self, X, y, **kwargs):
        self.estimator.fit(X, y, **kwargs)
        self.draw()
        return self

Response: A simple pass - through method that draws the alpha - error plot.


Instruction:
Explain what the following Python 3 code does
def draw(self):
        # Search for the correct parameters on the estimator.
        alphas = self._find_alphas_param()
        errors = self._find_errors_param()

        alpha = self.estimator.alpha_ # Get decision from the estimator
        name = self.name[:-2].lower() # Remove the CV from the label

        # Plot the alpha against the error
        self.ax.plot(alphas, errors, label=name)

        # Draw a dashed vline at the alpha
        label = "$\\alpha={:0.3f}$".format(alpha)
        self.ax.axvline(alpha, color='k', linestyle='dashed', label=label)

        return self.ax

Response: Draws the alpha plot based on the values on the estimator.


Instruction:
How would you implement a function in Python 3 that
searches for the parameter on the estimator that contains the array of alphas that was used to produce the error selection.

Response:
def _find_alphas_param(self):
        """
        Searches for the parameter on the estimator that contains the array of
        alphas that was used to produce the error selection. If it cannot find
        the parameter then a YellowbrickValueError is raised.
        """

        # NOTE: The order of the search is very important!
        for attr in ("cv_alphas_", "alphas_", "alphas",):
            try:
                return getattr(self.estimator, attr)
            except AttributeError:
                continue

        raise YellowbrickValueError(
            "could not find alphas param on {} estimator".format(
                self.estimator.__class__.__name__
            )
        )


Instruction:
Create a Python 3 function for
searching for the parameter that contains the array of errors that was used to determine the optimal alpha.

Response:
def _find_errors_param(self):
        """
        Searches for the parameter on the estimator that contains the array of
        errors that was used to determine the optimal alpha. If it cannot find
        the parameter then a YellowbrickValueError is raised.
        """

        # NOTE: The order of the search is very important!
        if hasattr(self.estimator, 'mse_path_'):
            return self.estimator.mse_path_.mean(1)

        if hasattr(self.estimator, 'cv_values_'):
            return self.estimator.cv_values_.mean(0)

        raise YellowbrickValueError(
            "could not find errors param on {} estimator".format(
                self.estimator.__class__.__name__
            )
        )


Instruction:
Can you implement a function in Python 3 that
draws the alphas values against their associated error in a similar similar fashion to the AlphaSelection visualizer.

Response:
def draw(self):
        """
        Draws the alphas values against their associated error in a similar
        fashion to the AlphaSelection visualizer.
        """
        # Plot the alpha against the error
        self.ax.plot(self.alphas, self.errors, label=self.name.lower())

        # Draw a dashed vline at the alpha with maximal error
        alpha = self.alphas[np.where(self.errors == self.errors.max())][0]
        label = "$\\alpha_{{max}}={:0.3f}$".format(alpha)
        self.ax.axvline(alpha, color='k', linestyle='dashed', label=label)

        # Draw a dashed vline at the alpha with minimal error
        alpha = self.alphas[np.where(self.errors == self.errors.min())][0]
        label = "$\\alpha_{{min}}={:0.3f}$".format(alpha)
        self.ax.axvline(alpha, color='k', linestyle='dashed', label=label)

        return self.ax


Instruction:
Given the following Python 3 function, write the documentation
def postag(
    X,
    ax=None,
    tagset="penn_treebank",
    colormap=None,
    colors=None,
    frequency=False,
    **kwargs
):
    """
    Display a barchart with the counts of different parts of speech
    in X, which consists of a part-of-speech-tagged corpus, which the
    visualizer expects to be a list of lists of lists of (token, tag)
    tuples.

    Parameters
    ----------
    X : list or generator
        Should be provided as a list of documents or a generator
        that yields a list of documents that contain a list of
        sentences that contain (token, tag) tuples.
    ax : matplotlib axes
        The axes to plot the figure on.
    tagset: string
        The tagset that was used to perform part-of-speech tagging.
        Either "penn_treebank" or "universal", defaults to "penn_treebank".
        Use "universal" if corpus has been tagged using SpaCy.
    colors : list or tuple of colors
        Specify the colors for each individual part-of-speech.
    colormap : string or matplotlib cmap
        Specify a colormap to color the parts-of-speech.
    frequency: bool {True, False}, default: False
        If set to True, part-of-speech tags will be plotted according to frequency,
        from most to least frequent.
    kwargs : dict
        Pass any additional keyword arguments to the PosTagVisualizer.

    Returns
    -------
    ax : matplotlib axes
        Returns the axes on which the PosTagVisualizer was drawn.
    """
    # Instantiate the visualizer
    visualizer = PosTagVisualizer(
        ax=ax, tagset=tagset, colors=colors, colormap=colormap,
        frequency=frequency, **kwargs
    )

    # Fit and transform the visualizer (calls draw)
    visualizer.fit(X, **kwargs)

    # Return the axes object on the visualizer
    return visualizer

Response:
Display a barchart with the counts of different parts of speech
    in X.


Instruction:
Write a Python 3 script to
fit the corpus to the appropriate tag map.

Response:
def fit(self, X, y=None, **kwargs):
        """
        Fits the corpus to the appropriate tag map.
        Text documents must be tokenized & tagged before passing to fit.

        Parameters
        ----------
        X : list or generator
            Should be provided as a list of documents or a generator
            that yields a list of documents that contain a list of
            sentences that contain (token, tag) tuples.

        y : ndarray or Series of length n
            An optional array of target values that are ignored by the
            visualizer.

        kwargs : dict
            Pass generic arguments to the drawing method

        Returns
        -------
        self : instance
            Returns the instance of the transformer/visualizer
        """
        # TODO: add support for other tagsets?
        if self.tagset == "penn_treebank":
            self.pos_tag_counts_ = self._penn_tag_map()
            self._handle_treebank(X)

        elif self.tagset == "universal":
            self.pos_tag_counts_ = self._uni_tag_map()
            self._handle_universal(X)

        self.draw()
        return self


Instruction:
Here you have a function in Python 3, explain what it does
def _handle_universal(self, X):
        jump = {
            # combine proper and regular nouns
            "NOUN": "noun", "PROPN": "noun",
            "ADJ": "adjective",
            "VERB": "verb",
            # include particles with adverbs
            "ADV": "adverb", "PART": "adverb",
            "ADP": "adposition",
            "PRON": "pronoun",
            "CCONJ": "conjunction",
            "PUNCT": "punctuation",
            "DET": "determiner",
            "NUM": "number",
            "INTJ": "interjection",
            "SYM": "symbol",
        }

        for tagged_doc in X:
            for tagged_sent in tagged_doc:
                for _, tag in tagged_sent:
                    if tag == "SPACE":
                        continue
                    self.pos_tag_counts_[jump.get(tag, "other")] += 1

Response: Handles Universal related information.


Instruction:
Explain what the following Python 3 code does
def _handle_treebank(self, X):
        for tagged_doc in X:
            for tagged_sent in tagged_doc:
                for _, tag in tagged_sent:
                    if tag.startswith("N"):
                        self.pos_tag_counts_["noun"] += 1
                    elif tag.startswith("J"):
                        self.pos_tag_counts_["adjective"] += 1
                    elif tag.startswith("V"):
                        self.pos_tag_counts_["verb"] += 1
                    # include particles with adverbs
                    elif tag.startswith("RB") or tag == "RP":
                        self.pos_tag_counts_["adverb"] += 1
                    elif tag.startswith("PR"):
                        self.pos_tag_counts_["pronoun"] += 1
                    elif tag.startswith("W"):
                        self.pos_tag_counts_["wh- word"] += 1
                    elif tag == "CC":
                        self.pos_tag_counts_["conjunction"] += 1
                    elif tag == "CD":
                        self.pos_tag_counts_["digit"] += 1
                    # combine predeterminer and determiner
                    elif tag in ["DT" or "PDT"]:
                        self.pos_tag_counts_["determiner"] += 1
                    elif tag == "EX":
                        self.pos_tag_counts_["existential"] += 1
                    elif tag == "FW":
                        self.pos_tag_counts_["non-English"] += 1
                    elif tag == "IN":
                        self.pos_tag_counts_["preposition"] += 1
                    elif tag == "POS":
                        self.pos_tag_counts_["possessive"] += 1
                    elif tag == "LS":
                        self.pos_tag_counts_["list"] += 1
                    elif tag == "MD":
                        self.pos_tag_counts_["modal"] += 1
                    elif tag in self.punct_tags:
                        self.pos_tag_counts_["punctuation"] += 1
                    elif tag == "TO":
                        self.pos_tag_counts_["infinitive"] += 1
                    elif tag == "UH":
                        self.pos_tag_counts_["interjection"] += 1
                    elif tag == "SYM":
                        self.pos_tag_counts_["symbol"] += 1
                    else:
                        self.pos_tag_counts_["other"] += 1

Response: This function handles the Penn Treebank tags and returns a dictionary of the part - of - speech tag mapping.


Instruction:
Write a Python 3 script for
calling from the fit method, this method creates the canvas and draws the part-of-speech tag mapping as a bar chart. Parameters ---------- kwargs: dict generic keyword arguments. Returns ------- ax : matplotlib axes Axes on which the PosTagVisualizer was drawn.

Response:
def draw(self, **kwargs):
        """
        Called from the fit method, this method creates the canvas and
        draws the part-of-speech tag mapping as a bar chart.

        Parameters
        ----------
        kwargs: dict
            generic keyword arguments.

        Returns
        -------
        ax : matplotlib axes
            Axes on which the PosTagVisualizer was drawn.
        """
        colors = resolve_colors(
            n_colors=len(self.pos_tag_counts_),
            colormap=self.colormap,
            colors=self.colors,
        )

        if self.frequency:
            # Sort tags with respect to frequency in corpus
            sorted_tags = sorted(
                self.pos_tag_counts_, key=self.pos_tag_counts_.get, reverse=True
            )
            sorted_counts = [self.pos_tag_counts_[tag] for tag in sorted_tags]

            self.ax.bar(range(len(sorted_tags)), sorted_counts, color=colors)
            return self.ax

        self.ax.bar(
            range(len(self.pos_tag_counts_)),
            list(self.pos_tag_counts_.values()),
            color=colors,
        )
        return self.ax


Instruction:
Can you generate a brief explanation for the following Python 3 code
def finalize(self, **kwargs):
        # NOTE: not deduping here, so this is total, not unique
        self.set_title(
            "PosTag plot for {}-token corpus".format(
                (sum(self.pos_tag_counts_.values()))
            )
        )

        self.ax.set_xticks(range(len(self.pos_tag_counts_)))
        self.ax.set_xticklabels(list(self.pos_tag_counts_.keys()), rotation=90)

        # Set the axis labels
        if self.frequency:
            self.ax.set_xlabel(
                "{} part-of-speech tags, sorted by frequency".format(self.tagset_names[self.tagset])
            )
        else:
            self.ax.set_xlabel(
                "{} part-of-speech tags".format(self.tagset_names[self.tagset])
            )
        self.ax.set_ylabel("Count")

Response: Finalize the plot with ticks labels and title


Instruction:
Make a summary of the following Python 3 code
def tsne(X, y=None, ax=None, decompose='svd', decompose_by=50, classes=None,
           colors=None, colormap=None, alpha=0.7, **kwargs):
    """
    Display a projection of a vectorized corpus in two dimensions using TSNE,
    a nonlinear dimensionality reduction method that is particularly well
    suited to embedding in two or three dimensions for visualization as a
    scatter plot. TSNE is widely used in text analysis to show clusters or
    groups of documents or utterances and their relative proximities.

    Parameters
    ----------

    X : ndarray or DataFrame of shape n x m
        A matrix of n instances with m features representing the corpus of
        vectorized documents to visualize with tsne.

    y : ndarray or Series of length n
        An optional array or series of target or class values for instances.
        If this is specified, then the points will be colored according to
        their class. Often cluster labels are passed in to color the documents
        in cluster space, so this method is used both for classification and
        clustering methods.

    ax : matplotlib axes
        The axes to plot the figure on.

    decompose : string or None
        A preliminary decomposition is often used prior to TSNE to make the
        projection faster. Specify `"svd"` for sparse data or `"pca"` for
        dense data. If decompose is None, the original data set will be used.

    decompose_by : int
        Specify the number of components for preliminary decomposition, by
        default this is 50; the more components, the slower TSNE will be.

    classes : list of strings
        The names of the classes in the target, used to create a legend.

    colors : list or tuple of colors
        Specify the colors for each individual class

    colormap : string or matplotlib cmap
        Sequential colormap for continuous target

    alpha : float, default: 0.7
        Specify a transparency where 1 is completely opaque and 0 is completely
        transparent. This property makes densely clustered points more visible.

    kwargs : dict
        Pass any additional keyword arguments to the TSNE transformer.

    Returns
    -------
    ax : matplotlib axes
        Returns the axes that the parallel coordinates were drawn on.
    """
    # Instantiate the visualizer
    visualizer = TSNEVisualizer(
        ax, decompose, decompose_by, classes, colors, colormap, alpha, **kwargs
    )

    # Fit and transform the visualizer (calls draw)
    visualizer.fit(X, y, **kwargs)
    visualizer.transform(X)

    # Return the axes object on the visualizer
    return visualizer.ax

Response: This function is used to plot a vectorized corpus using TSNE.


Instruction:
Write a Python 3 function that can
create a pipeline that will project the data set into the internal 2D space using TSNE.

Response:
def make_transformer(self, decompose='svd', decompose_by=50, tsne_kwargs={}):
        """
        Creates an internal transformer pipeline to project the data set into
        2D space using TSNE, applying an pre-decomposition technique ahead of
        embedding if necessary. This method will reset the transformer on the
        class, and can be used to explore different decompositions.

        Parameters
        ----------

        decompose : string or None, default: ``'svd'``
            A preliminary decomposition is often used prior to TSNE to make
            the projection faster. Specify ``"svd"`` for sparse data or ``"pca"``
            for dense data. If decompose is None, the original data set will
            be used.

        decompose_by : int, default: 50
            Specify the number of components for preliminary decomposition, by
            default this is 50; the more components, the slower TSNE will be.

        Returns
        -------

        transformer : Pipeline
            Pipelined transformer for TSNE projections
        """

        # TODO: detect decompose by inferring from sparse matrix or dense or
        # If number of features > 50 etc.
        decompositions = {
            'svd': TruncatedSVD,
            'pca': PCA,
        }

        if decompose and decompose.lower() not in decompositions:
            raise YellowbrickValueError(
                "'{}' is not a valid decomposition, use {}, or None".format(
                    decompose, ", ".join(decompositions.keys())
                )
            )

        # Create the pipeline steps
        steps = []

        # Add the pre-decomposition
        if decompose:
            klass = decompositions[decompose]
            steps.append((decompose, klass(
                n_components=decompose_by, random_state=self.random_state)))

        # Add the TSNE manifold
        steps.append(('tsne', TSNE(
            n_components=2, random_state=self.random_state, **tsne_kwargs)))

        # return the pipeline
        return Pipeline(steps)


Instruction:
Can you tell what is the following Python 3 function doing
def finalize(self, **kwargs):
        self.set_title(
            "TSNE Projection of {} Documents".format(self.n_instances_)
        )

        # Remove the ticks
        self.ax.set_yticks([])
        self.ax.set_xticks([])

        # Add the legend outside of the figure box.
        if not all(self.classes_ == np.array([self.NULL_CLASS])):
            box = self.ax.get_position()
            self.ax.set_position([box.x0, box.y0, box.width * 0.8, box.height])
            manual_legend(
                self, self.classes_, self.color_values_,
                loc='center left', bbox_to_anchor=(1, 0.5)
            )

Response: Finalize the drawing by removing ticks and axes objects that do not convey information about TNSE.


Instruction:
Can you implement a function in Python 3 that
reads the VERSION_PATH and returns the version string.

Response:
def get_version(path=VERSION_PATH):
    """
    Reads the python file defined in the VERSION_PATH to find the get_version
    function, and executes it to ensure that it is loaded correctly. Separating
    the version in this way ensures no additional code is executed.
    """
    namespace = {}
    exec(read(path), namespace)
    return namespace['get_version'](short=True)


Instruction:
Write a Python 3 function for
yielding a generator of requirements as defined by the REQUIRE_PATH which is the path to the requirements. txt file.

Response:
def get_requires(path=REQUIRE_PATH):
    """
    Yields a generator of requirements as defined by the REQUIRE_PATH which
    should point to a requirements.txt output by `pip freeze`.
    """
    for line in read(path).splitlines():
        line = line.strip()
        if line and not line.startswith('#'):
            yield line


Instruction:
Here you have a function in Python 3, explain what it does
def get_description_type(path=PKG_DESCRIBE):
    _, ext = os.path.splitext(path)
    return {
        ".rst": "text/x-rst",
        ".txt": "text/plain",
        ".md": "text/markdown",
    }[ext]

Response: Returns the long_description_content_type based on the extension of the passed in path.


Instruction:
Can you implement a function in Python 3 that
displays a learning curve based on number of samples vs training and cross validation scores. The learning curve aims to show how a model learns and improves with experience. This helper function is a quick wrapper to utilize the LearningCurve for one-off analysis. Parameters ---------- model : a scikit-learn estimator An object that implements ``fit`` and ``predict``, can be a classifier, regressor, or clusterer so long as there is also a valid associated scoring metric. Note that the object is cloned for each validation. X : array-like, shape (n_samples, n_features) Training vector, where n_samples is the number of samples and n_features is the number of features. y : array-like, shape (n_samples) or (n_samples, n_features), optional Target relative to X for classification or regression; None for unsupervised learning. ax : matplotlib.Axes object, optional The axes object to plot the figure on. groups : array-like, with shape (n_samples,) Optional group labels for the samples used while splitting the dataset into train/test sets. train_sizes : array-like, shape (n_ticks,) default: ``np.linspace(0.1,1.0,5)`` Relative or absolute numbers of training examples that will be used to generate the learning curve. If the dtype is float, it is regarded as a fraction of the maximum size of the training set, otherwise it is interpreted as absolute sizes of the training sets. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 3-fold cross-validation, - integer, to specify the number of folds. - An object to be used as a cross-validation generator. - An iterable yielding train/test splits. see the scikit-learn `cross-validation guide <http://scikit-learn.org/stable/modules/cross_validation.html>`_ for more information on the possible strategies that can be used here. scoring : string, callable or None, optional, default: None A string or scorer callable object / function with signature ``scorer(estimator, X, y)``. See scikit-learn model evaluation documentation for names of possible metrics. exploit_incremental_learning : boolean, default: False If the estimator supports incremental learning, this will be used to speed up fitting for different training set sizes. n_jobs : integer, optional Number of jobs to run in parallel (default 1). pre_dispatch : integer or string, optional Number of predispatched jobs for parallel execution (default is all). The option can reduce the allocated memory. The string can be an expression like '2*n_jobs'. shuffle : boolean, optional Whether to shuffle training data before taking prefixes of it based on``train_sizes``. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``shuffle`` is True. kwargs : dict Keyword arguments that are passed to the base class and may influence the visualization as defined in other Visualizers. These arguments are also passed to the `poof()` method, e.g. can pass a path to save the figure to. Returns ------- ax : matplotlib axes Returns the axes that the learning curve were drawn on.

Response:
def learning_curve(model, X, y, ax=None, groups=None,
               train_sizes=DEFAULT_TRAIN_SIZES, cv=None, scoring=None,
               exploit_incremental_learning=False, n_jobs=1,
               pre_dispatch="all", shuffle=False, random_state=None,
               **kwargs):
    """
    Displays a learning curve based on number of samples vs training and
    cross validation scores. The learning curve aims to show how a model
    learns and improves with experience.

    This helper function is a quick wrapper to utilize the LearningCurve
    for one-off analysis.

    Parameters
    ----------
    model : a scikit-learn estimator
        An object that implements ``fit`` and ``predict``, can be a
        classifier, regressor, or clusterer so long as there is also a valid
        associated scoring metric.

        Note that the object is cloned for each validation.

    X : array-like, shape (n_samples, n_features)
        Training vector, where n_samples is the number of samples and
        n_features is the number of features.

    y : array-like, shape (n_samples) or (n_samples, n_features), optional
        Target relative to X for classification or regression;
        None for unsupervised learning.

    ax : matplotlib.Axes object, optional
        The axes object to plot the figure on.

    groups : array-like, with shape (n_samples,)
        Optional group labels for the samples used while splitting the dataset
        into train/test sets.

    train_sizes : array-like, shape (n_ticks,)
        default: ``np.linspace(0.1,1.0,5)``

        Relative or absolute numbers of training examples that will be used to
        generate the learning curve. If the dtype is float, it is regarded as
        a fraction of the maximum size of the training set, otherwise it is
        interpreted as absolute sizes of the training sets.

    cv : int, cross-validation generator or an iterable, optional
        Determines the cross-validation splitting strategy.
        Possible inputs for cv are:

          - None, to use the default 3-fold cross-validation,
          - integer, to specify the number of folds.
          - An object to be used as a cross-validation generator.
          - An iterable yielding train/test splits.

        see the scikit-learn
        `cross-validation guide <http://scikit-learn.org/stable/modules/cross_validation.html>`_
        for more information on the possible strategies that can be used here.

    scoring : string, callable or None, optional, default: None
        A string or scorer callable object / function with signature
        ``scorer(estimator, X, y)``. See scikit-learn model evaluation
        documentation for names of possible metrics.

    exploit_incremental_learning : boolean, default: False
        If the estimator supports incremental learning, this will be used to
        speed up fitting for different training set sizes.

    n_jobs : integer, optional
        Number of jobs to run in parallel (default 1).

    pre_dispatch : integer or string, optional
        Number of predispatched jobs for parallel execution (default is
        all). The option can reduce the allocated memory. The string can
        be an expression like '2*n_jobs'.

    shuffle : boolean, optional
        Whether to shuffle training data before taking prefixes of it
        based on``train_sizes``.

    random_state : int, RandomState instance or None, optional (default=None)
        If int, random_state is the seed used by the random number generator;
        If RandomState instance, random_state is the random number generator;
        If None, the random number generator is the RandomState instance used
        by `np.random`. Used when ``shuffle`` is True.

    kwargs : dict
        Keyword arguments that are passed to the base class and may influence
        the visualization as defined in other Visualizers. These arguments are
        also passed to the `poof()` method, e.g. can pass a path to save the
        figure to.

    Returns
    -------
    ax : matplotlib axes
        Returns the axes that the learning curve were drawn on.
    """
    # Initialize the visualizer
    oz = LearningCurve(
        model, ax=ax, groups=groups, train_sizes=train_sizes, cv=cv,
        scoring=scoring, n_jobs=n_jobs, pre_dispatch=pre_dispatch,
        shuffle=shuffle, random_state=random_state,
        exploit_incremental_learning=exploit_incremental_learning,
    )

    # Fit and poof the visualizer
    oz.fit(X, y)
    oz.poof(**kwargs)
    return oz.ax


Instruction:
Can you tell what is the following Python 3 function doing
def fit(self, X, y=None):
        # arguments to pass to sk_learning_curve
        sklc_kwargs = {
            key: self.get_params()[key]
            for key in (
                'groups', 'train_sizes', 'cv', 'scoring',
                'exploit_incremental_learning', 'n_jobs',
                'pre_dispatch', 'shuffle', 'random_state',
            )
        }

        # compute the learning curve and store the scores on the estimator
        curve = sk_learning_curve(self.estimator, X, y, **sklc_kwargs)
        self.train_sizes_, self.train_scores_, self.test_scores_ = curve

        # compute the mean and standard deviation of the training data
        self.train_scores_mean_ = np.mean(self.train_scores_, axis=1)
        self.train_scores_std_ = np.std(self.train_scores_, axis=1)

        # compute the mean and standard deviation of the test data
        self.test_scores_mean_ = np.mean(self.test_scores_, axis=1)
        self.test_scores_std_ = np.std(self.test_scores_, axis=1)

        # draw the curves on the current axes
        self.draw()
        return self

Response: Fits the learning curve with the wrapped model to the specified data and saves the scores on the estimator.


Instruction:
Explain what the following Python 3 code does
def draw(self, **kwargs):
        # Specify the curves to draw and their labels
        labels = ("Training Score", "Cross Validation Score")
        curves = (
            (self.train_scores_mean_, self.train_scores_std_),
            (self.test_scores_mean_, self.test_scores_std_),
        )

        # Get the colors for the train and test curves
        colors = resolve_colors(n_colors=2)

        # Plot the fill betweens first so they are behind the curves.
        for idx, (mean, std) in enumerate(curves):
            # Plot one standard deviation above and below the mean
            self.ax.fill_between(
                self.train_sizes_, mean - std, mean+std, alpha=0.25,
                color=colors[idx],
            )

        # Plot the mean curves so they are in front of the variance fill
        for idx, (mean, _) in enumerate(curves):
            self.ax.plot(
                self.train_sizes_, mean, 'o-', color=colors[idx],
                label=labels[idx],
            )

        return self.ax

Response: Renders the training and test learning curves.