Instruction:
Explain what the following Python 3 code does
def module_path(self, filepath):
        possible_modbits = re.split('[\\/]', filepath.strip('\\/'))
        basename = possible_modbits[-1]
        prefixes = possible_modbits[0:-1]
        modpath = []
        discarded = []

        # find the first directory that has an __init__.py
        for i in range(len(prefixes)):
            path_args = ["/"]
            path_args.extend(prefixes[0:i+1])
            path_args.append('__init__.py')
            prefix_module = os.path.join(*path_args)
            #logger.debug("Checking prefix modulepath: {}".format(prefix_module))
            if os.path.isfile(prefix_module):
                #logger.debug("Found start of modulepath: {}".format(prefixes[i]))
                modpath = prefixes[i:]
                break

            else:
                discarded = path_args[0:-1]

        modpath.append(basename)

        # convert the remaining file path to a python module path that can be imported
        module_name = '.'.join(modpath)
        module_name = re.sub(r'(?:\.__init__)?\.py$', '', module_name, flags=re.I)
        logger.debug("Module path {} found in filepath {}".format(module_name, filepath))
        return module_name

Response:
given a filepath like base. path. to. module this will convert it to
        path. to. module so it can be imported


Instruction:
Write a Python 3 function for
removing paths in self. paths with confirmation.

Response:
def remove(self, auto_confirm=False):
        """Remove paths in ``self.paths`` with confirmation (unless
        ``auto_confirm`` is True)."""
        if not self._can_uninstall():
            return
        if not self.paths:
            logger.info(
                "Can't uninstall '%s'. No files were found to uninstall.",
                self.dist.project_name,
            )
            return
        logger.info(
            'Uninstalling %s-%s:',
            self.dist.project_name, self.dist.version
        )

        with indent_log():
            paths = sorted(self.compact(self.paths))

            if auto_confirm:
                response = 'y'
            else:
                for path in paths:
                    logger.info(path)
                response = ask('Proceed (y/n)? ', ('y', 'n'))
            if self._refuse:
                logger.info('Not removing or modifying (outside of prefix):')
                for path in self.compact(self._refuse):
                    logger.info(path)
            if response == 'y':
                self.save_dir = tempfile.mkdtemp(suffix='-uninstall',
                                                 prefix='pip-')
                for path in paths:
                    new_path = self._stash(path)
                    logger.debug('Removing file or directory %s', path)
                    self._moved_paths.append(path)
                    renames(path, new_path)
                for pth in self.pth.values():
                    pth.remove()
                logger.info(
                    'Successfully uninstalled %s-%s',
                    self.dist.project_name, self.dist.version
                )


Instruction:
How would you explain what the following Python 3 function does
def rollback(self):
        if self.save_dir is None:
            logger.error(
                "Can't roll back %s; was not uninstalled",
                self.dist.project_name,
            )
            return False
        logger.info('Rolling back uninstall of %s', self.dist.project_name)
        for path in self._moved_paths:
            tmp_path = self._stash(path)
            logger.debug('Replacing %s', path)
            renames(tmp_path, path)
        for pth in self.pth.values():
            pth.rollback()

Response: Rollback the changes made by remove.


Instruction:
How would you explain what the following Python 3 function does
def commit(self):
        if self.save_dir is not None:
            rmtree(self.save_dir)
            self.save_dir = None
            self._moved_paths = []

Response: Remove temporary save dir and rollback will no longer be possible.


Instruction:
Given the following Python 3 function, write the documentation
def _dump_arg_defaults(kwargs):
    if current_app:
        kwargs.setdefault('cls', current_app.json_encoder)
        if not current_app.config['JSON_AS_ASCII']:
            kwargs.setdefault('ensure_ascii', False)
        kwargs.setdefault('sort_keys', current_app.config['JSON_SORT_KEYS'])
    else:
        kwargs.setdefault('sort_keys', True)
        kwargs.setdefault('cls', JSONEncoder)

Response: Inject default arguments for dump functions.


Instruction:
Given the following Python 3 function, write the documentation
def _load_arg_defaults(kwargs):
    if current_app:
        kwargs.setdefault('cls', current_app.json_decoder)
    else:
        kwargs.setdefault('cls', JSONDecoder)

Response: Inject default arguments for load functions.


Instruction:
Can you write a function in Python 3 where it
serializes obj to a JSON formatted str by using the application s json_encoder.

Response:
def dumps(obj, **kwargs):
    """Serialize ``obj`` to a JSON formatted ``str`` by using the application's
    configured encoder (:attr:`~flask.Flask.json_encoder`) if there is an
    application on the stack.

    This function can return ``unicode`` strings or ascii-only bytestrings by
    default which coerce into unicode strings automatically.  That behavior by
    default is controlled by the ``JSON_AS_ASCII`` configuration variable
    and can be overriden by the simplejson ``ensure_ascii`` parameter.
    """
    _dump_arg_defaults(kwargs)
    encoding = kwargs.pop('encoding', None)
    rv = _json.dumps(obj, **kwargs)
    if encoding is not None and isinstance(rv, text_type):
        rv = rv.encode(encoding)
    return rv


Instruction:
Explain what the following Python 3 code does
def dump(obj, fp, **kwargs):
    _dump_arg_defaults(kwargs)
    encoding = kwargs.pop('encoding', None)
    if encoding is not None:
        fp = _wrap_writer_for_text(fp, encoding)
    _json.dump(obj, fp, **kwargs)

Response: Like dump but writes into a file object.


Instruction:
Given the following Python 3 function, write the documentation
def load(fp, **kwargs):
    _load_arg_defaults(kwargs)
    if not PY2:
        fp = _wrap_reader_for_text(fp, kwargs.pop('encoding', None) or 'utf-8')
    return _json.load(fp, **kwargs)

Response: Like load but reads from a file object.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def htmlsafe_dumps(obj, **kwargs):
    rv = dumps(obj, **kwargs) \
        .replace(u'<', u'\\u003c') \
        .replace(u'>', u'\\u003e') \
        .replace(u'&', u'\\u0026') \
        .replace(u"'", u'\\u0027')
    if not _slash_escape:
        rv = rv.replace('\\/', '/')
    return rv

Response: This function dumps is safe for use in HTML tags.


Instruction:
Write a Python 3 function that can
like htmlsafe_dumps but writes into a file object.

Response:
def htmlsafe_dump(obj, fp, **kwargs):
    """Like :func:`htmlsafe_dumps` but writes into a file object."""
    fp.write(unicode(htmlsafe_dumps(obj, **kwargs)))


Instruction:
Can you implement a function in Python 3 that
returns a Flask. Response object with the JSON representation of the given arguments with an application / json mimetype.

Response:
def jsonify(*args, **kwargs):
    """Creates a :class:`~flask.Response` with the JSON representation of
    the given arguments with an `application/json` mimetype.  The arguments
    to this function are the same as to the :class:`dict` constructor.

    Example usage::

        from flask import jsonify

        @app.route('/_get_current_user')
        def get_current_user():
            return jsonify(username=g.user.username,
                           email=g.user.email,
                           id=g.user.id)

    This will send a JSON response like this to the browser::

        {
            "username": "admin",
            "email": "admin@localhost",
            "id": 42
        }

    For security reasons only objects are supported toplevel.  For more
    information about this, have a look at :ref:`json-security`.

    This function's response will be pretty printed if it was not requested
    with ``X-Requested-With: XMLHttpRequest`` to simplify debugging unless
    the ``JSONIFY_PRETTYPRINT_REGULAR`` config parameter is set to false.

    .. versionadded:: 0.2
    """
    indent = None
    if current_app.config['JSONIFY_PRETTYPRINT_REGULAR'] \
        and not request.is_xhr:
        indent = 2
    return current_app.response_class(dumps(dict(*args, **kwargs),
        indent=indent),
        mimetype='application/json')


Instruction:
Write a Python 3 function for
implementing this method in a subclass such that it returns a serializable object for o.

Response:
def default(self, o):
        """Implement this method in a subclass such that it returns a
        serializable object for ``o``, or calls the base implementation (to
        raise a ``TypeError``).

        For example, to support arbitrary iterators, you could implement
        default like this::

            def default(self, o):
                try:
                    iterable = iter(o)
                except TypeError:
                    pass
                else:
                    return list(iterable)
                return JSONEncoder.default(self, o)
        """
        if isinstance(o, datetime):
            return http_date(o)
        if isinstance(o, uuid.UUID):
            return str(o)
        if hasattr(o, '__html__'):
            return text_type(o.__html__())
        return _json.JSONEncoder.default(self, o)


Instruction:
Write a Python 3 script for
converting the characters & and < and > and return a Markup string.

Response:
def escape(s):
    """Convert the characters &, <, >, ' and " in string s to HTML-safe
    sequences.  Use this if you need to display text that might contain
    such characters in HTML.  Marks return value as markup string.
    """
    if hasattr(s, '__html__'):
        return s.__html__()
    return Markup(text_type(s)
        .replace('&', '&')
        .replace('>', '>')
        .replace('<', '<')
        .replace("'", ''')
        .replace('"', '"')
    )


Instruction:
Can you tell what is the following Python 3 function doing
def set_many(self, mapping, timeout=None):
        rv = True
        for key, value in _items(mapping):
            if not self.set(key, value, timeout):
                rv = False
        return rv

Response: Sets multiple keys and values from a mapping.


Instruction:
Write a Python 3 function that can
increment the value of a key by delta.

Response:
def inc(self, key, delta=1):
        """Increments the value of a key by `delta`.  If the key does
        not yet exist it is initialized with `delta`.

        For supporting caches this is an atomic operation.

        :param key: the key to increment.
        :param delta: the delta to add.
        :returns: The new value or ``None`` for backend errors.
        """
        value = (self.get(key) or 0) + delta
        return value if self.set(key, value) else None


Instruction:
Write a Python 3 function for
dumping an object into a string for redis.

Response:
def dump_object(self, value):
        """Dumps an object into a string for redis.  By default it serializes
        integers as regular string and pickle dumps everything else.
        """
        t = type(value)
        if t in integer_types:
            return str(value).encode('ascii')
        return b'!' + pickle.dumps(value)


Instruction:
Can you generate the documentation for the following Python 3 function
def load_object(self, value):
        if value is None:
            return None
        if value.startswith(b'!'):
            try:
                return pickle.loads(value[1:])
            except pickle.PickleError:
                return None
        try:
            return int(value)
        except ValueError:
            # before 0.8 we did not have serialization.  Still support that.
            return value

Response: The reversal of dump_object.


Instruction:
Explain what the following Python 3 code does
def _strip_postfix(req):
    # FIXME: use package_to_requirement?
    match = re.search(r'^(.*?)(?:-dev|-\d.*)$', req)
    if match:
        # Strip off -dev, -0.2, etc.
        req = match.group(1)
    return req

Response: Strip the postfix from a package name.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _build_editable_options(req):

    """
        This method generates a dictionary of the query string
        parameters contained in a given editable URL.
    """
    regexp = re.compile(r"[\?#&](?P<name>[^&=]+)=(?P<value>[^&=]+)")
    matched = regexp.findall(req)

    if matched:
        ret = dict()
        for option in matched:
            (name, value) = option
            if name in ret:
                raise Exception("%s option already defined" % name)
            ret[name] = value
        return ret
    return None

Response:
This method builds a dictionary of the query string
AttributeNames parameters contained in a given editable URL.


Instruction:
Can you tell what is the following Python 3 function doing
def parse_editable(editable_req, default_vcs=None):

    url = editable_req
    extras = None

    # If a file path is specified with extras, strip off the extras.
    m = re.match(r'^(.+)(\[[^\]]+\])$', url)
    if m:
        url_no_extras = m.group(1)
        extras = m.group(2)
    else:
        url_no_extras = url

    if os.path.isdir(url_no_extras):
        if not os.path.exists(os.path.join(url_no_extras, 'setup.py')):
            raise InstallationError(
                "Directory %r is not installable. File 'setup.py' not found." %
                url_no_extras
            )
        # Treating it as code that has already been checked out
        url_no_extras = path_to_url(url_no_extras)

    if url_no_extras.lower().startswith('file:'):
        if extras:
            return (
                None,
                url_no_extras,
                pkg_resources.Requirement.parse(
                    '__placeholder__' + extras
                ).extras,
                {},
            )
        else:
            return None, url_no_extras, None, {}

    for version_control in vcs:
        if url.lower().startswith('%s:' % version_control):
            url = '%s+%s' % (version_control, url)
            break

    if '+' not in url:
        if default_vcs:
            url = default_vcs + '+' + url
        else:
            raise InstallationError(
                '%s should either be a path to a local project or a VCS url '
                'beginning with svn+, git+, hg+, or bzr+' %
                editable_req
            )

    vc_type = url.split('+', 1)[0].lower()

    if not vcs.get_backend(vc_type):
        error_message = 'For --editable=%s only ' % editable_req + \
            ', '.join([backend.name + '+URL' for backend in vcs.backends]) + \
            ' is currently supported'
        raise InstallationError(error_message)

    try:
        options = _build_editable_options(editable_req)
    except Exception as exc:
        raise InstallationError(
            '--editable=%s error in editable options:%s' % (editable_req, exc)
        )
    if not options or 'egg' not in options:
        req = _build_req_from_url(editable_req)
        if not req:
            raise InstallationError(
                '--editable=%s is not the right format; it must have '
                '#egg=Package' % editable_req
            )
    else:
        req = options['egg']

    package = _strip_postfix(req)
    return package, url, None, options

Response: Parses an editable requirement into a list of tuples.


Instruction:
How would you explain what the following Python 3 function does
def from_line(
            cls, name, comes_from=None, isolated=False, options=None,
            wheel_cache=None):
        """Creates an InstallRequirement from a name, which might be a
        requirement, directory containing 'setup.py', filename, or URL.
        """
        from pip.index import Link

        if is_url(name):
            marker_sep = '; '
        else:
            marker_sep = ';'
        if marker_sep in name:
            name, markers = name.split(marker_sep, 1)
            markers = markers.strip()
            if not markers:
                markers = None
        else:
            markers = None
        name = name.strip()
        req = None
        path = os.path.normpath(os.path.abspath(name))
        link = None
        extras = None

        if is_url(name):
            link = Link(name)
        else:
            p, extras = _strip_extras(path)
            if (os.path.isdir(p) and
                    (os.path.sep in name or name.startswith('.'))):

                if not is_installable_dir(p):
                    raise InstallationError(
                        "Directory %r is not installable. File 'setup.py' "
                        "not found." % name
                    )
                link = Link(path_to_url(p))
            elif is_archive_file(p):
                if not os.path.isfile(p):
                    logger.warning(
                        'Requirement %r looks like a filename, but the '
                        'file does not exist',
                        name
                    )
                link = Link(path_to_url(p))

        # it's a local file, dir, or url
        if link:
            # Handle relative file URLs
            if link.scheme == 'file' and re.search(r'\.\./', link.url):
                link = Link(
                    path_to_url(os.path.normpath(os.path.abspath(link.path))))
            # wheel file
            if link.is_wheel:
                wheel = Wheel(link.filename)  # can raise InvalidWheelFilename
                if not wheel.supported():
                    raise UnsupportedWheel(
                        "%s is not a supported wheel on this platform." %
                        wheel.filename
                    )
                req = "%s==%s" % (wheel.name, wheel.version)
            else:
                # set the req to the egg fragment.  when it's not there, this
                # will become an 'unnamed' requirement
                req = link.egg_fragment

        # a requirement specifier
        else:
            req = name

        options = options if options else {}
        res = cls(req, comes_from, link=link, markers=markers,
                  isolated=isolated, options=options,
                  wheel_cache=wheel_cache)

        if extras:
            res.extras = pkg_resources.Requirement.parse('__placeholder__' +
                                                         extras).extras

        return res

Response: Creates an InstallRequirement object from a line of text.


Instruction:
Write a Python 3 function that can
populate the link attribute with the link information from the given finder.

Response:
def populate_link(self, finder, upgrade):
        """Ensure that if a link can be found for this, that it is found.

        Note that self.link may still be None - if Upgrade is False and the
        requirement is already installed.
        """
        if self.link is None:
            self.link = finder.find_requirement(self, upgrade)


Instruction:
Make a summary of the following Python 3 code
def _correct_build_location(self):
        if self.source_dir is not None:
            return
        assert self.req is not None
        assert self._temp_build_dir
        assert self._ideal_build_dir
        old_location = self._temp_build_dir
        self._temp_build_dir = None
        new_location = self.build_location(self._ideal_build_dir)
        if os.path.exists(new_location):
            raise InstallationError(
                'A package already exists in %s; please remove it to continue'
                % display_path(new_location))
        logger.debug(
            'Moving package %s from %s to new location %s',
            self, display_path(old_location), display_path(new_location),
        )
        shutil.move(old_location, new_location)
        self._temp_build_dir = new_location
        self._ideal_build_dir = None
        self.source_dir = new_location
        self._egg_info_path = None

Response: Move the build directory to the correct location.


Instruction:
Make a summary of the following Python 3 code
def ensure_has_source_dir(self, parent_dir):
        if self.source_dir is None:
            self.source_dir = self.build_location(parent_dir)
        return self.source_dir

Response: Ensure that a source_dir is set.


Instruction:
Make a summary of the following Python 3 code
def remove_temporary_source(self):
        if self.source_dir and os.path.exists(
                os.path.join(self.source_dir, PIP_DELETE_MARKER_FILENAME)):
            logger.debug('Removing source in %s', self.source_dir)
            rmtree(self.source_dir)
        self.source_dir = None
        if self._temp_build_dir and os.path.exists(self._temp_build_dir):
            rmtree(self._temp_build_dir)
        self._temp_build_dir = None

Response:
Remove the source files from this requirement if they are marked
        for deletion


Instruction:
Write a Python 3 function for
returning a pkg_resources. Distribution built from self. egg_info_path

Response:
def get_dist(self):
        """Return a pkg_resources.Distribution built from self.egg_info_path"""
        egg_info = self.egg_info_path('').rstrip('/')
        base_dir = os.path.dirname(egg_info)
        metadata = pkg_resources.PathMetadata(base_dir, egg_info)
        dist_name = os.path.splitext(os.path.basename(egg_info))[0]
        return pkg_resources.Distribution(
            os.path.dirname(egg_info),
            project_name=dist_name,
            metadata=metadata)


Instruction:
Can you create a Python 3 function that
parses the contents of the egg - info file into a list of metadata items needed.

Response:
def parse_info(wininfo_name, egginfo_name):
    """Extract metadata from filenames.

    Extracts the 4 metadataitems needed (name, version, pyversion, arch) from
    the installer filename and the name of the egg-info directory embedded in
    the zipfile (if any).

    The egginfo filename has the format::

        name-ver(-pyver)(-arch).egg-info

    The installer filename has the format::

        name-ver.arch(-pyver).exe

    Some things to note:

    1. The installer filename is not definitive. An installer can be renamed
       and work perfectly well as an installer. So more reliable data should
       be used whenever possible.
    2. The egg-info data should be preferred for the name and version, because
       these come straight from the distutils metadata, and are mandatory.
    3. The pyver from the egg-info data should be ignored, as it is
       constructed from the version of Python used to build the installer,
       which is irrelevant - the installer filename is correct here (even to
       the point that when it's not there, any version is implied).
    4. The architecture must be taken from the installer filename, as it is
       not included in the egg-info data.
    5. Architecture-neutral installers still have an architecture because the
       installer format itself (being executable) is architecture-specific. We
       should therefore ignore the architecture if the content is pure-python.
    """

    egginfo = None
    if egginfo_name:
        egginfo = egg_info_re.search(egginfo_name)
        if not egginfo:
            raise ValueError("Egg info filename %s is not valid" %
                    (egginfo_name,))

    # Parse the wininst filename
    # 1. Distribution name (up to the first '-')
    w_name, sep, rest = wininfo_name.partition('-')
    if not sep:
        raise ValueError("Installer filename %s is not valid" %
                (wininfo_name,))
    # Strip '.exe'
    rest = rest[:-4]
    # 2. Python version (from the last '-', must start with 'py')
    rest2, sep, w_pyver = rest.rpartition('-')
    if sep and w_pyver.startswith('py'):
        rest = rest2
        w_pyver = w_pyver.replace('.', '')
    else:
        # Not version specific - use py2.py3. While it is possible that
        # pure-Python code is not compatible with both Python 2 and 3, there
        # is no way of knowing from the wininst format, so we assume the best
        # here (the user can always manually rename the wheel to be more
        # restrictive if needed).
        w_pyver = 'py2.py3'
    # 3. Version and architecture
    w_ver, sep, w_arch = rest.rpartition('.')
    if not sep:
        raise ValueError("Installer filename %s is not valid" %
                (wininfo_name,))

    if egginfo:
        w_name = egginfo.group('name')
        w_ver = egginfo.group('ver')

    return dict(name=w_name, ver=w_ver, arch=w_arch, pyver=w_pyver)


Instruction:
Make a summary of the following Python 3 code
def is_internal_attribute(obj, attr):
    if isinstance(obj, function_type):
        if attr in UNSAFE_FUNCTION_ATTRIBUTES:
            return True
    elif isinstance(obj, method_type):
        if attr in UNSAFE_FUNCTION_ATTRIBUTES or \
           attr in UNSAFE_METHOD_ATTRIBUTES:
            return True
    elif isinstance(obj, type):
        if attr == 'mro':
            return True
    elif isinstance(obj, (code_type, traceback_type, frame_type)):
        return True
    elif isinstance(obj, generator_type):
        if attr in UNSAFE_GENERATOR_ATTRIBUTES:
            return True
    return attr.startswith('__')

Response: Test if the attribute given is an internal python attribute.


Instruction:
How would you explain what the following Python 3 function does
def _get_stream_for_parsing(self):
        cached_data = getattr(self, '_cached_data', None)
        if cached_data is not None:
            return BytesIO(cached_data)
        return self.stream

Response:
This is the same as accessing the stream with the difference
        that is found in the cache.


Instruction:
Given the following Python 3 function, write the documentation
def get_data(self, cache=True, as_text=False, parse_form_data=False):
        rv = getattr(self, '_cached_data', None)
        if rv is None:
            if parse_form_data:
                self._load_form_data()
            rv = self.stream.read()
            if cache:
                self._cached_data = rv
        if as_text:
            rv = rv.decode(self.charset, self.encoding_errors)
        return rv

Response: This method reads the buffered incoming data from the client into one - element bytestring.


Instruction:
Explain what the following Python 3 code does
def url_fix(s, charset='utf-8'):
    # First step is to switch to unicode processing and to convert
    # backslashes (which are invalid in URLs anyways) to slashes.  This is
    # consistent with what Chrome does.
    s = to_unicode(s, charset, 'replace').replace('\\', '/')

    # For the specific case that we look like a malformed windows URL
    # we want to fix this up manually:
    if s.startswith('file://') and s[7:8].isalpha() and s[8:10] in (':/', '|/'):
        s = 'file:///' + s[7:]

    url = url_parse(s)
    path = url_quote(url.path, charset, safe='/%+$!*\'(),')
    qs = url_quote_plus(url.query, charset, safe=':&%=+$!*\'(),')
    anchor = url_quote_plus(url.fragment, charset, safe=':&%=+$!*\'(),')
    return to_native(url_unparse((url.scheme, url.encode_netloc(),
                                  path, qs, anchor)))

Response: Fixes the URL by a user that just isn t a real URL.


Instruction:
Given the following Python 3 function, write the documentation
def user_cache_dir(appname):
    if WINDOWS:
        # Get the base path
        path = os.path.normpath(_get_win_folder("CSIDL_LOCAL_APPDATA"))

        # Add our app name and Cache directory to it
        path = os.path.join(path, appname, "Cache")
    elif sys.platform == "darwin":
        # Get the base path
        path = os.path.expanduser("~/Library/Caches")

        # Add our app name to it
        path = os.path.join(path, appname)
    else:
        # Get the base path
        path = os.getenv("XDG_CACHE_HOME", os.path.expanduser("~/.cache"))

        # Add our app name to it
        path = os.path.join(path, appname)

    return path

Response: r Returns full path to the user - specific cache dir for this application.


Instruction:
Make a summary of the following Python 3 code
def user_data_dir(appname, roaming=False):
    if WINDOWS:
        const = roaming and "CSIDL_APPDATA" or "CSIDL_LOCAL_APPDATA"
        path = os.path.join(os.path.normpath(_get_win_folder(const)), appname)
    elif sys.platform == "darwin":
        path = os.path.join(
            os.path.expanduser('~/Library/Application Support/'),
            appname,
        )
    else:
        path = os.path.join(
            os.getenv('XDG_DATA_HOME', os.path.expanduser("~/.local/share")),
            appname,
        )

    return path

Response: Return full path to the user - specific data directory for this application.


Instruction:
How would you implement a function in Python 3 that
returns full path to the user - specific log dir for this application.

Response:
def user_log_dir(appname):
    """
    Return full path to the user-specific log dir for this application.

        "appname" is the name of application.
            If None, just the system directory is returned.

    Typical user cache directories are:
        Mac OS X:   ~/Library/Logs/<AppName>
        Unix:       ~/.cache/<AppName>/log  # or under $XDG_CACHE_HOME if
                    defined
        Win XP:     C:\Documents and Settings\<username>\Local Settings\ ...
                    ...Application Data\<AppName>\Logs
        Vista:      C:\\Users\<username>\AppData\Local\<AppName>\Logs

    On Windows the only suggestion in the MSDN docs is that local settings
    go in the `CSIDL_LOCAL_APPDATA` directory. (Note: I'm interested in
    examples of what some windows apps use for a logs dir.)

    OPINION: This function appends "Logs" to the `CSIDL_LOCAL_APPDATA`
    value for Windows and appends "log" to the user cache dir for Unix.
    """
    if WINDOWS:
        path = os.path.join(user_data_dir(appname), "Logs")
    elif sys.platform == "darwin":
        path = os.path.join(os.path.expanduser('~/Library/Logs'), appname)
    else:
        path = os.path.join(user_cache_dir(appname), "log")

    return path


Instruction:
Can you generate a brief explanation for the following Python 3 code
def user_config_dir(appname, roaming=True):
    if WINDOWS:
        path = user_data_dir(appname, roaming=roaming)
    elif sys.platform == "darwin":
        path = user_data_dir(appname)
    else:
        path = os.getenv('XDG_CONFIG_HOME', os.path.expanduser("~/.config"))
        path = os.path.join(path, appname)

    return path

Response: Return full path to the user - specific config dir for this application.


Instruction:
How would you code a function in Python 3 to
return a list of potential user - shared config dirs for this application.

Response:
def site_config_dirs(appname):
    """Return a list of potential user-shared config dirs for this application.

        "appname" is the name of application.

    Typical user config directories are:
        Mac OS X:   /Library/Application Support/<AppName>/
        Unix:       /etc or $XDG_CONFIG_DIRS[i]/<AppName>/ for each value in
                    $XDG_CONFIG_DIRS
        Win XP:     C:\Documents and Settings\All Users\Application ...
                    ...Data\<AppName>\
        Vista:      (Fail! "C:\ProgramData" is a hidden *system* directory
                    on Vista.)
        Win 7:      Hidden, but writeable on Win 7:
                    C:\ProgramData\<AppName>\
    """
    if WINDOWS:
        path = os.path.normpath(_get_win_folder("CSIDL_COMMON_APPDATA"))
        pathlist = [os.path.join(path, appname)]
    elif sys.platform == 'darwin':
        pathlist = [os.path.join('/Library/Application Support', appname)]
    else:
        # try looking in $XDG_CONFIG_DIRS
        xdg_config_dirs = os.getenv('XDG_CONFIG_DIRS', '/etc/xdg')
        if xdg_config_dirs:
            pathlist = [
                os.sep.join([os.path.expanduser(x), appname])
                for x in xdg_config_dirs.split(os.pathsep)
            ]
        else:
            pathlist = []

        # always look in /etc directly as well
        pathlist.append('/etc')

    return pathlist


Instruction:
Can you tell what is the following Python 3 function doing
def _iter_module_files():
    # The list call is necessary on Python 3 in case the module
    # dictionary modifies during iteration.
    for module in list(sys.modules.values()):
        if module is None:
            continue
        filename = getattr(module, '__file__', None)
        if filename:
            old = None
            while not os.path.isfile(filename):
                old = filename
                filename = os.path.dirname(filename)
                if filename == old:
                    break
            else:
                if filename[-4:] in ('.pyc', '.pyo'):
                    filename = filename[:-1]
                yield filename

Response: This function iterates over all relevant Python files in the modules and returns a generator of all relevant files.


Instruction:
Can you create a Python 3 function that
spawns a new Python interpreter with the same arguments as this one and running the reloader thread.

Response:
def restart_with_reloader(self):
        """Spawn a new Python interpreter with the same arguments as this one,
        but running the reloader thread.
        """
        while 1:
            _log('info', ' * Restarting with %s' % self.name)
            args = [sys.executable] + sys.argv
            new_environ = os.environ.copy()
            new_environ['WERKZEUG_RUN_MAIN'] = 'true'

            # a weird bug on windows. sometimes unicode strings end up in the
            # environment and subprocess.call does not like this, encode them
            # to latin1 and continue.
            if os.name == 'nt' and PY2:
                for key, value in iteritems(new_environ):
                    if isinstance(value, text_type):
                        new_environ[key] = value.encode('iso-8859-1')

            exit_code = subprocess.call(args, env=new_environ)
            if exit_code != 3:
                return exit_code


Instruction:
How would you implement a function in Python 3 that
wraps around six. text_type to convert None to empty string

Response:
def to_text(s, blank_if_none=True):
    """Wrapper around six.text_type to convert None to empty string"""
    if s is None:
        if blank_if_none:
            return ""
        else:
            return None
    elif isinstance(s, text_type):
        return s
    else:
        return text_type(s)


Instruction:
Write a Python 3 function for
returning an existing CA bundle path or None if no CA bundle exists.

Response:
def find_ca_bundle():
    """Return an existing CA bundle path, or None"""
    if os.name=='nt':
        return get_win_certfile()
    else:
        for cert_path in cert_paths:
            if os.path.isfile(cert_path):
                return cert_path
    try:
        return pkg_resources.resource_filename('certifi', 'cacert.pem')
    except (ImportError, ResolutionError, ExtractionError):
        return None


Instruction:
Write a Python 3 script to
parse a string or file - like object into a tree

Response:
def parse(doc, treebuilder="etree", encoding=None,
          namespaceHTMLElements=True):
    """Parse a string or file-like object into a tree"""
    tb = treebuilders.getTreeBuilder(treebuilder)
    p = HTMLParser(tb, namespaceHTMLElements=namespaceHTMLElements)
    return p.parse(doc, encoding=encoding)


Instruction:
How would you code a function in Python 3 to
parse a well - formed tree into a well - formed tree object

Response:
def parse(self, stream, encoding=None, parseMeta=True, useChardet=True):
        """Parse a HTML document into a well-formed tree

        stream - a filelike object or string containing the HTML to be parsed

        The optional encoding parameter must be a string that indicates
        the encoding.  If specified, that encoding will be used,
        regardless of any BOM or later declaration (such as in a meta
        element)
        """
        self._parse(stream, innerHTML=False, encoding=encoding,
                    parseMeta=parseMeta, useChardet=useChardet)
        return self.tree.getDocument()


Instruction:
Write a Python 3 script to
parse a fragment into a well - formed tree fragment

Response:
def parseFragment(self, stream, container="div", encoding=None,
                      parseMeta=False, useChardet=True):
        """Parse a HTML fragment into a well-formed tree fragment

        container - name of the element we're setting the innerHTML property
        if set to None, default to 'div'

        stream - a filelike object or string containing the HTML to be parsed

        The optional encoding parameter must be a string that indicates
        the encoding.  If specified, that encoding will be used,
        regardless of any BOM or later declaration (such as in a meta
        element)
        """
        self._parse(stream, True, container=container, encoding=encoding)
        return self.tree.getFragment()


Instruction:
How would you code a function in Python 3 to
translate a word into a list of transcription keys.

Response:
def translate(self, word):
        """
        pass in a word string that you
        would like to see probable matches for.
        """
        if (word not in self.transmissions):
            raise NoMatchError('no matches found')
        else:
            trans = self.transmissions[word]
            # print out a sorted list of all non-zero trans
            return sorted(((k, v) for k, v in trans.iteritems() if v != 0),
                                                                reverse=True)


Instruction:
Given the following Python 3 function, write the documentation
def initTef(self):
        '''
        get all probable matches
        and then initialize t(f|e)
        '''
        probs = {}
        transmissions = {}

        # go through each german word
        for word in self.en_words:
            word_poss = []
            # if word in sentence.. then
            for sent in self.en_dict:
                if word in sent:
                    matching = self.de_dict[self.en_dict.index(sent)]
                    word_poss = word_poss + matching.split()

            # remove the duplicates
            word_poss = list(set(word_poss))
            # add the probable matches
            probs[word] = word_poss

        self.probs = probs
        print self.probs

        for word in self.en_words:
            # print self.probs
            word_probs = self.probs[word]
            if (len(word_probs) == 0):
                print word, word_probs
            uniform_prob = 1.0 / len(word_probs)

            word_probs = dict([(w, uniform_prob) for w in word_probs])

            # save word_probs
            transmissions[word] = word_probs

        self.transmissions = transmissions

Response: get all probable matches and then initialize t


Instruction:
Given the following Python 3 function, write the documentation
def iterateEM(self, count):
        '''
        Iterate through all transmissions of english to
        foreign words. keep count of repeated occurences
         do until convergence
           set count(e|f) to 0 for all e,f
           set total(f) to 0 for all f
           for all sentence pairs (e_s,f_s)
             set total_s(e) = 0 for all e
             for all words e in e_s
               for all words f in f_s
                 total_s(e) += t(e|f)
             for all words e in e_s
               for all words f in f_s
                 count(e|f) += t(e|f) / total_s(e)
                 total(f)   += t(e|f) / total_s(e)
           for all f
             for all e
               t(e|f) = count(e|f) / total(f)
        '''

        for iter in range(count):

            countef = {}
            totalf = {}
            # set the count of the words to zero
            for word in self.en_words:
                if(word not in self.probs):
                    continue

                word_probs = self.probs[word]

                count = dict([(w, 0) for w in word_probs])
                countef[word] = count
                totalf[word] = 0

            self.countef = countef

            self.totalf = totalf

            # NOW iterate over each word pair
            for (es, ds) in self.sent_pairs:
                es_split = es.split()
                ds_split = ds.split()

                for d in ds_split:
                    self.totals[d] = 0
                    for e in es_split:

                        if (e not in self.transmissions):
                            continue

                        e_trans = self.transmissions[e]

                        if (d not in e_trans):
                            continue

                        self.totals[d] += e_trans[d]

                    # Get count(e|f) and total(f)
                    for e in es_split:
                        if(e not in self.transmissions):
                            continue
                        if (d not in self.transmissions[e]):
                            continue
                        self.countef[e][
                            d] += self.transmissions[e][d] / self.totals[d]
                        self.totalf[
                            e] += self.transmissions[e][d] / self.totals[d]

            for e in self.en_words:
                if (e not in self.probs):
                    continue
                e_prob = self.probs[e]
                for d in e_prob:
                    self.transmissions[e][d] = self.countef[
                        e][d] / self.totalf[e]

Response:
Iterate through all transmissions of english to
        foreign words and update the internal state of the object.


Instruction:
Given the following Python 3 function, write the documentation
def bind(self):

        HTTPServer.__init__(self, (self.host, self.port), HTTPRequestHandler)
        self.port = self.server_port

Response: Bind and activate HTTP server.


Instruction:
Create a Python 3 function to
report startup info to stdout.

Response:
def report(self):
        """Report startup info to stdout."""

        print(
            self.report_message.format(
                service=self.service,
                host=self.host,
                port=self.port,
            )
        )
        sys.stdout.flush()


Instruction:
Here you have a function in Python 3, explain what it does
def make_ssl_devcert(base_path, host=None, cn=None):
    from OpenSSL import crypto
    if host is not None:
        cn = '*.%s/CN=%s' % (host, host)
    cert, pkey = generate_adhoc_ssl_pair(cn=cn)

    cert_file = base_path + '.crt'
    pkey_file = base_path + '.key'

    with open(cert_file, 'wb') as f:
        f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
    with open(pkey_file, 'wb') as f:
        f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey))

    return cert_file, pkey_file

Response: Creates an SSL certificate and key for development.


Instruction:
Can you generate the documentation for the following Python 3 function
def load_bytecode(self, f):
        # make sure the magic header is correct
        magic = f.read(len(bc_magic))
        if magic != bc_magic:
            self.reset()
            return
        # the source code of the file changed, we need to reload
        checksum = pickle.load(f)
        if self.checksum != checksum:
            self.reset()
            return
        self.code = marshal_load(f)

Response: Loads bytecode from a file or file like object.


Instruction:
How would you code a function in Python 3 to
convert native Python keyword args to a dictionary of stylesheet parameters.

Response:
def stylesheet_params(**kwargs):
    """Convert keyword args to a dictionary of stylesheet parameters.
    XSL stylesheet parameters must be XPath expressions, i.e.:

    * string expressions, like "'5'"
    * simple (number) expressions, like "5"
    * valid XPath expressions, like "/a/b/text()"

    This function converts native Python keyword arguments to stylesheet
    parameters following these rules:
    If an arg is a string wrap it with XSLT.strparam().
    If an arg is an XPath object use its path string.
    If arg is None raise TypeError.
    Else convert arg to string.
    """
    result = {}
    for key, val in kwargs.items():
        if isinstance(val, basestring):
            val = _etree.XSLT.strparam(val)
        elif val is None:
            raise TypeError('None not allowed as a stylesheet parameter')
        elif not isinstance(val, _etree.XPath):
            val = unicode(val)
        result[key] = val
    return result


Instruction:
Can you create a Python 3 function that
returns a copy of paramsDict updated with kwargsDict entries wrapped as stylesheet arguments.

Response:
def _stylesheet_param_dict(paramsDict, kwargsDict):
    """Return a copy of paramsDict, updated with kwargsDict entries, wrapped as
    stylesheet arguments.
    kwargsDict entries with a value of None are ignored.
    """
    # beware of changing mutable default arg
    paramsDict = dict(paramsDict)
    for k, v in kwargsDict.items():
        if v is not None: # None values do not override
            paramsDict[k] = v
    paramsDict = stylesheet_params(**paramsDict)
    return paramsDict


Instruction:
Implement a function in Python 3 to
extract embedded schematron schema from an XML element.

Response:
def _extract(self, element):
        """Extract embedded schematron schema from non-schematron host schema.
        This method will only be called by __init__ if the given schema document
        is not a schematron schema by itself.
        Must return a schematron schema document tree or None.
        """
        schematron = None
        if element.tag == _xml_schema_root:
            schematron = self._extract_xsd(element)
        elif element.nsmap[element.prefix] == RELAXNG_NS:
            # RelaxNG does not have a single unique root element
            schematron = self._extract_rng(element)
        return schematron


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_backend_name(self, location):
        for vc_type in self._registry.values():
            logger.debug('Checking in %s for %s (%s)...',
                         location, vc_type.dirname, vc_type.name)
            path = os.path.join(location, vc_type.dirname)
            if os.path.exists(path):
                logger.debug('Determine that %s uses VCS: %s',
                             location, vc_type.name)
                return vc_type.name
        return None

Response: Return the name of the version control backend if found at given location.


Instruction:
Implement a function in Python 3 to
return True if the repository is local.

Response:
def _is_local_repository(self, repo):
        """
           posix absolute paths start with os.path.sep,
           win32 ones ones start with drive (like c:\\folder)
        """
        drive, tail = os.path.splitdrive(repo)
        return repo.startswith(os.path.sep) or drive


Instruction:
Can you tell what is the following Python 3 function doing
def get_info(self, location):
        assert not location.rstrip('/').endswith(self.dirname), \
            'Bad directory: %s' % location
        return self.get_url(location), self.get_revision(location)

Response: Returns (url, revision), where both are strings


Instruction:
Given the following Python 3 function, write the documentation
def unpack(self, location):
        if os.path.exists(location):
            rmtree(location)
        self.obtain(location)

Response: Unpack the current repository into location.


Instruction:
How would you explain what the following Python 3 function does
def run_command(self, cmd, show_stdout=True, cwd=None,
                    raise_on_returncode=True,
                    command_level=logging.DEBUG, command_desc=None,
                    extra_environ=None):
        """
        Run a VCS subcommand
        This is simply a wrapper around call_subprocess that adds the VCS
        command name, and checks that the VCS is available
        """
        cmd = [self.name] + cmd
        try:
            return call_subprocess(cmd, show_stdout, cwd,
                                   raise_on_returncode, command_level,
                                   command_desc, extra_environ)
        except OSError as e:
            # errno.ENOENT = no such file or directory
            # In other words, the VCS executable isn't available
            if e.errno == errno.ENOENT:
                raise BadCommand('Cannot find command %r' % self.name)
            else:
                raise

Response: Run a VCS command


Instruction:
Create a Python 3 function to
return a list of tags that are supported by each version specified in versions.

Response:
def get_supported(versions=None):
    """Return a list of supported tags for each version specified in
    `versions`.

    :param versions: a list of string versions, of the form ["33", "32"],
        or None. The first version will be assumed to support our ABI.
    """
    supported = []

    # Versions must be given with respect to the preference
    if versions is None:
        versions = []
        major = sys.version_info[0]
        # Support all previous minor Python versions.
        for minor in range(sys.version_info[1], -1, -1):
            versions.append(''.join(map(str, (major, minor))))

    impl = get_abbr_impl()

    abis = []

    soabi = sysconfig.get_config_var('SOABI')
    if soabi and soabi.startswith('cpython-'):
        abis[0:0] = ['cp' + soabi.split('-', 1)[-1]]

    abi3s = set()
    import imp
    for suffix in imp.get_suffixes():
        if suffix[0].startswith('.abi'):
            abi3s.add(suffix[0].split('.', 2)[1])

    abis.extend(sorted(list(abi3s)))

    abis.append('none')

    arch = get_platform()

    # Current version, current API (built specifically for our Python):
    for abi in abis:
        supported.append(('%s%s' % (impl, versions[0]), abi, arch))

    # No abi / arch, but requires our implementation:
    for i, version in enumerate(versions):
        supported.append(('%s%s' % (impl, version), 'none', 'any'))
        if i == 0:
            # Tagged specifically as being cross-version compatible
            # (with just the major version specified)
            supported.append(('%s%s' % (impl, versions[0][0]), 'none', 'any'))

    # No abi / arch, generic Python
    for i, version in enumerate(versions):
        supported.append(('py%s' % (version,), 'none', 'any'))
        if i == 0:
            supported.append(('py%s' % (version[0]), 'none', 'any'))

    return supported


Instruction:
Explain what the following Python 3 code does
def get_host(environ, trusted_hosts=None):
    if 'HTTP_X_FORWARDED_HOST' in environ:
        rv = environ['HTTP_X_FORWARDED_HOST'].split(',', 1)[0].strip()
    elif 'HTTP_HOST' in environ:
        rv = environ['HTTP_HOST']
    else:
        rv = environ['SERVER_NAME']
        if (environ['wsgi.url_scheme'], environ['SERVER_PORT']) not \
           in (('https', '443'), ('http', '80')):
            rv += ':' + environ['SERVER_PORT']
    if trusted_hosts is not None:
        if not host_is_trusted(rv, trusted_hosts):
            from werkzeug.exceptions import SecurityError
            raise SecurityError('Host "%s" is not trusted' % rv)
    return rv

Response: Returns the real host for the given WSGI environment.


Instruction:
Make a summary of the following Python 3 code
def distros_for_location(location, basename, metadata=None):
    if basename.endswith('.egg.zip'):
        basename = basename[:-4]    # strip the .zip
    if basename.endswith('.egg') and '-' in basename:
        # only one, unambiguous interpretation
        return [Distribution.from_location(location, basename, metadata)]
    if basename.endswith('.exe'):
        win_base, py_ver, platform = parse_bdist_wininst(basename)
        if win_base is not None:
            return interpret_distro_name(
                location, win_base, metadata, py_ver, BINARY_DIST, platform
            )
    # Try source distro extensions (.zip, .tgz, etc.)
    #
    for ext in EXTENSIONS:
        if basename.endswith(ext):
            basename = basename[:-len(ext)]
            return interpret_distro_name(location, basename, metadata)
    return []

Response: Yields a list of distributions for a given location.


Instruction:
Make a summary of the following Python 3 code
def find_external_links(url, page):

    for match in REL.finditer(page):
        tag, rel = match.groups()
        rels = set(map(str.strip, rel.lower().split(',')))
        if 'homepage' in rels or 'download' in rels:
            for match in HREF.finditer(tag):
                yield urljoin(url, htmldecode(match.group(1)))

    for tag in ("Home Page", "Download URL"):
        pos = page.find(tag)
        if pos!=-1:
            match = HREF.search(page,pos)
            if match:
                yield urljoin(url, htmldecode(match.group(1)))

Response: Find external links in page yielding URLs


Instruction:
Here you have a function in Python 3, explain what it does
def _encode_auth(auth):
    auth_s = unquote(auth)
    # convert to bytes
    auth_bytes = auth_s.encode()
    # use the legacy interface for Python 2.3 support
    encoded_bytes = base64.encodestring(auth_bytes)
    # convert back to a string
    encoded = encoded_bytes.decode()
    # strip the trailing carriage return
    return encoded.replace('\n','')

Response: Encode auth string into base64 encoded string.


Instruction:
Can you tell what is the following Python 3 function doing
def local_open(url):
    scheme, server, path, param, query, frag = urlparse(url)
    filename = url2pathname(path)
    if os.path.isfile(filename):
        return urllib2.urlopen(url)
    elif path.endswith('/') and os.path.isdir(filename):
        files = []
        for f in os.listdir(filename):
            if f=='index.html':
                with open(os.path.join(filename,f),'r') as fp:
                    body = fp.read()
                break
            elif os.path.isdir(os.path.join(filename,f)):
                f+='/'
            files.append("%s" % (f,f))
        else:
            body = ("<title>%s</title>" % url) + \
                "%s" % '\n'.join(files)
        status, message = 200, "OK"
    else:
        status, message, body = 404, "Path not found", "Not found"

    headers = {'content-type': 'text/html'}
    return HTTPError(url, status, message, headers, StringIO(body))

Response: Open a local path with special support for files and directories


Instruction:
How would you explain what the following Python 3 function does
def from_url(cls, url):
        "Construct a (possibly null) ContentChecker from a URL"
        fragment = urlparse(url)[-1]
        if not fragment:
            return ContentChecker()
        match = cls.pattern.search(fragment)
        if not match:
            return ContentChecker()
        return cls(**match.groupdict())

Response: Construct a ( possibly null ) ContentChecker from a URL


Instruction:
Write a Python 3 script to
evaluate a URL and process it

Response:
def process_url(self, url, retrieve=False):
        """Evaluate a URL as a possible download, and maybe retrieve it"""
        if url in self.scanned_urls and not retrieve:
            return
        self.scanned_urls[url] = True
        if not URL_SCHEME(url):
            self.process_filename(url)
            return
        else:
            dists = list(distros_for_url(url))
            if dists:
                if not self.url_ok(url):
                    return
                self.debug("Found link: %s", url)

        if dists or not retrieve or url in self.fetched_urls:
            list(map(self.add, dists))
            return  # don't need the actual page

        if not self.url_ok(url):
            self.fetched_urls[url] = True
            return

        self.info("Reading %s", url)
        self.fetched_urls[url] = True   # prevent multiple fetch attempts
        f = self.open_url(url, "Download error on %s: %%s -- Some packages may not be found!" % url)
        if f is None: return
        self.fetched_urls[f.url] = True
        if 'html' not in f.headers.get('content-type', '').lower():
            f.close()   # not html, we can't process it
            return

        base = f.url     # handle redirects
        page = f.read()
        if not isinstance(page, str): # We are in Python 3 and got bytes. We want str.
            if isinstance(f, HTTPError):
                # Errors have no charset, assume latin1:
                charset = 'latin-1'
            else:
                charset = f.headers.get_param('charset') or 'latin-1'
            page = page.decode(charset, "ignore")
        f.close()
        for match in HREF.finditer(page):
            link = urljoin(base, htmldecode(match.group(1)))
            self.process_url(link)
        if url.startswith(self.index_url) and getattr(f,'code',None)!=404:
            page = self.process_index(url, page)


Instruction:
Explain what the following Python 3 code does
def get_supported(versions=None, noarch=False):
    supported = []

    # Versions must be given with respect to the preference
    if versions is None:
        versions = []
        major = sys.version_info[0]
        # Support all previous minor Python versions.
        for minor in range(sys.version_info[1], -1, -1):
            versions.append(''.join(map(str, (major, minor))))

    impl = get_abbr_impl()

    abis = []

    try:
        soabi = sysconfig.get_config_var('SOABI')
    except IOError as e:  # Issue #1074
        warnings.warn("{0}".format(e), RuntimeWarning)
        soabi = None

    if soabi and soabi.startswith('cpython-'):
        abis[0:0] = ['cp' + soabi.split('-', 1)[-1]]

    abi3s = set()
    import imp
    for suffix in imp.get_suffixes():
        if suffix[0].startswith('.abi'):
            abi3s.add(suffix[0].split('.', 2)[1])

    abis.extend(sorted(list(abi3s)))

    abis.append('none')

    if not noarch:
        arch = get_platform()
        if sys.platform == 'darwin':
            # support macosx-10.6-intel on macosx-10.9-x86_64
            match = _osx_arch_pat.match(arch)
            if match:
                name, major, minor, actual_arch = match.groups()
                actual_arches = [actual_arch]
                if actual_arch in ('i386', 'ppc'):
                    actual_arches.append('fat')
                if actual_arch in ('i386', 'x86_64'):
                    actual_arches.append('intel')
                if actual_arch in ('i386', 'ppc', 'x86_64'):
                    actual_arches.append('fat3')
                if actual_arch in ('ppc64', 'x86_64'):
                    actual_arches.append('fat64')
                if actual_arch in ('i386', 'x86_64', 'intel', 'ppc', 'ppc64'):
                    actual_arches.append('universal')
                tpl = '{0}_{1}_%i_%s'.format(name, major)
                arches = []
                for m in range(int(minor) + 1):
                    for a in actual_arches:
                        arches.append(tpl % (m, a))
            else:
                # arch pattern didn't match (?!)
                arches = [arch]
        else:
            arches = [arch]

        # Current version, current API (built specifically for our Python):
        for abi in abis:
            for arch in arches:
                supported.append(('%s%s' % (impl, versions[0]), abi, arch))

        # Has binaries, does not use the Python API:
        supported.append(('py%s' % (versions[0][0]), 'none', arch))

    # No abi / arch, but requires our implementation:
    for i, version in enumerate(versions):
        supported.append(('%s%s' % (impl, version), 'none', 'any'))
        if i == 0:
            # Tagged specifically as being cross-version compatible
            # (with just the major version specified)
            supported.append(('%s%s' % (impl, versions[0][0]), 'none', 'any'))

    # No abi / arch, generic Python
    for i, version in enumerate(versions):
        supported.append(('py%s' % (version,), 'none', 'any'))
        if i == 0:
            supported.append(('py%s' % (version[0]), 'none', 'any'))

    return supported

Response: Return a list of supported tags for each version specified in versions.


Instruction:
Write a Python 3 script to
remove duplicate entries from sys. path along with making them absolute.

Response:
def removeduppaths():
    """ Remove duplicate entries from sys.path along with making them
    absolute"""
    # This ensures that the initial path provided by the interpreter contains
    # only absolute pathnames, even if we're running from the build directory.
    L = []
    known_paths = set()
    for dir in sys.path:
        # Filter out duplicate paths (on case-insensitive file systems also
        # if they only differ in case); turn relative paths into absolute
        # paths.
        dir, dircase = makepath(dir)
        if not dircase in known_paths:
            L.append(dir)
            known_paths.add(dircase)
    sys.path[:] = L
    return known_paths


Instruction:
Can you tell what is the following Python 3 function doing
def addbuilddir():
    from distutils.util import get_platform
    s = "build/lib.%s-%.3s" % (get_platform(), sys.version)
    if hasattr(sys, 'gettotalrefcount'):
        s += '-pydebug'
    s = os.path.join(os.path.dirname(sys.path[-1]), s)
    sys.path.append(s)

Response: Append. build. lib. <platform > in case we re running in the build dir


Instruction:
Implement a Python 3 function for
returning a set containing all existing directory entries from sys. path

Response:
def _init_pathinfo():
    """Return a set containing all existing directory entries from sys.path"""
    d = set()
    for dir in sys.path:
        try:
            if os.path.isdir(dir):
                dir, dircase = makepath(dir)
                d.add(dircase)
        except TypeError:
            continue
    return d


Instruction:
Explain what the following Python 3 code does
def addpackage(sitedir, name, known_paths):
    if known_paths is None:
        _init_pathinfo()
        reset = 1
    else:
        reset = 0
    fullname = os.path.join(sitedir, name)
    try:
        f = open(fullname, "rU")
    except IOError:
        return
    try:
        for line in f:
            if line.startswith("#"):
                continue
            if line.startswith("import"):
                exec(line)
                continue
            line = line.rstrip()
            dir, dircase = makepath(sitedir, line)
            if not dircase in known_paths and os.path.exists(dir):
                sys.path.append(dir)
                known_paths.add(dircase)
    finally:
        f.close()
    if reset:
        known_paths = None
    return known_paths

Response: Add a new path to known_paths by combining sitedir and name or execute it if it starts with import


Instruction:
Make a summary of the following Python 3 code
def addsitedir(sitedir, known_paths=None):
    if known_paths is None:
        known_paths = _init_pathinfo()
        reset = 1
    else:
        reset = 0
    sitedir, sitedircase = makepath(sitedir)
    if not sitedircase in known_paths:
        sys.path.append(sitedir)        # Add path component
    try:
        names = os.listdir(sitedir)
    except os.error:
        return
    names.sort()
    for name in names:
        if name.endswith(os.extsep + "pth"):
            addpackage(sitedir, name, known_paths)
    if reset:
        known_paths = None
    return known_paths

Response: Add the path of the given directory to sys. path if it doesn t already exist.


Instruction:
Can you tell what is the following Python 3 function doing
def addsitepackages(known_paths, sys_prefix=sys.prefix, exec_prefix=sys.exec_prefix):
    prefixes = [os.path.join(sys_prefix, "local"), sys_prefix]
    if exec_prefix != sys_prefix:
        prefixes.append(os.path.join(exec_prefix, "local"))

    for prefix in prefixes:
        if prefix:
            if sys.platform in ('os2emx', 'riscos') or _is_jython:
                sitedirs = [os.path.join(prefix, "Lib", "site-packages")]
            elif _is_pypy:
                sitedirs = [os.path.join(prefix, 'site-packages')]
            elif sys.platform == 'darwin' and prefix == sys_prefix:

                if prefix.startswith("/System/Library/Frameworks/"): # Apple's Python

                    sitedirs = [os.path.join("/Library/Python", sys.version[:3], "site-packages"),
                                os.path.join(prefix, "Extras", "lib", "python")]

                else: # any other Python distros on OSX work this way
                    sitedirs = [os.path.join(prefix, "lib",
                                             "python" + sys.version[:3], "site-packages")]

            elif os.sep == '/':
                sitedirs = [os.path.join(prefix,
                                         "lib",
                                         "python" + sys.version[:3],
                                         "site-packages"),
                            os.path.join(prefix, "lib", "site-python"),
                            os.path.join(prefix, "python" + sys.version[:3], "lib-dynload")]
                lib64_dir = os.path.join(prefix, "lib64", "python" + sys.version[:3], "site-packages")
                if (os.path.exists(lib64_dir) and
                    os.path.realpath(lib64_dir) not in [os.path.realpath(p) for p in sitedirs]):
                    if _is_64bit:
                        sitedirs.insert(0, lib64_dir)
                    else:
                        sitedirs.append(lib64_dir)
                try:
                    # sys.getobjects only available in --with-pydebug build
                    sys.getobjects
                    sitedirs.insert(0, os.path.join(sitedirs[0], 'debug'))
                except AttributeError:
                    pass
                # Debian-specific dist-packages directories:
                sitedirs.append(os.path.join(prefix, "local/lib",
                                             "python" + sys.version[:3],
                                             "dist-packages"))
                if sys.version[0] == '2':
                    sitedirs.append(os.path.join(prefix, "lib",
                                                 "python" + sys.version[:3],
                                                 "dist-packages"))
                else:
                    sitedirs.append(os.path.join(prefix, "lib",
                                                 "python" + sys.version[0],
                                                 "dist-packages"))
                sitedirs.append(os.path.join(prefix, "lib", "dist-python"))
            else:
                sitedirs = [prefix, os.path.join(prefix, "lib", "site-packages")]
            if sys.platform == 'darwin':
                # for framework builds *only* we add the standard Apple
                # locations. Currently only per-user, but /Library and
                # /Network/Library could be added too
                if 'Python.framework' in prefix:
                    home = os.environ.get('HOME')
                    if home:
                        sitedirs.append(
                            os.path.join(home,
                                         'Library',
                                         'Python',
                                         sys.version[:3],
                                         'site-packages'))
            for sitedir in sitedirs:
                if os.path.isdir(sitedir):
                    addsitedir(sitedir, known_paths)
    return None

Response: Add site - packages and possibly site - python to sys. path.


Instruction:
Write a Python 3 function that can
check if the user site directory is safe for inclusion.

Response:
def check_enableusersite():
    """Check if user site directory is safe for inclusion

    The function tests for the command line flag (including environment var),
    process uid/gid equal to effective uid/gid.

    None: Disabled for security reasons
    False: Disabled by user (command line option)
    True: Safe and enabled
    """
    if hasattr(sys, 'flags') and getattr(sys.flags, 'no_user_site', False):
        return False

    if hasattr(os, "getuid") and hasattr(os, "geteuid"):
        # check process uid == effective uid
        if os.geteuid() != os.getuid():
            return None
    if hasattr(os, "getgid") and hasattr(os, "getegid"):
        # check process gid == effective gid
        if os.getegid() != os.getgid():
            return None

    return True


Instruction:
Write a Python 3 script for
adding a per user site - package to sys. path.

Response:
def addusersitepackages(known_paths):
    """Add a per user site-package to sys.path

    Each user has its own python directory with site-packages in the
    home directory.

    USER_BASE is the root directory for all Python versions

    USER_SITE is the user specific site-packages directory

    USER_SITE/.. can be used for data.
    """
    global USER_BASE, USER_SITE, ENABLE_USER_SITE
    env_base = os.environ.get("PYTHONUSERBASE", None)

    def joinuser(*args):
        return os.path.expanduser(os.path.join(*args))

    #if sys.platform in ('os2emx', 'riscos'):
    #    # Don't know what to put here
    #    USER_BASE = ''
    #    USER_SITE = ''
    if os.name == "nt":
        base = os.environ.get("APPDATA") or "~"
        if env_base:
            USER_BASE = env_base
        else:
            USER_BASE = joinuser(base, "Python")
        USER_SITE = os.path.join(USER_BASE,
                                 "Python" + sys.version[0] + sys.version[2],
                                 "site-packages")
    else:
        if env_base:
            USER_BASE = env_base
        else:
            USER_BASE = joinuser("~", ".local")
        USER_SITE = os.path.join(USER_BASE, "lib",
                                 "python" + sys.version[:3],
                                 "site-packages")

    if ENABLE_USER_SITE and os.path.isdir(USER_SITE):
        addsitedir(USER_SITE, known_paths)
    if ENABLE_USER_SITE:
        for dist_libdir in ("lib", "local/lib"):
            user_site = os.path.join(USER_BASE, dist_libdir,
                                     "python" + sys.version[:3],
                                     "dist-packages")
            if os.path.isdir(user_site):
                addsitedir(user_site, known_paths)
    return known_paths


Instruction:
Can you implement a function in Python 3 that
sets the environment variable BEGINLIBPATH to the path where the DLLs are located.

Response:
def setBEGINLIBPATH():
    """The OS/2 EMX port has optional extension modules that do double duty
    as DLLs (and must use the .DLL file extension) for other extensions.
    The library search path needs to be amended so these will be found
    during module import.  Use BEGINLIBPATH so that these are at the start
    of the library search path.

    """
    dllpath = os.path.join(sys.prefix, "Lib", "lib-dynload")
    libpath = os.environ['BEGINLIBPATH'].split(';')
    if libpath[-1]:
        libpath.append(dllpath)
    else:
        libpath[-1] = dllpath
    os.environ['BEGINLIBPATH'] = ';'.join(libpath)


Instruction:
Implement a function in Python 3 to
define new built - ins quit and exit.

Response:
def setquit():
    """Define new built-ins 'quit' and 'exit'.
    These are simply strings that display a hint on how to exit.

    """
    if os.sep == ':':
        eof = 'Cmd-Q'
    elif os.sep == '\\':
        eof = 'Ctrl-Z plus Return'
    else:
        eof = 'Ctrl-D (i.e. EOF)'

    class Quitter(object):
        def __init__(self, name):
            self.name = name
        def __repr__(self):
            return 'Use %s() or %s to exit' % (self.name, eof)
        def __call__(self, code=None):
            # Shells like IDLE catch the SystemExit, but listen when their
            # stdin wrapper is closed.
            try:
                sys.stdin.close()
            except:
                pass
            raise SystemExit(code)
    builtins.quit = Quitter('quit')
    builtins.exit = Quitter('exit')


Instruction:
Implement a Python 3 function for
setting copyright and credits in __builtin__.

Response:
def setcopyright():
    """Set 'copyright' and 'credits' in __builtin__"""
    builtins.copyright = _Printer("copyright", sys.copyright)
    if _is_jython:
        builtins.credits = _Printer(
            "credits",
            "Jython is maintained by the Jython developers (www.jython.org).")
    elif _is_pypy:
        builtins.credits = _Printer(
            "credits",
            "PyPy is maintained by the PyPy developers: http://pypy.org/")
    else:
        builtins.credits = _Printer("credits", """\
    Thanks to CWI, CNRI, BeOpen.com, Zope Corporation and a cast of thousands
    for supporting Python development.  See www.python.org for more information.""")
    here = os.path.dirname(os.__file__)
    builtins.license = _Printer(
        "license", "See http://www.python.org/%.3s/license.html" % sys.version,
        ["LICENSE.txt", "LICENSE"],
        [os.path.join(here, os.pardir), here, os.curdir])


Instruction:
Can you create a Python 3 function that
aliases the default encodings of the current locale to the mbcs encoding.

Response:
def aliasmbcs():
    """On Windows, some default encodings are not provided by Python,
    while they are always available as "mbcs" in each locale. Make
    them usable by aliasing to "mbcs" in such a case."""
    if sys.platform == 'win32':
        import locale, codecs
        enc = locale.getdefaultlocale()[1]
        if enc.startswith('cp'):            # "cp***" ?
            try:
                codecs.lookup(enc)
            except LookupError:
                import encodings
                encodings._cache[enc] = encodings._unknown
                encodings.aliases.aliases[enc] = 'mbcs'


Instruction:
How would you explain what the following Python 3 function does
def setencoding():
    encoding = "ascii" # Default value set by _PyUnicode_Init()
    if 0:
        # Enable to support locale aware default string encodings.
        import locale
        loc = locale.getdefaultlocale()
        if loc[1]:
            encoding = loc[1]
    if 0:
        # Enable to switch off string to Unicode coercion and implicit
        # Unicode to string conversion.
        encoding = "undefined"
    if encoding != "ascii":
        # On Non-Unicode builds this will raise an AttributeError...
        sys.setdefaultencoding(encoding)

Response: Set the string encoding used by Unicode implementation.


Instruction:
Can you write a function in Python 3 where it
forces easy_installed eggs in the global environment to get placed in sys. path after all local site packages inside the virtualenv.

Response:
def force_global_eggs_after_local_site_packages():
    """
    Force easy_installed eggs in the global environment to get placed
    in sys.path after all packages inside the virtualenv.  This
    maintains the "least surprise" result that packages in the
    virtualenv always mask global packages, never the other way
    around.

    """
    egginsert = getattr(sys, '__egginsert', 0)
    for i, path in enumerate(sys.path):
        if i > egginsert and path.startswith(sys.prefix):
            egginsert = i
    sys.__egginsert = egginsert + 1


Instruction:
Write a Python 3 function that can
adjust the special classpath sys. path entries for Jython.

Response:
def fixclasspath():
    """Adjust the special classpath sys.path entries for Jython. These
    entries should follow the base virtualenv lib directories.
    """
    paths = []
    classpaths = []
    for path in sys.path:
        if path == '__classpath__' or path.startswith('__pyclasspath__'):
            classpaths.append(path)
        else:
            paths.append(path)
    sys.path = paths
    sys.path.extend(classpaths)


Instruction:
Explain what the following Python 3 code does
def Popen_nonblocking(*args, **kwargs):
	kwargs.setdefault('close_fds', 'posix' in sys.builtin_module_names)
	kwargs.setdefault('bufsize', 1)
	proc = subprocess.Popen(*args, **kwargs)
	if proc.stdout:
		q = queue.Queue()
		t = threading.Thread(
			target=enqueue_lines,
			args=(proc.stdout, q))
		proc.stdout = q
		# thread dies with the parent
		t.daemon = True
		t.start()
	if proc.stderr:
		q = queue.Queue()
		t = threading.Thread(
			target=enqueue_lines,
			args=(proc.stderr, q))
		proc.stderr = q
		t.daemon = True
		t.start()
	return proc

Response: Return a subprocess handle with any wcsdb output streams replaced by queues of lines from that stream.


Instruction:
Can you write a function in Python 3 where it
returns True if Cython or Pyrex can be imported.

Response:
def have_pyrex():
    """
    Return True if Cython or Pyrex can be imported.
    """
    pyrex_impls = 'Cython.Distutils.build_ext', 'Pyrex.Distutils.build_ext'
    for pyrex_impl in pyrex_impls:
        try:
            # from (pyrex_impl) import build_ext
            __import__(pyrex_impl, fromlist=['build_ext']).build_ext
            return True
        except Exception:
            pass
    return False


Instruction:
Given the following Python 3 function, write the documentation
def _convert_pyx_sources_to_lang(self):
        if have_pyrex():
            # the build has Cython, so allow it to compile the .pyx files
            return
        lang = self.language or ''
        target_ext = '.cpp' if lang.lower() == 'c++' else '.c'
        sub = functools.partial(re.sub, '.pyx$', target_ext)
        self.sources = list(map(sub, self.sources))

Response:
Replace sources with. pyx extensions to sources with the target
        language extension.


Instruction:
Implement a function in Python 3 to
run the application and conserve the traceback frames.

Response:
def debug_application(self, environ, start_response):
        """Run the application and conserve the traceback frames."""
        app_iter = None
        try:
            app_iter = self.app(environ, start_response)
            for item in app_iter:
                yield item
            if hasattr(app_iter, 'close'):
                app_iter.close()
        except Exception:
            if hasattr(app_iter, 'close'):
                app_iter.close()
            traceback = get_current_traceback(skip=1, show_hidden_frames=
                                              self.show_hidden_frames,
                                              ignore_system_exceptions=True)
            for frame in traceback.frames:
                self.frames[frame.id] = frame
            self.tracebacks[traceback.id] = traceback

            try:
                start_response('500 INTERNAL SERVER ERROR', [
                    ('Content-Type', 'text/html; charset=utf-8'),
                    # Disable Chrome's XSS protection, the debug
                    # output can cause false-positives.
                    ('X-XSS-Protection', '0'),
                ])
            except Exception:
                # if we end up here there has been output but an error
                # occurred.  in that situation we can do nothing fancy any
                # more, better log something into the error log and fall
                # back gracefully.
                environ['wsgi.errors'].write(
                    'Debugging middleware caught exception in streamed '
                    'response at a point where response headers were already '
                    'sent.\n')
            else:
                yield traceback.render_full(evalex=self.evalex,
                                            secret=self.secret) \
                               .encode('utf-8', 'replace')

            traceback.log(environ['wsgi.errors'])


Instruction:
How would you code a function in Python 3 to
return a static resource from the shared folder.

Response:
def get_resource(self, request, filename):
        """Return a static resource from the shared folder."""
        filename = join(dirname(__file__), 'shared', basename(filename))
        if isfile(filename):
            mimetype = mimetypes.guess_type(filename)[0] \
                or 'application/octet-stream'
            f = open(filename, 'rb')
            try:
                return Response(f.read(), mimetype=mimetype)
            finally:
                f.close()
        return Response('Not Found', status=404)


Instruction:
Can you tell what is the following Python 3 function doing
def user_agent():
    data = {
        "installer": {"name": "pip", "version": pip.__version__},
        "python": platform.python_version(),
        "implementation": {
            "name": platform.python_implementation(),
        },
    }

    if data["implementation"]["name"] == 'CPython':
        data["implementation"]["version"] = platform.python_version()
    elif data["implementation"]["name"] == 'PyPy':
        if sys.pypy_version_info.releaselevel == 'final':
            pypy_version_info = sys.pypy_version_info[:3]
        else:
            pypy_version_info = sys.pypy_version_info
        data["implementation"]["version"] = ".".join(
            [str(x) for x in pypy_version_info]
        )
    elif data["implementation"]["name"] == 'Jython':
        # Complete Guess
        data["implementation"]["version"] = platform.python_version()
    elif data["implementation"]["name"] == 'IronPython':
        # Complete Guess
        data["implementation"]["version"] = platform.python_version()

    if sys.platform.startswith("linux"):
        distro = dict(filter(
            lambda x: x[1],
            zip(["name", "version", "id"], platform.linux_distribution()),
        ))
        libc = dict(filter(
            lambda x: x[1],
            zip(["lib", "version"], platform.libc_ver()),
        ))
        if libc:
            distro["libc"] = libc
        if distro:
            data["distro"] = distro

    if sys.platform.startswith("darwin") and platform.mac_ver()[0]:
        data["distro"] = {"name": "OS X", "version": platform.mac_ver()[0]}

    if platform.system():
        data.setdefault("system", {})["name"] = platform.system()

    if platform.release():
        data.setdefault("system", {})["release"] = platform.release()

    if platform.machine():
        data["cpu"] = platform.machine()

    return "{data[installer][name]}/{data[installer][version]} {json}".format(
        data=data,
        json=json.dumps(data, separators=(",", ":"), sort_keys=True),
    )

Response: Return a string representing the user agent.


Instruction:
Can you implement a function in Python 3 that
gets the content of a file.

Response:
def get_file_content(url, comes_from=None, session=None):
    """Gets the content of a file; it may be a filename, file: URL, or
    http: URL.  Returns (location, content).  Content is unicode."""
    if session is None:
        raise TypeError(
            "get_file_content() missing 1 required keyword argument: 'session'"
        )

    match = _scheme_re.search(url)
    if match:
        scheme = match.group(1).lower()
        if (scheme == 'file' and comes_from and
                comes_from.startswith('http')):
            raise InstallationError(
                'Requirements file %s references URL %s, which is local'
                % (comes_from, url))
        if scheme == 'file':
            path = url.split(':', 1)[1]
            path = path.replace('\\', '/')
            match = _url_slash_drive_re.match(path)
            if match:
                path = match.group(1) + ':' + path.split('|', 1)[1]
            path = urllib_parse.unquote(path)
            if path.startswith('/'):
                path = '/' + path.lstrip('/')
            url = path
        else:
            # FIXME: catch some errors
            resp = session.get(url)
            resp.raise_for_status()

            if six.PY3:
                return resp.url, resp.text
            else:
                return resp.url, resp.content
    try:
        with open(url) as f:
            content = f.read()
    except IOError as exc:
        raise InstallationError(
            'Could not open requirements file: %s' % str(exc)
        )
    return url, content


Instruction:
Write a Python 3 script for
returning true if the name looks like a URL

Response:
def is_url(name):
    """Returns true if the name looks like a URL"""
    if ':' not in name:
        return False
    scheme = name.split(':', 1)[0].lower()
    return scheme in ['http', 'https', 'file', 'ftp'] + vcs.all_schemes


Instruction:
Make a summary of the following Python 3 code
def unpack_file_url(link, location, download_dir=None):

    link_path = url_to_path(link.url_without_fragment)

    # If it's a url to a local directory
    if os.path.isdir(link_path):
        if os.path.isdir(location):
            rmtree(location)
        shutil.copytree(link_path, location, symlinks=True)
        if download_dir:
            logger.info('Link is a directory, ignoring download_dir')
        return

    # if link has a hash, let's confirm it matches
    if link.hash:
        link_path_hash = _get_hash_from_file(link_path, link)
        _check_hash(link_path_hash, link)

    # If a download dir is specified, is the file already there and valid?
    already_downloaded_path = None
    if download_dir:
        already_downloaded_path = _check_download_dir(link, download_dir)

    if already_downloaded_path:
        from_path = already_downloaded_path
    else:
        from_path = link_path

    content_type = mimetypes.guess_type(from_path)[0]

    # unpack the archive to the build dir location. even when only downloading
    # archives, they have to be unpacked to parse dependencies
    unpack_file(from_path, location, content_type, link)

    # a download dir is specified and not already downloaded
    if download_dir and not already_downloaded_path:
        _copy_file(from_path, download_dir, content_type, link)

Response: Unpack a link into the build dir location.


Instruction:
Implement a Python 3 function for
downloading a link url into temp_dir using provided session

Response:
def _download_http_url(link, session, temp_dir):
    """Download link url into temp_dir using provided session"""
    target_url = link.url.split('#', 1)[0]
    try:
        resp = session.get(
            target_url,
            # We use Accept-Encoding: identity here because requests
            # defaults to accepting compressed responses. This breaks in
            # a variety of ways depending on how the server is configured.
            # - Some servers will notice that the file isn't a compressible
            #   file and will leave the file alone and with an empty
            #   Content-Encoding
            # - Some servers will notice that the file is already
            #   compressed and will leave the file alone and will add a
            #   Content-Encoding: gzip header
            # - Some servers won't notice anything at all and will take
            #   a file that's already been compressed and compress it again
            #   and set the Content-Encoding: gzip header
            # By setting this to request only the identity encoding We're
            # hoping to eliminate the third case. Hopefully there does not
            # exist a server which when given a file will notice it is
            # already compressed and that you're not asking for a
            # compressed file and will then decompress it before sending
            # because if that's the case I don't think it'll ever be
            # possible to make this work.
            headers={"Accept-Encoding": "identity"},
            stream=True,
        )
        resp.raise_for_status()
    except requests.HTTPError as exc:
        logger.critical(
            "HTTP error %s while getting %s", exc.response.status_code, link,
        )
        raise

    content_type = resp.headers.get('content-type', '')
    filename = link.filename  # fallback
    # Have a look at the Content-Disposition header for a better guess
    content_disposition = resp.headers.get('content-disposition')
    if content_disposition:
        type, params = cgi.parse_header(content_disposition)
        # We use ``or`` here because we don't want to use an "empty" value
        # from the filename param.
        filename = params.get('filename') or filename
    ext = splitext(filename)[1]
    if not ext:
        ext = mimetypes.guess_extension(content_type)
        if ext:
            filename += ext
    if not ext and link.url != resp.url:
        ext = os.path.splitext(resp.url)[1]
        if ext:
            filename += ext
    file_path = os.path.join(temp_dir, filename)
    with open(file_path, 'wb') as content_file:
        _download_url(resp, link, content_file)
    return file_path, content_type


Instruction:
Explain what the following Python 3 code does
def _check_download_dir(link, download_dir):
    download_path = os.path.join(download_dir, link.filename)
    if os.path.exists(download_path):
        # If already downloaded, does its hash match?
        logger.info('File was already downloaded %s', download_path)
        if link.hash:
            download_hash = _get_hash_from_file(download_path, link)
            try:
                _check_hash(download_hash, link)
            except HashMismatch:
                logger.warning(
                    'Previously-downloaded file %s has bad hash, '
                    're-downloading.',
                    download_path
                )
                os.unlink(download_path)
                return None
        return download_path
    return None

Response: Check download_dir for previously downloaded file with correct hash and return its path else None.


Instruction:
How would you implement a function in Python 3 that
handles currency format subdirectives.

Response:
def currencyFormat(_context, code, symbol, format,
                       currency_digits=True, decimal_quantization=True,
                       name=''):
        """Handle currencyFormat subdirectives."""
        _context.action(
            discriminator=('currency', name, code),
            callable=_register_currency,
            args=(name, code, symbol, format, currency_digits,
                  decimal_quantization)
            )


Instruction:
How would you explain what the following Python 3 function does
def print_results(distributions, list_all_files):
    results_printed = False
    for dist in distributions:
        results_printed = True
        logger.info("---")
        logger.info("Metadata-Version: %s" % dist.get('metadata-version'))
        logger.info("Name: %s" % dist['name'])
        logger.info("Version: %s" % dist['version'])
        logger.info("Summary: %s" % dist.get('summary'))
        logger.info("Home-page: %s" % dist.get('home-page'))
        logger.info("Author: %s" % dist.get('author'))
        logger.info("Author-email: %s" % dist.get('author-email'))
        logger.info("License: %s" % dist.get('license'))
        logger.info("Location: %s" % dist['location'])
        logger.info("Requires: %s" % ', '.join(dist['requires']))
        if list_all_files:
            logger.info("Files:")
            if dist['files'] is not None:
                for line in dist['files']:
                    logger.info("  %s" % line.strip())
            else:
                logger.info("Cannot locate installed-files.txt")
        if 'entry_points' in dist:
            logger.info("Entry-points:")
            for line in dist['entry_points']:
                logger.info("  %s" % line.strip())
    return results_printed

Response: Print the informations from installed distributions found.


Instruction:
Write a Python 3 function that can
decode the data passed in and potentially flush the decoder.

Response:
def _decode(self, data, decode_content, flush_decoder):
        """
        Decode the data passed in and potentially flush the decoder.
        """
        try:
            if decode_content and self._decoder:
                data = self._decoder.decompress(data)
        except (IOError, zlib.error) as e:
            content_encoding = self.headers.get('content-encoding', '').lower()
            raise DecodeError(
                "Received response with content-encoding: %s, but "
                "failed to decode it." % content_encoding, e)

        if flush_decoder and decode_content and self._decoder:
            buf = self._decoder.decompress(binary_type())
            data += buf + self._decoder.flush()

        return data


Instruction:
Can you generate the documentation for the following Python 3 function
def read(self, amt=None, decode_content=None, cache_content=False):
        self._init_decoder()
        if decode_content is None:
            decode_content = self.decode_content

        if self._fp is None:
            return

        flush_decoder = False

        try:
            try:
                if amt is None:
                    # cStringIO doesn't like amt=None
                    data = self._fp.read()
                    flush_decoder = True
                else:
                    cache_content = False
                    data = self._fp.read(amt)
                    if amt != 0 and not data:  # Platform-specific: Buggy versions of Python.
                        # Close the connection when no data is returned
                        #
                        # This is redundant to what httplib/http.client _should_
                        # already do.  However, versions of python released before
                        # December 15, 2012 (http://bugs.python.org/issue16298) do
                        # not properly close the connection in all cases. There is
                        # no harm in redundantly calling close.
                        self._fp.close()
                        flush_decoder = True

            except SocketTimeout:
                # FIXME: Ideally we'd like to include the url in the ReadTimeoutError but
                # there is yet no clean way to get at it from this context.
                raise ReadTimeoutError(self._pool, None, 'Read timed out.')

            except BaseSSLError as e:
                # FIXME: Is there a better way to differentiate between SSLErrors?
                if 'read operation timed out' not in str(e):  # Defensive:
                    # This shouldn't happen but just in case we're missing an edge
                    # case, let's avoid swallowing SSL errors.
                    raise

                raise ReadTimeoutError(self._pool, None, 'Read timed out.')

            except HTTPException as e:
                # This includes IncompleteRead.
                raise ProtocolError('Connection broken: %r' % e, e)

            self._fp_bytes_read += len(data)

            data = self._decode(data, decode_content, flush_decoder)

            if cache_content:
                self._body = data

            return data

        finally:
            if self._original_response and self._original_response.isclosed():
                self.release_conn()

Response: Read the content of the response and return it as a dict.


Instruction:
How would you explain what the following Python 3 function does
def _default_template_ctx_processor():
    reqctx = _request_ctx_stack.top
    appctx = _app_ctx_stack.top
    rv = {}
    if appctx is not None:
        rv['g'] = appctx.g
    if reqctx is not None:
        rv['request'] = reqctx.request
        rv['session'] = reqctx.session
    return rv

Response: Default template context processor.


Instruction:
Explain what the following Python 3 code does
def _render(template, context, app):
    rv = template.render(context)
    template_rendered.send(app, template=template, context=context)
    return rv

Response: Renders the template and fires the signal


Instruction:
How would you explain what the following Python 3 function does
def render_template(template_name_or_list, **context):
    ctx = _app_ctx_stack.top
    ctx.app.update_template_context(context)
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
                   context, ctx.app)

Response: Renders a template from the template folder with the given context.


Instruction:
Here you have a function in Python 3, explain what it does
def render_template_string(source, **context):
    ctx = _app_ctx_stack.top
    ctx.app.update_template_context(context)
    return _render(ctx.app.jinja_env.from_string(source),
                   context, ctx.app)

Response: Renders a template from the given source code string with the given context.


Instruction:
Create a Python 3 function for
using parse_version from pkg_resources or distutils as available.

Response:
def parse_version(version):
    """Use parse_version from pkg_resources or distutils as available."""
    global parse_version
    try:
        from pkg_resources import parse_version
    except ImportError:
        from distutils.version import LooseVersion as parse_version
    return parse_version(version)