Instruction:
Can you tell what is the following Python 3 function doing
def _setTag(self, tag):
        '''
            _setTag - INTERNAL METHOD. Associated a given AdvancedTag to this attributes dict.

                        If bool(#tag) is True, will set the weakref to that tag.

                        Otherwise, will clear the reference

                      @param tag <AdvancedTag/None> - Either the AdvancedTag to associate, or None to clear current association
        '''
        if tag:
            self._tagRef = weakref.ref(tag)
        else:
            self._tagRef = None

Response: Internal method to set the tag of the current object.


Instruction:
Can you tell what is the following Python 3 function doing
def _handleClassAttr(self):
        '''
            _handleClassAttr - Hack to ensure "class" and "style" show up in attributes when classes are set,
                and doesn't when no classes are present on associated tag.

                TODO: I don't like this hack.
        '''
        if len(self.tag._classNames) > 0:
            dict.__setitem__(self, "class", self.tag.className)
        else:
            try:
                dict.__delitem__(self, "class")
            except:
                pass

        styleAttr = self.tag.style
        if styleAttr.isEmpty() is False:
            dict.__setitem__(self, "style", styleAttr)
        else:
            try:
                dict.__delitem__(self, "style")
            except:
                pass

Response: Handle class and style attributes.


Instruction:
How would you explain what the following Python 3 function does
def get(self, key, default=None):
        '''
            get - Gets an attribute by key with the chance to provide a default value

                @param key <str> - The key to query

                @param default <Anything> Default None - The value to return if key is not found

             @return - The value of attribute at #key, or #default if not present.
        '''

        key = key.lower()

        if key == 'class':
            return self.tag.className

        if key in ('style', 'class') or key in self.keys():
            return self[key]
        return default

Response: Get an attribute by key.


Instruction:
Write a Python 3 script to
set the tag association for this style.

Response:
def setTag(self, tag):
        '''
            setTag - Set the tag association for this style.

              This will handle the underlying weakref to the tag.

              Call setTag(None) to clear the association, otherwise setTag(tag) to associate this style to that tag.

                @param tag <AdvancedTag/None> - The new association. If None, the association is cleared, otherwise the passed tag
                    becomes associated with this style.

        '''

        if tag:
            self._tagRef = weakref.ref(tag)
        else:
            self._tagRef = None


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _ensureHtmlAttribute(self):
        '''
            _ensureHtmlAttribute - INTERNAL METHOD.
                                    Ensure the "style" attribute is present in the html attributes when
                                        is has a value, and absent when it does not.

              This requires special linkage.
        '''
        tag = self.tag

        if tag:
            styleDict = self._styleDict
            tagAttributes = tag._attributes

            # If this is called before we have _attributes setup
            if not issubclass(tagAttributes.__class__, SpecialAttributesDict):
                return

            # If we have any styles set, ensure we have the style="whatever" in the HTML representation,
            #   otherwise ensure we don't have style=""
            if not styleDict:
                tagAttributes._direct_del('style')
            else: #if 'style' not in tagAttributes.keys():
                tagAttributes._direct_set('style', self)

Response:
Internal method.
                                    Ensure the style attribute is present in the html attributes when it does not exist.


Instruction:
Given the following Python 3 function, write the documentation
def dashNameToCamelCase(dashName):
        '''
            dashNameToCamelCase - Converts a "dash name" (like padding-top) to its camel-case name ( like "paddingTop" )

            @param dashName <str> - A name containing dashes

                NOTE: This method is currently unused, but may be used in the future. kept for completeness.

            @return <str> - The camel-case form
        '''
        nameParts = dashName.split('-')
        for i in range(1, len(nameParts), 1):
            nameParts[i][0] = nameParts[i][0].upper()

        return ''.join(nameParts)

Response: Converts a dash name to its camel - case name.


Instruction:
Can you implement a function in Python 3 that
converts a camel case name to a dash - name

Response:
def camelCaseToDashName(camelCase):
        '''
            camelCaseToDashName - Convert a camel case name to a dash-name (like paddingTop to padding-top)

            @param camelCase <str> - A camel-case string

            @return <str> - A dash-name
        '''

        camelCaseList = list(camelCase)

        ret = []

        for ch in camelCaseList:
            if ch.isupper():
                ret.append('-')
                ret.append(ch.lower())
            else:
                ret.append(ch)

        return ''.join(ret)


Instruction:
Can you tell what is the following Python 3 function doing
def styleToDict(styleStr):
        '''
            getStyleDict - Gets a dictionary of style attribute/value pairs.

              NOTE: dash-names (like padding-top) are used here

            @return - OrderedDict of "style" attribute.
        '''
        styleStr = styleStr.strip()
        styles = styleStr.split(';') # Won't work for strings containing semicolon..

        styleDict = OrderedDict()
        for item in styles:
            try:
                splitIdx = item.index(':')
                name = item[:splitIdx].strip().lower()
                value = item[splitIdx+1:].strip()
                styleDict[name] = value
            except:
                continue

        return styleDict

Response: Returns a dictionary of style attribute value pairs.


Instruction:
Can you create a Python 3 function that
gets the string representation of this style

Response:
def _asStr(self):
        '''
            _asStr - Get the string representation of this style

              @return <str> - A string representation of this style (semicolon separated, key: value format)
        '''

        styleDict = self._styleDict
        if styleDict:
            return '; '.join([name + ': ' + value for name, value in styleDict.items()])
        return ''


Instruction:
Create a Python 3 function for
handling rows special attribute which differs if tagName is a textarea or frameset

Response:
def _special_value_rows(em):
    '''
        _special_value_rows - Handle "rows" special attribute, which differs if tagName is a textarea or frameset
    '''
    if em.tagName == 'textarea':
        return convertToIntRange(em.getAttribute('rows', 2), minValue=1, maxValue=None, invalidDefault=2)
    else:
        # frameset
        return em.getAttribute('rows', '')


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _special_value_cols(em):
    '''
        _special_value_cols - Handle "cols" special attribute, which differs if tagName is a textarea or frameset
    '''
    if em.tagName == 'textarea':
        return convertToIntRange(em.getAttribute('cols', 20), minValue=1, maxValue=None, invalidDefault=20)
    else:
        # frameset
        return em.getAttribute('cols', '')

Response: Handle cols special attribute which differs if tagName is a textarea or frameset


Instruction:
Create a Python 3 function for
handling autocomplete property which has different behaviour for form vs input

Response:
def _special_value_autocomplete(em):
    '''
        handle "autocomplete" property, which has different behaviour for form vs input"
    '''
    if em.tagName == 'form':
        return convertPossibleValues(em.getAttribute('autocomplete', 'on'), POSSIBLE_VALUES_ON_OFF, invalidDefault='on', emptyValue=EMPTY_IS_INVALID)
    # else: input
    return convertPossibleValues(em.getAttribute('autocomplete', ''), POSSIBLE_VALUES_ON_OFF, invalidDefault="", emptyValue='')


Instruction:
Write a Python 3 script for
handling size property which has different behaviour for input vs everything else

Response:
def _special_value_size(em):
    '''
        handle "size" property, which has different behaviour for input vs everything else
    '''
    if em.tagName == 'input':
        # TODO: "size" on an input is implemented very weirdly. Negative values are treated as invalid,
        #          A value of "0" raises an exception (and does not set HTML attribute)
        #          No upper limit.
        return convertToPositiveInt(em.getAttribute('size', 20), invalidDefault=20)
    return em.getAttribute('size', '')


Instruction:
Can you implement a function in Python 3 that
converts a value into a corresponding ata object.

Response:
def get_by_value(cls, value, type):
        """ Converts a value into a corresponding  data object.
        For files, this looks up a file DataObject by name, uuid, and/or md5.
        For other types, it creates a new DataObject.
        """
        if type == 'file':
            return cls._get_file_by_value(value)
        else:
            data_object = DataObject(data={
                'value': cls._type_cast(value, type)}, type=type)
            data_object.full_clean()
            data_object.save()
            return data_object


Instruction:
Can you create a Python 3 function that
looks up a file DataObject by name uuid and or md5.

Response:
def _get_file_by_value(cls, value):
        """Look up a file DataObject by name, uuid, and/or md5.
        """
        # Ignore any FileResource with no DataObject. This is a typical state
        # for a deleted file that has not yet been cleaned up.
        queryset = FileResource.objects.exclude(data_object__isnull=True)
        matches = FileResource.filter_by_name_or_id_or_tag_or_hash(
            value, queryset=queryset)
        if matches.count() == 0:
            raise ValidationError(
                'No file found that matches value "%s"' % value)
        elif matches.count() > 1:
            match_id_list = ['%s@%s' % (match.filename, match.get_uuid())
                             for match in matches]
            match_id_string = ('", "'.join(match_id_list))
            raise ValidationError(
                'Multiple files were found matching value "%s": "%s". '\
                'Use a more precise identifier to select just one file.' % (
                    value, match_id_string))
        return matches.first().data_object


Instruction:
Can you create a Python 3 function that
creates a path for a given file in such a way that files end up being organized and browsable by run .

Response:
def _get_run_breadcrumbs(cls, source_type, data_object, task_attempt):
        """Create a path for a given file, in such a way
        that files end up being organized and browsable by run
        """
        # We cannot generate the path unless connect to a TaskAttempt
        # and a run
        if not task_attempt:
            return []
        # If multiple tasks exist, use the original.
        task = task_attempt.tasks.earliest('datetime_created')
        if task is None:
            return []
        run = task.run
        if run is None:
            return []

        breadcrumbs = [
            run.name,
            "task-%s" % str(task.uuid)[0:8],
            "attempt-%s" % str(task_attempt.uuid)[0:8],
        ]

        # Include any ancestors if run is nested
        while run.parent is not None:
            run = run.parent
            breadcrumbs = [run.name] + breadcrumbs

        # Prepend first breadcrumb with datetime and id
        breadcrumbs[0] = "%s-%s-%s" % (
            run.datetime_created.strftime('%Y-%m-%dT%H.%M.%SZ'),
            str(run.uuid)[0:8],
            breadcrumbs[0])

        breadcrumbs = ['runs'] + breadcrumbs
        return breadcrumbs


Instruction:
Write a Python 3 function that can
filter objects by name ID or tag.

Response:
def filter_by_name_or_id_or_tag(self, query_string, queryset = None):
        """Find objects that match the identifier of form {name}@{ID}, {name},
        or @{ID}, where ID may be truncated
        """
        assert self.Model.NAME_FIELD, \
            'NAME_FIELD is missing on model %s' % self.Model.__name__
        assert self.Model.ID_FIELD, \
            'ID_FIELD is missing on model %s' % self.Model.__name__
        assert self.Model.TAG_FIELD, \
            'TAG_FIELD is missing on model %s' % self.Model.__name__

        filter_args = {}
        name, uuid, tag = self._parse_as_name_or_id_or_tag(query_string)
        if name is not None:
            filter_args[self.Model.NAME_FIELD] = name
        if uuid is not None:
            filter_args[self.Model.ID_FIELD+'__startswith'] = uuid
        if tag is not None:
            filter_args[self.Model.TAG_FIELD] = tag
        if queryset is None:
            queryset = self.Model.objects.all()
        return queryset.filter(**filter_args)


Instruction:
How would you explain what the following Python 3 function does
def save(self, *args, **kwargs):
        cls = self.__class__
        if self.pk:
            rows = cls.objects.filter(
                pk=self.pk, _change=self._change).update(
                _change=self._change + 1)
            if not rows:
                raise ConcurrentModificationError(cls.__name__, self.pk)
            self._change += 1

        count = 0
        max_retries=3
        while True:
            try:
                return super(BaseModel, self).save(*args, **kwargs)
            except django.db.utils.OperationalError:
                if count >= max_retries:
                    raise
                count += 1

Response: This method protects against two processesses concurrently modifying the object.


Instruction:
Can you implement a function in Python 3 that
checks server. ini for server type.

Response:
def get_server_type():
    """Checks server.ini for server type."""
    server_location_file = os.path.expanduser(SERVER_LOCATION_FILE)
    if not os.path.exists(server_location_file):
        raise Exception(
            "%s not found. Please run 'loom server set "
            "<servertype>' first." % server_location_file)
    config = ConfigParser.SafeConfigParser()
    config.read(server_location_file)
    server_type = config.get('server', 'type')
    return server_type


Instruction:
How would you code a function in Python 3 to
set file_data_object. file_resource. upload_status

Response:
def _set_upload_status(self, file_data_object, upload_status):
        """ Set file_data_object.file_resource.upload_status
        """
        uuid = file_data_object['uuid']
        return self.connection.update_data_object(
            uuid,
            {'uuid': uuid, 'value': { 'upload_status': upload_status}}
        )


Instruction:
How would you implement a function in Python 3 that
gives a template and a list of file_dependencies this method will substitute the UUIDs of files that are not found in the file_dependencies list with the UUIDs of the files that were imported through the server.

Response:
def _substitute_file_uuids_throughout_template(self, template, file_dependencies):
        """Anywhere in "template" that refers to a data object but does not
        give a specific UUID, if a matching file can be found in "file_dependencies",
        we will change the data object reference to use that UUID. That way templates
        have a preference to connect to files nested under their ".dependencies" over
        files that were previously imported to the server.
        """
        if not isinstance(template, dict):
            # Nothing to do if this is a reference to a previously imported template.
            return
        for input in template.get('inputs', []):
            self._substitute_file_uuids_in_input(input, file_dependencies)
        for step in template.get('steps', []):
            self._substitute_file_uuids_throughout_template(step, file_dependencies)


Instruction:
Implement a function in Python 3 to
convert command line args into a list of template inputs

Response:
def _get_inputs(self):
        """Converts command line args into a list of template inputs
        """
        # Convert file inputs to a dict, to make it easier to override
        # them with commandline inputs
        file_inputs = self._get_file_inputs()
        try:
            jsonschema.validate(file_inputs, file_input_schema)
        except jsonschema.ValidationError:
            raise SystemExit("ERROR! Input file was invalid")
        input_dict = {}
        for (channel, input_id) in file_inputs.iteritems():
            input_dict[channel] = input_id

        if self.args.inputs:
            for kv_pair in self.args.inputs:
                (channel, input_id) = kv_pair.split('=')
                input_dict[channel] = self._parse_string_to_nested_lists(
                    input_id)

        inputs = []
        for (channel, contents) in input_dict.iteritems():
            inputs.append({
                'channel': channel,
                'data': {
                    'contents': contents
                }
            })
        return inputs


Instruction:
Make a summary of the following Python 3 code
def _get_inputs(self, old_inputs):
        # Convert inputs to dict to facilitate overriding by channel name
        # Also, drop DataNode ID and keep only contents.
        input_dict = {}
        for input in old_inputs:
            # Strip out DataNode UUID and URL
            input['data'] = {'contents': input['data']['contents']}
            input_dict[input['channel']] = input

        file_inputs = self._get_file_inputs()
        try:
            jsonschema.validate(file_inputs, file_input_schema)
        except jsonschema.ValidationError:
            raise SystemExit("ERROR! User inputs file is not valid")
        for (channel, input_id) in file_inputs.iteritems():
            input_dict[channel] = {
                'channel': channel,
                'data': {'contents': input_id}
            }
        # Override with cli user inputs if specified
        if self.args.inputs:
            for kv_pair in self.args.inputs:
                (channel, input_id) = kv_pair.split('=')
                input_dict[channel] = {
                    'channel': channel,
                    'data': {
                        'contents':
                        self._parse_string_to_nested_lists(input_id)}
                }
        return input_dict.values()

Response: Converts command line args into a list of template inputs


Instruction:
Can you create a Python 3 function that
suppresses warning about untrusted SSL certificate.

Response:
def disable_insecure_request_warning():
    """Suppress warning about untrusted SSL certificate."""
    import requests
    from requests.packages.urllib3.exceptions import InsecureRequestWarning
    requests.packages.urllib3.disable_warnings(InsecureRequestWarning)


Instruction:
Can you implement a function in Python 3 that
retries sending a request to the server until the response is received.

Response:
def _make_request_to_server(self, query_function, raise_for_status=True,
                                time_limit_seconds=2, retry_delay_seconds=0.2):
        """Retry sending request until timeout or until receiving a response.
        """
        start_time = datetime.datetime.now()
        while datetime.datetime.now() - start_time < datetime.timedelta(
                0, time_limit_seconds):
            error = None
            response = None
            try:
                response = query_function()
            except requests.exceptions.ConnectionError as e:
                error = ServerConnectionError(
                    "No response from server.\n%s" % e)
            except:
                if response:
                    logger.info(response.text)
                raise
            if response is not None and raise_for_status:
                # raises requests.exceptions.HTTPError
                self._raise_for_status(response)
            if error:
                time.sleep(retry_delay_seconds)
                continue
            else:
                return response
        raise error


Instruction:
Given the following Python 3 function, write the documentation
def TaskAttemptInput(input, task_attempt):

    (data_type, mode) = _get_input_info(input)

    if data_type != 'file':
        return NoOpInput(None, task_attempt)

    if mode == 'no_gather':
        return FileInput(input['data']['contents'], task_attempt)
    else:
        assert mode.startswith('gather')
        return FileListInput(input['data']['contents'], task_attempt)

Response: Returns the correct Input class for a given input data type and gather mode


Instruction:
Given the following Python 3 function, write the documentation
def execute(task_function, *args, **kwargs):

    if get_setting('TEST_DISABLE_ASYNC_DELAY'):
        # Delay disabled, run synchronously
        logger.debug('Running function "%s" synchronously because '\
                     'TEST_DISABLE_ASYNC_DELAY is True'
                     % task_function.__name__)
        return task_function(*args, **kwargs)

    db.connections.close_all()
    task_function.delay(*args, **kwargs)

Response: Execute a task asynchronously


Instruction:
How would you code a function in Python 3 to
execute a task asynchronously after at least delay_seconds

Response:
def execute_with_delay(task_function, *args, **kwargs):
    """Run a task asynchronously after at least delay_seconds
    """
    delay = kwargs.pop('delay', 0)
    if get_setting('TEST_DISABLE_ASYNC_DELAY'):
        # Delay disabled, run synchronously
        logger.debug('Running function "%s" synchronously because '\
                     'TEST_DISABLE_ASYNC_DELAY is True'
                     % task_function.__name__)
        return task_function(*args, **kwargs)

    db.connections.close_all()
    task_function.apply_async(args=args, kwargs=kwargs, countdown=delay)


Instruction:
Make a summary of the following Python 3 code
def check_for_stalled_tasks():
    from api.models.tasks import Task
    for task in Task.objects.filter(status_is_running=True):
        if not task.is_responsive():
            task.system_error()
        if task.is_timed_out():
            task.timeout_error()

Response: Check for tasks that are no longer sending a heartbeat


Instruction:
Here you have a function in Python 3, explain what it does
def check_for_missed_cleanup():
    if get_setting('PRESERVE_ALL'):
        return
    from api.models.tasks import TaskAttempt
    if get_setting('PRESERVE_ON_FAILURE'):
        for task_attempt in TaskAttempt.objects.filter(
                status_is_running=False).filter(
                    status_is_cleaned_up=False).exclude(
                        status_is_failed=True):
            task_attempt.cleanup()
    else:
        for task_attempt in TaskAttempt.objects.filter(
                status_is_running=False).filter(status_is_cleaned_up=False):
            task_attempt.cleanup()

Response: Check for TaskAttempts that were never cleaned up


Instruction:
Here you have a function in Python 3, explain what it does
def execute_with_retries(retryable_function,
                         retryable_errors,
                         logger,
                         human_readable_action_name='Action',
                         nonretryable_errors=None):
    """This attempts to execute "retryable_function" with exponential backoff
    on delay time.
    10 retries adds up to about 34 minutes total delay before the last attempt.
    "human_readable_action_name" is an option input to customize retry message.
    """
    max_retries = 10
    attempt = 0
    if not nonretryable_errors:
        nonretryable_errors = ()
    while True:
        try:
            return retryable_function()
        except tuple(nonretryable_errors):
            raise
        except tuple(retryable_errors) as e:
            attempt += 1
            if attempt > max_retries:
                raise
            # Exponentional backoff on retry delay as suggested by
            # https://cloud.google.com/storage/docs/exponential-backoff
            delay = 2**attempt + random.random()
            logger.info('"%s" failed with error "%s". '\
                        'Retry number %s of %s in %s seconds'
                        % (human_readable_action_name, str(e),
                           attempt, max_retries, delay))
            time.sleep(delay)

Response: Execute retryable_function with exponential backoff on delay time.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def export_file(self, data_object, destination_directory=None,
                    destination_filename=None, retry=False,
                    export_metadata=False, export_raw_file=True):
        """Export a file from Loom to some file storage location.
        Default destination_directory is cwd. Default destination_filename is the
        filename from the file data object associated with the given file_id.
        """
        if not destination_directory:
            destination_directory = os.getcwd()

        # We get filename from the dataobject
        if not destination_filename:
            destination_filename = data_object['value']['filename']

        destination_file_url = os.path.join(destination_directory,
                                            destination_filename)

        logger.info('Exporting file %s@%s ...' % (
            data_object['value']['filename'],
            data_object['uuid']))

        if export_raw_file:
            destination = File(
                destination_file_url, self.storage_settings, retry=retry)
            if destination.exists():
                raise FileAlreadyExistsError(
                    'File already exists at %s' % destination_file_url)
            logger.info('...copying file to %s' % (
                destination.get_url()))

            # Copy from the first file location
            file_resource = data_object.get('value')
            md5 = file_resource.get('md5')
            source_url = data_object['value']['file_url']
            File(source_url, self.storage_settings, retry=retry).copy_to(
                destination, expected_md5=md5)
            data_object['value'] = self._create_new_file_resource(
                data_object['value'], destination.get_url())
        else:
            logger.info('...skipping raw file')

        if export_metadata:
            data_object['value'].pop('link', None)
            data_object['value'].pop('upload_status', None)
            destination_metadata_url = os.path.join(
                destination_file_url + '.metadata.yaml')
            logger.info('...writing metadata to %s' % destination_metadata_url)
            metadata = yaml.safe_dump(data_object, default_flow_style=False)
            metadata_file = File(destination_metadata_url,
                                 self.storage_settings, retry=retry)
            metadata_file.write(metadata)
        else:
            logger.info('...skipping metadata')

        logger.info('...finished file export')

Response: Export a file from Loom to some file storage location.


Instruction:
Write a Python 3 script to
like urlparse except it assumes that the scheme is specified

Response:
def _urlparse(path):
    """Like urlparse except it assumes 'file://' if no scheme is specified
    """
    url = urlparse.urlparse(path)
    _validate_url(url)
    if not url.scheme or url.scheme == 'file://':
        # Normalize path, and set scheme to "file" if missing
        path = os.path.abspath(
            os.path.expanduser(path))
        url = urlparse.urlparse('file://'+path)
    return url


Instruction:
Can you generate the documentation for the following Python 3 function
def Copier(source, destination):

    if source.type == 'local' and destination.type == 'local':
        return LocalCopier(source, destination)
    elif source.type == 'local' and destination.type == 'google_storage':
        return Local2GoogleStorageCopier(source, destination)
    elif source.type == 'google_storage' and destination.type == 'local':
        return GoogleStorage2LocalCopier(source, destination)
    elif source.type == 'google_storage' and destination.type == 'google_storage':
        return GoogleStorageCopier(source, destination)
    else:
        raise FileUtilsError('Could not find method to copy from source '\
                        '"%s" to destination "%s".' % (source, destination))

Response: Factory method to select the right copier for a given source and destination.


Instruction:
Explain what the following Python 3 code does
def create_from_data_channel(cls, data_channel):
        gather_depth = cls._get_gather_depth(data_channel)

        generator = InputSetGeneratorNode()
        for (data_path, data_node) in data_channel.get_ready_data_nodes(
                [], gather_depth):
            flat_data_node = data_node.flattened_clone(save=False)
            input_item = InputItem(
                flat_data_node, data_channel.channel,
                data_channel.as_channel, mode=data_channel.mode)
            generator._add_input_item(data_path, input_item)
        return generator

Response: Create a corresponding InputSetGenerator from a data channel.


Instruction:
Implement a Python 3 function for
returning the correct Output class for a given task_attempt.

Response:
def TaskAttemptOutput(output, task_attempt):
    """Returns the correct Output class for a given
    data type, source type, and scatter mode
    """

    (data_type, mode, source_type) = _get_output_info(output)

    if data_type == 'file':
        if mode == 'scatter':
            assert source_type in ['filenames', 'glob'], \
                'source type "%s" not allowed' % source_type
            if source_type == 'filenames':
                return FileListScatterOutput(output, task_attempt)
            return GlobScatterOutput(output, task_attempt)
        else:
            assert mode == 'no_scatter'
            assert source_type == 'filename', \
                'source type "%s" not allowed' % source_type
            return FileOutput(output, task_attempt)
    else:  # data_type is non-file
        if mode == 'scatter':
            assert source_type in [
                'filename', 'filenames', 'glob', 'stream'], \
                'source type "%s" not allowed' % source_type
            if source_type == 'filename':
                return FileContentsScatterOutput(output, task_attempt)
            if source_type == 'filenames':
                return FileListContentsScatterOutput(output, task_attempt)
            if source_type == 'glob':
                return GlobContentsScatterOutput(output, task_attempt)
            assert source_type == 'stream'
            return StreamScatterOutput(output, task_attempt)
        else:
            assert mode == 'no_scatter'
            assert source_type in ['filename', 'stream'], \
                'source type "%s" not allowed' % source_type
            if source_type == 'filename':
                return FileContentsOutput(output, task_attempt)
            assert source_type == 'stream'
            return StreamOutput(output, task_attempt)


Instruction:
How would you code a function in Python 3 to
add a new leaf node at the given index with the given data_object.

Response:
def add_leaf(self, index, data_object, save=False):
        """Adds a new leaf node at the given index with the given data_object
        """
        assert self.type == data_object.type, 'data type mismatch'
        if self._get_child_by_index(index) is not None:
            raise NodeAlreadyExistsError(
                'Leaf data node already exists at this index')
        else:
            data_node = DataNode(
                parent=self,
                index=index,
                data_object=data_object,
                type=self.type)
            if save:
                data_node.full_clean()
                data_node.save()
            self._add_unsaved_child(data_node)
            return data_node


Instruction:
Write a Python 3 function for
returning a list of entries only for existing nodes with is_ready == True.

Response:
def get_ready_data_nodes(self, seed_path, gather_depth):
        """Returns a list [(path1,data_node1),...]
        with entries only for existing nodes with DataObjects where is_ready==True.
        Missing nodes or those with non-ready or non-existing data are ignored.
        """
        try:
            seed_node = self.get_node(seed_path)
        except MissingBranchError:
            return []
        all_paths = seed_node._get_all_paths(seed_path, gather_depth)
        ready_data_nodes = []
        for path in all_paths:
            if self.is_ready(data_path=path):
                ready_data_nodes.append((path, self.get_node(path)))
        return ready_data_nodes


Instruction:
How would you explain what the following Python 3 function does
def _check_index(self, index):
        if self.degree is None:
            raise UnknownDegreeError(
                'Cannot access child DataNode on a parent with degree of None. '\
                'Set the degree on the parent first.')
        if index < 0 or index >= self.degree:
            raise IndexOutOfRangeError(
                'Out of range index %s. DataNode parent has degree %s, so index '\
                'should be in the range 0 to %s' % (
                    index, self.degree, self.degree-1))

Response: Verify that the given index is consistent with the degree of the node.


Instruction:
Can you generate the documentation for the following Python 3 function
def on_gcloud_vm():
    r = None
    try:
        r = requests.get('http://metadata.google.internal')
    except requests.ConnectionError:
        return False

    try:
        if r.headers['Metadata-Flavor'] == 'Google' and \
           r.headers['Server'] == 'Metadata Server for VM':
            return True
    except KeyError:
        return False

Response: Determines if we re running on a GCE instance.


Instruction:
Can you generate the documentation for the following Python 3 function
def get_cheapest_instance_type(cores, memory):

    pricelist = get_gcloud_pricelist()

    # Filter out preemptible, shared-CPU, and non-US instance types
    us_instance_types = {k: v for k, v in pricelist.items()
                         if k.startswith('CP-COMPUTEENGINE-VMIMAGE-')
                         and not k.endswith('-PREEMPTIBLE')
                         and 'us' in v and v['cores'] != 'shared'}

    # Convert to array and add keys (instance type names) as type names
    price_array = []
    for key in us_instance_types:
        value = us_instance_types[key]
        value.update({'name': key.replace(
            'CP-COMPUTEENGINE-VMIMAGE-', '').lower()})
        price_array.append(value)

    # Sort by price in US
    price_array.sort(None, lambda x: x['us'])

    # Look for an instance type that satisfies requested
    # cores and memory; first will be cheapest
    for instance_type in price_array:
        if int(instance_type['cores']) >= int(cores) \
           and float(instance_type['memory']) >= float(memory):
            print instance_type['name']
            return instance_type['name']

    # No instance type found that can fulfill requested cores and memory
    raise Exception('No instance type found with at least %d cores '
                    'and %f GB of RAM.' % (cores, memory))

Response:
Determine the cheapest instance type given a minimum
    number of cores and minimum amount of RAM.


Instruction:
How would you implement a function in Python 3 that
retrieves latest pricelist from Google Cloud or use cached copy if not reachable.

Response:
def get_gcloud_pricelist():
    """Retrieve latest pricelist from Google Cloud, or use
    cached copy if not reachable.
    """
    try:
        r = requests.get('http://cloudpricingcalculator.appspot.com'
                         '/static/data/pricelist.json')
        content = json.loads(r.content)
    except ConnectionError:
        logger.warning(
            "Couldn't get updated pricelist from "
            "http://cloudpricingcalculator.appspot.com"
            "/static/data/pricelist.json. Falling back to cached "
            "copy, but prices may be out of date.")
        with open('gcloudpricelist.json') as infile:
            content = json.load(infile)

    pricelist = content['gcp_price_list']
    return pricelist


Instruction:
Here you have a function in Python 3, explain what it does
def _get_base_name(hostname, step_name, attempt_id, max_length):
    max_length = int(max_length)
    if len(hostname)+len(step_name)+MIN_TASK_ID_CHARS+2 > max_length:
        # round with ceil/floor such that extra char goes to hostname if odd
        hostname_chars = int(math.ceil(
            (max_length-MIN_TASK_ID_CHARS-2)/float(2)))
        step_name_chars = int(math.floor(
            (max_length-MIN_TASK_ID_CHARS-2)/float(2)))
        hostname = hostname[:hostname_chars]
        step_name = step_name[:step_name_chars]
    name_base = '-'.join([hostname, step_name, attempt_id])
    return _sanitize_instance_name(name_base, max_length)

Response:
Create a base name for the worker instance that will run the specified
    task run attempt from this server.


Instruction:
Given the following Python 3 function, write the documentation
def _sanitize_instance_name(name, max_length):
    name = str(name).lower()                # make all letters lowercase
    name = re.sub(r'[^-a-z0-9]', '', name)  # remove invalid characters
    # remove non-lowercase letters from the beginning
    name = re.sub(r'^[^a-z]+', '', name)
    name = name[:max_length]
    name = re.sub(r'-+$', '', name)         # remove hyphens from the end
    return name

Response:
Instance names must start with a lowercase letter.
    All following characters must be a dash, lowercase letter,
    or digit.


Instruction:
Can you generate the documentation for the following Python 3 function
def is_valid(self, max_age=None):
        ''' Determines if the cache files have expired, or if it is still valid '''

        if max_age is None:
            max_age = self.cache_max_age

        if os.path.isfile(self.cache_path_cache):
            mod_time = os.path.getmtime(self.cache_path_cache)
            current_time = time()
            if (mod_time + max_age) > current_time:
                return True

        return False

Response: Determines if the cache files have expired or still valid.


Instruction:
How would you explain what the following Python 3 function does
def get_all_data_from_cache(self, filename=''):
        ''' Reads the JSON inventory from the cache file. Returns Python dictionary. '''

        data = ''
        if not filename:
            filename = self.cache_path_cache
        with open(filename, 'r') as cache:
            data = cache.read()
        return json.loads(data)

Response: Reads the JSON inventory from the cache file. Returns Python dictionary.


Instruction:
Make a summary of the following Python 3 code
def write_to_cache(self, data, filename=''):
        ''' Writes data to file as JSON.  Returns True. '''
        if not filename:
            filename = self.cache_path_cache
        json_data = json.dumps(data)
        with open(filename, 'w') as cache:
            cache.write(json_data)
        return True

Response: Writes data to file as JSON. Returns True.


Instruction:
Create a Python 3 function for
reading the settings from the gce. ini file and returns a ConfigParser object with the values set as attributes.

Response:
def get_config(self):
        """
        Reads the settings from the gce.ini file.

        Populates a SafeConfigParser object with defaults and
        attempts to read an .ini-style configuration from the filename
        specified in GCE_INI_PATH. If the environment variable is
        not present, the filename defaults to gce.ini in the current
        working directory.
        """
        gce_ini_default_path = os.path.join(
            os.path.dirname(os.path.realpath(__file__)), "gce.ini")
        gce_ini_path = os.environ.get('GCE_INI_PATH', gce_ini_default_path)

        # Create a ConfigParser.
        # This provides empty defaults to each key, so that environment
        # variable configuration (as opposed to INI configuration) is able
        # to work.
        config = ConfigParser.SafeConfigParser(defaults={
            'gce_service_account_email_address': '',
            'gce_service_account_pem_file_path': '',
            'gce_project_id': '',
            'libcloud_secrets': '',
            'inventory_ip_type': '',
            'cache_path': '~/.ansible/tmp',
            'cache_max_age': '300'
        })
        if 'gce' not in config.sections():
            config.add_section('gce')
        if 'inventory' not in config.sections():
            config.add_section('inventory')
        if 'cache' not in config.sections():
            config.add_section('cache')

        config.read(gce_ini_path)

        #########
        # Section added for processing ini settings
        #########

        # Set the instance_states filter based on config file options
        self.instance_states = []
        if config.has_option('gce', 'instance_states'):
            states = config.get('gce', 'instance_states')
            # Ignore if instance_states is an empty string.
            if states:
                self.instance_states = states.split(',')

        # Caching
        cache_path = config.get('cache', 'cache_path')
        cache_max_age = config.getint('cache', 'cache_max_age')
        # TOOD(supertom): support project-specific caches
        cache_name = 'ansible-gce.cache'
        self.cache = CloudInventoryCache(cache_path=cache_path,
                                         cache_max_age=cache_max_age,
                                         cache_name=cache_name)
        return config


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_inventory_options(self):
        ip_type = self.config.get('inventory', 'inventory_ip_type')
        # If the appropriate environment variables are set, they override
        # other configuration
        ip_type = os.environ.get('INVENTORY_IP_TYPE', ip_type)
        return ip_type

Response:
Determine inventory options. Environment variables always
        take precedence over configuration files.


Instruction:
Can you implement a function in Python 3 that
determines the GCE authorization settings and return a libcloud driver.

Response:
def get_gce_driver(self):
        """Determine the GCE authorization settings and return a
        libcloud driver.
        """
        # Attempt to get GCE params from a configuration file, if one
        # exists.
        secrets_path = self.config.get('gce', 'libcloud_secrets')
        secrets_found = False
        try:
            import secrets
            args = list(getattr(secrets, 'GCE_PARAMS', []))
            kwargs = getattr(secrets, 'GCE_KEYWORD_PARAMS', {})
            secrets_found = True
        except:
            pass

        if not secrets_found and secrets_path:
            if not secrets_path.endswith('secrets.py'):
                err = "Must specify libcloud secrets file as "
                err += "/absolute/path/to/secrets.py"
                sys.exit(err)
            sys.path.append(os.path.dirname(secrets_path))
            try:
                import secrets
                args = list(getattr(secrets, 'GCE_PARAMS', []))
                kwargs = getattr(secrets, 'GCE_KEYWORD_PARAMS', {})
                secrets_found = True
            except:
                pass
        if not secrets_found:
            args = [
                self.config.get('gce','gce_service_account_email_address'),
                self.config.get('gce','gce_service_account_pem_file_path')
            ]
            kwargs = {'project': self.config.get('gce', 'gce_project_id')}

        # If the appropriate environment variables are set, they override
        # other configuration; process those into our args and kwargs.
        args[0] = os.environ.get('GCE_EMAIL', args[0])
        args[1] = os.environ.get('GCE_PEM_FILE_PATH', args[1])
        kwargs['project'] = os.environ.get('GCE_PROJECT', kwargs['project'])

        # Retrieve and return the GCE driver.
        gce = get_driver(Provider.GCE)(*args, **kwargs)
        gce.connection.user_agent_append(
            '%s/%s' % (USER_AGENT_PRODUCT, USER_AGENT_VERSION),
        )
        return gce


Instruction:
Create a Python 3 function for
returning a list of comma separated zones parsed from the GCE_ZONE environment variable.

Response:
def parse_env_zones(self):
        '''returns a list of comma separated zones parsed from the GCE_ZONE environment variable.
        If provided, this will be used to filter the results of the grouped_instances call'''
        import csv
        reader = csv.reader([os.environ.get('GCE_ZONE',"")], skipinitialspace=True)
        zones = [r for r in reader]
        return [z for z in zones[0]]


Instruction:
Can you implement a function in Python 3 that
commands line argument processing

Response:
def parse_cli_args(self):
        ''' Command line argument processing '''

        parser = argparse.ArgumentParser(
                description='Produce an Ansible Inventory file based on GCE')
        parser.add_argument('--list', action='store_true', default=True,
                           help='List instances (default: True)')
        parser.add_argument('--host', action='store',
                           help='Get all information about an instance')
        parser.add_argument('--pretty', action='store_true', default=False,
                           help='Pretty format (default: False)')
        parser.add_argument(
            '--refresh-cache', action='store_true', default=False,
            help='Force refresh of cache by making API requests (default: False - use cache files)')
        self.args = parser.parse_args()


Instruction:
Implement a function in Python 3 to
load inventory from JSON on disk.

Response:
def load_inventory_from_cache(self):
        ''' Loads inventory from JSON on disk. '''

        try:
            self.inventory = self.cache.get_all_data_from_cache()
            hosts = self.inventory['_meta']['hostvars']
        except Exception as e:
            print(
                "Invalid inventory file %s.  Please rebuild with -refresh-cache option."
                % (self.cache.cache_path_cache))
            raise


Instruction:
How would you explain what the following Python 3 function does
def do_api_calls_update_cache(self):
        ''' Do API calls and save data in cache. '''
        zones = self.parse_env_zones()
        data = self.group_instances(zones)
        self.cache.write_to_cache(data)
        self.inventory = data

Response: Do API calls and save data in cache.


Instruction:
Given the following Python 3 function, write the documentation
def group_instances(self, zones=None):
        '''Group all instances'''
        groups = {}
        meta = {}
        meta["hostvars"] = {}

        for node in self.list_nodes():

            # This check filters on the desired instance states defined in the
            # config file with the instance_states config option.
            #
            # If the instance_states list is _empty_ then _ALL_ states are returned.
            #
            # If the instance_states list is _populated_ then check the current
            # state against the instance_states list
            if self.instance_states and not node.extra['status'] in self.instance_states:
                continue

            name = node.name

            meta["hostvars"][name] = self.node_to_dict(node)

            zone = node.extra['zone'].name

            # To avoid making multiple requests per zone
            # we list all nodes and then filter the results
            if zones and zone not in zones:
                continue

            if zone in groups: groups[zone].append(name)
            else: groups[zone] = [name]

            tags = node.extra['tags']
            for t in tags:
                if t.startswith('group-'):
                    tag = t[6:]
                else:
                    tag = 'tag_%s' % t
                if tag in groups: groups[tag].append(name)
                else: groups[tag] = [name]

            net = node.extra['networkInterfaces'][0]['network'].split('/')[-1]
            net = 'network_%s' % net
            if net in groups: groups[net].append(name)
            else: groups[net] = [name]

            machine_type = node.size
            if machine_type in groups: groups[machine_type].append(name)
            else: groups[machine_type] = [name]

            image = node.image and node.image or 'persistent_disk'
            if image in groups: groups[image].append(name)
            else: groups[image] = [name]

            status = node.extra['status']
            stat = 'status_%s' % status.lower()
            if stat in groups: groups[stat].append(name)
            else: groups[stat] = [name]

        groups["_meta"] = meta

        return groups

Response: Group all instances in the cluster by their state.


Instruction:
Can you tell what is the following Python 3 function doing
def json_format_dict(self, data, pretty=False):
        ''' Converts a dict to a JSON object and dumps it as a formatted
        string '''

        if pretty:
            return json.dumps(data, sort_keys=True, indent=2)
        else:
            return json.dumps(data)

Response:
Converts a dict to a JSON object and dumps it as a formatted
            string


Instruction:
Write a Python 3 script for
streaming stdout and stderr from the task container to this process s stdout and stderr respectively.

Response:
def _stream_docker_logs(self):
        """Stream stdout and stderr from the task container to this
        process's stdout and stderr, respectively.
        """
        thread = threading.Thread(target=self._stderr_stream_worker)
        thread.start()
        for line in self.docker_client.logs(self.container, stdout=True,
                                            stderr=False, stream=True):
            sys.stdout.write(line)
        thread.join()


Instruction:
How would you explain what the following Python 3 function does
def to_internal_value(self, data):
        converted_data = _convert_template_id_to_dict(data)
        return super(TemplateSerializer, self)\
            .to_internal_value(converted_data)

Response: Convert the template ID string to a dictionary.


Instruction:
Write a Python 3 script for
comparing two identifier (for pre-release/build components).

Response:
def identifier_cmp(a, b):
    """Compare two identifier (for pre-release/build components)."""

    a_cmp, a_is_int = _to_int(a)
    b_cmp, b_is_int = _to_int(b)

    if a_is_int and b_is_int:
        # Numeric identifiers are compared as integers
        return base_cmp(a_cmp, b_cmp)
    elif a_is_int:
        # Numeric identifiers have lower precedence
        return -1
    elif b_is_int:
        return 1
    else:
        # Non-numeric identifiers are compared lexicographically
        return base_cmp(a_cmp, b_cmp)


Instruction:
Write a Python 3 script for
comparing two list of identifiers.

Response:
def identifier_list_cmp(a, b):
    """Compare two identifier list (pre-release/build components).

    The rule is:
        - Identifiers are paired between lists
        - They are compared from left to right
        - If all first identifiers match, the longest list is greater.

    >>> identifier_list_cmp(['1', '2'], ['1', '2'])
    0
    >>> identifier_list_cmp(['1', '2a'], ['1', '2b'])
    -1
    >>> identifier_list_cmp(['1'], ['1', '2'])
    -1
    """
    identifier_pairs = zip(a, b)
    for id_a, id_b in identifier_pairs:
        cmp_res = identifier_cmp(id_a, id_b)
        if cmp_res != 0:
            return cmp_res
    # alpha1.3 < alpha1.3.1
    return base_cmp(len(a), len(b))


Instruction:
Can you create a Python 3 function that
coerces an arbitrary version string into a semver - compatible version.

Response:
def coerce(cls, version_string, partial=False):
        """Coerce an arbitrary version string into a semver-compatible one.

        The rule is:
        - If not enough components, fill minor/patch with zeroes; unless
          partial=True
        - If more than 3 dot-separated components, extra components are "build"
          data. If some "build" data already appeared, append it to the
          extra components

        Examples:
            >>> Version.coerce('0.1')
            Version(0, 1, 0)
            >>> Version.coerce('0.1.2.3')
            Version(0, 1, 2, (), ('3',))
            >>> Version.coerce('0.1.2.3+4')
            Version(0, 1, 2, (), ('3', '4'))
            >>> Version.coerce('0.1+2-3+4_5')
            Version(0, 1, 0, (), ('2-3', '4-5'))
        """
        base_re = re.compile(r'^\d+(?:\.\d+(?:\.\d+)?)?')

        match = base_re.match(version_string)
        if not match:
            raise ValueError(
                "Version string lacks a numerical component: %r"
                % version_string
            )

        version = version_string[:match.end()]
        if not partial:
            # We need a not-partial version.
            while version.count('.') < 2:
                version += '.0'

        if match.end() == len(version_string):
            return Version(version, partial=partial)

        rest = version_string[match.end():]

        # Cleanup the 'rest'
        rest = re.sub(r'[^a-zA-Z0-9+.-]', '-', rest)

        if rest[0] == '+':
            # A 'build' component
            prerelease = ''
            build = rest[1:]
        elif rest[0] == '.':
            # An extra version component, probably 'build'
            prerelease = ''
            build = rest[1:]
        elif rest[0] == '-':
            rest = rest[1:]
            if '+' in rest:
                prerelease, build = rest.split('+', 1)
            else:
                prerelease, build = rest, ''
        elif '+' in rest:
            prerelease, build = rest.split('+', 1)
        else:
            prerelease, build = rest, ''

        build = build.replace('+', '.')

        if prerelease:
            version = '%s-%s' % (version, prerelease)
        if build:
            version = '%s+%s' % (version, build)

        return cls(version, partial=partial)


Instruction:
Write a Python 3 script to
parse a version string into a tuple of version numbers.

Response:
def parse(cls, version_string, partial=False, coerce=False):
        """Parse a version string into a Version() object.

        Args:
            version_string (str), the version string to parse
            partial (bool), whether to accept incomplete input
            coerce (bool), whether to try to map the passed in string into a
                valid Version.
        """
        if not version_string:
            raise ValueError('Invalid empty version string: %r' % version_string)

        if partial:
            version_re = cls.partial_version_re
        else:
            version_re = cls.version_re

        match = version_re.match(version_string)
        if not match:
            raise ValueError('Invalid version string: %r' % version_string)

        major, minor, patch, prerelease, build = match.groups()

        if _has_leading_zero(major):
            raise ValueError("Invalid leading zero in major: %r" % version_string)
        if _has_leading_zero(minor):
            raise ValueError("Invalid leading zero in minor: %r" % version_string)
        if _has_leading_zero(patch):
            raise ValueError("Invalid leading zero in patch: %r" % version_string)

        major = int(major)
        minor = cls._coerce(minor, partial)
        patch = cls._coerce(patch, partial)

        if prerelease is None:
            if partial and (build is None):
                # No build info, strip here
                return (major, minor, patch, None, None)
            else:
                prerelease = ()
        elif prerelease == '':
            prerelease = ()
        else:
            prerelease = tuple(prerelease.split('.'))
            cls._validate_identifiers(prerelease, allow_leading_zeroes=False)

        if build is None:
            if partial:
                build = None
            else:
                build = ()
        elif build == '':
            build = ()
        else:
            build = tuple(build.split('.'))
            cls._validate_identifiers(build, allow_leading_zeroes=True)

        return (major, minor, patch, prerelease, build)


Instruction:
Write a Python 3 script for
returning a list of 5 - tuple of comparison methods.

Response:
def _comparison_functions(cls, partial=False):
        """Retrieve comparison methods to apply on version components.

        This is a private API.

        Args:
            partial (bool): whether to provide 'partial' or 'strict' matching.

        Returns:
            5-tuple of cmp-like functions.
        """

        def prerelease_cmp(a, b):
            """Compare prerelease components.

            Special rule: a version without prerelease component has higher
            precedence than one with a prerelease component.
            """
            if a and b:
                return identifier_list_cmp(a, b)
            elif a:
                # Versions with prerelease field have lower precedence
                return -1
            elif b:
                return 1
            else:
                return 0

        def build_cmp(a, b):
            """Compare build metadata.

            Special rule: there is no ordering on build metadata.
            """
            if a == b:
                return 0
            else:
                return NotImplemented

        def make_optional(orig_cmp_fun):
            """Convert a cmp-like function to consider 'None == *'."""
            @functools.wraps(orig_cmp_fun)
            def alt_cmp_fun(a, b):
                if a is None or b is None:
                    return 0
                return orig_cmp_fun(a, b)

            return alt_cmp_fun

        if partial:
            return [
                base_cmp,  # Major is still mandatory
                make_optional(base_cmp),
                make_optional(base_cmp),
                make_optional(prerelease_cmp),
                make_optional(build_cmp),
            ]
        else:
            return [
                base_cmp,
                base_cmp,
                base_cmp,
                prerelease_cmp,
                build_cmp,
            ]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def __compare_helper(self, other, condition, notimpl_target):
        if not isinstance(other, self.__class__):
            return NotImplemented

        cmp_res = self.__cmp__(other)
        if cmp_res is NotImplemented:
            return notimpl_target

        return condition(cmp_res)

Response: Helper function for comparison.


Instruction:
Can you create a Python 3 function that
checks whether a Version satisfies the Spec.

Response:
def match(self, version):
        """Check whether a Version satisfies the Spec."""
        return all(spec.match(version) for spec in self.specs)


Instruction:
How would you explain what the following Python 3 function does
def select(self, versions):
        options = list(self.filter(versions))
        if options:
            return max(options)
        return None

Response: Select the best compatible version among an iterable of options.


Instruction:
Here you have a function in Python 3, explain what it does
def deconstruct(self):
        name, path, args, kwargs = super(VersionField, self).deconstruct()
        kwargs['partial'] = self.partial
        kwargs['coerce'] = self.coerce
        return name, path, args, kwargs

Response: Handle django. db. migrations.


Instruction:
Here you have a function in Python 3, explain what it does
def to_python(self, value):
        if value is None or value == '':
            return value
        if isinstance(value, base.Version):
            return value
        if self.coerce:
            return base.Version.coerce(value, partial=self.partial)
        else:
            return base.Version(value, partial=self.partial)

Response: Converts any value to a base. Version field.


Instruction:
Write a Python 3 script to
convert any value to a base. Spec field.

Response:
def to_python(self, value):
        """Converts any value to a base.Spec field."""
        if value is None or value == '':
            return value
        if isinstance(value, base.Spec):
            return value
        return base.Spec(value)


Instruction:
Here you have a function in Python 3, explain what it does
def move_left(self):
        self.at(ardrone.at.pcmd, True, -self.speed, 0, 0, 0)

Response: Make the drone move left.


Instruction:
Can you write a function in Python 3 where it
makes the drone move right.

Response:
def move_right(self):
        """Make the drone move right."""
        self.at(ardrone.at.pcmd, True, self.speed, 0, 0, 0)


Instruction:
Can you implement a function in Python 3 that
makes the drone rise upwards.

Response:
def move_up(self):
        """Make the drone rise upwards."""
        self.at(ardrone.at.pcmd, True, 0, 0, self.speed, 0)


Instruction:
How would you explain what the following Python 3 function does
def move_down(self):
        self.at(ardrone.at.pcmd, True, 0, 0, -self.speed, 0)

Response: Make the drone decent downwards.


Instruction:
Can you generate the documentation for the following Python 3 function
def move_forward(self):
        self.at(ardrone.at.pcmd, True, 0, -self.speed, 0, 0)

Response: Make the drone move forward.


Instruction:
Write a Python 3 function that can
make the drone move backwards.

Response:
def move_backward(self):
        """Make the drone move backwards."""
        self.at(ardrone.at.pcmd, True, 0, self.speed, 0, 0)


Instruction:
How would you explain what the following Python 3 function does
def turn_left(self):
        self.at(ardrone.at.pcmd, True, 0, 0, 0, -self.speed)

Response: Make the drone rotate left.


Instruction:
Write a Python 3 function for
making the drone rotate right.

Response:
def turn_right(self):
        """Make the drone rotate right."""
        self.at(ardrone.at.pcmd, True, 0, 0, 0, self.speed)


Instruction:
Create a Python 3 function for
toggling the drone s emergency state.

Response:
def reset(self):
        """Toggle the drone's emergency state."""
        self.at(ardrone.at.ref, False, True)
        time.sleep(0.1)
        self.at(ardrone.at.ref, False, False)


Instruction:
Explain what the following Python 3 code does
def at(self, cmd, *args, **kwargs):
        with self.lock:
            self.com_watchdog_timer.cancel()
            cmd(self.host, self.sequence, *args, **kwargs)
            self.sequence += 1
            self.com_watchdog_timer = threading.Timer(self.timer, self.commwdg)
            self.com_watchdog_timer.start()

Response: Wrapper for the low level at commands.


Instruction:
Write a Python 3 function for
making the drone move.

Response:
def move(self, lr, fb, vv, va):
        """Makes the drone move (translate/rotate).

        Parameters:
        lr -- left-right tilt: float [-1..1] negative: left, positive: right
        fb -- front-back tilt: float [-1..1] negative: forwards, positive:
            backwards
        vv -- vertical speed: float [-1..1] negative: go down, positive: rise
        va -- angular speed: float [-1..1] negative: spin left, positive: spin
            right"""
        self.at(ardrone.at.pcmd, True, lr, fb, vv, va)


Instruction:
How would you explain what the following Python 3 function does
def pcmd(host, seq, progressive, lr, fb, vv, va):
    p = 1 if progressive else 0
    at(host, 'PCMD', seq, [p, float(lr), float(fb), float(vv), float(va)])

Response: This function is used to make the drone move at the specified sequence number.


Instruction:
Can you write a function in Python 3 where it
sets configuration parameters of the drone.

Response:
def config(host, seq, option, value):
    """Set configuration parameters of the drone."""
    at(host, 'CONFIG', seq, [str(option), str(value)])


Instruction:
Write a Python 3 script for
sending control values directly to the engines overriding control loops.

Response:
def pwm(host, seq, m1, m2, m3, m4):
    """
    Sends control values directly to the engines, overriding control loops.

    Parameters:
    seq -- sequence number
    m1 -- Integer: front left command
    m2 -- Integer: front right command
    m3 -- Integer: back right command
    m4 -- Integer: back left command
    """
    at(host, 'PWM', seq, [m1, m2, m3, m4])


Instruction:
How would you explain what the following Python 3 function does
def led(host, seq, anim, f, d):
    at(host, 'LED', seq, [anim, float(f), d])

Response: Control the drones LED.


Instruction:
Here you have a function in Python 3, explain what it does
def anim(host, seq, anim, d):
    at(host, 'ANIM', seq, [anim, d])

Response: Execute an animation on the sequence seq.


Instruction:
Given the following Python 3 function, write the documentation
def at(host, command, seq, params):
    params_str = []
    for p in params:
        if type(p) == int:
            params_str.append('{:d}'.format(p))
        elif type(p) == float:
            params_str.append('{:d}'.format(f2i(p)))
        elif type(p) == str:
            params_str.append('"{:s}"'.format(p))
    msg = 'AT*{:s}={:d},{:s}\r'.format(command, seq, ','.join(params_str))
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.sendto(msg.encode(), (host, ardrone.constant.COMMAND_PORT))

Response: This function sends a command to the ardrone.


Instruction:
How would you explain what the following Python 3 function does
def decode(packet):
    offset = 0

    _ = struct.unpack_from('IIII', packet, offset)
    s = _[1]
    state = dict()
    state['fly']                 = s       & 1 # FLY MASK : (0) ardrone is landed, (1) ardrone is flying
    state['video']               = s >>  1 & 1 # VIDEO MASK : (0) video disable, (1) video enable
    state['vision']              = s >>  2 & 1 # VISION MASK : (0) vision disable, (1) vision enable
    state['control']             = s >>  3 & 1 # CONTROL ALGO (0) euler angles control, (1) angular speed control
    state['altitude']            = s >>  4 & 1 # ALTITUDE CONTROL ALGO : (0) altitude control inactive (1) altitude control active
    state['user_feedback_start'] = s >>  5 & 1 # USER feedback : Start button state
    state['command']             = s >>  6 & 1 # Control command ACK : (0) None, (1) one received
    state['fw_file']             = s >>  7 & 1 # Firmware file is good (1)
    state['fw_ver']              = s >>  8 & 1 # Firmware update is newer (1)
    state['fw_upd']              = s >>  9 & 1 # Firmware update is ongoing (1)
    state['navdata_demo']        = s >> 10 & 1 # Navdata demo : (0) All navdata, (1) only navdata demo
    state['navdata_bootstrap']   = s >> 11 & 1 # Navdata bootstrap : (0) options sent in all or demo mode, (1) no navdata options sent
    state['motors']              = s >> 12 & 1 # Motor status : (0) Ok, (1) Motors problem
    state['com_lost']            = s >> 13 & 1 # Communication lost : (1) com problem, (0) Com is ok
    state['vbat_low']            = s >> 15 & 1 # VBat low : (1) too low, (0) Ok
    state['user_el']             = s >> 16 & 1 # User Emergency Landing : (1) User EL is ON, (0) User EL is OFF
    state['timer_elapsed']       = s >> 17 & 1 # Timer elapsed : (1) elapsed, (0) not elapsed
    state['angles_out_of_range'] = s >> 19 & 1 # Angles : (0) Ok, (1) out of range
    state['ultrasound']          = s >> 21 & 1 # Ultrasonic sensor : (0) Ok, (1) deaf
    state['cutout']              = s >> 22 & 1 # Cutout system detection : (0) Not detected, (1) detected
    state['pic_version']         = s >> 23 & 1 # PIC Version number OK : (0) a bad version number, (1) version number is OK
    state['atcodec_thread_on']   = s >> 24 & 1 # ATCodec thread ON : (0) thread OFF (1) thread ON
    state['navdata_thread_on']   = s >> 25 & 1 # Navdata thread ON : (0) thread OFF (1) thread ON
    state['video_thread_on']     = s >> 26 & 1 # Video thread ON : (0) thread OFF (1) thread ON
    state['acq_thread_on']       = s >> 27 & 1 # Acquisition thread ON : (0) thread OFF (1) thread ON
    state['ctrl_watchdog']       = s >> 28 & 1 # CTRL watchdog : (1) delay in control execution (> 5ms), (0) control is well scheduled
    state['adc_watchdog']        = s >> 29 & 1 # ADC Watchdog : (1) delay in uart2 dsr (> 5ms), (0) uart2 is good
    state['com_watchdog']        = s >> 30 & 1 # Communication Watchdog : (1) com problem, (0) Com is ok
    state['emergency']           = s >> 31 & 1 # Emergency landing : (0) no emergency, (1) emergency

    data = dict()
    data['state'] = state
    data['header'] = _[0]
    data['sequence'] = _[2]
    data['vision'] = _[3]

    offset += struct.calcsize('IIII')

    demo_fields = [
        'ctrl_state',
        'battery',
        'theta',
        'phi',
        'psi',
        'altitude',
        'vx',
        'vy',
        'vz',
        'num_frames'
    ]
    angles = ['theta', 'phi', 'psi']
    while True:
        try:
            id_nr, size = struct.unpack_from('HH', packet, offset)
            offset += struct.calcsize('HH')
        except struct.error:
            break

        values = []
        for i in range(size - struct.calcsize('HH')):
            values.append(struct.unpack_from('c', packet, offset)[0])
            offset += struct.calcsize('c')

        if id_nr == 0:
            values = struct.unpack_from('IIfffIfffI', b''.join(values))
            demo = dict(zip(demo_fields, values))
            for a in angles:
                demo[a] = int(demo[a] / 1000)

            data['demo'] = demo

    return data

Response: Decode a navdata packet into a base - 16 state.


Instruction:
Create a Python 3 function to
save the current state of the object to a file.

Response:
def tofile(self, filename, format = 'ascii'):
        """Save VTK data to file.
        """
        if not common.is_string(filename):
            raise TypeError('argument filename must be string but got %s'%(type(filename)))
        if format not in ['ascii','binary']:
            raise TypeError('argument format must be ascii | binary')
        filename = filename.strip()
        if not filename:
            raise ValueError('filename must be non-empty string')
        if filename[-4:]!='.vtk':
            filename += '.vtk'
        f = open(filename,'wb')
        f.write(self.to_string(format))
        f.close()


Instruction:
How would you implement a function in Python 3 that
audits the provided customer s subscription against stripe and returns a pair of zebra_resource_id and zebra_resource_id.

Response:
def audit_customer_subscription(customer, unknown=True):
    """
    Audits the provided customer's subscription against stripe and returns a pair
    that contains a boolean and a result type.

    Default result types can be found in zebra.conf.defaults and can be
    overridden in your project's settings.
    """
    if (hasattr(customer, 'suspended') and customer.suspended):
        result = AUDIT_RESULTS['suspended']
    else:
        if hasattr(customer, 'subscription'):
            try:
                result = AUDIT_RESULTS[customer.subscription.status]
            except KeyError, err:
                # TODO should this be a more specific exception class?
                raise Exception("Unable to locate a result set for \
subscription status %s in ZEBRA_AUDIT_RESULTS") % str(err)
        else:
            result = AUDIT_RESULTS['no_subscription']
    return result


Instruction:
Create a Python 3 function for
using VtkData ( <filename >.

Response:
def polydata_fromfile(f, self):
    """Use VtkData(<filename>)."""
    points = []
    data = dict(vertices=[], lines=[], polygons=[], triangle_strips=[])
    l = common._getline(f).decode('ascii')
    k,n,datatype = [s.strip().lower() for s in l.split(' ')]
    if k!='points':
        raise ValueError('expected points but got %s'%(repr(k)))
    n = int(n)
    assert datatype in ['bit','unsigned_char','char','unsigned_short','short','unsigned_int','int','unsigned_long','long','float','double'],repr(datatype)

    log.debug('\tgetting %s points'%n)
    while len(points) < 3*n:
        l = common._getline(f).decode('ascii')
        points += map(eval,l.split(' '))
    assert len(points)==3*n
    while 1:
        l = common._getline(f)
        if l is None:
            break
        l = l.decode('ascii')
        sl = l.split(' ')
        k = sl[0].strip().lower()
        if k not in ['vertices','lines','polygons','triangle_strips']:
            break
        assert len(sl)==3
        n = int(sl[1])
        size = int(sl[2])
        lst = []
        while len(lst) < size:
            l = common._getline(f).decode('ascii')
            lst += map(eval, l.split(' '))
        assert len(lst)==size
        lst2 = []
        j = 0
        for i in range(n):
            lst2.append(lst[j+1:j+lst[j]+1])
            j += lst[j]+1
        data[k] = lst2

    return PolyData(points,data['vertices'], data['lines'], data['polygons'], data['triangle_strips']), l.encode()


Instruction:
Can you generate a brief explanation for the following Python 3 code
def webhooks(request):

    if request.method != "POST":
        return HttpResponse("Invalid Request.", status=400)

    json = simplejson.loads(request.POST["json"])

    if json["event"] == "recurring_payment_failed":
        zebra_webhook_recurring_payment_failed.send(sender=None, customer=_try_to_get_customer_from_customer_id(json["customer"]), full_json=json)

    elif json["event"] == "invoice_ready":
        zebra_webhook_invoice_ready.send(sender=None, customer=_try_to_get_customer_from_customer_id(json["customer"]), full_json=json)

    elif json["event"] == "recurring_payment_succeeded":
        zebra_webhook_recurring_payment_succeeded.send(sender=None, customer=_try_to_get_customer_from_customer_id(json["customer"]), full_json=json)

    elif json["event"] == "subscription_trial_ending":
        zebra_webhook_subscription_trial_ending.send(sender=None, customer=_try_to_get_customer_from_customer_id(json["customer"]), full_json=json)

    elif json["event"] == "subscription_final_payment_attempt_failed":
        zebra_webhook_subscription_final_payment_attempt_failed.send(sender=None, customer=_try_to_get_customer_from_customer_id(json["customer"]), full_json=json)

    elif json["event"] == "ping":
        zebra_webhook_subscription_ping_sent.send(sender=None)

    else:
        return HttpResponse(status=400)

    return HttpResponse(status=200)

Response: Handles all known webhooks from stripe and calls signals.


Instruction:
Can you tell what is the following Python 3 function doing
def webhooks_v2(request):
    if request.method != "POST":
        return HttpResponse("Invalid Request.", status=400)

    try:
        event_json = simplejson.loads(request.body)
    except AttributeError:
        # Backwords compatibility
        # Prior to Django 1.4, request.body was named request.raw_post_data
        event_json = simplejson.loads(request.raw_post_data)
    event_key = event_json['type'].replace('.', '_')

    if event_key in WEBHOOK_MAP:
        WEBHOOK_MAP[event_key].send(sender=None, full_json=event_json)

    return HttpResponse(status=200)

Response: Handles all known webhooks from stripe and calls signals.


Instruction:
Make a summary of the following Python 3 code
def is_number(obj):
    return isinstance(obj, (int, float, np.int_, np.float_))

Response: Check if obj is number.


Instruction:
Can you create a Python 3 function that
returns sequence of sequences.

Response:
def get_seq_seq(self,obj,default=None):
        """Return sequence of sequences."""
        if is_sequence2(obj):
            return [self.get_seq(o,default) for o in obj]
        else:
            return [self.get_seq(obj,default)]


Instruction:
Implement a function in Python 3 to
return 3 - tuple from a sequence number or number or None.

Response:
def get_3_tuple(self,obj,default=None):
        """Return 3-tuple from
        number -> (obj,default[1],default[2])
        0-sequence|None -> default
        1-sequence -> (obj[0],default[1],default[2])
        2-sequence -> (obj[0],obj[1],default[2])
        (3 or more)-sequence -> (obj[0],obj[1],obj[2])
        """
        if not (default is not None \
                and type(default) is tuple \
                and len(default)==3):
            raise ValueError('argument default must be 3-tuple|None but got %s'%(default))
        if is_sequence(obj):
            n = len(obj)
            if n>3:
                log.warning('expected 3-sequence but got %s-%s'%(n,type(obj)))
            if n>=3:
                return tuple(obj)
            log.warning('filling with default value (%s) to obtain size=3'%(default[0]))
            if default is not None:
                if n==0:
                    return default
                elif n==1:
                    return (obj[0],default[1],default[2])
                elif n==2:
                    return (obj[0],obj[1],default[2])
        elif is_number(obj) and default is not None:
            log.warning('filling with default value (%s) to obtain size=3'%(default[0]))
            return (obj,default[1],default[2])
        elif obj is None and default is not None:
            log.warning('filling with default value (%s) to obtain size=3'%(default[0]))
            return default
        raise ValueError('failed to construct 3-tuple from %s-%s'%(n,type(obj)))


Instruction:
Write a Python 3 script to
return list of 3 - tuples from a sequence of 3 - elements

Response:
def get_3_tuple_list(self,obj,default=None):
        """Return list of 3-tuples from
        sequence of a sequence,
        sequence - it is mapped to sequence of 3-sequences if possible
        number
        """
        if is_sequence2(obj):
            return [self.get_3_tuple(o,default) for o in obj]
        elif is_sequence(obj):
            return [self.get_3_tuple(obj[i:i+3],default) for i in range(0,len(obj),3)]
        else:
            return [self.get_3_tuple(obj,default)]


Instruction:
Write a Python 3 script for
returning tuple of 3 - tuples

Response:
def get_3_3_tuple(self,obj,default=None):
        """Return tuple of 3-tuples
        """
        if is_sequence2(obj):
            ret = []
            for i in range(3):
                if i<len(obj):
                    ret.append(self.get_3_tuple(obj[i],default))
                else:
                    ret.append(self.get_3_tuple(default,default))
            return tuple(ret)
        if is_sequence(obj):
            if len(obj)>9:
                log.warning('ignoring elements obj[i], i>=9')
            r = obj[:9]
            r = [self.get_3_tuple(r[j:j+3],default) for j in range(0,len(r),3)]
            if len(r)<3:
                log.warning('filling with default value (%s) to obtain size=3'%(default[0]))
            while len(r)<3:
                r.append(self.get_3_tuple(default,default))
            return tuple(r)
        log.warning('filling with default value (%s) to obtain size=3'%(default[0]))
        r1 = self.get_3_tuple(obj,default)
        r2 = self.get_3_tuple(default,default)
        r3 = self.get_3_tuple(default,default)
        return (r1,r2,r3)


Instruction:
Create a Python 3 function to
return list of 3x3 - tuples.

Response:
def get_3_3_tuple_list(self,obj,default=None):
        """Return list of 3x3-tuples.
        """
        if is_sequence3(obj):
            return [self.get_3_3_tuple(o,default) for o in obj]
        return [self.get_3_3_tuple(obj,default)]


Instruction:
Can you implement a function in Python 3 that
connects to the current application and instantiate the services.

Response:
def connect(self):
        """Iterate through the application configuration and instantiate
        the services.
        """
        requested_services = set(
            svc.lower() for svc in current_app.config.get('BOTO3_SERVICES', [])
        )

        region = current_app.config.get('BOTO3_REGION')
        sess_params = {
            'aws_access_key_id': current_app.config.get('BOTO3_ACCESS_KEY'),
            'aws_secret_access_key': current_app.config.get('BOTO3_SECRET_KEY'),
            'profile_name': current_app.config.get('BOTO3_PROFILE'),
            'region_name': region
        }
        sess = boto3.session.Session(**sess_params)

        try:
            cns = {}
            for svc in requested_services:
                # Check for optional parameters
                params = current_app.config.get(
                    'BOTO3_OPTIONAL_PARAMS', {}
                ).get(svc, {})

                # Get session params and override them with kwargs
                # `profile_name` cannot be passed to clients and resources
                kwargs = sess_params.copy()
                kwargs.update(params.get('kwargs', {}))
                del kwargs['profile_name']

                # Override the region if one is defined as an argument
                args = params.get('args', [])
                if len(args) >= 1:
                    del kwargs['region_name']

                if not(isinstance(args, list) or isinstance(args, tuple)):
                    args = [args]

                # Create resource or client
                if svc in sess.get_available_resources():
                    cns.update({svc: sess.resource(svc, *args, **kwargs)})
                else:
                    cns.update({svc: sess.client(svc, *args, **kwargs)})
        except UnknownServiceError:
            raise
        return cns


Instruction:
Can you generate a brief explanation for the following Python 3 code
def clients(self):
        clients = {}
        for k, v in self.connections.items():
            if hasattr(v.meta, 'client'):       # has boto3 resource
                clients[k] = v.meta.client
            else:                               # no boto3 resource
                clients[k] = v
        return clients

Response: Get all clients with and without associated resources


Instruction:
Given the following Python 3 function, write the documentation
def init_kerberos(app, service='HTTP', hostname=gethostname()):
    '''
    Configure the GSSAPI service name, and validate the presence of the
    appropriate principal in the kerberos keytab.

    :param app: a flask application
    :type app: flask.Flask
    :param service: GSSAPI service name
    :type service: str
    :param hostname: hostname the service runs under
    :type hostname: str
    '''
    global _SERVICE_NAME
    _SERVICE_NAME = "%s@%s" % (service, hostname)

    if 'KRB5_KTNAME' not in environ:
        app.logger.warn("Kerberos: set KRB5_KTNAME to your keytab file")
    else:
        try:
            principal = kerberos.getServerPrincipalDetails(service, hostname)
        except kerberos.KrbError as exc:
            app.logger.warn("Kerberos: %s" % exc.message[0])
        else:
            app.logger.info("Kerberos: server is %s" % principal)

Response: Initialize the Kerberos keytab.


Instruction:
How would you explain what the following Python 3 function does
def date_to_timestamp(date):
    date_tuple = date.timetuple()
    timestamp = calendar.timegm(date_tuple) * 1000
    return timestamp

Response: converts a date to unix timestamp in milliseconds


Instruction:
Make a summary of the following Python 3 code
def random_date(dt_from, dt_to):
    delta = dt_to - dt_from
    int_delta = (delta.days * 24 * 60 * 60) + delta.seconds
    random_second = randrange(int_delta)
    return dt_from + datetime.timedelta(seconds=random_second)

Response: This function will return a random datetime between two datetime objects.


Instruction:
Can you generate the documentation for the following Python 3 function
def object_to_json(obj, indent=2):
    instance_json = json.dumps(obj, indent=indent, ensure_ascii=False, cls=DjangoJSONEncoder)
    return instance_json

Response: transform object to json


Instruction:
Can you generate a brief explanation for the following Python 3 code
def qs_to_json(qs, fields=None):
    if not fields :
        fields = [f.name for f in qs.model._meta.fields]

    # сформируем список для сериализации
    objects = []
    for value_dict in qs.values(*fields):
        # сохраним порядок полей, как определено в моделе
        o = OrderedDict()
        for f in fields:
            o[f] = value_dict[f]
        objects.append(o)

    # сериализуем
    json_qs = json.dumps(objects, indent=2, ensure_ascii=False, cls=DjangoJSONEncoder)
    return json_qs

Response: transform QuerySet to json


Instruction:
Can you tell what is the following Python 3 function doing
def mongoqs_to_json(qs, fields=None):

    l = list(qs.as_pymongo())

    for element in l:
        element.pop('_cls')

    # use DjangoJSONEncoder for transform date data type to datetime
    json_qs = json.dumps(l, indent=2, ensure_ascii=False, cls=DjangoJSONEncoder)
    return json_qs

Response: transform mongoengine. QuerySet to json


Instruction:
Here you have a function in Python 3, explain what it does
def url_path(request, base_url=None, is_full=False, *args, **kwargs):
    if not base_url:
        base_url = request.path
        if is_full:
            protocol = 'https' if request.is_secure() else 'http'
            base_url = '%s://%s%s' % (protocol, request.get_host(), base_url)

    params = url_params(request, *args, **kwargs)
    url = '%s%s' % (base_url, params)
    return url

Response: Returns the full url path for the current page.


Instruction:
Write a Python 3 function for
creating string with GET - params of the current resource

Response:
def url_params(request, except_params=None, as_is=False):
    """
    create string with GET-params of request

    usage example:
        c['sort_url'] = url_params(request, except_params=('sort',))
        ...
        Лабораторный номер
    """
    if not request.GET:
        return ''
    params = []
    for key, value in request.GET.items():
        if except_params and key not in except_params:
            for v in request.GET.getlist(key):
                params.append('%s=%s' % (key, urlquote(v)))

    if as_is:
        str_params = '?' + '&'.join(params)
    else:
        str_params = '?' + '&'.join(params)
        str_params = urlquote(str_params)
    return mark_safe(str_params)


Instruction:
How would you implement a function in Python 3 that
prepares sort parameters for order.

Response:
def prepare_sort_params(params, request, sort_key='sort', revers_sort=None, except_params=None):
    """
        Prepare sort params. Add revers '-' if need.
        Params:
            params - list of sort parameters
            request
            sort_key
            revers_sort - list or set with keys that need reverse default sort direction
            except_params - GET-params that will be ignored
        Example:
            view:
                c['sort_params'] = prepare_sort_params(
                    ('order__lab_number', 'order__client__lname', 'organization', 'city', 'street', ),
                    request,
                )
            template:
                   Лабораторный номер
               or
                    {% load djutils %}
                    ...
                    {% sort_th 'order__lab_number' 'Лабораторный номер' %}

    """
    current_param, current_reversed = sort_key_process(request, sort_key)

    except_params = except_params or []
    except_params.append(sort_key)

    base_url = url_params(request, except_params=except_params, as_is=True)

    sort_params = {}
    revers_sort = revers_sort or set()
    url_connector = '?' if request.get_full_path() == request.path else "&"
    for p in params:
        sort_params[p] = {}
        if current_param and p == current_param:
            prefix = '' if current_reversed else '-'
            sort_params[p]['url'] = base_url + "%s%s=%s" % (url_connector, sort_key, prefix + current_param)
            sort_params[p]['is_reversed'] = current_reversed
            sort_params[p]['is_current'] = True
        else:
            default_direction = '-' if p in revers_sort else ''
            sort_params[p]['url'] = base_url + "%s%s=%s%s" % (url_connector, sort_key, default_direction, p)
            sort_params[p]['is_reversed'] = False
            sort_params[p]['is_current'] = False

    return sort_params


Instruction:
Create a Python 3 function for
processing sort key for sorting

Response:
def sort_key_process(request, sort_key='sort'):
    """
        process sort-parameter value (for example, "-name")
        return:
            current_param - field for sorting ("name)
            current_reversed - revers flag (True)
    """
    current = request.GET.get(sort_key)
    current_reversed = False
    current_param = None
    if current:
        mo = re.match(r'^(-?)(\w+)$', current)    # exclude first "-" (if exist)
        if mo:
            current_reversed = mo.group(1) == '-'
            current_param = mo.group(2)

    return current_param, current_reversed


Instruction:
Can you generate the documentation for the following Python 3 function
def transform_form_error(form, verbose=True):
    errors = []
    for field, err_msg in form.errors.items():
        if field == '__all__':  # general errors
            errors.append(', '.join(err_msg))
        else:                   # field errors
            field_name = field
            if verbose and field in form.fields:
                field_name = form.fields[field].label or field
            errors.append('%s: %s' % (field_name, ', '.join(err_msg)))
    return errors

Response:
transform form errors to list like
 articles


Instruction:
Here you have a function in Python 3, explain what it does
def process_date_from_to_options(options, to_datetime=False, default_dt_to=False):
    start_time = datetime.datetime.now()

    if options.get('last_week'):
        dt_from = start_time - datetime.timedelta(days=7)
        dt_to = start_time

    elif options.get('last_day'):
        dt_from = start_time - datetime.timedelta(days=1)
        dt_to = start_time

    elif options.get('last_2hours'):
        dt_from = start_time - datetime.timedelta(hours=2)
        dt_to = start_time

    else:
        from_str = options.get('from')
        if from_str:
            try:
                dt_from = iso_to_datetime(from_str)
            except:
                dt_from = iso_to_date(from_str)
        else:
            dt_from = None

        to_str = options.get('to')
        if to_str:
            try:
                dt_to = iso_to_datetime(to_str)
            except:
                dt_to = iso_to_date(to_str)
        else:
            dt_to = None

    if default_dt_to and not dt_to:
        dt_to = datetime.datetime(2100, 1, 1)

    if to_datetime:
        if isinstance(dt_from, datetime.date):
            dt_from = date_to_datetime(dt_from)
        if isinstance(dt_to, datetime.date):
            dt_to = date_to_datetime_lte(dt_to)

    return dt_from, dt_to

Response: Process date from options to options dt_to


Instruction:
Write a Python 3 script for
yielding n items from iterable into n - sized lists.

Response:
def _chunked(iterable, n):
    """
    Collect data into chunks of up to length n.
    :type iterable: Iterable[T]
    :type n: int
    :rtype: Iterator[list[T]]
    """
    it = iter(iterable)
    while True:
        chunk = list(islice(it, n))
        if chunk:
            yield chunk
        else:
            return


Instruction:
Can you implement a function in Python 3 that
gets gender information for a list of names.

Response:
def get(self, names, country_id=None, language_id=None, retheader=False):
        """
        Look up gender for a list of names.
        Can optionally refine search with locale info.
        May make multiple requests if there are more names than
        can be retrieved in one call.

        :param names: List of names.
        :type names: Iterable[str]
        :param country_id: Optional ISO 3166-1 alpha-2 country code.
        :type country_id: Optional[str]
        :param language_id: Optional ISO 639-1 language code.
        :type language_id: Optional[str]
        :param retheader: Optional
        :type retheader: Optional[boolean]
        :return:
        If retheader is False:
            List of dicts containing 'name', 'gender',
                     'probability', 'count' keys. If 'gender' is None,
                     'probability' and 'count' will be omitted.
        else:
            A dict containing 'data' and 'headers' keys.
            Data is the same as when retheader is False.
            Headers are the response header
            (a requests.structures.CaseInsensitiveDict).
            If multiple requests were made,
            the header will be from the last one.
        :rtype: Union[dict, Sequence[dict]]
        :raises GenderizeException: if API server returns HTTP error code.
        """
        responses = [
            self._get_chunk(name_chunk, country_id, language_id)
            for name_chunk
            in _chunked(names, Genderize.BATCH_SIZE)
        ]
        data = list(chain.from_iterable(
            response.data for response in responses
        ))
        if retheader:
            return {
                "data": data,
                "headers": responses[-1].headers,
            }
        else:
            return data


Instruction:
Write a Python 3 function for
looking up gender for a single name.

Response:
def get1(self, name, **kwargs):
        """
        Look up gender for a single name.
        See :py:meth:`get`.
        Doesn't support retheader option.
        """
        if 'retheader' in kwargs:
            raise GenderizeException(
                "get1() doesn't support the retheader option.")
        return self.get([name], **kwargs)[0]


Instruction:
Create a Python 3 function for
creating a proxy for a variable.

Response:
def _make_proxy(self, varname, parent=None, constructor=MlabObjectProxy):
        """Creates a proxy for a variable.

        XXX create and cache nested proxies also here.
        """
        # FIXME why not just use gensym here?
        proxy_val_name = "PROXY_VAL%d__" % self._proxy_count
        self._proxy_count += 1
        mlabraw.eval(self._session, "%s = %s;" % (proxy_val_name, varname))
        res = constructor(self, proxy_val_name, parent)
        self._proxies[proxy_val_name] = res
        return res


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _get(self, name, remove=False):
        # FIXME should this really be needed in normal operation?
        if name in self._proxies: return self._proxies[name]
        varname = name
        vartype = self._var_type(varname)
        if vartype in self._mlabraw_can_convert:
            var = mlabraw.get(self._session, varname)
            if isinstance(var, ndarray):
                if self._flatten_row_vecs and numpy.shape(var)[0] == 1:
                    var.shape = var.shape[1:2]
                elif self._flatten_col_vecs and numpy.shape(var)[1] == 1:
                    var.shape = var.shape[0:1]
                if self._array_cast:
                    var = self._array_cast(var)
        else:
            var = None
            if self._dont_proxy.get(vartype):
                # manual conversions may fail (e.g. for multidimensional
                # cell arrays), in that case just fall back on proxying.
                try:
                    var = self._manually_convert(varname, vartype)
                except MlabConversionError: pass
            if var is None:
                # we can't convert this to a python object, so we just
                # create a proxy, and don't delete the real matlab
                # reference until the proxy is garbage collected
                var = self._make_proxy(varname)
        if remove:
            mlabraw.eval(self._session, "clear('%s');" % varname)
        return var

Response: Directly access a variable in matlab space.


Instruction:
Can you tell what is the following Python 3 function doing
def open(self, visible=False):
    if self.client:
      raise MatlabConnectionError('Matlab(TM) COM client is still active. Use close to '
                      'close it')
    self.client = win32com.client.Dispatch('matlab.application')
    self.client.visible = visible

Response: Open the current COM client.


Instruction:
Given the following Python 3 function, write the documentation
def eval(self, expression, identify_erros=True):
    #print expression
    self._check_open()
    ret = self.client.Execute(expression)
    #print ret
    if identify_erros and ret.rfind('???') != -1:
      begin = ret.rfind('???') + 4
      raise MatlabError(ret[begin:])
    return ret

Response: Evaluates a matlab expression synchronously.


Instruction:
Make a summary of the following Python 3 code
def get(self, names_to_get, convert_to_numpy=True):
    self._check_open()
    single_itme = isinstance(names_to_get, (unicode, str))
    if single_itme:
      names_to_get = [names_to_get]
    ret = {}
    for name in names_to_get:
      ret[name] = self.client.GetWorkspaceData(name, 'base')
      # TODO(daniv): Do we really want to reduce dimensions like that? what if this a row vector?
      while isinstance(ret[name], (tuple, list)) and len(ret[name]) == 1:
        ret[name] = ret[name][0]
      if convert_to_numpy and isinstance(ret[name], (tuple, list)):
        ret[name] = np.array(ret[name])
    if single_itme:
      return ret.values()[0]
    return ret

Response: Loads the requested variables from the matlab com client.


Instruction:
Can you tell what is the following Python 3 function doing
def put(self, name_to_val):
    self._check_open()
    for name, val in name_to_val.iteritems():
      # First try to put data as a matrix:
      try:
        self.client.PutFullMatrix(name, 'base', val, None)
      except:
        self.client.PutWorkspaceData(name, 'base', val)

Response: Loads a dictionary of variable names into the matlab com client.