Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_queue_metadata(self, queue_name, timeout=None):
        '''
        Retrieves user-defined metadata and queue properties on the specified
        queue. Metadata is associated with the queue as name-value pairs.

        :param str queue_name:
            The name of an existing queue.
        :param int timeout:
            The server timeout, expressed in seconds.
        :return:
            A dictionary representing the queue metadata with an
            approximate_message_count int property on the dict estimating the
            number of messages in the queue.
        :rtype: a dict mapping str to str
        '''
        _validate_not_none('queue_name', queue_name)
        request = HTTPRequest()
        request.method = 'GET'
        request.host = self._get_host()
        request.path = _get_path(queue_name)
        request.query = [
            ('comp', 'metadata'),
            ('timeout', _int_to_str(timeout)),
        ]
        response = self._perform_request(request)

        return _parse_metadata_and_message_count(response)

Response: Retrieves user - defined metadata and queue properties on the specified queue.


Instruction:
Make a summary of the following Python 3 code
def set_queue_metadata(self, queue_name, metadata=None, timeout=None):
        '''
        Sets user-defined metadata on the specified queue. Metadata is
        associated with the queue as name-value pairs.

        :param str queue_name:
            The name of an existing queue.
        :param dict metadata:
            A dict containing name-value pairs to associate with the
            queue as metadata.
        :param int timeout:
            The server timeout, expressed in seconds.
        '''
        _validate_not_none('queue_name', queue_name)
        request = HTTPRequest()
        request.method = 'PUT'
        request.host = self._get_host()
        request.path = _get_path(queue_name)
        request.query = [
            ('comp', 'metadata'),
            ('timeout', _int_to_str(timeout)),
        ]
        request.headers = [('x-ms-meta-name-values', metadata)]
        self._perform_request(request)

Response: Sets user - defined metadata on the specified queue.


Instruction:
Write a Python 3 script to
return details about any stored access policies associated with the specified queue.

Response:
def get_queue_acl(self, queue_name, timeout=None):
        '''
        Returns details about any stored access policies specified on the
        queue that may be used with Shared Access Signatures.

        :param str queue_name:
            The name of an existing queue.
        :param int timeout:
            The server timeout, expressed in seconds.
        :return: A dictionary of access policies associated with the queue.
        :rtype: dict of str to :class:`~azure.storage.models.AccessPolicy`
        '''
        _validate_not_none('queue_name', queue_name)
        request = HTTPRequest()
        request.method = 'GET'
        request.host = self._get_host()
        request.path = _get_path(queue_name)
        request.query = [
            ('comp', 'acl'),
            ('timeout', _int_to_str(timeout)),
        ]
        response = self._perform_request(request)

        return _convert_xml_to_signed_identifiers(response.body)


Instruction:
Can you tell what is the following Python 3 function doing
def set_queue_acl(self, queue_name, signed_identifiers=None, timeout=None):
        '''
        Sets stored access policies for the queue that may be used with Shared
        Access Signatures.

        When you set permissions for a queue, the existing permissions are replaced.
        To update the queue’s permissions, call :func:`~get_queue_acl` to fetch
        all access policies associated with the queue, modify the access policy
        that you wish to change, and then call this function with the complete
        set of data to perform the update.

        When you establish a stored access policy on a queue, it may take up to
        30 seconds to take effect. During this interval, a shared access signature
        that is associated with the stored access policy will throw an
        :class:`AzureHttpError` until the access policy becomes active.

        :param str queue_name:
            The name of an existing queue.
        :param signed_identifiers:
            A dictionary of access policies to associate with the queue. The
            dictionary may contain up to 5 elements. An empty dictionary
            will clear the access policies set on the service.
        :type signed_identifiers: dict of str to :class:`~azure.storage.models.AccessPolicy`
        :param int timeout:
            The server timeout, expressed in seconds.
        '''
        _validate_not_none('queue_name', queue_name)
        request = HTTPRequest()
        request.method = 'PUT'
        request.host = self._get_host()
        request.path = _get_path(queue_name)
        request.query = [
            ('comp', 'acl'),
            ('timeout', _int_to_str(timeout)),
        ]
        request.body = _get_request_body(
            _convert_signed_identifiers_to_xml(signed_identifiers))
        self._perform_request(request)

Response: Sets the permissions for a queue.


Instruction:
How would you explain what the following Python 3 function does
def put_message(self, queue_name, content, visibility_timeout=None,
                    time_to_live=None, timeout=None):
        '''
        Adds a new message to the back of the message queue.

        The visibility timeout specifies the time that the message will be
        invisible. After the timeout expires, the message will become visible.
        If a visibility timeout is not specified, the default value of 0 is used.

        The message time-to-live specifies how long a message will remain in the
        queue. The message will be deleted from the queue when the time-to-live
        period expires.

        :param str queue_name:
            The name of the queue to put the message into.
        :param obj content:
            Message content. Allowed type is determined by the encode_function
            set on the service. Default is str. The encoded message can be up to
            64KB in size.
        :param int visibility_timeout:
            If not specified, the default value is 0. Specifies the
            new visibility timeout value, in seconds, relative to server time.
            The value must be larger than or equal to 0, and cannot be
            larger than 7 days. The visibility timeout of a message cannot be
            set to a value later than the expiry time. visibility_timeout
            should be set to a value smaller than the time-to-live value.
        :param int time_to_live:
            Specifies the time-to-live interval for the message, in
            seconds. The maximum time-to-live allowed is 7 days. If this
            parameter is omitted, the default time-to-live is 7 days.
        :param int timeout:
            The server timeout, expressed in seconds.
        '''
        _validate_not_none('queue_name', queue_name)
        _validate_not_none('content', content)
        request = HTTPRequest()
        request.method = 'POST'
        request.host = self._get_host()
        request.path = _get_path(queue_name, True)
        request.query = [
            ('visibilitytimeout', _to_str(visibility_timeout)),
            ('messagettl', _to_str(time_to_live)),
            ('timeout', _int_to_str(timeout))
        ]
        request.body = _get_request_body(_convert_queue_message_xml(content, self.encode_function))
        self._perform_request(request)

Response: Adds a new message into the back of the message queue.


Instruction:
Can you create a Python 3 function that
retrieves one or more messages from the front of a queue.

Response:
def get_messages(self, queue_name, num_messages=None,
                     visibility_timeout=None, timeout=None):
        '''
        Retrieves one or more messages from the front of the queue.

        When a message is retrieved from the queue, the response includes the message
        content and a pop_receipt value, which is required to delete the message.
        The message is not automatically deleted from the queue, but after it has
        been retrieved, it is not visible to other clients for the time interval
        specified by the visibility_timeout parameter.

        :param str queue_name:
            The name of the queue to get messages from.
        :param int num_messages:
            A nonzero integer value that specifies the number of
            messages to retrieve from the queue, up to a maximum of 32. If
            fewer are visible, the visible messages are returned. By default,
            a single message is retrieved from the queue with this operation.
        :param int visibility_timeout:
            Specifies the new visibility timeout value, in seconds, relative
            to server time. The new value must be larger than or equal to 1
            second, and cannot be larger than 7 days. The visibility timeout of
            a message can be set to a value later than the expiry time.
        :param int timeout:
            The server timeout, expressed in seconds.
        :return: A list of :class:`~azure.storage.queue.models.QueueMessage` objects.
        :rtype: list of :class:`~azure.storage.queue.models.QueueMessage`
        '''
        _validate_not_none('queue_name', queue_name)
        request = HTTPRequest()
        request.method = 'GET'
        request.host = self._get_host()
        request.path = _get_path(queue_name, True)
        request.query = [
            ('numofmessages', _to_str(num_messages)),
            ('visibilitytimeout', _to_str(visibility_timeout)),
            ('timeout', _int_to_str(timeout))
        ]
        response = self._perform_request(request)

        return _convert_xml_to_queue_messages(response, self.decode_function)


Instruction:
Can you tell what is the following Python 3 function doing
def clear_messages(self, queue_name, timeout=None):
        '''
        Deletes all messages from the specified queue.

        :param str queue_name:
            The name of the queue whose messages to clear.
        :param int timeout:
            The server timeout, expressed in seconds.
        '''
        _validate_not_none('queue_name', queue_name)
        request = HTTPRequest()
        request.method = 'DELETE'
        request.host = self._get_host()
        request.path = _get_path(queue_name, True)
        request.query = [('timeout', _int_to_str(timeout))]
        self._perform_request(request)

Response: Deletes all messages from the specified queue.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def update_message(self, queue_name, message_id, pop_receipt, visibility_timeout,
                       content=None, timeout=None):
        '''
        Updates the visibility timeout of a message. You can also use this
        operation to update the contents of a message.

        This operation can be used to continually extend the invisibility of a
        queue message. This functionality can be useful if you want a worker role
        to “lease” a queue message. For example, if a worker role calls get_messages
        and recognizes that it needs more time to process a message, it can
        continually extend the message’s invisibility until it is processed. If
        the worker role were to fail during processing, eventually the message
        would become visible again and another worker role could process it.

        :param str queue_name:
            The name of the queue containing the message to update.
        :param str message_id:
            The message id identifying the message to update.
        :param str pop_receipt:
            A valid pop receipt value returned from an earlier call
            to the :func:`~get_messages` or :func:`~update_message` operation.
        :param int visibility_timeout:
            Specifies the new visibility timeout value, in seconds,
            relative to server time. The new value must be larger than or equal
            to 0, and cannot be larger than 7 days. The visibility timeout of a
            message cannot be set to a value later than the expiry time. A
            message can be updated until it has been deleted or has expired.
        :param obj content:
            Message content. Allowed type is determined by the encode_function
            set on the service. Default is str.
        :param int timeout:
            The server timeout, expressed in seconds.
        :return:
            A list of :class:`~azure.storage.queue.models.QueueMessage` objects. Note that
            only time_next_visible and pop_receipt will be populated.
        :rtype: list of :class:`~azure.storage.queue.models.QueueMessage`
        '''
        _validate_not_none('queue_name', queue_name)
        _validate_not_none('message_id', message_id)
        _validate_not_none('pop_receipt', pop_receipt)
        _validate_not_none('visibility_timeout', visibility_timeout)
        request = HTTPRequest()
        request.method = 'PUT'
        request.host = self._get_host()
        request.path = _get_path(queue_name, True, message_id)
        request.query = [
            ('popreceipt', _to_str(pop_receipt)),
            ('visibilitytimeout', _int_to_str(visibility_timeout)),
            ('timeout', _int_to_str(timeout))
        ]

        if content is not None:
            request.body = _get_request_body(_convert_queue_message_xml(content, self.encode_function))

        response = self._perform_request(request)
        return _parse_queue_message_from_headers(response)

Response: Updates the visibility timeout of a message.


Instruction:
Implement a Python 3 function for
enabling or disable secure boot on the server.

Response:
def enable_secure_boot(self, secure_boot_enable):
        """Enable/Disable secure boot on the server.

        Caller needs to reset the server after issuing this command
        to bring this into effect.
        :param secure_boot_enable: True, if secure boot needs to be
               enabled for next boot, else False.
        :raises: InvalidInputError, if the validation of the input fails
        :raises: SushyError, on an error from iLO.
        """
        if not isinstance(secure_boot_enable, bool):
            msg = ('The parameter "%(parameter)s" value "%(value)s" is '
                   'invalid. Valid values are: True/False.' %
                   {'parameter': 'secure_boot_enable',
                    'value': secure_boot_enable})
            raise exception.InvalidInputError(msg)

        self._conn.patch(self.path,
                         data={'SecureBootEnable': secure_boot_enable})


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_allowed_reset_keys_values(self):
        reset_keys_action = self._get_reset_keys_action_element()

        if not reset_keys_action.allowed_values:
            LOG.warning('Could not figure out the allowed values for the '
                        'reset keys in secure boot %s', self.path)
            return set(mappings.SECUREBOOT_RESET_KEYS_MAP_REV)

        return set([mappings.SECUREBOOT_RESET_KEYS_MAP[v] for v in
                    set(mappings.SECUREBOOT_RESET_KEYS_MAP).
                    intersection(reset_keys_action.allowed_values)])

Response: Get the allowed values for resetting the system.


Instruction:
Here you have a function in Python 3, explain what it does
def reset_keys(self, target_value):
        valid_keys_resets = self.get_allowed_reset_keys_values()
        if target_value not in valid_keys_resets:
            msg = ('The parameter "%(parameter)s" value "%(target_value)s" is '
                   'invalid. Valid values are: %(valid_keys_reset_values)s' %
                   {'parameter': 'target_value', 'target_value': target_value,
                    'valid_keys_reset_values': valid_keys_resets})
            raise exception.InvalidInputError(msg)

        value = mappings.SECUREBOOT_RESET_KEYS_MAP_REV[target_value]
        target_uri = (
            self._get_reset_keys_action_element().target_uri)

        self._conn.post(target_uri, data={'ResetKeysType': value})

Response: Resets the secure boot keys.


Instruction:
Create a Python 3 function to
extract out resource properties and metadata information.

Response:
def _parse_properties(response, result_class):
    '''
    Extracts out resource properties and metadata information.
    Ignores the standard http headers.
    '''

    if response is None or response.headers is None:
        return None

    props = result_class()
    for key, value in response.headers:
        info = GET_PROPERTIES_ATTRIBUTE_MAP.get(key)
        if info:
            if info[0] is None:
                setattr(props, info[1], info[2](value))
            else:
                attr = getattr(props, info[0])
                setattr(attr, info[1], info[2](value))

    return props


Instruction:
Create a Python 3 function for
extracting name - values from response header. Filter out the standard http headers.

Response:
def _parse_response_for_dict(response):
    ''' Extracts name-values from response header. Filter out the standard
    http headers.'''

    if response is None:
        return None
    http_headers = ['server', 'date', 'location', 'host',
                    'via', 'proxy-connection', 'connection']
    return_dict = _HeaderDict()
    if response.headers:
        for name, value in response.headers:
            if not name.lower() in http_headers:
                return_dict[name] = value

    return return_dict


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _convert_xml_to_service_properties(xml):
    '''
    <?xml version="1.0" encoding="utf-8"?>
    <StorageServiceProperties>
        <Logging>
            <Version>version-number</Version>
            <Delete>true|false</Delete>
            <Read>true|false</Read>
            <Write>true|false</Write>
            <RetentionPolicy>
                <Enabled>true|false</Enabled>
                <Days>number-of-days</Days>
            </RetentionPolicy>
        </Logging>
        <HourMetrics>
            <Version>version-number</Version>
            <Enabled>true|false</Enabled>
            <IncludeAPIs>true|false</IncludeAPIs>
            <RetentionPolicy>
                <Enabled>true|false</Enabled>
                <Days>number-of-days</Days>
            </RetentionPolicy>
        </HourMetrics>
        <MinuteMetrics>
            <Version>version-number</Version>
            <Enabled>true|false</Enabled>
            <IncludeAPIs>true|false</IncludeAPIs>
            <RetentionPolicy>
                <Enabled>true|false</Enabled>
                <Days>number-of-days</Days>
            </RetentionPolicy>
        </MinuteMetrics>
        <Cors>
            <CorsRule>
                <AllowedOrigins>comma-separated-list-of-allowed-origins</AllowedOrigins>
                <AllowedMethods>comma-separated-list-of-HTTP-verb</AllowedMethods>
                <MaxAgeInSeconds>max-caching-age-in-seconds</MaxAgeInSeconds>
                <ExposedHeaders>comma-seperated-list-of-response-headers</ExposedHeaders>
                <AllowedHeaders>comma-seperated-list-of-request-headers</AllowedHeaders>
            </CorsRule>
        </Cors>
    </StorageServiceProperties>
    '''
    service_properties_element = ETree.fromstring(xml)
    service_properties = ServiceProperties()

    # Logging
    logging = service_properties_element.find('Logging')
    if logging is not None:
        service_properties.logging = Logging()
        service_properties.logging.version = logging.find('Version').text
        service_properties.logging.delete = _bool(logging.find('Delete').text)
        service_properties.logging.read = _bool(logging.find('Read').text)
        service_properties.logging.write = _bool(logging.find('Write').text)

        _convert_xml_to_retention_policy(logging.find('RetentionPolicy'),
                                            service_properties.logging.retention_policy)
    # HourMetrics
    hour_metrics_element = service_properties_element.find('HourMetrics')
    if hour_metrics_element is not None:
        service_properties.hour_metrics = Metrics()
        _convert_xml_to_metrics(hour_metrics_element, service_properties.hour_metrics)

    # MinuteMetrics
    minute_metrics_element = service_properties_element.find('MinuteMetrics')
    if minute_metrics_element is not None:
        service_properties.minute_metrics = Metrics()
        _convert_xml_to_metrics(minute_metrics_element, service_properties.minute_metrics)

    # CORS
    cors = service_properties_element.find('Cors')
    if cors is not None:
        service_properties.cors = list()
        for rule in cors.findall('CorsRule'):
            allowed_origins = rule.find('AllowedOrigins').text.split(',')

            allowed_methods = rule.find('AllowedMethods').text.split(',')

            max_age_in_seconds = int(rule.find('MaxAgeInSeconds').text)

            cors_rule = CorsRule(allowed_origins, allowed_methods, max_age_in_seconds)

            exposed_headers = rule.find('ExposedHeaders').text
            if exposed_headers is not None:
                cors_rule.exposed_headers = exposed_headers.split(',')

            allowed_headers = rule.find('AllowedHeaders').text
            if allowed_headers is not None:
                cors_rule.allowed_headers = allowed_headers.split(',')

            service_properties.cors.append(cors_rule)

    # Target version
    target_version = service_properties_element.find('DefaultServiceVersion')
    if target_version is not None:
        service_properties.target_version = target_version.text

    return service_properties

Response: Convert XML to ServiceProperties.


Instruction:
Write a Python 3 script to
add an insert entity operation to the BatchService.

Response:
def insert_entity(self, entity):
        '''
        Adds an insert entity operation to the batch. See
        :func:`~azure.storage.table.tableservice.TableService.insert_entity` for more
        information on inserts.

        The operation will not be executed until the batch is committed.

        :param entity:
            The entity to insert. Could be a dict or an entity object.
            Must contain a PartitionKey and a RowKey.
        :type entity: a dict or :class:`azure.storage.table.models.Entity`
        '''
        request = _insert_entity(entity)
        self._add_to_batch(entity['PartitionKey'], entity['RowKey'], request)


Instruction:
How would you implement a function in Python 3 that
updates the metadata of an entity in the table.

Response:
def update_entity(self, entity, if_match='*'):
        '''
        Adds an update entity operation to the batch. See
        :func:`~azure.storage.table.tableservice.TableService.update_entity` for more
        information on updates.

        The operation will not be executed until the batch is committed.

        :param entity:
            The entity to update. Could be a dict or an entity object.
            Must contain a PartitionKey and a RowKey.
        :type entity: a dict or :class:`azure.storage.table.models.Entity`
        :param str if_match:
            The client may specify the ETag for the entity on the
            request in order to compare to the ETag maintained by the service
            for the purpose of optimistic concurrency. The update operation
            will be performed only if the ETag sent by the client matches the
            value maintained by the server, indicating that the entity has
            not been modified since it was retrieved by the client. To force
            an unconditional update, set If-Match to the wildcard character (*).
        '''
        request = _update_entity(entity, if_match)
        self._add_to_batch(entity['PartitionKey'], entity['RowKey'], request)


Instruction:
Write a Python 3 script for
adding a merge entity operation to the BatchService.

Response:
def merge_entity(self, entity, if_match='*'):
        '''
        Adds a merge entity operation to the batch. See
        :func:`~azure.storage.table.tableservice.TableService.merge_entity` for more
        information on merges.

        The operation will not be executed until the batch is committed.

        :param entity:
            The entity to merge. Could be a dict or an entity object.
            Must contain a PartitionKey and a RowKey.
        :type entity: a dict or :class:`azure.storage.table.models.Entity`
        :param str if_match:
            The client may specify the ETag for the entity on the
            request in order to compare to the ETag maintained by the service
            for the purpose of optimistic concurrency. The merge operation
            will be performed only if the ETag sent by the client matches the
            value maintained by the server, indicating that the entity has
            not been modified since it was retrieved by the client. To force
            an unconditional merge, set If-Match to the wildcard character (*).
        '''
        request = _merge_entity(entity, if_match)
        self._add_to_batch(entity['PartitionKey'], entity['RowKey'], request)


Instruction:
Can you generate the documentation for the following Python 3 function
def insert_or_replace_entity(self, entity):
        '''
        Adds an insert or replace entity operation to the batch. See
        :func:`~azure.storage.table.tableservice.TableService.insert_or_replace_entity` for more
        information on insert or replace operations.

        The operation will not be executed until the batch is committed.

        :param entity:
            The entity to insert or replace. Could be a dict or an entity object.
            Must contain a PartitionKey and a RowKey.
        :type entity: a dict or :class:`azure.storage.table.models.Entity`
       '''
        request = _insert_or_replace_entity(entity)
        self._add_to_batch(entity['PartitionKey'], entity['RowKey'], request)

Response: Adds an insert or replace entity operation to the batch.


Instruction:
Can you implement a function in Python 3 that
adds an insert or merge entity operation to the Batch.

Response:
def insert_or_merge_entity(self, entity):
        '''
        Adds an insert or merge entity operation to the batch. See
        :func:`~azure.storage.table.tableservice.TableService.insert_or_merge_entity` for more
        information on insert or merge operations.

        The operation will not be executed until the batch is committed.

        :param entity:
            The entity to insert or merge. Could be a dict or an entity object.
            Must contain a PartitionKey and a RowKey.
        :type entity: a dict or :class:`azure.storage.table.models.Entity`
        '''
        request = _insert_or_merge_entity(entity)
        self._add_to_batch(entity['PartitionKey'], entity['RowKey'], request)


Instruction:
Write a Python 3 script to
update iscsi data :param data: default iscsi config data

Response:
def update_iscsi_settings(self, iscsi_data):
        """Update iscsi data

        :param data: default iscsi config data
        """
        self._conn.patch(self.path, data=iscsi_data)


Instruction:
Here you have a function in Python 3, explain what it does
def array_controllers(self):
        return array_controller.HPEArrayControllerCollection(
            self._conn, utils.get_subresource_path_by(
                self, ['Links', 'ArrayControllers']),
            redfish_version=self.redfish_version)

Response: This property gets the list of instances of array controllers


Instruction:
How would you explain what the following Python 3 function does
def wait_for_operation_to_complete(
        has_operation_completed, retries=10, delay_bw_retries=5,
        delay_before_attempts=10, failover_exc=exception.IloError,
        failover_msg=("Operation did not complete even after multiple "
                      "attempts."), is_silent_loop_exit=False):
    """Attempts the provided operation for a specified number of times.

    If it runs out of attempts, then it raises an exception. On success,
    it breaks out of the loop.
    :param has_operation_completed: the method to retry and it needs to return
                                    a boolean to indicate success or failure.
    :param retries: number of times the operation to be (re)tried, default 10
    :param delay_bw_retries: delay in seconds before attempting after
                             each failure, default 5.
    :param delay_before_attempts: delay in seconds before beginning any
                                  operation attempt, default 10.
    :param failover_exc: the exception which gets raised in case of failure
                         upon exhausting all the attempts, default IloError.
    :param failover_msg: the msg with which the exception gets raised in case
                         of failure upon exhausting all the attempts.
    :param is_silent_loop_exit: decides if exception has to be raised (in case
                                of failure upon exhausting all the attempts)
                                or not, default False (will be raised).
    :raises: failover_exc, if failure happens even after all the attempts,
             default IloError.
    """
    retry_count = retries
    # Delay for ``delay_before_attempts`` secs, before beginning any attempt
    time.sleep(delay_before_attempts)

    while retry_count:
        try:
            LOG.debug("Calling '%s', retries left: %d",
                      has_operation_completed.__name__, retry_count)
            if has_operation_completed():
                break
        except exception.IloError:
            pass
        time.sleep(delay_bw_retries)
        retry_count -= 1
    else:
        LOG.debug("Max retries exceeded with: '%s'",
                  has_operation_completed.__name__)
        if not is_silent_loop_exit:
            raise failover_exc(failover_msg)

Response: Waits until the operation is completed.


Instruction:
Explain what the following Python 3 code does
def get_filename_and_extension_of(target_file):
    base_target_filename = os.path.basename(target_file)
    file_name, file_ext_with_dot = os.path.splitext(base_target_filename)
    return file_name, file_ext_with_dot

Response: Gets the base filename and extension of the target file.


Instruction:
Can you tell what is the following Python 3 function doing
def add_exec_permission_to(target_file):
    mode = os.stat(target_file).st_mode
    os.chmod(target_file, mode | stat.S_IXUSR)

Response: Add executable permissions to the file


Instruction:
Write a Python 3 script to
extract the major and minor number from the passed string

Response:
def get_major_minor(ilo_ver_str):
    """Extract the major and minor number from the passed string

    :param ilo_ver_str: the string that contains the version information
    :returns: String of the form "<major>.<minor>" or None
    """
    if not ilo_ver_str:
        return None
    try:
        # Note(vmud213):This logic works for all strings
        # that contain the version info as <major>.<minor>
        # Formats of the strings:
        #    Release version ->  "2.50 Feb 18  2016"
        #    Debug version   ->  "iLO 4 v2.50"
        #    random version  ->  "XYZ ABC 2.30"
        pattern = re.search(ILO_VER_STR_PATTERN, ilo_ver_str)
        if pattern:
            matched = pattern.group(0)
            if matched:
                return matched
            return None
    except Exception:
        return None


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_supported_boot_modes(supported_boot_mode_constant):
    boot_mode_bios = 'false'
    boot_mode_uefi = 'false'
    if (supported_boot_mode_constant ==
            constants.SUPPORTED_BOOT_MODE_LEGACY_BIOS_ONLY):
        boot_mode_bios = 'true'
    elif (supported_boot_mode_constant ==
            constants.SUPPORTED_BOOT_MODE_UEFI_ONLY):
        boot_mode_uefi = 'true'
    elif (supported_boot_mode_constant ==
            constants.SUPPORTED_BOOT_MODE_LEGACY_BIOS_AND_UEFI):
        boot_mode_bios = 'true'
        boot_mode_uefi = 'true'

    return SupportedBootModes(boot_mode_bios=boot_mode_bios,
                              boot_mode_uefi=boot_mode_uefi)

Response: Retrieves the server supported boot modes as a namedtuple containing boot_mode_bios and boot_mode_uefi as true/'false.


Instruction:
Here you have a function in Python 3, explain what it does
def _get_key_value(string):
    # Normally all properties look like this:
    #   Unique Identifier: 600508B1001CE4ACF473EE9C826230FF
    #   Disk Name: /dev/sda
    #   Mount Points: None
    key = ''
    value = ''
    try:
        key, value = string.split(': ')
    except ValueError:
        # This handles the case when the property of a logical drive
        # returned is as follows. Here we cannot split by ':' because
        # the disk id has colon in it. So if this is about disk,
        # then strip it accordingly.
        #   Mirror Group 0: physicaldrive 6I:1:5
        string = string.lstrip(' ')
        if string.startswith('physicaldrive'):
            fields = string.split(' ')
            # Include fields[1] to key to avoid duplicate pairs
            # with the same 'physicaldrive' key
            key = fields[0] + " " + fields[1]
            value = fields[1]
        else:
            # TODO(rameshg87): Check if this ever occurs.
            return string.strip(' '), None

    return key.strip(' '), value.strip(' ')

Response: Return the key and value from a string.


Instruction:
Write a Python 3 script for
wrapping function for parsing hpssacli command.

Response:
def _convert_to_dict(stdout):
    """Wrapper function for parsing hpssacli/ssacli command.

    This function gets the output from hpssacli/ssacli command
    and calls the recursive function _get_dict to return
    the complete dictionary containing the RAID information.
    """

    lines = stdout.split("\n")
    lines = list(filter(None, lines))
    info_dict, j = _get_dict(lines, 0, 0, 0)
    return info_dict


Instruction:
How would you code a function in Python 3 to
wrap function for executing the ssacli command.

Response:
def _ssacli(*args, **kwargs):
    """Wrapper function for executing hpssacli/ssacli command.

    This function executes ssacli command if it exists, else it
    falls back to hpssacli.
    :param args: args to be provided to hpssacli/ssacli command
    :param kwargs: kwargs to be sent to processutils except the
        following:
        - dont_transform_to_hpssa_exception - Set to True if this
          method shouldn't transform other exceptions to hpssa
          exceptions only when hpssa controller is available. This is
          useful when the return code from hpssacli/ssacli is useful for
          analysis.
    :returns: a tuple containing the stdout and stderr after running
        the process.
    :raises: HPSSAOperationError, if some error was encountered and
        dont_dont_transform_to_hpssa_exception was set to False.
    :raises: OSError or processutils.ProcessExecutionError if execution
        failed and dont_transform_to_hpssa_exception was set to True.
    """

    dont_transform_to_hpssa_exception = kwargs.get(
        'dont_transform_to_hpssa_exception', False)
    kwargs.pop('dont_transform_to_hpssa_exception', None)

    try:
        if os.path.exists("/usr/sbin/ssacli"):
            stdout, stderr = processutils.execute("ssacli",
                                                  *args, **kwargs)
        else:
            stdout, stderr = processutils.execute("hpssacli",
                                                  *args, **kwargs)
    except (OSError, processutils.ProcessExecutionError) as e:
        if 'No controllers detected' in str(e):
            msg = ("SSA controller not found. Enable ssa controller"
                   " to continue with the desired operation")
            raise exception.HPSSAOperationError(reason=msg)
        elif not dont_transform_to_hpssa_exception:
            raise exception.HPSSAOperationError(reason=e)
        else:
            raise

    return stdout, stderr


Instruction:
Implement a Python 3 function for
refreshing the server and it s child objects.

Response:
def refresh(self):
        """Refresh the server and it's child objects.

        This method removes all the cache information in the server
        and it's child objects, and fetches the information again from
        the server using hpssacli/ssacli command.

        :raises: HPSSAOperationError, if hpssacli/ssacli operation failed.
        """
        config = self._get_all_details()

        raid_info = _convert_to_dict(config)
        self.controllers = []

        for key, value in raid_info.items():
            self.controllers.append(Controller(key, value, self))

        self.last_updated = time.time()


Instruction:
Explain what the following Python 3 code does
def get_controller_by_id(self, id):
        for controller in self.controllers:
            if controller.id == id:
                return controller
        return None

Response: Get the controller object given the id.


Instruction:
How would you explain what the following Python 3 function does
def get_logical_drives(self):
        logical_drives = []
        for controller in self.controllers:
            for array in controller.raid_arrays:
                for logical_drive in array.logical_drives:
                    logical_drives.append(logical_drive)
        return logical_drives

Response: Get all the RAID logical drives in the Server.


Instruction:
How would you explain what the following Python 3 function does
def get_physical_drives(self):
        physical_drives = []
        for controller in self.controllers:
            # First add unassigned physical drives.
            for physical_drive in controller.unassigned_physical_drives:
                physical_drives.append(physical_drive)
            # Now add physical drives part of RAID arrays.
            for array in controller.raid_arrays:
                for physical_drive in array.physical_drives:
                    physical_drives.append(physical_drive)

        return physical_drives

Response: Get all the RAID physical drives on the Server.


Instruction:
Create a Python 3 function for
getting the logical drive object given the wwn. This method returns the logical drive object with the given wwn. :param wwn: wwn of the logical drive :returns: LogicalDrive object which has the wwn or None if logical drive is not found.

Response:
def get_logical_drive_by_wwn(self, wwn):
        """Get the logical drive object given the wwn.

        This method returns the logical drive object with the given wwn.

        :param wwn: wwn of the logical drive
        :returns: LogicalDrive object which has the wwn or None if
            logical drive is not found.
        """
        disk = [x for x in self.get_logical_drives() if x.wwn == wwn]
        if disk:
            return disk[0]
        return None


Instruction:
Given the following Python 3 function, write the documentation
def get_physical_drive_by_id(self, id):
        for phy_drive in self.unassigned_physical_drives:
            if phy_drive.id == id:
                return phy_drive
        for array in self.raid_arrays:
            for phy_drive in array.physical_drives:
                if phy_drive.id == id:
                    return phy_drive
        return None

Response: Get a PhysicalDrive object for given id.


Instruction:
Write a Python 3 function that can
execute a given hpssacli command on the controller.

Response:
def execute_cmd(self, *args, **kwargs):
        """Execute a given hpssacli/ssacli command on the controller.

        This method executes a given command on the controller.

        :params args: a tuple consisting of sub-commands to be appended
            after specifying the controller in hpssacli/ssacli command.
        :param kwargs: kwargs to be passed to execute() in processutils
        :raises: HPSSAOperationError, if hpssacli/ssacli operation failed.
        """
        slot = self.properties['Slot']
        base_cmd = ("controller", "slot=%s" % slot)
        cmd = base_cmd + args
        return _ssacli(*cmd, **kwargs)


Instruction:
Can you implement a function in Python 3 that
creates a logical drive on the controller. This method creates a logical drive on the controller when the logical drive details and physical drive ids are passed to it. :param logical_drive_info: a dictionary containing the details of the logical drive as specified in raid config. :raises: HPSSAOperationError, if hpssacli/ssacli operation failed.

Response:
def create_logical_drive(self, logical_drive_info):
        """Create a logical drive on the controller.

        This method creates a logical drive on the controller when the
        logical drive details and physical drive ids are passed to it.

        :param logical_drive_info: a dictionary containing the details
            of the logical drive as specified in raid config.
        :raises: HPSSAOperationError, if hpssacli/ssacli operation failed.
        """
        cmd_args = []
        if 'array' in logical_drive_info:
            cmd_args.extend(['array', logical_drive_info['array']])

        cmd_args.extend(['create', "type=logicaldrive"])

        if 'physical_disks' in logical_drive_info:
            phy_drive_ids = ','.join(logical_drive_info['physical_disks'])
            cmd_args.append("drives=%s" % phy_drive_ids)

        raid_level = logical_drive_info['raid_level']
        # For RAID levels (like 5+0 and 6+0), HPSSA names them differently.
        # Check if we have mapping stored, otherwise use the same.
        raid_level = constants.RAID_LEVEL_INPUT_TO_HPSSA_MAPPING.get(
            raid_level, raid_level)
        cmd_args.append("raid=%s" % raid_level)

        # If size_gb is MAX, then don't pass size argument.  HPSSA will
        # automatically allocate the maximum # disks size possible to the
        # logical disk.
        if logical_drive_info['size_gb'] != "MAX":
            size_mb = logical_drive_info['size_gb'] * 1024
            cmd_args.append("size=%s" % size_mb)

        self.execute_cmd(*cmd_args, process_input='y')


Instruction:
How would you implement a function in Python 3 that
returns the command arguments based on the pattern.

Response:
def _get_erase_command(self, drive, pattern):
        """Return the command arguments based on the pattern.

        Erase command examples:
        1) Sanitize: "ssacli ctrl slot=0 pd 1I:1:1 modify erase
                      erasepattern=overwrite unrestricted=off forced"
        2) Zeros: "ssacli ctrl slot=0 pd 1I:1:1 modify erase
                   erasepattern=zero forced"

        :param drive: A string with comma separated list of drives.
        :param pattern: A string which defines the type of erase.
        :returns: A list of ssacli command arguments.
        """
        cmd_args = []
        cmd_args.append("pd %s" % drive)
        cmd_args.extend(['modify', 'erase', pattern])

        if pattern != 'erasepattern=zero':
            cmd_args.append('unrestricted=off')

        cmd_args.append('forced')
        return cmd_args


Instruction:
Write a Python 3 function that can
erase all the HDD and SSD drives in the controller.

Response:
def erase_devices(self, drives):
        """Perform Erase on all the drives in the controller.

        This method erases all the hdd and ssd drives in the controller
        by overwriting the drives with patterns for hdd and erasing storage
        blocks for ssd drives. The drives would be unavailable until
        successful completion or failure of erase operation.

        If the sanitize erase is not supported on any disk it will try to
        populate zeros on disk drives.

        :param drives: A list of drive objects in the controller.
        :raises: HPSSAOperationError, if sanitize erase is not supported.
        """
        for drive in drives:
            pattern = 'overwrite' if (
                drive.disk_type == constants.DISK_TYPE_HDD) else 'block'
            cmd_args = self._get_erase_command(
                drive.id, 'erasepattern=%s' % pattern)
            stdout = self.execute_cmd(*cmd_args)
            LOG.debug("Sanitize disk erase invoked with erase pattern as "
                      "'%(pattern)s' on disk type: %(disk_type)s."
                      % {'pattern': pattern, 'disk_type': drive.disk_type})

            if "not supported" in str(stdout):
                new_pattern = 'zero'
                cmd_args = self._get_erase_command(drive.id,
                                                   'erasepattern=zero')
                self.execute_cmd(*cmd_args)
                LOG.debug("Sanitize disk erase invoked with erase pattern as "
                          "'%(pattern)s' is not supported on disk type: "
                          "%(disk_type)s. Now its invoked with erase pattern "
                          "as %(new_pattern)s."
                          % {'pattern': pattern, 'disk_type': drive.disk_type,
                             'new_pattern': new_pattern})


Instruction:
Here you have a function in Python 3, explain what it does
def can_accomodate(self, logical_disk):
        raid_level = constants.RAID_LEVEL_INPUT_TO_HPSSA_MAPPING.get(
            logical_disk['raid_level'], logical_disk['raid_level'])
        args = ("array", self.id, "create", "type=logicaldrive",
                "raid=%s" % raid_level, "size=?")

        if logical_disk['size_gb'] != "MAX":
            desired_disk_size = logical_disk['size_gb']
        else:
            desired_disk_size = constants.MINIMUM_DISK_SIZE

        try:
            stdout, stderr = self.parent.execute_cmd(
                *args, dont_transform_to_hpssa_exception=True)
        except processutils.ProcessExecutionError as ex:
            # hpssacli/ssacli returns error code 1 when RAID level of the
            # logical disk is not supported on the array.
            # If that's the case, just return saying the logical disk
            # cannot be accomodated in the array.
            # If exist_code is not 1, then it's some other error that we
            # don't expect to appear and hence raise it back.
            if ex.exit_code == 1:
                return False
            else:
                raise exception.HPSSAOperationError(reason=ex)
        except Exception as ex:
            raise exception.HPSSAOperationError(reason=ex)

        # TODO(rameshg87): This always returns in MB, but confirm with
        # HPSSA folks.
        match = re.search('Max: (\d+)', stdout)
        if not match:
            return False

        max_size_gb = int(match.group(1)) / 1024
        return desired_disk_size <= max_size_gb

Response: Check if this RAID array can accomodate the logical disk.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_physical_drive_dict(self):

        if isinstance(self.parent, RaidArray):
            controller = self.parent.parent.id
            status = 'active'
        else:
            controller = self.parent.id
            status = 'ready'

        return {'size_gb': self.size_gb,
                'controller': controller,
                'id': self.id,
                'disk_type': self.disk_type,
                'interface_type': self.interface_type,
                'model': self.model,
                'firmware': self.firmware,
                'status': status,
                'erase_status': self.erase_status}

Response: Returns a dictionary of with the details of the physical drive.


Instruction:
Can you write a function in Python 3 where it
gets the set of raid levels for each logical volume

Response:
def logical_raid_levels(self):
        """Gets the raid level for each logical volume

        :returns the set of list of raid levels configured.
        """
        lg_raid_lvls = set()
        for member in self.get_members():
            lg_raid_lvls.add(mappings.RAID_LEVEL_MAP_REV.get(member.raid))
        return lg_raid_lvls


Instruction:
Can you create a Python 3 function that
sends a file to the client.

Response:
def send_stream(stream, filename, size, mtime, mimetype=None, restricted=True,
                as_attachment=False, etag=None, content_md5=None,
                chunk_size=None, conditional=True, trusted=False):
    """Send the contents of a file to the client.

    .. warning::

        It is very easy to be exposed to Cross-Site Scripting (XSS) attacks if
        you serve user uploaded files. Here are some recommendations:

            1. Serve user uploaded files from a separate domain
               (not a subdomain). This way a malicious file can only attack
               other user uploaded files.
            2. Prevent the browser from rendering and executing HTML files (by
               setting ``trusted=False``).
            3. Force the browser to download the file as an attachment
               (``as_attachment=True``).

    :param stream: The file stream to send.
    :param filename: The file name.
    :param size: The file size.
    :param mtime: A Unix timestamp that represents last modified time (UTC).
    :param mimetype: The file mimetype. If ``None``, the module will try to
        guess. (Default: ``None``)
    :param restricted: If the file is not restricted, the module will set the
        cache-control. (Default: ``True``)
    :param as_attachment: If the file is an attachment. (Default: ``False``)
    :param etag: If defined, it will be set as HTTP E-Tag.
    :param content_md5: If defined, a HTTP Content-MD5 header will be set.
    :param chunk_size: The chunk size.
    :param conditional: Make the response conditional to the request.
        (Default: ``True``)
    :param trusted: Do not enable this option unless you know what you are
        doing. By default this function will send HTTP headers and MIME types
        that prevents your browser from rendering e.g. a HTML file which could
        contain a malicious script tag.
        (Default: ``False``)
    :returns: A Flask response instance.
    """
    chunk_size = chunk_size_or_default(chunk_size)

    # Guess mimetype from filename if not provided.
    if mimetype is None and filename:
        mimetype = mimetypes.guess_type(filename)[0]
    if mimetype is None:
        mimetype = 'application/octet-stream'

    # Construct headers
    headers = Headers()
    headers['Content-Length'] = size
    if content_md5:
        headers['Content-MD5'] = content_md5

    if not trusted:
        # Sanitize MIME type
        mimetype = sanitize_mimetype(mimetype, filename=filename)
        # See https://www.owasp.org/index.php/OWASP_Secure_Headers_Project
        # Prevent JavaScript execution
        headers['Content-Security-Policy'] = "default-src 'none';"
        # Prevent MIME type sniffing for browser.
        headers['X-Content-Type-Options'] = 'nosniff'
        # Prevent opening of downloaded file by IE
        headers['X-Download-Options'] = 'noopen'
        # Prevent cross domain requests from Flash/Acrobat.
        headers['X-Permitted-Cross-Domain-Policies'] = 'none'
        # Prevent files from being embedded in frame, iframe and object tags.
        headers['X-Frame-Options'] = 'deny'
        # Enable XSS protection (IE, Chrome, Safari)
        headers['X-XSS-Protection'] = '1; mode=block'

    # Force Content-Disposition for application/octet-stream to prevent
    # Content-Type sniffing.
    if as_attachment or mimetype == 'application/octet-stream':
        # See https://github.com/pallets/flask/commit/0049922f2e690a6d
        try:
            filenames = {'filename': filename.encode('latin-1')}
        except UnicodeEncodeError:
            filenames = {'filename*': "UTF-8''%s" % url_quote(filename)}
            encoded_filename = (unicodedata.normalize('NFKD', filename)
                                .encode('latin-1', 'ignore'))
            if encoded_filename:
                filenames['filename'] = encoded_filename
        headers.add('Content-Disposition', 'attachment', **filenames)
    else:
        headers.add('Content-Disposition', 'inline')

    # Construct response object.
    rv = current_app.response_class(
        FileWrapper(stream, buffer_size=chunk_size),
        mimetype=mimetype,
        headers=headers,
        direct_passthrough=True,
    )

    # Set etag if defined
    if etag:
        rv.set_etag(etag)

    # Set last modified time
    if mtime is not None:
        rv.last_modified = int(mtime)

    # Set cache-control
    if not restricted:
        rv.cache_control.public = True
        cache_timeout = current_app.get_send_file_max_age(filename)
        if cache_timeout is not None:
            rv.cache_control.max_age = cache_timeout
            rv.expires = int(time() + cache_timeout)

    if conditional:
        rv = rv.make_conditional(request)

    return rv


Instruction:
Implement a Python 3 function for
sanitizing a MIME type so the browser does not render the file.

Response:
def sanitize_mimetype(mimetype, filename=None):
    """Sanitize a MIME type so the browser does not render the file."""
    # Allow some few mime type like plain text, images and audio.
    if mimetype in MIMETYPE_WHITELIST:
        return mimetype
    # Rewrite HTML, JavaScript, CSS etc to text/plain.
    if mimetype in MIMETYPE_PLAINTEXT or \
            (filename and filename.lower() in MIMETYPE_TEXTFILES):
        return 'text/plain'
    # Default
    return 'application/octet-stream'


Instruction:
Here you have a function in Python 3, explain what it does
def make_path(base_uri, path, filename, path_dimensions, split_length):
    assert len(path) > path_dimensions * split_length

    uri_parts = []
    for i in range(path_dimensions):
        uri_parts.append(path[0:split_length])
        path = path[split_length:]
    uri_parts.append(path)
    uri_parts.append(filename)

    return os.path.join(base_uri, *uri_parts)

Response: Generates a path as base location for a file instance.


Instruction:
Can you implement a function in Python 3 that
computes checksum from a file - like object.

Response:
def compute_checksum(stream, algo, message_digest, chunk_size=None,
                     progress_callback=None):
    """Get helper method to compute checksum from a stream.

    :param stream: File-like object.
    :param algo: Identifier for checksum algorithm.
    :param messsage_digest: A message digest instance.
    :param chunk_size: Read at most size bytes from the file at a time.
    :param progress_callback: Function accepting one argument with number
        of bytes read. (Default: ``None``)
    :returns: The checksum.
    """
    chunk_size = chunk_size_or_default(chunk_size)

    bytes_read = 0
    while 1:
        chunk = stream.read(chunk_size)
        if not chunk:
            if progress_callback:
                progress_callback(bytes_read)
            break
        message_digest.update(chunk)
        bytes_read += len(chunk)
        if progress_callback:
            progress_callback(bytes_read)
    return "{0}:{1}".format(algo, message_digest.hexdigest())


Instruction:
How would you explain what the following Python 3 function does
def populate_from_path(bucket, source, checksum=True, key_prefix='',
                       chunk_size=None):
    """Populate a ``bucket`` from all files in path.

    :param bucket: The bucket (instance or id) to create the object in.
    :param source: The file or directory path.
    :param checksum: If ``True`` then a MD5 checksum will be computed for each
        file. (Default: ``True``)
    :param key_prefix: The key prefix for the bucket.
    :param chunk_size: Chunk size to read from file.
    :returns: A iterator for all
        :class:`invenio_files_rest.models.ObjectVersion` instances.
    """
    from .models import FileInstance, ObjectVersion

    def create_file(key, path):
        """Create new ``ObjectVersion`` from path or existing ``FileInstance``.

        It checks MD5 checksum and size of existing ``FileInstance``s.
        """
        key = key_prefix + key

        if checksum:
            file_checksum = compute_md5_checksum(
                    open(path, 'rb'), chunk_size=chunk_size)
            file_instance = FileInstance.query.filter_by(
                checksum=file_checksum, size=os.path.getsize(path)
            ).first()
            if file_instance:
                return ObjectVersion.create(
                    bucket, key, _file_id=file_instance.id
                )
        return ObjectVersion.create(bucket, key, stream=open(path, 'rb'))

    if os.path.isfile(source):
        yield create_file(os.path.basename(source), source)
    else:
        for root, dirs, files in os.walk(source, topdown=False):
            for name in files:
                filename = os.path.join(root, name)
                assert filename.startswith(source)
                parts = [p for p in filename[len(source):].split(os.sep) if p]
                yield create_file('/'.join(parts), os.path.join(root, name))

Response: Populate a list of objects from a file or directory path.


Instruction:
Implement a function in Python 3 to
set instance variables based on an options dict.

Response:
def set_options(self, options):
        """
        Sets instance variables based on an options dict
        """
        # COMMAND LINE OPTIONS
        self.wipe = options.get("wipe")
        self.test_run = options.get("test_run")
        self.quiet = options.get("test_run")
        self.container_name = options.get("container")
        self.verbosity = int(options.get("verbosity"))
        self.syncmedia = options.get("syncmedia")
        self.syncstatic = options.get("syncstatic")
        if self.test_run:
            self.verbosity = 2
        cli_includes = options.get("includes")
        cli_excludes = options.get("excludes")

        # CUMULUS CONNECTION AND SETTINGS FROM SETTINGS.PY
        if self.syncmedia and self.syncstatic:
            raise CommandError("options --media and --static are mutually exclusive")
        if not self.container_name:
            if self.syncmedia:
                self.container_name = CUMULUS["CONTAINER"]
            elif self.syncstatic:
                self.container_name = CUMULUS["STATIC_CONTAINER"]
            else:
                raise CommandError("must select one of the required options, either --media or --static")
        settings_includes = CUMULUS["INCLUDE_LIST"]
        settings_excludes = CUMULUS["EXCLUDE_LIST"]

        # PATH SETTINGS
        if self.syncmedia:
            self.file_root = os.path.abspath(settings.MEDIA_ROOT)
            self.file_url = settings.MEDIA_URL
        elif self.syncstatic:
            self.file_root = os.path.abspath(settings.STATIC_ROOT)
            self.file_url = settings.STATIC_URL
        if not self.file_root.endswith("/"):
            self.file_root = self.file_root + "/"
        if self.file_url.startswith("/"):
            self.file_url = self.file_url[1:]

        # SYNCSTATIC VARS
        # combine includes and excludes from the cli and django settings file
        self.includes = list(set(cli_includes + settings_includes))
        self.excludes = list(set(cli_excludes + settings_excludes))
        # transform glob patterns to regular expressions
        self.local_filenames = []
        self.create_count = 0
        self.upload_count = 0
        self.update_count = 0
        self.skip_count = 0
        self.delete_count = 0


Instruction:
How would you code a function in Python 3 to
return the cloud objects that match the include and exclude patterns.

Response:
def match_cloud(self, includes, excludes):
        """
        Returns the cloud objects that match the include and exclude patterns.
        """
        cloud_objs = [cloud_obj.name for cloud_obj in self.container.get_objects()]
        includes_pattern = r"|".join([fnmatch.translate(x) for x in includes])
        excludes_pattern = r"|".join([fnmatch.translate(x) for x in excludes]) or r"$."
        excludes = [o for o in cloud_objs if re.match(excludes_pattern, o)]
        includes = [o for o in cloud_objs if re.match(includes_pattern, o)]
        return [o for o in includes if o not in excludes]


Instruction:
Explain what the following Python 3 code does
def upload_files(self, abspaths, relpaths, remote_objects):
        for relpath in relpaths:
            abspath = [p for p in abspaths if p[len(self.file_root):] == relpath][0]
            cloud_datetime = remote_objects[relpath] if relpath in remote_objects else None
            local_datetime = datetime.datetime.utcfromtimestamp(os.stat(abspath).st_mtime)

            if cloud_datetime and local_datetime < cloud_datetime:
                self.skip_count += 1
                if not self.quiet:
                    print("Skipped {0}: not modified.".format(relpath))
                continue
            if relpath in remote_objects:
                self.update_count += 1
            else:
                self.create_count += 1
            self.upload_file(abspath, relpath)

Response: Determines files to be uploaded and call upload_file on each.


Instruction:
Create a Python 3 function for
uploading a file to the container.

Response:
def upload_file(self, abspath, cloud_filename):
        """
        Uploads a file to the container.
        """
        if not self.test_run:
            content = open(abspath, "rb")
            content_type = get_content_type(cloud_filename, content)
            headers = get_headers(cloud_filename, content_type)

            if headers.get("Content-Encoding") == "gzip":
                content = get_gzipped_contents(content)
                size = content.size
            else:
                size = os.stat(abspath).st_size
            self.container.create(
                obj_name=cloud_filename,
                data=content,
                content_type=content_type,
                content_length=size,
                content_encoding=headers.get("Content-Encoding", None),
                headers=headers,
                ttl=CUMULUS["FILE_TTL"],
                etag=None,
            )

        self.upload_count += 1
        if not self.quiet or self.verbosity > 1:
            print("Uploaded: {0}".format(cloud_filename))


Instruction:
Write a Python 3 script for
deleting any extra files from the container that do not exist locally.

Response:
def delete_extra_files(self, relpaths, cloud_objs):
        """
        Deletes any objects from the container that do not exist locally.
        """
        for cloud_obj in cloud_objs:
            if cloud_obj not in relpaths:
                if not self.test_run:
                    self.delete_cloud_obj(cloud_obj)
                self.delete_count += 1
                if not self.quiet or self.verbosity > 1:
                    print("Deleted: {0}".format(cloud_obj))


Instruction:
Make a summary of the following Python 3 code
def delete_cloud_obj(self, cloud_obj):
        self._connection.delete_object(
            container=self.container_name,
            obj=cloud_obj,
        )

Response: Deletes an object from the container.


Instruction:
Write a Python 3 function for
printing the final tally to stdout.

Response:
def print_tally(self):
        """
        Prints the final tally to stdout.
        """
        self.update_count = self.upload_count - self.create_count
        if self.test_run:
            print("Test run complete with the following results:")
        print("Skipped {0}. Created {1}. Updated {2}. Deleted {3}.".format(
            self.skip_count, self.create_count, self.update_count, self.delete_count))


Instruction:
Implement a Python 3 function for
listing all the items in a container to stdout.

Response:
def handle(self, *args, **options):
        """
        Lists all the items in a container to stdout.
        """
        self._connection = Auth()._get_connection()

        if len(args) == 0:
            containers = self._connection.list_containers()
            if not containers:
                print("No containers were found for this account.")
        elif len(args) == 1:
            containers = self._connection.list_container_object_names(args[0])
            if not containers:
                print("No matching container found.")
        else:
            raise CommandError("Pass one and only one [container_name] as an argument")

        for container in containers:
            print(container)


Instruction:
Write a Python 3 script to
progress reporter for checksum verification.

Response:
def progress_updater(size, total):
    """Progress reporter for checksum verification."""
    current_task.update_state(
        state=state('PROGRESS'),
        meta=dict(size=size, total=total)
    )


Instruction:
Here you have a function in Python 3, explain what it does
def verify_checksum(file_id, pessimistic=False, chunk_size=None, throws=True,
                    checksum_kwargs=None):
    """Verify checksum of a file instance.

    :param file_id: The file ID.
    """
    f = FileInstance.query.get(uuid.UUID(file_id))

    # Anything might happen during the task, so being pessimistic and marking
    # the file as unchecked is a reasonable precaution
    if pessimistic:
        f.clear_last_check()
        db.session.commit()
    f.verify_checksum(
        progress_callback=progress_updater, chunk_size=chunk_size,
        throws=throws, checksum_kwargs=checksum_kwargs)
    db.session.commit()

Response: Verify checksum of a file instance.


Instruction:
Implement a function in Python 3 to
schedule a batch of files for checksum verification.

Response:
def schedule_checksum_verification(frequency=None, batch_interval=None,
                                   max_count=None, max_size=None,
                                   files_query=None,
                                   checksum_kwargs=None):
    """Schedule a batch of files for checksum verification.

    The purpose of this task is to be periodically called through `celerybeat`,
    in order achieve a repeated verification cycle of all file checksums, while
    following a set of constraints in order to throttle the execution rate of
    the checks.

    :param dict frequency: Time period over which a full check of all files
        should be performed. The argument is a dictionary that will be passed
        as arguments to the `datetime.timedelta` class. Defaults to a month (30
        days).
    :param dict batch_interval: How often a batch is sent. If not supplied,
        this information will be extracted, if possible, from the
        celery.conf['CELERYBEAT_SCHEDULE'] entry of this task. The argument is
        a dictionary that will be passed as arguments to the
        `datetime.timedelta` class.
    :param int max_count: Max count of files of a single batch. When set to `0`
        it's automatically calculated to be distributed equally through the
        number of total batches.
    :param int max_size: Max size of a single batch in bytes. When set to `0`
        it's automatically calculated to be distributed equally through the
        number of total batches.
    :param str files_query: Import path for a function returning a
        FileInstance query for files that should be checked.
    :param dict checksum_kwargs: Passed to ``FileInstance.verify_checksum``.
    """
    assert max_count is not None or max_size is not None
    frequency = timedelta(**frequency) if frequency else timedelta(days=30)
    if batch_interval:
        batch_interval = timedelta(**batch_interval)
    else:
        celery_schedule = current_celery.conf.get('CELERYBEAT_SCHEDULE', {})
        batch_interval = batch_interval or next(
            (v['schedule'] for v in celery_schedule.values()
             if v.get('task') == schedule_checksum_verification.name), None)
    if not batch_interval or not isinstance(batch_interval, timedelta):
        raise Exception(u'No "batch_interval" could be decided')

    total_batches = int(
        frequency.total_seconds() / batch_interval.total_seconds())

    files = obj_or_import_string(
        files_query, default=default_checksum_verification_files_query)()
    files = files.order_by(
        sa.func.coalesce(FileInstance.last_check_at, date.min))

    if max_count is not None:
        all_files_count = files.count()
        min_count = int(math.ceil(all_files_count / total_batches))
        max_count = min_count if max_count == 0 else max_count
        if max_count < min_count:
            current_app.logger.warning(
                 u'The "max_count" you specified ({0}) is smaller than the '
                 'minimum batch file count required ({1}) in order to achieve '
                 'the file checks over the specified period ({2}).'
                 .format(max_count, min_count, frequency))
        files = files.limit(max_count)

    if max_size is not None:
        all_files_size = db.session.query(
            sa.func.sum(FileInstance.size)).scalar()
        min_size = int(math.ceil(all_files_size / total_batches))
        max_size = min_size if max_size == 0 else max_size
        if max_size < min_size:
            current_app.logger.warning(
                 u'The "max_size" you specified ({0}) is smaller than the '
                 'minimum batch total file size required ({1}) in order to '
                 'achieve the file checks over the specified period ({2}).'
                 .format(max_size, min_size, frequency))

    files = files.yield_per(1000)
    scheduled_file_ids = []
    total_size = 0
    for f in files:
        # Add at least the first file, since it might be larger than "max_size"
        scheduled_file_ids.append(str(f.id))
        total_size += f.size
        if max_size and max_size <= total_size:
            break
    group(
        verify_checksum.s(
            file_id, pessimistic=True, throws=False,
            checksum_kwargs=(checksum_kwargs or {}))
        for file_id in scheduled_file_ids
    ).apply_async()


Instruction:
Create a Python 3 function to
task to migrate a file instance to a new location. .. note:: If something goes wrong during the content copy, the destination file instance is removed. :param src_id: The :class:`invenio_files_rest.models.FileInstance` ID. :param location_name: Where to migrate the file. :param post_fixity_check: Verify checksum after migration. (Default: ``False``)

Response:
def migrate_file(src_id, location_name, post_fixity_check=False):
    """Task to migrate a file instance to a new location.

    .. note:: If something goes wrong during the content copy, the destination
        file instance is removed.

    :param src_id: The :class:`invenio_files_rest.models.FileInstance` ID.
    :param location_name: Where to migrate the file.
    :param post_fixity_check: Verify checksum after migration.
        (Default: ``False``)
    """
    location = Location.get_by_name(location_name)
    f_src = FileInstance.get(src_id)

    # Create destination
    f_dst = FileInstance.create()
    db.session.commit()

    try:
        # Copy contents
        f_dst.copy_contents(
            f_src,
            progress_callback=progress_updater,
            default_location=location.uri,
        )
        db.session.commit()
    except Exception:
        # Remove destination file instance if an error occurred.
        db.session.delete(f_dst)
        db.session.commit()
        raise

    # Update all objects pointing to file.
    ObjectVersion.relink_all(f_src, f_dst)
    db.session.commit()

    # Start a fixity check
    if post_fixity_check:
        verify_checksum.delay(str(f_dst.id))


Instruction:
Can you write a function in Python 3 where it
removes the file instance and associated data.

Response:
def remove_file_data(file_id, silent=True):
    """Remove file instance and associated data.

    :param file_id: The :class:`invenio_files_rest.models.FileInstance` ID.
    :param silent: It stops propagation of a possible arised IntegrityError
        exception. (Default: ``True``)
    :raises sqlalchemy.exc.IntegrityError: Raised if the database removal goes
        wrong and silent is set to ``False``.
    """
    try:
        # First remove FileInstance from database and commit transaction to
        # ensure integrity constraints are checked and enforced.
        f = FileInstance.get(file_id)
        if not f.writable:
            return
        f.delete()
        db.session.commit()
        # Next, remove the file on disk. This leaves the possibility of having
        # a file on disk dangling in case the database removal works, and the
        # disk file removal doesn't work.
        f.storage().delete()
    except IntegrityError:
        if not silent:
            raise


Instruction:
Make a summary of the following Python 3 code
def merge_multipartobject(upload_id, version_id=None):
    mp = MultipartObject.query.filter_by(upload_id=upload_id).one_or_none()
    if not mp:
        raise RuntimeError('Upload ID does not exists.')
    if not mp.completed:
        raise RuntimeError('MultipartObject is not completed.')

    try:
        obj = mp.merge_parts(
            version_id=version_id,
            progress_callback=progress_updater
        )
        db.session.commit()
        return str(obj.version_id)
    except Exception:
        db.session.rollback()
        raise

Response: Merge a multipart object.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def remove_expired_multipartobjects():
    delta = current_app.config['FILES_REST_MULTIPART_EXPIRES']
    expired_dt = datetime.utcnow() - delta

    file_ids = []
    for mp in MultipartObject.query_expired(expired_dt):
        file_ids.append(str(mp.file_id))
        mp.delete()

    for fid in file_ids:
        remove_file_data.delay(fid)

Response: Remove expired multipart objects.


Instruction:
Explain what the following Python 3 code does
def pyfs_storage_factory(fileinstance=None, default_location=None,
                         default_storage_class=None,
                         filestorage_class=PyFSFileStorage, fileurl=None,
                         size=None, modified=None, clean_dir=True):
    """Get factory function for creating a PyFS file storage instance."""
    # Either the FileInstance needs to be specified or all filestorage
    # class parameters need to be specified
    assert fileinstance or (fileurl and size)

    if fileinstance:
        # FIXME: Code here should be refactored since it assumes a lot on the
        # directory structure where the file instances are written
        fileurl = None
        size = fileinstance.size
        modified = fileinstance.updated

        if fileinstance.uri:
            # Use already existing URL.
            fileurl = fileinstance.uri
        else:
            assert default_location
            # Generate a new URL.
            fileurl = make_path(
                default_location,
                str(fileinstance.id),
                'data',
                current_app.config['FILES_REST_STORAGE_PATH_DIMENSIONS'],
                current_app.config['FILES_REST_STORAGE_PATH_SPLIT_LENGTH'],
            )

    return filestorage_class(
        fileurl, size=size, modified=modified, clean_dir=clean_dir)

Response: Returns a factory function for creating a PyFS file storage instance.


Instruction:
Here you have a function in Python 3, explain what it does
def _get_fs(self, create_dir=True):
        filedir = dirname(self.fileurl)
        filename = basename(self.fileurl)

        return (
            opener.opendir(filedir, writeable=True, create_dir=create_dir),
            filename
        )

Response: Return tuple with filesystem and filename.


Instruction:
Can you create a Python 3 function that
opens file. The caller is responsible for closing the file.

Response:
def open(self, mode='rb'):
        """Open file.

        The caller is responsible for closing the file.
        """
        fs, path = self._get_fs()
        return fs.open(path, mode=mode)


Instruction:
Implement a function in Python 3 to
delete a file. The base directory is also removed, as it is assumed that only one file exists in the directory.

Response:
def delete(self):
        """Delete a file.

        The base directory is also removed, as it is assumed that only one file
        exists in the directory.
        """
        fs, path = self._get_fs(create_dir=False)
        if fs.exists(path):
            fs.remove(path)
        if self.clean_dir and fs.exists('.'):
            fs.removedir('.')
        return True


Instruction:
Implement a Python 3 function for
initializing file on storage and truncate to given size.

Response:
def initialize(self, size=0):
        """Initialize file on storage and truncate to given size."""
        fs, path = self._get_fs()

        # Required for reliably opening the file on certain file systems:
        if fs.exists(path):
            fp = fs.open(path, mode='r+b')
        else:
            fp = fs.open(path, mode='wb')

        try:
            fp.truncate(size)
        except Exception:
            fp.close()
            self.delete()
            raise
        finally:
            fp.close()

        self._size = size

        return self.fileurl, size, None


Instruction:
Write a Python 3 function for
saving file in the file system.

Response:
def save(self, incoming_stream, size_limit=None, size=None,
             chunk_size=None, progress_callback=None):
        """Save file in the file system."""
        fp = self.open(mode='wb')
        try:
            bytes_written, checksum = self._write_stream(
                incoming_stream, fp, chunk_size=chunk_size,
                progress_callback=progress_callback,
                size_limit=size_limit, size=size)
        except Exception:
            fp.close()
            self.delete()
            raise
        finally:
            fp.close()

        self._size = bytes_written

        return self.fileurl, bytes_written, checksum


Instruction:
Write a Python 3 script to
update a file in the file system.

Response:
def update(self, incoming_stream, seek=0, size=None, chunk_size=None,
               progress_callback=None):
        """Update a file in the file system."""
        fp = self.open(mode='r+b')
        try:
            fp.seek(seek)
            bytes_written, checksum = self._write_stream(
                incoming_stream, fp, chunk_size=chunk_size,
                size=size, progress_callback=progress_callback)
        finally:
            fp.close()

        return bytes_written, checksum


Instruction:
Can you create a Python 3 function that
gets default permission factory.

Response:
def permission_factory(obj, action):
    """Get default permission factory.

    :param obj: An instance of :class:`invenio_files_rest.models.Bucket` or
        :class:`invenio_files_rest.models.ObjectVersion` or
        :class:`invenio_files_rest.models.MultipartObject` or ``None`` if
        the action is global.
    :param action: The required action.
    :raises RuntimeError: If the object is unknown.
    :returns: A :class:`invenio_access.permissions.Permission` instance.
    """
    need_class = _action2need_map[action]

    if obj is None:
        return Permission(need_class(None))

    arg = None
    if isinstance(obj, Bucket):
        arg = str(obj.id)
    elif isinstance(obj, ObjectVersion):
        arg = str(obj.bucket_id)
    elif isinstance(obj, MultipartObject):
        arg = str(obj.bucket_id)
    else:
        raise RuntimeError('Unknown object')

    return Permission(need_class(arg))


Instruction:
Can you implement a function in Python 3 that
checks if the target file should be deleted if it already exists and if it should be deleted if it should be skipped if it should be deleted if it already exists

Response:
def delete_file(self, path, prefixed_path, source_storage):
        """
        Checks if the target file should be deleted if it already exists
        """
        if isinstance(self.storage, CumulusStorage):
            if self.storage.exists(prefixed_path):
                try:
                    etag = self.storage._get_object(prefixed_path).etag
                    digest = "{0}".format(hashlib.md5(source_storage.open(path).read()).hexdigest())
                    if etag == digest:
                        self.log(u"Skipping '{0}' (not modified based on file hash)".format(path))
                        return False
                except:
                    raise
        return super(Command, self).delete_file(path, prefixed_path, source_storage)


Instruction:
Can you write a function in Python 3 where it
creates new bucket from all files in directory.

Response:
def cp(source, bucket, checksum, key_prefix):
    """Create new bucket from all files in directory."""
    from .models import Bucket
    from .helpers import populate_from_path
    for object_version in populate_from_path(
            Bucket.get(bucket), source, checksum=checksum,
            key_prefix=key_prefix):
        click.secho(str(object_version))
    db.session.commit()


Instruction:
Make a summary of the following Python 3 code
def get_content_type(name, content):
    if hasattr(content, "content_type"):
        content_type = content.content_type
    else:
        mime_type, encoding = mimetypes.guess_type(name)
        content_type = mime_type
    return content_type

Response: Returns the content type of the given name.


Instruction:
Given the following Python 3 function, write the documentation
def sync_headers(cloud_obj, headers=None, header_patterns=HEADER_PATTERNS):
    if headers is None:
        headers = {}

    # don't set headers on directories
    content_type = getattr(cloud_obj, "content_type", None)
    if content_type == "application/directory":
        return
    matched_headers = {}
    for pattern, pattern_headers in header_patterns:
        if pattern.match(cloud_obj.name):
            matched_headers.update(pattern_headers.copy())
    # preserve headers already set
    matched_headers.update(cloud_obj.headers)
    # explicitly set headers overwrite matches and already set headers
    matched_headers.update(headers)
    if matched_headers != cloud_obj.headers:
        cloud_obj.headers = matched_headers
        cloud_obj.sync_metadata()

Response: Synchronizes the given cloud_obj s headers with the ones given as headers.


Instruction:
Implement a Python 3 function for
returning a gzipped version of a previously opened file s buffer.

Response:
def get_gzipped_contents(input_file):
    """
    Returns a gzipped version of a previously opened file's buffer.
    """
    zbuf = StringIO()
    zfile = GzipFile(mode="wb", compresslevel=6, fileobj=zbuf)
    zfile.write(input_file.read())
    zfile.close()
    return ContentFile(zbuf.getvalue())


Instruction:
Can you tell what is the following Python 3 function doing
def schema_from_context(context):
    item_class = context.get('class')
    return (
        serializer_mapping[item_class] if item_class else BaseSchema,
        context.get('many', False)
    )

Response: Determine which schema to use based on context.


Instruction:
Make a summary of the following Python 3 code
def wait_for_taskresult(task_result, content, interval, max_rounds):
    assert max_rounds > 0

    def _whitespace_waiting():
        current = 0
        while current < max_rounds and current != -1:
            if task_result.ready():
                # Task is done and we return
                current = -1
                if task_result.successful():
                    yield content
                else:
                    yield FilesException(
                        description='Job failed.'
                    ).get_body()
            else:
                # Send whitespace to prevent connection from closing.
                current += 1
                sleep(interval)
                yield b' '

        # Timed-out reached
        if current == max_rounds:
            yield FilesException(
                description='Job timed out.'
            ).get_body()

    return _whitespace_waiting()

Response: Get helper to wait for async task result to finish.


Instruction:
Can you generate the documentation for the following Python 3 function
def json_serializer(data=None, code=200, headers=None, context=None,
                    etag=None, task_result=None):
    """Build a json flask response using the given data.

    :param data: The data to serialize. (Default: ``None``)
    :param code: The HTTP status code. (Default: ``200``)
    :param headers: The HTTP headers to include. (Default: ``None``)
    :param context: The schema class context. (Default: ``None``)
    :param etag: The ETag header. (Default: ``None``)
    :param task_result: Optionally you can pass async task to wait for.
        (Default: ``None``)
    :returns: A Flask response with json data.
    :rtype: :py:class:`flask.Response`
    """
    schema_class, many = schema_from_context(context or {})

    if data is not None:
        # Generate JSON response
        data = json.dumps(
            schema_class(context=context).dump(data, many=many).data,
            **_format_args()
        )

        interval = current_app.config['FILES_REST_TASK_WAIT_INTERVAL']
        max_rounds = int(
            current_app.config['FILES_REST_TASK_WAIT_MAX_SECONDS'] // interval
        )

        response = current_app.response_class(
            # Stream response if waiting for task result.
            data if task_result is None else wait_for_taskresult(
                task_result, data, interval, max_rounds, ),
            mimetype='application/json'
        )
    else:
        response = current_app.response_class(mimetype='application/json')

    response.status_code = code
    if headers is not None:
        response.headers.extend(headers)

    if etag:
        response.set_etag(etag)

    return response

Response: Builds a json flask response using the given data.


Instruction:
Create a Python 3 function for
dumping links for a specific object.

Response:
def dump_links(self, o):
        """Dump links."""
        return {
            'self': url_for('.bucket_api', bucket_id=o.id, _external=True),
            'versions': url_for(
                '.bucket_api', bucket_id=o.id, _external=True) + '?versions',
            'uploads': url_for(
                '.bucket_api', bucket_id=o.id, _external=True) + '?uploads',
        }


Instruction:
Here you have a function in Python 3, explain what it does
def wrap(self, data, many):
        if not many:
            return data
        else:
            data = {'contents': data}
            bucket = self.context.get('bucket')
            if bucket:
                data.update(BucketSchema().dump(bucket).data)
            return data

Response: Wrap response in envelope.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def wrap(self, data, many):
        if not many:
            return data
        else:
            data = {'parts': data}
            multipart = self.context.get('multipart')
            if multipart:
                data.update(MultipartObjectSchema(context={
                    'bucket': multipart.bucket}).dump(multipart).data)
            return data

Response: Wrap response in envelope.


Instruction:
Write a Python 3 script to
get directions between locations in a specific language.

Response:
def directions(self, origin, destination, mode=None, alternatives=None,
                   waypoints=None, optimize_waypoints=False,
                   avoid=None, language=None, units=None,
                   region=None, departure_time=None,
                   arrival_time=None, sensor=None):
        """Get directions between locations

        :param origin: Origin location - string address; (latitude, longitude)
            two-tuple, dict with ("lat", "lon") keys or object with (lat, lon)
            attributes
        :param destination: Destination location - type same as origin
        :param mode: Travel mode as string, defaults to "driving".
            See `google docs details <https://developers.google.com/maps/documentation/directions/#TravelModes>`_
        :param alternatives: True if provide it has to return more then one
            route alternative
        :param waypoints: Iterable with set of intermediate stops,
            like ("Munich", "Dallas")
            See `google docs details <https://developers.google.com/maps/documentation/javascript/reference#DirectionsRequest>`_
        :param optimize_waypoints: if true will attempt to re-order supplied
            waypoints to minimize overall cost of the route. If waypoints are
            optimized, the route returned will show the optimized order under
            "waypoint_order". See `google docs details <https://developers.google.com/maps/documentation/javascript/reference#DirectionsRequest>`_
        :param avoid: Iterable with set of restrictions,
            like ("tolls", "highways"). For full list refer to
            `google docs details <https://developers.google.com/maps/documentation/directions/#Restrictions>`_
        :param language: The language in which to return results.
            See `list of supported languages <https://developers.google.com/maps/faq#languagesupport>`_
        :param units: Unit system for result. Defaults to unit system of
            origin's country.
            See `google docs details <https://developers.google.com/maps/documentation/directions/#UnitSystems>`_
        :param region: The region code. Affects geocoding of origin and
            destination (see `gmaps.Geocoding.geocode` region parameter)
        :param departure_time: Desired time of departure as
            seconds since midnight, January 1, 1970 UTC
        :param arrival_time: Desired time of arrival for transit directions as
            seconds since midnight, January 1, 1970 UTC.
        """  # noqa
        if optimize_waypoints:
            waypoints.insert(0, "optimize:true")
        parameters = dict(
            origin=self.assume_latlon_or_address(origin),
            destination=self.assume_latlon_or_address(destination),
            mode=mode,
            alternatives=alternatives,
            waypoints=waypoints or [],
            avoid=avoid,
            language=language,
            units=units,
            region=region,
            departure_time=departure_time,
            arrival_time=arrival_time,
            sensor=sensor,
        )
        return self._make_request(self.DIRECTIONS_URL, parameters, "routes")


Instruction:
Explain what the following Python 3 code does
def upgrade():
    # Variant types:
    def created():
        """Return instance of a column."""
        return sa.Column(
            'created',
            sa.DateTime().with_variant(mysql.DATETIME(fsp=6), 'mysql'),
            nullable=False
        )

    def updated():
        """Return instance of a column."""
        return sa.Column(
            'updated',
            sa.DateTime().with_variant(mysql.DATETIME(fsp=6), 'mysql'),
            nullable=False
        )

    def uri():
        """Return instance of a column."""
        return sa.Column(
            'uri',
            sa.Text().with_variant(mysql.VARCHAR(255), 'mysql'),
            nullable=True
        )

    def key(nullable=True):
        """Return instance of a column."""
        return sa.Column(
            'key',
            sa.Text().with_variant(mysql.VARCHAR(255), 'mysql'),
            nullable=nullable
        )

    op.create_table(
        'files_files',
        created(),
        updated(),
        sa.Column(
            'id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        uri(),
        sa.Column('storage_class', sa.String(length=1), nullable=True),
        sa.Column('size', sa.BigInteger(), nullable=True),
        sa.Column('checksum', sa.String(length=255), nullable=True),
        sa.Column('readable', sa.Boolean(name='readable'), nullable=False),
        sa.Column('writable', sa.Boolean(name='writable'), nullable=False),
        sa.Column('last_check_at', sa.DateTime(), nullable=True),
        sa.Column('last_check', sa.Boolean(name='last_check'), nullable=False),
        sa.PrimaryKeyConstraint('id'),
        sa.UniqueConstraint('uri')
    )
    op.create_table(
        'files_location',
        created(),
        updated(),
        sa.Column('id', sa.Integer(), nullable=False),
        sa.Column('name', sa.String(length=20), nullable=False),
        sa.Column('uri', sa.String(length=255), nullable=False),
        sa.Column('default', sa.Boolean(name='default'), nullable=False),
        sa.PrimaryKeyConstraint('id'),
        sa.UniqueConstraint('name')
    )
    op.create_table(
        'files_bucket',
        created(),
        updated(),
        sa.Column(
            'id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        sa.Column('default_location', sa.Integer(), nullable=False),
        sa.Column(
            'default_storage_class',
            sa.String(length=1),
            nullable=False
        ),
        sa.Column('size', sa.BigInteger(), nullable=False),
        sa.Column('quota_size', sa.BigInteger(), nullable=True),
        sa.Column('max_file_size', sa.BigInteger(), nullable=True),
        sa.Column('locked', sa.Boolean(name='locked'), nullable=False),
        sa.Column('deleted', sa.Boolean(name='deleted'), nullable=False),
        sa.ForeignKeyConstraint(
            ['default_location'],
            [u'files_location.id'],
            ondelete='RESTRICT'),
        sa.PrimaryKeyConstraint('id')
    )
    op.create_table(
        'files_buckettags',
        sa.Column(
            'bucket_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        sa.Column('key', sa.String(length=255), nullable=False),
        sa.Column('value', sa.Text(), nullable=False),
        sa.ForeignKeyConstraint(
            ['bucket_id'],
            [u'files_bucket.id'],
            ondelete='CASCADE'),
        sa.PrimaryKeyConstraint('bucket_id', 'key')
    )
    op.create_table(
        'files_multipartobject',
        created(),
        updated(),
        sa.Column(
            'upload_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        sa.Column(
            'bucket_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=True),
        key(),
        sa.Column(
            'file_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        sa.Column('chunk_size', sa.Integer(), nullable=True),
        sa.Column('size', sa.BigInteger(), nullable=True),
        sa.Column('completed', sa.Boolean(name='completed'), nullable=False),
        sa.ForeignKeyConstraint(
            ['bucket_id'],
            [u'files_bucket.id'],
            ondelete='RESTRICT'),
        sa.ForeignKeyConstraint(
            ['file_id'],
            [u'files_files.id'],
            ondelete='RESTRICT'),
        sa.PrimaryKeyConstraint('upload_id'),
        sa.UniqueConstraint('upload_id', 'bucket_id', 'key', name='uix_item')
    )
    op.create_table(
        'files_object',
        created(),
        updated(),
        sa.Column(
            'bucket_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        key(nullable=False),
        sa.Column(
            'version_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        sa.Column(
            'file_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=True),
        sa.Column(
            '_mimetype', sa.String(
                length=255), nullable=True),
        sa.Column('is_head', sa.Boolean(name='is_head'), nullable=False),
        sa.ForeignKeyConstraint(
            ['bucket_id'],
            [u'files_bucket.id'],
            ondelete='RESTRICT'),
        sa.ForeignKeyConstraint(
            ['file_id'],
            [u'files_files.id'],
            ondelete='RESTRICT'),
        sa.PrimaryKeyConstraint('bucket_id', 'key', 'version_id')
    )
    op.create_index(
        op.f('ix_files_object__mimetype'),
        'files_object',
        ['_mimetype'],
        unique=False
    )
    op.create_table(
        'files_multipartobject_part',
        created(),
        updated(),
        sa.Column(
            'upload_id',
            sqlalchemy_utils.types.uuid.UUIDType(),
            nullable=False),
        sa.Column(
            'part_number',
            sa.Integer(),
            autoincrement=False,
            nullable=False),
        sa.Column('checksum', sa.String(length=255), nullable=True),
        sa.ForeignKeyConstraint(
            ['upload_id'],
            [u'files_multipartobject.upload_id'],
            ondelete='RESTRICT'),
        sa.PrimaryKeyConstraint('upload_id', 'part_number')
    )

Response: Upgrade database to version 1. 0.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def guess_mimetype(filename):
    m, encoding = mimetypes.guess_type(filename)
    if encoding:
        m = ENCODING_MIMETYPES.get(encoding, None)
    return m or 'application/octet-stream'

Response: Map extra mimetype with the encoding provided.


Instruction:
How would you explain what the following Python 3 function does
def link(text, link_func):
    def object_formatter(v, c, m, p):
        """Format object view link."""
        return Markup('{1}'.format(
            link_func(m), text))
    return object_formatter

Response: Generate a formatter for links.


Instruction:
Create a Python 3 function for
validating a tag. Keys must be less than 128 chars and values must be less than 256 chars.

Response:
def validate_tag(key, value):
    """Validate a tag.

    Keys must be less than 128 chars and values must be less than 256 chars.
    """
    # Note, parse_sql does not include a keys if the value is an empty string
    # (e.g. 'key=&test=a'), and thus technically we should not get strings
    # which have zero length.
    klen = len(key)
    vlen = len(value)

    return klen > 0 and klen < 256 and vlen > 0 and vlen < 256


Instruction:
Given the following Python 3 function, write the documentation
def parse_header_tags():
    # Get the value of the custom HTTP header and interpret it as an query
    # string
    qs = request.headers.get(
        current_app.config['FILES_REST_FILE_TAGS_HEADER'], '')

    tags = {}
    for key, value in parse_qsl(qs):
        # Check for duplicate keys
        if key in tags:
            raise DuplicateTagError()
        # Check for too short/long keys and values.
        if not validate_tag(key, value):
            raise InvalidTagError()
        tags[key] = value
    return tags or None

Response: Parse tags specified in the HTTP request header.