Instruction:
How would you code a function in Python 3 to
return a dry run report of the items that need to be uploaded or a message saying there are no files or folders that need to be uploaded.

Response:
def dry_run_report(self):
        """
        Returns text displaying the items that need to be uploaded or a message saying there are no files/folders
        to upload.
        :return: str: report text
        """
        project_uploader = ProjectUploadDryRun()
        project_uploader.run(self.local_project)
        items = project_uploader.upload_items
        if not items:
            return "\n\nNo changes found. Nothing needs to be uploaded.\n\n"
        else:
            result = "\n\nFiles/Folders that need to be uploaded:\n"
            for item in items:
                result += "{}\n".format(item)
            result += "\n"
            return result


Instruction:
Can you generate the documentation for the following Python 3 function
def get_upload_report(self):
        project = self.remote_store.fetch_remote_project(self.project_name_or_id,
                                                         must_exist=True,
                                                         include_children=False)
        report = UploadReport(project.name)
        report.walk_project(self.local_project)
        return report.get_content()

Response: Generate and print a report onto stdout.


Instruction:
Create a Python 3 function to
print url to view the project via dds portal.

Response:
def get_url_msg(self):
        """
        Print url to view the project via dds portal.
        """
        msg = 'URL to view project'
        project_id = self.local_project.remote_id
        url = '{}: https://{}/#/project/{}'.format(msg, self.config.get_portal_url_base(), project_id)
        return url


Instruction:
Explain what the following Python 3 code does
def visit_file(self, item, parent):
        if item.need_to_send:
            self.files += 1
            self.chunks += item.count_chunks(self.bytes_per_chunk)

Response: Increments counter if item needs to be sent.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def result_str(self):
        return '{}, {}, {}'.format(LocalOnlyCounter.plural_fmt('project', self.projects),
                                   LocalOnlyCounter.plural_fmt('folder', self.folders),
                                   LocalOnlyCounter.plural_fmt('file', self.files))

Response: Return a string representing the totals contained herein.


Instruction:
Can you implement a function in Python 3 that
pluralizes name if necessary and combine with cnt

Response:
def plural_fmt(name, cnt):
        """
        pluralize name if necessary and combine with cnt
        :param name: str name of the item type
        :param cnt: int number items of this type
        :return: str name and cnt joined
        """
        if cnt == 1:
            return '{} {}'.format(cnt, name)
        else:
            return '{} {}s'.format(cnt, name)


Instruction:
How would you code a function in Python 3 to
add folder to the report if it was sent to remote.

Response:
def visit_folder(self, item, parent):
        """
        Add folder to the report if it was sent.
        :param item: LocalFolder folder to possibly add
        :param parent: LocalFolder/LocalContent not used here
        """
        if item.sent_to_remote:
            self._add_report_item(item.path, item.remote_id)


Instruction:
Can you tell what is the following Python 3 function doing
def visit_file(self, item, parent):
        if item.sent_to_remote:
            self._add_report_item(item.path, item.remote_id, item.size, item.get_hash_value())

Response: Add file to the report if it was sent to remote.


Instruction:
Can you tell what is the following Python 3 function doing
def str_with_sizes(self, max_name, max_remote_id, max_size):
        name_str = self.name.ljust(max_name)
        remote_id_str = self.remote_id.ljust(max_remote_id)
        size_str = self.size.ljust(max_size)
        return u'{}    {}    {}    {}'.format(name_str, remote_id_str, size_str, self.file_hash)

Response: Create string for the internal properties of the item based on the sizes of the internal properties.


Instruction:
Here you have a function in Python 3, explain what it does
def upload(self, project_id, parent_kind, parent_id):
        path_data = self.local_file.get_path_data()
        hash_data = path_data.get_hash()
        self.upload_id = self.upload_operations.create_upload(project_id, path_data, hash_data,
                                                              storage_provider_id=self.config.storage_provider_id)
        ParallelChunkProcessor(self).run()
        parent_data = ParentData(parent_kind, parent_id)
        remote_file_data = self.upload_operations.finish_upload(self.upload_id, hash_data, parent_data,
                                                                self.local_file.remote_id)
        if self.file_upload_post_processor:
            self.file_upload_post_processor.run(self.data_service, remote_file_data)
        return remote_file_data['id']

Response: Upload file contents to project within a specified parent.


Instruction:
Implement a function in Python 3 to
create upload for uploading multiple chunks or the non - chunked variety.

Response:
def _create_upload(self, project_id, path_data, hash_data, remote_filename=None, storage_provider_id=None,
                       chunked=True):
        """
        Create upload for uploading multiple chunks or the non-chunked variety (includes upload url).
        :param project_id: str: uuid of the project
        :param path_data: PathData: holds file system data about the file we are uploading
        :param hash_data: HashData: contains hash alg and value for the file we are uploading
        :param remote_filename: str: name to use for our remote file (defaults to path_data basename otherwise)
        :param storage_provider_id: str: optional storage provider id
        :param chunked: bool: should we create a chunked upload
        :return: str: uuid for the upload
        """
        if not remote_filename:
            remote_filename = path_data.name()
        mime_type = path_data.mime_type()
        size = path_data.size()

        def func():
            return self.data_service.create_upload(project_id, remote_filename, mime_type, size,
                                                   hash_data.value, hash_data.alg,
                                                   storage_provider_id=storage_provider_id,
                                                   chunked=chunked)

        resp = retry_until_resource_is_consistent(func, self.waiting_monitor)
        return resp.json()


Instruction:
Can you generate a brief explanation for the following Python 3 code
def create_upload(self, project_id, path_data, hash_data, remote_filename=None, storage_provider_id=None):
        upload_response = self._create_upload(project_id, path_data, hash_data, remote_filename=remote_filename,
                                              storage_provider_id=storage_provider_id, chunked=True)
        return upload_response['id']

Response: Create a chunked upload id for the current file system entry


Instruction:
How would you code a function in Python 3 to
create an upload that returns upload id and upload url.

Response:
def create_upload_and_chunk_url(self, project_id, path_data, hash_data, remote_filename=None,
                                    storage_provider_id=None):
        """
        Create an non-chunked upload that returns upload id and upload url. This type of upload doesn't allow
        additional upload urls. For single chunk files this method is more efficient than
        create_upload/create_file_chunk_url.
        :param project_id: str: uuid of the project
        :param path_data: PathData: holds file system data about the file we are uploading
        :param hash_data: HashData: contains hash alg and value for the file we are uploading
        :param remote_filename: str: name to use for our remote file (defaults to path_data basename otherwise)
        :param storage_provider_id:str: optional storage provider id
        :return: str, dict: uuid for the upload, upload chunk url dict
        """
        upload_response = self._create_upload(project_id, path_data, hash_data, remote_filename=remote_filename,
                                              storage_provider_id=storage_provider_id, chunked=False)
        return upload_response['id'], upload_response['signed_url']


Instruction:
Given the following Python 3 function, write the documentation
def create_file_chunk_url(self, upload_id, chunk_num, chunk):
        chunk_len = len(chunk)
        hash_data = HashData.create_from_chunk(chunk)
        one_based_index = chunk_num + 1

        def func():
            return self.data_service.create_upload_url(upload_id, one_based_index, chunk_len,
                                                       hash_data.value, hash_data.alg)

        resp = retry_until_resource_is_consistent(func, self.waiting_monitor)
        return resp.json()

Response: Create a url for uploading a particular chunk to the datastore.


Instruction:
Create a Python 3 function for
sending a file to external store

Response:
def send_file_external(self, url_json, chunk):
        """
        Send chunk to external store specified in url_json.
        Raises ValueError on upload failure.
        :param data_service: data service to use for sending chunk
        :param url_json: dict contains where/how to upload chunk
        :param chunk: data to be uploaded
        """
        http_verb = url_json['http_verb']
        host = url_json['host']
        url = url_json['url']
        http_headers = url_json['http_headers']
        resp = self._send_file_external_with_retry(http_verb, host, url, http_headers, chunk)
        if resp.status_code != 200 and resp.status_code != 201:
            raise ValueError("Failed to send file to external store. Error:" + str(resp.status_code) + host + url)


Instruction:
Create a Python 3 function for
sending a file to the remote host using http_verb.

Response:
def _send_file_external_with_retry(self, http_verb, host, url, http_headers, chunk):
        """
        Send chunk to host, url using http_verb. If http_verb is PUT and a connection error occurs
        retry a few times. Pauses between retries. Raises if unsuccessful.
        """
        count = 0
        retry_times = 1
        if http_verb == 'PUT':
            retry_times = SEND_EXTERNAL_PUT_RETRY_TIMES
        while True:
            try:
                return self.data_service.send_external(http_verb, host, url, http_headers, chunk)
            except requests.exceptions.ConnectionError:
                count += 1
                if count < retry_times:
                    if count == 1:  # Only show a warning the first time we fail to send a chunk
                        self._show_retry_warning(host)
                    time.sleep(SEND_EXTERNAL_RETRY_SECONDS)
                    self.data_service.recreate_requests_session()
                else:
                    raise


Instruction:
Can you create a Python 3 function that
displays a message on stderr that we lost connection to a host.

Response:
def _show_retry_warning(host):
        """
        Displays a message on stderr that we lost connection to a host and will retry.
        :param host: str: name of the host we are trying to communicate with
        """
        sys.stderr.write("\nConnection to {} failed. Retrying.\n".format(host))
        sys.stderr.flush()


Instruction:
Can you implement a function in Python 3 that
completes the upload and create or update the file.

Response:
def finish_upload(self, upload_id, hash_data, parent_data, remote_file_id):
        """
        Complete the upload and create or update the file.
        :param upload_id: str: uuid of the upload we are completing
        :param hash_data: HashData: hash info about the file
        :param parent_data: ParentData: info about the parent of this file
        :param remote_file_id: str: uuid of this file if it already exists or None if it is a new file
        :return: dict: DukeDS details about this file
        """
        self.data_service.complete_upload(upload_id, hash_data.value, hash_data.alg)
        if remote_file_id:
            result = self.data_service.update_file(remote_file_id, upload_id)
            return result.json()
        else:
            result = self.data_service.create_file(parent_data.kind, parent_data.id, upload_id)
            return result.json()


Instruction:
Can you generate a brief explanation for the following Python 3 code
def run(self):
        processes = []
        progress_queue = ProgressQueue(Queue())
        num_chunks = ParallelChunkProcessor.determine_num_chunks(self.config.upload_bytes_per_chunk,
                                                                 self.local_file.size)
        work_parcels = ParallelChunkProcessor.make_work_parcels(self.config.upload_workers, num_chunks)
        for (index, num_items) in work_parcels:
            processes.append(self.make_and_start_process(index, num_items, progress_queue))
        wait_for_processes(processes, num_chunks, progress_queue, self.watcher, self.local_file)

Response: Runs the local file upload.


Instruction:
Can you create a Python 3 function that
determines how many pieces we can upload in a file.

Response:
def determine_num_chunks(chunk_size, file_size):
        """
        Figure out how many pieces we are sending the file in.
        NOTE: duke-data-service requires an empty chunk to be uploaded for empty files.
        """
        if file_size == 0:
            return 1
        return int(math.ceil(float(file_size) / float(chunk_size)))


Instruction:
Write a Python 3 function that can
make a list of parcels for the given number of workers and number of items.

Response:
def make_work_parcels(upload_workers, num_chunks):
        """
        Make groups so we can split up num_chunks into similar sizes.
        Rounds up trying to keep work evenly split so sometimes it will not use all workers.
        For very small numbers it can result in (upload_workers-1) total workers.
        For example if there are two few items to distribute.
        :param upload_workers: int target number of workers
        :param num_chunks: int number of total items we need to send
        :return [(index, num_items)] -  an array of tuples where array element will be in a separate process.
        """
        chunks_per_worker = int(math.ceil(float(num_chunks) / float(upload_workers)))
        return ParallelChunkProcessor.divide_work(range(num_chunks), chunks_per_worker)


Instruction:
How would you explain what the following Python 3 function does
def divide_work(list_of_indexes, batch_size):
        grouped_indexes = [list_of_indexes[i:i + batch_size] for i in range(0, len(list_of_indexes), batch_size)]
        return [(batch[0], len(batch)) for batch in grouped_indexes]

Response: Given a sequential list of indexes split them into num_parts.


Instruction:
Given the following Python 3 function, write the documentation
def make_and_start_process(self, index, num_items, progress_queue):
        process = Process(target=upload_async,
                          args=(self.data_service.auth.get_auth_data(), self.config, self.upload_id,
                                self.local_file.path, index, num_items, progress_queue))
        process.start()
        return process

Response: Create and start a process to upload num_items chunks from our local file starting at index.


Instruction:
Write a Python 3 function that can
send the file to the server.

Response:
def send(self):
        """
        For each chunk we need to send, create upload url and send bytes. Raises exception on error.
        """
        sent_chunks = 0
        chunk_num = self.index
        with open(self.filename, 'rb') as infile:
            infile.seek(self.index * self.chunk_size)
            while sent_chunks != self.num_chunks_to_send:
                chunk = infile.read(self.chunk_size)
                self._send_chunk(chunk, chunk_num)
                self.progress_queue.processed(1)
                chunk_num += 1
                sent_chunks += 1


Instruction:
Make a summary of the following Python 3 code
def _send_chunk(self, chunk, chunk_num):
        url_info = self.upload_operations.create_file_chunk_url(self.upload_id, chunk_num, chunk)
        self.upload_operations.send_file_external(url_info, chunk)

Response: Send a single chunk to the remote service.


Instruction:
Can you write a function in Python 3 where it
returns True if the given parameter is within its valid interval.

Response:
def _in_valid_interval(self, parameter, value):
        """Returns if the parameter is within its valid interval.

        :param string parameter:     Name of the parameter that has to be checked.
        :param numeric value:     Value of the parameter.

        :return:    Returns :py:const:`True` it the value for the given parameter is valid,
            :py:const:`False` otherwise.
        :rtype: boolean
        """
        # return True, if not interval is defined for the parameter
        if parameter not in self._parameterIntervals:
            return True

        interval = self._parameterIntervals[parameter]

        if interval[2] and interval[3]:
            return interval[0] <= value <= interval[1]

        if not interval[2] and interval[3]:
            return interval[0] <  value <= interval[1]

        if interval[2] and not interval[3]:
            return interval[0] <= value <  interval[1]

        #if False == interval[2] and False == interval[3]:
        return interval[0] < value < interval[1]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _get_value_error_message_for_invalid_prarameter(self, parameter, value):
        # return if not interval is defined for the parameter
        if parameter not in self._parameterIntervals:
            return

        interval = self._parameterIntervals[parameter]
        return "%s has to be in %s%s, %s%s. Current value is %s." % (
            parameter,
            BaseMethod._interval_definitions[interval[2]][0],
            interval[0], interval[1],
            BaseMethod._interval_definitions[interval[3]][1],
            value
        )

Response: Returns the error message for the given parameter and value.


Instruction:
Can you tell what is the following Python 3 function doing
def set_parameter(self, name, value):
        if not self._in_valid_interval(name, value):
            raise ValueError(self._get_value_error_message_for_invalid_prarameter(name, value))

        #if name in self._parameters:
        #    print "Parameter %s already existed. It's old value will be replaced with %s" % (name, value)

        self._parameters[name] = value

Response: Sets a parameter for the BaseMethod.


Instruction:
Implement a function in Python 3 to
return if the method can already be executed.

Response:
def can_be_executed(self):
        """Returns if the method can already be executed.

        :return:    Returns :py:const:`True` if all required parameters where already set, False otherwise.
        :rtype: boolean
        """
        missingParams = filter(lambda rp: rp not in self._parameters, self._requiredParameters)
        return len(missingParams) == 0


Instruction:
Implement a function in Python 3 to
set a parameter for the BaseForecastingMethod.

Response:
def set_parameter(self, name, value):
        """Sets a parameter for the BaseForecastingMethod.

        :param string name:    Name of the parameter.
        :param numeric value:    Value of the parameter.
        """
        # set the furecast until variable to None if necessary
        if name == "valuesToForecast":
            self._forecastUntil = None

        # continue with the parents implementation
        return super(BaseForecastingMethod, self).set_parameter(name, value)


Instruction:
Write a Python 3 function that can
set the forecasting goal ( timestamp wise.

Response:
def forecast_until(self, timestamp, tsformat=None):
        """Sets the forecasting goal (timestamp wise).

        This function enables the automatic determination of valuesToForecast.

        :param timestamp:    timestamp containing the end date of the forecast.
        :param string tsformat:    Format of the timestamp. This is used to convert the
            timestamp from UNIX epochs, if necessary. For valid examples
            take a look into the :py:func:`time.strptime` documentation.
        """
        if tsformat is not None:
            timestamp = TimeSeries.convert_timestamp_to_epoch(timestamp, tsformat)

        self._forecastUntil = timestamp


Instruction:
Write a Python 3 function that can
calculate the number of values that need to be forecasted to match the goal set in forecast_until.

Response:
def _calculate_values_to_forecast(self, timeSeries):
        """Calculates the number of values, that need to be forecasted to match the goal set in forecast_until.

        This sets the parameter "valuesToForecast" and should be called at the beginning of the :py:meth:`BaseMethod.execute` implementation.

        :param TimeSeries timeSeries:    Should be a sorted and normalized TimeSeries instance.

        :raise:    Raises a :py:exc:`ValueError` if the TimeSeries is either not normalized or sorted.
        """
        # do not set anything, if it is not required
        if self._forecastUntil is None:
            return

        # check the TimeSeries for correctness
        if not timeSeries.is_sorted():
            raise ValueError("timeSeries has to be sorted.")
        if not timeSeries.is_normalized():
            raise ValueError("timeSeries has to be normalized.")

        timediff = timeSeries[-1][0] - timeSeries[-2][0]
        forecastSpan = self._forecastUntil - timeSeries[-1][0]

        self.set_parameter("valuesToForecast", int(forecastSpan / timediff) + 1)


Instruction:
Implement a function in Python 3 to
execute the simple moving average.

Response:
def execute(self, timeSeries):
        """Creates a new TimeSeries containing the SMA values for the predefined windowsize.

        :param TimeSeries timeSeries:    The TimeSeries used to calculate the simple moving average values.

        :return:    TimeSeries object containing the smooth moving average.
        :rtype:     TimeSeries

        :raise:   Raises a :py:exc:`ValueError` wif the defined windowsize is larger than the number of elements
            in timeSeries

        :note:    This implementation aims to support independent for loop execution.
        """
        windowsize    = self._parameters["windowsize"]

        if len (timeSeries) < windowsize:
            raise ValueError("windowsize is larger than the number of elements in timeSeries.")

        tsLength      = len(timeSeries)
        nbrOfLoopRuns = tsLength - windowsize + 1

        res = TimeSeries()
        for idx in xrange(nbrOfLoopRuns):
            end = idx + windowsize
            data = timeSeries[idx:end]

            timestamp = data[windowsize//2][0]
            value     = sum([i[1] for i in data])/windowsize

            res.add_entry(timestamp, value)

        res.sort_timeseries()
        return res


Instruction:
Explain what the following Python 3 code does
def add(self, child, min_occurs=1):
        if not min_occurs in (0, 1):
            raise RuntimeError("Unexpected min bound for node schema")
        self.children[child.tag] = child
        self.children_min_occurs[child.tag] = min_occurs
        return child

Response: Add a child node.


Instruction:
How would you explain what the following Python 3 function does
def _create_child(self, tag):
        return etree.SubElement(self._root, self._get_namespace_tag(tag))

Response: Create a new child element with the given tag.


Instruction:
Can you generate the documentation for the following Python 3 function
def _find_child(self, tag):
        tag = self._get_namespace_tag(tag)
        children = self._root.findall(tag)
        if len(children) > 1:
            raise WSDLParseError("Duplicate tag '%s'" % tag)
        if len(children) == 0:
            return None
        return children[0]

Response: Find the child with the matching tag.


Instruction:
How would you implement a function in Python 3 that
ensures that the element matching the given tag can have the given value.

Response:
def _check_value(self, tag, value):
        """Ensure that the element matching C{tag} can have the given C{value}.

        @param tag: The tag to consider.
        @param value: The value to check
        @return: The unchanged L{value}, if valid.
        @raises L{WSDLParseError}: If the value is invalid.
        """
        if value is None:
            if self._schema.children_min_occurs[tag] > 0:
                raise WSDLParseError("Missing tag '%s'" % tag)
            return value
        return value


Instruction:
Can you create a Python 3 function that
gets the L { NodeItem } attribute name for the given C { tag.

Response:
def _get_tag(self, name):
        """Get the L{NodeItem} attribute name for the given C{tag}."""
        if name.endswith("_"):
            if name[:-1] in self._schema.reserved:
                return name[:-1]
        return name


Instruction:
Can you generate the documentation for the following Python 3 function
def _get_namespace_tag(self, tag):
        if self._namespace is not None:
            tag = "{%s}%s" % (self._namespace, tag)
        return tag

Response: Return the given C { tag with the namespace prefix added if any.


Instruction:
Implement a Python 3 function for
returning the child schema for the given tag.

Response:
def _get_schema(self, tag):
        """Return the child schema for the given C{tag}.

        @raises L{WSDLParseError}: If the tag doesn't belong to the schema.
        """
        schema = self._schema.children.get(tag)
        if not schema:
            raise WSDLParseError("Unknown tag '%s'" % tag)
        return schema


Instruction:
Write a Python 3 script for
creating a sequence element with the given root.

Response:
def create(self, root=None, namespace=None):
        """Create a sequence element with the given root.

        @param root: The C{etree.Element} to root the sequence at, if C{None} a
            new one will be created..
        @result: A L{SequenceItem} with the given root.
        @raises L{ECResponseError}: If the given C{root} has a bad tag.
        """
        if root is not None:
            tag = root.tag
            if root.nsmap:
                namespace = root.nsmap[None]
                tag = tag[len(namespace) + 2:]
            if tag != self.tag:
                raise WSDLParseError("Expected response with tag '%s', but "
                                       "got '%s' instead" % (self.tag, tag))
        return SequenceItem(self, root, namespace)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def set(self, child, min_occurs=1, max_occurs=1):
        if isinstance(child, LeafSchema):
            raise RuntimeError("Sequence can't have leaf children")
        if self.child is not None:
            raise RuntimeError("Sequence has already a child")
        if min_occurs is None or max_occurs is None:
            raise RuntimeError("Sequence node without min or max")
        if isinstance(child, LeafSchema):
            raise RuntimeError("Sequence node with leaf child type")
        if not child.tag == "item":
            raise RuntimeError("Sequence node with bad child tag")

        self.child = child
        self.min_occurs = min_occurs
        self.max_occurs = max_occurs
        return child

Response: Set the schema for the sequence children.


Instruction:
Implement a function in Python 3 to
append a new item to the sequence.

Response:
def append(self):
        """Append a new item to the sequence, appending it to the end.

        @return: The newly created item.
        @raises L{WSDLParseError}: If the operation would result in having
             more child elements than the allowed max.
        """
        tag = self._schema.tag
        children = self._root.getchildren()
        if len(children) >= self._schema.max_occurs:
            raise WSDLParseError("Too many items in tag '%s'" % tag)
        schema = self._schema.child
        tag = "item"
        if self._namespace is not None:
            tag = "{%s}%s" % (self._namespace, tag)
        child = etree.SubElement(self._root, tag)
        return schema.create(child)


Instruction:
Can you generate the documentation for the following Python 3 function
def remove(self, item):
        for index, child in enumerate(self._root.getchildren()):
            if child is item._root:
                del self[index]
                return item
        raise WSDLParseError("Non existing item in tag '%s'" %
                               self._schema.tag)

Response: Removes the given item from the sequence.


Instruction:
How would you implement a function in Python 3 that
returns the child with the given index.

Response:
def _get_child(self, children, index):
        """Return the child with the given index."""
        try:
            return children[index]
        except IndexError:
            raise WSDLParseError("Non existing item in tag '%s'" %
                                   self._schema.tag)


Instruction:
Make a summary of the following Python 3 code
def parse(self, wsdl):
        parser = etree.XMLParser(remove_blank_text=True, remove_comments=True)
        root = etree.fromstring(wsdl, parser=parser)
        types = {}
        responses = {}
        schemas = {}
        namespace = root.attrib["targetNamespace"]

        for element in root[0][0]:
            self._remove_namespace_from_tag(element)
            if element.tag in ["annotation", "group"]:
                continue
            name = element.attrib["name"]
            if element.tag == "element":
                if name.endswith("Response"):
                    if name in responses:
                        raise RuntimeError("Schema already defined")
                    responses[name] = element
            elif element.tag == "complexType":
                types[name] = [element, False]
            else:
                raise RuntimeError("Top-level element with unexpected tag")

        for name, element in responses.iteritems():
            schemas[name] = self._parse_type(element, types)
            schemas[name].namespace = namespace

        return schemas

Response:
Parse the given WDSL definition
            and build the associated schemas.


Instruction:
Given the following Python 3 function, write the documentation
def _parse_type(self, element, types):
        name = element.attrib["name"]
        type = element.attrib["type"]
        if not type.startswith("tns:"):
            raise RuntimeError("Unexpected element type %s" % type)
        type = type[4:]

        [children] = types[type][0]
        types[type][1] = True

        self._remove_namespace_from_tag(children)
        if children.tag not in ("sequence", "choice"):
            raise RuntimeError("Unexpected children type %s" % children.tag)

        if children[0].attrib["name"] == "item":
            schema = SequenceSchema(name)
        else:
            schema = NodeSchema(name)

        for child in children:
            self._remove_namespace_from_tag(child)
            if child.tag == "element":
                name, type, min_occurs, max_occurs = self._parse_child(child)
                if type in self.leaf_types:
                    if max_occurs != 1:
                        raise RuntimeError("Unexpected max value for leaf")
                    if not isinstance(schema, NodeSchema):
                        raise RuntimeError("Attempt to add leaf to a non-node")
                    schema.add(LeafSchema(name), min_occurs=min_occurs)
                else:
                    if name == "item":  # sequence
                        if not isinstance(schema, SequenceSchema):
                            raise RuntimeError("Attempt to set child for "
                                               "non-sequence")
                        schema.set(self._parse_type(child, types),
                                   min_occurs=min_occurs,
                                   max_occurs=max_occurs)
                    else:
                        if max_occurs != 1:
                            raise RuntimeError("Unexpected max for node")
                        if not isinstance(schema, NodeSchema):
                            raise RuntimeError("Unexpected schema type")
                        schema.add(self._parse_type(child, types),
                                   min_occurs=min_occurs)
            elif child.tag == "choice":
                pass
            else:
                raise RuntimeError("Unexpected child type")
        return schema

Response: Parse a complexType element.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _parse_child(self, child):
        if set(child.attrib) - set(["name", "type", "minOccurs", "maxOccurs"]):
            raise RuntimeError("Unexpected attribute in child")
        name = child.attrib["name"]
        type = child.attrib["type"].split(":")[1]
        min_occurs = child.attrib.get("minOccurs")
        max_occurs = child.attrib.get("maxOccurs")
        if min_occurs is None:
            min_occurs = "1"
        min_occurs = int(min_occurs)
        if max_occurs is None:
            max_occurs = "1"
        if max_occurs != "unbounded":
            max_occurs = int(max_occurs)
        return name, type, min_occurs, max_occurs

Response: Parse a single child element.


Instruction:
Can you write a function in Python 3 where it
gets existing item from remote service based on keys.

Response:
def get_existing_item(self, item):
        """
        Lookup item in remote service based on keys.
        :param item: D4S2Item data contains keys we will use for lookup.
        :return: requests.Response containing the successful result
        """
        params = {
            'project_id': item.project_id,
            'from_user_id': item.from_user_id,
            'to_user_id': item.to_user_id,
        }
        resp = requests.get(self.make_url(item.destination), headers=self.json_headers, params=params)
        self.check_response(resp)
        return resp


Instruction:
Here you have a function in Python 3, explain what it does
def create_item(self, item):
        item_dict = {
            'project_id': item.project_id,
            'from_user_id': item.from_user_id,
            'to_user_id': item.to_user_id,
            'role': item.auth_role,
            'user_message': item.user_message
        }
        if item.share_user_ids:
            item_dict['share_user_ids'] = item.share_user_ids
        data = json.dumps(item_dict)
        resp = requests.post(self.make_url(item.destination), headers=self.json_headers, data=data)
        self.check_response(resp)
        return resp

Response: Create a new item in D4S2 service for the specified item.


Instruction:
Create a Python 3 function to
send an item to a specific service

Response:
def send_item(self, destination, item_id, force_send):
        """
        Run send method for item_id at destination.
        :param destination: str which type of operation are we doing (SHARE_DESTINATION or DELIVER_DESTINATION)
        :param item_id: str D4S2 service id representing the item we want to send
        :param force_send: bool it's ok to email the item again
        :return: requests.Response containing the successful result
        """
        data = json.dumps({
            'force': force_send,
        })
        url_suffix = "{}/send/".format(item_id)
        resp = requests.post(self.make_url(destination, url_suffix), headers=self.json_headers, data=data)
        self.check_response(resp)
        return resp


Instruction:
Write a Python 3 function that can
check if the response is successful. Raises error if the response is not 200.

Response:
def check_response(self, response):
        """
        Raises error if the response isn't successful.
        :param response: requests.Response response to be checked
        """
        if response.status_code == 401:
            raise D4S2Error(UNAUTHORIZED_MESSAGE)
        if not 200 <= response.status_code < 300:
            raise D4S2Error("Request to {} failed with {}:\n{}.".format(response.url, response.status_code,
                                                                        response.text))


Instruction:
Here you have a function in Python 3, explain what it does
def send(self, api, force_send):
        item_id = self.get_existing_item_id(api)
        if not item_id:
            item_id = self.create_item_returning_id(api)
            api.send_item(self.destination, item_id, force_send)
        else:
            if force_send:
                api.send_item(self.destination, item_id, force_send)
            else:
                item_type = D4S2Api.DEST_TO_NAME.get(self.destination, "Item")
                msg = "{} already sent. Run with --resend argument to resend."
                raise D4S2Error(msg.format(item_type), warning=True)

Response: Send this item using D4S2Api


Instruction:
Can you implement a function in Python 3 that
gets the id of the item if it exists else None.

Response:
def get_existing_item_id(self, api):
        """
        Lookup the id for this item via the D4S2 service.
        :param api: D4S2Api object who communicates with D4S2 server.
        :return str id of this item or None if not found
        """
        resp = api.get_existing_item(self)
        items = resp.json()
        num_items = len(items)
        if num_items == 0:
            return None
        else:
            return items[0]['id']


Instruction:
Given the following Python 3 function, write the documentation
def create_item_returning_id(self, api):
        resp = api.create_item(self)
        item = resp.json()
        return item['id']

Response: Create this item in the D4S2 service and return the id of the newly created item


Instruction:
Given the following Python 3 function, write the documentation
def share(self, project, to_user, force_send, auth_role, user_message):
        if self._is_current_user(to_user):
            raise ShareWithSelfError(SHARE_WITH_SELF_MESSAGE.format("share"))
        if not to_user.email:
            self._raise_user_missing_email_exception("share")
        self.set_user_project_permission(project, to_user, auth_role)
        return self._share_project(D4S2Api.SHARE_DESTINATION, project, to_user, force_send, auth_role, user_message)

Response: Send mail and give user specified access to the project.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def set_user_project_permission(self, project, user, auth_role):
        self.remote_store.set_user_project_permission(project, user, auth_role)

Response: Set user access permissions for a project.


Instruction:
Write a Python 3 script for
sending a message to the service to email user for the project to_user.

Response:
def deliver(self, project, new_project_name, to_user, share_users, force_send, path_filter, user_message):
        """
        Remove access to project_name for to_user, copy to new_project_name if not None,
        send message to service to email user so they can have access.
        :param project: RemoteProject pre-existing project to be delivered
        :param new_project_name: str name of non-existing project to copy project_name to, if None we don't copy
        :param to_user: RemoteUser user we are handing over the project to
        :param share_users: [RemoteUser] who will have project shared with them once to_user accepts the project
        :param force_send: boolean enables resending of email for existing projects
        :param path_filter: PathFilter: filters what files are shared
        :param user_message: str message to be sent with the share
        :return: str email we sent deliver to
        """
        if self._is_current_user(to_user):
            raise ShareWithSelfError(SHARE_WITH_SELF_MESSAGE.format("deliver"))
        if not to_user.email:
            self._raise_user_missing_email_exception("deliver")
        self.remove_user_permission(project, to_user)
        if new_project_name:
            project = self._copy_project(project, new_project_name, path_filter)
        return self._share_project(D4S2Api.DELIVER_DESTINATION, project, to_user,
                                   force_send, user_message=user_message, share_users=share_users)


Instruction:
Explain what the following Python 3 code does
def _share_project(self, destination, project, to_user, force_send, auth_role='', user_message='',
                       share_users=None):
        """
        Send message to remote service to email/share project with to_user.
        :param destination: str which type of sharing we are doing (SHARE_DESTINATION or DELIVER_DESTINATION)
        :param project: RemoteProject project we are sharing
        :param to_user: RemoteUser user we are sharing with
        :param auth_role: str project role eg 'project_admin' email is customized based on this setting.
        :param user_message: str message to be sent with the share
        :param share_users: [RemoteUser] users to have this project shared with after delivery (delivery only)
        :return: the email the user should receive a message on soon
        """
        from_user = self.remote_store.get_current_user()
        share_user_ids = None
        if share_users:
            share_user_ids = [share_user.id for share_user in share_users]
        item = D4S2Item(destination=destination,
                        from_user_id=from_user.id,
                        to_user_id=to_user.id,
                        project_id=project.id,
                        project_name=project.name,
                        auth_role=auth_role,
                        user_message=user_message,
                        share_user_ids=share_user_ids)
        item.send(self.api, force_send)
        return to_user.email

Response: Send message to remote service to email to share project with to_user.


Instruction:
How would you implement a function in Python 3 that
copies project to new_project_name.

Response:
def _copy_project(self, project, new_project_name, path_filter):
        """
        Copy pre-existing project with name project_name to non-existing project new_project_name.
        :param project: remotestore.RemoteProject project to copy from
        :param new_project_name: str project to copy to
        :param path_filter: PathFilter: filters what files are shared
        :return: RemoteProject new project we copied data to
        """
        temp_directory = tempfile.mkdtemp()
        new_project_name_or_id = ProjectNameOrId.create_from_name(new_project_name)
        remote_project = self.remote_store.fetch_remote_project(new_project_name_or_id)
        if remote_project:
            raise ValueError("A project with name '{}' already exists.".format(new_project_name))
        activity = CopyActivity(self.remote_store.data_service, project, new_project_name)
        self._download_project(activity, project, temp_directory, path_filter)
        self._upload_project(activity, new_project_name, temp_directory)
        activity.finished()
        shutil.rmtree(temp_directory)
        return self.remote_store.fetch_remote_project(new_project_name_or_id, must_exist=True)


Instruction:
Write a Python 3 function for
downloading the project with project_name to temp_directory.

Response:
def _download_project(self, activity, project, temp_directory, path_filter):
        """
        Download the project with project_name to temp_directory.
        :param activity: CopyActivity: info about the copy activity are downloading for
        :param project: remotestore.RemoteProject project to download
        :param temp_directory: str path to directory we can download into
        :param path_filter: PathFilter: filters what files are shared
        """
        self.print_func("Downloading a copy of '{}'.".format(project.name))
        project_download = ProjectDownload(self.remote_store, project, temp_directory, path_filter,
                                           file_download_pre_processor=DownloadedFileRelations(activity))
        project_download.run()


Instruction:
Create a Python 3 function for
uploading the contents of temp_directory into project_name

Response:
def _upload_project(self, activity, project_name, temp_directory):
        """
        Upload the contents of temp_directory into project_name
        :param activity: CopyActivity: info about the copy activity are uploading for
        :param project_name: str project name we will upload files to
        :param temp_directory: str path to directory who's files we will upload
        """
        self.print_func("Uploading to '{}'.".format(project_name))
        items_to_send = [os.path.join(temp_directory, item) for item in os.listdir(os.path.abspath(temp_directory))]
        project_name_or_id = ProjectNameOrId.create_from_name(project_name)
        project_upload = ProjectUpload(self.config, project_name_or_id, items_to_send,
                                       file_upload_post_processor=UploadedFileRelations(activity))
        project_upload.run()


Instruction:
Write a Python 3 script to
check if the current user is the passed in user

Response:
def _is_current_user(self, some_user):
        """
        Is the specified user the current user?
        :param some_user: RemoteUser user we want to check against the current user
        :return: boolean: True if the current user is the passed in user
        """
        current_user = self.remote_store.get_current_user()
        return current_user.id == some_user.id


Instruction:
Given the following Python 3 function, write the documentation
def finished(self):
        self.data_service.update_activity(self.id, self.name, self.desc,
                                          started_on=self.started,
                                          ended_on=self._current_timestamp_str())

Response: Mark the activity as finished.


Instruction:
Here you have a function in Python 3, explain what it does
def run(self, data_service, project_file):
        remote_path = project_file.path
        file_dict = data_service.get_file(project_file.id).json()
        file_version_id = file_dict['current_version']['id']
        data_service.create_used_relation(self.activity.id, KindType.file_str, file_version_id)
        self.activity.remote_path_to_file_version_id[remote_path] = file_version_id

Response: Attach a remote file to activity with used relationship.


Instruction:
Can you generate the documentation for the following Python 3 function
def run(self, data_service, file_details):
        file_version_id = file_details['current_version']['id']
        data_service.create_was_generated_by_relation(self.activity.id, KindType.file_str, file_version_id)
        used_entity_id = self._lookup_used_entity_id(file_details)
        data_service.create_was_derived_from_relation(used_entity_id, KindType.file_str,
                                                      file_version_id, KindType.file_str)

Response: Attach a remote file to activity with was generated by relationship.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _lookup_used_entity_id(self, file_details):
        # Since this uses the response from POST to /files/ this will include the ancestors and not be
        # effected by exclude_response_fields that were used when listing the project
        name_parts = [ancestor['name'] for ancestor in file_details['ancestors']
                      if ancestor['kind'] == KindType.folder_str]
        name_parts.append(file_details['name'])
        remote_path = os.sep.join(name_parts)
        return self.activity.remote_path_to_file_version_id[remote_path]

Response: Return the file_version_id associated with the path from file_details.


Instruction:
Make a summary of the following Python 3 code
def create_config(allow_insecure_config_file=False):
    config = Config()
    config.add_properties(GLOBAL_CONFIG_FILENAME)
    user_config_filename = get_user_config_filename()
    if user_config_filename == LOCAL_CONFIG_FILENAME and not allow_insecure_config_file:
        verify_file_private(user_config_filename)
    config.add_properties(user_config_filename)
    return config

Response: Create config based on the user s configuration file.


Instruction:
Explain what the following Python 3 code does
def add_properties(self, filename):
        filename = os.path.expanduser(filename)
        if os.path.exists(filename):
            with open(filename, 'r') as yaml_file:
                self.update_properties(yaml.safe_load(yaml_file))

Response: Add properties from a YAML file to the config based on filename replacing previous values.


Instruction:
How would you explain what the following Python 3 function does
def get_portal_url_base(self):
        api_url = urlparse(self.url).hostname
        portal_url = re.sub('^api\.', '', api_url)
        portal_url = re.sub(r'api', '', portal_url)
        return portal_url

Response: Determine the root url of the data service from the url specified.


Instruction:
Can you generate the documentation for the following Python 3 function
def auth(self):
        return self.values.get(Config.AUTH, os.environ.get(AUTH_ENV_KEY_NAME, None))

Response: Returns the auth token for use with connecting to the dataservice.


Instruction:
Can you create a Python 3 function that
returns the number of bytes per chunk to be sent to external store.

Response:
def upload_bytes_per_chunk(self):
        """
        Return the bytes per chunk to be sent to external store.
        :return: int bytes per upload chunk
        """
        value = self.values.get(Config.UPLOAD_BYTES_PER_CHUNK, DDS_DEFAULT_UPLOAD_CHUNKS)
        return Config.parse_bytes_str(value)


Instruction:
Implement a Python 3 function for
returning the number of parallel works to use when downloading a file.

Response:
def download_workers(self):
        """
        Return the number of parallel works to use when downloading a file.
        :return: int number of workers. Specify None or 1 to disable parallel downloading
        """
        # Profiling download on different servers showed half the number of CPUs to be optimum for speed.
        default_workers = int(math.ceil(default_num_workers() / 2))
        return self.values.get(Config.DOWNLOAD_WORKERS, default_workers)


Instruction:
Can you tell what is the following Python 3 function doing
def parse_bytes_str(value):
        if type(value) == str:
            if "MB" in value:
                return int(value.replace("MB", "")) * MB_TO_BYTES
            else:
                return int(value)
        else:
            return value

Response: Given a value return the integer number of bytes it represents. Trailing Mb causes the value multiplied by 1024 * 1024


Instruction:
Can you implement a function in Python 3 that
calculates the error between two given values.

Response:
def local_error(self, originalValue, calculatedValue):
        """Calculates the error between the two given values.

        :param list originalValue:    List containing the values of the original data.
        :param list calculatedValue:    List containing the values of the calculated TimeSeries that
            corresponds to originalValue.

        :return:    Returns the error measure of the two given values.
        :rtype:     numeric
        """
        originalValue = originalValue[0]
        calculatedValue = calculatedValue[0]

        return abs(originalValue - calculatedValue)


Instruction:
Can you tell what is the following Python 3 function doing
def run_instances(self, image_id, min_count, max_count,
        security_groups=None, key_name=None, instance_type=None,
        user_data=None, availability_zone=None, kernel_id=None,
        ramdisk_id=None, subnet_id=None, security_group_ids=None):
        """Run new instances.

        TODO: blockDeviceMapping, monitoring, subnetId
        """
        params = {"ImageId": image_id, "MinCount": str(min_count),
                  "MaxCount": str(max_count)}
        if key_name is not None:
            params["KeyName"] = key_name
        if subnet_id is not None:
            params["SubnetId"] = subnet_id
            if security_group_ids is not None:
                for i, id in enumerate(security_group_ids):
                    params["SecurityGroupId.%d" % (i + 1)] = id
            else:
                msg = "You must specify the security_group_ids with the subnet_id"
                raise ValueError(msg)
        elif security_groups is not None:
            for i, name in enumerate(security_groups):
                params["SecurityGroup.%d" % (i + 1)] = name
        else:
            msg = ("You must specify either the subnet_id and "
                   "security_group_ids or security_groups")
            raise ValueError(msg)
        if user_data is not None:
            params["UserData"] = b64encode(user_data)
        if instance_type is not None:
            params["InstanceType"] = instance_type
        if availability_zone is not None:
            params["Placement.AvailabilityZone"] = availability_zone
        if kernel_id is not None:
            params["KernelId"] = kernel_id
        if ramdisk_id is not None:
            params["RamdiskId"] = ramdisk_id
        query = self.query_factory(
            action="RunInstances", creds=self.creds, endpoint=self.endpoint,
            other_params=params)
        d = query.submit()
        return d.addCallback(self.parser.run_instances)

Response:
Run new instances.

        TODO: blockDeviceMapping, monitoring, subnetId


Instruction:
Explain what the following Python 3 code does
def get_console_output(self, instance_id):
        InstanceIDParam = {"InstanceId": instance_id}
        query = self.query_factory(
            action="GetConsoleOutput", creds=self.creds,
            endpoint=self.endpoint, other_params=InstanceIDParam)
        d = query.submit()
        return d.addCallback(self.parser.get_console_output)

Response: Get the console output for a single instance.


Instruction:
Can you tell what is the following Python 3 function doing
def describe_security_groups(self, *names):
        group_names = {}
        if names:
            group_names = dict([("GroupName.%d" % (i + 1), name)
                                for i, name in enumerate(names)])
        query = self.query_factory(
            action="DescribeSecurityGroups", creds=self.creds,
            endpoint=self.endpoint, other_params=group_names)
        d = query.submit()
        return d.addCallback(self.parser.describe_security_groups)

Response:
Describe security groups.

        @param names: Optionally, a list of security group names to describe.
            Defaults to all security groups in the account.
        @return: A C{Deferred} that will fire with a list of L{SecurityGroup}s
            retrieved from the cloud.


Instruction:
Can you generate the documentation for the following Python 3 function
def create_security_group(self, name, description, vpc_id=None):
        parameters = {"GroupName":  name, "GroupDescription": description}
        if vpc_id:
            parameters["VpcId"] = vpc_id
        query = self.query_factory(
            action="CreateSecurityGroup", creds=self.creds,
            endpoint=self.endpoint, other_params=parameters)
        d = query.submit()
        return d.addCallback(self.parser.create_security_group)

Response: Create a security group.


Instruction:
Can you generate the documentation for the following Python 3 function
def authorize_security_group(
        self, group_name=None, group_id=None, source_group_name="", source_group_owner_id="",
        ip_protocol="", from_port="", to_port="", cidr_ip=""):
        """
        There are two ways to use C{authorize_security_group}:
            1) associate an existing group (source group) with the one that you
            are targeting (group_name) with an authorization update; or
            2) associate a set of IP permissions with the group you are
            targeting with an authorization update.

        @param group_name: The group you will be modifying with a new
            authorization.
        @param group_id: The id of the group you will be modifying with
            a new authorization.

        Optionally, the following parameters:
        @param source_group_name: Name of security group to authorize access to
            when operating on a user/group pair.
        @param source_group_owner_id: Owner of security group to authorize
            access to when operating on a user/group pair.

        If those parameters are not specified, then the following must be:
        @param ip_protocol: IP protocol to authorize access to when operating
            on a CIDR IP.
        @param from_port: Bottom of port range to authorize access to when
            operating on a CIDR IP. This contains the ICMP type if ICMP is
            being authorized.
        @param to_port: Top of port range to authorize access to when operating
            on a CIDR IP. This contains the ICMP code if ICMP is being
            authorized.
        @param cidr_ip: CIDR IP range to authorize access to when operating on
            a CIDR IP.

        @return: A C{Deferred} that will fire with a truth value for the
            success of the operation.
        """
        if source_group_name and source_group_owner_id:
            parameters = {
                "SourceSecurityGroupName": source_group_name,
                "SourceSecurityGroupOwnerId": source_group_owner_id,
                }
        elif ip_protocol and from_port and to_port and cidr_ip:
            parameters = {
                "IpProtocol": ip_protocol,
                "FromPort": from_port,
                "ToPort": to_port,
                "CidrIp": cidr_ip,
                }
        else:
            msg = ("You must specify either both group parameters or "
                   "all the ip parameters.")
            raise ValueError(msg)
        if group_id:
            parameters["GroupId"] = group_id
        elif group_name:
            parameters["GroupName"] = group_name
        else:
            raise ValueError("You must specify either the group name of the group id.")
        query = self.query_factory(
            action="AuthorizeSecurityGroupIngress", creds=self.creds,
            endpoint=self.endpoint, other_params=parameters)
        d = query.submit()
        return d.addCallback(self.parser.truth_return)

Response: This method is used to associate an existing security group with a new security group.


Instruction:
Here you have a function in Python 3, explain what it does
def authorize_ip_permission(
        self, group_name, ip_protocol, from_port, to_port, cidr_ip):
        """
        This is a convenience function that wraps the "authorize ip
        permission" functionality of the C{authorize_security_group} method.

        For an explanation of the parameters, see C{authorize_security_group}.
        """
        d = self.authorize_security_group(
            group_name,
            ip_protocol=ip_protocol, from_port=from_port, to_port=to_port,
            cidr_ip=cidr_ip)
        return d

Response:
This is a convenience method that wraps the authorize ip
        permission functionality of the C { authorize_security_group method }.


Instruction:
Can you tell what is the following Python 3 function doing
def revoke_group_permission(
        self, group_name, source_group_name, source_group_owner_id):
        """
        This is a convenience function that wraps the "authorize group"
        functionality of the C{authorize_security_group} method.

        For an explanation of the parameters, see C{revoke_security_group}.
        """
        d = self.revoke_security_group(
            group_name,
            source_group_name=source_group_name,
            source_group_owner_id=source_group_owner_id)
        return d

Response: This method is used to revoke group permissions for a group.


Instruction:
Can you write a function in Python 3 where it
creates a new volume.

Response:
def create_volume(self, availability_zone, size=None, snapshot_id=None):
        """Create a new volume."""
        params = {"AvailabilityZone": availability_zone}
        if ((snapshot_id is None and size is None) or
            (snapshot_id is not None and size is not None)):
            raise ValueError("Please provide either size or snapshot_id")
        if size is not None:
            params["Size"] = str(size)
        if snapshot_id is not None:
            params["SnapshotId"] = snapshot_id
        query = self.query_factory(
            action="CreateVolume", creds=self.creds, endpoint=self.endpoint,
            other_params=params)
        d = query.submit()
        return d.addCallback(self.parser.create_volume)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def describe_snapshots(self, *snapshot_ids):
        snapshot_set = {}
        for pos, snapshot_id in enumerate(snapshot_ids):
            snapshot_set["SnapshotId.%d" % (pos + 1)] = snapshot_id
        query = self.query_factory(
            action="DescribeSnapshots", creds=self.creds,
            endpoint=self.endpoint, other_params=snapshot_set)
        d = query.submit()
        return d.addCallback(self.parser.snapshots)

Response:
Describe available snapshots.

        TODO: ownerSet, restorableBySet


Instruction:
Implement a function in Python 3 to
remove a previously created snapshot.

Response:
def delete_snapshot(self, snapshot_id):
        """Remove a previously created snapshot."""
        query = self.query_factory(
            action="DeleteSnapshot", creds=self.creds, endpoint=self.endpoint,
            other_params={"SnapshotId": snapshot_id})
        d = query.submit()
        return d.addCallback(self.parser.truth_return)


Instruction:
Can you tell what is the following Python 3 function doing
def attach_volume(self, volume_id, instance_id, device):
        query = self.query_factory(
            action="AttachVolume", creds=self.creds, endpoint=self.endpoint,
            other_params={"VolumeId": volume_id, "InstanceId": instance_id,
                          "Device": device})
        d = query.submit()
        return d.addCallback(self.parser.attach_volume)

Response: Attach the given volume to the specified instance at C { device }.


Instruction:
Here you have a function in Python 3, explain what it does
def describe_keypairs(self, *keypair_names):
        keypairs = {}
        for index, keypair_name in enumerate(keypair_names):
            keypairs["KeyName.%d" % (index + 1)] = keypair_name
        query = self.query_factory(
            action="DescribeKeyPairs", creds=self.creds,
            endpoint=self.endpoint, other_params=keypairs)
        d = query.submit()
        return d.addCallback(self.parser.describe_keypairs)

Response: Returns information about key pairs available.


Instruction:
Explain what the following Python 3 code does
def create_keypair(self, keypair_name):
        query = self.query_factory(
            action="CreateKeyPair", creds=self.creds, endpoint=self.endpoint,
            other_params={"KeyName": keypair_name})
        d = query.submit()
        return d.addCallback(self.parser.create_keypair)

Response: Create a new 2048 bit RSA key pair and return a unique ID that can be used to reference the created key pair.


Instruction:
Can you tell what is the following Python 3 function doing
def import_keypair(self, keypair_name, key_material):
        query = self.query_factory(
            action="ImportKeyPair", creds=self.creds, endpoint=self.endpoint,
            other_params={"KeyName": keypair_name,
                          "PublicKeyMaterial": b64encode(key_material)})
        d = query.submit()
        return d.addCallback(self.parser.import_keypair, key_material)

Response: Import an existing SSH key into EC2.


Instruction:
Make a summary of the following Python 3 code
def allocate_address(self):
        # XXX remove empty other_params
        query = self.query_factory(
            action="AllocateAddress", creds=self.creds, endpoint=self.endpoint,
            other_params={})
        d = query.submit()
        return d.addCallback(self.parser.allocate_address)

Response:
Acquire an elastic IP address to be attached subsequently to EC2
        instances.


Instruction:
Write a Python 3 function that can
release a previously allocated address returned by C { allocate_address.

Response:
def release_address(self, address):
        """
        Release a previously allocated address returned by C{allocate_address}.

        @return: C{True} if the operation succeeded.
        """
        query = self.query_factory(
            action="ReleaseAddress", creds=self.creds, endpoint=self.endpoint,
            other_params={"PublicIp": address})
        d = query.submit()
        return d.addCallback(self.parser.truth_return)


Instruction:
How would you explain what the following Python 3 function does
def describe_addresses(self, *addresses):
        address_set = {}
        for pos, address in enumerate(addresses):
            address_set["PublicIp.%d" % (pos + 1)] = address
        query = self.query_factory(
            action="DescribeAddresses", creds=self.creds,
            endpoint=self.endpoint, other_params=address_set)
        d = query.submit()
        return d.addCallback(self.parser.describe_addresses)

Response: Describe the elastic IPs allocated in this account.


Instruction:
How would you code a function in Python 3 to
parse the instancesSet XML payload and return a list of L { Instance s.

Response:
def instances_set(self, root, reservation):
        """Parse instance data out of an XML payload.

        @param root: The root node of the XML payload.
        @param reservation: The L{Reservation} associated with the instances
            from the response.
        @return: A C{list} of L{Instance}s.
        """
        instances = []
        for instance_data in root.find("instancesSet"):
            instances.append(self.instance(instance_data, reservation))
        return instances


Instruction:
Create a Python 3 function for
parsing the instance data out of an XML payload.

Response:
def instance(self, instance_data, reservation):
        """Parse instance data out of an XML payload.

        @param instance_data: An XML node containing instance data.
        @param reservation: The L{Reservation} associated with the instance.
        @return: An L{Instance}.

        TODO: reason, platform, monitoring, subnetId, vpcId, privateIpAddress,
              ipAddress, stateReason, architecture, rootDeviceName,
              blockDeviceMapping, instanceLifecycle, spotInstanceRequestId.
        """
        for group_data in instance_data.find("groupSet"):
            group_id = group_data.findtext("groupId")
            group_name = group_data.findtext("groupName")
            reservation.groups.append((group_id, group_name))
        instance_id = instance_data.findtext("instanceId")
        instance_state = instance_data.find(
            "instanceState").findtext("name")
        private_dns_name = instance_data.findtext("privateDnsName")
        dns_name = instance_data.findtext("dnsName")
        private_ip_address = instance_data.findtext("privateIpAddress")
        ip_address = instance_data.findtext("ipAddress")
        key_name = instance_data.findtext("keyName")
        ami_launch_index = instance_data.findtext("amiLaunchIndex")
        products = []
        product_codes = instance_data.find("productCodes")
        if product_codes is not None:
            for product_data in instance_data.find("productCodes"):
                products.append(product_data.text)
        instance_type = instance_data.findtext("instanceType")
        launch_time = instance_data.findtext("launchTime")
        placement = instance_data.find("placement").findtext(
            "availabilityZone")
        kernel_id = instance_data.findtext("kernelId")
        ramdisk_id = instance_data.findtext("ramdiskId")
        image_id = instance_data.findtext("imageId")
        instance = model.Instance(
            instance_id, instance_state, instance_type, image_id,
            private_dns_name, dns_name, private_ip_address, ip_address,
            key_name, ami_launch_index, launch_time, placement, products,
            kernel_id, ramdisk_id, reservation=reservation)
        return instance


Instruction:
Can you generate the documentation for the following Python 3 function
def describe_instances(self, xml_bytes):
        root = XML(xml_bytes)
        results = []
        # May be a more elegant way to do this:
        for reservation_data in root.find("reservationSet"):
            # Create a reservation object with the parsed data.
            reservation = model.Reservation(
                reservation_id=reservation_data.findtext("reservationId"),
                owner_id=reservation_data.findtext("ownerId"))
            # Get the list of instances.
            instances = self.instances_set(
                reservation_data, reservation)
            results.extend(instances)
        return results

Response: Parse the reservations XML payload that is returned from an AWS CloudFormation describeInstances API call and return a list of model. Instances.


Instruction:
Here you have a function in Python 3, explain what it does
def run_instances(self, xml_bytes):
        root = XML(xml_bytes)
        # Get the security group information.
        groups = []
        for group_data in root.find("groupSet"):
            group_id = group_data.findtext("groupId")
            groups.append(group_id)
        # Create a reservation object with the parsed data.
        reservation = model.Reservation(
            reservation_id=root.findtext("reservationId"),
            owner_id=root.findtext("ownerId"),
            groups=groups)
        # Get the list of instances.
        instances = self.instances_set(root, reservation)
        return instances

Response:
Parse the reservations XML payload that is returned from an AWS
        RunInstances API call.


Instruction:
Implement a function in Python 3 to
parse the XML returned by the C { TerminateInstances function and return an iterable of tuples of instanceId previousState currentState

Response:
def terminate_instances(self, xml_bytes):
        """Parse the XML returned by the C{TerminateInstances} function.

        @param xml_bytes: XML bytes with a C{TerminateInstancesResponse} root
            element.
        @return: An iterable of C{tuple} of (instanceId, previousState,
            currentState) for the ec2 instances that where terminated.
        """
        root = XML(xml_bytes)
        result = []
        # May be a more elegant way to do this:
        instances = root.find("instancesSet")
        if instances is not None:
            for instance in instances:
                instanceId = instance.findtext("instanceId")
                previousState = instance.find("previousState").findtext(
                    "name")
                currentState = instance.find("currentState").findtext(
                    "name")
                result.append((instanceId, previousState, currentState))
        return result


Instruction:
Write a Python 3 function that can
parse the XML returned by the C { DescribeSecurityGroups} function and return a list of L { SecurityGroup } instances.

Response:
def describe_security_groups(self, xml_bytes):
        """Parse the XML returned by the C{DescribeSecurityGroups} function.

        @param xml_bytes: XML bytes with a C{DescribeSecurityGroupsResponse}
            root element.
        @return: A list of L{SecurityGroup} instances.
        """
        root = XML(xml_bytes)
        result = []
        for group_info in root.findall("securityGroupInfo/item"):
            id = group_info.findtext("groupId")
            name = group_info.findtext("groupName")
            description = group_info.findtext("groupDescription")
            owner_id = group_info.findtext("ownerId")
            allowed_groups = []
            allowed_ips = []
            ip_permissions = group_info.find("ipPermissions")
            if ip_permissions is None:
                ip_permissions = ()
            for ip_permission in ip_permissions:

                # openstack doesn't handle self authorized groups properly
                # XXX this is an upstream problem and should be addressed there
                # lp bug #829609
                ip_protocol = ip_permission.findtext("ipProtocol")
                from_port = ip_permission.findtext("fromPort")
                to_port = ip_permission.findtext("toPort")

                if from_port:
                    from_port = int(from_port)

                if to_port:
                    to_port = int(to_port)

                for groups in ip_permission.findall("groups/item") or ():
                    user_id = groups.findtext("userId")
                    group_name = groups.findtext("groupName")
                    if user_id and group_name:
                        if (user_id, group_name) not in allowed_groups:
                            allowed_groups.append((user_id, group_name))
                for ip_ranges in ip_permission.findall("ipRanges/item") or ():
                    cidr_ip = ip_ranges.findtext("cidrIp")
                    allowed_ips.append(
                        model.IPPermission(
                            ip_protocol, from_port, to_port, cidr_ip))

            allowed_groups = [model.UserIDGroupPair(user_id, group_name)
                              for user_id, group_name in allowed_groups]

            security_group = model.SecurityGroup(
                id, name, description, owner_id=owner_id,
                groups=allowed_groups, ips=allowed_ips)
            result.append(security_group)
        return result


Instruction:
Can you write a function in Python 3 where it
parses the XML returned by the C{DescribeVolumes function and return a list of L { Volume } instances.

Response:
def describe_volumes(self, xml_bytes):
        """Parse the XML returned by the C{DescribeVolumes} function.

        @param xml_bytes: XML bytes with a C{DescribeVolumesResponse} root
            element.
        @return: A list of L{Volume} instances.

        TODO: attachementSetItemResponseType#deleteOnTermination
        """
        root = XML(xml_bytes)
        result = []
        for volume_data in root.find("volumeSet"):
            volume_id = volume_data.findtext("volumeId")
            size = int(volume_data.findtext("size"))
            snapshot_id = volume_data.findtext("snapshotId")
            availability_zone = volume_data.findtext("availabilityZone")
            status = volume_data.findtext("status")
            create_time = volume_data.findtext("createTime")
            create_time = datetime.strptime(
                create_time[:19], "%Y-%m-%dT%H:%M:%S")
            volume = model.Volume(
                volume_id, size, status, create_time, availability_zone,
                snapshot_id)
            result.append(volume)
            for attachment_data in volume_data.find("attachmentSet"):
                instance_id = attachment_data.findtext("instanceId")
                device = attachment_data.findtext("device")
                status = attachment_data.findtext("status")
                attach_time = attachment_data.findtext("attachTime")
                attach_time = datetime.strptime(
                    attach_time[:19], "%Y-%m-%dT%H:%M:%S")
                attachment = model.Attachment(
                    instance_id, device, status, attach_time)
                volume.attachments.append(attachment)
        return result


Instruction:
How would you code a function in Python 3 to
parse the XML returned by the C { CreateVolume function.

Response:
def create_volume(self, xml_bytes):
        """Parse the XML returned by the C{CreateVolume} function.

        @param xml_bytes: XML bytes with a C{CreateVolumeResponse} root
            element.
        @return: The L{Volume} instance created.
        """
        root = XML(xml_bytes)
        volume_id = root.findtext("volumeId")
        size = int(root.findtext("size"))
        snapshot_id = root.findtext("snapshotId")
        availability_zone = root.findtext("availabilityZone")
        status = root.findtext("status")
        create_time = root.findtext("createTime")
        create_time = datetime.strptime(
            create_time[:19], "%Y-%m-%dT%H:%M:%S")
        volume = model.Volume(
            volume_id, size, status, create_time, availability_zone,
            snapshot_id)
        return volume


Instruction:
Can you generate a brief explanation for the following Python 3 code
def snapshots(self, xml_bytes):
        root = XML(xml_bytes)
        result = []
        for snapshot_data in root.find("snapshotSet"):
            snapshot_id = snapshot_data.findtext("snapshotId")
            volume_id = snapshot_data.findtext("volumeId")
            status = snapshot_data.findtext("status")
            start_time = snapshot_data.findtext("startTime")
            start_time = datetime.strptime(
                start_time[:19], "%Y-%m-%dT%H:%M:%S")
            progress = snapshot_data.findtext("progress")[:-1]
            progress = float(progress or "0") / 100.
            snapshot = model.Snapshot(
                snapshot_id, volume_id, status, start_time, progress)
            result.append(snapshot)
        return result

Response: Parse the XML returned by the DescribeSnapshots function and return a list of L { Snapshot } instances.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def create_snapshot(self, xml_bytes):
        root = XML(xml_bytes)
        snapshot_id = root.findtext("snapshotId")
        volume_id = root.findtext("volumeId")
        status = root.findtext("status")
        start_time = root.findtext("startTime")
        start_time = datetime.strptime(
            start_time[:19], "%Y-%m-%dT%H:%M:%S")
        progress = root.findtext("progress")[:-1]
        progress = float(progress or "0") / 100.
        return model.Snapshot(
            snapshot_id, volume_id, status, start_time, progress)

Response: Parse the XML returned by the C { CreateSnapshot function.


Instruction:
Here you have a function in Python 3, explain what it does
def attach_volume(self, xml_bytes):
        root = XML(xml_bytes)
        status = root.findtext("status")
        attach_time = root.findtext("attachTime")
        attach_time = datetime.strptime(
            attach_time[:19], "%Y-%m-%dT%H:%M:%S")
        return {"status": status, "attach_time": attach_time}

Response: Parse the XML returned by the C { AttachVolume function.


Instruction:
Write a Python 3 script for
parsing the XML returned by the C { DescribeKeyPairs function.

Response:
def describe_keypairs(self, xml_bytes):
        """Parse the XML returned by the C{DescribeKeyPairs} function.

        @param xml_bytes: XML bytes with a C{DescribeKeyPairsResponse} root
            element.
        @return: a C{list} of L{Keypair}.
        """
        results = []
        root = XML(xml_bytes)
        keypairs = root.find("keySet")
        if keypairs is None:
            return results
        for keypair_data in keypairs:
            key_name = keypair_data.findtext("keyName")
            key_fingerprint = keypair_data.findtext("keyFingerprint")
            results.append(model.Keypair(key_name, key_fingerprint))
        return results


Instruction:
Write a Python 3 function that can
parse the XML returned by the C { CreateKeyPair function.

Response:
def create_keypair(self, xml_bytes):
        """Parse the XML returned by the C{CreateKeyPair} function.

        @param xml_bytes: XML bytes with a C{CreateKeyPairResponse} root
            element.
        @return: The L{Keypair} instance created.
        """
        keypair_data = XML(xml_bytes)
        key_name = keypair_data.findtext("keyName")
        key_fingerprint = keypair_data.findtext("keyFingerprint")
        key_material = keypair_data.findtext("keyMaterial")
        return model.Keypair(key_name, key_fingerprint, key_material)


Instruction:
Create a Python 3 function to
parse the XML returned by the C { DescribeAddresses function.

Response:
def describe_addresses(self, xml_bytes):
        """Parse the XML returned by the C{DescribeAddresses} function.

        @param xml_bytes: XML bytes with a C{DescribeAddressesResponse} root
            element.
        @return: a C{list} of L{tuple} of (publicIp, instancId).
        """
        results = []
        root = XML(xml_bytes)
        for address_data in root.find("addressesSet"):
            address = address_data.findtext("publicIp")
            instance_id = address_data.findtext("instanceId")
            results.append((address, instance_id))
        return results


Instruction:
Create a Python 3 function to
parse the XML returned by the C { DescribeAvailibilityZonesResponse function.

Response:
def describe_availability_zones(self, xml_bytes):
        """Parse the XML returned by the C{DescribeAvailibilityZones} function.

        @param xml_bytes: XML bytes with a C{DescribeAvailibilityZonesResponse}
            root element.
        @return: a C{list} of L{AvailabilityZone}.

        TODO: regionName, messageSet
        """
        results = []
        root = XML(xml_bytes)
        for zone_data in root.find("availabilityZoneInfo"):
            zone_name = zone_data.findtext("zoneName")
            zone_state = zone_data.findtext("zoneState")
            results.append(model.AvailabilityZone(zone_name, zone_state))
        return results


Instruction:
Can you implement a function in Python 3 that
signs this query using its built in credentials.

Response:
def sign(self, hash_type="sha256"):
        """Sign this query using its built in credentials.

        @param hash_type: if the SignatureVersion is 2, specify the type of
            hash to use, either "sha1" or "sha256". It defaults to the latter.

        This prepares it to be sent, and should be done as the last step before
        submitting the query. Signing is done automatically - this is a public
        method to facilitate testing.
        """
        version = self.params["SignatureVersion"]
        if version == "2":
            self.params["SignatureMethod"] = "Hmac%s" % hash_type.upper()
        self.params["Signature"] = self.signature.compute()


Instruction:
Explain what the following Python 3 code does
def submit(self):
        self.sign()
        url = self.endpoint.get_uri()
        method = self.endpoint.method
        params = self.signature.get_canonical_query_params()
        headers = {}
        kwargs = {"method": method}
        if method == "POST":
            headers["Content-Type"] = "application/x-www-form-urlencoded"
            kwargs["postdata"] = params
        else:
            url += "?%s" % params
        if self.endpoint.get_host() != self.endpoint.get_canonical_host():
            headers["Host"] = self.endpoint.get_canonical_host()
        if headers:
            kwargs["headers"] = headers
        if self.timeout:
            kwargs["timeout"] = self.timeout
        d = self.get_page(url, **kwargs)
        return d.addErrback(ec2_error_wrapper)

Response:
Submit this query.

        @return: A deferred from get_page


Instruction:
How would you explain what the following Python 3 function does
def compute(self):
        if "Signature" in self.params:
            raise RuntimeError("Existing signature in parameters")
        if self.signature_version is not None:
            version = self.signature_version
        else:
            version = self.params["SignatureVersion"]
        if str(version) == "1":
            bytes = self.old_signing_text()
            hash_type = "sha1"
        elif str(version) == "2":
            bytes = self.signing_text()
            if self.signature_method is not None:
                signature_method = self.signature_method
            else:
                signature_method = self.params["SignatureMethod"]
            hash_type = signature_method[len("Hmac"):].lower()
        else:
            raise RuntimeError("Unsupported SignatureVersion: '%s'" % version)
        return self.creds.sign(bytes, hash_type)

Response: Compute and return the signature according to the given data.


Instruction:
Make a summary of the following Python 3 code
def old_signing_text(self):
        result = []
        lower_cmp = lambda x, y: cmp(x[0].lower(), y[0].lower())
        for key, value in sorted(self.params.items(), cmp=lower_cmp):
            result.append("%s%s" % (key, value))
        return "".join(result)

Response: Return the text needed for signing using SignatureVersion 1.


Instruction:
Implement a Python 3 function for
returning the text to be signed when signing the query.

Response:
def signing_text(self):
        """Return the text to be signed when signing the query."""
        result = "%s\n%s\n%s\n%s" % (self.endpoint.method,
                                     self.endpoint.get_canonical_host(),
                                     self.endpoint.path,
                                     self.get_canonical_query_params())
        return result


Instruction:
Implement a Python 3 function for
returning the canonical query params used in signing.

Response:
def get_canonical_query_params(self):
        """Return the canonical query params (used in signing)."""
        result = []
        for key, value in self.sorted_params():
            result.append("%s=%s" % (self.encode(key), self.encode(value)))
        return "&".join(result)


Instruction:
Can you implement a function in Python 3 that
encodes a_string as per the canonicalisation encoding rules.

Response:
def encode(self, string):
        """Encode a_string as per the canonicalisation encoding rules.

        See the AWS dev reference page 186 (2009-11-30 version).
        @return: a_string encoded.
        """
        if isinstance(string, unicode):
            string = string.encode("utf-8")
        return quote(string, safe="~")


Instruction:
Create a Python 3 function to
create an instance of this from XML bytes.

Response:
def from_xml(cls, xml_bytes):
        """
        Create an instance of this from XML bytes.

        @param xml_bytes: C{str} bytes of XML to parse
        @return: an instance of L{MultipartInitiationResponse}
        """
        root = XML(xml_bytes)
        return cls(root.findtext('Bucket'),
                   root.findtext('Key'),
                   root.findtext('UploadId'))


Instruction:
Can you generate the documentation for the following Python 3 function
def pythag(a, b):
    absA = abs(a)
    absB = abs(b)
    if absA > absB:
        return absA * sqrt(1.0 + (absB / float(absA)) ** 2)
    elif absB == 0.0:
        return 0.0
    else:
        return absB * sqrt(1.0 + (absA / float(absB)) ** 2)

Response: Compute the Pythagorean theorem


Instruction:
Given the following Python 3 function, write the documentation
def _initialize_with_array(self, data, rowBased=True):
        if rowBased:
            self.matrix = []
            if len(data) != self._rows:
                raise ValueError("Size of Matrix does not match")
            for col in xrange(self._columns):
                self.matrix.append([])
                for row in xrange(self._rows):
                    if len(data[row]) != self._columns:
                        raise ValueError("Size of Matrix does not match")
                    self.matrix[col].append(data[row][col])
        else:
            if len(data) != self._columns:
                raise ValueError("Size of Matrix does not match")
            for col in data:
                if len(col) != self._rows:
                    raise ValueError("Size of Matrix does not match")
            self.matrix = copy.deepcopy(data)

Response: Set the matrix values from a two dimensional list.


Instruction:
Create a Python 3 function for
creating a new Matrix instance from a TimeSeries or MultiDimensionalTimeSeries.

Response:
def from_timeseries(cls, timeSeries):
        """Create a new Matrix instance from a TimeSeries or MultiDimensionalTimeSeries

        :param TimeSeries timeSeries: The TimeSeries, which should be used to
                create a new Matrix.
        :return:    A Matrix with the values of the timeSeries. Each row of
                    the Matrix represents one entry of the timeSeries.
                    The time of an entry is ignored in the matrix.
        :rtype:     Matrix

        :raise:     Raises an :py:exc:`ValueError`, if the timeSeries is empty.
        """
        width = 1

        if isinstance(timeSeries, MultiDimensionalTimeSeries):
            width = timeSeries.dimension_count()

        matrixData = [[] for dummy in xrange(width)]

        for entry in timeSeries:
            for col in xrange(1, len(entry)):
                matrixData[col - 1].append(entry[col])
        if not matrixData[0]:
            raise ValueError("Cannot create Matrix from empty Timeseries")
        mtrx = Matrix.from_two_dim_array(len(matrixData), len(matrixData[0]), matrixData)

        # mtrx.initialize(matrixData, rowBased=False)
        return mtrx


Instruction:
Can you generate a brief explanation for the following Python 3 code
def from_two_dim_array(cls, cols, rows, twoDimArray):
        return Matrix(cols, rows, twoDimArray, rowBased=False, isOneDimArray=False)

Response: Create a new Matrix instance from a two dimensional array.


Instruction:
Explain what the following Python 3 code does
def to_multi_dim_timeseries(self):
        ts = MultiDimensionalTimeSeries(dimensions=self.get_width())
        for row in xrange(self.get_height()):
            newEntry = []
            for col in xrange(self.get_width()):
                newEntry.append(self.get_value(col, row))
            ts.add_entry(row, newEntry)
        return ts

Response: Return a MultiDimensionalTimeSeries with the values of self


Instruction:
Explain what the following Python 3 code does
def get_array(self, rowBased=True):
        if rowBased:
            array = []
            for row in xrange(self._rows):
                newRow = []
                for col in xrange(self._columns):
                    newRow.append(self.get_value(col, row))
                array.append(newRow)
            return array
        return copy.deepcopy(self.matrix)

Response: Return a two dimensional list with the values of the i'th column based.


Instruction:
Can you generate the documentation for the following Python 3 function
def get_matrix_from_list(self, rows, columns, matrix_list, rowBased=True):
        resultMatrix = Matrix(columns, rows, matrix_list, rowBased)
        return resultMatrix

Response: Create a Matrix instance from a list of values.


Instruction:
Given the following Python 3 function, write the documentation
def set_value(self, column, row, value):
        self.matrix[column][row] = value

Response: Set the value of the matrix at the specified column and row.


Instruction:
Given the following Python 3 function, write the documentation
def invers(self):

        if self._columns != self._rows:
            raise ValueError("A square matrix is needed")
        mArray = self.get_array(False)
        appList = [0] * self._columns

        # add identity matrix to array in order to use gauss jordan algorithm
        for col in xrange(self._columns):
            mArray.append(appList[:])
            mArray[self._columns + col][col] = 1
        # create new Matrix and execute gass jordan algorithm
        exMatrix = Matrix.from_two_dim_array(2 * self._columns, self._rows, mArray)
        gjResult = exMatrix.gauss_jordan()
        # remove identity matrix from left side
        # TODO Implement slicing directly for Matrix
        gjResult.matrix = gjResult.matrix[self._columns:]
        gjResult._columns = len(gjResult.matrix)
        return gjResult

Response: Return the invers matrix if it can be calculated.


Instruction:
Implement a Python 3 function for
multiplying the matrix with the given matrix and return the result.

Response:
def matrix_multiplication(self, matrix):
        """Multiply :py:obj:`self` with the given matrix and return result matrix.

        param Matrix matrix: The matrix, which should be multiplied.

        :return:    Returns a new Matrix with the result of the multiplication
        :rtype:     Matrix

        :note: Make sure, that the matrices can be multiplied.
                The number of columns of the Matrix instance must match with
                the number of rows of the Matrix given as parameter.
                Use is_matrix_mult_possible(matrix) to test.
        """
        resultMatrix = Matrix(matrix.get_width(), self.get_height())
        for r_row in xrange(self._rows):
            for r_col in xrange(matrix.get_width()):
                #blockwise matrix multiplication hack
                if isinstance(self.get_array()[0][0], Matrix):
                    blocksize = self.get_array()[0][0].get_width()
                    valueT = Matrix(blocksize, blocksize)
                else:
                    valueT = 0
                for column in xrange(matrix.get_height()):
                    valueT += self.get_value(column, r_row) * matrix.get_value(r_col, column)
                resultMatrix.set_value(r_col, r_row, valueT)
        return resultMatrix


Instruction:
Write a Python 3 function that can
return a new Matrix with the columns of the current Matrix.

Response:
def flatten(self):
        """
        If the current Matrix consists of Blockmatrixes as elementes method
        flattens the Matrix into one Matrix only consisting of the 2nd level
        elements

        [[[1 2] [[3 4]   to [[1 2 3 4]
          [5 6]] [7 8]]]     [5 6 7 8]]
        """
        blocksize = self.get_array()[0][0].get_width()
        width = self.get_width() * blocksize

        columnsNew = [[] for dummy in xrange(width)]

        for row in self.get_array():
            index = 0
            for submatrix in row:
                for column in submatrix.get_array(False):
                    columnsNew[index] += column
                    index += 1

        columnsFlat = sum(columnsNew, [])
        return Matrix(width, len(columnsNew[0]), columnsFlat, rowBased=False)


Instruction:
How would you code a function in Python 3 to
convert a matrix into a n * m matrix.

Response:
def matrix_to_blockmatrix(self, blocksize):
        """
        turns an n*m Matrix into a (n/blocksize)*(m/blocksize matrix).
        Each element is another blocksize*blocksize matrix.
        """
        if self.get_width() % blocksize or self.get_height() % blocksize:
            raise ValueError("Number of rows and columns have to be evenly dividable by blocksize")
        selfBlocks = []
        for columnIndex in range(0, self.get_width() - 1, blocksize):
            for rowIndex in range(0, self.get_height() - 1, blocksize):
                currentBlock = []
                for blockRows in self.get_array(False)[columnIndex:columnIndex + blocksize]:
                    currentBlock += blockRows[rowIndex:rowIndex + blocksize]
                selfBlocks.append(Matrix(blocksize, blocksize, currentBlock, rowBased=False))
        return Matrix(self.get_width() / blocksize, self.get_height() / blocksize, selfBlocks, rowBased=False)


Instruction:
Explain what the following Python 3 code does
def multiply(self, multiplicator):
        result = Matrix(self.get_width(), self.get_height())
        for row in xrange(self.get_height()):
            for col in xrange(self.get_width()):
                result.set_value(col, row, self.get_value(col, row) * multiplicator)
        return result

Response: Return a new Matrix with a multiple.


Instruction:
How would you explain what the following Python 3 function does
def transform(self):
        t_matrix = Matrix(self._rows, self._columns)
        for col_i, col in enumerate(self.matrix):
            for row_i, entry in enumerate(col):
                t_matrix.set_value(row_i, col_i, entry)
        return t_matrix

Response: Return a new transformed matrix.


Instruction:
Explain what the following Python 3 code does
def gauss_jordan(self):

        mArray = self.get_array(rowBased=False)
        width = self.get_width()
        height = self.get_height()
        if not height < width:
            raise ValueError("""Not enough rows""")
        # Start with complete matrix and remove in each iteration
        # the first row and the first column
        for offset in xrange(height):
            # Switch lines, if current first value is 0
            if mArray[offset][offset] == 0:
                for i in xrange(offset + 1, height):
                    if mArray[offset][i] != 0:
                        tmp = []
                        for j in xrange(offset, width):
                            tmp.append(mArray[j][offset])
                        # tmp = mArray[offset][offset:]
                        for j in xrange(offset, width):
                            mArray[j][offset] = mArray[j][i]
                            mArray[j][i] = tmp[j]
                        # mArray[offset][offset:] = mArray[i][offset:]
                        # mArray[i] = tmp
                        break

            currentRow = [mArray[j][offset] for j in xrange(offset, width)]
            devider = float(currentRow[0])
            # If no line is found with an value != 0
            # the matrix is not invertible
            if devider == 0:
                raise ValueError("Matrix is not invertible")
            transformedRow = []
            # Devide current row by first element of current row
            for value in currentRow:
                transformedRow.append(value / devider)
            # put transformed row back into matrix
            for j in xrange(offset, width):
                mArray[j][offset] = transformedRow[j - offset]
            # subtract multiples of the current row, from all remaining rows
            # in order to become a 0 at the current first column
            for i in xrange(offset + 1, height):
                multi = mArray[offset][i]
                for j in xrange(offset, width):
                    mArray[j][i] = mArray[j][i] - mArray[j][offset] * multi
        for i in xrange(1, height):
            # subtract multiples of the i-the row from all above rows
            for j in xrange(0, i):
                multi = mArray[i][j]
                for col in xrange(i, width):
                    mArray[col][j] = mArray[col][j] - mArray[col][i] * multi
        self.matrix = mArray
        return self

Response: Reduce the matrix to row echelon form.