Instruction:
Given the following Python 3 function, write the documentation
def scan(self,
             table_name,
             index_name=None,
             consistent_read=None,
             projection_expression=None,
             filter_expression=None,
             expression_attribute_names=None,
             expression_attribute_values=None,
             segment=None,
             total_segments=None,
             select=None,
             limit=None,
             exclusive_start_key=None,
             return_consumed_capacity=None):
        """The `Scan`_ operation returns one or more items and item attributes
        by accessing every item in a table or a secondary index.

        If the total number of scanned items exceeds the maximum data set size
        limit of 1 MB, the scan stops and results are returned to the user as a
        ``LastEvaluatedKey`` value to continue the scan in a subsequent
        operation. The results also include the number of items exceeding the
        limit. A scan can result in no table data meeting the filter criteria.

        By default, Scan operations proceed sequentially; however, for faster
        performance on a large table or secondary index, applications can
        request a parallel *Scan* operation by providing the ``segment`` and
        ``total_segments`` parameters. For more information, see
        `Parallel Scan <http://docs.aws.amazon.com/amazondynamodb/latest/
        developerguide/QueryAndScan.html#QueryAndScanParallelScan>`_ in the
        Amazon DynamoDB Developer Guide.

        By default, *Scan* uses eventually consistent reads when accessing the
        data in a table; therefore, the result set might not include the
        changes to data in the table immediately before the operation began. If
        you need a consistent copy of the data, as of the time that the *Scan*
        begins, you can set the ``consistent_read`` parameter to ``True``.

        :rtype: dict

        .. _Scan: http://docs.aws.amazon.com/amazondynamodb/
           latest/APIReference/API_Scan.html

        """
        payload = {'TableName': table_name}
        if index_name:
            payload['IndexName'] = index_name
        if consistent_read is not None:
            payload['ConsistentRead'] = consistent_read
        if filter_expression:
            payload['FilterExpression'] = filter_expression
        if expression_attribute_names:
            payload['ExpressionAttributeNames'] = expression_attribute_names
        if expression_attribute_values:
            payload['ExpressionAttributeValues'] = \
                utils.marshall(expression_attribute_values)
        if projection_expression:
            payload['ProjectionExpression'] = projection_expression
        if segment:
            payload['Segment'] = segment
        if total_segments:
            payload['TotalSegments'] = total_segments
        if select:
            _validate_select(select)
            payload['Select'] = select
        if exclusive_start_key:
            payload['ExclusiveStartKey'] = utils.marshall(exclusive_start_key)
        if limit:
            payload['Limit'] = limit
        if return_consumed_capacity:
            _validate_return_consumed_capacity(return_consumed_capacity)
            payload['ReturnConsumedCapacity'] = return_consumed_capacity
        return self.execute('Scan', payload)

Response: This operation scans the table and returns one or more items and item attributes in a secondary index.


Instruction:
Create a Python 3 function to
execute a DynamoDB action with the given parameters.

Response:
def execute(self, action, parameters):
        """
        Execute a DynamoDB action with the given parameters. The method will
        retry requests that failed due to OS level errors or when being
        throttled by DynamoDB.

        :param str action: DynamoDB action to invoke
        :param dict parameters: parameters to send into the action
        :rtype: tornado.concurrent.Future

        This method creates a future that will resolve to the result
        of calling the specified DynamoDB function.  It does it's best
        to unwrap the response from the function to make life a little
        easier for you.  It does this for the ``GetItem`` and ``Query``
        functions currently.

        :raises:
            :exc:`~sprockets_dynamodb.exceptions.DynamoDBException`
            :exc:`~sprockets_dynamodb.exceptions.ConfigNotFound`
            :exc:`~sprockets_dynamodb.exceptions.NoCredentialsError`
            :exc:`~sprockets_dynamodb.exceptions.NoProfileError`
            :exc:`~sprockets_dynamodb.exceptions.TimeoutException`
            :exc:`~sprockets_dynamodb.exceptions.RequestException`
            :exc:`~sprockets_dynamodb.exceptions.InternalFailure`
            :exc:`~sprockets_dynamodb.exceptions.LimitExceeded`
            :exc:`~sprockets_dynamodb.exceptions.MissingParameter`
            :exc:`~sprockets_dynamodb.exceptions.OptInRequired`
            :exc:`~sprockets_dynamodb.exceptions.ResourceInUse`
            :exc:`~sprockets_dynamodb.exceptions.RequestExpired`
            :exc:`~sprockets_dynamodb.exceptions.ResourceNotFound`
            :exc:`~sprockets_dynamodb.exceptions.ServiceUnavailable`
            :exc:`~sprockets_dynamodb.exceptions.ThroughputExceeded`
            :exc:`~sprockets_dynamodb.exceptions.ValidationException`

        """
        measurements = collections.deque([], self._max_retries)
        for attempt in range(1, self._max_retries + 1):
            try:
                result = yield self._execute(
                    action, parameters, attempt, measurements)
            except (exceptions.InternalServerError,
                    exceptions.RequestException,
                    exceptions.ThrottlingException,
                    exceptions.ThroughputExceeded,
                    exceptions.ServiceUnavailable) as error:
                if attempt == self._max_retries:
                    if self._instrumentation_callback:
                        self._instrumentation_callback(measurements)
                    self._on_exception(error)
                duration = self._sleep_duration(attempt)
                self.logger.warning('%r on attempt %i, sleeping %.2f seconds',
                                    error, attempt, duration)
                yield gen.sleep(duration)
            except exceptions.DynamoDBException as error:
                if self._instrumentation_callback:
                    self._instrumentation_callback(measurements)
                self._on_exception(error)
            else:
                if self._instrumentation_callback:
                    self._instrumentation_callback(measurements)
                self.logger.debug('%s result: %r', action, result)
                raise gen.Return(_unwrap_result(action, result))


Instruction:
Here you have a function in Python 3, explain what it does
def set_error_callback(self, callback):
        self.logger.debug('Setting error callback: %r', callback)
        self._on_error = callback

Response: Assign a method to invoke when a request has encountered an ancillary unrecoverable error in an action execution.


Instruction:
How would you implement a function in Python 3 that
assigns a method to invoke when a request has completed gathering measurements.

Response:
def set_instrumentation_callback(self, callback):
        """Assign a method to invoke when a request has completed gathering
        measurements.

        :param method callback: The method to invoke

        """
        self.logger.debug('Setting instrumentation callback: %r', callback)
        self._instrumentation_callback = callback


Instruction:
Can you generate the documentation for the following Python 3 function
def _execute(self, action, parameters, attempt, measurements):
        future = concurrent.Future()
        start = time.time()

        def handle_response(request):
            """Invoked by the IOLoop when fetch has a response to process.

            :param tornado.concurrent.Future request: The request future

            """
            self._on_response(
                action, parameters.get('TableName', 'Unknown'), attempt,
                start, request, future, measurements)

        ioloop.IOLoop.current().add_future(self._client.fetch(
            'POST', '/',
            body=json.dumps(parameters).encode('utf-8'),
            headers={
                'x-amz-target': 'DynamoDB_20120810.{}'.format(action),
                'Content-Type': 'application/x-amz-json-1.0',
            }), handle_response)
        return future

Response: Invoke a DynamoDB action and return a concurrent. Future that will be completed when the action completes.


Instruction:
Can you create a Python 3 function that
invokes when the HTTP request to the DynamoDB has returned and sets the future result or exception based upon the response.

Response:
def _on_response(self, action, table, attempt, start, response, future,
                     measurements):
        """Invoked when the HTTP request to the DynamoDB has returned and
        is responsible for setting the future result or exception based upon
        the HTTP response provided.

        :param str action: The action that was taken
        :param str table: The table name the action was made against
        :param int attempt: The attempt number for the action
        :param float start: When the request was submitted
        :param tornado.concurrent.Future response: The HTTP request future
        :param tornado.concurrent.Future future: The action execution future
        :param list measurements: The measurement accumulator

        """
        self.logger.debug('%s on %s request #%i = %r',
                          action, table, attempt, response)
        now, exception = time.time(), None
        try:
            future.set_result(self._process_response(response))
        except aws_exceptions.ConfigNotFound as error:
            exception = exceptions.ConfigNotFound(str(error))
        except aws_exceptions.ConfigParserError as error:
            exception = exceptions.ConfigParserError(str(error))
        except aws_exceptions.NoCredentialsError as error:
            exception = exceptions.NoCredentialsError(str(error))
        except aws_exceptions.NoProfileError as error:
            exception = exceptions.NoProfileError(str(error))
        except aws_exceptions.AWSError as error:
            exception = exceptions.DynamoDBException(error)
        except (ConnectionError, ConnectionResetError, OSError,
                aws_exceptions.RequestException, ssl.SSLError,
                _select.error, ssl.socket_error, socket.gaierror) as error:
            exception = exceptions.RequestException(str(error))
        except TimeoutError:
            exception = exceptions.TimeoutException()
        except httpclient.HTTPError as error:
            if error.code == 599:
                exception = exceptions.TimeoutException()
            else:
                exception = exceptions.RequestException(
                    getattr(getattr(error, 'response', error),
                            'body', str(error.code)))
        except Exception as error:
            exception = error

        if exception:
            future.set_exception(exception)

        measurements.append(
            Measurement(now, action, table, attempt, max(now, start) - start,
                        exception.__class__.__name__
                        if exception else exception))


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _process_response(response):
        error = response.exception()
        if error:
            if isinstance(error, aws_exceptions.AWSError):
                if error.args[1]['type'] in exceptions.MAP:
                    raise exceptions.MAP[error.args[1]['type']](
                        error.args[1]['message'])
            raise error
        http_response = response.result()
        if not http_response or not http_response.body:
            raise exceptions.DynamoDBException('empty response')
        return json.loads(http_response.body.decode('utf-8'))

Response:
Process the response from DynamoDB returning either the mapped exception
COOKIE or deserialized response.


Instruction:
Make a summary of the following Python 3 code
def write(self, obj, resource_id=None):
        if resource_id is not None:
            if self.read(resource_id):
                raise ValueError("There are one object already with this id.")
            obj['_id'] = resource_id
        prepared_creation_tx = self.driver.instance.transactions.prepare(
            operation='CREATE',
            signers=self.user.public_key,
            asset={
                'namespace': self.namespace,
                'data': obj
            },
            metadata={
                'namespace': self.namespace,
                'data': obj
            }
        )

        signed_tx = self.driver.instance.transactions.fulfill(
            prepared_creation_tx,
            private_keys=self.user.private_key
        )
        self.logger.debug('bdb::write::{}'.format(signed_tx['id']))
        self.driver.instance.transactions.send_commit(signed_tx)
        return signed_tx

Response: Write and return the id of the transaction that created the object.


Instruction:
Make a summary of the following Python 3 code
def _get(self, tx_id):
        # tx_id=self._find_tx_id(resource_id)
        value = [
            {
                'data': transaction['metadata'],
                'id': transaction['id']
            }
            for transaction in self.driver.instance.transactions.get(asset_id=self.get_asset_id(tx_id))
        ][-1]
        if value['data']['data']:
            self.logger.debug('bdb::read::{}'.format(value['data']))
            return value
        else:
            return False

Response: Read and obj in bdb using the tx_id.


Instruction:
Here you have a function in Python 3, explain what it does
def _update(self, metadata, tx_id, resource_id):
        try:
            if not tx_id:
                sent_tx = self.write(metadata, resource_id)
                self.logger.debug('bdb::put::{}'.format(sent_tx['id']))
                return sent_tx
            else:
                txs = self.driver.instance.transactions.get(asset_id=self.get_asset_id(tx_id))
                unspent = txs[-1]
                sent_tx = self._put(metadata, unspent, resource_id)
                self.logger.debug('bdb::put::{}'.format(sent_tx))
                return sent_tx

        except BadRequest as e:
            logging.error(e)

Response: Update and obj in bdb using the tx_id.


Instruction:
Implement a Python 3 function for
listing all the objects saved in the namespace.

Response:
def list(self, search_from=None, search_to=None, limit=None):
        """List all the objects saved in the namespace.

         :param search_from: TBI
         :param search_to: TBI
         :param offset: TBI
         :param limit: max number of values to be shows.
         :return: list with transactions.
         """
        l = []
        for i in self._list():
            l.append(i['data']['data'])
        return l[0:limit]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _list(self):
        all = self.driver.instance.metadata.get(search=self.namespace)
        list = []
        for id in all:
            try:
                if not self._get(id['id']) in list:
                    list.append(self._get(id['id']))
            except Exception:
                pass

        return list

Response: List all the objects saved in the namespace.


Instruction:
Can you generate the documentation for the following Python 3 function
def query(self, search_model: QueryModel):
        self.logger.debug('bdb::get::{}'.format(search_model.query))
        assets = json.loads(requests.post("http://localhost:4000/query", data=search_model.query).content)['data']
        self.logger.debug('bdb::result::len {}'.format(len(assets)))
        assets_metadata = []
        for i in assets:
            try:
                assets_metadata.append(self._get(i['id'])['data']['data'])
            except:
                pass
        return assets_metadata

Response: Query to bdb namespace.


Instruction:
Here you have a function in Python 3, explain what it does
def _delete(self, tx_id):
        txs = self.driver.instance.transactions.get(asset_id=self.get_asset_id(tx_id))
        unspent = txs[-1]
        output_index = 0
        output = unspent['outputs'][output_index]

        transfer_input = {
            'fulfillment': output['condition']['details'],
            'fulfills': {
                'output_index': output_index,
                'transaction_id': unspent['id']
            },
            'owners_before': output['public_keys']
        }

        prepared_transfer_tx = self.driver.instance.transactions.prepare(
            operation='TRANSFER',
            asset=unspent['asset'] if 'id' in unspent['asset'] else {'id': unspent['id']},
            inputs=transfer_input,
            recipients=self.BURN_ADDRESS,
            metadata={
                'namespace': 'burned',

            }
        )
        signed_tx = self.driver.instance.transactions.fulfill(
            prepared_transfer_tx,
            private_keys=self.user.private_key,
        )
        self.driver.instance.transactions.send_commit(signed_tx)

Response:
Delete a transaction. Read documentation about CRAB model in https://blog.bigchaindb.com/crab-create-retrieve-append-burn-b9f6d111f460.

        :param tx_id: transaction id
        :return:


Instruction:
Can you generate the documentation for the following Python 3 function
def get_asset_id(self, tx_id):
        tx = self.driver.instance.transactions.retrieve(txid=tx_id)
        assert tx is not None
        return tx['id'] if tx['operation'] == 'CREATE' else tx['asset']['id']

Response: Return the asset id of the first transaction.


Instruction:
How would you explain what the following Python 3 function does
def hdr(data, filename):
    hdrobj = data if isinstance(data, HDRobject) else HDRobject(data)
    hdrobj.write(filename)

Response: Writes ENVI header files


Instruction:
Make a summary of the following Python 3 code
def __hdr2dict(self):
        with open(self.filename, 'r') as infile:
            lines = infile.readlines()
        i = 0
        out = dict()
        while i < len(lines):
            line = lines[i].strip('\r\n')
            if '=' in line:
                if '{' in line and '}' not in line:
                    while '}' not in line:
                        i += 1
                        line += lines[i].strip('\n').lstrip()
                line = list(filter(None, re.split(r'\s+=\s+', line)))
                line[1] = re.split(',[ ]*', line[1].strip('{}'))
                key = line[0].replace(' ', '_')
                val = line[1] if len(line[1]) > 1 else line[1][0]
                out[key] = parse_literal(val)
            i += 1
        if 'band_names' in out.keys() and not isinstance(out['band_names'], list):
            out['band_names'] = [out['band_names']]
        return out

Response: read a HDR file into a dictionary


Instruction:
Create a Python 3 function for
writing object to an ENVI header file

Response:
def write(self, filename='same'):
        """
        write object to an ENVI header file
        """
        if filename == 'same':
            filename = self.filename
        if not filename.endswith('.hdr'):
            filename += '.hdr'
        with open(filename, 'w') as out:
            out.write(self.__str__())


Instruction:
Can you create a Python 3 function that
goes to Zero position

Response:
def GoZero(self, speed):
        ' Go to Zero position '
        self.ReleaseSW()

        spi.SPI_write_byte(self.CS, 0x82 | (self.Dir & 1))       # Go to Zero
        spi.SPI_write_byte(self.CS, 0x00)
        spi.SPI_write_byte(self.CS, speed)
        while self.IsBusy():
            pass
        time.sleep(0.3)
        self.ReleaseSW()


Instruction:
Given the following Python 3 function, write the documentation
def Move(self, units):
        ' Move some distance units from current position '
        steps = units * self.SPU  # translate units to steps
        if steps > 0:                                          # look for direction
            spi.SPI_write_byte(self.CS, 0x40 | (~self.Dir & 1))
        else:
            spi.SPI_write_byte(self.CS, 0x40 | (self.Dir & 1))
        steps = int(abs(steps))
        spi.SPI_write_byte(self.CS, (steps >> 16) & 0xFF)
        spi.SPI_write_byte(self.CS, (steps >> 8) & 0xFF)
        spi.SPI_write_byte(self.CS, steps & 0xFF)

Response: Move some distance units from current position


Instruction:
Can you generate the documentation for the following Python 3 function
def ReadStatusBit(self, bit):
        ' Report given status bit '
        spi.SPI_write_byte(self.CS, 0x39)   # Read from address 0x19 (STATUS)
        spi.SPI_write_byte(self.CS, 0x00)
        data0 = spi.SPI_read_byte()           # 1st byte
        spi.SPI_write_byte(self.CS, 0x00)
        data1 = spi.SPI_read_byte()           # 2nd byte
        #print hex(data0), hex(data1)
        if bit > 7:                                   # extract requested bit
            OutputBit = (data0 >> (bit - 8)) & 1
        else:
            OutputBit = (data1 >> bit) & 1
        return OutputBit

Response: Report given status bit


Instruction:
Explain what the following Python 3 code does
def SPI_write_byte(self, chip_select, data):
        'Writes a data to a SPI device selected by chipselect bit. '
        self.bus.write_byte_data(self.address, chip_select, data)

Response: Writes a data to a SPI device selected by chipselect bit.


Instruction:
Make a summary of the following Python 3 code
def SPI_write(self, chip_select, data):
        'Writes data to SPI device selected by chipselect bit. '
        dat = list(data)
        dat.insert(0, chip_select)
        return self.bus.write_i2c_block(self.address, dat);

Response: Writes data to SPI device selected by chipselect bit.


Instruction:
Can you tell what is the following Python 3 function doing
def SPI_config(self,config):
        'Configure SPI interface parameters.'
        self.bus.write_byte_data(self.address, 0xF0, config)
        return self.bus.read_byte_data(self.address, 0xF0)

Response: Configure SPI interface parameters.


Instruction:
Write a Python 3 function that can
read logic state on GPIO enabled slave - selects pins.

Response:
def GPIO_read(self):
        'Reads logic state on GPIO enabled slave-selects pins.'
        self.bus.write_byte_data(self.address, 0xF5, 0x0f)
        status = self.bus.read_byte(self.address)
        bits_values = dict([('SS0',status & 0x01 == 0x01),('SS1',status & 0x02 == 0x02),('SS2',status & 0x04 == 0x04),('SS3',status & 0x08 == 0x08)])
        return bits_values


Instruction:
Here you have a function in Python 3, explain what it does
def GPIO_config(self, gpio_enable, gpio_config):
        'Enable or disable slave-select pins as gpio.'
        self.bus.write_byte_data(self.address, 0xF6, gpio_enable)
        self.bus.write_byte_data(self.address, 0xF7, gpio_config)
        return

Response: Enable or disable slave - select pins as gpio.


Instruction:
Given the following Python 3 function, write the documentation
def get_address(self):
        LOGGER.debug("Reading RPS01A sensor's address.",)
        return self.bus.read_byte_data(self.address, self.address_reg)

Response: Returns sensors I2C address.


Instruction:
Here you have a function in Python 3, explain what it does
def get_zero_position(self):

        LSB = self.bus.read_byte_data(self.address, self.zero_position_MSB)
        MSB = self.bus.read_byte_data(self.address, self.zero_position_LSB)
        DATA = (MSB << 6) + LSB
        return DATA

Response: Returns the programmed zero position in OTP memory.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_agc_value(self):
        LOGGER.debug("Reading RPS01A sensor's AGC settings",)
        return self.bus.read_byte_data(self.address, self.AGC_reg)

Response: Returns sensor s Automatic Gain Control actual value.


Instruction:
Can you implement a function in Python 3 that
reads the diagnostic data from the sensor and returns a dictionary of the values.

Response:
def get_diagnostics(self):
        """
        Reads diagnostic data from the sensor.
        OCF (Offset Compensation Finished) - logic high indicates the finished Offset Compensation Algorithm. After power up the flag remains always to logic high.

        COF (Cordic Overflow) - logic high indicates an out of range error in the CORDIC part. When this bit is set, the angle and magnitude data is invalid.
            The absolute output maintains the last valid angular value.

        COMP low, indicates a high magnetic field. It is recommended to monitor in addition the magnitude value.
        COMP high, indicated a weak magnetic field. It is recommended to monitor the magnitude value.
        """

        status = self.bus.read_byte_data(self.address, self.diagnostics_reg)
        bits_values = dict([('OCF',status & 0x01 == 0x01),
                            ('COF',status & 0x02 == 0x02),
                            ('Comp_Low',status & 0x04 == 0x04),
                            ('Comp_High',status & 0x08 == 0x08)])
        return bits_values


Instruction:
Can you write a function in Python 3 where it
returns measured angle in degrees in range 0 - 360.

Response:
def get_angle(self, verify = False):
        """
        Retuns measured angle in degrees in range 0-360.
        """
        LSB = self.bus.read_byte_data(self.address, self.angle_LSB)
        MSB = self.bus.read_byte_data(self.address, self.angle_MSB)
        DATA = (MSB << 6) + LSB
        if not verify:
            return  (360.0 / 2**14) * DATA
        else:
            status = self.get_diagnostics()
            if not (status['Comp_Low']) and  not(status['Comp_High']) and not(status['COF']):
                return  (360.0 / 2**14) * DATA
            else:
                return  None


Instruction:
Given the following Python 3 function, write the documentation
def main(input_bed, output_file, output_features=False, genome=None,
         only_canonical=False, short=False, extended=False, high_confidence=False,
         ambiguities_method=False, coding_only=False, collapse_exons=False, work_dir=False, is_debug=False):
    """ Annotating BED file based on reference features annotations.
    """
    logger.init(is_debug_=is_debug)

    if not genome:
        raise click.BadParameter('Error: please, specify genome build name with -g (e.g. `-g hg19`)', param='genome')

    if short:
        if extended:        raise click.BadParameter('--short and --extended can\'t be set both', param='extended')
        if output_features: raise click.BadParameter('--short and --output-features can\'t be set both', param='output_features')
    elif output_features or extended:
        extended = True
        short    = False

    if not verify_file(input_bed):
        click.BadParameter(f'Usage: {__file__} Input_BED_file -g hg19 -o Annotated_BED_file [options]', param='input_bed')
    input_bed = verify_file(input_bed, is_critical=True, description=f'Input BED file for {__file__}')

    if work_dir:
        work_dir = join(adjust_path(work_dir), os.path.splitext(basename(input_bed))[0])
        safe_mkdir(work_dir)
        info(f'Created work directory {work_dir}')
    else:
        work_dir = mkdtemp('bed_annotate')
        debug('Created temporary work directory {work_dir}')

    input_bed = clean_bed(input_bed, work_dir)
    input_bed = verify_bed(input_bed, is_critical=True, description=f'Input BED file for {__file__} after cleaning')

    output_file = adjust_path(output_file)

    output_file = annotate(
        input_bed, output_file, work_dir, genome=genome,
        only_canonical=only_canonical, short=short, extended=extended,
        high_confidence=high_confidence, collapse_exons=collapse_exons,
        output_features=output_features,
        ambiguities_method=ambiguities_method, coding_only=coding_only,
        is_debug=is_debug)

    if not work_dir:
        debug(f'Removing work directory {work_dir}')
        shutil.rmtree(work_dir)

    info(f'Done, saved to {output_file}')

Response: This function is used to annotate a single BED file based on reference features annotations.


Instruction:
Write a Python 3 function for
returning the state of charge in MHz.

Response:
def StateOfCharge(self):
        """ % of Full Charge """
        return (self.bus.read_byte_data(self.address, 0x02) + self.bus.read_byte_data(self.address, 0x03) * 256)


Instruction:
Can you generate the documentation for the following Python 3 function
def backpropagate_2d(uSin, angles, res, nm, lD=0, coords=None,
                     weight_angles=True,
                     onlyreal=False, padding=True, padval=0,
                     count=None, max_count=None, verbose=0):
    r"""2D backpropagation with the Fourier diffraction theorem

    Two-dimensional diffraction tomography reconstruction
    algorithm for scattering of a plane wave
    :math:`u_0(\mathbf{r}) = u_0(x,z)`
    by a dielectric object with refractive index
    :math:`n(x,z)`.

    This method implements the 2D backpropagation algorithm
    :cite:`Mueller2015arxiv`.

    .. math::
        f(\mathbf{r}) =
            -\frac{i k_\mathrm{m}}{2\pi}
            \sum_{j=1}^{N} \! \Delta \phi_0 D_{-\phi_j} \!\!
            \left \{
            \text{FFT}^{-1}_{\mathrm{1D}}
            \left \{
            \left| k_\mathrm{Dx} \right|
            \frac{\text{FFT}_{\mathrm{1D}} \left \{
            u_{\mathrm{B},\phi_j}(x_\mathrm{D}) \right \}
            }{u_0(l_\mathrm{D})}
            \exp \! \left[i k_\mathrm{m}(M - 1) \cdot
            (z_{\phi_j}-l_\mathrm{D}) \right]
            \right \}
            \right \}

    with the forward :math:`\text{FFT}_{\mathrm{1D}}` and inverse
    :math:`\text{FFT}^{-1}_{\mathrm{1D}}` 1D fast Fourier transform, the
    rotational operator :math:`D_{-\phi_j}`, the angular distance between the
    projections :math:`\Delta \phi_0`, the ramp filter in Fourier space
    :math:`|k_\mathrm{Dx}|`, and the propagation distance
    :math:`(z_{\phi_j}-l_\mathrm{D})`.

    Parameters
    ----------
    uSin: (A,N) ndarray
        Two-dimensional sinogram of line recordings
        :math:`u_{\mathrm{B}, \phi_j}(x_\mathrm{D})`
        divided by the incident plane wave :math:`u_0(l_\mathrm{D})`
        measured at the detector.
    angles: (A,) ndarray
        Angular positions :math:`\phi_j` of `uSin` in radians.
    res: float
        Vacuum wavelength of the light :math:`\lambda` in pixels.
    nm: float
        Refractive index of the surrounding medium :math:`n_\mathrm{m}`.
    lD: float
        Distance from center of rotation to detector plane
        :math:`l_\mathrm{D}` in pixels.
    coords: None [(2,M) ndarray]
        Computes only the output image at these coordinates. This
        keyword is reserved for future versions and is not
        implemented yet.
    weight_angles: bool
        If `True`, weights each backpropagated projection with a factor
        proportional to the angular distance between the neighboring
        projections.

        .. math::
            \Delta \phi_0 \longmapsto \Delta \phi_j =
                \frac{\phi_{j+1} - \phi_{j-1}}{2}

        .. versionadded:: 0.1.1
    onlyreal: bool
        If `True`, only the real part of the reconstructed image
        will be returned. This saves computation time.
    padding: bool
        Pad the input data to the second next power of 2 before
        Fourier transforming. This reduces artifacts and speeds up
        the process for input image sizes that are not powers of 2.
    padval: float
        The value used for padding. This is important for the Rytov
        approximation, where an approximate zero in the phase might
        translate to 2πi due to the unwrapping algorithm. In that
        case, this value should be a multiple of 2πi.
        If `padval` is `None`, then the edge values are used for
        padding (see documentation of :func:`numpy.pad`).
    count, max_count: multiprocessing.Value or `None`
        Can be used to monitor the progress of the algorithm.
        Initially, the value of `max_count.value` is incremented
        by the total number of steps. At each step, the value
        of `count.value` is incremented.
    verbose: int
        Increment to increase verbosity.

    Returns
    -------
    f: ndarray of shape (N,N), complex if `onlyreal` is `False`
        Reconstructed object function :math:`f(\mathbf{r})` as defined
        by the Helmholtz equation.
        :math:`f(x,z) =
        k_m^2 \left(\left(\frac{n(x,z)}{n_m}\right)^2 -1\right)`

    See Also
    --------
    odt_to_ri: conversion of the object function :math:`f(\mathbf{r})`
        to refractive index :math:`n(\mathbf{r})`

    radontea.backproject: backprojection based on the Fourier slice
        theorem

    Notes
    -----
    Do not use the parameter `lD` in combination with the Rytov
    approximation - the propagation is not correctly described.
    Instead, numerically refocus the sinogram prior to converting
    it to Rytov data (using e.g. :func:`odtbrain.sinogram_as_rytov`)
    with a numerical focusing algorithm (available in the Python
    package :py:mod:`nrefocus`).
    """
    ##
    ##
    # TODO:
    # - combine the 2nd filter and the rotation in the for loop
    # to save memory. However, memory is not a big issue in 2D.
    ##
    ##
    A = angles.shape[0]
    if max_count is not None:
        max_count.value += A + 2
    # Check input data
    assert len(uSin.shape) == 2, "Input data `uB` must have shape (A,N)!"
    assert len(uSin) == A, "`len(angles)` must be  equal to `len(uSin)`!"

    if coords is not None:
        raise NotImplementedError("Output coordinates cannot yet be set " +
                                  + "for the 2D backrpopagation algorithm.")
    # Cut-Off frequency
    # km [1/px]
    km = (2 * np.pi * nm) / res
    # Here, the notation defines
    # a wave propagating to the right as:
    #
    #    u0(x) = exp(ikx)
    #
    # However, in physics usually we use the other sign convention:
    #
    #    u0(x) = exp(-ikx)
    #
    # In order to be consistent with programs like Meep or our
    # scattering script for a dielectric cylinder, we want to use the
    # latter sign convention.
    # This is not a big problem. We only need to multiply the imaginary
    # part of the scattered wave by -1.

    # Perform weighting
    if weight_angles:
        weights = util.compute_angle_weights_1d(angles).reshape(-1, 1)
        sinogram = uSin * weights
    else:
        sinogram = uSin

    # Size of the input data
    ln = sinogram.shape[1]

    # We perform padding before performing the Fourier transform.
    # This gets rid of artifacts due to false periodicity and also
    # speeds up Fourier transforms of the input image size is not
    # a power of 2.
    order = max(64., 2**np.ceil(np.log(ln * 2.1) / np.log(2)))

    if padding:
        pad = order - ln
    else:
        pad = 0

    padl = np.int(np.ceil(pad / 2))
    padr = np.int(pad - padl)

    if padval is None:
        sino = np.pad(sinogram, ((0, 0), (padl, padr)),
                      mode="edge")
        if verbose > 0:
            print("......Padding with edge values.")
    else:
        sino = np.pad(sinogram, ((0, 0), (padl, padr)),
                      mode="linear_ramp",
                      end_values=(padval,))
        if verbose > 0:
            print("......Verifying padding value: {}".format(padval))

    # zero-padded length of sinogram.
    lN = sino.shape[1]

    # Ask for the filter. Do not include zero (first element).
    #
    # Integrals over ϕ₀ [0,2π]; kx [-kₘ,kₘ]
    #   - double coverage factor 1/2 already included
    #   - unitary angular frequency to unitary ordinary frequency
    #     conversion performed in calculation of UB=FT(uB).
    #
    # f(r) = -i kₘ / ((2π)^(3/2) a₀)            (prefactor)
    #      * iint dϕ₀ dkx                       (prefactor)
    #      * |kx|                               (prefactor)
    #      * exp(-i kₘ M lD )                   (prefactor)
    #      * UBϕ₀(kx)                             (dependent on ϕ₀)
    #      * exp( i (kx t⊥ + kₘ (M - 1) s₀) r )   (dependent on ϕ₀ and r)
    #
    # (r and s₀ are vectors. In the last term we perform the dot-product)
    #
    # kₘM = sqrt( kₘ² - kx² )
    # t⊥  = (  cos(ϕ₀), sin(ϕ₀) )
    # s₀  = ( -sin(ϕ₀), cos(ϕ₀) )
    #
    # The filter can be split into two parts
    #
    # 1) part without dependence on the z-coordinate
    #
    #        -i kₘ / ((2π)^(3/2) a₀)
    #      * iint dϕ₀ dkx
    #      * |kx|
    #      * exp(-i kₘ M lD )
    #
    # 2) part with dependence of the z-coordinate
    #
    #        exp( i (kx t⊥ + kₘ (M - 1) s₀) r )
    #
    # The filter (1) can be performed using the classical filter process
    # as in the backprojection algorithm.
    #
    #
    if count is not None:
        count.value += 1

    # Corresponding sample frequencies
    fx = np.fft.fftfreq(lN)  # 1D array
    # kx is a 1D array.
    kx = 2 * np.pi * fx
    # Differentials for integral
    dphi0 = 2 * np.pi / A
    # We will later multiply with phi0.
    #               a, x
    kx = kx.reshape(1, -1)
    # Low-pass filter:
    # less-than-or-equal would give us zero division error.
    filter_klp = (kx**2 < km**2)

    # Filter M so there are no nans from the root
    M = 1. / km * np.sqrt((km**2 - kx**2) * filter_klp)

    prefactor = -1j * km / (2 * np.pi)
    prefactor *= dphi0
    prefactor *= np.abs(kx) * filter_klp
    # new in version 0.1.4:
    # We multiply by the factor (M-1) instead of just (M)
    # to take into account that we have a scattered
    # wave that is normalized by u0.
    prefactor *= np.exp(-1j * km * (M-1) * lD)
    # Perform filtering of the sinogram
    projection = np.fft.fft(sino, axis=-1) * prefactor

    #
    # filter (2) must be applied before rotation as well
    # exp( i (kx t⊥ + kₘ (M - 1) s₀) r )
    #
    # t⊥  = (  cos(ϕ₀), sin(ϕ₀) )
    # s₀  = ( -sin(ϕ₀), cos(ϕ₀) )
    #
    # This filter is effectively an inverse Fourier transform
    #
    # exp(i kx xD) exp(i kₘ (M - 1) yD )
    #
    # xD =   x cos(ϕ₀) + y sin(ϕ₀)
    # yD = - x sin(ϕ₀) + y cos(ϕ₀)

    # Everything is in pixels
    center = ln / 2.0
    x = np.arange(lN) - center + .5
    # Meshgrid for output array
    yv = x.reshape(-1, 1)

    Mp = M.reshape(1, -1)
    filter2 = np.exp(1j * yv * km * (Mp - 1))  # .reshape(1,lN,lN)

    projection = projection.reshape(A, 1, lN)  # * filter2

    # Prepare complex output image
    if onlyreal:
        outarr = np.zeros((ln, ln))
    else:
        outarr = np.zeros((ln, ln), dtype=np.dtype(complex))

    if count is not None:
        count.value += 1

    # Calculate backpropagations
    for i in np.arange(A):
        # Create an interpolation object of the projection.

        # interpolation of the rotated fourier transformed projection
        # this is already tiled onto the entire image.
        sino_filtered = np.fft.ifft(projection[i] * filter2, axis=-1)

        # Resize filtered sinogram back to original size
        sino = sino_filtered[:ln, padl:padl + ln]

        rotated_projr = scipy.ndimage.interpolation.rotate(
            sino.real, -angles[i] * 180 / np.pi,
            reshape=False, mode="constant", cval=0)
        # Append results

        outarr += rotated_projr

        if not onlyreal:
            outarr += 1j * scipy.ndimage.interpolation.rotate(
                sino.imag, -angles[i] * 180 / np.pi,
                reshape=False, mode="constant", cval=0)

        if count is not None:
            count.value += 1

    return outarr

Response: r2D backpropagation of a single object in a 2D manner.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def color(x, y):

    if (x-4) > (y-4) and -(y-4) <= (x-4):
        # right
        return "#CDB95B"
    elif (x-4) > (y-4) and -(y-4) > (x-4):
        # top
        return "#CD845B"
    elif (x-4) <= (y-4) and -(y-4) <= (x-4):
        # bottom
        return "#57488E"
    elif (x-4) <= (y-4) and -(y-4) > (x-4):
        # left
        return "#3B8772"

    # should not happen
    return "black"

Response: Color of the current image.


Instruction:
Can you tell what is the following Python 3 function doing
def single_read(self, register):
        '''
        Reads data from desired register only once.
        '''

        comm_reg = (0b00010 << 3) + register

        if register == self.AD7730_STATUS_REG:
            bytes_num = 1
        elif register == self.AD7730_DATA_REG:
            bytes_num = 3
        elif register == self.AD7730_MODE_REG:
            bytes_num = 2
        elif register == self.AD7730_FILTER_REG:
            bytes_num = 3
        elif register == self.AD7730_DAC_REG:
            bytes_num = 1
        elif register == self.AD7730_OFFSET_REG:
            bytes_num = 3
        elif register == self.AD7730_GAIN_REG:
            bytes_num = 3
        elif register == self.AD7730_TEST_REG:
            bytes_num = 3

        command = [comm_reg] + ([0x00] * bytes_num)
        spi.SPI_write(self.CS, command)
        data = spi.SPI_read(bytes_num + 1)
        return data[1:]

Response: Reads data from the specified register only once.


Instruction:
Given the following Python 3 function, write the documentation
def getStatus(self):

        """
        RDY - Ready Bit. This bit provides the status of the RDY flag from the part. The status and function of this bit is the same as the RDY output pin. A number of events set the RDY bit high as indicated in Table XVIII in datasheet

        STDY - Steady Bit. This bit is updated when the filter writes a result to the Data Register. If the filter is
        in FASTStep mode (see Filter Register section) and responding to a step input, the STDY bit
        remains high as the initial conversion results become available. The RDY output and bit are set
        low on these initial conversions to indicate that a result is available. If the STDY is high, however,
        it indicates that the result being provided is not from a fully settled second-stage FIR filter. When the
        FIR filter has fully settled, the STDY bit will go low coincident with RDY. If the part is never placed
        into its FASTStep mode, the STDY bit will go low at the first Data Register read and it is
        not cleared by subsequent Data Register reads. A number of events set the STDY bit high as indicated in Table XVIII. STDY is set high along with RDY by all events in the table except a Data Register read.

        STBY - Standby Bit. This bit indicates whether the AD7730 is in its Standby Mode or normal mode of
        operation. The part can be placed in its standby mode using the STANDBY input pin or by
        writing 011 to the MD2 to MD0 bits of the Mode Register. The power-on/reset status of this bit
        is 0 assuming the STANDBY pin is high.

NOREF - No Reference Bit. If the voltage between the REF IN(+) and REF IN(-) pins is below 0.3 V, or either of these inputs is open-circuit, the NOREF bit goes to 1. If NOREF is active on completion of a conversion, the Data Register is loaded with all 1s. If NOREF is active on completion of a calibration, updating of the calibration registers is inhibited."""

        status = self.single_read(self.AD7730_STATUS_REG)
        bits_values = dict([('NOREF',status[0] & 0x10 == 0x10),
                            ('STBY',status[0] & 0x20 == 0x20),
                            ('STDY',status[0] & 0x40 == 0x40),
                            ('RDY',status[0] & 0x80 == 0x80)])
        return bits_values

Response: This method returns the status of the specific entry in the filter.


Instruction:
Create a Python 3 function to
set the mode of the specified channel.

Response:
def setMode(self
                    ,mode
                    ,polarity
                    ,den
                    ,iovalue
                    ,data_length
                    ,reference
                    ,input_range
                    ,clock_enable
                    ,burn_out
                    ,channel):
        '''
        def setMode(self
                    ,mode = self.AD7730_IDLE_MODE
                    ,polarity = self.AD7730_UNIPOLAR_MODE
                    ,den = self.AD7730_IODISABLE_MODE
                    ,iovalue = 0b00
                    ,data_lenght = self.AD7730_24bitDATA_MODE
                    ,reference = self.AD7730_REFERENCE_5V
                    ,input_range = self.AD7730_40mVIR_MODE
                    ,clock_enable = self.AD7730_MCLK_ENABLE_MODE
                    ,burn_out = self.AD7730_BURNOUT_DISABLE
                    ,channel = self.AD7730_AIN1P_AIN1N
               ):
        '''
        mode_MSB = (mode << 5) + (polarity << 4) + (den << 3) + (iovalue << 1) + data_length
        mode_LSB = (reference << 7) + (0b0 << 6) + (input_range << 4) + (clock_enable << 3) + (burn_out << 2) + channel

        self.single_write(self.AD7730_MODE_REG, [mode_MSB, mode_LSB])


Instruction:
Make a summary of the following Python 3 code
def downsample(work_dir, sample_name, fastq_left_fpath, fastq_right_fpath, downsample_to, num_pairs=None):
    sample_name = sample_name or splitext(''.join(lc if lc == rc else '' for lc, rc in zip(fastq_left_fpath, fastq_right_fpath)))[0]

    l_out_fpath = make_downsampled_fpath(work_dir, fastq_left_fpath)
    r_out_fpath = make_downsampled_fpath(work_dir, fastq_right_fpath)
    if can_reuse(l_out_fpath, [fastq_left_fpath, fastq_right_fpath]):
        return l_out_fpath, r_out_fpath

    info('Processing ' + sample_name)
    if num_pairs is None:
        info(sample_name + ': counting number of reads in fastq...')
        num_pairs = _count_records_in_fastq(fastq_left_fpath)
    if num_pairs > LIMIT:
        info(sample_name + ' the number of reads is higher than ' + str(LIMIT) +
             ', sampling from only first ' + str(LIMIT))
        num_pairs = LIMIT
    info(sample_name + ': ' + str(num_pairs) + ' reads')
    num_downsample_pairs = int(downsample_to * num_pairs) if isinstance(downsample_to, float) else downsample_to
    if num_pairs <= num_downsample_pairs:
        info(sample_name + ': and it is less than ' + str(num_downsample_pairs) + ', so no downsampling.')
        return fastq_left_fpath, fastq_right_fpath
    else:
        info(sample_name + ': downsampling to ' + str(num_downsample_pairs))
        rand_records = sorted(random.sample(range(num_pairs), num_downsample_pairs))

    info('Opening ' + fastq_left_fpath)
    fh1 = open_gzipsafe(fastq_left_fpath)
    info('Opening ' + fastq_right_fpath)
    fh2 = open_gzipsafe(fastq_right_fpath) if fastq_right_fpath else None

    out_files = (l_out_fpath, r_out_fpath) if r_out_fpath else (l_out_fpath,)

    written_records = 0
    with file_transaction(work_dir, out_files) as tx_out_files:
        if isinstance(tx_out_files, six.string_types):
            tx_out_f1 = tx_out_files
        else:
            tx_out_f1, tx_out_f2 = tx_out_files
        info('Opening ' + str(tx_out_f1) + ' to write')
        sub1 = open_gzipsafe(tx_out_f1, "w")
        info('Opening ' + str(tx_out_f2) + ' to write')
        sub2 = open_gzipsafe(tx_out_f2, "w") if r_out_fpath else None
        rec_no = -1
        for rr in rand_records:
            while rec_no < rr:
                rec_no += 1
                for i in range(4): fh1.readline()
                if fh2:
                    for i in range(4): fh2.readline()
            for i in range(4):
                sub1.write(fh1.readline())
                if sub2:
                    sub2.write(fh2.readline())
            written_records += 1
            if written_records % 10000 == 0:
                info(sample_name + ': written ' + str(written_records) + ', rec_no ' + str(rec_no + 1))
            if rec_no > num_pairs:
                info(sample_name + ' reached the limit of ' + str(num_pairs), ' read lines, stopping.')
                break
        info(sample_name + ': done, written ' + str(written_records) + ', rec_no ' + str(rec_no))
        fh1.close()
        sub1.close()
        if fastq_right_fpath:
            fh2.close()
            sub2.close()

    info(sample_name + ': done downsampling, saved to ' + l_out_fpath + ' and ' + r_out_fpath + ', total ' + str(written_records) + ' paired reads written')
    return l_out_fpath, r_out_fpath

Response: Downsample a fastq file into memory and return the new file and the new file.


Instruction:
Here you have a function in Python 3, explain what it does
def _calculate_checksum(value):
        # CRC
        polynomial = 0x131  # //P(x)=x^8+x^5+x^4+1 = 100110001
        crc = 0xFF

        # calculates 8-Bit checksum with given polynomial
        for byteCtr in [ord(x) for x in struct.pack(">H", value)]:
            crc ^= byteCtr
            for bit in range(8, 0, -1):
                if crc & 0x80:
                    crc = (crc << 1) ^ polynomial
                else:
                    crc = (crc << 1)
        return crc

Response: 4. 12 Checksum Calculation from an unsigned short input


Instruction:
Make a summary of the following Python 3 code
def run(self):

        inside = 0
        for draws in range(1, self.data['samples']):
            # generate points and check whether they are inside the unit circle
            r1, r2 = (random(), random())
            if r1 ** 2 + r2 ** 2 < 1.0:
                inside += 1

            if draws % 1000 != 0:
                continue

            # debug
            yield self.emit('log', {'draws': draws, 'inside': inside})

            # calculate pi and its uncertainty given the current draws
            p = inside / draws
            pi = {
                'estimate': 4.0 * inside / draws,
                'uncertainty': 4.0 * math.sqrt(draws * p * (1.0 - p)) / draws,
            }

            # send status to frontend
            yield self.set_state(pi=pi)

        yield self.emit('log', {'action': 'done'})

Response: Run when button is pressed.


Instruction:
Make a summary of the following Python 3 code
def on_action(action):
    @wrapt.decorator
    @tornado.gen.coroutine
    def _execute(wrapped, instance, args, kwargs):
        return wrapped(*args, **kwargs)

    _execute.action = action
    return _execute

Response: Decorator for action handlers.


Instruction:
How would you explain what the following Python 3 function does
def init_datastores(self):
        self.data = Datastore(self.id_)
        self.data.subscribe(lambda data: self.emit('data', data))
        self.class_data = Datastore(type(self).__name__)
        self.class_data.subscribe(lambda data: self.emit('class_data', data))

Response: Initialize datastores for this analysis instance.


Instruction:
Can you implement a function in Python 3 that
emits a signal to the frontend.

Response:
def emit(self, signal, message='__nomessagetoken__'):
        """Emit a signal to the frontend.

        :param str signal: name of the signal
        :param message: message to send
        :returns: return value from frontend emit function
        :rtype: tornado.concurrent.Future
        """
        # call pre-emit hooks
        if signal == 'log':
            self.log_backend.info(message)
        elif signal == 'warn':
            self.log_backend.warn(message)
        elif signal == 'error':
            self.log_backend.error(message)

        return self.emit_to_frontend(signal, message)


Instruction:
Explain what the following Python 3 code does
def set(self, i, value):
        value_encoded = encode(value, self.get_change_trigger(i))

        if i in self.data and self.data[i] == value_encoded:
            return self

        self.data[i] = value_encoded
        return self.trigger_changed(i)

Response: Set value at position i and return a Future.


Instruction:
Create a Python 3 function for
setting a value at key and return a Future.

Response:
def set(self, key, value):
        """Set a value at key and return a Future.

        :rtype: tornado.concurrent.Future
        """
        value_encoded = encode(value, self.get_change_trigger(key))

        if key in self.data and self.data[key] == value_encoded:
            return self

        self.data[key] = value_encoded
        return self.trigger_changed(key)


Instruction:
Can you generate the documentation for the following Python 3 function
def update(self, new_data):
        for k, v in new_data.items():
            self[k] = v

        return self

Response: Update the dictionary with the new data.


Instruction:
Create a Python 3 function for
triggering all callbacks that were set with on_change.

Response:
def trigger_all_change_callbacks(self):
        """Trigger all callbacks that were set with on_change()."""
        return [
            ret
            for key in DatastoreLegacy.store[self.domain].keys()
            for ret in self.trigger_change_callbacks(key)
        ]


Instruction:
Can you write a function in Python 3 where it
sets value at key and return a Future

Response:
def set(self, key, value):
        """Set value at key and return a Future

        :rtype: tornado.concurrent.Future
        """
        return DatastoreLegacy.store[self.domain].set(key, value)


Instruction:
Here you have a function in Python 3, explain what it does
def init(self, key_value_pairs):
        for k, v in key_value_pairs.items():
            if k not in DatastoreLegacy.store[self.domain]:
                DatastoreLegacy.store[self.domain][k] = v

Response:
Initialize datastore.

        Only sets values for keys that are not in the datastore already.

        :param dict key_value_pairs:
            A set of key value pairs to use to initialize the datastore.


Instruction:
Can you write a function in Python 3 where it
closes and delete instance.

Response:
def close(self):
        """Close and delete instance."""

        # remove callbacks
        DatastoreLegacy.datastores[self.domain].remove(self)

        # delete data after the last instance is gone
        if self.release_storage and \
           not DatastoreLegacy.datastores[self.domain]:
            del DatastoreLegacy.store[self.domain]

        del self


Instruction:
Write a Python 3 script to
generate html and dirhtml output.

Response:
def gen(skipdirhtml=False):
    """Generate html and dirhtml output."""
    docs_changelog = 'docs/changelog.rst'
    check_git_unchanged(docs_changelog)
    pandoc('--from=markdown', '--to=rst', '--output=' + docs_changelog, 'CHANGELOG.md')
    if not skipdirhtml:
        sphinx_build['-b', 'dirhtml', '-W', '-E', 'docs', 'docs/_build/dirhtml'] & FG
    sphinx_build['-b', 'html', '-W', '-E', 'docs', 'docs/_build/html'] & FG


Instruction:
Can you tell what is the following Python 3 function doing
def __reset_crosshair(self):
        self.lhor.set_ydata(self.y_coord)
        self.lver.set_xdata(self.x_coord)

Response: redraw the cross - hair on the horizontal slice plot


Instruction:
Implement a Python 3 function for
initializing the vertical plot for the current locale.

Response:
def __init_vertical_plot(self):
        """
        set up the vertical profile plot

        Returns
        -------
        """
        # clear the plot if lines have already been drawn on it
        if len(self.ax2.lines) > 0:
            self.ax2.cla()
        # set up the vertical profile plot
        self.ax2.set_ylabel(self.datalabel, fontsize=self.fontsize)
        self.ax2.set_xlabel(self.spectrumlabel, fontsize=self.fontsize)
        self.ax2.set_title('vertical point profiles', fontsize=self.fontsize)
        self.ax2.set_xlim([1, self.bands])
        # plot vertical line at the slider position
        self.vline = self.ax2.axvline(self.slider.value, color='black')


Instruction:
Here you have a function in Python 3, explain what it does
def __onclick(self, event):
        # only do something if the first plot has been clicked on
        if event.inaxes == self.ax1:

            # retrieve the click coordinates
            self.x_coord = event.xdata
            self.y_coord = event.ydata

            # redraw the cross-hair
            self.__reset_crosshair()

            x, y = self.__map2img(self.x_coord, self.y_coord)
            subset_vertical = self.__read_timeseries(x, y)

            # redraw/clear the vertical profile plot in case stacking is disabled
            if not self.checkbox.value:
                self.__init_vertical_plot()

            # plot the vertical profile
            label = 'x: {0:03}; y: {1:03}'.format(x, y)
            self.ax2.plot(self.timestamps, subset_vertical, label=label)
            self.ax2_legend = self.ax2.legend(loc=0, prop={'size': 7}, markerscale=1)

Response: This function is called when the mouse is clicked on the base calendar. It updates the vertical profile and slice plots.


Instruction:
Write a Python 3 script to
return a function suitable for passing as the more_formatters argument athrow.

Response:
def LookupChain(lookup_func_list):
  """Returns a *function* suitable for passing as the more_formatters argument
  to Template.

  NOTE: In Java, this would be implemented using the 'Composite' pattern.  A
  *list* of formatter lookup function behaves the same as a *single* formatter
  lookup funcion.

  Note the distinction between formatter *lookup* functions and formatter
  functions here.
  """
  def MoreFormatters(formatter_name):
    for lookup_func in lookup_func_list:
      formatter_func = lookup_func(formatter_name)
      if formatter_func is not None:
        return formatter_func

  return MoreFormatters


Instruction:
Create a Python 3 function for
using Python % format strings as template format specifiers.

Response:
def PythonPercentFormat(format_str):
  """Use Python % format strings as template format specifiers."""

  if format_str.startswith('printf '):
    fmt = format_str[len('printf '):]
    return lambda value: fmt % value
  else:
    return None


Instruction:
Given the following Python 3 function, write the documentation
def Plural(format_str):
  if format_str.startswith('plural?'):
    i = len('plural?')

    try:
      splitchar = format_str[i]  # Usually a space, but could be something else
      _, plural_val, singular_val = format_str.split(splitchar)
    except IndexError:
      raise Error('plural? must have exactly 2 arguments')

    def Formatter(value):
      plural = False
      if isinstance(value, int) and value > 1:
        plural = True
      if isinstance(value, list) and len(value) > 1:
        plural = True

      if plural:
        return plural_val
      else:
        return singular_val

    return Formatter

  else:
    return None

Response: Returns whether the value should be considered a plural value.


Instruction:
Write a Python 3 function for
returning all merged CDSs for a genome

Response:
def get_merged_cds(genome):
    """
    Returns all CDS merged, used:
    - for TargQC general reports CDS coverage statistics for WGS
    - for Seq2C CNV calling when no capture BED available
    """
    bed = get_all_features(genome)
    debug('Filtering BEDTool for high confidence CDS and stop codons')
    return bed\
        .filter(lambda r: r.fields[BedCols.FEATURE] in ['CDS', 'stop_codon'])\
        .filter(high_confidence_filter)\
        .merge()


Instruction:
Can you tell what is the following Python 3 function doing
def _get(relative_path, genome=None):
    chrom = None
    if genome:
        if '-chr' in genome:
            genome, chrom = genome.split('-')
        check_genome(genome)
        relative_path = relative_path.format(genome=genome)

    path = abspath(join(dirname(__file__), relative_path))
    if not isfile(path) and isfile(path + '.gz'):
        path += '.gz'

    if path.endswith('.bed') or path.endswith('.bed.gz'):
        if path.endswith('.bed.gz'):
            bedtools = which('bedtools')
            if not bedtools:
                critical('bedtools not found in PATH: ' + str(os.environ['PATH']))
            debug('BED is compressed, creating BedTool')
            bed = BedTool(path)
        else:
            debug('BED is uncompressed, creating BedTool')
            bed = BedTool(path)

        if chrom:
            debug('Filtering BEDTool for chrom ' + chrom)
            bed = bed.filter(lambda r: r.chrom == chrom)
        return bed
    else:
        return path

Response: Returns the BedTools object for the given path.


Instruction:
Can you tell what is the following Python 3 function doing
def run(cmd, output_fpath=None, input_fpath=None, checks=None, stdout_to_outputfile=True,
        stdout_tx=True, reuse=False, env_vars=None):
    """Run the provided command, logging details and checking for errors.
    """
    if output_fpath and reuse:
        if verify_file(output_fpath, silent=True):
            info(output_fpath + ' exists, reusing')
            return output_fpath
        if not output_fpath.endswith('.gz') and verify_file(output_fpath + '.gz', silent=True):
            info(output_fpath + '.gz exists, reusing')
            return output_fpath

    env = os.environ.copy()
    if env_vars:
        for k, v in env_vars.items():
            if v is None:
                if k in env:
                    del env[k]
            else:
                env[k] = v

    if checks is None:
        checks = [file_nonempty_check]

    def _try_run(_cmd, _output_fpath, _input_fpath):
        try:
            info(' '.join(str(x) for x in _cmd) if not isinstance(_cmd, six.string_types) else _cmd)
            _do_run(_cmd, checks, env, _output_fpath, _input_fpath)
        except:
            raise

    if output_fpath:
        if isfile(output_fpath):
            os.remove(output_fpath)
    if output_fpath:
        if stdout_tx:
            with file_transaction(None, output_fpath) as tx_out_file:
                if stdout_to_outputfile:
                    cmd += ' > ' + tx_out_file
                else:
                    cmd += '\n'
                    cmd = cmd.replace(' ' + output_fpath + ' ', ' ' + tx_out_file + ' ') \
                             .replace(' "' + output_fpath + '" ', ' ' + tx_out_file + '" ') \
                             .replace(' \'' + output_fpath + '\' ', ' ' + tx_out_file + '\' ') \
                             .replace(' ' + output_fpath + '\n', ' ' + tx_out_file) \
                             .replace(' "' + output_fpath + '"\n', ' ' + tx_out_file + '"') \
                             .replace(' \'' + output_fpath + '\'\n', ' ' + tx_out_file + '\'') \
                             .replace('\n', '')
                _try_run(cmd, tx_out_file, input_fpath)
        else:
            _try_run(cmd, output_fpath, input_fpath)

    else:
        _try_run(cmd, None, input_fpath)

Response: Run the provided command and return the path to the output file.


Instruction:
Make a summary of the following Python 3 code
def _correct_qualimap_genome_results(samples):
    for s in samples:
        if verify_file(s.qualimap_genome_results_fpath):
            correction_is_needed = False
            with open(s.qualimap_genome_results_fpath, 'r') as f:
                content = f.readlines()
                metrics_started = False
                for line in content:
                    if ">> Reference" in line:
                        metrics_started = True
                    if metrics_started:
                        if line.find(',') != -1:
                            correction_is_needed = True
                            break
            if correction_is_needed:
                with open(s.qualimap_genome_results_fpath, 'w') as f:
                    metrics_started = False
                    for line in content:
                        if ">> Reference" in line:
                            metrics_started = True
                        if metrics_started:
                            if line.find(',') != -1:
                                line = line.replace(',', '')
                        f.write(line)

Response: fix java. lang. Double. parseDouble error on entries like 6082. 49


Instruction:
Can you generate the documentation for the following Python 3 function
def _correct_qualimap_insert_size_histogram(work_dir, samples):
    for s in samples:
        qualimap1_dirname = dirname(s.qualimap_ins_size_hist_fpath).replace('raw_data_qualimapReport', 'raw_data')
        qualimap2_dirname = dirname(s.qualimap_ins_size_hist_fpath)
        if exists(qualimap1_dirname):
            if not exists(qualimap2_dirname):
                shutil.move(qualimap1_dirname, qualimap2_dirname)
            else:
                shutil.rmtree(qualimap1_dirname)
        elif not exists(qualimap2_dirname):
            continue  # no data from both Qualimap v.1 and Qualimap v.2

        # if qualimap histogram exits and reuse_intermediate, skip
        if verify_file(s.qualimap_ins_size_hist_fpath, silent=True) and tc.reuse_intermediate:
            pass
        else:
            if verify_file(s.picard_ins_size_hist_txt_fpath):
                with open(s.picard_ins_size_hist_txt_fpath, 'r') as picard_f:
                    one_line_to_stop = False
                    for line in picard_f:
                        if one_line_to_stop:
                            break
                        if line.startswith('## HISTOGRAM'):
                            one_line_to_stop = True

                    with file_transaction(work_dir, s.qualimap_ins_size_hist_fpath) as tx:
                        with open(tx, 'w') as qualimap_f:
                            for line in picard_f:
                                qualimap_f.write(line)

Response: This function takes a list of samples and corrects the insert size histogram.


Instruction:
Here you have a function in Python 3, explain what it does
def norm_vec(vector):
    assert len(vector) == 3
    v = np.array(vector)
    return v/np.sqrt(np.sum(v**2))

Response: Normalize the length of a vector to one


Instruction:
Can you tell what is the following Python 3 function doing
def rotate_points_to_axis(points, axis):
    axis = norm_vec(axis)
    u, v, w = axis
    points = np.array(points)
    # Determine the rotational angle in the x-z plane
    phi = np.arctan2(u, w)
    # Determine the tilt angle w.r.t. the y-axis
    theta = np.arccos(v)

    # Negative rotation about y-axis
    Rphi = np.array([
        [np.cos(phi), 0, -np.sin(phi)],
        [0, 1,            0],
        [np.sin(phi), 0,  np.cos(phi)],
    ])

    # Negative rotation about x-axis
    Rtheta = np.array([
        [1,             0,              0],
        [0, np.cos(theta),  np.sin(theta)],
        [0, -np.sin(theta),  np.cos(theta)],
    ])

    DR1 = np.dot(Rtheta, Rphi)
    # Rotate back by -phi such that effective rotation was only
    # towards [0,1,0].
    DR = np.dot(Rphi.T, DR1)
    rotpoints = np.zeros((len(points), 3))
    for ii, pnt in enumerate(points):
        rotpoints[ii] = np.dot(DR, pnt)

    # For visualiztaion:
    # import matplotlib.pylab as plt
    # from mpl_toolkits.mplot3d import Axes3D
    # from matplotlib.patches import FancyArrowPatch
    # from mpl_toolkits.mplot3d import proj3d
    #
    # class Arrow3D(FancyArrowPatch):
    #     def __init__(self, xs, ys, zs, *args, **kwargs):
    #         FancyArrowPatch.__init__(self, (0,0), (0,0), *args, **kwargs)
    #         self._verts3d = xs, ys, zs
    #
    #     def draw(self, renderer):
    #         xs3d, ys3d, zs3d = self._verts3d
    #         xs, ys, zs = proj3d.proj_transform(xs3d, ys3d, zs3d, renderer.M)
    #         self.set_positions((xs[0],ys[0]),(xs[1],ys[1]))
    #         FancyArrowPatch.draw(self, renderer)
    #
    # fig = plt.figure(figsize=(10,10))
    # ax = fig.add_subplot(111, projection='3d')
    # for vec in rotpoints:
    #     u,v,w = vec
    #     a = Arrow3D([0,u],[0,v],[0,w],
    #                 mutation_scale=20, lw=1, arrowstyle="-|>")
    #     ax.add_artist(a)
    #
    # radius=1
    # ax.set_xlabel('X')
    # ax.set_ylabel('Y')
    # ax.set_zlabel('Z')
    # ax.set_xlim(-radius*1.5, radius*1.5)
    # ax.set_ylim(-radius*1.5, radius*1.5)
    # ax.set_zlim(-radius*1.5, radius*1.5)
    # plt.tight_layout()
    # plt.show()

    return rotpoints

Response: Rotates a list of points along a reference axis.


Instruction:
Write a Python 3 function that can
compute the rotation matrix that rotates from [ 0 1 ) to point.

Response:
def rotation_matrix_from_point(point, ret_inv=False):
    """Compute rotation matrix to go from [0,0,1] to `point`.

    First, the matrix rotates to in the polar direction. Then,
    a rotation about the y-axis is performed to match the
    azimuthal angle in the x-z-plane.

    This rotation matrix is required for the correct 3D orientation
    of the backpropagated projections.

    Parameters
    ----------
    points: list-like, length 3
        The coordinates of the point in 3D.
    ret_inv: bool
        Also return the inverse of the rotation matrix. The inverse
        is required for :func:`scipy.ndimage.interpolation.affine_transform`
        which maps the output coordinates to the input coordinates.

    Returns
    -------
    Rmat [, Rmat_inv]: 3x3 ndarrays
        The rotation matrix that rotates [0,0,1] to `point` and
        optionally its inverse.
    """
    x, y, z = point
    # azimuthal angle
    phi = np.arctan2(x, z)
    # angle in polar direction (negative)
    theta = -np.arctan2(y, np.sqrt(x**2+z**2))

    # Rotation in polar direction
    Rtheta = np.array([
        [1,             0,              0],
        [0, np.cos(theta), -np.sin(theta)],
        [0, np.sin(theta),  np.cos(theta)],
    ])

    # rotation in x-z-plane
    Rphi = np.array([
        [np.cos(phi), 0, -np.sin(phi)],
        [0, 1,            0],
        [np.sin(phi), 0, np.cos(phi)],
    ])

    D = np.dot(Rphi, Rtheta)
    # The inverse of D
    Dinv = np.dot(Rtheta.T, Rphi.T)

    if ret_inv:
        return D, Dinv
    else:
        return D


Instruction:
Given the following Python 3 function, write the documentation
def rotation_matrix_from_point_planerot(point, plane_angle, ret_inv=False):
    # These matrices are correct if there is no tilt of the
    # rotational axis within the detector plane (x-y).
    D, Dinv = rotation_matrix_from_point(point, ret_inv=True)

    # We need an additional rotation about the z-axis to correct
    # for the tilt for all the the other cases.
    angz = plane_angle

    Rz = np.array([
        [np.cos(angz), -np.sin(angz), 0],
        [np.sin(angz),  np.cos(angz), 0],
        [0,             0,            1],
    ])

    DR = np.dot(D, Rz)
    DRinv = np.dot(Rz.T, Dinv)

    if ret_inv:
        return DR, DRinv
    else:
        return DR

Response: Compute the rotation matrix that rotates to a point on a 3D plane.


Instruction:
Here you have a function in Python 3, explain what it does
def sphere_points_from_angles_and_tilt(angles, tilted_axis):
    assert len(angles.shape) == 1
    # Normalize tilted axis.
    tilted_axis = norm_vec(tilted_axis)
    [u, v, w] = tilted_axis
    # Initial distribution of points about great circle (x-z).
    newang = np.zeros((angles.shape[0], 3), dtype=float)
    # We subtract angles[0], because in step (a) we want that
    # newang[0]==[0,0,1]. This only works if we actually start
    # at that point.
    newang[:, 0] = np.sin(angles-angles[0])
    newang[:, 2] = np.cos(angles-angles[0])

    # Compute rotational angles w.r.t. [0,1,0].
    # - Draw a unit sphere with the y-axis pointing up and the
    #   z-axis pointing right
    # - The rotation of `tilted_axis` can be described by two
    #   separate rotations. We will use these two angles:
    #   (a) Rotation from y=1 within the y-z plane: theta
    #       This is the rotation that is critical for data
    #       reconstruction. If this angle is zero, then we
    #       have a rotational axis in the imaging plane. If
    #       this angle is PI/2, then our sinogram consists
    #       of a rotating image and 3D reconstruction is
    #       impossible. This angle is counted from the y-axis
    #       onto the x-z plane.
    #   (b) Rotation in the x-z plane: phi
    #       This angle is responsible for matching up the angles
    #       with the correct sinogram images. If this angle is zero,
    #       then the projection of the rotational axis onto the
    #       x-y plane is aligned with the y-axis. If this angle is
    #       PI/2, then the axis and its projection onto the x-y
    #       plane are identical. This angle is counted from the
    #       positive z-axis towards the positive x-axis. By default,
    #       angles[0] is the point that touches the great circle
    #       that lies in the x-z plane. angles[1] is the next point
    #       towards the x-axis if phi==0.

    # (a) This angle is the azimuthal angle theta measured from the
    #     y-axis.
    theta = np.arccos(v)

    # (b) This is the polar angle measured in the x-z plane starting
    #     at the x-axis and measured towards the positive z-axis.
    if np.allclose(u, 0) and np.allclose(w, 0):
        # Avoid flipping the axis of rotation due to numerical
        # errors during its computation.
        phi = 0
    else:
        phi = np.arctan2(u, w)

    # Determine the projection points on the unit sphere.
    # The resulting circle meets the x-z-plane at phi, and
    # is tilted by theta w.r.t. the y-axis.

    # (a) Create a tilted data set. This is achieved in 3 steps.

    # a1) Determine radius of tilted circle and get the centered
    #     circle with a smaller radius.
    rtilt = np.cos(theta)
    newang *= rtilt

    # a2) Rotate this circle about the x-axis by theta
    #     (right-handed/counter-clockwise/basic/elemental rotation)
    Rx = np.array([
        [1,             0,              0],
        [0, np.cos(theta), -np.sin(theta)],
        [0, np.sin(theta),  np.cos(theta)]
    ])
    for ii in range(newang.shape[0]):
        newang[ii] = np.dot(Rx, newang[ii])

    # a3) Shift newang such that newang[0] is located at (0,0,1)
    newang = newang - (newang[0] - np.array([0, 0, 1])).reshape(1, 3)

    # (b) Rotate the entire thing with phi about the y-axis
    #     (right-handed/counter-clockwise/basic/elemental rotation)
    Ry = np.array([
        [+np.cos(phi), 0, np.sin(phi)],
        [0,            1,           0],
        [-np.sin(phi), 0, np.cos(phi)]
    ])

    for jj in range(newang.shape[0]):
        newang[jj] = np.dot(Ry, newang[jj])

    # For visualiztaion:
    # import matplotlib.pylab as plt
    # from mpl_toolkits.mplot3d import Axes3D
    # from matplotlib.patches import FancyArrowPatch
    # from mpl_toolkits.mplot3d import proj3d
    #
    # class Arrow3D(FancyArrowPatch):
    #     def __init__(self, xs, ys, zs, *args, **kwargs):
    #         FancyArrowPatch.__init__(self, (0,0), (0,0), *args, **kwargs)
    #         self._verts3d = xs, ys, zs
    #
    #     def draw(self, renderer):
    #         xs3d, ys3d, zs3d = self._verts3d
    #         xs, ys, zs = proj3d.proj_transform(xs3d, ys3d, zs3d, renderer.M)
    #         self.set_positions((xs[0],ys[0]),(xs[1],ys[1]))
    #         FancyArrowPatch.draw(self, renderer)
    #
    # fig = plt.figure(figsize=(10,10))
    # ax = fig.add_subplot(111, projection='3d')
    # for vec in newang:
    #     u,v,w = vec
    #     a = Arrow3D([0,u],[0,v],[0,w],
    #                 mutation_scale=20, lw=1, arrowstyle="-|>")
    #     ax.add_artist(a)
    #
    # radius=1
    # ax.set_xlabel('X')
    # ax.set_ylabel('Y')
    # ax.set_zlabel('Z')
    # ax.set_xlim(-radius*1.5, radius*1.5)
    # ax.set_ylim(-radius*1.5, radius*1.5)
    # ax.set_zlim(-radius*1.5, radius*1.5)
    # plt.tight_layout()
    # plt.show()

    return newang

Response: Compute the points on a unit sphere that correspond to the distribution of points about the given angles and tilt.


Instruction:
Create a Python 3 function for
giving a set of file names produce a list of names consisting of the uniq parts of the names. This works from the end of the name.

Response:
def filenames_to_uniq(names,new_delim='.'):
    '''
    Given a set of file names, produce a list of names consisting of the
    uniq parts of the names. This works from the end of the name.  Chunks of
    the name are split on '.' and '-'.

    For example:
        A.foo.bar.txt
        B.foo.bar.txt
        returns: ['A','B']

        AA.BB.foo.txt
        CC.foo.txt
        returns: ['AA.BB','CC']

    '''
    name_words = []
    maxlen = 0
    for name in names:
        name_words.append(name.replace('.',' ').replace('-',' ').strip().split())
        name_words[-1].reverse()
        if len(name_words[-1]) > maxlen:
            maxlen = len(name_words[-1])

    common = [False,] * maxlen
    for i in range(maxlen):
        last = None
        same = True
        for nameword in name_words:
            if i >= len(nameword):
                same = False
                break
            if not last:
                last = nameword[i]
            elif nameword[i] != last:
                same = False
                break
        common[i] = same

    newnames = []
    for nameword in name_words:
        nn = []
        for (i, val) in enumerate(common):
            if not val and i < len(nameword):
                nn.append(nameword[i])
        nn.reverse()
        newnames.append(new_delim.join(nn))

    return newnames


Instruction:
Here you have a function in Python 3, explain what it does
def run_process(self, analysis, action_name, message='__nomessagetoken__'):

        if action_name == 'connect':
            analysis.on_connect(self.executable, self.zmq_publish)

        while not analysis.zmq_handshake:
            yield tornado.gen.sleep(0.1)

        log.debug('sending action {}'.format(action_name))
        analysis.zmq_send({'signal': action_name, 'load': message})

        if action_name == 'disconnected':
            # Give kernel time to process disconnected message.
            yield tornado.gen.sleep(0.1)
            analysis.on_disconnected()

Response: Executes an action in the analysis with the given message.


Instruction:
Given the following Python 3 function, write the documentation
def check_git_unchanged(filename, yes=False):
    if check_staged(filename):
        s = 'There are staged changes in {}, overwrite? [y/n] '.format(filename)
        if yes or input(s) in ('y', 'yes'):
            return
        else:
            raise RuntimeError('There are staged changes in '
                               '{}, aborting.'.format(filename))
    if check_unstaged(filename):
        s = 'There are unstaged changes in {}, overwrite? [y/n] '.format(filename)
        if yes or input(s) in ('y', 'yes'):
            return
        else:
            raise RuntimeError('There are unstaged changes in '
                               '{}, aborting.'.format(filename))

Response: Check git to avoid overwriting user changes.


Instruction:
Can you generate the documentation for the following Python 3 function
def check_staged(filename=None):
    retcode, _, stdout = git['diff-index', '--quiet', '--cached', 'HEAD',
                             filename].run(retcode=None)
    if retcode == 1:
        return True
    elif retcode == 0:
        return False
    else:
        raise RuntimeError(stdout)

Response: Check if there are changes to be committed in the index.


Instruction:
Write a Python 3 function for
executing an action in the analysis with the given message.

Response:
def run_process(analysis, action_name, message='__nomessagetoken__'):
        """Executes an action in the analysis with the given message.

        It also handles the start and stop signals in the case that message
        is a `dict` with a key ``__process_id``.

        :param str action_name: Name of the action to trigger.
        :param message: Message.
        :param callback:
            A callback function when done (e.g.
            :meth:`~tornado.testing.AsyncTestCase.stop` in tests).
        :rtype: tornado.concurrent.Future
        """

        if analysis is None:
            return

        # detect process_id
        process_id = None
        if isinstance(message, dict) and '__process_id' in message:
            process_id = message['__process_id']
            del message['__process_id']

        if process_id:
            yield analysis.emit('__process',
                                {'id': process_id, 'status': 'start'})

        fns = [
            functools.partial(handler, analysis)
            for handler in (analysis._action_handlers.get(action_name, []) +
                            analysis._action_handlers.get('*', []))
        ]
        if fns:
            args, kwargs = [], {}

            # Check whether this is a list (positional arguments)
            # or a dictionary (keyword arguments).
            if isinstance(message, list):
                args = message
            elif isinstance(message, dict):
                kwargs = message
            elif message == '__nomessagetoken__':
                pass
            else:
                args = [message]

            for fn in fns:
                log.debug('calling {}'.format(fn))
                try:
                    yield tornado.gen.maybe_future(fn(*args, **kwargs))
                except Exception as e:
                    yield analysis.emit('error', 'an Exception occured')
                    raise e
        else:
            yield analysis.emit('warn',
                                'no handler for {}'.format(action_name))

        if process_id:
            yield analysis.emit('__process',
                                {'id': process_id, 'status': 'end'})


Instruction:
Can you create a Python 3 function that
downloads url to dest

Response:
def dl_file(url, dest, chunk_size=6553):
    """Download `url` to `dest`"""
    import urllib3
    http = urllib3.PoolManager()
    r = http.request('GET', url, preload_content=False)
    with dest.open('wb') as out:
        while True:
            data = r.read(chunk_size)
            if data is None or len(data) == 0:
                break
            out.write(data)
    r.release_conn()


Instruction:
Implement a function in Python 3 to
extract an lzma file and return the temporary file name

Response:
def extract_lzma(path):
    """Extract an lzma file and return the temporary file name"""
    tlfile = pathlib.Path(path)
    # open lzma file
    with tlfile.open("rb") as td:
        data = lzma.decompress(td.read())
    # write temporary tar file
    fd, tmpname = tempfile.mkstemp(prefix="odt_ex_", suffix=".tar")
    with open(fd, "wb") as fo:
        fo.write(data)
    return tmpname


Instruction:
Can you implement a function in Python 3 that
downloads a file from the ODTbrain GitHub repository.

Response:
def get_file(fname, datapath=datapath):
    """Return path of an example data file

    Return the full path to an example data file name.
    If the file does not exist in the `datapath` directory,
    tries to download it from the ODTbrain GitHub repository.
    """
    # download location
    datapath = pathlib.Path(datapath)
    datapath.mkdir(parents=True, exist_ok=True)

    dlfile = datapath / fname
    if not dlfile.exists():
        print("Attempting to download file {} from {} to {}.".
              format(fname, webloc, datapath))
        try:
            dl_file(url=webloc+fname, dest=dlfile)
        except BaseException:
            warnings.warn("Download failed: {}".format(fname))
            raise
    return dlfile


Instruction:
Implement a function in Python 3 to
load example sinogram data from a. tar. lzma file

Response:
def load_tar_lzma_data(tlfile):
    """Load example sinogram data from a .tar.lzma file"""
    tmpname = extract_lzma(tlfile)

    # open tar file
    fields_real = []
    fields_imag = []
    phantom = []
    parms = {}

    with tarfile.open(tmpname, "r") as t:
        members = t.getmembers()
        members.sort(key=lambda x: x.name)

        for m in members:
            n = m.name
            f = t.extractfile(m)
            if n.startswith("fdtd_info"):
                for ln in f.readlines():
                    ln = ln.decode()
                    if ln.count("=") == 1:
                        key, val = ln.split("=")
                        parms[key.strip()] = float(val.strip())
            elif n.startswith("phantom"):
                phantom.append(np.loadtxt(f))
            elif n.startswith("field"):
                if n.endswith("imag.txt"):
                    fields_imag.append(np.loadtxt(f))
                elif n.endswith("real.txt"):
                    fields_real.append(np.loadtxt(f))

    try:
        os.remove(tmpname)
    except OSError:
        pass

    phantom = np.array(phantom)
    sino = np.array(fields_real) + 1j * np.array(fields_imag)
    angles = np.linspace(0, 2 * np.pi, sino.shape[0], endpoint=False)

    return sino, angles, phantom, parms


Instruction:
Can you tell what is the following Python 3 function doing
def load_zip_data(zipname, f_sino_real, f_sino_imag,
                  f_angles=None, f_phantom=None, f_info=None):
    """Load example sinogram data from a .zip file"""
    ret = []
    with zipfile.ZipFile(str(zipname)) as arc:
        sino_real = np.loadtxt(arc.open(f_sino_real))
        sino_imag = np.loadtxt(arc.open(f_sino_imag))
        sino = sino_real + 1j * sino_imag
        ret.append(sino)
        if f_angles:
            angles = np.loadtxt(arc.open(f_angles))
            ret.append(angles)
        if f_phantom:
            phantom = np.loadtxt(arc.open(f_phantom))
            ret.append(phantom)
        if f_info:
            with arc.open(f_info) as info:
                cfg = {}
                for li in info.readlines():
                    li = li.decode()
                    if li.count("=") == 1:
                        key, val = li.split("=")
                        cfg[key.strip()] = float(val.strip())
            ret.append(cfg)
    return ret

Response: Load example sinogram data from a. zip file


Instruction:
Explain what the following Python 3 code does
def fourier_map_2d(uSin, angles, res, nm, lD=0, semi_coverage=False,
                   coords=None, count=None, max_count=None, verbose=0):
    r"""2D Fourier mapping with the Fourier diffraction theorem

    Two-dimensional diffraction tomography reconstruction
    algorithm for scattering of a plane wave
    :math:`u_0(\mathbf{r}) = u_0(x,z)`
    by a dielectric object with refractive index
    :math:`n(x,z)`.

    This function implements the solution by interpolation in
    Fourier space.

    Parameters
    ----------
    uSin: (A,N) ndarray
        Two-dimensional sinogram of line recordings
        :math:`u_{\mathrm{B}, \phi_j}(x_\mathrm{D})`
        divided by the incident plane wave :math:`u_0(l_\mathrm{D})`
        measured at the detector.
    angles: (A,) ndarray
        Angular positions :math:`\phi_j` of `uSin` in radians.
    res: float
        Vacuum wavelength of the light :math:`\lambda` in pixels.
    nm: float
        Refractive index of the surrounding medium :math:`n_\mathrm{m}`.
    lD: float
        Distance from center of rotation to detector plane
        :math:`l_\mathrm{D}` in pixels.
    semi_coverage: bool
        If set to `True`, it is assumed that the sinogram does not
        necessarily cover the full angular range from 0 to 2π, but an
        equidistant coverage over 2π can be achieved by inferring point
        (anti)symmetry of the (imaginary) real parts of the Fourier
        transform of f. Valid for any set of angles {X} that result in
        a 2π coverage with the union set {X}U{X+π}.
    coords: None [(2,M) ndarray]
        Computes only the output image at these coordinates. This
        keyword is reserved for future versions and is not
        implemented yet.
    count, max_count: multiprocessing.Value or `None`
        Can be used to monitor the progress of the algorithm.
        Initially, the value of `max_count.value` is incremented
        by the total number of steps. At each step, the value
        of `count.value` is incremented.
    verbose: int
        Increment to increase verbosity.

    Returns
    -------
    f: ndarray of shape (N,N), complex if `onlyreal` is `False`
        Reconstructed object function :math:`f(\mathbf{r})` as defined
        by the Helmholtz equation.
        :math:`f(x,z) =
        k_m^2 \left(\left(\frac{n(x,z)}{n_m}\right)^2 -1\right)`

    See Also
    --------
    backpropagate_2d: implementation by backpropagation
    odt_to_ri: conversion of the object function :math:`f(\mathbf{r})`
        to refractive index :math:`n(\mathbf{r})`

    Notes
    -----
    Do not use the parameter `lD` in combination with the Rytov
    approximation - the propagation is not correctly described.
    Instead, numerically refocus the sinogram prior to converting
    it to Rytov data (using e.g. :func:`odtbrain.sinogram_as_rytov`)
    with a numerical focusing algorithm (available in the Python
    package :py:mod:`nrefocus`).

    The interpolation in Fourier space (which is done with
    :func:`scipy.interpolate.griddata`) may be unstable and lead to
    artifacts if the data to interpolate contains sharp spikes. This
    issue is not handled at all by this method (in fact, a test has
    been removed in version 0.2.6 because ``griddata`` gave different
    results on Windows and Linux).
    """
    ##
    ##
    # TODO:
    # - zero-padding as for backpropagate_2D - However this is not
    # necessary as Fourier interpolation is not parallelizable with
    # multiprocessing and thus unattractive. Could be interesting for
    # specific environments without the Python GIL.
    # - Deal with oversampled data. Maybe issue a warning.
    ##
    ##
    A = angles.shape[0]
    if max_count is not None:
        max_count.value += 4
    # Check input data
    assert len(uSin.shape) == 2, "Input data `uSin` must have shape (A,N)!"
    assert len(uSin) == A, "`len(angles)` must be  equal to `len(uSin)`!"

    if coords is not None:
        raise NotImplementedError("Output coordinates cannot yet be set"
                                  + "for the 2D backrpopagation algorithm.")
    # Cut-Off frequency
    # km [1/px]
    km = (2 * np.pi * nm) / res

    # Fourier transform of all uB's
    # In the script we used the unitary angular frequency (uaf) Fourier
    # Transform. The discrete Fourier transform is equivalent to the
    # unitary ordinary frequency (uof) Fourier transform.
    #
    # uof: f₁(ξ) = int f(x) exp(-2πi xξ)
    #
    # uaf: f₃(ω) = (2π)^(-n/2) int f(x) exp(-i ωx)
    #
    # f₁(ω/(2π)) = (2π)^(n/2) f₃(ω)
    # ω = 2πξ
    #
    # Our Backpropagation Formula is with uaf convention of the Form
    #
    # F(k) = 1/sqrt(2π) U(kD)
    #
    # If we convert now to uof convention, we get
    #
    # F(k) = U(kD)
    #
    # This means that if we divide the Fourier transform of the input
    # data by sqrt(2π) to convert f₃(ω) to f₁(ω/(2π)), the resulting
    # value for F is off by a factor of 2π.
    #
    # Instead, we can just multiply *UB* by sqrt(2π) and calculate
    # everything in uof.
    # UB =  np.fft.fft(np.fft.ifftshift(uSin, axes=-1))/np.sqrt(2*np.pi)
    #
    #
    # Furthermore, we define
    # a wave propagating to the right as:
    #
    #    u0(x) = exp(ikx)
    #
    # However, in physics usually we use the other sign convention:
    #
    #    u0(x) = exp(-ikx)
    #
    # In order to be consistent with programs like Meep or our
    # scattering script for a dielectric cylinder, we want to use the
    # latter sign convention.
    # This is not a big problem. We only need to multiply the imaginary
    # part of the scattered wave by -1.

    UB = np.fft.fft(np.fft.ifftshift(uSin, axes=-1)) * np.sqrt(2 * np.pi)

    # Corresponding sample frequencies
    fx = np.fft.fftfreq(len(uSin[0]))  # 1D array

    # kx is a 1D array.
    kx = 2 * np.pi * fx

    if count is not None:
        count.value += 1

    # Undersampling/oversampling?
    # Determine if the resolution of the image is too low by looking
    # at the maximum value for kx. This is no comparison between
    # Nyquist and Rayleigh frequency.
    if verbose and np.max(kx**2) <= km**2:
        # Detector is not set up properly. Higher resolution
        # can be achieved.
        print("......Measurement data is undersampled.")
    else:
        print("......Measurement data is oversampled.")
        # raise NotImplementedError("Oversampled data not yet supported."+
        #                   " Please rescale xD-axis of the input data.")
        # DEAL WITH OVERSAMPLED DATA?
        # lenk = len(kx)
        # kx = np.fft.ifftshift(np.linspace(-np.sqrt(km),
        #                                   np.sqrt(km),
        #                                   len(fx), endpoint=False))

    #
    # F(kD-kₘs₀) = - i kₘ sqrt(2/π) / a₀ * M exp(-i kₘ M lD) * UB(kD)
    # kₘM = sqrt( kₘ² - kx² )
    # s₀  = ( -sin(ϕ₀), cos(ϕ₀) )
    #
    # We create the 2D interpolation object F
    #   - We compute the real coordinates (krx,kry) = kD-kₘs₀
    #   - We set as grid points the right side of the equation
    #
    # The interpolated griddata may go up to sqrt(2)*kₘ for kx and ky.

    kx = kx.reshape(1, -1)
    # a0 should have same shape as kx and UB
    # a0 = np.atleast_1d(a0)
    # a0 = a0.reshape(1,-1)

    filter_klp = (kx**2 < km**2)
    M = 1. / km * np.sqrt(km**2 - kx**2)
    # Fsin =  -1j * km * np.sqrt(2/np.pi) / a0 * M * np.exp(-1j*km*M*lD)
    # new in version 0.1.4:
    # We multiply by the factor (M-1) instead of just (M)
    # to take into account that we have a scattered
    # wave that is normalized by u0.
    Fsin = -1j * km * np.sqrt(2 / np.pi) * M * np.exp(-1j * km * (M-1) * lD)

    # UB has same shape (len(angles), len(kx))
    Fsin = Fsin * UB * filter_klp

    ang = angles.reshape(-1, 1)

    if semi_coverage:
        Fsin = np.vstack((Fsin, np.conj(Fsin)))
        ang = np.vstack((ang, ang + np.pi))

    if count is not None:
        count.value += 1

    # Compute kxl and kyl (in rotated system ϕ₀)
    kxl = kx
    kyl = np.sqrt((km**2 - kx**2) * filter_klp) - km
    # rotate kxl and kyl to where they belong
    krx = np.cos(ang) * kxl + np.sin(ang) * kyl
    kry = - np.sin(ang) * kxl + np.cos(ang) * kyl

    Xf = krx.flatten()
    Yf = kry.flatten()
    Zf = Fsin.flatten()

    # DEBUG: plot kry vs krx
    # from matplotlib import pylab as plt
    # plt.figure()
    # for i in range(len(krx)):
    #    plt.plot(krx[i],kry[i],"x")
    # plt.axes().set_aspect('equal')
    # plt.show()

    # interpolation on grid with same resolution as input data
    kintp = np.fft.fftshift(kx.reshape(-1))

    Fcomp = intp.griddata((Xf, Yf), Zf, (kintp[None, :], kintp[:, None]))

    if count is not None:
        count.value += 1

    # removed nans
    Fcomp[np.where(np.isnan(Fcomp))] = 0

    # Filter data
    kinx, kiny = np.meshgrid(np.fft.fftshift(kx), np.fft.fftshift(kx))
    Fcomp[np.where((kinx**2 + kiny**2) > np.sqrt(2) * km)] = 0
    # Fcomp[np.where(kinx**2+kiny**2<km)] = 0

    # Fcomp is centered at K = 0 due to the way we chose kintp/coords
    f = np.fft.fftshift(np.fft.ifft2(np.fft.ifftshift(Fcomp)))

    if count is not None:
        count.value += 1

    return f[::-1]

Response: r Returns a 2D Fourier mapping of theorem at the given sinograms and angle res.


Instruction:
Here you have a function in Python 3, explain what it does
def transform_to(ext):

    def decor(f):
        @functools.wraps(f)
        def wrapper(*args, **kwargs):
            out_file = kwargs.get("out_file", None)
            if not out_file:
                in_path = kwargs.get("in_file", args[0])
                out_dir = kwargs.get("out_dir", os.path.dirname(in_path))
                safe_mkdir(out_dir)
                out_name = replace_suffix(os.path.basename(in_path), ext)
                out_file = os.path.join(out_dir, out_name)
            kwargs["out_file"] = out_file
            if not file_exists(out_file):
                out_file = f(*args, **kwargs)
            return out_file
        return wrapper
    return decor

Response: Decorator to create an output filename from an input file with the specified extension.


Instruction:
Here you have a function in Python 3, explain what it does
def chdir(new_dir):
    cur_dir = os.getcwd()
    safe_mkdir(new_dir)
    os.chdir(new_dir)
    try:
        yield
    finally:
        os.chdir(cur_dir)

Response: Context manager to temporarily change to a new directory.


Instruction:
Can you create a Python 3 function that
checks if a file exists is non - empty and is more recent than cmp_fname.

Response:
def file_uptodate(fname, cmp_fname):
    """Check if a file exists, is non-empty and is more recent than cmp_fname.
    """
    try:
        return (file_exists(fname) and file_exists(cmp_fname) and
                getmtime(fname) >= getmtime(cmp_fname))
    except OSError:
        return False


Instruction:
Write a Python 3 script to
overwrite a file in place with a short message to save disk.

Response:
def save_diskspace(fname, reason, config):
    """Overwrite a file in place with a short message to save disk.

    This keeps files as a sanity check on processes working, but saves
    disk by replacing them with a short message.
    """
    if config["algorithm"].get("save_diskspace", False):
        with open(fname, "w") as out_handle:
            out_handle.write("File removed to save disk space: %s" % reason)


Instruction:
Implement a Python 3 function for
creating relative symlinks and handle associated biological index files.

Response:
def symlink_plus(orig, new):
    """Create relative symlinks and handle associated biological index files.
    """
    for ext in ["", ".idx", ".gbi", ".tbi", ".bai"]:
        if os.path.exists(orig + ext) and not os.path.lexists(new + ext):
            with chdir(os.path.dirname(new)):
                os.symlink(os.path.relpath(orig + ext), os.path.basename(new + ext))
    orig_noext = splitext_plus(orig)[0]
    new_noext = splitext_plus(new)[0]
    for sub_ext in [".bai"]:
        if os.path.exists(orig_noext + sub_ext) and not os.path.lexists(new_noext + sub_ext):
            with chdir(os.path.dirname(new_noext)):
                os.symlink(os.path.relpath(orig_noext + sub_ext), os.path.basename(new_noext + sub_ext))


Instruction:
Implement a Python 3 function for
using a predicate to partition entries into false entries and true entries

Response:
def partition(pred, iterable):
    'Use a predicate to partition entries into false entries and true entries'
    # partition(is_odd, range(10)) --> 0 2 4 6 8   and  1 3 5 7 9
    t1, t2 = itertools.tee(iterable)
    try:
        return itertools.ifilterfalse(pred, t1), itertools.ifilter(pred, t2)
    except:
        return itertools.filterfalse(pred, t1), filter(pred, t2)