Instruction:
Here you have a function in Python 3, explain what it does
def _get_candidate_swap(resources, location,
                        l2v, vertices_resources, fixed_vertices, machine):
    """Given a chip location, select a set of vertices which would have to be
    moved elsewhere to accommodate the arrival of the specified set of
    resources.

    Parameters
    ----------
    resources : {resource: value, ...}
        The amount of resources which are required at the specified location.
    location : (x, y)
        The coordinates of the chip where the resources are sought.
    l2v : {(x, y): [vertex, ...], ...}
    vertices_resources : {vertex: {resource: value, ...}, ...}
    fixed_vertices : {vertex, ...}
    machine : :py:class:`rig.place_and_route.Machine`

    Returns
    -------
    [Vertex, ...] or None
        If a (possibly empty) list, gives the set of vertices which should be
        removed from the specified location to make room.

        If None, the situation is impossible.
    """
    # The resources already available at the given location
    chip_resources = machine[location]

    # The set of vertices at that location
    vertices = l2v[location]

    # The set of vertices to be moved from the location to free up the
    # specified amount of resources
    to_move = []

    # While there's not enough free resource, remove an arbitrary (movable)
    # vertex from the chip.
    i = 0
    while overallocated(subtract_resources(chip_resources, resources)):
        if i >= len(vertices):
            # Run out of vertices to remove from this chip, thus the situation
            # must be impossible.
            return None
        elif vertices[i] in fixed_vertices:
            # Can't move fixed vertices, just skip them.
            i += 1
            continue
        else:
            # Work out the cost change when we remove the specified vertex
            vertex = vertices[i]
            chip_resources = add_resources(chip_resources,
                                           vertices_resources[vertex])
            to_move.append(vertex)
            i += 1

    return to_move

Response: Given a set of vertices and a set of resources and a machine return a list of vertices that should be moved elsewhere to accommodate the arrival of the specified set of resources.


Instruction:
Write a Python 3 script to
swap the positions of two sets of vertices in a single tree.

Response:
def _swap(vas, vas_location, vbs, vbs_location, l2v, vertices_resources,
          placements, machine):
    """Swap the positions of two sets of vertices.

    Parameters
    ----------
    vas : [vertex, ...]
        A set of vertices currently at vas_location.
    vas_location : (x, y)
    vbs : [vertex, ...]
        A set of vertices currently at vbs_location.
    vbs_location : (x, y)
    l2v : {(x, y): [vertex, ...], ...}
    vertices_resources : {vertex: {resource: value, ...}, ...}
    placements : {vertex: (x, y), ...}
    machine : :py:class:`rig.place_and_route.Machine`
    """
    # Get the lists of vertices at either location
    vas_location2v = l2v[vas_location]
    vbs_location2v = l2v[vbs_location]

    # Get the resource availability at either location
    vas_resources = machine[vas_location]
    vbs_resources = machine[vbs_location]

    # Move all the vertices in vas into vbs.
    for va in vas:
        # Update the placements
        placements[va] = vbs_location

        # Update the location-to-vertex lookup
        vas_location2v.remove(va)
        vbs_location2v.append(va)

        # Update the resource consumption after the move
        resources = vertices_resources[va]
        vas_resources = add_resources(vas_resources, resources)
        vbs_resources = subtract_resources(vbs_resources, resources)

    for vb in vbs:
        # Update the placements
        placements[vb] = vas_location

        # Update the location-to-vertex lookup
        vbs_location2v.remove(vb)
        vas_location2v.append(vb)

        # Update the resource consumption after the move
        resources = vertices_resources[vb]
        vas_resources = subtract_resources(vas_resources, resources)
        vbs_resources = add_resources(vbs_resources, resources)

    # Update the resources in the machine
    machine[vas_location] = vas_resources
    machine[vbs_location] = vbs_resources


Instruction:
How would you implement a function in Python 3 that
attempts a single swap operation: the kernel of the Simulated Annealing algorithm. Parameters ---------- vertices : [vertex, ...] The set of *movable* vertices. d_limit : int The maximum distance over-which swaps are allowed. temperature : float > 0.0 or None The temperature (i.e. likelihood of accepting a non-advantageous swap). Higher temperatures mean higher chances of accepting a swap. placements : {vertex: (x, y), ...} The positions of all vertices, will be updated if a swap is made. l2v : {(x, y): [vertex, ...], ...} Lookup from chip to vertices, will be updated if a swap is made. v2n : {vertex: [:py:class:`rig.netlist.Net`, ...], ...} Lookup from vertex to all nets that vertex is in. vertices_resources : {vertex: {resource: value, ...}, ...} fixed_vertices : {vertex, ...} The set of vertices which must not be moved. machine : :py:class:`rig.place_and_route.Machine` Describes the state of the machine including the resources actually available on each chip given the current placements. Updated if a swap is made. has_wrap_around_links : bool Should the placements attempt to make use of wrap-around links? random : :py:class:`random.Random` The random number generator to use. Returns ------- (swapped, delta) swapped is a boolean indicating if a swap was made. delta is a float indicating the change in cost resulting from the swap (or 0.0 when no swap is made).

Response:
def _step(vertices, d_limit, temperature, placements, l2v, v2n,
          vertices_resources, fixed_vertices, machine, has_wrap_around_links,
          random):
    """Attempt a single swap operation: the kernel of the Simulated Annealing
    algorithm.

    Parameters
    ----------
    vertices : [vertex, ...]
        The set of *movable* vertices.
    d_limit : int
        The maximum distance over-which swaps are allowed.
    temperature : float > 0.0 or None
        The temperature (i.e. likelihood of accepting a non-advantageous swap).
        Higher temperatures mean higher chances of accepting a swap.
    placements : {vertex: (x, y), ...}
        The positions of all vertices, will be updated if a swap is made.
    l2v : {(x, y): [vertex, ...], ...}
        Lookup from chip to vertices, will be updated if a swap is made.
    v2n : {vertex: [:py:class:`rig.netlist.Net`, ...], ...}
        Lookup from vertex to all nets that vertex is in.
    vertices_resources : {vertex: {resource: value, ...}, ...}
    fixed_vertices : {vertex, ...}
        The set of vertices which must not be moved.
    machine : :py:class:`rig.place_and_route.Machine`
        Describes the state of the machine including the resources actually
        available on each chip given the current placements. Updated if a swap
        is made.
    has_wrap_around_links : bool
        Should the placements attempt to make use of wrap-around links?
    random : :py:class:`random.Random`
        The random number generator to use.

    Returns
    -------
    (swapped, delta)
        swapped is a boolean indicating if a swap was made.

        delta is a float indicating the change in cost resulting from the swap
        (or 0.0 when no swap is made).
    """
    # Special case: If the machine is a singleton, no swaps can be made so just
    # terminate.
    if machine.width == 1 and machine.height == 1:
        return (False, 0.0)

    # Select a vertex to swap at random
    src_vertex = random.choice(vertices)

    # Select a random (nearby) location to swap the vertex with. Note: this is
    # guaranteed to be different from the selected vertex, otherwise the swap
    # cannot change the cost of the placements.
    # XXX: Does not consider hexagonal properties of the system!
    src_location = placements[src_vertex]
    dst_location = src_location
    while dst_location == src_location:
        if has_wrap_around_links:
            dst_location = tuple(random.randint(v - d_limit,
                                                v + d_limit) % limit
                                 for v, limit
                                 in [(src_location[0], machine.width),
                                     (src_location[1], machine.height)])
        else:
            dst_location = tuple(random.randint(max(v - d_limit, 0),
                                                min(v + d_limit, limit-1))
                                 for v, limit
                                 in [(src_location[0], machine.width),
                                     (src_location[1], machine.height)])

    # If we've inadvertently selected a dead chip to swap to, abort the swap.
    if dst_location not in machine:
        return (False, 0.0)

    # Find out which vertices (if any) must be swapped out of the destination
    # to make room for the vertex we're moving.
    src_resources = vertices_resources[src_vertex]
    dst_vertices = _get_candidate_swap(src_resources, dst_location,
                                       l2v, vertices_resources,
                                       fixed_vertices, machine)

    # The destination simply isn't big enough (no matter how many vertices at
    # the destination are moved), abort the swap.
    if dst_vertices is None:
        return (False, 0.0)

    # Make sure that any vertices moved out of the destination will fit in the
    # space left in the source location. If there isn't enough space, abort the
    # swap.
    resources = machine[src_location]
    resources = add_resources(resources, src_resources)
    for dst_vertex in dst_vertices:
        resources = subtract_resources(resources,
                                       vertices_resources[dst_vertex])
    if overallocated(resources):
        return (False, 0.0)

    # Work out the cost of the nets involved *before* swapping
    cost_before = _vertex_net_cost(src_vertex, v2n, placements,
                                   has_wrap_around_links, machine)
    for dst_vertex in dst_vertices:
        cost_before += _vertex_net_cost(dst_vertex, v2n, placements,
                                        has_wrap_around_links, machine)

    # Swap the vertices
    _swap([src_vertex], src_location,
          dst_vertices, dst_location,
          l2v, vertices_resources, placements, machine)

    # Work out the new cost
    cost_after = _vertex_net_cost(src_vertex, v2n, placements,
                                  has_wrap_around_links, machine)
    for dst_vertex in dst_vertices:
        cost_after += _vertex_net_cost(dst_vertex, v2n, placements,
                                       has_wrap_around_links, machine)

    # If the swap was beneficial, keep it, otherwise keep it with a probability
    # related to just how bad the cost change is is and the temperature.
    delta = cost_after - cost_before
    if delta <= 0.0 or random.random() < math.exp(-delta/temperature):
        # Keep the swap!
        return (True, delta)
    else:
        # Revert the swap
        _swap([src_vertex], dst_location,
              dst_vertices, src_location,
              l2v, vertices_resources, placements, machine)
        return (False, 0.0)


Instruction:
Make a summary of the following Python 3 code
def _reset(self):
        self._subscriber_thread = None
        self._must_stop = threading.Event()
        self._number_periodic_tasks = 0

Response: Reset the internal state of the broker.


Instruction:
Make a summary of the following Python 3 code
def _load_script(self, filename: str) -> Script:
        with open(path.join(here, 'redis_scripts', filename), mode='rb') as f:
            script_data = f.read()
        rv = self._r.register_script(script_data)
        if script_data.startswith(b'-- idempotency protected script'):
            self._idempotency_protected_scripts.append(rv)
        return rv

Response: Load a Lua script file and return a Script object.


Instruction:
How would you explain what the following Python 3 function does
def enqueue_jobs(self, jobs: Iterable[Job]):
        jobs_to_queue = list()
        for job in jobs:
            if job.should_start:
                job.status = JobStatus.QUEUED
            else:
                job.status = JobStatus.WAITING
            jobs_to_queue.append(job.serialize())

        if jobs_to_queue:
            self._run_script(
                self._enqueue_job,
                self._to_namespaced(NOTIFICATIONS_KEY),
                self._to_namespaced(RUNNING_JOBS_KEY.format(self._id)),
                self.namespace,
                self._to_namespaced(FUTURE_JOBS_KEY),
                *jobs_to_queue
            )

Response: Enqueue a batch of jobs.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_jobs_from_queue(self, queue: str, max_jobs: int) -> List[Job]:
        jobs_json_string = self._run_script(
            self._get_jobs_from_queue,
            self._to_namespaced(queue),
            self._to_namespaced(RUNNING_JOBS_KEY.format(self._id)),
            JobStatus.RUNNING.value,
            max_jobs
        )

        jobs = json.loads(jobs_json_string.decode())
        jobs = [Job.deserialize(job) for job in jobs]

        return jobs

Response: Get jobs from a queue.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def register_periodic_tasks(self, tasks: Iterable[Task]):
        tasks = [task.serialize() for task in tasks]
        self._number_periodic_tasks = len(tasks)
        self._run_script(
            self._register_periodic_tasks,
            math.ceil(datetime.now(timezone.utc).timestamp()),
            self._to_namespaced(PERIODIC_TASKS_HASH_KEY),
            self._to_namespaced(PERIODIC_TASKS_QUEUE_KEY),
            *tasks
        )

Response: Register tasks that need to be scheduled periodically.


Instruction:
Write a Python 3 script for
getting the next periodic task schedule.

Response:
def inspect_periodic_tasks(self) -> List[Tuple[int, str]]:
        """Get the next periodic task schedule.

        Used only for debugging and during tests.
        """
        rv = self._r.zrangebyscore(
            self._to_namespaced(PERIODIC_TASKS_QUEUE_KEY),
            '-inf', '+inf', withscores=True
        )
        return [(int(r[1]), r[0].decode()) for r in rv]


Instruction:
Create a Python 3 function for
giving the amount of seconds before the next periodic task is due.

Response:
def next_future_periodic_delta(self) -> Optional[float]:
        """Give the amount of seconds before the next periodic task is due."""
        rv = self._r.zrangebyscore(
            self._to_namespaced(PERIODIC_TASKS_QUEUE_KEY),
            '-inf', '+inf', start=0, num=1, withscores=True,
            score_cast_func=int
        )
        if not rv:
            return None

        now = datetime.now(timezone.utc).timestamp()
        next_event_time = rv[0][1]
        if next_event_time < now:
            return 0

        return next_event_time - now


Instruction:
Explain what the following Python 3 code does
def _provision_network(self, port_id, net_uuid, network_type,
                           physical_network, segmentation_id):
        """Provision the network with the received information."""
        LOG.info("Provisioning network %s", net_uuid)

        vswitch_name = self._get_vswitch_name(network_type, physical_network)
        vswitch_map = {
            'network_type': network_type,
            'vswitch_name': vswitch_name,
            'ports': [],
            'vlan_id': segmentation_id}
        self._network_vswitch_map[net_uuid] = vswitch_map

Response: Provision the network with the received information.


Instruction:
Given the following Python 3 function, write the documentation
def _port_bound(self, port_id, network_id, network_type, physical_network,
                    segmentation_id, port_security_enabled, set_port_sriov):
        """Bind the port to the recived network."""
        super(HNVAgent, self)._port_bound(port_id, network_id, network_type,
                                          physical_network, segmentation_id,
                                          port_security_enabled,
                                          set_port_sriov)
        LOG.debug("Getting the profile id for the current port.")
        profile_id = self._neutron_client.get_port_profile_id(port_id)

        LOG.debug("Trying to set port profile id %r for the current port %r.",
                  profile_id, port_id)
        self._utils.set_vswitch_port_profile_id(
            switch_port_name=port_id,
            profile_id=profile_id,
            profile_data=h_const.PROFILE_DATA,
            profile_name=h_const.PROFILE_NAME,
            net_cfg_instance_id=h_const.NET_CFG_INSTANCE_ID,
            cdn_label_id=h_const.CDN_LABEL_ID,
            cdn_label_string=h_const.CDN_LABEL_STRING,
            vendor_id=h_const.VENDOR_ID,
            vendor_name=h_const.VENDOR_NAME)

Response: Bind the port to the recived network.


Instruction:
Can you write a function in Python 3 where it
transmits a SCP packet to the SpiNNaker machine and block until acknowledgement is received.

Response:
def send_scp(self, buffer_size, x, y, p, cmd, arg1=0, arg2=0, arg3=0,
                 data=b'', expected_args=3, timeout=0.0):
        """Transmit a packet to the SpiNNaker machine and block until an
        acknowledgement is received.

        Parameters
        ----------
        buffer_size : int
            Number of bytes held in an SCP buffer by SARK, determines how many
            bytes will be expected in a socket.
        x : int
        y : int
        p : int
        cmd : int
        arg1 : int
        arg2 : int
        arg3 : int
        data : bytestring
        expected_args : int
            The number of arguments (0-3) that are expected in the returned
            packet.
        timeout : float
            Additional timeout in seconds to wait for a reply on top of the
            default specified upon instantiation.

        Returns
        -------
        :py:class:`~rig.machine_control.packets.SCPPacket`
            The packet that was received in acknowledgement of the transmitted
            packet.
        """
        # This is implemented as a single burst packet sent using the bursty
        # interface.  This significantly reduces code duplication.
        # Construct a callable to retain the returned packet for us
        class Callback(object):
            def __init__(self):
                self.packet = None

            def __call__(self, packet):
                self.packet = SCPPacket.from_bytestring(
                    packet, n_args=expected_args
                )

        # Create the packet to send
        callback = Callback()
        packets = [
            scpcall(x, y, p, cmd, arg1, arg2, arg3, data, callback, timeout)
        ]

        # Send the burst
        self.send_scp_burst(buffer_size, 1, packets)

        # Return the received packet
        assert callback.packet is not None
        return callback.packet


Instruction:
How would you code a function in Python 3 to
send a burst of SCP packets and call a callback for each returned ArcGIS Packet.

Response:
def send_scp_burst(self, buffer_size, window_size,
                       parameters_and_callbacks):
        """Send a burst of SCP packets and call a callback for each returned
        packet.

        Parameters
        ----------
        buffer_size : int
            Number of bytes held in an SCP buffer by SARK, determines how many
            bytes will be expected in a socket.
        window_size : int
            Number of packets which can be awaiting replies from the SpiNNaker
            board.
        parameters_and_callbacks: iterable of :py:class:`.scpcall`
            Iterable of :py:class:`.scpcall` elements.  These elements can
            specify a callback which will be called with the returned packet.
        """
        parameters_and_callbacks = iter(parameters_and_callbacks)

        self.sock.setblocking(False)

        # Calculate the receive length, this should be the smallest power of
        # two greater than the required size
        max_length = buffer_size + consts.SDP_HEADER_LENGTH
        receive_length = int(2**math.ceil(math.log(max_length, 2)))

        class TransmittedPacket(object):
            """A packet which has been transmitted and still awaits a response.
            """
            __slots__ = ["callback", "packet", "bytestring", "n_tries",
                         "timeout", "timeout_time"]

            def __init__(self, callback, packet, timeout):
                self.callback = callback
                self.packet = packet
                self.bytestring = packet.bytestring
                self.n_tries = 1
                self.timeout = timeout
                self.timeout_time = time.time() + self.timeout

        queued_packets = True
        outstanding_packets = {}
        outstanding_callbacks = collections.deque()

        # While there are packets in the queue or packets for which we are
        # still awaiting returns then continue to loop.
        while queued_packets or outstanding_packets or outstanding_callbacks:
            # If there are fewer outstanding packets than the window can take
            # and we still might have packets left to send then transmit a
            # packet and add it to the list of outstanding packets.
            while len(outstanding_packets) < window_size and queued_packets:
                try:
                    args = next(parameters_and_callbacks)
                except StopIteration:
                    queued_packets = False

                if queued_packets:
                    # If we extracted a new packet to send then create a new
                    # outstanding packet and transmit it.
                    seq = next(self.seq)
                    while seq in outstanding_packets:
                        # The seq should rarely be already taken, it normally
                        # means that one packet is taking such a long time to
                        # send that the sequence has wrapped around.  It's not
                        # a problem provided that we don't reuse the number.
                        seq = next(self.seq)

                    # Construct the packet that we'll be sending
                    packet = SCPPacket(
                        reply_expected=True, tag=0xff, dest_port=0,
                        dest_cpu=args.p, src_port=7, src_cpu=31,
                        dest_x=args.x, dest_y=args.y, src_x=0, src_y=0,
                        cmd_rc=args.cmd, seq=seq,
                        arg1=args.arg1, arg2=args.arg2, arg3=args.arg3,
                        data=args.data
                    )

                    # Create a reference to this packet so that we know we're
                    # expecting a response for it and can retransmit it if
                    # necessary.
                    outstanding_packets[seq] = TransmittedPacket(
                        args.callback, packet,
                        self.default_timeout + args.timeout
                    )

                    # Actually send the packet
                    self.sock.send(outstanding_packets[seq].bytestring)

            # Call all outstanding callbacks
            while outstanding_callbacks:
                callback, packet = outstanding_callbacks.pop()
                callback(packet)

            # Listen on the socket for an acknowledgement packet, there may not
            # be one.
            if outstanding_packets:
                timeout = min((o.timeout_time for o in
                               six.itervalues(outstanding_packets)
                               )) - time.time()
            else:
                timeout = 0.0
            r, w, x = select.select([self.sock], [], [], max(timeout, 0.0))
            # Process the received packet (if there is one).
            while r:
                # Note that 'r' is never changed so this while loop will either
                # get skipped if r is empty or loop until 'break' of r is not.
                # Since we may receive multiple packets at once, it is better
                # to try and pull all out of the socket immediately rather than
                # running around the parent loop again and incuring the
                # 'select' cost.
                try:
                    ack = self.sock.recv(receive_length)
                except IOError:
                    break

                # Extract the sequence number from the bytestring, iff possible
                rc, seq = struct.unpack_from("<2H", ack,
                                             consts.SDP_HEADER_LENGTH + 2)

                # If the code is an error then we respond immediately
                if rc != consts.SCPReturnCodes.ok:
                    if rc in consts.RETRYABLE_SCP_RETURN_CODES:
                        # If the error is timeout related then treat the packet
                        # as though it timed out, just discard.  This avoids us
                        # hammering the board when it's most vulnerable.
                        pass
                    else:
                        # For all other errors, we'll just fall over
                        # immediately.
                        packet = outstanding_packets.get(seq)
                        if packet is not None:
                            packet = packet.packet
                        raise FatalReturnCodeError(rc, packet)
                else:
                    # Look up the sequence index of packet in the list of
                    # outstanding packets.  We may have already processed a
                    # response for this packet (indicating that the response
                    # was delayed and we retransmitted the initial message) in
                    # which case we can silently ignore the returned packet.
                    # XXX: There is a danger that a response was so delayed
                    # that we already reused the seq number... this is probably
                    # sufficiently unlikely that there is no problem.
                    outstanding = outstanding_packets.pop(seq, None)
                    if outstanding is not None:
                        outstanding_callbacks.appendleft((outstanding.callback,
                                                          ack))

            # Look through all the remaining outstanding packets, if any of
            # them have timed out then we retransmit them.
            current_time = time.time()
            for outstanding in six.itervalues(outstanding_packets):
                if outstanding.timeout_time < current_time:
                    # This packet has timed out, if we have sent it more than
                    # the given number of times then raise a timeout error for
                    # it.
                    if outstanding.n_tries >= self.n_tries:
                        raise TimeoutError(
                            "No response after {} attempts.".format(
                                self.n_tries),
                            outstanding.packet)

                    # Otherwise we retransmit it
                    self.sock.send(outstanding.bytestring)
                    outstanding.n_tries += 1
                    outstanding.timeout_time = (current_time +
                                                outstanding.timeout)


Instruction:
Implement a Python 3 function for
reading a bytestring from an address in memory.

Response:
def read(self, buffer_size, window_size, x, y, p, address, length_bytes):
        """Read a bytestring from an address in memory.

        ..note::
            This method is included here to maintain API compatibility with an
            `alternative implementation of SCP
            <https://github.com/project-rig/rig-scp>`_.

        Parameters
        ----------
        buffer_size : int
            Number of bytes held in an SCP buffer by SARK, determines how many
            bytes will be expected in a socket and how many bytes of data will
            be read back in each packet.
        window_size : int
        x : int
        y : int
        p : int
        address : int
            The address at which to start reading the data.
        length_bytes : int
            The number of bytes to read from memory. Large reads are
            transparently broken into multiple SCP read commands.

        Returns
        -------
        :py:class:`bytes`
            The data is read back from memory as a bytestring.
        """
        # Prepare the buffer to receive the incoming data
        data = bytearray(length_bytes)
        mem = memoryview(data)

        # Create a callback which will write the data from a packet into a
        # memoryview.
        def callback(mem, data):
            mem[:] = data[6 + consts.SDP_HEADER_LENGTH:]

        # Create a generator that will generate request packets and store data
        # until all data has been returned
        def packets(length_bytes, data):
            offset = 0
            while length_bytes > 0:
                # Get the next block of data
                block_size = min((length_bytes, buffer_size))
                read_address = address + offset
                dtype = consts.address_length_dtype[(read_address % 4,
                                                     block_size % 4)]

                # Create the call spec and yield
                yield scpcall(
                    x, y, p, consts.SCPCommands.read, read_address,
                    block_size, dtype,
                    callback=functools.partial(callback,
                                               mem[offset:offset + block_size])
                )

                # Update the number of bytes remaining and the offset
                offset += block_size
                length_bytes -= block_size

        # Run the event loop and then return the retrieved data
        self.send_scp_burst(buffer_size, window_size,
                            list(packets(length_bytes, data)))
        return bytes(data)


Instruction:
Write a Python 3 script for
writing a bytestring to an address in memory.

Response:
def write(self, buffer_size, window_size, x, y, p, address, data):
        """Write a bytestring to an address in memory.

        ..note::
            This method is included here to maintain API compatibility with an
            `alternative implementation of SCP
            <https://github.com/project-rig/rig-scp>`_.

        Parameters
        ----------
        buffer_size : int
            Number of bytes held in an SCP buffer by SARK, determines how many
            bytes will be expected in a socket and how many bytes will be
            written in each packet.
        window_size : int
        x : int
        y : int
        p : int
        address : int
            The address at which to start writing the data. Addresses are given
            within the address space of a SpiNNaker core. See the SpiNNaker
            datasheet for more information.
        data : :py:class:`bytes`
            Data to write into memory. Writes are automatically broken into a
            sequence of SCP write commands.
        """
        # While there is still data perform a write: get the block to write
        # this time around, determine the data type, perform the write and
        # increment the address
        def packets(address, data):
            end = len(data)
            pos = 0
            while pos < end:
                block = data[pos:pos + buffer_size]
                block_size = len(block)

                dtype = consts.address_length_dtype[(address % 4,
                                                     block_size % 4)]

                yield scpcall(x, y, p, consts.SCPCommands.write, address,
                              block_size, dtype, block)

                address += block_size
                pos += block_size

        # Run the event loop and then return the retrieved data
        self.send_scp_burst(buffer_size, window_size,
                            list(packets(address, data)))


Instruction:
Explain what the following Python 3 code does
def convert_msg_to_html(msg):

    msg = re.sub("\n", "<br />", msg, flags=re.MULTILINE)
    return msg

Response: Convert \ n into a <BR > for an HTML formatted message


Instruction:
Given the following Python 3 function, write the documentation
def validate(nanopub: dict, error_level: str = "WARNING") -> Tuple[str, str, str]:

    # Validation results
    v = []

    bel_version = config["bel"]["lang"]["default_bel_version"]

    # Structural checks
    try:
        if not isinstance(nanopub["nanopub"]["assertions"], list):
            msg = "Assertions must be a list/array"
            v.append(
                {
                    "level": "Error",
                    "section": "Structure",
                    "label": "Error-Structure",
                    "msg": msg,
                    "msg_html": msg,
                }
            )
    except Exception as e:
        msg = 'Missing nanopub["nanopub"]["assertions"]'
        v.append(
            {
                "level": "Error",
                "section": "Structure",
                "label": "Error-Structure",
                "msg": msg,
                "msg_html": msg,
            }
        )

    try:
        if (
            "name" in nanopub["nanopub"]["type"]
            and "version" in nanopub["nanopub"]["type"]
        ):
            pass
        if nanopub["nanopub"]["type"]["name"].upper() == "BEL":
            bel_version = nanopub["nanopub"]["type"]["version"]

    except Exception as e:
        msg = 'Missing or badly formed type - must have nanopub["nanopub"]["type"] = {"name": <name>, "version": <version}'
        v.append(
            {
                "level": "Error",
                "section": "Structure",
                "label": "Error-Structure",
                "msg": msg,
                "msg_html": msg,
            }
        )

    try:
        for key in ["uri", "database", "reference"]:
            if key in nanopub["nanopub"]["citation"]:
                break
        else:
            msg = 'nanopub["nanopub"]["citation"] must have either a uri, database or reference key.'
            v.append(
                {
                    "level": "Error",
                    "section": "Structure",
                    "label": "Error-Structure",
                    "msg": msg,
                    "msg_html": msg,
                }
            )
    except Exception as e:
        msg = 'nanopub["nanopub"] must have a "citation" key with either a uri, database or reference key.'
        v.append(
            {
                "level": "Error",
                "section": "Structure",
                "label": "Error-Structure",
                "msg": msg,
                "msg_html": msg,
            }
        )

    # Assertion checks
    if "assertions" in nanopub["nanopub"]:
        for idx, assertion in enumerate(nanopub["nanopub"]["assertions"]):
            bo = bel.lang.belobj.BEL(
                bel_version, config["bel_api"]["servers"]["api_url"]
            )
            belstr = f'{assertion.get("subject")} {assertion.get("relation", "")} {assertion.get("object", "")}'
            belstr = belstr.replace("None", "")
            try:
                messages = (
                    bo.parse(belstr)
                    .semantic_validation(error_level=error_level)
                    .validation_messages
                )

                for message in messages:
                    (level, msg) = message
                    if error_level == "ERROR":
                        if level == "ERROR":
                            v.append(
                                {
                                    "level": f"{level.title()}",
                                    "section": "Assertion",
                                    "label": f"{level.title()}-Assertion",
                                    "index": idx,
                                    "msg": msg,
                                    "msg_html": convert_msg_to_html(msg),
                                }
                            )
                    else:
                        v.append(
                            {
                                "level": f"{level.title()}",
                                "section": "Assertion",
                                "label": f"{level.title()}-Assertion",
                                "index": idx,
                                "msg": msg,
                                "msg_html": convert_msg_to_html(msg),
                            }
                        )

            except Exception as e:
                msg = f"Could not parse: {belstr}"
                v.append(
                    {
                        "level": "Error",
                        "section": "Assertion",
                        "label": "Error-Assertion",
                        "index": idx,
                        "msg": msg,
                        "msg_html": msg,
                    }
                )
                log.exception(f"Could not parse: {belstr}")

    # Annotation checks
    if error_level == "WARNING":
        for idx, annotation in enumerate(nanopub["nanopub"].get("annotations", [])):
            term_type = annotation["type"]
            term_id = annotation["id"]
            # term_label = annotation['label']
            log.info(f"Annotation: {term_type}  ID: {term_id}")

            search_body = {
                "_source": ["src_id", "id", "name", "label", "annotation_types"],
                "query": {"term": {"id": term_id}},
            }

            results = es.search(index="terms", doc_type="term", body=search_body)
            if len(results["hits"]["hits"]) > 0:
                result = results["hits"]["hits"][0]["_source"]
                if term_type not in result["annotation_types"]:
                    msg = f'Annotation type: {term_type} for {term_id} does not match annotation types in database: {result["annotation_types"]}'
                    v.append(
                        {
                            "level": "Warning",
                            "section": "Annotation",
                            "index": idx,
                            "label": "Warning-Annotation",
                            "msg": msg,
                            "msg_html": msg,
                        }
                    )
            else:
                msg = f"Annotation term: {term_id} not found in database"
                v.append(
                    {
                        "level": "Warning",
                        "section": "Annotation",
                        "index": idx,
                        "label": "Warning-Annotation",
                        "msg": msg,
                        "msg_html": msg,
                    }
                )

    return v

Response:
Validate Nanopub

    Error Levels are similar to log levels - selecting WARNING includes both
    WARNING and ERROR, selecting ERROR just includes ERROR

    The validation result is a list of objects containing
        {
            'level': 'Warning|Error',
            'section': 'Assertion|Annotation|Structure',
            'label': '{Error|Warning}-{Assertion|Annotation|Structure}',  # to be used for faceting in Elasticsearch
            'index': idx,  # Index of Assertion or Annotation in Nanopub - starts at 0
            'msg': msg,  # Error or Warning message
        }

    Args:
        nanopub: nanopub record starting with nanopub...
        level: return WARNING or just ERROR?  defaults to warnings and errors
    Returns:
        list(tuples): [{'level': 'Warning', 'section': 'Assertion', 'label': 'Warning-Assertion', 'index': 0, 'msg': <msg>}]


Instruction:
Write a Python 3 script to
get BEL Specification The json file this depends on is generated by belspec_yaml2json as part of the update_specifications function Args: version: e.g. 2.0.0 where the filename

Response:
def get_specification(version: str) -> Mapping[str, Any]:
    """Get BEL Specification

    The json file this depends on is generated by belspec_yaml2json as
    part of the update_specifications function

    Args:
        version: e.g. 2.0.0 where the filename
    """

    spec_dir = config["bel"]["lang"]["specifications"]
    spec_dict = {}

    bel_versions = get_bel_versions()
    if version not in bel_versions:
        log.error("Cannot get unknown version BEL specification")
        return {"error": "unknown version of BEL"}

    # use this variable to find our parser file since periods aren't recommended in python module names
    version_underscored = version.replace(".", "_")

    json_fn = f"{spec_dir}/bel_v{version_underscored}.json"

    with open(json_fn, "r") as f:
        spec_dict = json.load(f)

    return spec_dict


Instruction:
Can you tell what is the following Python 3 function doing
def get_bel_versions() -> List[str]:

    spec_dir = config["bel"]["lang"]["specifications"]

    fn = f"{spec_dir}/versions.json"
    with open(fn, "r") as f:
        versions = json.load(f)

    return versions

Response: Get the list of all BEL Language versions supported by BEL


Instruction:
Make a summary of the following Python 3 code
def update_specifications(force: bool = False):

    spec_dir = config["bel"]["lang"]["specifications"]
    if not os.path.isdir(spec_dir):
        os.mkdir(spec_dir)

    log.info(f"Updating BEL Specifications - stored in {spec_dir}")

    # Collect new specifications from Git repository
    if config["bel"]["lang"]["specification_github_repo"]:
        github_belspec_files(spec_dir, force=force)

    # Ensure that files use 'yaml' extension
    files = glob.glob(f"{spec_dir}/*.yml")
    for fn in files:
        new_fn = fn.replace("yml", "yaml")
        os.rename(fn, new_fn)

    # Convert YAML to enhanced JSON
    files = glob.glob(f"{spec_dir}/*.yaml")
    versions = {}
    for fn in files:
        filename = os.path.basename(fn)

        check_version = filename.replace("bel_v", "").replace(".yaml", "").replace("_", ".")

        json_fn = fn.replace(".yaml", ".json")
        version = belspec_yaml2json(fn, json_fn)

        if version != check_version:
            log.error(
                f"Version mis-match for {fn} - fn version: {check_version} version: {version}"
            )
        versions[version] = filename

    with open(f"{spec_dir}/versions.json", "w") as f:
        json.dump(list(set(versions)), f, indent=4)

    # Convert YAML file to EBNF and then parser module

    create_ebnf_parser(files)

Response: Update BEL Specifications in local directory and store in local directory specified in belbio_conf. yaml


Instruction:
Write a Python 3 script for
getting BEL Spec files from Github repo

Response:
def github_belspec_files(spec_dir, force: bool = False):
    """Get belspec files from Github repo

    Args:
        spec_dir: directory to store the BEL Specification and derived files
        force: force update of BEL Specifications from Github - skipped if local files less than 1 day old
    """

    if not force:
        dtnow = datetime.datetime.utcnow()
        delta = datetime.timedelta(1)
        yesterday = dtnow - delta

        for fn in glob.glob(f"{spec_dir}/bel*yaml"):
            if datetime.datetime.fromtimestamp(os.path.getmtime(fn)) > yesterday:
                log.info("Skipping BEL Specification update - specs less than 1 day old")
                return

    repo_url = "https://api.github.com/repos/belbio/bel_specifications/contents/specifications"
    params = {}
    github_access_token = os.getenv("GITHUB_ACCESS_TOKEN", "")
    if github_access_token:
        params = {"access_token": github_access_token}

    r = requests.get(repo_url, params=params)
    if r.status_code == 200:
        results = r.json()
        for f in results:
            url = f["download_url"]
            fn = os.path.basename(url)

            if "yaml" not in fn and "yml" in fn:
                fn = fn.replace("yml", "yaml")

            r = requests.get(url, params=params, allow_redirects=True)
            if r.status_code == 200:
                open(f"{spec_dir}/{fn}", "wb").write(r.content)
            else:
                sys.exit(
                    f"Could not get BEL Spec file {url} from Github -- Status: {r.status_code}  Msg: {r.content}"
                )
    else:
        sys.exit(
            f"Could not get BEL Spec directory listing from Github -- Status: {r.status_code}  Msg: {r.content}"
        )


Instruction:
Implement a function in Python 3 to
enhance BEL Specification and save as JSON file

Response:
def belspec_yaml2json(yaml_fn: str, json_fn: str) -> str:
    """Enhance BEL specification and save as JSON file

    Load all BEL Specification YAML files and convert to JSON files
    after enhancing them.  Also create a bel_versions.json file with
    all available BEL versions for fast loading.

    Args:
        yaml_fn: original YAML version of BEL Spec
        json_fn: enhanced JSON version of BEL Spec
    Returns:
        str: version of BEL Spec
    """

    try:
        spec_dict = yaml.load(open(yaml_fn, "r").read(), Loader=yaml.SafeLoader)

        # admin-related keys
        spec_dict["admin"] = {}
        spec_dict["admin"]["version_underscored"] = spec_dict["version"].replace(".", "_")
        spec_dict["admin"]["parser_fn"] = yaml_fn.replace(".yaml", "_parser.py")

        # add relation keys list, to_short, to_long
        add_relations(spec_dict)
        # add function keys list, to_short, to_long
        add_functions(spec_dict)
        # add namespace keys list, list_short, list_long, to_short, to_long
        add_namespaces(spec_dict)

        enhance_function_signatures(spec_dict)

        add_function_signature_help(spec_dict)

        with open(json_fn, "w") as f:
            json.dump(spec_dict, f)

    except Exception as e:
        log.error(
            "Warning: BEL Specification {yaml_fn} could not be read. Cannot proceed.".format(
                yaml_fn
            )
        )
        sys.exit()

    return spec_dict["version"]


Instruction:
Can you implement a function in Python 3 that
adds function signature help to BEL Editor users

Response:
def add_function_signature_help(spec_dict: dict) -> dict:
    """Add function signature help

    Simplify the function signatures for presentation to BEL Editor users
    """
    for f in spec_dict["functions"]["signatures"]:
        for argset_idx, argset in enumerate(spec_dict["functions"]["signatures"][f]["signatures"]):
            args_summary = ""
            args_list = []
            arg_idx = 0
            for arg_idx, arg in enumerate(
                spec_dict["functions"]["signatures"][f]["signatures"][argset_idx]["arguments"]
            ):
                if arg["type"] in ["Function", "Modifier"]:
                    vals = [
                        spec_dict["functions"]["to_short"].get(
                            val, spec_dict["functions"]["to_short"].get(val)
                        )
                        for val in arg["values"]
                    ]
                    args_summary += "|".join(vals) + "()"
                    arg_idx += 1

                    if arg.get("optional", False) and arg.get("multiple", False) is False:
                        args_summary += "?"
                        text = f'Zero or one of each function(s): {", ".join([val for val in arg["values"]])}'
                    elif arg.get("optional", False):
                        args_summary += "*"
                        text = f'Zero or more of each function(s): {", ".join([val for val in arg["values"]])}'
                    else:
                        text = f'One of following function(s): {", ".join([val for val in arg["values"]])}'

                elif arg["type"] in ["NSArg", "StrArg", "StrArgNSArg"]:
                    args_summary += f'{arg["type"]}'
                    if arg.get("optional", False) and arg.get("multiple", False) is False:
                        args_summary += "?"
                        if arg["type"] in ["NSArg"]:
                            text = f'Zero or one namespace argument of following type(s): {", ".join([val for val in arg["values"]])}'
                        elif arg["type"] == "StrArgNSArg":
                            text = f'Zero or one amespace argument or default namespace argument (without prefix) of following type(s): {", ".join([val for val in arg["values"]])}'
                        else:
                            text = f'Zero or one string argument of following type(s): {", ".join([val for val in arg["values"]])}'
                    elif arg.get("optional", False):
                        args_summary += "*"
                        if arg["type"] in ["NSArg"]:
                            text = f'Zero or more namespace arguments of following type(s): {", ".join([val for val in arg["values"]])}'
                        elif arg["type"] == "StrArgNSArg":
                            text = f'Zero or more namespace arguments or default namespace arguments (without prefix) of following type(s): {", ".join([val for val in arg["values"]])}'
                        else:
                            text = f'Zero or more of string arguments of following type(s): {", ".join([val for val in arg["values"]])}'
                    else:
                        if arg["type"] in ["NSArg"]:
                            text = f'Namespace argument of following type(s): {", ".join([val for val in arg["values"]])}'
                        elif arg["type"] == "StrArgNSArg":
                            text = f'Namespace argument or default namespace argument (without prefix) of following type(s): {", ".join([val for val in arg["values"]])}'
                        else:
                            text = f'String argument of following type(s): {", ".join([val for val in arg["values"]])}'

                args_summary += ", "
                args_list.append(text)

            args_summary = re.sub(", $", "", args_summary)
            spec_dict["functions"]["signatures"][f]["signatures"][argset_idx][
                "argument_summary"
            ] = f"{f}({args_summary})"
            spec_dict["functions"]["signatures"][f]["signatures"][argset_idx][
                "argument_help_listing"
            ] = args_list

            # print(f'{f}({args_summary})')
            # print(args_list)

    return spec_dict


Instruction:
Make a summary of the following Python 3 code
def add_relations(spec_dict: Mapping[str, Any]) -> Mapping[str, Any]:

    # Class 'Mapping' does not define '__setitem__', so the '[]' operator cannot be used on its instances
    spec_dict["relations"]["list"] = []
    spec_dict["relations"]["list_short"] = []
    spec_dict["relations"]["list_long"] = []
    spec_dict["relations"]["to_short"] = {}
    spec_dict["relations"]["to_long"] = {}

    for relation_name in spec_dict["relations"]["info"]:

        abbreviated_name = spec_dict["relations"]["info"][relation_name]["abbreviation"]
        spec_dict["relations"]["list"].extend((relation_name, abbreviated_name))
        spec_dict["relations"]["list_long"].append(relation_name)
        spec_dict["relations"]["list_short"].append(abbreviated_name)

        spec_dict["relations"]["to_short"][relation_name] = abbreviated_name
        spec_dict["relations"]["to_short"][abbreviated_name] = abbreviated_name

        spec_dict["relations"]["to_long"][abbreviated_name] = relation_name
        spec_dict["relations"]["to_long"][relation_name] = relation_name

    return spec_dict

Response: Add relation keys to the BEL specification dictionary


Instruction:
Can you generate a brief explanation for the following Python 3 code
def add_functions(spec_dict: Mapping[str, Any]) -> Mapping[str, Any]:

    # Class 'Mapping' does not define '__setitem__', so the '[]' operator cannot be used on its instances
    spec_dict["functions"]["list"] = []
    spec_dict["functions"]["list_long"] = []
    spec_dict["functions"]["list_short"] = []

    spec_dict["functions"]["primary"] = {}
    spec_dict["functions"]["primary"]["list_long"] = []
    spec_dict["functions"]["primary"]["list_short"] = []

    spec_dict["functions"]["modifier"] = {}
    spec_dict["functions"]["modifier"]["list_long"] = []
    spec_dict["functions"]["modifier"]["list_short"] = []

    spec_dict["functions"]["to_short"] = {}
    spec_dict["functions"]["to_long"] = {}

    for func_name in spec_dict["functions"]["info"]:

        abbreviated_name = spec_dict["functions"]["info"][func_name]["abbreviation"]

        spec_dict["functions"]["list"].extend((func_name, abbreviated_name))

        spec_dict["functions"]["list_long"].append(func_name)
        spec_dict["functions"]["list_short"].append(abbreviated_name)

        if spec_dict["functions"]["info"][func_name]["type"] == "primary":
            spec_dict["functions"]["primary"]["list_long"].append(func_name)
            spec_dict["functions"]["primary"]["list_short"].append(abbreviated_name)
        else:
            spec_dict["functions"]["modifier"]["list_long"].append(func_name)
            spec_dict["functions"]["modifier"]["list_short"].append(abbreviated_name)

        spec_dict["functions"]["to_short"][abbreviated_name] = abbreviated_name
        spec_dict["functions"]["to_short"][func_name] = abbreviated_name

        spec_dict["functions"]["to_long"][abbreviated_name] = func_name
        spec_dict["functions"]["to_long"][func_name] = func_name

    return spec_dict

Response: Add function keys to BEL specification dictionary.


Instruction:
Here you have a function in Python 3, explain what it does
def add_namespaces(spec_dict):

    for ns in spec_dict["namespaces"]:
        spec_dict["namespaces"][ns]["list"] = []
        spec_dict["namespaces"][ns]["list_long"] = []
        spec_dict["namespaces"][ns]["list_short"] = []

        spec_dict["namespaces"][ns]["to_short"] = {}
        spec_dict["namespaces"][ns]["to_long"] = {}

        for obj in spec_dict["namespaces"][ns]["info"]:
            spec_dict["namespaces"][ns]["list"].extend([obj["name"], obj["abbreviation"]])
            spec_dict["namespaces"][ns]["list_short"].append(obj["abbreviation"])
            spec_dict["namespaces"][ns]["list_long"].append(obj["name"])

            spec_dict["namespaces"][ns]["to_short"][obj["abbreviation"]] = obj["abbreviation"]
            spec_dict["namespaces"][ns]["to_short"][obj["name"]] = obj["abbreviation"]

            spec_dict["namespaces"][ns]["to_long"][obj["abbreviation"]] = obj["name"]
            spec_dict["namespaces"][ns]["to_long"][obj["name"]] = obj["name"]

            # For AminoAcid namespace
            if "abbrev1" in obj:
                spec_dict["namespaces"][ns]["to_short"][obj["abbrev1"]] = obj["abbreviation"]
                spec_dict["namespaces"][ns]["to_long"][obj["abbrev1"]] = obj["name"]

Response: Add namespace convenience keys list list_long to_short to_long


Instruction:
Can you generate a brief explanation for the following Python 3 code
def enhance_function_signatures(spec_dict: Mapping[str, Any]) -> Mapping[str, Any]:

    for func in spec_dict["functions"]["signatures"]:
        for i, sig in enumerate(spec_dict["functions"]["signatures"][func]["signatures"]):
            args = sig["arguments"]
            req_args = []
            pos_args = []
            opt_args = []
            mult_args = []
            for arg in args:
                # Multiple argument types
                if arg.get("multiple", False):
                    if arg["type"] in ["Function", "Modifier"]:
                        mult_args.extend(arg.get("values", []))
                    elif arg["type"] in ["StrArgNSArg", "NSArg", "StrArg"]:
                        # Complex signature has this
                        mult_args.append(arg["type"])

                # Optional, position dependent - will be added after req_args based on order in bel_specification
                elif arg.get("optional", False) and arg.get("position", False):
                    if arg["type"] in ["Function", "Modifier"]:
                        pos_args.append(arg.get("values", []))
                    elif arg["type"] in ["StrArgNSArg", "NSArg", "StrArg"]:
                        pos_args.append(arg["type"])

                # Optional, position independent
                elif arg.get("optional", False):
                    if arg["type"] in ["Function", "Modifier"]:
                        opt_args.extend(arg.get("values", []))
                    elif arg["type"] in ["StrArgNSArg", "NSArg", "StrArg"]:
                        opt_args.append(arg["type"])

                # Required arguments, position dependent
                else:
                    if arg["type"] in ["Function", "Modifier"]:
                        req_args.append(arg.get("values", []))
                    elif arg["type"] in ["StrArgNSArg", "NSArg", "StrArg"]:
                        req_args.append(arg["type"])

            spec_dict["functions"]["signatures"][func]["signatures"][i]["req_args"] = copy.deepcopy(
                req_args
            )
            spec_dict["functions"]["signatures"][func]["signatures"][i]["pos_args"] = copy.deepcopy(
                pos_args
            )
            spec_dict["functions"]["signatures"][func]["signatures"][i]["opt_args"] = copy.deepcopy(
                opt_args
            )
            spec_dict["functions"]["signatures"][func]["signatures"][i][
                "mult_args"
            ] = copy.deepcopy(mult_args)

    return spec_dict

Response: Enhance function signatures in BEL Specification dictionary.


Instruction:
Can you write a function in Python 3 where it
gets the EBNF template from Github belbio - bel. ebnf. j2

Response:
def get_ebnf_template():
    """Get EBNF template from Github belbio/bel_specifications repo"""

    spec_dir = config["bel"]["lang"]["specifications"]
    local_fp = f"{spec_dir}/bel.ebnf.j2"

    repo_url = (
        "https://api.github.com/repos/belbio/bel_specifications/contents/resources/bel.ebnf.j2"
    )

    params = {}
    github_access_token = os.getenv("GITHUB_ACCESS_TOKEN", "")
    if github_access_token:
        params = {"access_token": github_access_token}

    try:
        # Get download url for template file
        r = requests.get(repo_url, params=params)

        if r.status_code == 200:
            template_url = r.json()["download_url"]
        else:
            log.warning("Could not get EBNF file download url from Github")

        # Get template file
        try:
            r = requests.get(template_url, params=params, allow_redirects=True)
            if r.status_code == 200:
                open(local_fp, "wt").write(r.text)
            else:
                log.warning(
                    f"Could not download EBNF file from Github -- Status: {r.status_code}  Msg: {r.text}"
                )

        except Exception as e:
            log.warning(
                f"Could not download EBNF file from Github -- Status: {r.status_code}  Msg: {e}"
            )

    except Exception as e:
        log.warning("Could not download BEL EBNF template file")
        if not os.path.exists(f"{spec_dir}/local_fp"):
            log.error("No BEL EBNF template file available")

    return local_fp


Instruction:
How would you code a function in Python 3 to
create EBNF files and EBNF - based parsers

Response:
def create_ebnf_parser(files):
    """Create EBNF files and EBNF-based parsers"""

    flag = False
    for belspec_fn in files:
        # Get EBNF Jinja template from Github if enabled
        if config["bel"]["lang"]["specification_github_repo"]:
            tmpl_fn = get_ebnf_template()

        # Check if EBNF file is more recent than belspec_fn
        ebnf_fn = belspec_fn.replace(".yaml", ".ebnf")
        if not os.path.exists(ebnf_fn) or os.path.getmtime(belspec_fn) > os.path.getmtime(ebnf_fn):
            with open(belspec_fn, "r") as f:
                belspec = yaml.load(f, Loader=yaml.SafeLoader)

            tmpl_dir = os.path.dirname(tmpl_fn)
            tmpl_basename = os.path.basename(tmpl_fn)

            bel_major_version = belspec["version"].split(".")[0]

            env = jinja2.Environment(
                loader=jinja2.FileSystemLoader(tmpl_dir)
            )  # create environment for template
            template = env.get_template(tmpl_basename)  # get the template

            # replace template placeholders with appropriate variables
            relations_list = [
                (relation, belspec["relations"]["info"][relation]["abbreviation"])
                for relation in belspec["relations"]["info"]
            ]
            relations_list = sorted(list(itertools.chain(*relations_list)), key=len, reverse=True)

            functions_list = [
                (function, belspec["functions"]["info"][function]["abbreviation"])
                for function in belspec["functions"]["info"]
                if belspec["functions"]["info"][function]["type"] == "primary"
            ]
            functions_list = sorted(list(itertools.chain(*functions_list)), key=len, reverse=True)

            modifiers_list = [
                (function, belspec["functions"]["info"][function]["abbreviation"])
                for function in belspec["functions"]["info"]
                if belspec["functions"]["info"][function]["type"] == "modifier"
            ]
            modifiers_list = sorted(list(itertools.chain(*modifiers_list)), key=len, reverse=True)

            created_time = datetime.datetime.now().strftime("%B %d, %Y - %I:%M:%S%p")

            ebnf = template.render(
                functions=functions_list,
                m_functions=modifiers_list,
                relations=relations_list,
                bel_version=belspec["version"],
                bel_major_version=bel_major_version,
                created_time=created_time,
            )

            with open(ebnf_fn, "w") as f:
                f.write(ebnf)

            parser_fn = ebnf_fn.replace(".ebnf", "_parser.py")

            parser = tatsu.to_python_sourcecode(ebnf, filename=parser_fn)
            flag = True
            with open(parser_fn, "wt") as f:
                f.write(parser)

    if flag:
        # In case we created new parser modules
        importlib.invalidate_caches()


Instruction:
Write a Python 3 function for
getting function_help given function name

Response:
def get_function_help(function: str, bel_spec: BELSpec):
    """Get function_help given function name

    This will get the function summary template (argument summary in signature)
    and the argument help listing.
    """

    function_long = bel_spec["functions"]["to_long"].get(function)
    function_help = []

    if function_long:
        for signature in bel_spec["functions"]["signatures"][function_long]["signatures"]:
            function_help.append(
                {
                    "function_summary": signature["argument_summary"],
                    "argument_help": signature["argument_help_listing"],
                    "description": bel_spec["functions"]["info"][function_long]["description"],
                }
            )

    return function_help


Instruction:
How would you implement a function in Python 3 that
checks if loc is inside span

Response:
def in_span(loc: int, span: Span) -> bool:
    """Checks if loc is inside span"""

    if loc >= span[0] and loc <= span[1]:
        return True
    else:
        return False


Instruction:
Can you write a function in Python 3 where it
finds BEL function or argument at cursor location

Response:
def cursor(
    belstr: str, ast: AST, cursor_loc: int, result: Mapping[str, Any] = None
) -> Mapping[str, Any]:
    """Find BEL function or argument at cursor location

    Args:
        belstr: BEL String used to create the completion_text
        ast (Mapping[str, Any]): AST (dict) of BEL String
        cursor_loc (int): given cursor location from input field
            cursor_loc starts at 0, think of it like a block cursor covering each char
        result: used to recursively return the result
    Returns:
        result dict
    """

    log.debug(f"SubAST: {json.dumps(ast, indent=4)}")

    # Recurse down through subject, object, nested to functions
    log.debug(f"Cursor keys {ast.keys()}, BELStr: {belstr}")

    if len(belstr) == 0:
        return {"type": "Function", "replace_span": (0, 0), "completion_text": ""}

    if "relation" in ast and in_span(cursor_loc, ast["relation"]["span"]):
        log.debug("In relation")

        completion_text = belstr[ast["relation"]["span"][0] : cursor_loc + 1]
        return {
            "type": "Relation",
            "replace_span": ast["relation"]["span"],
            "completion_text": completion_text,
        }

    # Handle subject, object and nested keys in tree
    elif "span" not in ast and isinstance(ast, dict):
        for key in ast:
            if key in ["subject", "object", "nested"]:
                log.debug(f"Recursing Keys {key}")
                result = cursor(belstr, ast[key], cursor_loc, result=result)
                if result:
                    return result

    # Matches Functions, NSArgs and StrArgs/StrArgNSArg
    if "span" in ast and in_span(cursor_loc, ast["span"]):
        log.debug("Inside subject/object subAST")
        if "function" in ast:
            name_span = ast["function"]["name_span"]
            if in_span(cursor_loc, name_span):
                return {
                    "type": "Function",
                    "replace_span": name_span,
                    "completion_text": belstr[name_span[0] : cursor_loc + 1],
                }
            for idx, arg in enumerate(ast["args"]):
                if (
                    cursor_loc == ast["function"]["parens_span"][0]
                    and ast["function"]["parens_span"][1] == -1
                ):
                    return {
                        "type": "StrArg",  # default type if unknown
                        "arg_idx": idx,
                        "replace_span": arg["span"],  # replace entire strarg
                        "parent_function": ast["function"]["name"],
                        "completion_text": "",
                    }

                elif in_span(cursor_loc, arg["span"]):
                    log.debug(
                        f'In argument span {arg["span"]}  Cursor_loc: {cursor_loc}'
                    )
                    if arg["type"] == "Function":
                        if in_span(cursor_loc, arg["function"]["name_span"]):
                            log.debug("Found replace_span in args: Function type")
                            return {
                                "type": "Function",
                                "replace_span": arg["function"][
                                    "name_span"
                                ],  # replace function name only
                                "arg_idx": idx,
                                "args": copy.deepcopy(ast["args"]),
                                "parent_function": ast["function"]["name"],
                                "completion_text": belstr[
                                    arg["function"]["name_span"][0] : cursor_loc + 1
                                ],
                            }
                        else:
                            log.debug(f'Recursing Function  {arg["span"]}')
                            result = cursor(belstr, arg, cursor_loc, result=result)
                    elif arg["type"] == "NSArg":

                        result = {
                            "type": "NSArg",
                            "replace_span": arg["span"],  # replace entire nsarg
                            "arg_idx": idx,
                            "args": copy.deepcopy(ast["args"]),
                            "parent_function": ast["function"]["name"],
                        }

                        # Filter on namespace and query on ns_val chars up to cursor_loc
                        if in_span(cursor_loc, arg["nsarg"]["ns_val_span"]):
                            result["namespace"] = arg["nsarg"][
                                "ns"
                            ]  # provide namespace for nsarg filtering
                            result["completion_text"] = belstr[
                                arg["nsarg"]["ns_val_span"][0] : cursor_loc + 1
                            ]
                        # Query on nsarg chars up to cursor_loc
                        else:
                            result["completion_text"] = belstr[
                                arg["nsarg"]["ns_span"][0] : cursor_loc + 1
                            ]

                        log.debug(f"Found replace_span in args: NSArg {result}")
                        return result
                    elif (
                        arg["type"] == "StrArg"
                    ):  # in case this is a default namespace StrArg
                        if (
                            arg["span"][0] == arg["span"][1]
                        ):  # handle case like p() cursor=2
                            completion_text = arg["arg"]
                        else:
                            completion_text = belstr[arg["span"][0] : cursor_loc + 1]

                        return {
                            "type": "StrArg",
                            "arg_idx": idx,
                            "replace_span": arg["span"],  # replace entire strarg
                            "parent_function": ast["function"]["name"],
                            "completion_text": completion_text.lstrip(),
                        }
    return result


Instruction:
Can you generate a brief explanation for the following Python 3 code
def nsarg_completions(
    completion_text: str,
    entity_types: list,
    bel_spec: BELSpec,
    namespace: str,
    species_id: str,
    bel_fmt: str,
    size: int,
):
    """Namespace completions

    Args:
        completion_text
        entity_types: used to filter namespace search results
        bel_spec: used to search default namespaces
        namespace: used to filter namespace search results
        species_id: used to filter namespace search results
        bel_fmt: used to select full name or abbrev for default namespaces
        size: how many completions to return

    Results:
        list of replacement text objects
    """

    minimal_nsarg_completion_len = 1

    species = [species_id]
    namespaces = [namespace]
    replace_list = []

    if len(completion_text) >= minimal_nsarg_completion_len:
        # Use BEL.bio API module if running bel module in BEL.bio API, otherwise call BEL.bio API endpoint
        #   is there a better way to  handle this?

        url = f'{config["bel_api"]["servers"]["api_url"]}/terms/completions/{url_path_param_quoting(completion_text)}'
        params = {
            "size": size,
            "entity_types": entity_types,
            "namespaces": namespaces,
            "species": species,
        }
        r = get_url(url, params=params)

        if r.status_code == 200:
            ns_completions = r.json()
        else:
            log.error(f"Status code of {r.status_code} for {url}")
            ns_completions = {}

        for complete in ns_completions.get("completions", []):
            replace_list.append(
                {
                    "replacement": complete["id"],
                    "label": f"{complete['id']} ({complete['label']})",
                    "highlight": complete["highlight"][-1],
                    "type": "NSArg",
                }
            )

    # Check default namespaces
    for entity_type in entity_types:
        default_namespace = bel_spec["namespaces"].get(entity_type, [])
        if default_namespace:
            for obj in default_namespace["info"]:
                replacement = None
                if bel_fmt == "long" and re.match(
                    completion_text, obj["name"], re.IGNORECASE
                ):
                    replacement = obj["name"]
                elif bel_fmt in ["short", "medium"] and re.match(
                    completion_text, obj["abbreviation"], re.IGNORECASE
                ):
                    replacement = obj["abbreviation"]

                if replacement:
                    highlight = replacement.replace(
                        completion_text, f"<em>{completion_text}</em>"
                    )
                    replace_list.insert(
                        0,
                        {
                            "replacement": replacement,
                            "label": replacement,
                            "highlight": highlight,
                            "type": "NSArg",
                        },
                    )

    return replace_list[:size]

Response: Search for completions in the current namespace.


Instruction:
How would you code a function in Python 3 to
filter BEL relations by prefix

Response:
def relation_completions(
    completion_text: str, bel_spec: BELSpec, bel_fmt: str, size: int
) -> list:
    """Filter BEL relations by prefix

    Args:
        prefix: completion string
        bel_fmt: short, medium, long BEL formats
        spec: BEL specification

    Returns:
        list: list of BEL relations that match prefix
    """

    if bel_fmt == "short":
        relation_list = bel_spec["relations"]["list_short"]
    else:
        relation_list = bel_spec["relations"]["list_long"]

    matches = []
    for r in relation_list:
        if re.match(completion_text, r):
            matches.append(r)

    replace_list = []
    for match in matches:
        highlight = match.replace(completion_text, f"<em>{completion_text}</em>")
        replace_list.append(
            {
                "replacement": match,
                "label": match,
                "highlight": highlight,
                "type": "Relation",
            }
        )

    return replace_list[:size]


Instruction:
Create a Python 3 function for
filtering function list by prefix

Response:
def function_completions(
    completion_text: str,
    bel_spec: BELSpec,
    function_list: list,
    bel_fmt: str,
    size: int,
) -> list:
    """Filter BEL functions by prefix

    Args:
        prefix: completion string
        bel_fmt: short, medium, long BEL formats
        spec: BEL specification

    Returns:
        list: list of BEL functions that match prefix
    """

    # Convert provided function list to correct bel_fmt
    if isinstance(function_list, list):
        if bel_fmt in ["short", "medium"]:
            function_list = [
                bel_spec["functions"]["to_short"][fn] for fn in function_list
            ]
        else:
            function_list = [
                bel_spec["functions"]["to_long"][fn] for fn in function_list
            ]
    elif bel_fmt in ["short", "medium"]:
        function_list = bel_spec["functions"]["primary"]["list_short"]
    else:
        function_list = bel_spec["functions"]["primary"]["list_long"]

    matches = []
    for f in function_list:
        escaped_completion_text = completion_text.replace(r"(", r"\(").replace(
            r")", r"\)"
        )
        log.debug(f"Completion match: {escaped_completion_text}  F: {f}")
        if re.match(escaped_completion_text, f):
            matches.append(f)

    replace_list = []
    for match in matches:
        if completion_text:
            highlight = match.replace(completion_text, f"<em>{completion_text}</em>")
        else:
            highlight = completion_text

        replace_list.append(
            {
                "replacement": match,
                "label": f"{match}()",
                "highlight": highlight,
                "type": "Function",
            }
        )

    return replace_list[:size]


Instruction:
Explain what the following Python 3 code does
def arg_completions(
    completion_text: str,
    parent_function: str,
    args: list,
    arg_idx: int,
    bel_spec: BELSpec,
    bel_fmt: str,
    species_id: str,
    namespace: str,
    size: int,
):
    """Function argument completion

    Only allow legal options for completion given function name, arguments and index of argument
    to replace.

    Args:
        completion_text: text to use for completion - used for creating highlight
        parent_function: BEL function containing these args
        args: arguments of BEL function
        arg_idx: completing on this argument identified by this index
        bel_spec: BEL Specification
        bel_fmt: short, medium, long BEL function/relation formats
        species_id: filter on this species id, e.g. TAX:9606 if available
        namespace: filter on this namespace if available
        size: number of completions to return

    Return:
        list of replacements
    """

    function_long = bel_spec["functions"]["to_long"].get(parent_function)
    if not function_long:
        return []

    signatures = bel_spec["functions"]["signatures"][function_long]["signatures"]

    # Position based argument  ###################################
    function_list = []
    entity_types = []
    fn_replace_list, ns_arg_replace_list = [], []
    position_flag = False  # Signature matches position-based argument

    # Check for position based argument
    for signature in signatures:
        sig_arg = signature["arguments"][arg_idx]
        sig_type = sig_arg["type"]

        if sig_arg.get("position", False) and arg_idx == sig_arg["position"] - 1:
            position_flag = True
            if sig_type in ["Function", "Modifier"]:
                function_list.extend(sig_arg["values"])
            elif sig_type in ["NSArg", "StrArgNSArg"]:
                entity_types.extend(sig_arg["values"])

    if not position_flag:
        # Collect optional and multiple signature arguments for completion
        opt_fn_sig_args = []
        opt_nsarg_sig_args = []
        mult_fn_sig_args = []
        mult_nsarg_sig_args = []
        for signature in signatures:
            signature_opt_fn_sig_args = []
            signature_opt_nsarg_sig_args = []
            signature_mult_fn_sig_args = []
            signature_mult_nsarg_sig_args = []
            max_position = -1
            for sig_arg in signature["arguments"]:
                if "position" in sig_arg:
                    max_position = sig_arg["position"]
                    continue  # Skip position based signature arguments

                if (
                    sig_arg.get("optional", False) is True
                    and sig_arg.get("multiple", False) is False
                ):
                    if sig_arg["type"] in ["Function", "Modifier"]:
                        signature_opt_fn_sig_args.extend(sig_arg["values"])
                    elif sig_arg["type"] in ["NSArg", "StrArgNSArg"]:
                        signature_opt_nsarg_sig_args.extend(sig_arg["values"])
                elif sig_arg.get("multiple", False) is True:
                    if sig_arg["type"] in ["Function", "Modifier"]:
                        signature_mult_fn_sig_args.extend(sig_arg["values"])
                    elif sig_arg["type"] in ["NSArg", "StrArgNSArg"]:
                        signature_mult_nsarg_sig_args.extend(sig_arg["values"])

            # Remove signature non-multiple, optional arguments that are already in args list
            for idx, arg in enumerate(args):
                if idx <= max_position - 1:  # Skip positional arguments
                    continue
                if idx == arg_idx:  # Skip argument to be completed
                    continue
                log.debug(f"Remove Optional Args {arg}  {signature_opt_fn_sig_args}")

            opt_fn_sig_args.extend(signature_opt_fn_sig_args)
            opt_nsarg_sig_args.extend(signature_opt_nsarg_sig_args)
            mult_fn_sig_args.extend(signature_mult_fn_sig_args)
            mult_nsarg_sig_args.extend(signature_mult_nsarg_sig_args)

        function_list.extend(list(set(opt_fn_sig_args + mult_fn_sig_args)))
        entity_types.extend(list(set(opt_nsarg_sig_args + mult_nsarg_sig_args)))

    if function_list:
        log.debug(f"ArgComp - position-based Function list: {function_list}")
        fn_replace_list = function_completions(
            completion_text, bel_spec, function_list, bel_fmt, size
        )

    if entity_types:
        log.debug(f"ArgComp - position-based Entity types: {entity_types}")
        ns_arg_replace_list = nsarg_completions(
            completion_text,
            entity_types,
            bel_spec,
            namespace,
            species_id,
            bel_fmt,
            size,
        )

    replace_list = fn_replace_list + ns_arg_replace_list

    return replace_list

Response: Function argument completion function


Instruction:
Can you generate the documentation for the following Python 3 function
def add_completions(
    replace_list: list, belstr: str, replace_span: Span, completion_text: str
) -> List[Mapping[str, Any]]:
    """Create completions to return given replacement list

    Args:
        replace_list: list of completion replacement values
        belstr: BEL String
        replace_span: start, stop of belstr to replace
        completion_text: text to use for completion - used for creating highlight
    Returns:
        [{
            "replacement": replacement,
            "cursor_loc": cursor_loc,
            "highlight": highlight,
            "label": label,
        }]
    """

    completions = []

    for r in replace_list:

        # if '(' not in belstr:
        #     replacement = f'{r["replacement"]}()'
        #     cursor_loc = len(replacement) - 1  # inside parenthesis
        # elif r['type'] == 'Function' and replace_span[1] == len(belstr) - 1:

        if len(belstr) > 0:
            belstr_end = len(belstr) - 1
        else:
            belstr_end = 0

        log.debug(
            f'Replace list {r}  Replace_span {replace_span}  BELstr: {belstr} Len: {belstr_end} Test1 {r["type"] == "Function"}  Test2 {replace_span[1] + 1 == len(belstr)}'
        )

        # Put a space between comma and following function arg
        if (
            r["type"] == "Function"
            and replace_span[0] > 0
            and belstr[replace_span[0] - 1] == ","
        ):
            log.debug("prior char is a comma")
            replacement = (
                belstr[0 : replace_span[0]]
                + " "
                + f"{r['replacement']}()"
                + belstr[replace_span[1] + 1 :]
            )
            cursor_loc = len(
                belstr[0 : replace_span[0]] + " " + f"{r['replacement']}()"
            )
        # Put a space between comman and following NSArg or StrArg
        elif replace_span[0] > 0 and belstr[replace_span[0] - 1] == ",":
            log.debug("prior char is a comma")
            replacement = (
                belstr[0 : replace_span[0]]
                + " "
                + r["replacement"]
                + belstr[replace_span[1] + 1 :]
            )
            cursor_loc = len(belstr[0 : replace_span[0]] + " " + r["replacement"])
        # Add function to end of belstr
        elif r["type"] == "Function" and replace_span[1] >= belstr_end:
            replacement = belstr[0 : replace_span[0]] + f"{r['replacement']}()"
            cursor_loc = len(replacement) - 1  # inside parenthesis
            log.debug(f"Replacement: {replacement}")
        # Insert replacement in beginning or middle of belstr
        else:
            replacement = (
                belstr[0 : replace_span[0]]
                + r["replacement"]
                + belstr[replace_span[1] + 1 :]
            )
            cursor_loc = len(
                belstr[0 : replace_span[0]] + r["replacement"]
            )  # move cursor just past replacement

        completions.append(
            {
                "replacement": replacement,
                "cursor_loc": cursor_loc,
                "highlight": r["highlight"],
                "label": r["label"],
            }
        )

    return completions

Response: Create completions for given replacement list and BEL string.


Instruction:
How would you explain what the following Python 3 function does
def get_completions(
    belstr: str,
    cursor_loc: int,
    bel_spec: BELSpec,
    bel_comp: str,
    bel_fmt: str,
    species_id: str,
    size: int,
):
    """Get BEL Assertion completions

    Args:

    Results:

    """

    ast, errors = pparse.get_ast_dict(belstr)

    spans = pparse.collect_spans(ast)

    completion_text = ""
    completions = []
    function_help = []

    log.debug(f"Cursor location BELstr: {belstr}  Cursor idx: {cursor_loc}")
    cursor_results = cursor(belstr, ast, cursor_loc)
    log.debug(f"Cursor results: {cursor_results}")

    if not cursor_results:
        log.debug("Cursor results is empty")
        return (completion_text, completions, function_help, spans)

    completion_text = cursor_results.get("completion_text", "")

    replace_span = cursor_results["replace_span"]
    namespace = cursor_results.get("namespace", None)

    if "parent_function" in cursor_results:
        parent_function = cursor_results["parent_function"]
        function_help = bel_specification.get_function_help(
            cursor_results["parent_function"], bel_spec
        )

        args = cursor_results.get("args", [])
        arg_idx = cursor_results.get("arg_idx")

        replace_list = arg_completions(
            completion_text,
            parent_function,
            args,
            arg_idx,
            bel_spec,
            bel_fmt,
            species_id,
            namespace,
            size,
        )
    elif cursor_results["type"] == "Function":
        function_list = None
        replace_list = function_completions(
            completion_text, bel_spec, function_list, bel_fmt, size
        )
    elif cursor_results["type"] == "Relation":
        replace_list = relation_completions(completion_text, bel_spec, bel_fmt, size)

    completions.extend(
        add_completions(replace_list, belstr, replace_span, completion_text)
    )

    return completion_text, completions, function_help, spans

Response: Get BEL Assertion completions


Instruction:
Write a Python 3 script for
scanning BEL string to find the corresponding set of character locations and return a list of character locations and a list of errors.

Response:
def parse_chars(bels: list, errors: Errors) -> Tuple[CharLocs, Errors]:
    """Scan BEL string to map parens, quotes, commas

    Args:
        bels: bel string as an array of characters
        errors: list of error tuples ('<type>', '<msg>')

    Returns:
        (char_locs, errors): character locations and errors
    """
    pstack, qstack, nested_pstack = [], [], []
    parens, nested_parens, quotes, commas = {}, {}, {}, {}

    notquoted_flag = True

    for i, c in enumerate(bels):
        prior_char = i - 1
        # print('BEL', prior_char, b[prior_char])

        # Find starting quote
        if c == '"' and bels[prior_char] != "\\" and len(qstack) == 0:
            qstack.append(i)
            notquoted_flag = False

        # Find closing quote
        elif c == '"' and bels[prior_char] != "\\":
            quotes[qstack.pop()] = i
            notquoted_flag = True

        # Find all escaped quotes outside of quoted string
        elif c == '"' and bels[prior_char] == "\\" and len(qstack) == 0:
            errors.append(
                (
                    "ERROR",
                    f"Escaped quote outside of quoted string at location: {i - 1}",
                    (i - 1, i - 1),
                )
            )

        # Find all nested object opening parens
        elif notquoted_flag and c == "(" and bels[prior_char] == " ":
            if len(nested_pstack) > 1:
                errors.append(
                    (
                        "ERROR",
                        f"More than one nested parenthesis or left parenthesis following a space character",
                        (i, i),
                    )
                )

            nested_pstack.append(i)

        # Find all opening parens
        elif notquoted_flag and c == "(" and bels[prior_char] not in ["\\"]:
            pstack.append(i)

        # Find all closing parens
        elif notquoted_flag and c == ")" and bels[prior_char] != "\\":
            if len(pstack):
                if len(pstack) > 1:
                    parens[pstack.pop()] = (i, "child")
                else:
                    parens[pstack.pop()] = (i, "top")
            elif len(nested_pstack):
                nested_parens[nested_pstack.pop()] = (i, "top")
            else:
                errors.append(
                    (
                        "ERROR",
                        f"Missing left parenthesis for right parenthesis at location {i}",
                        (i, i),
                    )
                )
        # Find comma outside of quoted string
        elif notquoted_flag and c == "," and len(qstack) == 0:
            sparen = pstack[-1]
            if sparen not in commas:
                commas[sparen] = [i]
            else:
                commas[sparen].append(i)

    while len(pstack):
        errors.append(
            (
                "ERROR",
                f"Missing right parenthesis for left parenthesis at location {pstack[-1]}",
                (pstack[-1], pstack[-1]),
            )
        )
        if len(pstack) > 1:
            parens[pstack.pop()] = (-1, "child")
        else:
            parens[pstack.pop()] = (-1, "top")

    while len(nested_pstack):
        errors.append(
            (
                "ERROR",
                f"Missing right parenthesis for nested object left parenthesis at location {nested_pstack[-1]}",
                (nested_pstack[-1], nested_pstack[-1]),
            )
        )
        nested_parens[nested_pstack.pop()] = (-1, "top")

    if len(qstack):
        missing_quote = qstack.pop()
        errors.append(
            (
                "ERROR",
                f"Missing right quote for left quote at location {missing_quote}",
                (missing_quote, missing_quote),
            )
        )

    return (
        {
            "parens": parens,
            "nested_parens": nested_parens,
            "quotes": quotes,
            "commas": commas,
        },
        errors,
    )


Instruction:
Can you create a Python 3 function that
parses functions from BEL string using paren comma quote character locations and return parsed and errors.

Response:
def parse_functions(
    bels: list, char_locs: CharLocs, parsed: Parsed, errors: Errors
) -> Tuple[Parsed, Errors]:
    """Parse functions from BEL using paren, comma, quote character locations

    Args:
        bels: BEL string as list of chars
        char_locs: paren, comma, quote character locations
        errors: Any error messages generated during the parse

    Returns:
        (functions, errors): function names and locations and error messages
    """
    parens = char_locs["parens"]

    # Handle partial top-level function name
    if not parens:
        bels_len = len(bels) - 1
        span = (0, bels_len)
        parsed[span] = {
            "name": "".join(bels),
            "type": "Function",
            "span": span,
            "name_span": (span),
            "function_level": "top",
        }
        return parsed, errors

    for sp in sorted(parens):  # sp = starting paren, ep = ending_paren
        ep, function_level = parens[sp]

        # Functions can't have a space between function name and left paren
        if bels[sp - 1] == " ":
            continue

        # look in front of start paren for function name
        for i in range(sp - 1, 0, -1):
            if bels[i] in [" ", ",", "("]:  # function name upstream boundary chars
                if i < sp - 1:
                    if ep == -1:
                        span = (i + 1, len(bels) - 1)
                    else:
                        span = (i + 1, ep)

                    parsed[span] = {
                        "name": "".join(bels[i + 1 : sp]),
                        "type": "Function",
                        "span": span,
                        "name_span": (i + 1, sp - 1),
                        "parens_span": (sp, ep),
                        "function_level": function_level,
                    }
                break
        else:
            if ep == -1:
                span = (0, len(bels) - 1)
            else:
                span = (0, ep)

            parsed[span] = {
                "name": "".join(bels[0:sp]),
                "type": "Function",
                "span": span,
                "name_span": (0, sp - 1),
                "parens_span": (sp, ep),
                "function_level": function_level,
            }

    return parsed, errors


Instruction:
Implement a Python 3 function for
parsing arguments from functions Formula.

Response:
def parse_args(
    bels: list, char_locs: CharLocs, parsed: Parsed, errors: Errors
) -> Tuple[Parsed, Errors]:
    """Parse arguments from functions

    Args:
        bels: BEL string as list of chars
        char_locs: char locations for parens, commas and quotes
        parsed: function locations
        errors: error messages

    Returns:
        (functions, errors): function and arg locations plus error messages
    """

    commas = char_locs["commas"]

    # Process each span key in parsed from beginning
    for span in parsed:
        if parsed[span]["type"] != "Function" or "parens_span" not in parsed[span]:
            continue  # Skip if not argument-less
        sp, ep = parsed[span]["parens_span"]

        # calculate args_end position
        if ep == -1:  # supports bel completion
            args_end = len(bels) - 1  # 1
        else:
            args_end = ep - 1  # 1

        # Parse arguments
        args = []
        arg_start = sp + 1
        each_arg_end_list = sorted([end - 1 for end in commas.get(sp, [])] + [args_end])
        for arg_end in each_arg_end_list:
            # log.debug(f'Arg_start: {arg_start}  Arg_end: {arg_end}')

            # Skip blanks at beginning of argument
            while arg_start < args_end and bels[arg_start] == " ":
                arg_start += 1

            # Trim arg_end (e.g. HGNC:AKT1  , HGNC:EGF) - if there are spaces before comma
            trimmed_arg_end = arg_end
            while trimmed_arg_end > arg_start and bels[trimmed_arg_end] == " ":
                trimmed_arg_end -= 1

            if trimmed_arg_end < arg_start:
                trimmed_arg_end = arg_start

            arg = "".join(bels[arg_start : trimmed_arg_end + 1])

            # log.debug(f'Adding arg to args: {arg_start} {trimmed_arg_end}')
            args.append({"arg": arg, "span": (arg_start, trimmed_arg_end)})
            arg_start = arg_end + 2

        parsed[span]["args"] = args

    return parsed, errors


Instruction:
Given the following Python 3 function, write the documentation
def arg_types(parsed: Parsed, errors: Errors) -> Tuple[Parsed, Errors]:

    func_pattern = re.compile(r"\s*[a-zA-Z]+\(")
    nsarg_pattern = re.compile(r"^\s*([A-Z]+):(.*?)\s*$")

    for span in parsed:
        if parsed[span]["type"] != "Function" or "parens_span" not in parsed[span]:
            continue

        for i, arg in enumerate(parsed[span]["args"]):
            nsarg_matches = nsarg_pattern.match(arg["arg"])
            if func_pattern.match(arg["arg"]):
                parsed[span]["args"][i].update({"type": "Function"})
            elif nsarg_matches:
                (start, end) = arg["span"]
                ns = nsarg_matches.group(1)
                ns_val = nsarg_matches.group(2)
                ns_span = nsarg_matches.span(1)
                ns_span = (ns_span[0] + start, ns_span[1] + start - 1)
                ns_val_span = nsarg_matches.span(2)
                ns_val_span = (ns_val_span[0] + start, ns_val_span[1] + start - 1)

                parsed[span]["args"][i].update(
                    {
                        "type": "NSArg",
                        "ns": ns,
                        "ns_span": ns_span,
                        "ns_val": ns_val,
                        "ns_val_span": ns_val_span,
                    }
                )
            else:
                parsed[span]["args"][i].update({"type": "StrArg"})

    return parsed, errors

Response: Add argument types to the function data structure.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def parse_relations(
    belstr: str, char_locs: CharLocs, parsed: Parsed, errors: Errors
) -> Tuple[Parsed, Errors]:
    """Parse relations from BEL string

    Args:
        belstr: BEL string as one single string (not list of chars)
        char_locs: paren, comma and quote char locations
        parsed: data structure for parsed functions, relations, nested
        errors: error messages

    Returns:
        (parsed, errors):
    """
    quotes = char_locs["quotes"]
    quoted_range = set([i for start, end in quotes.items() for i in range(start, end)])

    for match in relations_pattern_middle.finditer(belstr):
        (start, end) = match.span(1)
        # log.debug(f'Relation-middle {match}')
        end = end - 1  # adjust end to match actual end character index
        if start != end:
            test_range = set(range(start, end))
        else:
            test_range = set(start)

        # Skip if relation overlaps with quoted string
        if test_range.intersection(quoted_range):
            continue

        span_key = (start, end)
        parsed[span_key] = {
            "type": "Relation",
            "name": match.group(1),
            "span": (start, end),
        }

    for match in relations_pattern_end.finditer(belstr):
        (start, end) = match.span(1)
        log.debug(f"Relation-end {match}")
        end = end - 1  # adjust end to match actual end character index
        if start != end:
            test_range = set(range(start, end))
        else:
            test_range = set(start)

        # Skip if relation overlaps with quoted string
        if test_range.intersection(quoted_range):
            continue

        span_key = (start, end)
        parsed[span_key] = {
            "type": "Relation",
            "name": match.group(1),
            "span": (start, end),
        }

    return parsed, errors

Response: Parse BEL relations from BEL string into a nested BEL tree structure.


Instruction:
Explain what the following Python 3 code does
def parse_nested(
    bels: list, char_locs: CharLocs, parsed: Parsed, errors: Errors
) -> Tuple[Parsed, Errors]:
    """ Parse nested BEL object """

    for sp in char_locs[
        "nested_parens"
    ]:  # sp = start parenthesis, ep = end parenthesis
        ep, level = char_locs["nested_parens"][sp]
        if ep == -1:
            ep = len(bels) + 1
        parsed[(sp, ep)] = {"type": "Nested", "span": (sp, ep)}

    return parsed, errors

Response: Parse nested BEL object.