Instruction:
Implement a Python 3 function for
stripping lock quality from the internal table

Response:
def strip_lock_quality(self, idx):
        """strip(2 byte) lock quality
        :idx: int
        :return: int
            idx
        :return: int
        """
        idx = Radiotap.align(idx, 2)
        lock_quality, = struct.unpack_from('<H', self._rtap, idx)
        return idx + 2, lock_quality


Instruction:
Can you tell what is the following Python 3 function doing
def strip_tx_attenuation(self, idx):
        idx = Radiotap.align(idx, 2)
        tx_attenuation, = struct.unpack_from('<H', self._rtap, idx)
        return idx + 2, tx_attenuation

Response: strip tx_attenuation - strips the first byte from the internal array and returns the index and the last byte


Instruction:
Explain what the following Python 3 code does
def strip_db_tx_attenuation(self, idx):
        idx = Radiotap.align(idx, 2)
        db_tx_attenuation, = struct.unpack_from('<H', self._rtap, idx)
        return idx + 2, db_tx_attenuation

Response: strip db_tx_attenuation - strips db_tx_attenuation - return the index and db_tx_attenuation - 2 byte


Instruction:
How would you explain what the following Python 3 function does
def strip_dbm_tx_power(self, idx):
        idx = Radiotap.align(idx, 1)
        dbm_tx_power, = struct.unpack_from('<b', self._rtap, idx)
        return idx + 1, dbm_tx_power

Response: strip 1 byte from the internal table of DBM tx power


Instruction:
Write a Python 3 script to
strip 1 byte radiotap. antenna idx

Response:
def strip_antenna(self, idx):
        """strip(1 byte) radiotap.antenna
        :return: int
            idx
        :return: int
        """
        antenna, = struct.unpack_from('<B', self._rtap, idx)
        return idx + 1, antenna


Instruction:
Can you tell what is the following Python 3 function doing
def strip_db_antsignal(self, idx):
        db_antsignal, = struct.unpack_from('<B', self._rtap, idx)
        return idx + 1, db_antsignal

Response: strip 1 byte from radiotap. db_antsignal


Instruction:
Can you generate a brief explanation for the following Python 3 code
def strip_db_antnoise(self, idx):
        db_antnoise, = struct.unpack_from('<B', self._rtap, idx)
        return idx + 1, db_antnoise

Response:
strip 1 byte from radiotap. db_antnoise
            idx


Instruction:
Here you have a function in Python 3, explain what it does
def strip_rx_flags(self, idx):
        rx_flags = collections.namedtuple('rx_flags', ['reserved', 'badplcp'])

        idx = Radiotap.align(idx, 2)
        flags, = struct.unpack_from('<H', self._rtap, idx)
        flag_bits = format(flags, '08b')[::-1]
        rx_flags.reserved = int(flag_bits[0])
        rx_flags.badplcp = int(flag_bits[1])
        return idx + 2, rx_flags

Response: strips the rx flags from the internal structure


Instruction:
Explain what the following Python 3 code does
def strip_tx_flags(self, idx):
        idx = Radiotap.align(idx, 2)
        tx_flags, = struct.unpack_from('<B', self._rtap, idx)
        return idx + 1, tx_flags

Response: strip tx_flags - strips tx_flags from the internal structure


Instruction:
Create a Python 3 function to
strip 1 byte from the rts_retries structure and returns the index and the remaining rts_retries bytes.

Response:
def strip_rts_retries(self, idx):
        """strip(1 byte) rts_retries
        :idx: int
        :return: int
            idx
        :return: int
        """
        rts_retries, = struct.unpack_from('<B', self._rtap, idx)
        return idx + 1, rts_retries


Instruction:
Create a Python 3 function to
strip 1 byte from the data_retries field of the record

Response:
def strip_data_retries(self, idx):
        """strip(1 byte) data_retries
        :idx: int
        :return: int
            idx
        :return: int
        """
        data_retries, = struct.unpack_from('<B', self._rtap, idx)
        return idx + 1, data_retries


Instruction:
Can you generate a brief explanation for the following Python 3 code
def strip_xchannel(self, idx):
        xchannel = collections.namedtuple(
            'xchannel', ['flags', 'freq', 'channel', 'max_power'])

        flags = collections.namedtuple(
            'flags', ['turbo', 'cck', 'ofdm', 'two_g', 'five_g', 'passive',
                      'dynamic', 'gfsk', 'gsm', 'sturbo', 'hafl', 'quarter',
                      'ht_20', 'ht_40u', 'ht_40d'])

        idx = Radiotap.align(idx, 2)
        flag_val, freq, channel, max_power = struct.unpack_from('<lHBB', self._rtap, idx)

        xchannel.freq = freq
        xchannel.channel = channel
        xchannel.max_power = max_power

        bits = format(flag_val, '032b')[::-1]
        flags.turbo = int(bits[4])
        flags.cck = int(bits[5])
        flags.ofdm = int(bits[6])
        flags.two_g = int(bits[7])
        flags.five_g = int(bits[8])
        flags.passive = int(bits[9])
        flags.dynamic = int(bits[10])
        flags.gfsk = int(bits[11])
        flags.gsm = int(bits[12])
        flags.sturbo = int(bits[13])
        flags.half = int(bits[14])
        flags.quarter = int(bits[15])
        flags.ht_20 = int(bits[16])
        flags.ht_40u = int(bits[17])
        flags.ht_40d = int(bits[18])
        xchannel.flags = flags

        return idx + 8, xchannel

Response: Strip the X - Channel information from the current instance.


Instruction:
Write a Python 3 script for
stripping 8 byte ampdu from the internal structure

Response:
def strip_ampdu(self, idx):
        """strip(8 byte) radiotap.ampdu
        :idx: int
        :return: int
            idx
        :return: collections.namedtuple
        """
        ampdu = collections.namedtuple(
            'ampdu', ['reference', 'crc_val', 'reservered', 'flags'])
        flags = collections.namedtuple(
            'flags', ['report_zerolen', 'is_zerolen', 'lastknown', 'last',
                      'delim_crc_error'])

        idx = Radiotap.align(idx, 4)
        refnum, flag_vals, crc_val, reserved = struct.unpack_from('<LHBB', self._rtap, idx)
        ampdu.flags = flags
        ampdu.reference = refnum
        ampdu.crc_val = crc_val
        ampdu.reserved = reserved

        bits = format(flag_vals, '032b')[::-1]
        ampdu.flags.report_zerolen = int(bits[0])
        ampdu.flags.is_zerolen = int(bits[1])
        ampdu.flags.lastknown = int(bits[2])
        ampdu.flags.last = int(bits[3])
        ampdu.flags.delim_crc_error = int(bits[4])
        return idx + 8, ampdu


Instruction:
Given the following Python 3 function, write the documentation
def strip_vht(self, idx):
        vht = collections.namedtuple(
            'vht', ['known_bits', 'have_stbc', 'have_txop_ps', 'have_gi',
                    'have_sgi_nsym_da', 'have_ldpc_extra', 'have_beamformed',
                    'have_bw', 'have_gid', 'have_paid', 'stbc', 'txop_ps', 'gi',
                    'sgi_nysm_da', 'ldpc_extra', 'group_id', 'partial_id',
                    'beamformed', 'user_0', 'user_1', 'user_2', 'user_3'])
        user = collections.namedtuple('user', ['nss', 'mcs', 'coding'])

        idx = Radiotap.align(idx, 2)
        known, flags, bw = struct.unpack_from('<HBB', self._rtap, idx)
        mcs_nss_0, mcs_nss_1, mcs_nss_2, mcs_nss_3 = struct.unpack_from('<BBBB', self._rtap, idx + 4)
        coding, group_id, partial_id = struct.unpack_from('<BBH', self._rtap, idx + 8)

        known_bits = format(known, '032b')[::-1]
        vht.known_bits = known_bits
        vht.have_stbc = int(known_bits[0])         # Space Time Block Coding
        vht.have_txop_ps = int(known_bits[1])      # TXOP_PS_NOT_ALLOWD
        vht.have_gi = int(known_bits[2])           # Short/Long Guard Interval
        vht.have_sgi_nsym_da = int(known_bits[3])  # Short Guard Interval Nsym Disambiguation
        vht.have_ldpc_extra = int(known_bits[4])   # LDPC(Low Density Parity Check)
        vht.have_beamformed = int(known_bits[5])   # Beamformed
        vht.have_bw = int(known_bits[6])           # Bandwidth
        vht.have_gid = int(known_bits[7])          # Group ID
        vht.have_paid = int(known_bits[8])         # Partial AID

        flag_bits = format(flags, '032b')[::-1]
        vht.flag_bits = flag_bits
        vht.stbc = int(flag_bits[0])
        vht.txop_ps = int(flag_bits[1])
        vht.gi = int(flag_bits[2])
        vht.sgi_nysm_da = int(flag_bits[3])
        vht.ldpc_extra = int(flag_bits[4])
        vht.beamformed = int(flag_bits[5])
        vht.group_id = group_id
        vht.partial_id = partial_id

        vht.bw = bw
        vht.user_0 = user(None, None, None)
        vht.user_1 = user(None, None, None)
        vht.user_2 = user(None, None, None)
        vht.user_3 = user(None, None, None)
        for (i, mcs_nss) in enumerate([mcs_nss_0, mcs_nss_1, mcs_nss_2, mcs_nss_3]):
            if mcs_nss:
                nss = mcs_nss & 0xf0 >> 4
                mcs = (mcs_nss & 0xf0) >> 4
                coding = (coding & 2**i) >> i
                if i == 0:
                    vht.user_0 = user(nss, mcs, coding)
                elif i == 1:
                    vht.user_1 = user(nss, mcs, coding)
                elif i == 2:
                    vht.user_2 = user(nss, mcs, coding)
                elif i == 3:
                    vht.user_3 = user(nss, mcs, coding)

        return idx + 12, vht

Response: strips the vht entry from the internal buffer and returns the index in the internal buffer.


Instruction:
Explain what the following Python 3 code does
def extract_protocol(self):
        if self.present.mcs:
            return '.11n'

        if self.present.vht:
            return '.11ac'

        if self.present.channel and hasattr(self, 'chan'):
            if self.chan.five_g:
                if self.chan.ofdm:
                    return '.11a'
            elif self.chan.two_g:
                if self.chan.cck:
                    return '.11b'
                elif self.chan.ofdm or self.chan.dynamic:
                    return '.11g'
        return 'None'

Response: extract 802. 11 protocol from radiotap. channel. flags


Instruction:
Given the following Python 3 function, write the documentation
def get_shark_field(self, fields):
        keys, exist, out = None, {}, None

        if isinstance(fields, str):
            fields = [fields]
        elif not isinstance(fields, list):
            logging.error('invalid input type')
            return None

        out = dict.fromkeys(fields)

        if hasattr(self, '_shark_'):
            exist.update(self._shark_)

        if hasattr(self, '_s_shark_'):
            exist.update(self._s_shark_)

        if hasattr(self.radiotap, '_r_shark_'):
            exist.update(self.radiotap._r_shark_)

        keys = exist.keys()

        for elem in fields:
            if elem in keys:
                obj_field, tmp = exist[elem], None
                try:
                    tmp = operator.attrgetter(obj_field)(self)
                except AttributeError:
                    tmp = None
                if not tmp:
                    try:
                        tmp = operator.attrgetter(obj_field)(self.radiotap)
                    except AttributeError:
                        tmp = None
                out[elem] = tmp
        return out

Response: get parameters via wireshark syntax.


Instruction:
How would you explain what the following Python 3 function does
def get_mac_addr(mac_addr):
        mac_addr = bytearray(mac_addr)
        mac = b':'.join([('%02x' % o).encode('ascii') for o in mac_addr])
        return mac

Response:
converts bytes to mac addr format
            11 : 22 : 33. aa. cc. cz


Instruction:
Make a summary of the following Python 3 code
def strip_mac_addrs(self):
        qos_idx, seq_idx = 0, 0
        sa, ta, ra, da, bssid = None, None, None, None, None

        if self.to_ds == 1 and self.from_ds == 1:
            (ra, ta, da) = struct.unpack('!6s6s6s', self._packet[4:22])
            sa = struct.unpack('!6s', self._packet[24:30])[0]
            qos_idx = 30
            seq_idx = 22
        elif self.to_ds == 0 and self.from_ds == 1:
            (ra, ta, sa) = struct.unpack('!6s6s6s', self._packet[4:22])
            qos_idx = 24
            seq_idx = 22
        elif self.to_ds == 1 and self.from_ds == 0:
            (ra, ta, da) = struct.unpack('!6s6s6s', self._packet[4:22])
            qos_idx = 24
            seq_idx = 22
        elif self.to_ds == 0 and self.from_ds == 0:
            (ra, ta, bssid) = struct.unpack('!6s6s6s', self._packet[4:22])
            qos_idx = 24
            seq_idx = 22

        if ta is not None:
            ta = Wifi.get_mac_addr(ta)
        if ra is not None:
            ra = Wifi.get_mac_addr(ra)
        if sa is not None:
            sa = Wifi.get_mac_addr(sa)
        if da is not None:
            da = Wifi.get_mac_addr(da)
        if bssid is not None:
            bssid = Wifi.get_mac_addr(bssid)

        return seq_idx, qos_idx, sa, ta, ra, da, bssid

Response: strips the MAC addresses from the packet and returns the corresponding entry in the sequence control structure.


Instruction:
Can you tell what is the following Python 3 function doing
def strip_seq_cntrl(self, idx):
        seq_cntrl = struct.unpack('H', self._packet[idx:idx + 2])[0]
        seq_num = seq_cntrl >> 4
        frag_num = seq_cntrl & 0x000f
        return seq_num, frag_num

Response: strips the seq and fram number information.


Instruction:
Can you generate the documentation for the following Python 3 function
def strip_qos_cntrl(self, idx, prot_type):
        qos_cntrl, = struct.unpack('H', self._packet[idx:idx + 2])
        qos_cntrl_bits = format(qos_cntrl, '016b')[::-1]
        qos_pri = qos_cntrl & 0x000f
        qos_bit = int(qos_cntrl_bits[5])
        qos_ack = int(qos_cntrl_bits[6:8], 2)
        amsdupresent = 0
        if prot_type == '.11ac':
            amsdupresent = int(qos_cntrl_bits[7])
        return 2, qos_pri, qos_bit, qos_ack, amsdupresent

Response: strips the QoS control packet from the packet and returns the tuple.


Instruction:
Given the following Python 3 function, write the documentation
def strip_ccmp(self, idx):
        ccmp_extiv = None
        if len(self._packet[idx:]) >= 8:
            raw_bytes = self._packet[idx:idx + 8]
            ccmp_extiv, = struct.unpack_from('Q', raw_bytes, 0)
        return 8, ccmp_extiv

Response: strips the CCMP extended initialization vector from the packet at the given index


Instruction:
How would you code a function in Python 3 to
strip a single mac servis data unit from the packet at the given index

Response:
def strip_msdu(self, idx):
        """strip single mac servis data unit(msdu)
        see -> https://mrncciew.com/2014/11/01/cwap-802-11-data-frame-aggregation/
        :idx: int
        :return: dict
            msdu
        :return: int
            number of processed bytes
        """
        # length of msdu payload has to be multiple of 4,
        # this guaranteed with padding
        padding = 0
        len_payload = 0
        msdu = {
            'llc': {},
            'wlan.da': None,
            'wlan.sa': None,
            'payload': None,
            'length': 0
        }

        (da_mac, sa_mac) = struct.unpack('!6s6s', self._packet[idx:idx + 12])
        msdu['wlan.da'] = Wifi.get_mac_addr(da_mac)
        msdu['wlan.sa'] = Wifi.get_mac_addr(sa_mac)
        idx += 12
        msdu['length'] = struct.unpack('!H', self._packet[idx:idx + 2])[0]
        idx += 2
        offset, msdu['llc'] = self.strip_llc(idx)
        idx += offset
        len_payload = msdu['length'] - offset
        msdu['payload'] = self._packet[idx:idx + len_payload]
        padding = 4 - (len_payload % 4)
        return msdu, msdu['length'] + padding + 12


Instruction:
Make a summary of the following Python 3 code
def strip_llc(self, idx):
        llc = {}
        snap = 170
        llc_dsap = struct.unpack('B', self._packet[idx:idx + 1])[0]
        llc['dsap.dsap'] = llc_dsap >> 1
        llc['dsap.ig'] = llc_dsap & 0b01
        idx += 1
        llc_ssap = struct.unpack('B', self._packet[idx:idx + 1])[0]
        llc['ssap.sap'] = llc_ssap >> 1
        llc['ssap.cr'] = llc_ssap & 0b01
        idx += 1
        if llc_dsap == snap and llc_ssap == snap:
            llc_control = struct.unpack('B', self._packet[idx:idx + 1])[0]
            llc['control.u_modifier_cmd'] = llc_control >> 2
            llc['control.ftype'] = llc_control & 0x03
            idx += 1
            llc['organization_code'] = self._packet[idx:idx + 3]
            idx += 3
            llc['type'] = self._packet[idx:idx + 2]
            return 8, llc
        else:
            return 4, llc

Response: strips the 4 or 8 byte logical link control headers from the packet at the given index and returns the number of processed bytes and the object that is returned.


Instruction:
Can you generate the documentation for the following Python 3 function
def parse_tagged_params(raw_tagged_params):
        fcs_len = 4  # wlan.fcs (4 bytes)
        idx = 0
        tagged_params = []
        while idx < len(raw_tagged_params) - fcs_len:
            tag_num, tag_len = struct.unpack('BB', raw_tagged_params[idx:idx + 2])
            idx += 2
            if len(raw_tagged_params) >= idx + tag_len:
                param = {}
                param['number'], param['length'] = tag_num, tag_len
                payload = raw_tagged_params[idx:idx + tag_len]
                if tag_num in MNGMT_TAGS:
                    param['name'] = MNGMT_TAGS[tag_num]
                    if MNGMT_TAGS[tag_num] == 'TAG_VENDOR_SPECIFIC_IE':
                        param['payload'] = Management.parse_vendor_ie(payload)
                    else:
                        param['payload'] = payload
                else:
                    param['name'] = None
                tagged_params.append(param)
                idx += tag_len
            else:
                logging.warning('out tag length header points out of boundary')
                log_msg = 'index: {p_idx}, pack_len: {p_len}'
                log_msg = log_msg.format(p_idx=idx + tag_len,
                                         p_len=len(raw_tagged_params))
                logging.warning(log_msg)
                return 1, tagged_params
        return 0, tagged_params

Response:
parse tagged information elements wlan_mgt. tag
        which has generic type - length - value structure
        type - length - value structure
        length - value structure
osity - length of structure
osity - payload of payload of payload of tag


Instruction:
Make a summary of the following Python 3 code
def get_fixed_capabils(payload):
        if len(payload) != 2:
            return None
        capabils = {}
        fix_cap = struct.unpack('H', payload)[0]
        cap_bits = format(fix_cap, '016b')[::-1]
        capabils['ess'] = int(cap_bits[0])             # Extended Service Set
        capabils['ibss'] = int(cap_bits[1])            # Independent Basic Service Set
        capabils['priv'] = int(cap_bits[4])            # Privacy
        capabils['short_preamble'] = int(cap_bits[5])  # Short Preamble
        capabils['pbcc'] = int(cap_bits[6])            # Packet Binary Convolutional Code
        capabils['chan_agility'] = int(cap_bits[7])    # Channel Agility
        capabils['spec_man'] = int(cap_bits[8])        # Spectrum Management
        capabils['short_slot'] = int(cap_bits[10])     # Short Slot Time
        capabils['apsd'] = int(cap_bits[11])           # Automatic Power Save Delivery
        capabils['radio_meas'] = int(cap_bits[12])     # Radio Measurement
        capabils['dss_ofdm'] = int(cap_bits[13])       # Direct Spread Spectrum
        capabils['del_back'] = int(cap_bits[14])       # Delayed Block Acknowledgement
        capabils['imm_back'] = int(cap_bits[15])       # Immediate Block Acknowledgement
        return capabils

Response:
Returns dict of fixed capabilities from wlan_mgt. fixed. capabilities
       .


Instruction:
Write a Python 3 function for
parsing vendor specific information element oui -> organizationally unique identifier

Response:
def parse_vendor_ie(payload):
        """parse vendor specific information element
        oui -> organizationally unique identifier
        first 3 bytes of mac addresses
        see:https://www.wireshark.org/tools/oui-lookup.html
        strip wlan_mgt.tag.oui(3 bytes),
        wlan_mgt.tag.vendor.oui.type(1 byte)
        wlan_mgt.tag.vendor.data (varies)
        :payload: ctypes.structure
        :return: dict
            {'oui':00-11-22, 'oui_type':1, 'oui_data':ctypes.structure}
        """
        output = {}
        oui = struct.unpack('BBB', payload[0:3])
        oui = b'-'.join([('%02x' % o).encode('ascii') for o in oui])
        oui_type = struct.unpack('B', payload[3:4])[0]
        oui_data = payload[4:]
        output['oui'] = oui.upper()
        output['oui_type'] = oui_type
        output['oui_data'] = oui_data
        return output


Instruction:
Implement a function in Python 3 to
strip the fixed parameters from the payload and returns the timestamp interval and capabils

Response:
def strip_fixed_params(payload):
        """strip(12 byte) wlan_mgt.fixed.all
        :payload: ctypes.structure
        :return: int
            timestamp
        :return: int
            beacon interval
        :return: dict
            capabilities
        """
        if len(payload) != 12:
            return None, None, None
        idx = 0
        timestamp = Management.get_timestamp(payload[idx:idx + 8])
        idx += 8
        interval = Management.get_interval(payload[idx:idx + 2])
        idx += 2
        capabils = Management.get_fixed_capabils(payload[idx:idx + 2])
        return timestamp, interval, capabils


Instruction:
Explain what the following Python 3 code does
def is_valid_mac_oui(mac_block):
        if len(mac_block) != 8:
            return 0
        if ':' in mac_block:
            if len(mac_block.split(':')) != 3:
                return 0
        elif '-' in mac_block:
            if len(mac_block.split('-')) != 3:
                return 0
        return 1

Response:
checks whether the given mac block is in format of
        00 - 11 - 22 or 00 - 11 - 22 or 00 - 11 - 22 or 00 - 11 - 22.


Instruction:
Can you generate the documentation for the following Python 3 function
def set_fixed_capabils(self, capabils):
        self.ess = capabils['ess']
        self.ibss = capabils['ibss']
        self.priv = capabils['priv']
        self.short_preamble = capabils['short_preamble']
        self.pbcc = capabils['pbcc']
        self.chan_agility = capabils['chan_agility']
        self.spec_man = capabils['spec_man']
        self.short_slot = capabils['short_slot']
        self.apsd = capabils['apsd']
        self.radio_meas = capabils['radio_meas']
        self.dss_ofdm = capabils['dss_ofdm']
        self.del_back = capabils['del_back']
        self.imm_back = capabils['imm_back']

Response: set keys of capabils into fields of object


Instruction:
Given the following Python 3 function, write the documentation
def get_vendor_ies(self, mac_block=None, oui_type=None):
        vendor_ies = []
        if mac_block is not None:
            if Management.is_valid_mac_oui(mac_block):
                mac_block = mac_block.upper()
                if ':' in mac_block:
                    mac_block.replace(':', '-')
            else:
                logging.warning("invalid oui macblock")
                return None

        for elem in self.tagged_params:
            tag_num = elem['number']
            if MNGMT_TAGS[tag_num] == 'TAG_VENDOR_SPECIFIC_IE':
                if mac_block is None:
                    vendor_ies.append(elem)
                elif elem['payload']['oui'] == mac_block.encode('ascii'):
                    if oui_type is None:
                        vendor_ies.append(elem)
                    elif elem['payload']['oui_type'] == oui_type:
                        vendor_ies.append(elem)
        return vendor_ies

Response: get vendor information elements querying the tag


Instruction:
Here you have a function in Python 3, explain what it does
def get_shark_field(self, fields):
        out = super(BACK, self).get_shark_field(fields)
        out.update({'acked_seqs': self.acked_seqs,
                    'bitmap_str': self.bitmap_str})
        return out

Response: Returns a dictionary of the shark fields for the current object.


Instruction:
Create a Python 3 function to
strip the cntrl from the payload and returns the ackpolicy and multitid

Response:
def strip_cntrl(payload):
        """strip(2 byte) wlan.ba.control
        :payload: ctypes.structure
        :return: int
            multitid (tid: traffic indicator)
        :return: int
            ackpolicy
        """
        cntrl = struct.unpack('H', payload)[0]
        cntrl_bits = format(cntrl, '016b')[::-1]
        ackpolicy = int(cntrl_bits[0])
        multitid = int(cntrl_bits[1])
        return ackpolicy, multitid


Instruction:
Can you generate the documentation for the following Python 3 function
def strip_ssc(payload):
        ssc = struct.unpack('H', payload)[0]
        ssc_seq = ssc >> 4
        ssc_frag = ssc & 0x000f
        return ssc_seq, ssc_frag

Response: strips the SSC from the given payload and returns the sequence and fragment number of the SSC.


Instruction:
Implement a function in Python 3 to
strip 8 byte bitmap

Response:
def strip_bitmap_str(payload):
        """strip(8 byte) wlan.ba.bm
        :payload: ctypes.structure
        :return: str
            bitmap
        """
        bitmap = struct.unpack('BBBBBBBB', payload)
        bitmap_str = ''
        for elem in bitmap:
            bitmap_str += format(elem, '08b')[::-1]
        return bitmap_str


Instruction:
Create a Python 3 function to
extract acknowledged sequences from bitmap and starting sequence number.

Response:
def extract_acked_seqs(bitmap, ssc_seq):
        """extracts acknowledged sequences from bitmap and
        starting sequence number.
        :bitmap: str
        :ssc_seq: int
        :return: int[]
            acknowledged sequence numbers
        """
        acked_seqs = []
        for idx, val in enumerate(bitmap):
            if int(val) == 1:
                seq = (ssc_seq + idx) % 4096
                acked_seqs.append(seq)
        return acked_seqs


Instruction:
How would you implement a function in Python 3 that
sends a request and gets a response from the Plivo REST API

Response:
def request(self, path, method=None, data={}):
        """sends a request and gets a response from the Plivo REST API

        path: the URL (relative to the endpoint URL, after the /v1
        method: the HTTP method to use, defaults to POST
        data: for POST or PUT, a dict of data to send

        returns Plivo response in XML or raises an exception on error
        """
        if not path:
            raise ValueError('Invalid path parameter')
        if method and method not in ['GET', 'POST', 'DELETE', 'PUT']:
            raise NotImplementedError(
                'HTTP %s method not implemented' % method)

        if path[0] == '/':
            uri = self.url + path
        else:
            uri = self.url + '/' + path

        if APPENGINE:
            return json.loads(self._appengine_fetch(uri, data, method))
        return json.loads(self._urllib2_fetch(uri, data, method))


Instruction:
Can you implement a function in Python 3 that
rests Reload Plivo Config helper

Response:
def reload_config(self, call_params):
        """REST Reload Plivo Config helper
        """
        path = '/' + self.api_version + '/ReloadConfig/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Write a Python 3 function for
resting Reload Plivo Cache Config helper

Response:
def reload_cache_config(self, call_params):
        """REST Reload Plivo Cache Config helper
        """
        path = '/' + self.api_version + '/ReloadCacheConfig/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Write a Python 3 function that can
rest Group Call Helper

Response:
def group_call(self, call_params):
        """REST GroupCalls Helper
        """
        path = '/' + self.api_version + '/GroupCall/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Can you create a Python 3 function that
rests Transfer Live Call Helper

Response:
def transfer_call(self, call_params):
        """REST Transfer Live Call Helper
        """
        path = '/' + self.api_version + '/TransferCall/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Can you implement a function in Python 3 that
rests Hangup All Live Calls Helper

Response:
def hangup_all_calls(self):
        """REST Hangup All Live Calls Helper
        """
        path = '/' + self.api_version + '/HangupAllCalls/'
        method = 'POST'
        return self.request(path, method)


Instruction:
Here you have a function in Python 3, explain what it does
def hangup_call(self, call_params):
        path = '/' + self.api_version + '/HangupCall/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: Hangup a live call


Instruction:
Can you write a function in Python 3 where it
rests Schedule Hangup Helper

Response:
def schedule_hangup(self, call_params):
        """REST Schedule Hangup Helper
        """
        path = '/' + self.api_version + '/ScheduleHangup/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Explain what the following Python 3 code does
def cancel_scheduled_hangup(self, call_params):
        path = '/' + self.api_version + '/CancelScheduledHangup/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: This method cancels a scheduled hangup


Instruction:
Write a Python 3 script for
resting Conference Mute helper

Response:
def conference_mute(self, call_params):
        """REST Conference Mute helper
        """
        path = '/' + self.api_version + '/ConferenceMute/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Can you generate the documentation for the following Python 3 function
def play(self, call_params):
        path = '/' + self.api_version + '/Play/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST Play something on a Call Helper


Instruction:
Here you have a function in Python 3, explain what it does
def play_stop(self, call_params):
        path = '/' + self.api_version + '/PlayStop/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST PlayStop on a Call Helper


Instruction:
How would you implement a function in Python 3 that
rests Schedule playing something on a call Helper

Response:
def schedule_play(self, call_params):
        """REST Schedule playing something on a call Helper
        """
        path = '/' + self.api_version + '/SchedulePlay/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Can you tell what is the following Python 3 function doing
def cancel_scheduled_play(self, call_params):
        path = '/' + self.api_version + '/CancelScheduledPlay/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: This method cancels a scheduled play


Instruction:
Can you write a function in Python 3 where it
rests Add soundtouch audio effects to a Call

Response:
def sound_touch(self, call_params):
        """REST Add soundtouch audio effects to a Call
        """
        path = '/' + self.api_version + '/SoundTouch/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Create a Python 3 function to
rest Remove soundtouch audio effects on a Call

Response:
def sound_touch_stop(self, call_params):
        """REST Remove soundtouch audio effects on a Call
        """
        path = '/' + self.api_version + '/SoundTouchStop/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Create a Python 3 function to
rest Send digits to a Call

Response:
def send_digits(self, call_params):
        """REST Send digits to a Call
        """
        path = '/' + self.api_version + '/SendDigits/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Here you have a function in Python 3, explain what it does
def conference_unmute(self, call_params):
        path = '/' + self.api_version + '/ConferenceUnmute/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST Conference Unmute helper


Instruction:
How would you explain what the following Python 3 function does
def conference_kick(self, call_params):
        path = '/' + self.api_version + '/ConferenceKick/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST Conference Kick helper


Instruction:
Can you create a Python 3 function that
rests Conference Hangup helper

Response:
def conference_hangup(self, call_params):
        """REST Conference Hangup helper
        """
        path = '/' + self.api_version + '/ConferenceHangup/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Write a Python 3 script to
rest Conference Deaf helper

Response:
def conference_deaf(self, call_params):
        """REST Conference Deaf helper
        """
        path = '/' + self.api_version + '/ConferenceDeaf/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
How would you implement a function in Python 3 that
rests Conference Undeaf helper

Response:
def conference_undeaf(self, call_params):
        """REST Conference Undeaf helper
        """
        path = '/' + self.api_version + '/ConferenceUndeaf/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
How would you explain what the following Python 3 function does
def conference_record_start(self, call_params):
        path = '/' + self.api_version + '/ConferenceRecordStart/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST Conference Record Start


Instruction:
Can you tell what is the following Python 3 function doing
def conference_record_stop(self, call_params):
        path = '/' + self.api_version + '/ConferenceRecordStop/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: This method stops a conference record for a specific set of attributes.


Instruction:
How would you explain what the following Python 3 function does
def conference_play(self, call_params):
        path = '/' + self.api_version + '/ConferencePlay/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST Conference Play helper


Instruction:
Implement a Python 3 function for
resting Conference Speak helper

Response:
def conference_speak(self, call_params):
        """REST Conference Speak helper
        """
        path = '/' + self.api_version + '/ConferenceSpeak/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Explain what the following Python 3 code does
def conference_list(self, call_params):
        path = '/' + self.api_version + '/ConferenceList/'
        method = 'POST'
        return self.request(path, method, call_params)

Response: REST Conference List Helper


Instruction:
Create a Python 3 function for
resting Conference List Members Helper

Response:
def conference_list_members(self, call_params):
        """REST Conference List Members Helper
        """
        path = '/' + self.api_version + '/ConferenceListMembers/'
        method = 'POST'
        return self.request(path, method, call_params)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _xml(self, root):
        element = root.createElement(self.name)

        # Add attributes
        keys = self.attrs.keys()
        keys.sort()
        for a in keys:
            element.setAttribute(a, self.attrs[a])

        if self.body:
            text = root.createTextNode(self.body)
            element.appendChild(text)

        for c in self.elements:
            element.appendChild(c._xml(root))

        return element

Response: Return an XML element representing this element.


Instruction:
Create a Python 3 function to
validate a request from plivo

Response:
def validateRequest(self, uri, postVars, expectedSignature):
        """validate a request from plivo

        uri: the full URI that Plivo requested on your server
        postVars: post vars that Plivo sent with the request
        expectedSignature: signature in HTTP X-Plivo-Signature header

        returns true if the request passes validation, false if not
        """

        # append the POST variables sorted by key to the uri
        s = uri
        for k, v in sorted(postVars.items()):
            s += k + v

        # compute signature and compare signatures
        return (base64.encodestring(hmac.new(self.auth_token, s, sha1).digest()).\
            strip() == expectedSignature)


Instruction:
Can you generate the documentation for the following Python 3 function
def add_node(self,label):
      '''Return a node with label. Create node if label is new'''
      try:
          n = self._nodes[label]
      except KeyError:
          n = Node()
          n['label'] = label
          self._nodes[label]=n
      return n

Response: Return a node with label. Create node if it does not exist.


Instruction:
How would you code a function in Python 3 to
add an edge between nodes n1 and n2.

Response:
def add_edge(self, n1_label, n2_label,directed=False):
      """
      Get or create edges using get_or_create_node
      """
      n1 = self.add_node(n1_label)
      n2 = self.add_node(n2_label)
      e = Edge(n1, n2, directed)
      self._edges.append(e)
      return e


Instruction:
Create a Python 3 function to
parse a DOM into a Graph object.

Response:
def parse_dom(dom):
        """Parse dom into a Graph.

        :param dom: dom as returned by minidom.parse or minidom.parseString
        :return: A Graph representation
        """
        root = dom.getElementsByTagName("graphml")[0]
        graph = root.getElementsByTagName("graph")[0]
        name = graph.getAttribute('id')

        g = Graph(name)

        # # Get attributes
        # attributes = []
        # for attr in root.getElementsByTagName("key"):
        #     attributes.append(attr)

        # Get nodes
        for node in graph.getElementsByTagName("node"):
            n = g.add_node(id=node.getAttribute('id'))

            for attr in node.getElementsByTagName("data"):
                if attr.firstChild:
                    n[attr.getAttribute("key")] = attr.firstChild.data
                else:
                    n[attr.getAttribute("key")] = ""

        # Get edges
        for edge in graph.getElementsByTagName("edge"):
            source = edge.getAttribute('source')
            dest = edge.getAttribute('target')

            # source/target attributes refer to IDs: http://graphml.graphdrawing.org/xmlns/1.1/graphml-structure.xsd
            e = g.add_edge_by_id(source, dest)

            for attr in edge.getElementsByTagName("data"):
                if attr.firstChild:
                    e[attr.getAttribute("key")] = attr.firstChild.data
                else:
                    e[attr.getAttribute("key")] = ""

        return g


Instruction:
Explain what the following Python 3 code does
def parse_string(self, string):
        dom = minidom.parseString(string)
        return self.parse_dom(dom)

Response:
Parse a string into a Graph.
        object


Instruction:
Can you write a function in Python 3 where it
returns the other node that is in the tree

Response:
def node(self, node):
        """
        Return the other node
        """

        if node == self.node1:
            return self.node2
        elif node == self.node2:
            return self.node1
        else:
            return None


Instruction:
Given the following Python 3 function, write the documentation
def consume(self, **kwargs):
        kwargs.setdefault('block', True)
        try:
            while True:
                msg = self.get(**kwargs)
                if msg is None:
                    break
                yield msg
        except KeyboardInterrupt:
            print; return

Response:
Return a generator that yields whenever a message is waiting in the the
            queue. Will block if a KeyboardInterrupt is raised.


Instruction:
Write a Python 3 script for
getting a message from the queue.

Response:
def get(self, block=False, timeout=None):
        """Return a message from the queue. Example:

        >>> queue.get()
        'my message'
        >>> queue.get()
        'another message'

        :param block: whether or not to wait until a msg is available in
            the queue before returning; ``False`` by default
        :param timeout: when using :attr:`block`, if no msg is available
            for :attr:`timeout` in seconds, give up and return ``None``
        """
        if block:
            if timeout is None:
                timeout = 0
            msg = self.__redis.blpop(self.key, timeout=timeout)
            if msg is not None:
                msg = msg[1]
        else:
            msg = self.__redis.lpop(self.key)
        if msg is not None and self.serializer is not None:
            msg = self.serializer.loads(msg)
        return msg


Instruction:
Write a Python 3 script to
put one or more messages onto the queue.

Response:
def put(self, *msgs):
        """Put one or more messages onto the queue. Example:

        >>> queue.put("my message")
        >>> queue.put("another message")

        To put messages onto the queue in bulk, which can be significantly
        faster if you have a large number of messages:

        >>> queue.put("my message", "another message", "third message")
        """
        if self.serializer is not None:
            msgs = map(self.serializer.dumps, msgs)
        self.__redis.rpush(self.key, *msgs)


Instruction:
Explain what the following Python 3 code does
def worker(self, *args, **kwargs):
        def decorator(worker):
            @wraps(worker)
            def wrapper(*args):
                for msg in self.consume(**kwargs):
                    worker(*args + (msg,))
            return wrapper
        if args:
            return decorator(*args)
        return decorator

Response: Decorator for using a function as a queue worker.


Instruction:
Given the following Python 3 function, write the documentation
def to_bytes(self, previous: bytes):
        # First, validate the lengths.
        if len(self.conditions) != len(self.body):
            raise exc.CompileError("Conditions and body length mismatch!")

        bc = b""

        prev_len = len(previous)

        # Loop over the conditions and bodies
        for condition, body in zip(self.conditions, self.body):
            # Generate the conditional data.
            cond_bytecode = condition.to_bytecode(previous)
            bc += cond_bytecode
            # Complex calculation. First, generate the bytecode for all tokens in the body. Then
            # we calculate the len() of that. We create a POP_JUMP_IF_FALSE operation that jumps
            # to the instructions after the body code + 3 for the pop call. This is done for all
            # chained IF calls, as if it was an elif call. Else calls are not possible to be
            # auto-generated, but it is possible to emulate them using an elif call that checks
            # for the opposite of the above IF.

            # Call the _compile_func method from compiler to compile the body.
            body_bc = compiler.compile_bytecode(body)

            bdyl = len(body_bc)
            # Add together the lengths.
            gen_len = prev_len + len(cond_bytecode) + bdyl + 1
            # Generate the POP_JUMP_IF_FALSE instruction
            bc += generate_simple_call(tokens.POP_JUMP_IF_FALSE, gen_len)
            # Add the body_bc
            bc += body_bc

            # Update previous_len
            prev_len = len(previous) + len(bc)

        return bc

Response: Generate the Complex Code ahead. Comments have been added in as needed.


Instruction:
How would you explain what the following Python 3 function does
def to_bytes_35(self, previous: bytes):

        # Calculations ahead.
        bc = b""

        # Calculate the length of the iterator.
        it_bc = util.generate_bytecode_from_obb(self.iterator, previous)
        bc += it_bc

        # Push a get_iter on.
        bc += util.generate_bytecode_from_obb(tokens.GET_ITER, b"")
        prev_len = len(previous) + len(bc)
        # Calculate the bytecode for the body.
        body_bc = b""
        for op in self._body:
            # Add padding bytes to the bytecode to allow if blocks to work.
            padded_bc = previous
            # Add padding for SETUP_LOOP
            padded_bc += b"\x00\x00\x00"
            padded_bc += bc
            # Add padding for FOR_ITER
            padded_bc += b"\x00\x00\x00"
            # Add previous body
            padded_bc += body_bc
            body_bc += util.generate_bytecode_from_obb(op, padded_bc)

        # Add a JUMP_ABSOLUTE
        body_bc += util.generate_simple_call(tokens.JUMP_ABSOLUTE, prev_len + 3)

        # Add a POP_TOP
        body_bc += util.generate_bytecode_from_obb(tokens.POP_BLOCK, b"")

        # Calculate the right lengths.
        # Add a FOR_ITER, using len(body_bc)
        body_bc = util.generate_simple_call(tokens.FOR_ITER, len(body_bc) - 1) + body_bc
        # Add the SETUP_LOOP call
        bc = util.generate_simple_call(tokens.SETUP_LOOP, prev_len + len(body_bc) - 6) + bc + body_bc

        return bc

Response: A to - bytes version of the internal method.


Instruction:
Can you generate the documentation for the following Python 3 function
def to_bytes_36(self, previous: bytes):
        # Calculations ahead.
        bc = b""

        # Calculate the length of the iterator.
        it_bc = util.generate_bytecode_from_obb(self.iterator, previous)
        bc += it_bc

        bc += util.ensure_instruction(tokens.GET_ITER)

Response: A to - bytes specific to Python 3. 6 and above.


Instruction:
Can you write a function in Python 3 where it
runs the correct validator for given objects. Assumes all same type.

Response:
def validate_content(*objs):
    """Runs the correct validator for given `obj`ects. Assumes all same type"""
    from .main import Collection, Module
    validator = {
        Collection: cnxml.validate_collxml,
        Module: cnxml.validate_cnxml,
    }[type(objs[0])]
    return validator(*[obj.file for obj in objs])


Instruction:
Here you have a function in Python 3, explain what it does
def validate_litezip(struct):
    msgs = []

    def _fmt_err(err):
        return (Path(err.filename), "{}:{} -- {}: {}".format(*(err[1:])))

    obj_by_type = {}
    for obj in struct:
        if not is_valid_identifier(obj.id):
            msg = (obj.file.parent,
                   "{} is not a valid identifier".format(obj.id),)
            logger.info("{}: {}".format(*msg))
            msgs.append(msg)
        obj_by_type.setdefault(type(obj), []).append(obj)

    for obtype in obj_by_type:
        content_msgs = list([_fmt_err(err) for err in
                             validate_content(*obj_by_type[obtype])])
        for msg in content_msgs:
            logger.info("{}: {}".format(*msg))
        msgs.extend(content_msgs)
    return msgs

Response:
Validate the given litezip as struct.
    Returns a list of validation messages.


Instruction:
How would you code a function in Python 3 to
ensure that an instruction is Python 3. 6 + compatible.

Response:
def ensure_instruction(instruction: int) -> bytes:
    """
    Wraps an instruction to be Python 3.6+ compatible. This does nothing on Python 3.5 and below.

    This is most useful for operating on bare, single-width instructions such as
    ``RETURN_FUNCTION`` in a version portable way.

    :param instruction: The instruction integer to use.
    :return: A safe bytes object, if applicable.
    """
    if PY36:
        return instruction.to_bytes(2, byteorder="little")
    else:
        return instruction.to_bytes(1, byteorder="little")


Instruction:
Write a Python 3 function for
generating a simple call for the current node.

Response:
def generate_simple_call(opcode: int, index: int):
    """
    Generates a simple call, with an index for something.

    :param opcode: The opcode to generate.
    :param index: The index to use as an argument.
    :return:
    """
    bs = b""
    # add the opcode
    bs += opcode.to_bytes(1, byteorder="little")
    # Add the index
    if isinstance(index, int):
        if PY36:
            bs += index.to_bytes(1, byteorder="little")
        else:
            bs += index.to_bytes(2, byteorder="little")
    else:
        bs += index
    return bs


Instruction:
Here you have a function in Python 3, explain what it does
def generate_bytecode_from_obb(obb: object, previous: bytes) -> bytes:
    # Generates bytecode from a specified object, be it a validator or an int or bytes even.
    if isinstance(obb, pyte.superclasses._PyteOp):
        return obb.to_bytes(previous)
    elif isinstance(obb, (pyte.superclasses._PyteAugmentedComparator,
                          pyte.superclasses._PyteAugmentedValidator._FakeMathematicalOP)):
        return obb.to_bytes(previous)
    elif isinstance(obb, pyte.superclasses._PyteAugmentedValidator):
        obb.validate()
        return obb.to_load()
    elif isinstance(obb, int):
        return obb.to_bytes((obb.bit_length() + 7) // 8, byteorder="little") or b''
    elif isinstance(obb, bytes):
        return obb
    else:
        raise TypeError("`{}` was not a valid bytecode-encodable item".format(obb))

Response: Generates a bytecode from an object.


Instruction:
Can you generate the documentation for the following Python 3 function
def _get_const_info(const_index, const_list):
    argval = const_index
    if const_list is not None:
        try:
            argval = const_list[const_index]
        except IndexError:
            raise ValidationError("Consts value out of range: {}".format(const_index)) from None
    return argval, repr(argval)

Response: Helper to get optional details about a constant in a resource tree.


Instruction:
Write a Python 3 function that can
add arguments described in the usage string to the argument parser.

Response:
def add_to_parser(parser, usage, ignore_existing=False):
    """
    Add arguments described in the usage string to the
    parser. View more details at the <create_parser> docs

    Args:
        parser (ArgumentParser): parser to add arguments to
        usage (str): Usage string in the format described above
        ignore_existing (bool): Ignore any arguments that have already been defined
    """
    usage = '\n' + usage
    first_line = [i for i in usage.split('\n') if i][0]
    indent = ' ' * (len(first_line) - len(first_line.lstrip(' ')))
    usage = usage.replace('\n' + indent, '\n')
    usage = usage.replace('\n...', '')

    defaults = {}
    description, *descriptors = usage.split('\n:')
    description = description.replace('\n', ' ').strip()
    if description and (not parser.description or not ignore_existing):
        parser.description = description
    for descriptor in descriptors:
        try:
            options, *info = descriptor.split('\n')
            info = ' '.join(i for i in info if i).replace('    ', '')
            if options.count(' ') == 1:
                if options[0] == '-':
                    short, long = options.split(' ')
                    var_name = long.strip('-').replace('-', '_')
                    parser.add_argument(short, long, dest=var_name, action='store_true', help=info)
                    defaults[var_name] = False
                else:
                    short, typ = options.split(' ')
                    parser.add_argument(short, type=types[typ], help=info)
            else:
                short, long, typ, default = options.split(' ')
                info = info.rstrip() + '. Default: ' + default
                default = '' if default == '-' else default
                parser.add_argument(short, long, type=types[typ], default=default, help=info)
        except ArgumentError:
            if not ignore_existing:
                raise
        except Exception as e:
            print(e.__class__.__name__ + ': ' + str(e))
            print('While parsing:')
            print(descriptor)
            raise ValueError('Failed to create parser from usage string')


Instruction:
Implement a Python 3 function for
returning a byte string containing the bytecode of the given list of Pyte objects.

Response:
def compile_bytecode(code: list) -> bytes:
    """
    Compiles Pyte objects into a bytecode list.

    :param code: A list of objects to compile.
    :return: The computed bytecode.
    """
    bc = b""
    for i, op in enumerate(code):
        try:
            # Get the bytecode.
            if isinstance(op, _PyteOp) or isinstance(op, _PyteAugmentedComparator):
                bc_op = op.to_bytes(bc)
            elif isinstance(op, int):
                bc_op = op.to_bytes(1, byteorder="little")
            elif isinstance(op, bytes):
                bc_op = op
            else:
                raise CompileError("Could not compile code of type {}".format(type(op)))
            bc += bc_op
        except Exception as e:
            print("Fatal compiliation error on operator {i} ({op}).".format(i=i, op=op))
            raise e

    return bc


Instruction:
How would you implement a function in Python 3 that
simulates the actions of the stack.

Response:
def _simulate_stack(code: list) -> int:
    """
    Simulates the actions of the stack, to check safety.

    This returns the maximum needed stack.
    """

    max_stack = 0
    curr_stack = 0

    def _check_stack(ins):
        if curr_stack < 0:
            raise CompileError("Stack turned negative on instruction: {}".format(ins))
        if curr_stack > max_stack:
            return curr_stack

    # Iterate over the bytecode.
    for instruction in code:
        assert isinstance(instruction, dis.Instruction)
        if instruction.arg is not None:
            try:
                effect = dis.stack_effect(instruction.opcode, instruction.arg)
            except ValueError as e:
                raise CompileError("Invalid opcode `{}` when compiling"
                                   .format(instruction.opcode)) from e
        else:
            try:
                effect = dis.stack_effect(instruction.opcode)
            except ValueError as e:
                raise CompileError("Invalid opcode `{}` when compiling"
                                   .format(instruction.opcode)) from e
        curr_stack += effect
        # Re-check the stack.
        _should_new_stack = _check_stack(instruction)
        if _should_new_stack:
            max_stack = _should_new_stack

    return max_stack


Instruction:
How would you explain what the following Python 3 function does
def compile(code: list, consts: list, names: list, varnames: list,
            func_name: str = "<unknown, compiled>",
            arg_count: int = 0, kwarg_defaults: Tuple[Any] = (), use_safety_wrapper: bool = True):
    """
    Compiles a set of bytecode instructions into a working function, using Python's bytecode
    compiler.

    :param code: A list of bytecode instructions.
    :param consts: A list of constants to compile into the function.
    :param names: A list of names to compile into the function.
    :param varnames: A list of ``varnames`` to compile into the function.
    :param func_name: The name of the function to use.
    :param arg_count: The number of arguments this function takes. Must be ``<= len(varnames)``.
    :param kwarg_defaults: A tuple of defaults for kwargs.
    :param use_safety_wrapper: Use the safety wrapper? This hijacks SystemError to print better \
        stack traces.
    """
    varnames = tuple(varnames)
    consts = tuple(consts)
    names = tuple(names)

    # Flatten the code list.
    code = util.flatten(code)

    if arg_count > len(varnames):
        raise CompileError("arg_count > len(varnames)")

    if len(kwarg_defaults) > len(varnames):
        raise CompileError("len(kwarg_defaults) > len(varnames)")

    # Compile it.
    bc = compile_bytecode(code)

    dis.dis(bc)

    # Check for a final RETURN_VALUE.
    if PY36:
        # TODO: Add Python 3.6 check
        pass
    else:
        if bc[-1] != tokens.RETURN_VALUE:
            raise CompileError(
                "No default RETURN_VALUE. Add a `pyte.tokens.RETURN_VALUE` to the end of your "
                "bytecode if you don't need one.")

    # Set default flags
    flags = 1 | 2 | 64

    frame_data = inspect.stack()[1]

    if sys.version_info[0:2] > (3, 3):
        # Validate the stack.
        stack_size = _simulate_stack(dis._get_instructions_bytes(
            bc, constants=consts, names=names, varnames=varnames)
        )
    else:
        warnings.warn("Cannot check stack for safety.")
        stack_size = 99

    # Generate optimization warnings.
    _optimize_warn_pass(dis._get_instructions_bytes(bc, constants=consts, names=names, varnames=varnames))

    obb = types.CodeType(
        arg_count,  # Varnames - used for arguments.
        0,  # Kwargs are not supported yet
        len(varnames),  # co_nlocals -> Non-argument local variables
        stack_size,  # Auto-calculated
        flags,  # 67 is default for a normal function.
        bc,  # co_code - use the bytecode we generated.
        consts,  # co_consts
        names,  # co_names, used for global calls.
        varnames,  # arguments
        frame_data[1],  # use <unknown, compiled>
        func_name,  # co_name
        frame_data[2],  # co_firstlineno, ignore this.
        b'',  # https://svn.python.org/projects/python/trunk/Objects/lnotab_notes.txt
        (),  # freevars - no idea what this does
        ()  # cellvars - used for nested functions - we don't use these.
    )
    # Update globals
    f_globals = frame_data[0].f_globals

    # Create a function type.
    f = types.FunctionType(obb, f_globals)
    f.__name__ = func_name
    f.__defaults__ = kwarg_defaults

    if use_safety_wrapper:
        def __safety_wrapper(*args, **kwargs):
            try:
                return f(*args, **kwargs)
            except SystemError as e:
                if 'opcode' not in ' '.join(e.args):
                    # Re-raise any non opcode related errors.
                    raise
                msg = "Bytecode exception!" \
                      "\nFunction {} returned an invalid opcode." \
                      "\nFunction dissection:\n\n".format(f.__name__)
                # dis sucks and always prints to stdout
                # so we capture it
                file = io.StringIO()
                with contextlib.redirect_stdout(file):
                    dis.dis(f)
                msg += file.getvalue()
                raise SystemError(msg) from e

        returned_func = __safety_wrapper
        returned_func.wrapped = f
    else:
        returned_func = f

    # return the func
    return returned_func

Response: Compiles a list of bytecode instructions into a working function.


Instruction:
Implement a Python 3 function for
giving the parsed xml to an ElementTree parse the id from the content.

Response:
def _parse_document_id(elm_tree):
    """Given the parsed xml to an `ElementTree`,
    parse the id from the content.

    """
    xpath = '//md:content-id/text()'
    return [x for x in elm_tree.xpath(xpath, namespaces=COLLECTION_NSMAP)][0]


Instruction:
Explain what the following Python 3 code does
def _find_resources(directory, excludes=[]):
    return sorted([r for r in directory.glob('*')
                   if True not in [e(r) for e in excludes]])

Response: Find all the resource paths in the directory.


Instruction:
Write a Python 3 script to
parse the file structure given the path to a module directory.

Response:
def parse_module(path, excludes=None):
    """Parse the file structure to a data structure given the path to
    a module directory.

    """
    file = path / MODULE_FILENAME

    if not file.exists():
        raise MissingFile(file)
    id = _parse_document_id(etree.parse(file.open()))

    excludes = excludes or []
    excludes.extend([
        lambda filepath: filepath.name == MODULE_FILENAME,
    ])

    resources_paths = _find_resources(path, excludes=excludes)
    resources = tuple(_resource_from_path(res) for res in resources_paths)

    return Module(id, file, resources)


Instruction:
Make a summary of the following Python 3 code
def parse_collection(path, excludes=None):
    file = path / COLLECTION_FILENAME
    if not file.exists():
        raise MissingFile(file)
    id = _parse_document_id(etree.parse(file.open()))

    excludes = excludes or []
    excludes.extend([
        lambda filepath: filepath.name == COLLECTION_FILENAME,
        lambda filepath: filepath.is_dir(),
    ])
    resources_paths = _find_resources(path, excludes=excludes)
    resources = tuple(_resource_from_path(res) for res in resources_paths)

    return Collection(id, file, resources)

Response:
Parse a file structure given the path to
    a collection directory.


Instruction:
Implement a Python 3 function for
parsing a litezip file structure given the path to the litezip directory.

Response:
def parse_litezip(path):
    """Parse a litezip file structure to a data structure given the path
    to the litezip directory.

    """
    struct = [parse_collection(path)]
    struct.extend([parse_module(x) for x in path.iterdir()
                   if x.is_dir() and x.name.startswith('m')])
    return tuple(sorted(struct))


Instruction:
Make a summary of the following Python 3 code
def convert_completezip(path):
    for filepath in path.glob('**/index_auto_generated.cnxml'):
        filepath.rename(filepath.parent / 'index.cnxml')
        logger.debug('removed {}'.format(filepath))
    for filepath in path.glob('**/index.cnxml.html'):
        filepath.unlink()
    return parse_litezip(path)

Response: Converts a completezip file structure to a litezip file structure.


Instruction:
How would you code a function in Python 3 to
return a sequence of Instructions in a bytecode string.

Response:
def _get_instructions_bytes(code, varnames=None, names=None, constants=None,
                            cells=None, linestarts=None, line_offset=0):
    """Iterate over the instructions in a bytecode string.

    Generates a sequence of Instruction namedtuples giving the details of each
    opcode.  Additional information about the code's runtime environment
    (e.g. variable names, constants) can be specified using optional
    arguments.

    """
    labels = dis.findlabels(code)
    extended_arg = 0
    starts_line = None
    free = None
    # enumerate() is not an option, since we sometimes process
    # multiple elements on a single pass through the loop
    n = len(code)
    i = 0
    while i < n:
        op = code[i]
        offset = i
        if linestarts is not None:
            starts_line = linestarts.get(i, None)
            if starts_line is not None:
                starts_line += line_offset
        is_jump_target = i in labels
        i = i + 1
        arg = None
        argval = None
        argrepr = ''
        if op >= dis.HAVE_ARGUMENT:
            arg = code[i] + code[i + 1] * 256 + extended_arg
            extended_arg = 0
            i = i + 2
            if op == dis.EXTENDED_ARG:
                extended_arg = arg * 65536
            # Set argval to the dereferenced value of the argument when
            #  availabe, and argrepr to the string representation of argval.
            #    _disassemble_bytes needs the string repr of the
            #    raw name index for LOAD_GLOBAL, LOAD_CONST, etc.
            argval = arg
            if op in dis.hasconst:
                argval, argrepr = dis._get_const_info(arg, constants)
            elif op in dis.hasname:
                argval, argrepr = dis._get_name_info(arg, names)
            elif op in dis.hasjrel:
                argval = i + arg
                argrepr = "to " + repr(argval)
            elif op in dis.haslocal:
                argval, argrepr = dis._get_name_info(arg, varnames)
            elif op in dis.hascompare:
                argval = dis.cmp_op[arg]
                argrepr = argval
            elif op in dis.hasfree:
                argval, argrepr = dis._get_name_info(arg, cells)
            elif op in dis.hasnargs:
                argrepr = "%d positional, %d keyword pair" % (code[i - 2], code[i - 1])
        yield dis.Instruction(dis.opname[op], op,
                              arg, argval, argrepr,
                              offset, starts_line, is_jump_target)


Instruction:
Explain what the following Python 3 code does
def _disassemble(self, lineno_width=3, mark_as_current=False):
        fields = []
        # Column: Source code line number
        if lineno_width:
            if self.starts_line is not None:
                lineno_fmt = "%%%dd" % lineno_width
                fields.append(lineno_fmt % self.starts_line)
            else:
                fields.append(' ' * lineno_width)
        # Column: Current instruction indicator
        if mark_as_current:
            fields.append('-->')
        else:
            fields.append('   ')
        # Column: Jump target marker
        if self.is_jump_target:
            fields.append('>>')
        else:
            fields.append('  ')
        # Column: Instruction offset from start of code sequence
        fields.append(repr(self.offset).rjust(4))
        # Column: Opcode name
        fields.append(self.opname.ljust(20))
        # Column: Opcode argument
        if self.arg is not None:
            fields.append(repr(self.arg).rjust(5))
            # Column: Opcode argument details
            if self.argrepr:
                fields.append('(' + self.argrepr + ')')
        return ' '.join(fields).rstrip()

Response: Format instruction details for inclusion in disassembly output


Instruction:
Write a Python 3 script for
returning the intersection of two lists. Assumes the lists are sorted by start positions.

Response:
def intersection(l1, l2):
    '''Returns intersection of two lists.  Assumes the lists are sorted by start positions'''
    if len(l1) == 0 or len(l2) == 0:
        return []

    out = []
    l2_pos = 0

    for l in l1:
        while l2_pos < len(l2) and l2[l2_pos].end < l.start:
            l2_pos += 1

        if l2_pos == len(l2):
            break

        while l2_pos < len(l2) and l.intersects(l2[l2_pos]):
            out.append(l.intersection(l2[l2_pos]))
            l2_pos += 1

        l2_pos = max(0, l2_pos - 1)

    return out


Instruction:
Can you tell what is the following Python 3 function doing
def merge_overlapping_in_list(l):
    '''Sorts list, merges any overlapping intervals, and also adjacent intervals. e.g.
       [0,1], [1,2] would be merge to [0,.2].'''
    i = 0
    l.sort()

    while i < len(l) - 1:
        u = l[i].union(l[i+1])
        if u is not None:
            l[i] = u
            l.pop(i+1)
        else:
            i += 1

Response: Sorts list merges any overlapping intervals and also adjacent intervals. e. g. A list of all entries in the list are merged to [ 0 1 2 ).


Instruction:
Can you tell what is the following Python 3 function doing
def remove_contained_in_list(l):
    '''Sorts list in place, then removes any intervals that are completely
       contained inside another interval'''
    i = 0
    l.sort()

    while i < len(l) - 1:
       if l[i+1].contains(l[i]):
           l.pop(i)
       elif l[i].contains(l[i+1]):
           l.pop(i+1)
       else:
           i += 1

Response:
Sorts list in place then removes any intervals that are completely
       contained inside another interval


Instruction:
Write a Python 3 function that can
return the distance from the point to the interval. Zero if the point lies inside the interval.

Response:
def distance_to_point(self, p):
        '''Returns the distance from the point to the interval. Zero if the point lies inside the interval.'''
        if self.start <= p <= self.end:
            return 0
        else:
            return min(abs(self.start - p), abs(self.end - p))


Instruction:
Can you generate the documentation for the following Python 3 function
def intersects(self, i):
        '''Returns true iff this interval intersects the interval i'''
        return self.start <= i.end and i.start <= self.end

Response: Returns true iff this interval intersects the interval i


Instruction:
Write a Python 3 script for
returning true iff this interval contains the interval i

Response:
def contains(self, i):
        '''Returns true iff this interval contains the interval i'''
        return self.start <= i.start and i.end <= self.end


Instruction:
Can you generate the documentation for the following Python 3 function
def union(self, i):
        '''If intervals intersect, returns their union, otherwise returns None'''
        if self.intersects(i) or self.end + 1 == i.start or i.end + 1 == self.start:
            return Interval(min(self.start, i.start), max(self.end, i.end))
        else:
            return None

Response: Returns the union of two intervals.


Instruction:
How would you explain what the following Python 3 function does
def union_fill_gap(self, i):
        '''Like union, but ignores whether the two intervals intersect or not'''
        return Interval(min(self.start, i.start), max(self.end, i.end))

Response: Like union but ignores whether the two intervals intersect or not


Instruction:
Make a summary of the following Python 3 code
def intersection(self, i):
        '''If intervals intersect, returns their intersection, otherwise returns None'''
        if self.intersects(i):
            return Interval(max(self.start, i.start), min(self.end, i.end))
        else:
            return None

Response: Returns the intersection of two sets or None if there is no intersection.


Instruction:
How would you implement a function in Python 3 that
reads a FASTA or FASTQ file and returns a dictionary of all the sequence IDs that are available in memory.

Response:
def file_reader(fname, read_quals=False):
    '''Iterates over a FASTA or FASTQ file, yielding the next sequence in the file until there are no more sequences'''
    f = utils.open_file_read(fname)
    line = f.readline()
    phylip_regex = re.compile('^\s*[0-9]+\s+[0-9]+$')
    gbk_regex = re.compile('^LOCUS\s+\S')

    if line.startswith('>'):
        seq = Fasta()
        previous_lines[f] = line
    elif line.startswith('##gff-version 3'):
        seq = Fasta()
        # if a GFF file, need to skip past all the annotation
        # and get to the fasta sequences at the end of the file
        while not line.startswith('>'):
            line = f.readline()
            if not line:
                utils.close(f)
                raise Error('No sequences found in GFF file "' + fname + '"')

        seq = Fasta()
        previous_lines[f] = line
    elif line.startswith('ID   ') and line[5] != ' ':
        seq = Embl()
        previous_lines[f] = line
    elif gbk_regex.search(line):
        seq = Embl()
        previous_lines[f] = line
    elif line.startswith('@'):
        seq = Fastq()
        previous_lines[f] = line
    elif phylip_regex.search(line):
        # phylip format could be interleaved or not, need to look at next
        # couple of lines to figure that out. Don't expect these files to
        # be too huge, so just store all the sequences in memory
        number_of_seqs, bases_per_seq = line.strip().split()
        number_of_seqs = int(number_of_seqs)
        bases_per_seq = int(bases_per_seq)
        got_blank_line = False

        first_line = line
        seq_lines = []
        while 1:
            line = f.readline()
            if line == '':
                break
            elif line == '\n':
                got_blank_line = True
            else:
                seq_lines.append(line.rstrip())
        utils.close(f)

        if len(seq_lines) == 1 or len(seq_lines) == number_of_seqs:
            sequential = True
        elif seq_lines[0][10] != ' ' and seq_lines[1][10] == ' ':
            sequential = True
        else:
            sequential = False

        # if the 11th char of second sequence line is a space,  then the file is sequential, e.g.:
        # GAGCCCGGGC AATACAGGGT AT
        # as opposed to:
        # Salmo gairAAGCCTTGGC AGTGCAGGGT
        if sequential:
            current_id = None
            current_seq = ''
            for line in seq_lines:
                if len(current_seq) == bases_per_seq or len(current_seq) == 0:
                    if current_id is not None:
                        yield Fasta(current_id, current_seq.replace('-', ''))
                    current_seq = ''
                    current_id, new_bases = line[0:10].rstrip(), line.rstrip()[10:]
                else:
                    new_bases = line.rstrip()

                current_seq += new_bases.replace(' ','')

            yield Fasta(current_id, current_seq.replace('-', ''))
        else:
            # seaview files start all seqs at pos >=12. Other files start
            # their sequence at the start of the line
            if seq_lines[number_of_seqs + 1][0] == ' ':
                first_gap_pos = seq_lines[0].find(' ')
                end_of_gap = first_gap_pos
                while seq_lines[0][end_of_gap] == ' ':
                    end_of_gap += 1
                first_seq_base = end_of_gap
            else:
                first_seq_base = 10

            seqs = []
            for i in range(number_of_seqs):
                name, bases = seq_lines[i][0:first_seq_base].rstrip(), seq_lines[i][first_seq_base:]
                seqs.append(Fasta(name, bases))

            for i in range(number_of_seqs, len(seq_lines)):
                seqs[i%number_of_seqs].seq += seq_lines[i]

            for fa in seqs:
                fa.seq = fa.seq.replace(' ','').replace('-','')
                yield fa

        return
    elif line == '':
        utils.close(f)
        return
    else:
        utils.close(f)
        raise Error('Error determining file type from file "' + fname + '". First line is:\n' + line.rstrip())

    try:
        while seq.get_next_from_file(f, read_quals):
            yield seq
    finally:
        utils.close(f)


Instruction:
Make a summary of the following Python 3 code
def subseq(self, start, end):
        '''Returns Fasta object with the same name, of the bases from start to end, but not including end'''
        return Fasta(self.id, self.seq[start:end])

Response: Returns a Fasta object with the same name of the bases from start to end


Instruction:
Implement a Python 3 function for
getting the prefix and suffix of a capillary read e. g. xxxxx. p1k or xxxxx. q1k Returns a tuple ( prefix suffix )

Response:
def split_capillary_id(self):
        '''Gets the prefix and suffix of an name of a capillary read, e.g. xxxxx.p1k or xxxx.q1k. Returns a tuple (prefix, suffx)'''
        try:
            a = self.id.rsplit('.', 1)
            if a[1].startswith('p'):
                dir = 'fwd'
            elif a[1].startswith('q'):
                dir = 'rev'
            else:
                dir = 'unk'

            return {'prefix': a[0], 'dir': dir, 'suffix':a[1]}
        except:
            raise Error('Error in split_capillary_id() on ID', self.id)


Instruction:
Given the following Python 3 function, write the documentation
def expand_nucleotides(self):
        '''Assumes sequence is nucleotides. Returns list of all combinations of redundant nucleotides. e.g. R is A or G, so CRT would have combinations CAT and CGT'''
        s = list(self.seq)
        for i in range(len(s)):
            if s[i] in redundant_nts:
                s[i] = ''.join(redundant_nts[s[i]])

        seqs = []
        for x in itertools.product(*s):
            seqs.append(Fasta(self.id + '.' + str(len(seqs) + 1), ''.join(x)))
        return seqs

Response: Assumes sequence is nucleotides. Returns list of all combinations of redundant nucleotides. e. g. R is A or G so CRT would have combinations CAT and CGT.


Instruction:
Make a summary of the following Python 3 code
def strip_illumina_suffix(self):
        '''Removes any trailing /1 or /2 off the end of the name'''
        if self.id.endswith('/1') or self.id.endswith('/2'):
            self.id = self.id[:-2]

Response: Removes any trailing illumina suffix off the end of the name


Instruction:
Implement a function in Python 3 to
return true if the sequence is all Ns

Response:
def is_all_Ns(self, start=0, end=None):
        '''Returns true if the sequence is all Ns (upper or lower case)'''
        if end is not None:
            if start > end:
                raise Error('Error in is_all_Ns. Start coord must be <= end coord')
            end += 1
        else:
            end = len(self)

        if len(self) == 0:
            return False
        else:
            return re.search('[^Nn]', self.seq[start:end]) is None


Instruction:
Can you generate a brief explanation for the following Python 3 code
def add_insertions(self, skip=10, window=1, test=False):
        '''Adds a random base within window bases around every skip bases. e.g. skip=10, window=1 means a random base added somwhere in theintervals [9,11], [19,21] ... '''
        assert 2 * window < skip
        new_seq = list(self.seq)
        for i in range(len(self) - skip, 0, -skip):
            pos = random.randrange(i - window, i + window + 1)
            base = random.choice(['A', 'C', 'G', 'T'])
            if test:
                base = 'N'
            new_seq.insert(pos, base)

        self.seq = ''.join(new_seq)

Response: Adds a random base within window bases around every skip bases. e. g. a random base added somwhere in the intervals [ 9 11 19 21...


Instruction:
Can you generate a brief explanation for the following Python 3 code
def replace_bases(self, old, new):
        '''Replaces all occurrences of 'old' with 'new' '''
        self.seq = self.seq.replace(old, new)

Response: Replaces all occurrences of old with new


Instruction:
Given the following Python 3 function, write the documentation
def replace_interval(self, start, end, new):
        '''Replaces the sequence from start to end with the sequence "new"'''
        if start > end or start > len(self) - 1 or end > len(self) - 1:
            raise Error('Error replacing bases ' + str(start) + '-' + str(end) + ' in sequence ' + self.id)

        self.seq = self.seq[0:start] + new + self.seq[end + 1:]

Response: Replaces the sequence from start to end with the sequence new


Instruction:
Can you implement a function in Python 3 that
finds the positions of all gaps in the sequence that are at least min_length long. Returns a list of Intervals. Coords are zero - based

Response:
def gaps(self, min_length = 1):
        '''Finds the positions of all gaps in the sequence that are at least min_length long. Returns a list of Intervals. Coords are zero-based'''
        gaps = []
        regex = re.compile('N+', re.IGNORECASE)
        for m in regex.finditer(self.seq):
             if m.span()[1] - m.span()[0] + 1 >= min_length:
                 gaps.append(intervals.Interval(m.span()[0], m.span()[1] - 1))
        return gaps


Instruction:
Write a Python 3 function that can
find the coordinates of contigs i. e. everything that s not a gap ( N or n. Returns a list of Intervals. Coords are zero - based.

Response:
def contig_coords(self):
        '''Finds coords of contigs, i.e. everything that's not a gap (N or n). Returns a list of Intervals. Coords are zero-based'''
        # contigs are the opposite of gaps, so work out the coords from the gap coords
        gaps = self.gaps()

        if len(gaps) == 0:
            return [intervals.Interval(0, len(self) - 1)]

        coords = [0]
        for g in gaps:
            if g.start == 0:
                coords = [g.end + 1]
            else:
                coords += [g.start - 1, g.end + 1]

        if coords[-1] < len(self):
            coords.append(len(self) - 1)

        return [intervals.Interval(coords[i], coords[i+1]) for i in range(0, len(coords)-1,2)]


Instruction:
Write a Python 3 function for
returning a list of ORFs that the sequence has starting on the given frame.

Response:
def orfs(self, frame=0, revcomp=False):
        '''Returns a list of ORFs that the sequence has, starting on the given
           frame. Each returned ORF is an interval.Interval object.
           If revomp=True, then finds the ORFs of the reverse complement
           of the sequence.'''
        assert frame in [0,1,2]
        if revcomp:
            self.revcomp()

        aa_seq = self.translate(frame=frame).seq.rstrip('X')
        if revcomp:
            self.revcomp()

        orfs = _orfs_from_aa_seq(aa_seq)
        for i in range(len(orfs)):
            if revcomp:
                start = len(self) - (orfs[i].end * 3 + 3) - frame
                end = len(self) - (orfs[i].start * 3) - 1 - frame
            else:
                start = orfs[i].start * 3 + frame
                end = orfs[i].end * 3 + 2 + frame

            orfs[i] = intervals.Interval(start, end)

        return orfs


Instruction:
Given the following Python 3 function, write the documentation
def is_complete_orf(self):
        '''Returns true iff length is >= 6, is a multiple of 3, and there is exactly one stop codon in the sequence and it is at the end'''
        if len(self) %3 != 0 or len(self) < 6:
            return False

        orfs = self.orfs()
        complete_orf = intervals.Interval(0, len(self) - 1)
        for orf in orfs:
            if orf == complete_orf:
                return True
        return False

Response: Returns true iff length is > = 6 and there is exactly one stop codon in the sequence and it is at the end.


Instruction:
Explain what the following Python 3 code does
def looks_like_gene(self):
        '''Returns true iff: length >=6, length is a multiple of 3, first codon is start, last codon is a stop and has no other stop codons'''
        return self.is_complete_orf() \
          and len(self) >= 6 \
          and len(self) %3 == 0 \
          and self.seq[0:3].upper() in genetic_codes.starts[genetic_code]

Response: Returns true iff the sequence looks like a gene


Instruction:
Write a Python 3 script for
trying to make into a gene sequence. Returns a tuple ( new sequence strand frame ). Returns None.

Response:
def make_into_gene(self):
        '''Tries to make into a gene sequence. Tries all three reading frames and both strands. Returns a tuple (new sequence, strand, frame) if it was successful. Otherwise returns None.'''
        for reverse in [True, False]:
            for frame in range(3):
                new_seq = copy.copy(self)
                if reverse:
                    new_seq.revcomp()
                new_seq.seq = new_seq[frame:]
                if len(new_seq) % 3:
                    new_seq.seq = new_seq.seq[:-(len(new_seq) % 3)]

                new_aa_seq = new_seq.translate()
                if len(new_aa_seq) >= 2 and new_seq[0:3] in genetic_codes.starts[genetic_code] and new_aa_seq[-1] == '*' and '*' not in new_aa_seq[:-1]:
                    strand = '-' if reverse else '+'
                    return new_seq, strand, frame

        return None


Instruction:
Implement a function in Python 3 to
remove first start/'end bases off the start of the sequence

Response:
def trim(self, start, end):
        '''Removes first 'start'/'end' bases off the start/end of the sequence'''
        self.seq = self.seq[start:len(self.seq) - end]


Instruction:
Write a Python 3 script to
return a Fastq object.

Response:
def to_Fastq(self, qual_scores):
        '''Returns a Fastq object. qual_scores expected to be a list of numbers, like you would get in a .qual file'''
        if len(self) != len(qual_scores):
            raise Error('Error making Fastq from Fasta, lengths differ.', self.id)
        return Fastq(self.id, self.seq, ''.join([chr(max(0, min(x, 93)) + 33) for x in qual_scores]))


Instruction:
How would you explain what the following Python 3 function does
def translate(self, frame=0):
        '''Returns a Fasta sequence, translated into amino acids. Starts translating from 'frame', where frame expected to be 0,1 or 2'''
        return Fasta(self.id, ''.join([genetic_codes.codes[genetic_code].get(self.seq[x:x+3].upper(), 'X') for x in range(frame, len(self)-1-frame, 3)]))

Response: Returns a Fasta sequence translated into amino acids. Starts translating from frame where frame expected to be 0 1 or 2


Instruction:
Write a Python 3 script to
return the GC content for the sequence.

Response:
def gc_content(self, as_decimal=True):
        """Returns the GC content for the sequence.
        Notes:
            This method ignores N when calculating the length of the sequence.
            It does not, however ignore other ambiguous bases. It also only
            includes the ambiguous base S (G or C). In this sense the method is
            conservative with its calculation.

        Args:
            as_decimal (bool): Return the result as a decimal. Setting to False
            will return as a percentage. i.e for the sequence GCAT it will
            return 0.5 by default and 50.00 if set to False.

        Returns:
            float: GC content calculated as the number of G, C, and S divided
            by the number of (non-N) bases (length).

        """
        gc_total = 0.0
        num_bases = 0.0
        n_tuple = tuple('nN')
        accepted_bases = tuple('cCgGsS')

        # counter sums all unique characters in sequence. Case insensitive.
        for base, count in Counter(self.seq).items():

            # dont count N in the number of bases
            if base not in n_tuple:
                num_bases += count

                if base in accepted_bases:  # S is a G or C
                    gc_total += count

        gc_content = gc_total / num_bases

        if not as_decimal:  # return as percentage
            gc_content *= 100

        return gc_content


Instruction:
Given the following Python 3 function, write the documentation
def subseq(self, start, end):
        '''Returns Fastq object with the same name, of the bases from start to end, but not including end'''
        return Fastq(self.id, self.seq[start:end], self.qual[start:end])

Response: Returns a new Fastq object with the same name of the bases from start to end


Instruction:
Can you generate a brief explanation for the following Python 3 code
def trim(self, start, end):
        '''Removes first 'start'/'end' bases off the start/end of the sequence'''
        super().trim(start, end)
        self.qual = self.qual[start:len(self.qual) - end]

Response: Removes first start/'end bases off the start of the sequence


Instruction:
Explain what the following Python 3 code does
def trim_Ns(self):
        '''Removes any leading or trailing N or n characters from the sequence'''
        # get index of first base that is not an N
        i = 0
        while i < len(self) and self.seq[i] in 'nN':
            i += 1

        # strip off start of sequence and quality
        self.seq = self.seq[i:]
        self.qual = self.qual[i:]

        # strip the ends
        self.seq = self.seq.rstrip('Nn')
        self.qual = self.qual[:len(self.seq)]

Response: Removes any leading or trailing N or n characters from the sequence


Instruction:
Implement a Python 3 function for
replacing the sequence from start to end with the sequence new

Response:
def replace_interval(self, start, end, new, qual_string):
        '''Replaces the sequence from start to end with the sequence "new"'''
        if len(new) != len(qual_string):
            raise Error('Length of new seq and qual string in replace_interval() must be equal. Cannot continue')
        super().replace_interval(start, end, new)
        self.qual = self.qual[0:start] + qual_string + self.qual[end + 1:]


Instruction:
Can you write a function in Python 3 where it
returns a Fasta sequence translated into amino acids. Starts translating from frame where frame expected to be 0 1 2

Response:
def translate(self):
        '''Returns a Fasta sequence, translated into amino acids. Starts translating from 'frame', where frame expected to be 0,1 or 2'''
        fa = super().translate()
        return Fastq(fa.id, fa.seq, 'I'*len(fa.seq))


Instruction:
Can you tell what is the following Python 3 function doing
def acgtn_only(infile, outfile):
    '''Replace every non-acgtn (case insensitve) character with an N'''
    f = utils.open_file_write(outfile)
    for seq in sequences.file_reader(infile):
        seq.replace_non_acgt()
        print(seq, file=f)
    utils.close(f)

Response: Replace every non - acgtn character with an N


Instruction:
Can you implement a function in Python 3 that
converts a CAF file to fastq.

Response:
def caf_to_fastq(infile, outfile, min_length=0, trim=False):
    '''Convert a CAF file to fastq. Reads shorter than min_length are not output. If clipping information is in the CAF file (with a line Clipping QUAL ...) and trim=True, then trim the reads'''
    caf_reader = caf.file_reader(infile)
    fout = utils.open_file_write(outfile)

    for c in caf_reader:
        if trim:
            if c.clip_start is not None and c.clip_end is not None:
                c.seq.seq = c.seq.seq[c.clip_start:c.clip_end + 1]
                c.seq.qual = c.seq.qual[c.clip_start:c.clip_end + 1]
            else:
                print('Warning: no clipping info for sequence', c.id, file=sys.stderr)

        if len(c.seq) >= min_length:
            print(c.seq, file=fout)

    utils.close(fout)


Instruction:
Can you tell what is the following Python 3 function doing
def count_sequences(infile):
    '''Returns the number of sequences in a file'''
    seq_reader = sequences.file_reader(infile)
    n = 0
    for seq in seq_reader:
        n += 1
    return n

Response: Returns the number of sequences in a file


Instruction:
Explain what the following Python 3 code does
def interleave(infile_1, infile_2, outfile, suffix1=None, suffix2=None):
    '''Makes interleaved file from two sequence files. If used, will append suffix1 onto end
    of every sequence name in infile_1, unless it already ends with suffix1. Similar for sufffix2.'''
    seq_reader_1 = sequences.file_reader(infile_1)
    seq_reader_2 = sequences.file_reader(infile_2)
    f_out = utils.open_file_write(outfile)

    for seq_1 in seq_reader_1:
        try:
            seq_2 = next(seq_reader_2)
        except:
            utils.close(f_out)
            raise Error('Error getting mate for sequence', seq_1.id, ' ... cannot continue')

        if suffix1 is not None and not seq_1.id.endswith(suffix1):
            seq_1.id += suffix1
        if suffix2 is not None and not seq_2.id.endswith(suffix2):
            seq_2.id += suffix2

        print(seq_1, file=f_out)
        print(seq_2, file=f_out)

    try:
        seq_2 = next(seq_reader_2)
    except:
        seq_2 = None

    if seq_2 is not None:
        utils.close(f_out)
        raise Error('Error getting mate for sequence', seq_2.id, ' ... cannot continue')

    utils.close(f_out)

Response:
Makes an interleaved file from two sequence files. If used will append suffix1 onto end
    of every sequence name in infile_1 unless it already ends with suffix2. Similar for sufffix2.


Instruction:
Can you create a Python 3 function that
makes a multi fasta file of random sequences all the same length

Response:
def make_random_contigs(contigs, length, outfile, name_by_letters=False, prefix='', seed=None, first_number=1):
    '''Makes a multi fasta file of random sequences, all the same length'''
    random.seed(a=seed)
    fout = utils.open_file_write(outfile)
    letters = list('ABCDEFGHIJKLMNOPQRSTUVWXYZ')
    letters_index = 0

    for i in range(contigs):
        if name_by_letters:
            name = letters[letters_index]
            letters_index += 1
            if letters_index == len(letters):
                letters_index = 0
        else:
            name = str(i + first_number)

        fa = sequences.Fasta(prefix + name, ''.join([random.choice('ACGT') for x in range(length)]))
        print(fa, file=fout)

    utils.close(fout)


Instruction:
Can you implement a function in Python 3 that
returns the mean length of the sequences in the input file. By default uses all sequences. By default uses all sequences.

Response:
def mean_length(infile, limit=None):
    '''Returns the mean length of the sequences in the input file. By default uses all sequences. To limit to the first N sequences, use limit=N'''
    total = 0
    count = 0
    seq_reader = sequences.file_reader(infile)
    for seq in seq_reader:
        total += len(seq)
        count += 1
        if limit is not None and count >= limit:
            break

    assert count > 0
    return total / count


Instruction:
Implement a function in Python 3 to
take a multi fasta file and writes a new file that contains just one sequence with the original sequences catted together preserving their order

Response:
def merge_to_one_seq(infile, outfile, seqname='union'):
    '''Takes a multi fasta or fastq file and writes a new file that contains just one sequence, with the original sequences catted together, preserving their order'''
    seq_reader = sequences.file_reader(infile)
    seqs = []

    for seq in seq_reader:
        seqs.append(copy.copy(seq))

    new_seq = ''.join([seq.seq for seq in seqs])

    if type(seqs[0]) == sequences.Fastq:
        new_qual = ''.join([seq.qual for seq in seqs])
        seqs[:] = []
        merged = sequences.Fastq(seqname, new_seq, new_qual)
    else:
        merged = sequences.Fasta(seqname, new_seq)
        seqs[:] = []

    f = utils.open_file_write(outfile)
    print(merged, file=f)
    utils.close(f)


Instruction:
Write a Python 3 function for
making a file of contigs from scaffolds by splitting at every N. contig.

Response:
def scaffolds_to_contigs(infile, outfile, number_contigs=False):
    '''Makes a file of contigs from scaffolds by splitting at every N.
       Use number_contigs=True to add .1, .2, etc onto end of each
       contig, instead of default to append coordinates.'''
    seq_reader = sequences.file_reader(infile)
    fout = utils.open_file_write(outfile)

    for seq in seq_reader:
        contigs = seq.contig_coords()
        counter = 1
        for contig in contigs:
            if number_contigs:
                name = seq.id + '.' + str(counter)
                counter += 1
            else:
                name = '.'.join([seq.id, str(contig.start + 1), str(contig.end + 1)])
            print(sequences.Fasta(name, seq[contig.start:contig.end+1]), file=fout)

    utils.close(fout)


Instruction:
Can you generate the documentation for the following Python 3 function
def sort_by_size(infile, outfile, smallest_first=False):
    '''Sorts input sequence file by biggest sequence first, writes sorted output file. Set smallest_first=True to have smallest first'''
    seqs = {}
    file_to_dict(infile, seqs)
    seqs = list(seqs.values())
    seqs.sort(key=lambda x: len(x), reverse=not smallest_first)
    fout = utils.open_file_write(outfile)
    for seq in seqs:
        print(seq, file=fout)
    utils.close(fout)

Response: Sorts input sequence file by biggest sequence first writes sorted output file.


Instruction:
Implement a function in Python 3 to
sort input sequence file by sort - d - k1 1 writes sorted output file.

Response:
def sort_by_name(infile, outfile):
    '''Sorts input sequence file by sort -d -k1,1, writes sorted output file.'''
    seqs = {}
    file_to_dict(infile, seqs)
    #seqs = list(seqs.values())
    #seqs.sort()
    fout = utils.open_file_write(outfile)
    for name in sorted(seqs):
        print(seqs[name], file=fout)
    utils.close(fout)


Instruction:
Explain what the following Python 3 code does
def to_fastg(infile, outfile, circular=None):
    '''Writes a FASTG file in SPAdes format from input file. Currently only whether or not a sequence is circular is supported. Put circular=set of ids, or circular=filename to make those sequences circular in the output. Puts coverage=1 on all contigs'''
    if circular is None:
        to_circularise = set()
    elif type(circular) is not set:
        f = utils.open_file_read(circular)
        to_circularise = set([x.rstrip() for x in f.readlines()])
        utils.close(f)
    else:
        to_circularise = circular

    seq_reader = sequences.file_reader(infile)
    fout = utils.open_file_write(outfile)
    nodes = 1

    for seq in seq_reader:
        new_id = '_'.join([
            'NODE', str(nodes),
            'length', str(len(seq)),
            'cov', '1',
            'ID', seq.id
        ])

        if seq.id in to_circularise:
            seq.id = new_id + ':' + new_id + ';'
            print(seq, file=fout)
            seq.revcomp()
            seq.id = new_id + "':" + new_id + "';"
            print(seq, file=fout)
        else:
            seq.id = new_id + ';'
            print(seq, file=fout)
            seq.revcomp()
            seq.id = new_id + "';"
            print(seq, file=fout)

        nodes += 1

    utils.close(fout)

Response: Writes a FASTG file in SPAdes format from input file. Currently only supports circular sequences.


Instruction:
How would you implement a function in Python 3 that
returns a dictionary of positions of the start of each sequence in the file as sgi is a dictionary of name = > length of each sequence in the file.

Response:
def length_offsets_from_fai(fai_file):
    '''Returns a dictionary of positions of the start of each sequence, as
       if all the sequences were catted into one sequence.
       eg if file has three sequences, seq1 10bp, seq2 30bp, seq3 20bp, then
       the output would be: {'seq1': 0, 'seq2': 10, 'seq3': 40}'''
    positions = {}
    total_length = 0
    f = utils.open_file_read(fai_file)

    for line in f:
        try:
            (name, length) = line.rstrip().split()[:2]
            length = int(length)
        except:
            raise Error('Error reading the following line of fai file ' + fai_file + '\n' + line)

        positions[name] = total_length
        total_length += length

    utils.close(f)
    return positions