Instruction:
How would you code a function in Python 3 to
get neighbors from a CURIE.

Response:
def getNeighbors(self, id, depth=1, blankNodes='false', relationshipType=None, direction='BOTH', project='*', callback=None, output='application/json'):
        """ Get neighbors from: /graph/neighbors/{id}
            Arguments:
            id: This ID should be either a CURIE or an IRI
            depth: How far to traverse neighbors
            blankNodes: Traverse blank nodes
            relationshipType: Which relationship to traverse
            direction: Which direction to traverse: INCOMING, OUTGOING, BOTH (default). Only used if relationshipType is specified.
            project: Which properties to project. Defaults to '*'.
            callback: Name of the JSONP callback ('fn' by default). Supplying this parameter or
            requesting a javascript media type will cause a JSONP response to be
            rendered.
            outputs:
                application/json
                application/graphson
                application/xml
                application/graphml+xml
                application/xgmml
                text/gml
                text/csv
                text/tab-separated-values
                image/jpeg
                image/png
        """

        kwargs = {'id':id, 'depth':depth, 'blankNodes':blankNodes, 'relationshipType':relationshipType, 'direction':direction, 'project':project, 'callback':callback}
        kwargs = {k:dumps(v) if type(v) is dict else v for k, v in kwargs.items()}
        param_rest = self._make_rest('id', **kwargs)
        url = self._basePath + ('/graph/neighbors/{id}').format(**kwargs)
        requests_params = {k:v for k, v in kwargs.items() if k != 'id'}
        return self._get('GET', url, requests_params, output)


Instruction:
Can you implement a function in Python 3 that
classes Enrichment Service from: /analyzer/enrichment Arguments: sample: A list of CURIEs for nodes whose attributes are to be tested for enrichment. For example, a list of genes. ontologyClass: CURIE for parent ontology class for the attribute to be tested. For example, GO biological process path: A path expression that connects sample nodes to attribute class nodes callback: Name of the JSONP callback ('fn' by default). Supplying this parameter or requesting a javascript media type will cause a JSONP response to be rendered. outputs: application/json

Response:
def enrich(self, sample, ontologyClass, path, callback=None, output='application/json'):
        """ Class Enrichment Service from: /analyzer/enrichment
            Arguments:
            sample: A list of CURIEs for nodes whose attributes are to be tested for enrichment. For example, a list of genes.
            ontologyClass: CURIE for parent ontology class for the attribute to be tested. For example, GO biological process
            path: A path expression that connects sample nodes to attribute class nodes
            callback: Name of the JSONP callback ('fn' by default). Supplying this parameter or
            requesting a javascript media type will cause a JSONP response to be
            rendered.
            outputs:
                application/json
        """

        kwargs = {'sample':sample, 'ontologyClass':ontologyClass, 'path':path, 'callback':callback}
        kwargs = {k:dumps(v) if type(v) is dict else v for k, v in kwargs.items()}
        param_rest = self._make_rest('path', **kwargs)
        url = self._basePath + ('/analyzer/enrichment').format(**kwargs)
        requests_params = {k:v for k, v in kwargs.items() if k != 'path'}
        return self._get('GET', url, requests_params, output)


Instruction:
How would you implement a function in Python 3 that
annotates text from the annotations API.

Response:
def annotatePost(self, content, includeCat=None, excludeCat=None, minLength=4, longestOnly='false', includeAbbrev='false', includeAcronym='false', includeNumbers='false', ignoreTag=None, stylesheet=None, scripts=None, targetId=None, targetClass=None):
        """ Annotate text from: /annotations
            Arguments:
            content: The content to annotate
            includeCat: A set of categories to include
            excludeCat: A set of categories to exclude
            minLength: The minimum number of characters in annotated entities
            longestOnly: Should only the longest entity be returned for an overlapping group
            includeAbbrev: Should abbreviations be included
            includeAcronym: Should acronyms be included
            includeNumbers: Should numbers be included
            ignoreTag: HTML tags that should not be annotated
            stylesheet: CSS stylesheets to add to the HEAD
            scripts: JavaScripts that should to add to the HEAD
            targetId: A set of element IDs to annotate
            targetClass: A set of CSS class names to annotate
        """

        kwargs = {'content':content, 'includeCat':includeCat, 'excludeCat':excludeCat, 'minLength':minLength, 'longestOnly':longestOnly, 'includeAbbrev':includeAbbrev, 'includeAcronym':includeAcronym, 'includeNumbers':includeNumbers, 'ignoreTag':ignoreTag, 'stylesheet':stylesheet, 'scripts':scripts, 'targetId':targetId, 'targetClass':targetClass}
        kwargs = {k:dumps(v) if type(v) is dict else v for k, v in kwargs.items()}
        param_rest = self._make_rest('content', **kwargs)
        url = self._basePath + ('/annotations').format(**kwargs)
        requests_params = {k:v for k, v in kwargs.items() if k != 'content'}
        return self._get('POST', url, requests_params)


Instruction:
Can you tell what is the following Python 3 function doing
def getPos(self, text):

        kwargs = {'text':text}
        kwargs = {k:dumps(v) if type(v) is dict else v for k, v in kwargs.items()}
        param_rest = self._make_rest('text', **kwargs)
        url = self._basePath + ('/lexical/pos').format(**kwargs)
        requests_params = {k:v for k, v in kwargs.items() if k != 'text'}
        return self._get('GET', url, requests_params)

Response: Get the POS of the text in the current language.


Instruction:
Can you tell what is the following Python 3 function doing
def findByPrefix(self, term, limit=20, searchSynonyms='true', searchAbbreviations='false', searchAcronyms='false', includeDeprecated='false', category=None, prefix=None):

        kwargs = {'term':term, 'limit':limit, 'searchSynonyms':searchSynonyms, 'searchAbbreviations':searchAbbreviations, 'searchAcronyms':searchAcronyms, 'includeDeprecated':includeDeprecated, 'category':category, 'prefix':prefix}
        kwargs = {k:dumps(v) if type(v) is dict else v for k, v in kwargs.items()}
        param_rest = self._make_rest('term', **kwargs)
        url = self._basePath + ('/vocabulary/autocomplete/{term}').format(**kwargs)
        requests_params = {k:v for k, v in kwargs.items() if k != 'term'}
        return self._get('GET', url, requests_params)

Response: Find a concept by its prefix from the Vocabulary Autocomplete API.


Instruction:
Can you implement a function in Python 3 that
loads zip file and return complex field

Response:
def load_cell(fname="HL60_field.zip"):
    "Load zip file and return complex field"
    here = op.dirname(op.abspath(__file__))
    data = op.join(here, "data")
    arc = zipfile.ZipFile(op.join(data, fname))
    for f in arc.filelist:
        with arc.open(f) as fd:
            if f.filename.count("imag"):
                imag = np.loadtxt(fd)

            elif f.filename.count("real"):
                real = np.loadtxt(fd)

    field = real + 1j * imag
    return field


Instruction:
Make a summary of the following Python 3 code
def bootstrap(nside, rand, nbar, *data):

    def split(data, indices, axis):
        """ This function splits array. It fixes the bug
            in numpy that zero length array are improperly handled.

            In the future this will be fixed.
        """
        s = []
        s.append(slice(0, indices[0]))
        for i in range(len(indices) - 1):
            s.append(slice(indices[i], indices[i+1]))
        s.append(slice(indices[-1], None))

        rt = []
        for ss in s:
            ind = [slice(None, None, None) for i in range(len(data.shape))]
            ind[axis] = ss
            ind = tuple(ind)
            rt.append(data[ind])
        return rt

    def hpsplit(nside, data):
        # data is (RA, DEC)
        RA, DEC = data
        pix = radec2pix(nside, RA, DEC)
        n = numpy.bincount(pix)
        a = numpy.argsort(pix)
        data = numpy.array(data)[:, a]
        rt = split(data, n.cumsum(), axis=-1)
        return rt

    # mean area of sky.

    Abar =  41252.96 / nside2npix(nside)
    rand = hpsplit(nside, rand)
    if len(data) > 0:
        data = [list(i) for i in zip(*[hpsplit(nside, d1) for d1 in data])]
    else:
        data = [[] for i in range(len(rand))]

    heap = []
    j = 0
    for r, d in zip(rand, data):
        if len(r[0]) == 0: continue
        a = 1.0 * len(r[0]) / nbar
        j = j + 1
        if len(heap) == 0:
            heapq.heappush(heap, (a, j, r, d))
        else:
            a0, j0, r0, d0 = heapq.heappop(heap)
            if a0 + a < Abar:
                a0 += a
                d0 = [
                     numpy.concatenate((d0[i], d[i]), axis=-1)
                     for i in range(len(d))
                    ]
                r0 = numpy.concatenate((r0, r), axis=-1)
            else:
                heapq.heappush(heap, (a, j, r, d))
            heapq.heappush(heap, (a0, j0, r0, d0))

    for i in range(len(heap)):
        area, j, r, d = heapq.heappop(heap)
        rt = [area, r] + d
        yield rt

Response: This function will bootstrap the data based on the sky coverage of a random sample.


Instruction:
Create a Python 3 function to
load project specific flag list file

Response:
def load_project_flag_list_file(self, project_exceptions, project):
        """ Loads project specific lists """
        if self.loaded:
            return
        exception_file = None
        for item in project_exceptions:
            if project in item:
                exception_file = item.get(project)
        if exception_file is not None:
            try:
                with open(exception_file, 'r') as f:
                    ex = yaml.safe_load(f)
            except IOError:
                logger.error('File not found: %s', exception_file)
                sys.exit(0)
            for key in ex:
                if key in fl:
                    fl[key][project] = _merge(fl[key][project], ex.get(key, None)) \
                        if project in fl[key] else ex.get(key, None)
            self.loaded = True
        else:
            logger.info('%s not found in %s', project, ignore_list)
            logger.info('No project specific exceptions will be applied')


Instruction:
Create a Python 3 function to
gather sha256 hashes from binary lists

Response:
def binary_hash(self, project, patch_file):
        """ Gathers sha256 hashes from binary lists """
        global il
        exception_file = None
        try:
            project_exceptions = il.get('project_exceptions')
        except KeyError:
            logger.info('project_exceptions missing in %s for %s', ignore_list, project)

        for project_files in project_exceptions:
            if project in project_files:
                exception_file = project_files.get(project)
                with open(exception_file, 'r') as f:
                    bl = yaml.safe_load(f)

                for key, value in bl.items():
                    if key == 'binaries':
                        if patch_file in value:
                            hashvalue = value[patch_file]
                            return hashvalue
                        else:
                            for key, value in il.items():
                                if key == 'binaries':
                                    if patch_file in value:
                                        hashvalue = value[patch_file]
                                        return hashvalue
                                    else:
                                        hashvalue = ""
                                        return hashvalue
            else:
                logger.info('%s not found in %s', project, ignore_list)
                logger.info('No project specific exceptions will be applied')
                hashvalue = ""
                return hashvalue


Instruction:
Given the following Python 3 function, write the documentation
def file_audit_list(self, project):
        project_list = False
        self.load_project_flag_list_file(il.get('project_exceptions'), project)
        try:
            default_list = set((fl['file_audits']['file_names']))
        except KeyError:
            logger.error('Key Error processing file_names list values')
        try:
            project_list = set((fl['file_audits'][project]['file_names']))
            logger.info('Loaded %s specific file_audits entries', project)
        except KeyError:
            logger.info('No project specific file_names section for project %s', project)

        file_names_re = re.compile("|".join(default_list),
                                   flags=re.IGNORECASE)

        if project_list:
            file_names_proj_re = re.compile("|".join(project_list),
                                            flags=re.IGNORECASE)
            return file_names_re, file_names_proj_re
        else:
            file_names_proj_re = re.compile("")
            return file_names_re, file_names_proj_re

Response: Gathers file name lists


Instruction:
Write a Python 3 function for
gathering a list of directories to ignore

Response:
def ignore_directories(self, project):
        """ Gathers a list of directories to ignore """
        project_list = False
        try:
            ignore_directories = il['ignore_directories']
        except KeyError:
            logger.error('Key Error processing ignore_directories list values')

        try:
            project_exceptions = il.get('project_exceptions')
            for item in project_exceptions:
                if project in item:
                    exception_file = item.get(project)
                    with open(exception_file, 'r') as f:
                        test_list = yaml.safe_load(f)
                        project_list = test_list['ignore_directories']
        except KeyError:
            logger.info('No ignore_directories for %s', project)

        if project_list:
            ignore_directories = ignore_directories + project_list
            return ignore_directories
        else:
            return ignore_directories


Instruction:
Can you generate a brief explanation for the following Python 3 code
def url_ignore(self, project):
        project_list = False
        try:
            url_ignore = il['url_ignore']
        except KeyError:
            logger.error('Key Error processing url_ignore list values')

        try:
            project_exceptions = il.get('project_exceptions')
            for item in project_exceptions:
                if project in item:
                    exception_file = item.get(project)
                    with open(exception_file, 'r') as f:
                        url_list = yaml.safe_load(f)
                        project_list = url_list['url_ignore']
        except KeyError:
            logger.info('No url_ignore for %s', project)

        if project_list:
            url_ignore = url_ignore + project_list
            url_ignore_re = re.compile("|".join(url_ignore), flags=re.IGNORECASE)
            return url_ignore_re
        else:
            url_ignore_re = re.compile("|".join(url_ignore), flags=re.IGNORECASE)
            return url_ignore_re

Response: Gathers a list of URLs to ignore for a project


Instruction:
Given the following Python 3 function, write the documentation
def ip_ignore(self, project):
        project_list = False
        try:
            ip_ignore = il['ip_ignore']
        except KeyError:
            logger.error('Key Error processing ip_ignore list values')

        try:
            project_exceptions = il.get('project_exceptions')
            for item in project_exceptions:
                if project in item:
                    exception_file = item.get(project)
                    with open(exception_file, 'r') as f:
                        ip_list = yaml.safe_load(f)
                        project_list = ip_list['ip_ignore']
        except KeyError:
            logger.info('No ip_ignore for %s', project)

        if project_list:
            ip_ignore = ip_ignore + project_list
            ip_ignore_re = re.compile("|".join(ip_ignore), flags=re.IGNORECASE)
            return ip_ignore_re
        else:
            ip_ignore_re = re.compile("|".join(ip_ignore), flags=re.IGNORECASE)
            return ip_ignore_re

Response: Gathers a list of URLs to ignore for a project


Instruction:
Here you have a function in Python 3, explain what it does
def download_from_github(fname, path):
    base_url = 'https://github.com/ornlneutronimaging/ImagingReso/blob/master/ImagingReso/reference_data/'
    # Add GitHub junk to the file name for downloading.
    f = fname + '?raw=true'
    url = base_url + f
    block_size = 16384
    req = urlopen(url)

    # Get file size from header
    if sys.version_info[0] < 3:
        file_size = int(req.info().getheaders('Content-Length')[0])
    else:
        file_size = req.length
    # downloaded = 0

    # Check if file already downloaded
    if os.path.exists(fname):
        if os.path.getsize(fname) == file_size:
            print("Skipping downloading '{}'".format(fname))
        else:
            overwrite = input("File size changed, overwrite '{}'? ([y]/n) ".format(fname))
            if overwrite.lower().startswith('n'):
                print("Local file '{}' kept without overwriting.".format(fname))

    # Copy file to disk
    print("Downloading '{}'... ".format(fname))
    with open(fname, 'wb') as fh:
        while True:
            chunk = req.read(block_size)
            if not chunk:
                break
            fh.write(chunk)
            # downloaded += len(chunk)
        print('')
    print('Download completed.')
    print("Unzipping '{}'... ".format(fname))

    _database_zip = zipfile.ZipFile(fname)
    _database_zip.extractall(path=path)
    print("'{}' has been unzipped and database '{}' is ready to use.".format(fname, fname.replace('.zip', '')))

    os.remove(fname)
    print("'{}' has been deleted".format(fname))

Response: Download database from GitHub and save it to path.


Instruction:
Here you have a function in Python 3, explain what it does
def get_list_element_from_database(database='ENDF_VII'):
    _file_path = os.path.abspath(os.path.dirname(__file__))
    _ref_data_folder = os.path.join(_file_path, 'reference_data')
    _database_folder = os.path.join(_ref_data_folder, database)

    if not os.path.exists(_ref_data_folder):
        os.makedirs(_ref_data_folder)
        print("Folder to store database files has been created: '{}'".format(_ref_data_folder))

    if not os.path.exists(_database_folder):
        print("First time using database '{}'? ".format(database))
        print("I will retrieve and store a local copy of database'{}': ".format(database))
        download_from_github(fname=database + '.zip', path=_ref_data_folder)

    # if '/_elements_list.csv' NOT exist
    if not os.path.exists(_database_folder + '/_elements_list.csv'):
        # glob all .csv files
        _list_files = glob.glob(_database_folder + '/*.csv')

        # glob all .h5 files if NO .csv file exist
        if not _list_files:
            _list_files = glob.glob(_database_folder + '/*.h5')

        # test if files globed
        _empty_list_boo = not _list_files
        if _empty_list_boo is True:
            raise ValueError("'{}' does not contain any '*.csv' or '*.h5' file.".format(_database_folder))

        # convert path/to/file to filename only
        _list_short_filename_without_extension = [os.path.splitext(os.path.basename(_file))[0] for _file in _list_files]

        # isolate element names and output as list
        if '-' in _list_short_filename_without_extension[0]:
            _list_element = list(set([_name.split('-')[0] for _name in _list_short_filename_without_extension]))
        else:
            _list_letter_part = list(
                set([re.split(r'(\d+)', _name)[0] for _name in _list_short_filename_without_extension]))
            _list_element = []
            for each_letter_part in _list_letter_part:
                if len(each_letter_part) <= 2:
                    _list_element.append(each_letter_part)
        # save to current dir
        _list_element.sort()
        df_to_save = pd.DataFrame()
        df_to_save['elements'] = _list_element
        df_to_save.to_csv(_database_folder + '/_elements_list.csv')
        # print("NOT FOUND '{}'".format(_database_folder + '/_elements_list.csv'))
        # print("SAVED '{}'".format(_database_folder + '/_elements_list.csv'))

    # '/_elements_list.csv' exist
    else:
        df_to_read = pd.read_csv(_database_folder + '/_elements_list.csv')
        _list_element = list(df_to_read['elements'])
        # print("FOUND '{}'".format(_database_folder + '/_elements_list.csv'))
        # print("READ '{}'".format(_database_folder + '/_elements_list.csv'))

    return _list_element

Response: returns a string array of all the element from the database


Instruction:
Can you generate a brief explanation for the following Python 3 code
def is_element_in_database(element='', database='ENDF_VII'):
    if element == '':
        return False

    list_entry_from_database = get_list_element_from_database(database=database)
    if element in list_entry_from_database:
        return True
    return False

Response: returns True if the element is found in the database False otherwise


Instruction:
Implement a function in Python 3 to
function to create a dictionary from a formula

Response:
def formula_to_dictionary(formula='', thickness=np.NaN, density=np.NaN, database='ENDF_VII'):
    """create dictionary based on formula given

    Parameters:
    ===========
    formula: string
       ex: 'AgCo2'
       ex: 'Ag'
    thickness: float (in mm) default is np.NaN
    density: float (in g/cm3) default is np.NaN
    database: string (default is ENDV_VIII). Database where to look for elements

    Raises:
    =======
    ValueError if one of the element is missing from the database

    Return:
    =======
    the dictionary of the elements passed
      ex: {'AgCo2': {'elements': ['Ag','Co'],
                     'stoichiometric_ratio': [1,2],
                     'thickness': {'value': thickness,
                                   'units': 'mm'},
                     'density': {'value': density,
                                 'units': 'g/cm3'},
                     'molar_mass': {'value': np.nan,
                                    'units': 'g/mol'},
                    }
    """

    if '.' in formula:
        raise ValueError("formula '{}' is invalid, containing symbol '{}' !".format(formula, '.'))
    _formula_parsed = re.findall(r'([A-Z][a-z]*)(\d*)', formula)
    if len(_formula_parsed) == 0:
        raise ValueError("formula '{}' is invalid !".format(formula))

    # _dictionary = {}
    _elements_array = []
    _atomic_ratio_array = []
    for _element in _formula_parsed:
        [_single_element, _atomic_ratio] = list(_element)
        if not is_element_in_database(element=_single_element, database=database):
            raise ValueError("element '{}' not found in the database '{}'!".format(_single_element, database))

        if _atomic_ratio == '':
            _atomic_ratio = 1

        _atomic_ratio_array.append(int(_atomic_ratio))
        _elements_array.append(_single_element)
    _dict = {formula: {'elements': _elements_array,
                       'stoichiometric_ratio': _atomic_ratio_array,
                       'thickness': {'value': thickness,
                                     'units': 'mm'},
                       'density': {'value': density,
                                   'units': 'g/cm3'},
                       'molar_mass': {'value': np.nan,
                                      'units': 'g/mol'}
                       }
             }
    return _dict


Instruction:
Can you tell what is the following Python 3 function doing
def get_isotope_dicts(element='', database='ENDF_VII'):
    _file_path = os.path.abspath(os.path.dirname(__file__))
    _database_folder = os.path.join(_file_path, 'reference_data', database)
    _element_search_path = os.path.join(_database_folder, element + '-*.csv')

    list_files = glob.glob(_element_search_path)
    if not list_files:
        raise ValueError("File names contains NO '-', the name should in the format of 'Cd-115_m1' or 'Cd-114'")
    list_files.sort()
    isotope_dict = {'isotopes': {'list': [],
                                 'file_names': [],
                                 'density': {'value': np.NaN,
                                             'units': 'g/cm3'},
                                 'mass': {'value': [],
                                          'units': 'g/mol',
                                          },
                                 'isotopic_ratio': [], },
                    'density': {'value': np.NaN,
                                'units': 'g/cm3'},
                    'molar_mass': {'value': np.NaN,
                                   'units': 'g/mol'},
                    }

    # isotope_dict_mirror = {}
    _isotopes_list = []
    _isotopes_list_files = []
    _isotopes_mass = []
    _isotopes_density = []
    _isotopes_atomic_ratio = []
    _density = np.NaN
    _molar_mass = np.NaN

    for file in list_files:
        # Obtain element, z number from the basename
        _basename = os.path.basename(file)
        filename = os.path.splitext(_basename)[0]
        if '-' in filename:
            [_name, _number] = filename.split('-')
            if '_' in _number:
                [aaa, meta] = _number.split('_')
                _number = aaa[:]
        else:
            _split_list = re.split(r'(\d+)', filename)
            if len(_split_list) == 2:
                [_name, _number] = _split_list
            else:
                _name = _split_list[0]
                _number = _split_list[1]
        if _number == '0':
            _number = '12'
        _symbol = _number + '-' + _name
        isotope = str(_symbol)

        _isotopes_list.append(isotope)
        _isotopes_list_files.append(_basename)
        _isotopes_mass.append(get_mass(isotope))
        _isotopes_atomic_ratio.append(get_abundance(isotope))
        _isotopes_density.append(get_density(isotope))
        _density = get_density(element)
        _molar_mass = get_mass(element)

    isotope_dict['isotopes']['list'] = _isotopes_list
    isotope_dict['isotopes']['file_names'] = _isotopes_list_files
    isotope_dict['isotopes']['mass']['value'] = _isotopes_mass
    isotope_dict['isotopes']['isotopic_ratio'] = _isotopes_atomic_ratio
    isotope_dict['isotopes']['density']['value'] = _isotopes_density
    isotope_dict['density']['value'] = _density
    isotope_dict['molar_mass']['value'] = _molar_mass
    return isotope_dict

Response: returns a dictionary with list of isotopes found in database and name of database files


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_database_data(file_name=''):
    if not os.path.exists(file_name):
        raise IOError("File {} does not exist!".format(file_name))
    df = pd.read_csv(file_name, header=1)
    return df

Response: return the energy and sigma from the file_name


Instruction:
Explain what the following Python 3 code does
def get_interpolated_data(df: pd.DataFrame, e_min=np.nan, e_max=np.nan, e_step=np.nan):
    nbr_point = int((e_max - e_min) / e_step + 1)
    x_axis = np.linspace(e_min, e_max, nbr_point).round(6)
    y_axis_function = interp1d(x=df['E_eV'], y=df['Sig_b'], kind='linear')
    y_axis = y_axis_function(x_axis)

    return {'x_axis': x_axis, 'y_axis': y_axis}

Response: return the interpolated x and y axis for the given x range e_min e_max with step defined


Instruction:
Make a summary of the following Python 3 code
def get_sigma(database_file_name='', e_min=np.NaN, e_max=np.NaN, e_step=np.NaN, t_kelvin=None):

    file_extension = os.path.splitext(database_file_name)[1]

    if t_kelvin is None:
        # '.csv' files
        if file_extension != '.csv':
            raise IOError("Cross-section File type must be '.csv'")
        else:
            _df = get_database_data(file_name=database_file_name)
            _dict = get_interpolated_data(df=_df, e_min=e_min, e_max=e_max,
                                          e_step=e_step)
            return {'energy_eV': _dict['x_axis'],
                    'sigma_b': _dict['y_axis']}
    else:
        raise ValueError("Doppler broadened cross-section in not yet supported in current version.")

Response: retrieve the Energy and sigma axis for the given isotope


Instruction:
Here you have a function in Python 3, explain what it does
def get_atoms_per_cm3_of_layer(compound_dict: dict):
    # atoms_per_cm3 = {}

    _list_of_elements = compound_dict['elements']
    _stoichiometric_list = compound_dict['stoichiometric_ratio']

    _element_stoichio = zip(_list_of_elements, _stoichiometric_list)
    _molar_mass_sum = 0
    for _element, _stoichio in _element_stoichio:
        _molar_mass_sum += _stoichio * compound_dict[_element]['molar_mass']['value']

    atoms_per_cm3 = Avogadro * compound_dict['density']['value'] / _molar_mass_sum
    # _element_stoichio = zip(_list_of_elements, _stoichiometric_list)
    # for _element, _stoichio in _element_stoichio:
    #     _step1 = (compound_dict['density']['value'] * _stoichio) / _molar_mass_sum
    #     atoms_per_cm3[_element] = Avogadro * _step1

    return _molar_mass_sum, atoms_per_cm3

Response: calculate the atoms per cm3 of the given compound


Instruction:
Given the following Python 3 function, write the documentation
def calculate_linear_attenuation_coefficient(atoms_per_cm3: np.float, sigma_b: np.array):
    miu_per_cm = 1e-24 * sigma_b * atoms_per_cm3
    return np.array(miu_per_cm)

Response: calculate the linear attenuation coefficient using the formula


Instruction:
How would you code a function in Python 3 to
calculate the transmission signal using the formula

Response:
def calculate_trans(thickness_cm: np.float, miu_per_cm: np.array):
    """calculate the transmission signal using the formula

    transmission = exp( - thickness_cm * atoms_per_cm3 * 1e-24 * sigma_b)

    Parameters:
    ===========
    thickness: float (in cm)
    atoms_per_cm3: float (number of atoms per cm3 of element/isotope)
    sigma_b: np.array of sigma retrieved from database

    Returns:
    ========
    transmission array
    """
    transmission = np.exp(-thickness_cm * miu_per_cm)
    return np.array(transmission)


Instruction:
Can you generate the documentation for the following Python 3 function
def calculate_transmission(thickness_cm: np.float, atoms_per_cm3: np.float, sigma_b: np.array):
    miu_per_cm = calculate_linear_attenuation_coefficient(atoms_per_cm3=atoms_per_cm3, sigma_b=sigma_b)
    transmission = calculate_trans(thickness_cm=thickness_cm, miu_per_cm=miu_per_cm)
    return miu_per_cm, transmission

Response: calculate the transmission signal using the formula


Instruction:
Can you write a function in Python 3 where it
converts distance into new units

Response:
def set_distance_units(value=np.NaN, from_units='mm', to_units='cm'):
    """convert distance into new units

    Parameters:
    ===========
    value: float. value to convert
    from_units: string. Must be 'mm', 'cm' or 'm'
    to_units: string. must be 'mm','cm' or 'm'

    Returns:
    ========
    converted value

    Raises:
    =======
    ValueError if from_units is not a valid unit (see above)
    ValueError if to_units is not a valid unit
    """
    if from_units == to_units:
        return value

    if from_units == 'cm':
        if to_units == 'mm':
            coeff = 10
        elif to_units == 'm':
            coeff = 0.01
        else:
            raise ValueError("to_units not supported ['cm','m','mm']!")
    elif from_units == 'mm':
        if to_units == 'cm':
            coeff = 0.1
        elif to_units == 'm':
            coeff = 0.001
        else:
            raise ValueError("to_units not supported ['cm','m','mm']!")
    elif from_units == 'm':
        if to_units == 'mm':
            coeff = 1000
        elif to_units == 'cm':
            coeff = 100
        else:
            raise ValueError("to_units not supported ['cm','m','mm']!")
    else:
        raise ValueError("to_units not supported ['cm','m','mm']!")

    return coeff * value


Instruction:
Can you write a function in Python 3 where it
converts energy to time

Response:
def ev_to_s(offset_us, source_to_detector_m, array):
    # delay values is normal 2.99 us with NONE actual MCP delay settings
    """convert energy (eV) to time (us)

    Parameters:
    ===========
    array: array (in eV)
    offset_us: float. Delay of detector in us
    source_to_detector_m: float. Distance source to detector in m

    Returns:
    ========
    time: array in s
    """
    # 1000 is used to convert eV to meV
    time_s = np.sqrt(81.787 / (array * 1000.)) * source_to_detector_m / 3956.
    time_record_s = time_s - offset_us * 1e-6
    return time_record_s


Instruction:
Can you generate a brief explanation for the following Python 3 code
def s_to_ev(offset_us, source_to_detector_m, array):
    lambda_a = 3956. * (array + offset_us * 1e-6) / source_to_detector_m
    return (81.787 / pow(lambda_a, 2)) / 1000.

Response: convert time ( s ) to energy ( eV


Instruction:
Write a Python 3 function for
converting energy in eV to image number

Response:
def ev_to_image_number(offset_us, source_to_detector_m, time_resolution_us, t_start_us, array):
    # delay values is normal 2.99 us with NONE actual MCP delay settings
    """convert energy (eV) to image numbers (#)

    Parameters:
    ===========
    numpy array of energy in eV
    offset_us: float. Delay of detector in us
    source_to_detector_m: float. Distance source to detector in m

    Returns:
    ========
    image numbers: array of image number
    """
    time_tot_us = np.sqrt(81.787 / (array * 1000)) * source_to_detector_m * 100 / 0.3956
    time_record_us = (time_tot_us + offset_us)
    image_number = (time_record_us - t_start_us) / time_resolution_us
    return image_number


Instruction:
Can you write a function in Python 3 where it
clones a git repository inside a temporary folder then delete the folder.

Response:
def temp_repo(url, branch, commit=''):
    """ Clone a git repository inside a temporary folder, yield the folder then delete the folder.

    :param string url: url of the repo to clone.
    :param string branch: name of the branch to checkout to.
    :param string commit: Optional commit rev to checkout to. If mentioned, that take over the branch
    :return: yield the path to the temporary folder
    :rtype: string
    """
    tmp_folder = tempfile.mkdtemp()
    git.Repo.clone_from(
        url, tmp_folder, branch=branch
    )
    if commit:
        git_cmd = git.Git(tmp_folder)
        git_cmd.checkout(commit)
    yield tmp_folder
    shutil.rmtree(tmp_folder)


Instruction:
Here you have a function in Python 3, explain what it does
def force_move(source, destination):
    if not os.path.exists(destination):
        raise RuntimeError(
            'The code could not be moved to {destination} '
            'because the folder does not exist'.format(destination=destination))

    destination_folder = os.path.join(destination, os.path.split(source)[-1])

    if os.path.exists(destination_folder):
        shutil.rmtree(destination_folder)
    shutil.move(source, destination)

Response: Force the move of the source folder to the destination folder.


Instruction:
Explain what the following Python 3 code does
def _run_command_inside_folder(command, folder):
    logger.debug("command: %s", command)
    # avoid usage of shell = True
    # see https://docs.openstack.org/bandit/latest/plugins/subprocess_popen_with_shell_equals_true.html
    process = subprocess.Popen(command.split(), stdout=subprocess.PIPE, cwd=folder)
    stream_data = process.communicate()[0]
    logger.debug("%s stdout: %s (RC %s)", command, stream_data, process.returncode)
    return process.returncode, stream_data

Response: Run a command inside the folder.


Instruction:
Can you create a Python 3 function that
parses the given url and update the environment variable if required.

Response:
def parse_url(url):
    """ Parse the given url and update it with environment value if required.

    :param basestring url:
    :rtype: basestring
    :raise: KeyError if environment variable is needed but not found.
    """
    # the url has to be a unicode by pystache's design, but the unicode concept has been rewamped in py3
    # we use a try except to make the code compatible with py2 and py3
    try:
        url = unicode(url)
    except NameError:
        url = url

    parsed = pystache.parse(url)
    # pylint: disable=protected-access
    variables = (element.key for element in parsed._parse_tree if isinstance(element, _EscapeNode))
    return pystache.render(url, {variable: os.environ[variable] for variable in variables})


Instruction:
Write a Python 3 function that can
install a third party odoo add - on to destination.

Response:
def install(self, destination):
        """ Install a third party odoo add-on

        :param string destination: the folder where the add-on should end up at.
        """
        logger.info(
            "Installing %s@%s to %s",
            self.repo, self.commit if self.commit else self.branch, destination
        )
        with temp_repo(self.repo, self.branch, self.commit) as tmp:
            self._apply_patches(tmp)
            self._move_modules(tmp, destination)


Instruction:
Write a Python 3 function for
moving modules froom the temp directory to the destination.

Response:
def _move_modules(self, temp_repo, destination):
        """Move modules froom the temp directory to the destination.

        :param string temp_repo: the folder containing the code.
        :param string destination: the folder where the add-on should end up at.
        """
        folders = self._get_module_folders(temp_repo)
        for folder in folders:
            force_move(folder, destination)


Instruction:
Make a summary of the following Python 3 code
def _get_module_folders(self, temp_repo):
        paths = (
            os.path.join(temp_repo, path) for path in os.listdir(temp_repo)
            if self._is_module_included(path)
        )
        return (path for path in paths if os.path.isdir(path))

Response: Get a list of module paths contained in a temp directory.


Instruction:
Can you write a function in Python 3 where it
evaluates if the module must be included in the Odoo addons.

Response:
def _is_module_included(self, module):
        """Evaluate if the module must be included in the Odoo addons.

        :param string module: the name of the module
        :rtype: bool
        """
        if module in self.exclude_modules:
            return False

        if self.include_modules is None:
            return True

        return module in self.include_modules


Instruction:
Explain what the following Python 3 code does
def _move_modules(self, temp_repo, destination):
        tmp_addons = os.path.join(temp_repo, 'addons')
        tmp_odoo_addons = os.path.join(temp_repo, 'odoo/addons')
        folders = self._get_module_folders(tmp_addons)
        for folder in folders:
            force_move(folder, tmp_odoo_addons)

        tmp_odoo = os.path.join(temp_repo, 'odoo')
        force_move(tmp_odoo, destination)

Response: Move modules from the temp directory to the destination.


Instruction:
Write a Python 3 function that can
merge code from the given repo url to the git repo contained in the given folder.

Response:
def apply(self, folder):
        """ Merge code from the given repo url to the git repo contained in the given folder.

        :param string folder: path of the folder where is the git repo cloned at.
        :raise: RuntimeError if the patch could not be applied.
        """
        logger.info("Apply Patch %s@%s (commit %s)", self.url, self.branch, self.commit)
        remote_name = 'patch'
        commands = [
            "git remote add {} {}".format(remote_name, self.url),
            "git fetch {} {}".format(remote_name, self.branch),
            'git merge {} -m "patch"'.format(self.commit),
            "git remote remove {}".format(remote_name),
        ]
        for command in commands:
            return_code, stream_data = _run_command_inside_folder(command, folder)
            if return_code:
                msg = "Could not apply patch from {}@{}: {}. Error: {}".format(
                    self.url, self.branch, command, stream_data)
                logger.error(msg)
                raise RuntimeError(msg)


Instruction:
Here you have a function in Python 3, explain what it does
def apply(self, folder):
        logger.info("Apply Patch File %s", self.file_path)
        command = "git apply {}".format(self.file_path)
        return_code, stream_data = _run_command_inside_folder(command, folder)

        if return_code:
            msg = "Could not apply patch file at {}. Error: {}".format(self.file_path, stream_data)
            logger.error(msg)
            raise RuntimeError(msg)

Response: Apply a git patch file to a folder.


Instruction:
Write a Python 3 function for
creating context to keep all needed variables in.

Response:
def _set_up_context(cls):
        """Create context to keep all needed variables in."""
        cls.context = AttributeDict()
        cls.context.new_meta = {}
        cls.context.new_transitions = {}
        cls.context.new_methods = {}


Instruction:
Here you have a function in Python 3, explain what it does
def _check_states_enum(cls):
        states_enum_name = cls.context.get_config('states_enum_name')
        try:
            cls.context['states_enum'] = getattr(
                cls.context.new_class, states_enum_name)
        except AttributeError:
            raise ValueError('No states enum given!')

        proper = True
        try:
            if not issubclass(cls.context.states_enum, Enum):
                proper = False
        except TypeError:
            proper = False

        if not proper:
            raise ValueError(
                'Please provide enum instance to define available states.')

Response: Check if states enum exists and is proper one.


Instruction:
How would you explain what the following Python 3 function does
def _check_if_states_are_strings(cls):
        for item in list(cls.context.states_enum):
            if not isinstance(item.value, six.string_types):
                raise ValueError(
                    'Item {name} is not string. Only strings are allowed.'
                    .format(name=item.name)
                )

Response: Check if all states are strings.


Instruction:
Given the following Python 3 function, write the documentation
def _check_state_value(cls):
        state_value = cls.context.get_config('initial_state', None)
        state_value = state_value or getattr(
            cls.context.new_class, cls.context.state_name, None
        )

        if not state_value:
            raise ValueError(
                "Empty state is disallowed, yet no initial state is given!"
            )
        state_value = (
            cls.context
            .new_meta['translator']
            .translate(state_value)
        )
        cls.context.state_value = state_value

Response: Check initial state value - if is proper and translate it.


Instruction:
Create a Python 3 function for
adding attributes common to all state machines.

Response:
def _add_standard_attributes(cls):
        """Add attributes common to all state machines.

        These are methods for setting and checking state etc.

        """
        setattr(
            cls.context.new_class,
            cls.context.new_meta['state_attribute_name'],
            cls.context.state_value)
        setattr(
            cls.context.new_class,
            cls.context.state_name,
            utils.state_property)

        setattr(cls.context.new_class, 'is_', utils.is_)
        setattr(cls.context.new_class, 'can_be_', utils.can_be_)
        setattr(cls.context.new_class, 'set_', utils.set_)


Instruction:
How would you explain what the following Python 3 function does
def _generate_standard_transitions(cls):
        allowed_transitions = cls.context.get_config('transitions', {})
        for key, transitions in allowed_transitions.items():
            key = cls.context.new_meta['translator'].translate(key)

            new_transitions = set()
            for trans in transitions:
                if not isinstance(trans, Enum):
                    trans = cls.context.new_meta['translator'].translate(trans)
                new_transitions.add(trans)

            cls.context.new_transitions[key] = new_transitions

        for state in cls.context.states_enum:
            if state not in cls.context.new_transitions:
                cls.context.new_transitions[state] = set()

Response: Generate methods used for transitions.


Instruction:
Implement a Python 3 function for
generating standard setters getters and checkers.

Response:
def _generate_standard_methods(cls):
        """Generate standard setters, getters and checkers."""
        for state in cls.context.states_enum:
            getter_name = 'is_{name}'.format(name=state.value)
            cls.context.new_methods[getter_name] = utils.generate_getter(state)

            setter_name = 'set_{name}'.format(name=state.value)
            cls.context.new_methods[setter_name] = utils.generate_setter(state)

            checker_name = 'can_be_{name}'.format(name=state.value)
            checker = utils.generate_checker(state)
            cls.context.new_methods[checker_name] = checker

        cls.context.new_methods['actual_state'] = utils.actual_state
        cls.context.new_methods['as_enum'] = utils.as_enum
        cls.context.new_methods['force_set'] = utils.force_set


Instruction:
Make a summary of the following Python 3 code
def _add_new_methods(cls):
        for name, method in cls.context.new_methods.items():
            if hasattr(cls.context.new_class, name):
                raise ValueError(
                    "Name collision in state machine class - '{name}'."
                    .format(name)
                )

            setattr(cls.context.new_class, name, method)

Response: Add all generated methods to result class.


Instruction:
Write a Python 3 script to
check and set complete option.

Response:
def _set_complete_option(cls):
        """Check and set complete option."""
        get_config = cls.context.get_config
        complete = get_config('complete', None)
        if complete is None:
            conditions = [
                get_config('transitions', False),
                get_config('named_transitions', False),
            ]
            complete = not any(conditions)

        cls.context.new_meta['complete'] = complete


Instruction:
Create a Python 3 function to
return the absolute path to the directory containing the package data.

Response:
def data_directory():
    """Return the absolute path to the directory containing the package data."""
    package_directory = os.path.abspath(os.path.dirname(__file__))
    return os.path.join(package_directory, "data")


Instruction:
How would you implement a function in Python 3 that
selects a map by name or critiera

Response:
def selectMap(name=None, excludeName=False, closestMatch=True, **tags):
    """select a map by name and/or critiera"""
    matches = filterMapAttrs(**tags)
    if not matches: raise c.InvalidMapSelection("could not find any matching maps given criteria: %s"%tags)
    if name: # if name is specified, consider only the best-matching names only
        matches = filterMapNames(name, excludeRegex=excludeName, closestMatch=closestMatch, records=matches)
    try:
        if closestMatch:    return random.choice(matches) # pick any map at random that matches all criteria
        elif matches:       return matches
    except IndexError: pass # matches is empty still
    raise c.InvalidMapSelection("requested map '%s', but could not locate "\
        "it within %s or its subdirectories.  Submit the map to https://"\
        "github.com/ttinies/sc2gameMapRepo/tree/master/sc2maptool/maps"%(
            name, c.PATH_MAP_INSTALL))


Instruction:
Here you have a function in Python 3, explain what it does
def filterMapAttrs(records=getIndex(), **tags):
    if len(tags) == 0: return records # otherwise if unspecified, all given records match
    ret = []
    for record in records: # attempt to match attributes
        if matchRecordAttrs(record, tags):
            ret.append(record)
    return ret

Response: returns a list of records that match the given tags


Instruction:
How would you explain what the following Python 3 function does
def matchRecordAttrs(mapobj, attrs):
    for k,v in iteritems(attrs):
        try:    val = getattr(mapobj, k)
        except AttributeError:       # k isn't an attr of record
            if bool(v): return False # if k doesn't exist in mapobj but was required, no match
            else:       continue     # otherwise ignore attributes that aren't defined for the given map record
        if val != v:    return False # if any criteria matches, it's considered a match
    return True

Response: attempt to match given attributes against a single map object s attributes


Instruction:
Make a summary of the following Python 3 code
def filterMapNames(regexText,  records=getIndex(), excludeRegex=False, closestMatch=True):
    bestScr = 99999 # a big enough number to not be a valid file system path
    regex = re.compile(regexText, flags=re.IGNORECASE)
    ret = []
    if excludeRegex: # match only records that do NOT contain regex
        if regexText and closestMatch: # then maps with fewer characters are better matches
            for m in list(records):
                if re.search(regex, m.name): continue # map must NOT contain specified phrase
                score = len(m.name) # the map with the smallest map name means it has the largets matching character percentage
                if score == bestScr:
                    bestScr = score
                    ret.append(m)
                elif score <  bestScr: # new set of best maps
                    bestScr = score
                    ret = [m]
        else: # all maps that match regex are included
            for m in list(records):
                if re.search(regex, m.name): continue # map must NOT contain specified phrase
                ret.append(m) # any mapname containing regex matches
    else: # only match records that contain regex
        if regexText and closestMatch: # then maps with fewer characters are better matches
            for m in records:
                if not re.search(regex, m.name): continue # map must contain specified phrase if excludeRegex==True
                score = len(m.name) # the map with the smallest map name means it has the largets matching character percentage
                if score == bestScr:
                    bestScr = score
                    ret.append(m)
                elif score <  bestScr: # new group of best maps
                    bestScr = score
                    ret = [m]
        else: # all maps that match regex are included
            for m in records:
                if not re.search(regex, m.name): continue # map must contain specified phrase if excludeRegex==True
                ret.append(m) # any mapname containing regex matches
    return ret

Response:
matches each record against regexText according to parameters
    returns a list of maps that match the regexText


Instruction:
Create a Python 3 function to
try conversion of any value to a boolean

Response:
def to_boolean(value, ctx):
    """
    Tries conversion of any value to a boolean
    """
    if isinstance(value, bool):
        return value
    elif isinstance(value, int):
        return value != 0
    elif isinstance(value, Decimal):
        return value != Decimal(0)
    elif isinstance(value, str):
        value = value.lower()
        if value == 'true':
            return True
        elif value == 'false':
            return False
    elif isinstance(value, datetime.date) or isinstance(value, datetime.time):
        return True

    raise EvaluationError("Can't convert '%s' to a boolean" % str(value))


Instruction:
Write a Python 3 script for
trying conversion of any value to an integer

Response:
def to_integer(value, ctx):
    """
    Tries conversion of any value to an integer
    """
    if isinstance(value, bool):
        return 1 if value else 0
    elif isinstance(value, int):
        return value
    elif isinstance(value, Decimal):
        try:
            val = int(value.to_integral_exact(ROUND_HALF_UP))
            if isinstance(val, int):
                return val
        except ArithmeticError:
            pass
    elif isinstance(value, str):
        try:
            return int(value)
        except ValueError:
            pass

    raise EvaluationError("Can't convert '%s' to an integer" % str(value))


Instruction:
Can you implement a function in Python 3 that
tries conversion of any value to a decimal

Response:
def to_decimal(value, ctx):
    """
    Tries conversion of any value to a decimal
    """
    if isinstance(value, bool):
        return Decimal(1) if value else Decimal(0)
    elif isinstance(value, int):
        return Decimal(value)
    elif isinstance(value, Decimal):
        return value
    elif isinstance(value, str):
        try:
            return Decimal(value)
        except Exception:
            pass

    raise EvaluationError("Can't convert '%s' to a decimal" % str(value))


Instruction:
Can you generate a brief explanation for the following Python 3 code
def to_string(value, ctx):
    if isinstance(value, bool):
        return "TRUE" if value else "FALSE"
    elif isinstance(value, int):
        return str(value)
    elif isinstance(value, Decimal):
        return format_decimal(value)
    elif isinstance(value, str):
        return value
    elif type(value) == datetime.date:
        return value.strftime(ctx.get_date_format(False))
    elif isinstance(value, datetime.time):
        return value.strftime('%H:%M')
    elif isinstance(value, datetime.datetime):
        return value.astimezone(ctx.timezone).isoformat()

    raise EvaluationError("Can't convert '%s' to a string" % str(value))

Response: Converts any value to a string


Instruction:
Given the following Python 3 function, write the documentation
def to_date(value, ctx):
    if isinstance(value, str):
        temporal = ctx.get_date_parser().auto(value)
        if temporal is not None:
            return to_date(temporal, ctx)
    elif type(value) == datetime.date:
        return value
    elif isinstance(value, datetime.datetime):
        return value.date()  # discard time

    raise EvaluationError("Can't convert '%s' to a date" % str(value))

Response: Tries conversion of any value to a date


Instruction:
Create a Python 3 function to
try conversion of any value to a datetime

Response:
def to_datetime(value, ctx):
    """
    Tries conversion of any value to a datetime
    """
    if isinstance(value, str):
        temporal = ctx.get_date_parser().auto(value)
        if temporal is not None:
            return to_datetime(temporal, ctx)
    elif type(value) == datetime.date:
        return ctx.timezone.localize(datetime.datetime.combine(value, datetime.time(0, 0)))
    elif isinstance(value, datetime.datetime):
        return value.astimezone(ctx.timezone)

    raise EvaluationError("Can't convert '%s' to a datetime" % str(value))


Instruction:
Here you have a function in Python 3, explain what it does
def to_date_or_datetime(value, ctx):
    if isinstance(value, str):
        temporal = ctx.get_date_parser().auto(value)
        if temporal is not None:
            return temporal
    elif type(value) == datetime.date:
        return value
    elif isinstance(value, datetime.datetime):
        return value.astimezone(ctx.timezone)

    raise EvaluationError("Can't convert '%s' to a date or datetime" % str(value))

Response: Tries conversion of any value to a date or datetime


Instruction:
How would you code a function in Python 3 to
try conversion of any value to a time

Response:
def to_time(value, ctx):
    """
    Tries conversion of any value to a time
    """
    if isinstance(value, str):
        time = ctx.get_date_parser().time(value)
        if time is not None:
            return time
    elif isinstance(value, datetime.time):
        return value
    elif isinstance(value, datetime.datetime):
        return value.astimezone(ctx.timezone).time()

    raise EvaluationError("Can't convert '%s' to a time" % str(value))


Instruction:
How would you explain what the following Python 3 function does
def to_same(value1, value2, ctx):
    if type(value1) == type(value2):
        return value1, value2

    try:
        # try converting to two decimals
        return to_decimal(value1, ctx), to_decimal(value2, ctx)
    except EvaluationError:
        pass

    try:
        # try converting to two dates
        d1, d2 = to_date_or_datetime(value1, ctx), to_date_or_datetime(value2, ctx)

        # if either one is a datetime, then the other needs to become a datetime
        if type(value1) != type(value2):
            d1, d2 = to_datetime(d1, ctx), to_datetime(d2, ctx)
        return d1, d2
    except EvaluationError:
        pass

    # try converting to two strings
    return to_string(value1, ctx), to_string(value2, ctx)

Response: Converts two values to the most - likely type.


Instruction:
Explain what the following Python 3 code does
def to_repr(value, ctx):
    as_string = to_string(value, ctx)

    if isinstance(value, str) or isinstance(value, datetime.date) or isinstance(value, datetime.time):
        as_string = as_string.replace('"', '""')  # escape quotes by doubling
        as_string = '"%s"' % as_string

    return as_string

Response: Converts a value back to its representation form e. g. x -> x


Instruction:
Can you write a function in Python 3 where it
formats a decimal number

Response:
def format_decimal(decimal):
    """
    Formats a decimal number
    :param decimal: the decimal value
    :return: the formatted string value
    """
    # strip trailing fractional zeros
    normalized = decimal.normalize()
    sign, digits, exponent = normalized.as_tuple()
    if exponent >= 1:
        normalized = normalized.quantize(1)

    return str(normalized)


Instruction:
How would you code a function in Python 3 to
check if I am running inside a Linux container.

Response:
def is_containerized() -> bool:
    '''
    Check if I am running inside a Linux container.
    '''
    try:
        cginfo = Path('/proc/self/cgroup').read_text()
        if '/docker/' in cginfo or '/lxc/' in cginfo:
            return True
    except IOError:
        return False


Instruction:
Make a summary of the following Python 3 code
def detect_cloud() -> str:
    '''
    Detect the cloud provider where I am running on.
    '''
    # NOTE: Contributions are welcome!
    # Please add other cloud providers such as Rackspace, IBM BlueMix, etc.
    if sys.platform.startswith('linux'):
        # Google Cloud Platform or Amazon AWS (hvm)
        try:
            # AWS Nitro-based instances
            mb = Path('/sys/devices/virtual/dmi/id/board_vendor').read_text().lower()
            if 'amazon' in mb:
                return 'amazon'
        except IOError:
            pass
        try:
            bios = Path('/sys/devices/virtual/dmi/id/bios_version').read_text().lower()
            if 'google' in bios:
                return 'google'
            if 'amazon' in bios:
                return 'amazon'
        except IOError:
            pass
        # Microsoft Azure
        # https://gallery.technet.microsoft.com/scriptcenter/Detect-Windows-Azure-aed06d51
        # TODO: this only works with Debian/Ubuntu instances.
        # TODO: this does not work inside containers.
        try:
            dhcp = Path('/var/lib/dhcp/dhclient.eth0.leases').read_text()
            if 'unknown-245' in dhcp:
                return 'azure'
            # alternative method is to read /var/lib/waagent/GoalState.1.xml
            # but it requires sudo privilege.
        except IOError:
            pass
    else:
        log.warning('Cloud detection is implemented for Linux only yet.')
    return None

Response: Detect the cloud provider where I am running on.


Instruction:
How would you explain what the following Python 3 function does
def addBlock(self,branch=None,btype=None,mtype=None,attributes=None):
        '''
        Add block definition to list of blocks in material
        Order for list entry: branch level (0=root), block type (material,solid,fluid,etc.), matid (integer if root, False otherwise),
            material name (string if root, False otherwise), material type, dictionary of attributes or False if none
        '''
        if branch == 0:
            attributes = self.attributes
        blk = {'branch': branch,
               'btype': btype,
               'mtype': mtype,
               'attributes': attributes}

        self.blocks.append(blk)

Response: Add block definition to list of blocks in material


Instruction:
Explain what the following Python 3 code does
def refocus(field, d, nm, res, method="helmholtz", num_cpus=1, padding=True):
    # FFT of field
    fshape = len(field.shape)
    assert fshape in [1, 2], "Dimension of `field` must be 1 or 2."

    func = fft_propagate
    names = func.__code__.co_varnames[:func.__code__.co_argcount]

    loc = locals()
    vardict = dict()

    for name in names:
        if name in loc:
            vardict[name] = loc[name]

    if padding:
        field = pad.pad_add(field)

    vardict["fftfield"] = np.fft.fftn(field)

    refoc = func(**vardict)

    if padding:
        refoc = pad.pad_rem(refoc)

    return refoc

Response: Refocus a 1D or 2D array of electric fields at d.


Instruction:
Explain what the following Python 3 code does
def refocus_stack(fieldstack, d, nm, res, method="helmholtz",
                  num_cpus=_cpu_count, copy=True, padding=True):
    """Refocus a stack of 1D or 2D fields

    Parameters
    ----------
    fieldstack : 2d or 3d array
        Stack of 1D or 2D background corrected electric fields (Ex/BEx).
        The first axis iterates through the individual fields.
    d : float
        Distance to be propagated in pixels (negative for backwards)
    nm : float
        Refractive index of medium
    res : float
        Wavelenth in pixels
    method : str
        Defines the method of propagation;
        one of

            - "helmholtz" : the optical transfer function `exp(idkₘ(M-1))`
            - "fresnel"   : paraxial approximation `exp(idk²/kₘ)`

    num_cpus : str
        Defines the number of CPUs to be used for refocusing.
    copy : bool
        If False, overwrites input stack.
    padding : bool
        Perform padding with linear ramp from edge to average
        to reduce ringing artifacts.

        .. versionadded:: 0.1.4

    Returns
    -------
    Electric field stack at `d`.
    """

    func = refocus
    names = func.__code__.co_varnames[:func.__code__.co_argcount]

    loc = locals()
    vardict = dict()
    for name in names:
        if name in loc.keys():
            vardict[name] = loc[name]
    # default keyword arguments
    func_def = func.__defaults__[::-1]

    # child processes should only use one cpu
    vardict["num_cpus"] = 1
    vardict["padding"] = padding

    M = fieldstack.shape[0]
    stackargs = list()

    # Create individual arglists for all fields
    for m in range(M):
        kwarg = vardict.copy()
        kwarg["field"] = fieldstack[m]
        # now we turn the kwarg into an arglist
        args = list()
        for i, a in enumerate(names[::-1]):
            # first set default
            if i < len(func_def):
                val = func_def[i]
            if a in kwarg:
                val = kwarg[a]
            args.append(val)
        stackargs.append(args[::-1])

    p = mp.Pool(num_cpus)
    result = p.map_async(_refocus_wrapper, stackargs).get()
    p.close()
    p.terminate()
    p.join()

    if copy:
        data = np.zeros(fieldstack.shape, dtype=result[0].dtype)
    else:
        data = fieldstack

    for m in range(M):
        data[m] = result[m]

    return data

Response: Refocuses a stack of electric fields at d.


Instruction:
How would you implement a function in Python 3 that
propagates a 1D or 2D Fourier transformed field at d nm res

Response:
def fft_propagate(fftfield, d, nm, res, method="helmholtz",
                  ret_fft=False):
    """Propagates a 1D or 2D Fourier transformed field

    Parameters
    ----------
    fftfield : 1-dimensional or 2-dimensional ndarray
        Fourier transform of 1D Electric field component
    d : float
        Distance to be propagated in pixels (negative for backwards)
    nm : float
        Refractive index of medium
    res : float
        Wavelength in pixels
    method : str
        Defines the method of propagation;
        one of

            - "helmholtz" : the optical transfer function `exp(idkₘ(M-1))`
            - "fresnel"   : paraxial approximation `exp(idk²/kₘ)`

    ret_fft : bool
        Do not perform an inverse Fourier transform and return the field
        in Fourier space.

    Returns
    -------
    Electric field at `d`. If `ret_fft` is True, then the
    Fourier transform of the electric field will be returned (faster).
    """
    fshape = len(fftfield.shape)
    assert fshape in [1, 2], "Dimension of `fftfield` must be 1 or 2."

    if fshape == 1:
        func = fft_propagate_2d
    else:
        func = fft_propagate_3d

    names = func.__code__.co_varnames[:func.__code__.co_argcount]

    loc = locals()
    vardict = dict()
    for name in names:
        vardict[name] = loc[name]

    return func(**vardict)


Instruction:
Can you generate the documentation for the following Python 3 function
def fft_propagate_2d(fftfield, d, nm, res, method="helmholtz",
                     ret_fft=False):
    """Propagate a 1D  Fourier transformed field in 2D

    Parameters
    ----------
    fftfield : 1d array
        Fourier transform of 1D Electric field component
    d : float
        Distance to be propagated in pixels (negative for backwards)
    nm : float
        Refractive index of medium
    res : float
        Wavelength in pixels
    method : str
        Defines the method of propagation;
        one of

            - "helmholtz" : the optical transfer function `exp(idkₘ(M-1))`
            - "fresnel"   : paraxial approximation `exp(idk²/kₘ)`

    ret_fft : bool
        Do not perform an inverse Fourier transform and return the field
        in Fourier space.

    Returns
    -------
    Electric field at `d`. If `ret_fft` is True, then the
    Fourier transform of the electric field will be returned (faster).
    """
    assert len(fftfield.shape) == 1, "Dimension of `fftfield` must be 1."
    km = (2 * np.pi * nm) / res
    kx = np.fft.fftfreq(len(fftfield)) * 2 * np.pi

    # free space propagator is
    if method == "helmholtz":
        # exp(i*sqrt(km²-kx²)*d)
        # Also subtract incoming plane wave. We are only considering
        # the scattered field here.
        root_km = km**2 - kx**2
        rt0 = (root_km > 0)
        # multiply by rt0 (filter in Fourier space)
        fstemp = np.exp(1j * (np.sqrt(root_km * rt0) - km) * d) * rt0
    elif method == "fresnel":
        # exp(i*d*(km-kx²/(2*km))
        # fstemp = np.exp(-1j * d * (kx**2/(2*km)))
        fstemp = np.exp(-1j * d * (kx**2/(2*km)))
    else:
        raise ValueError("Unknown method: {}".format(method))

    if ret_fft:
        return fftfield * fstemp
    else:
        return np.fft.ifft(fftfield * fstemp)

Response: Propagate a 1D Fourier transformed field in 2D.


Instruction:
Here you have a function in Python 3, explain what it does
def fft_propagate_3d(fftfield, d, nm, res, method="helmholtz",
                     ret_fft=False):
    """Propagate a 2D  Fourier transformed field in 3D

    Parameters
    ----------
    fftfield : 2d array
        Fourier transform of 2D Electric field component
    d : float
        Distance to be propagated in pixels (negative for backwards)
    nm : float
        Refractive index of medium
    res : float
        Wavelength in pixels
    method : str
        Defines the method of propagation;
        one of

            - "helmholtz" : the optical transfer function `exp(idkₘ(M-1))`
            - "fresnel"   : paraxial approximation `exp(idk²/kₘ)`

    ret_fft : bool
        Do not perform an inverse Fourier transform and return the field
        in Fourier space.

    Returns
    -------
    Electric field at `d`. If `ret_fft` is True, then the
    Fourier transform of the electric field will be returned (faster).
    """
    assert len(fftfield.shape) == 2, "Dimension of `fftfield` must be 2."
    # if fftfield.shape[0] != fftfield.shape[1]:
    #    raise NotImplementedError("Field must be square shaped.")
    # free space propagator is
    # exp(i*sqrt(km**2-kx**2-ky**2)*d)
    km = (2 * np.pi * nm) / res
    kx = (np.fft.fftfreq(fftfield.shape[0]) * 2 * np.pi).reshape(-1, 1)
    ky = (np.fft.fftfreq(fftfield.shape[1]) * 2 * np.pi).reshape(1, -1)
    if method == "helmholtz":
        # exp(i*sqrt(km²-kx²-ky²)*d)
        root_km = km**2 - kx**2 - ky**2
        rt0 = (root_km > 0)
        # multiply by rt0 (filter in Fourier space)
        fstemp = np.exp(1j * (np.sqrt(root_km * rt0) - km) * d) * rt0
    elif method == "fresnel":
        # exp(i*d*(km-(kx²+ky²)/(2*km))
        # fstemp = np.exp(-1j * d * (kx**2+ky**2)/(2*km))
        fstemp = np.exp(-1j * d * (kx**2 + ky**2)/(2*km))
    else:
        raise ValueError("Unknown method: {}".format(method))
    # fstemp[np.where(np.isnan(fstemp))] = 0
    # Also subtract incoming plane wave. We are only considering
    # the scattered field here.
    if ret_fft:
        return fftfield * fstemp
    else:
        return np.fft.ifft2(fftfield * fstemp)

Response: Propagate a 2D Fourier transformed field in 3D


Instruction:
Write a Python 3 script for
pushing the object to the GIT repository.

Response:
def push(self):
        """
         Adding the no_thin argument to the GIT push because we had some issues pushing previously.
         According to http://stackoverflow.com/questions/16586642/git-unpack-error-on-push-to-gerrit#comment42953435_23610917,
         "a new optimization which causes git to send as little data as possible over the network caused this bug to manifest,
          so my guess is --no-thin just turns these optimizations off. From git push --help: "A thin transfer significantly
                  reduces the amount of sent data when the sender and receiver share many of the same objects in common." (--thin is the default)."
        """

        if not self.canRunRemoteCmd():
            return None

        try:
            fetchInfo = self.repo.remotes.origin.push(no_thin=True)[0]
        except exc.GitCommandError as e:
            print(dir(e))
            print(e)
            raise

        if fetchInfo.flags & fetchInfo.ERROR:
            try:
                raise IOError("An error occured while trying to push the GIT repository from the server. Error flag: '" +
                          str(fetchInfo.flags) + "', message: '" + str(fetchInfo.note) + "'.")
            except:
                IOError("An error occured while trying to push the GIT repository from the server.")
        return fetchInfo


Instruction:
Can you generate a brief explanation for the following Python 3 code
def autofocus_stack(fieldstack, nm, res, ival, roi=None,
                    metric="average gradient", padding=True,
                    same_dist=False, ret_ds=False, ret_grads=False,
                    num_cpus=_cpu_count, copy=True):
    """Numerical autofocusing of a stack using the Helmholtz equation.

    Parameters
    ----------
    fieldstack : 2d or 3d ndarray
        Electric field is BG-Corrected, i.e. Field = EX/BEx
    nm : float
        Refractive index of medium.
    res : float
        Size of wavelength in pixels.
    ival : tuple of floats
        Approximate interval to search for optimal focus in px.
    metric : str
        see `autofocus_field`.
    padding : bool
        Perform padding with linear ramp from edge to average
        to reduce ringing artifacts.

        .. versionchanged:: 0.1.4
           improved padding value and padding location
    ret_dopt : bool
        Return optimized distance and gradient plotting data.
    same_dist : bool
        Refocus entire sinogram with one distance.
    red_ds : bool
        Return the autofocusing distances in pixels. Defaults to False.
        If sam_dist is True, still returns autofocusing distances
        of first pass. The used refocusing distance is the
        average.
    red_grads : bool
        Return the computed gradients as a list.
    copy : bool
        If False, overwrites input array.

    Returns
    -------
    The focused field (and the refocussing distance + data if d is None)
    """
    dopt = list()
    grad = list()

    M = fieldstack.shape[0]

    # setup arguments
    stackargs = list()
    for s in range(M):
        stackargs.append([fieldstack[s].copy(copy), nm, res, ival,
                          roi, metric, padding, True, True, 1])
    # perform first pass
    p = mp.Pool(num_cpus)
    result = p.map_async(_autofocus_wrapper, stackargs).get()
    p.close()
    p.terminate()
    p.join()
    # result = []
    # for arg in stackargs:
    #    result += _autofocus_wrapper(arg)

    newstack = np.zeros(fieldstack.shape, dtype=fieldstack.dtype)

    for s in range(M):
        field, ds, gs = result[s]
        dopt.append(ds)
        grad.append(gs)
        newstack[s] = field

    # perform second pass if `same_dist` is True
    if same_dist:
        # find average dopt
        davg = np.average(dopt)
        newstack = refocus_stack(fieldstack, davg, nm, res,
                                 num_cpus=num_cpus, copy=copy,
                                 padding=padding)

    ret_list = [newstack]
    if ret_ds:
        ret_list += [dopt]
    if ret_grads:
        ret_list += [grad]

    if len(ret_list) == 1:
        return ret_list[0]
    else:
        return tuple(ret_list)

Response: This function calculates the autofocusing of a single stack of sinograms.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def minimize_metric(field, metric_func, nm, res, ival, roi=None,
                    coarse_acc=1, fine_acc=.005,
                    return_gradient=True, padding=True):
    """Find the focus by minimizing the `metric` of an image

    Parameters
    ----------
    field : 2d array
        electric field
    metric_func : callable
        some metric to be minimized
    ival : tuple of floats
        (minimum, maximum) of interval to search in pixels
    nm : float
        RI of medium
    res : float
        wavelength in pixels
    roi : rectangular region of interest (x1, y1, x2, y2)
        Region of interest of `field` for which the metric will be
        minimized. If not given, the entire `field` will be used.
    coarse_acc : float
        accuracy for determination of global minimum in pixels
    fine_acc : float
        accuracy for fine localization percentage of gradient change
    return_gradient:
        return x and y values of computed gradient
    padding : bool
        perform padding with linear ramp from edge to average
        to reduce ringing artifacts.

        .. versionchanged:: 0.1.4
           improved padding value and padding location
    """
    if roi is not None:
        assert len(roi) == len(field.shape) * \
            2, "ROI must match field dimension"

    initshape = field.shape
    Fshape = len(initshape)
    propfunc = fft_propagate

    if roi is None:
        if Fshape == 2:
            roi = (0, 0, field.shape[0], field.shape[1])
        else:
            roi = (0, field.shape[0])

    roi = 1*np.array(roi)

    if padding:
        # Pad with correct complex number
        field = pad.pad_add(field)

    if ival[0] > ival[1]:
        ival = (ival[1], ival[0])
    # set coarse interval
    # coarse_acc = int(np.ceil(ival[1]-ival[0]))/100
    N = 100 / coarse_acc
    zc = np.linspace(ival[0], ival[1], N, endpoint=True)

    # compute fft of field
    fftfield = np.fft.fftn(field)

    # fftplan = fftw3.Plan(fftfield.copy(), None, nthreads = _ncores,
    #                     direction="backward", flags=_fftwflags)

    # initiate gradient vector
    gradc = np.zeros(zc.shape)
    for i in range(len(zc)):
        d = zc[i]
        # fsp = propfunc(fftfield, d, nm, res, fftplan=fftplan)
        fsp = propfunc(fftfield, d, nm, res)
        if Fshape == 2:
            gradc[i] = metric_func(fsp[roi[0]:roi[2], roi[1]:roi[3]])
        else:
            gradc[i] = metric_func(fsp[roi[0]:roi[1]])

    minid = np.argmin(gradc)
    if minid == 0:
        zc -= zc[1] - zc[0]
        minid += 1
    if minid == len(zc) - 1:
        zc += zc[1] - zc[0]
        minid -= 1
    zf = 1*zc
    gradf = 1 * gradc

    numfine = 10
    mingrad = gradc[minid]

    while True:
        gradf = np.zeros(numfine)
        ival = (zf[minid - 1], zf[minid + 1])
        zf = np.linspace(ival[0], ival[1], numfine)
        for i in range(len(zf)):
            d = zf[i]
            fsp = propfunc(fftfield, d, nm, res)
            if Fshape == 2:
                gradf[i] = metric_func(fsp[roi[0]:roi[2], roi[1]:roi[3]])
            else:
                gradf[i] = metric_func(fsp[roi[0]:roi[1]])
        minid = np.argmin(gradf)
        if minid == 0:
            zf -= zf[1] - zf[0]
            minid += 1
        if minid == len(zf) - 1:
            zf += zf[1] - zf[0]
            minid -= 1
        if abs(mingrad - gradf[minid]) / 100 < fine_acc:
            break

    minid = np.argmin(gradf)
    fsp = propfunc(fftfield, zf[minid], nm, res)

    if padding:
        fsp = pad.pad_rem(fsp)

    if return_gradient:
        return fsp, zf[minid], [(zc, gradc), (zf, gradf)]
    return fsp, zf[minid]

Response: Find the focus by minimizing a metric of an image.


Instruction:
Can you generate the documentation for the following Python 3 function
def prepare_project(project, project_dir, binaries, ips, urls):

    # Get Various Lists / Project Waivers
    lists = get_lists.GetLists()

    # Get file name black list and project waivers
    file_audit_list, file_audit_project_list = lists.file_audit_list(project)

    # Get file content black list and project waivers
    flag_list, ignore_list = lists.file_content_list(project)

    # Get File Ignore Lists
    file_ignore = lists.file_ignore()

    ignore_directories = lists.ignore_directories(project)

    # Get URL Ignore Lists
    url_ignore = lists.url_ignore(project)

    # Get IP Ignore Lists
    ip_ignore = lists.ip_ignore(project)

    # Get Binary Ignore Lists
    hashlist = get_lists.GetLists()

    if binaries or ips or urls:
        try:
            apikey = os.environ["VT_KEY"]
        except KeyError:
            logger.error("Please set your virustotal.com API key as an environment variable")
            sys.exit(1)
        try:
            vt_rate_type = config.get('config', 'vt_rate_type')
        except six.moves.configparser.NoSectionError:
            logger.error("A config section is required for vt_rate_type with a public | private option")
            sys.exit(1)

        patten = re.compile(r'\bpublic\b|\bprivate\b')
        if not patten.match(vt_rate_type):
            logger.error("Unrecognized %s option for vt_rate_type", vt_rate_type)
            sys.exit(1)

    # Perform rudimentary scans
    scan_file(project, project_dir, binaries, ips, urls, file_audit_list,
              file_audit_project_list, flag_list, ignore_list, hashlist,
              file_ignore, ignore_directories, url_ignore, ip_ignore, apikey)

Response: Prepares the project for rudimentary analysis


Instruction:
Can you write a function in Python 3 where it
scans a file for a specific project.

Response:
def scan_file(project, project_dir, binaries, ips, urls, file_audit_list,
              file_audit_project_list, flag_list, ignore_list, hashlist,
              file_ignore, ignore_directories, url_ignore, ip_ignore, apikey):
    """
    Main scan tasks begin
    """
    logger.info("Commencing scan tasks..")
    for root, dirs, files in os.walk(project_dir):
        # Filter out ignored directories from list.
        for items in files:
            full_path = os.path.join(root, items)
            split_path = full_path.split(project + '/', 1)[-1]
            # Check for Blacklisted file names
            if not any(x in split_path for x in ignore_directories):
                if file_audit_list.search(full_path) and not \
                        file_audit_project_list.search(full_path):
                    match = file_audit_list.search(full_path)
                    logger.error('Blacklisted filename: %s', full_path)
                    logger.error('Matched String: %s', match.group())
                    with open(reports_dir + "file-names_" + project + ".log",
                              "a") as gate_report:
                                gate_report. \
                                    write('Blacklisted filename: {0}\n'.
                                          format(full_path))
                                gate_report. \
                                    write('Matched String: {0}'.
                                          format(match.group()))

                # Check if Binary is whitelisted
                if is_binary(full_path) and binaries:

                    split_path = full_path.split(project + '/', 1)[-1]
                    binary_hash = hashlist.binary_hash(project, split_path)
                    with open(full_path, 'rb') as afile:
                        hasher = hashlib.sha256()
                        buf = afile.read()
                        hasher.update(buf)
                        sha256hash = hasher.hexdigest()
                    if sha256hash in binary_hash:
                        logger.info('Found matching file hash for: %s', full_path)
                        logger.info('No further action needed for: %s', full_path)
                    else:
                        logger.info('Non Whitelisted Binary file: %s', full_path)
                        scan_binary(full_path, split_path, project, sha256hash, apikey)

                else:
                    if not items.endswith(tuple(file_ignore)) and not is_binary(full_path):
                        try:
                            fo = open(full_path, 'r')
                            lines = fo.readlines()
                        except IOError:
                            logger.error('%s does not exist', full_path)

                        for line in lines:
                            # Find IP Addresses and send for report to Virus Total
                            if ips:
                                ipaddr = re.findall(r'(?:\d{1,3}\.)+(?:\d{1,3})', line)
                                if ipaddr:
                                    ipaddr = ipaddr[0]
                                    if re.search(ip_ignore, ipaddr):
                                        logger.info('%s is in IP ignore list.', ipaddr)
                                    else:
                                        try:
                                            ipaddress.ip_address(ipaddr).is_global
                                            scan_ipaddr(ipaddr, line, project, split_path, apikey)
                                        except:
                                            pass  # Ok to pass here, as this captures the odd string which is nt an IP Address

                            # Check for URLs and send for report to Virus Total
                            if urls:
                                url = re.search("(?P<url>https?://[^\s]+)", line) or re.search("(?P<url>www[^\s]+)", line)
                                if url:
                                    url = url.group("url")
                                    if re.search(url_ignore, url):
                                        logger.info('%s is in URL ignore list.', url)
                                    else:
                                        scan_url(url, line, project, split_path, apikey)

                            # Check flagged content in files
                            for key, value in flag_list.items():
                                regex = value['regex']
                                desc = value['desc']
                                if re.search(regex, line) and not re.search(ignore_list, line):
                                    logger.error('File contains violation: %s', full_path)
                                    logger.error('Flagged Content: %s', line.rstrip())
                                    logger.error('Matched Regular Exp: %s', regex)
                                    logger.error('Rationale: %s', desc.rstrip())
                                    with open(reports_dir + "contents-" + project + ".log", "a") as gate_report:
                                        gate_report.write('File contains violation: {0}\n'.format(full_path))
                                        gate_report.write('Flagged Content: {0}'.format(line))
                                        gate_report.write('Matched Regular Exp: {0}'.format(regex))
                                        gate_report.write('Rationale: {0}\n'.format(desc.rstrip()))


Instruction:
Here you have a function in Python 3, explain what it does
def scan_ipaddr(ipaddr, line, project, split_path, apikey):
    logger.info('Found what I believe is an IP Address: %s', line.strip())
    logger.info('File %s. Parsed IP Address: %s', split_path, ipaddr)
    with open(reports_dir + "ips-" + project + ".log", "a") as gate_report:
        gate_report.write('File {} contains what I believe is an IP Address: {}\n'.format(split_path, ipaddr))
    v_api = virus_total.VirusTotal()
    scan_ip = v_api.send_ip(ipaddr, apikey)
    response_code = scan_ip['response_code']
    verbose_msg = scan_ip['verbose_msg']
    urls = scan_ip['detected_urls']
    with open(reports_dir + "ips-" + project + ".log", "a") as gate_report:
        if urls:
            logger.error('%s has been known to resolve to the following malicious urls:', ipaddr)
            gate_report.write('{} has been known to resolve to the following malicious urls:\n'.format(ipaddr))

            for url in urls:
                logger.info('%s on date: %s', url['url'], url['scan_date'])
                gate_report.write('{} on {}\n'.format(url['url'], url['scan_date']))
                sleep(0.2)
        else:
            logger.info('No malicious DNS history found for: %s', ipaddr)
            gate_report.write('No malicious DNS history found for: {}\n'.format(ipaddr))

Response: Scan the VirusTotal API for a single IP Address.


Instruction:
Write a Python 3 script for
scanning a URL and return a list of the VirusTotal objects.

Response:
def scan_url(url, line, project, split_path, apikey):
    """
    If URL is found, scan it
    """
    logger.info('File %s contains what I believe is a URL: %s', split_path, line.strip())
    logger.info('Scanning: %s', url)
    with open(reports_dir + "urls-" + project + ".log", "a") as gate_report:
        gate_report.write('File {} contains what I believe is a URL: {}\n'.format(split_path, url))

    v_api = virus_total.VirusTotal()
    while True:
        url_report = v_api.url_report(url, apikey)
        response_code = url_report['response_code']

        # report does not exist, need to scan
        if response_code == -2:
            logger.info('Report job still queued..')

        if response_code == 0:
            logger.info('No report  for %s', url)
            break

        if response_code == 1:
            logger.info('Report found, job complete for %s.', url)
            break

    try:
        positives = url_report['positives']
        if positives > 0:
            for site, results in url_report['scans'].items():
                printed = False
                if results['detected']:
                    with open(reports_dir + "urls-" + project + ".log", "a") as gate_report:
                        logger.error("%s is recorded as a %s by %s", url, results['result'], site)
                        gate_report.write('{} is recorded as a {} by {}\n'.format(url, results['result'], site))
            if not printed:
                printed = True
                with open(reports_dir + "urls-" + project + ".log", "a") as gate_report:
                    logger.error("Full report available here: %s", url_report['permalink'])
                    gate_report.write('Full report available here: {}\n'.format(url_report['permalink']))

        else:
            logger.info("%s is recorded as a clean", url)
            with open(reports_dir + "urls-" + project + ".log", "a") as gate_report:
                gate_report.write('{} is recorded as a clean\n'.format(url))

    except:
        # No positives so we can pass this
        pass


Instruction:
Can you generate the documentation for the following Python 3 function
def getIndex(folderPath=None):
    try:    return cache.structure
    except AttributeError: pass # if it doesn't exist, generate and cache the map file data
    if folderPath == None:
        from sc2maptool.startup import setup
        folderPath = setup()
    ############################################################################
    def folderSearch(path, attrList=[]):
        ret = []
        for item in glob(os.path.join(path, '*')):
            if item == os.sep: continue
            itemName = os.path.basename(item)
            if   os.path.isdir(item):               ret += folderSearch(item, attrList + [itemName])
            elif itemName.endswith(c.SC2_MAP_EXT):  ret.append( MapRecord(itemName, item, attrList) )
        return ret
    ############################################################################
    cache.structure = folderSearch(folderPath)
    return cache.structure

Response: parse the Maps subfolder directory divining criteria for valid maps


Instruction:
Can you generate the documentation for the following Python 3 function
def _stringify_number(v):
    '''
    Stringify a number, preventing unwanted scientific notations.
    '''
    if isinstance(v, (float, Decimal)):
        if math.isinf(v) and v > 0:
            v = 'Infinity'
        elif math.isinf(v) and v < 0:
            v = '-Infinity'
        else:
            v = '{:f}'.format(v)
    elif isinstance(v, BinarySize):
        v = '{:d}'.format(int(v))
    elif isinstance(v, int):
        v = '{:d}'.format(v)
    else:
        v = str(v)
    return v

Response: Stringify a number preventing unwanted scientific notations.


Instruction:
How would you explain what the following Python 3 function does
async def resolve_alias(cls, alias_key: str, etcd: etcd.AsyncEtcd):
        '''
        Resolve the tag using etcd so that the current instance indicates
        a concrete, latest image.

        Note that alias resolving does not take the registry component into
        account.
        '''
        alias_target = None
        repeats = 0
        while repeats < 8:
            prev_alias_key = alias_key
            alias_key = await etcd.get(f'images/_aliases/{alias_key}')
            if alias_key is None:
                alias_target = prev_alias_key
                break
            repeats += 1
        else:
            raise AliasResolutionFailed('Could not resolve the given image name!')
        known_registries = await get_known_registries(etcd)
        return cls(alias_target, known_registries)

Response: Resolve the tag using etcd so that the current instance indicates that the given image name is a concrete latest image.


Instruction:
Can you write a function in Python 3 where it
gets the list of categories for a given curie.

Response:
def getOntoCategory(curie, alwaysFetch=False):
    """
     Accessing web-based ontology service is too long, so we cache the
     information in a pickle file and query the services only if the info
     has not already been cached.
    """

    fileName = os.path.join(os.path.dirname(__file__), "ontoCategories.bin")
    if not alwaysFetch:
        try:
            with open(fileName, "rb") as catFile:
                ontoCat = pickle.load(catFile)

            if curie in ontoCat:
                return ontoCat[curie]
        except:
            ontoCat = {}

    base = bases["KS"]
    query = base + "/vocabulary/id/" + curie
    response = requests.get(query)
    if not response.ok:
        ontoCat[curie] = []
    else:

        try:
            concepts = response.json()
        except ValueError:
            print(query)
            print(response)
            raise

        if len(concepts["categories"]):
            ontoCat[curie] = concepts["categories"]
        else:
            ontoCat[curie] = []

    try:
        with open(fileName, "wb") as catFile:
            pickle.dump(ontoCat, catFile)
    except:
        pass

    return ontoCat[curie]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _init_logging(anteater_log):

    LOG.setLevel(logging.DEBUG)
    ch = logging.StreamHandler()
    formatter = logging.Formatter('%(asctime)s - %(name)s - '
                                  '%(levelname)s - %(message)s')
    ch.setFormatter(formatter)
    ch.setLevel(logging.DEBUG)

    # create the directory if it does not exist
    path = os.path.dirname(anteater_log)
    try:
        os.makedirs(path)
    except OSError as e:
        if e.errno != errno.EEXIST:
            raise

    handler = logging.FileHandler(anteater_log)
    handler.setFormatter(formatter)
    handler.setLevel(logging.DEBUG)
    del logging.root.handlers[:]
    logging.root.addHandler(ch)
    logging.root.addHandler(handler)

Response: Setup root logger for the AETTLE package


Instruction:
Implement a function in Python 3 to
create a directory for scan reports

Response:
def check_dir():
    """ Creates a directory for scan reports """
    try:
        os.makedirs(reports_dir)
        logger.info('Creating reports directory: %s', reports_dir)
    except OSError as e:
        if e.errno != errno.EEXIST:
            raise


Instruction:
Can you generate the documentation for the following Python 3 function
def fit(model_code, *args, **kwargs):
    kwargs = dict(kwargs)
    kwargs['model_code'] = model_code
    if 'n_jobs' not in kwargs:
        kwargs['n_jobs'] = -1
    if model_code in FIT_CACHE:
        print("Reusing model.")
        kwargs['fit'] = FIT_CACHE[model_code]
    else:
        print("NOT reusing model.")
    start = time.time()
    FIT_CACHE[model_code] = pystan.stan(*args, **kwargs)
    print("Ran in %0.3f sec." % (time.time() - start))
    return FIT_CACHE[model_code]

Response: Fit a Stan model.


Instruction:
Can you write a function in Python 3 where it
iterates over all pairs of pairs of KDNode objects with distance rmax

Response:
def enumiter(self, other, rmax, bunch=100000):
        """ cross correlate with other, for all pairs
            closer than rmax, iterate.

            for r, i, j in A.enumiter(...):
                ...
            where r is the distance, i and j are the original
            input array index of the data.

            This uses a thread to convert from KDNode.enum.
        """

        def feeder(process):
            self.enum(other, rmax, process, bunch)
        for r, i, j in makeiter(feeder):
            yield r, i, j


Instruction:
Here you have a function in Python 3, explain what it does
def enum(self, other, rmax, process=None, bunch=100000, **kwargs):
        rall = None
        if process is None:
            rall = [numpy.empty(0, 'f8')]
            iall = [numpy.empty(0, 'intp')]
            jall = [numpy.empty(0, 'intp')]
            def process(r1, i1, j1, **kwargs):
                rall[0] = numpy.append(rall[0], r1)
                iall[0] = numpy.append(iall[0], i1)
                jall[0] = numpy.append(jall[0], j1)

        _core.KDNode.enum(self, other, rmax, process, bunch, **kwargs)

        if rall is not None:
            return rall[0], iall[0], jall[0]
        else:
            return None

Response:
cross correlate with other for all pairs
            closer than rmax and iterate.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def count(self, other, r, attrs=None, info={}):

        r = numpy.array(r, dtype='f8')

        return _core.KDNode.count(self, other, r, attrs, info=info)

Response: Gray & Moore based fast dual tree counting.


Instruction:
Make a summary of the following Python 3 code
def integrate(self, min, max, attr=None, info={}):
        if numpy.isscalar(min):
            min = [min for i in range(self.ndims)]
        if numpy.isscalar(max):
            max = [max for i in range(self.ndims)]

        min = numpy.array(min, dtype='f8', order='C')
        max = numpy.array(max, dtype='f8', order='C')

        if (min).shape[-1] != self.ndims:
            raise ValueError("dimension of min does not match Node")
        if (max).shape[-1] != self.ndims:
            raise ValueError("dimension of max does not match Node")
        min, max = broadcast_arrays(min, max)
        return _core.KDNode.integrate(self, min, max, attr, info)

Response: Integrate the number of points between min and max.


Instruction:
Write a Python 3 script to
divide a tree branch to a forest each subtree of size at most chunksize

Response:
def make_forest(self, chunksize):
        """ Divide a tree branch to a forest,
            each subtree of size at most chunksize """
        heap = []
        heappush(heap, (-self.size, self))
        while True:
            w, x = heappop(heap)
            if w == 0:
                heappush(heap, (0, x))
                break
            if x.less is None \
            or (x.size < chunksize):
                heappush(heap, (0, x))
                continue
            heappush(heap, (x.less.size, x.less))
            heappush(heap, (x.greater.size, x.greater))
        for w, x in heap:
            yield x