Instruction:
Write a Python 3 function for
functioning to fetch a list of filled trades for the parameters requested. Execution of this function is as follows:: get_trades(pair="SWTH_NEO", limit=3) The expected return result for this function is as follows:: [{ 'id': '15bb16e2-7a80-4de1-bb59-bcaff877dee0', 'fill_amount': 100000000, 'take_amount': 100000000, 'event_time': '2018-08-04T15:00:12.634Z', 'is_buy': True }, { 'id': 'b6f9e530-60ff-46ff-9a71-362097a2025e', 'fill_amount': 47833882, 'take_amount': 97950000000, 'event_time': '2018-08-03T02:44:47.706Z', 'is_buy': True }, { 'id': '7a308ccc-b7f5-46a3-bf6b-752ab076cc9f', 'fill_amount': 1001117, 'take_amount': 2050000000, 'event_time': '2018-08-03T02:32:50.703Z', 'is_buy': True }] :param pair: The trading pair that will be used to request filled trades. :type pair: str :param start_time: Only return trades after this time (in epoch seconds). :type start_time: int :param end_time: Only return trades before this time (in epoch seconds). :type end_time: int :param limit: The number of filled trades to return. Min: 1, Max: 10000, Default: 5000 :type limit: int :return: List of dictionaries consisting of filled orders that meet requirements of the parameters passed to it.

Response:
def get_trades(self, pair="SWTH_NEO", start_time=None, end_time=None, limit=5000):
        """
        Function to fetch a list of filled trades for the parameters requested.
        Execution of this function is as follows::

            get_trades(pair="SWTH_NEO", limit=3)

        The expected return result for this function is as follows::

            [{
                'id': '15bb16e2-7a80-4de1-bb59-bcaff877dee0',
                'fill_amount': 100000000,
                'take_amount': 100000000,
                'event_time': '2018-08-04T15:00:12.634Z',
                'is_buy': True
            }, {
                'id': 'b6f9e530-60ff-46ff-9a71-362097a2025e',
                'fill_amount': 47833882,
                'take_amount': 97950000000,
                'event_time': '2018-08-03T02:44:47.706Z',
                'is_buy': True
            }, {
                'id': '7a308ccc-b7f5-46a3-bf6b-752ab076cc9f',
                'fill_amount': 1001117,
                'take_amount': 2050000000,
                'event_time': '2018-08-03T02:32:50.703Z',
                'is_buy': True
            }]

        :param pair: The trading pair that will be used to request filled trades.
        :type pair: str
        :param start_time: Only return trades after this time (in epoch seconds).
        :type start_time: int
        :param end_time: Only return trades before this time (in epoch seconds).
        :type end_time: int
        :param limit: The number of filled trades to return. Min: 1, Max: 10000, Default: 5000
        :type limit: int
        :return: List of dictionaries consisting of filled orders that meet requirements of the parameters passed to it.
        """
        if limit > 10000 or limit < 1:
            raise ValueError("Attempting to request more trades than allowed by the API.")
        api_params = {
            "blockchain": self.blockchain,
            "pair": pair,
            "contract_hash": self.contract_hash
        }
        if start_time is not None:
            api_params['from'] = start_time
        if end_time is not None:
            api_params['to'] = end_time
        if limit != 5000:
            api_params['limit'] = limit
        return self.request.get(path='/trades', params=api_params)


Instruction:
Implement a function in Python 3 to
function to fetch a list of the 20 most recently filled trades for the parameters requested. Execution of this function is as follows:: get_recent_trades(pair="SWTH_NEO") The expected return result for this function is as follows:: [{ 'id': '15bb16e2-7a80-4de1-bb59-bcaff877dee0', 'fill_amount': 100000000, 'take_amount': 100000000, 'event_time': '2018-08-04T15:00:12.634Z', 'is_buy': True }, { 'id': 'b6f9e530-60ff-46ff-9a71-362097a2025e', 'fill_amount': 47833882, 'take_amount': 97950000000, 'event_time': '2018-08-03T02:44:47.706Z', 'is_buy': True }, ...., { 'id': '7a308ccc-b7f5-46a3-bf6b-752ab076cc9f', 'fill_amount': 1001117, 'take_amount': 2050000000, 'event_time': '2018-08-03T02:32:50.703Z', 'is_buy': True }] :param pair: The trading pair that will be used to request filled trades. :type pair: str :return: List of 20 dictionaries consisting of filled orders for the trade pair.

Response:
def get_recent_trades(self, pair="SWTH_NEO"):
        """
        Function to fetch a list of the 20 most recently filled trades for the parameters requested.
        Execution of this function is as follows::

            get_recent_trades(pair="SWTH_NEO")

        The expected return result for this function is as follows::

            [{
                'id': '15bb16e2-7a80-4de1-bb59-bcaff877dee0',
                'fill_amount': 100000000,
                'take_amount': 100000000,
                'event_time': '2018-08-04T15:00:12.634Z',
                'is_buy': True
            }, {
                'id': 'b6f9e530-60ff-46ff-9a71-362097a2025e',
                'fill_amount': 47833882,
                'take_amount': 97950000000,
                'event_time': '2018-08-03T02:44:47.706Z',
                'is_buy': True
            }, ...., {
                'id': '7a308ccc-b7f5-46a3-bf6b-752ab076cc9f',
                'fill_amount': 1001117,
                'take_amount': 2050000000,
                'event_time': '2018-08-03T02:32:50.703Z',
                'is_buy': True
            }]

        :param pair: The trading pair that will be used to request filled trades.
        :type pair: str
        :return: List of 20 dictionaries consisting of filled orders for the trade pair.
        """
        api_params = {
            "pair": pair
        }
        return self.request.get(path='/trades/recent', params=api_params)


Instruction:
Make a summary of the following Python 3 code
def get_pairs(self, base=None, show_details=False):
        api_params = {}
        if show_details:
            api_params["show_details"] = show_details
        if base is not None and base in ["NEO", "GAS", "SWTH", "USD", "ETH"]:
            api_params["bases"] = [base]
        return self.request.get(path='/exchange/pairs', params=api_params)

Response: This function returns a list of trading pairs offered on Switcheo decentralized exchange.


Instruction:
Can you generate the documentation for the following Python 3 function
def get_orders(self, address, chain_name='NEO', contract_version='V2', pair=None, from_epoch_time=None,
                   order_status=None, before_id=None, limit=50):
        """
        Function to fetch the order history of the given address.
        Execution of this function is as follows::

            get_orders(address=neo_get_scripthash_from_address(address=address))

        The expected return result for this function is as follows::

            [{
                'id': '7cbdf481-6acf-4bf3-a1ed-4773f31e6931',
                'blockchain': 'neo',
                'contract_hash': 'a195c1549e7da61b8da315765a790ac7e7633b82',
                'address': 'fea2b883725ef2d194c9060f606cd0a0468a2c59',
                'side': 'buy',
                'offer_asset_id': 'c56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b',
                'want_asset_id': 'ab38352559b8b203bde5fddfa0b07d8b2525e132',
                'offer_amount': '53718500',
                'want_amount': '110000000000',
                'transfer_amount': '0',
                'priority_gas_amount': '0',
                'use_native_token': True,
                'native_fee_transfer_amount': 0,
                'deposit_txn': None,
                'created_at': '2018-08-03T02:44:47.692Z',
                'status': 'processed',
                'fills': [{
                    'id': 'b6f9e530-60ff-46ff-9a71-362097a2025e',
                    'offer_hash': '95b3b03be0bff8f58aa86a8dd599700bbaeaffc05078329d5b726b6b995f4cda',
                    'offer_asset_id': 'c56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b',
                    'want_asset_id': 'ab38352559b8b203bde5fddfa0b07d8b2525e132',
                    'fill_amount': '47833882',
                    'want_amount': '97950000000',
                    'filled_amount': '',
                    'fee_asset_id': 'ab38352559b8b203bde5fddfa0b07d8b2525e132',
                    'fee_amount': '73462500',
                    'price': '0.00048835',
                    'txn': None,
                    'status': 'success',
                    'created_at': '2018-08-03T02:44:47.706Z',
                    'transaction_hash': '694745a09e33845ec008cfb79c73986a556e619799ec73274f82b30d85bda13a'
                }],
                'makes': [{
                    'id': '357088a0-cc80-49ab-acdd-980589c2d7d8',
                    'offer_hash': '420cc85abf02feaceb1bcd91489a0c1949c972d2a9a05ae922fa15d79de80c00',
                    'available_amount': '0',
                    'offer_asset_id': 'c56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b',
                    'offer_amount': '5884618',
                    'want_asset_id': 'ab38352559b8b203bde5fddfa0b07d8b2525e132',
                    'want_amount': '12050000000',
                    'filled_amount': '0.0',
                    'txn': None,
                    'cancel_txn': None,
                    'price': '0.000488350041493775933609958506224066390041494',
                    'status': 'cancelled',
                    'created_at': '2018-08-03T02:44:47.708Z',
                    'transaction_hash': '1afa946546550151bbbd19f197a87cec92e9be58c44ec431cae42076298548b7',
                    'trades': []
                }]
            }, {
            ....
            }]

        :param address: The ScriptHash of the address to filter orders for.
        :type address: str
        :param pair: The trading pair to filter order requests on.
        :type pair: str
        :param chain_name: The name of the chain to find orders against.
        :type chain_name: str
        :param contract_version: The version of the contract to find orders against.
        :type contract_version: str
        :param from_epoch_time: Only return orders that are last updated at or after this time.
        :type from_epoch_time: int
        :param order_status: Only return orders have this status. Possible values are open, cancelled, completed.
        :type order_status: str
        :param before_id: Only return orders that are created before the order with this id.
        :type before_id: str
        :param limit: Only return up to this number of orders (min: 1, max: 200, default: 50).
        :type limit: int
        :return: List of dictionaries containing the orders for the given NEO address and (optional) trading pair.
        """
        api_params = {
            "address": address,
            "contract_hash": self.get_contracts()[chain_name.upper()][contract_version.upper()],
            "limit": limit
        }
        if pair is not None:
            api_params['pair'] = pair
        if from_epoch_time is not None:
            api_params['from_epoch_time'] = from_epoch_time
        if order_status is not None:
            api_params['order_status'] = order_status
        if before_id is not None:
            api_params['before_id'] = before_id
        return self.request.get(path='/orders', params=api_params)

Response: This function returns the order history for the given address.


Instruction:
How would you explain what the following Python 3 function does
def get_balance(self, addresses, contracts):
        api_params = {
            "addresses[]": addresses,
            "contract_hashes[]": contracts
        }
        return self.request.get(path='/balances', params=api_params)

Response: This function returns the current balance for the given address in the Switcheo smart contract.


Instruction:
Write a Python 3 script to
configure a file extension to be ignored.

Response:
def ignore_file_extension(self, extension):
        """
        Configure a file extension to be ignored.

        :param extension: file extension to be ignored
                          (ex. .less, .scss, etc)
        """
        logger.info('Ignoring file extension: {}'.format(extension))
        self.watcher.ignore_file_extension(extension)


Instruction:
Implement a Python 3 function for
adding the given filepath for watcher list.

Response:
def watch(self, filepath, func=None, delay=None):
        """Add the given filepath for watcher list.

        Once you have intialized a server, watch file changes before
        serve the server::

            server.watch('static/*.stylus', 'make static')
            def alert():
                print('foo')
            server.watch('foo.txt', alert)
            server.serve()

        :param filepath: files to be watched, it can be a filepath,
                         a directory, or a glob pattern
        :param func: the function to be called, it can be a string of
                     shell command, or any callable object without
                     parameters
        :param delay: Delay sending the reload message. Use 'forever' to
                      not send it. This is useful to compile sass files to
                      css, but reload on changed css files then only.
        """
        if isinstance(func, string_types):
            func = shell(func)

        self.watcher.watch(filepath, func, delay)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def serve(self, liveport=None, host=None, restart_delay=2):
        host = host or '127.0.0.1'
        logger.info('Serving on http://%s:%s' % (host, liveport))

        self.application(host, liveport=liveport)

        try:
            self.watcher._changes.append(('__livereload__', restart_delay))
            LiveReloadHandler.start_tasks()
            IOLoop.instance().start()
        except KeyboardInterrupt:
            logger.info('Shutting down...')

Response: Start serving the application with the given port.


Instruction:
Write a Python 3 script for
checking if there are changes run the given task.

Response:
def examine(self):
        """Check if there are changes, if true, run the given task."""
        if self._changes:
            return self._changes.pop()

        # clean filepath
        self.filepath = None
        delays = set([0])
        for path in self._tasks:
            item = self._tasks[path]
            if self.is_changed(path, item['ignore']):
                func = item['func']
                func and func()
                delay = item['delay']
                if delay:
                    delays.add(delay)

        if 'forever' in delays:
            delay = 'forever'
        else:
            delay = max(delays)
        return self.filepath, delay


Instruction:
Write a Python 3 function that can
perform the LiveReload request.

Response:
def livereload_request(self, **options):
        """
        Performs the LiveReload request.
        """
        style = color_style()
        verbosity = int(options['verbosity'])
        host = '%s:%d' % (
            options['livereload_host'],
            options['livereload_port'],
        )
        try:
            urlopen('http://%s/forcereload' % host)
            self.message('LiveReload request emitted.\n',
                         verbosity, style.HTTP_INFO)
        except IOError:
            pass


Instruction:
Implement a function in Python 3 to
get the handler for this command.

Response:
def get_handler(self, *args, **options):
        """
        Entry point to plug the LiveReload feature.
        """
        handler = super(Command, self).get_handler(*args, **options)
        if options['use_livereload']:
            threading.Timer(1, self.livereload_request, kwargs=options).start()
        return handler


Instruction:
Write a Python 3 function for
returning the unsigned value along with its timestamp the time when it got dumped.

Response:
def loads_with_timestamp(value, salt):
    """Returns the unsigned value along with its timestamp, the time when it
    got dumped."""
    try:
        signing.loads(value, salt=salt, max_age=-999999)
    except signing.SignatureExpired as e:
        age = float(str(e).split('Signature age ')[1].split(' >')[0])
        timestamp = timezone.now() - datetime.timedelta(seconds=age)
        return timestamp, signing.loads(value, salt=salt)


Instruction:
Can you write a function in Python 3 where it
sets TCP keepalive on an open socket.

Response:
def set_keepalive(sock, after_idle_sec=1, interval_sec=3, max_fails=5):
    """Set TCP keepalive on an open socket.

    It activates after 1 second (after_idle_sec) of idleness,
    then sends a keepalive ping once every 3 seconds (interval_sec),
    and closes the connection after 5 failed ping (max_fails), or 15 seconds
    """
    if hasattr(socket, "SO_KEEPALIVE"):
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
    if hasattr(socket, "TCP_KEEPIDLE"):
        sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, after_idle_sec)
    if hasattr(socket, "TCP_KEEPINTVL"):
        sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, interval_sec)
    if hasattr(socket, "TCP_KEEPCNT"):
        sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, max_fails)


Instruction:
Create a Python 3 function to
ingest the veron catalogue into the catalogues database

Response:
def ingest(self):
        """ingest the veron catalogue into the catalogues database

        See class docstring for usage.
        """
        self.log.debug('starting the ``get`` method')

        dictList = self._create_dictionary_of_veron()

        tableName = self.dbTableName
        createStatement = """
    CREATE TABLE `%(tableName)s` (
      `primaryId` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'An internal counter',
      `B_V` float DEFAULT NULL,
      `U_B` float DEFAULT NULL,
      `abs_magnitude` float DEFAULT NULL,
      `dateCreated` datetime DEFAULT  CURRENT_TIMESTAMP,
      `decDeg` double DEFAULT NULL,
      `magnitude` float DEFAULT NULL,
      `raDeg` double DEFAULT NULL,
      `class` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
      `name` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL,
      `redshift` float DEFAULT NULL,
      `not_radio` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
      `magnitude_filter` varchar(10) COLLATE utf8_unicode_ci DEFAULT 'V',
      `htm16ID` bigint(20) DEFAULT NULL,
      `redshift_flag` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL,
      `spectral_classification` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL,
      `dateLastModified` datetime DEFAULT CURRENT_TIMESTAMP,
      `updated` varchar(45) DEFAULT '0',
      `htm10ID` bigint(20) DEFAULT NULL,
      `htm13ID` bigint(20) DEFAULT NULL,
      PRIMARY KEY (`primaryId`),
      UNIQUE KEY `radeg_decdeg` (`raDeg`,`decDeg`),
      KEY `idx_htm16ID` (`htm16ID`),
      KEY `idx_htm10ID` (`htm10ID`),
      KEY `idx_htm13ID` (`htm13ID`)
    ) ENGINE=MyISAM AUTO_INCREMENT=168945 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
""" % locals()

        self.add_data_to_database_table(
            dictList=dictList,
            createStatement=createStatement
        )

        self.log.debug('completed the ``get`` method')
        return None


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _create_dictionary_of_veron(
            self):
        """create a list of dictionaries containing all the rows in the veron catalogue

        **Return:**
            - ``dictList`` - a list of dictionaries containing all the rows in the veron catalogue

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``_create_dictionary_of_veron`` method')

        dictList = []
        lines = string.split(self.catData, '\n')

        totalCount = len(lines)
        count = 0
        switch = 0
        for line in lines:
            if (len(line) == 0 or line[0] in ["#", " "]) and switch == 0:
                continue
            else:
                switch = 1
            count += 1
            if count > 1:
                # Cursor up one line and clear line
                sys.stdout.write("\x1b[1A\x1b[2K")
            print "%(count)s / %(totalCount)s veron data added to memory" % locals()

            if count == 1:
                theseKeys = []
                someKeys = string.split(line, '|')
                for key in someKeys:
                    if key == "_RAJ2000":
                        key = "raDeg"
                    if key == "_DEJ2000":
                        key = "decDeg"
                    if key == "Cl":
                        key = "class"
                    if key == "nR":
                        key = "not_radio"
                    if key == "Name":
                        key = "name"
                    if key == "l_z":
                        key = "redshift_flag"
                    if key == "z":
                        key = "redshift"
                    if key == "Sp":
                        key = "spectral_classification"
                    if key == "n_Vmag":
                        key = "magnitude_filter"
                    if key == "Vmag":
                        key = "magnitude"
                    if key == "B-V":
                        key = "B_V"
                    if key == "U-B":
                        key = "U_B"
                    if key == "Mabs":
                        key = "abs_magnitude"
                    theseKeys.append(key)
                continue

            if count in [2, 3]:
                continue

            thisDict = {}
            theseValues = string.split(line, '|')

            for k, v in zip(theseKeys, theseValues):
                v = v.strip()
                if len(v) == 0 or v == "-":
                    v = None
                thisDict[k] = v
            dictList.append(thisDict)

        self.log.debug(
            'completed the ``_create_dictionary_of_veron`` method')
        return dictList

Response: create a list of dictionaries containing all the rows in the catalogue


Instruction:
Make a summary of the following Python 3 code
def main(arguments=None):
    # setup the command-line util settings

    su = tools(
        arguments=arguments,
        docString=__doc__,
        logLevel="WARNING",
        options_first=False,
        projectName="sherlock"
    )
    arguments, settings, log, dbConn = su.setup()

    # unpack remaining cl arguments using `exec` to setup the variable names
    # automatically
    for arg, val in arguments.iteritems():
        if arg[0] == "-":
            varname = arg.replace("-", "") + "Flag"
        else:
            varname = arg.replace("<", "").replace(">", "")
            if varname == "import":
                varname = "iimport"
        if isinstance(val, str) or isinstance(val, unicode):
            exec(varname + " = '%s'" % (val,))
        else:
            exec(varname + " = %s" % (val,))
        if arg == "--dbConn":
            dbConn = val
        log.debug('%s = %s' % (varname, val,))

    ## START LOGGING ##
    startTime = times.get_now_sql_datetime()
    log.debug(
        '--- STARTING TO RUN THE cl_utils.py AT %s' %
        (startTime,))

    # call the worker function
    # x-if-settings-or-database-credientials
    if init:
        from os.path import expanduser
        home = expanduser("~")
        filepath = home + "/.config/sherlock/sherlock.yaml"
        cmd = """open %(filepath)s""" % locals()
        p = Popen(cmd, stdout=PIPE, stderr=PIPE, shell=True)
        try:
            cmd = """open %(filepath)s""" % locals()
            p = Popen(cmd, stdout=PIPE, stderr=PIPE, shell=True)
        except:
            pass
        try:
            cmd = """start %(filepath)s""" % locals()
            p = Popen(cmd, stdout=PIPE, stderr=PIPE, shell=True)
        except:
            pass

    if match or dbmatch:
        if verboseFlag:
            verbose = 2
        else:
            verbose = 1

        if skipNedUpdateFlag:
            updateNed = False
        else:
            updateNed = True

        if skipMagUpdateFlag:
            updatePeakMags = False
        else:
            updatePeakMags = True

        classifier = transient_classifier.transient_classifier(
            log=log,
            settings=settings,
            ra=ra,
            dec=dec,
            name=False,
            verbose=verbose,
            update=updateFlag,
            updateNed=updateNed,
            updatePeakMags=updatePeakMags
        )
        classifier.classify()

    if clean:
        cleaner = database_cleaner(
            log=log,
            settings=settings
        )
        cleaner.clean()
    if wiki:
        updateWiki = update_wiki_pages(
            log=log,
            settings=settings
        )
        updateWiki.update()

    if iimport and ned:
        ned = nedStreamImporter(
            log=log,
            settings=settings,
            coordinateList=["%(ra)s %(dec)s" % locals()],
            radiusArcsec=radiusArcsec
        )
        ned.ingest()
    if iimport and cat:

        if cat_name == "veron":
            catalogue = veronImporter(
                log=log,
                settings=settings,
                pathToDataFile=pathToDataFile,
                version=cat_version,
                catalogueName=cat_name
            )
            catalogue.ingest()

        if "ned_d" in cat_name:
            catalogue = nedImporter(
                log=log,
                settings=settings,
                pathToDataFile=pathToDataFile,
                version=cat_version,
                catalogueName=cat_name
            )
            catalogue.ingest()
    if iimport and stream:
        if "marshall" in stream_name:
            stream = marshallImporter(
                log=log,
                settings=settings,
            )
            stream.ingest()
        if "ifs" in stream_name:
            stream = ifsImporter(
                log=log,
                settings=settings
            )
            stream.ingest()
    if not init and not match and not clean and not wiki and not iimport and ra:

        classifier = transient_classifier.transient_classifier(
            log=log,
            settings=settings,
            ra=ra,
            dec=dec,
            name=False,
            verbose=verboseFlag
        )
        classifier.classify()

    if info:
        print "sherlock-catalogues"
        wiki = update_wiki_pages(
            log=log,
            settings=settings
        )
        table = list(wiki._get_table_infos(trimmed=True))

        dataSet = list_of_dictionaries(
            log=log,
            listOfDictionaries=table
        )
        tableData = dataSet.reST(filepath=None)
        print tableData
        print

        print "Crossmatch Streams"
        table = list(wiki._get_stream_view_infos(trimmed=True))
        dataSet = list_of_dictionaries(
            log=log,
            listOfDictionaries=table
        )
        tableData = dataSet.reST(filepath=None)
        print tableData
        print

        print "Views on Catalogues and Streams"

        table = list(wiki._get_view_infos(trimmed=True))
        dataSet = list_of_dictionaries(
            log=log,
            listOfDictionaries=table
        )
        tableData = dataSet.reST(filepath=None)
        print tableData

    if "dbConn" in locals() and dbConn:
        dbConn.commit()
        dbConn.close()
    ## FINISH LOGGING ##
    endTime = times.get_now_sql_datetime()
    runningTime = times.calculate_time_difference(startTime, endTime)
    log.debug('-- FINISHED ATTEMPT TO RUN THE cl_utils.py AT %s (RUNTIME: %s) --' %
              (endTime, runningTime, ))

    return

Response: This is the main function used by the cl_utils. py script.


Instruction:
Can you implement a function in Python 3 that
displays the effect of this filter

Response:
def display(self, display_before=False):
        """shows the effect of this filter"""
        try:
            copy = self.image.copy()
        except AttributeError:
            raise Exception("You need to set the Filter.image attribute for displaying")
        copy = BrightnessProcessor(brightness=0.6).process(copy)
        s, g = self._input, self.good_segments_indexes
        draw_segments(copy, s[g], (0, 255, 0))
        draw_segments(copy, s[True ^ g], (0, 0, 255))
        show_image_and_wait_for_key(copy, "segments filtered by " + self.__class__.__name__)


Instruction:
Implement a Python 3 function for
parsing the output analysis files from MIP for adding info to trend database

Response:
def parse_mip_analysis(mip_config_raw: dict, qcmetrics_raw: dict, sampleinfo_raw: dict) -> dict:
    """Parse the output analysis files from MIP for adding info
    to trend database

    Args:
        mip_config_raw (dict): raw YAML input from MIP analysis config file
        qcmetrics_raw (dict): raw YAML input from MIP analysis qc metric file
        sampleinfo_raw (dict): raw YAML input from MIP analysis qc sample info file
    Returns:
        dict: parsed data
    """
    outdata = _define_output_dict()

    _config(mip_config_raw, outdata)
    _qc_metrics(outdata, qcmetrics_raw)
    _qc_sample_info(outdata, sampleinfo_raw)

    return outdata


Instruction:
Implement a Python 3 function for
cleaning up and run some maintenance tasks on the crossmatch catalogue helper tables and views

Response:
def clean(self):
        """*clean up and run some maintance tasks on the crossmatch catalogue helper tables*

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug('starting the ``get`` method')

        self._update_tcs_helper_catalogue_tables_info_with_new_tables()
        self._updated_row_counts_in_tcs_helper_catalogue_tables_info()
        self._clean_up_columns()
        self._update_tcs_helper_catalogue_views_info_with_new_views()
        self._clean_up_columns()
        self._updated_row_counts_in_tcs_helper_catalogue_tables_info()

        print "`tcs_helper_catalogue_tables_info` & `tcs_helper_catalogue_views_info` database tables updated"

        self.log.debug('completed the ``get`` method')
        return None


Instruction:
How would you explain what the following Python 3 function does
def _updated_row_counts_in_tcs_helper_catalogue_tables_info(
            self):
        """ updated row counts in tcs catalogue tables

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``_updated_row_counts_in_tcs_helper_catalogue_tables_info`` method')

        sqlQuery = u"""
            select * from tcs_helper_catalogue_tables_info where table_name like "%%stream" or (number_of_rows is null and legacy_table = 0)
        """ % locals()
        rows = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )

        for row in rows:
            tbName = row["table_name"]

            sqlQuery = u"""
                update tcs_helper_catalogue_tables_info set number_of_rows = (select count(*) as count from %(tbName)s) where table_name = "%(tbName)s"
            """ % locals()
            writequery(
                log=self.log,
                sqlQuery=sqlQuery,
                dbConn=self.cataloguesDbConn,
            )

        sqlQuery = u"""
            select * from tcs_helper_catalogue_views_info where (number_of_rows is null and legacy_view = 0)
        """ % locals()
        rows = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )

        for row in rows:
            tbName = row["view_name"]
            print tbName

            sqlQuery = u"""
                update tcs_helper_catalogue_views_info set number_of_rows = (select count(*) as count from %(tbName)s) where view_name = "%(tbName)s"
            """ % locals()
            writequery(
                log=self.log,
                sqlQuery=sqlQuery,
                dbConn=self.cataloguesDbConn,
            )

        self.log.debug(
            'completed the ``_updated_row_counts_in_tcs_helper_catalogue_tables_info`` method')
        return None

Response: update row counts in tcs catalogue tables info


Instruction:
How would you implement a function in Python 3 that
updates tcs helper catalogue tables info with new tables

Response:
def _update_tcs_helper_catalogue_tables_info_with_new_tables(
            self):
        """update tcs helper catalogue tables info with new tables

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``_update_tcs_helper_catalogue_tables_info_with_new_tables`` method')

        sqlQuery = u"""
            SELECT max(id) as thisId FROM tcs_helper_catalogue_tables_info;
        """ % locals()
        thisId = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )
        try:
            highestId = thisId[0]["thisId"] + 1
        except:
            highestId = 1

        sqlQuery = u"""
            SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE='BASE TABLE' AND TABLE_SCHEMA like '%%catalogues%%' and TABLE_NAME like "tcs_cat%%";
        """ % locals()
        tablesInDatabase = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )

        sqlQuery = u"""
            SELECT table_name, old_table_name FROM tcs_helper_catalogue_tables_info;
        """ % locals()
        tableList = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )
        tbList = []
        oldList = []
        for tb in tableList:
            oldList.append(tb["old_table_name"])

        for tb in tableList:
            if tb["old_table_name"] not in tbList:
                tbList.append(tb["old_table_name"])
            if tb["table_name"] not in tbList:
                tbList.append(tb["table_name"])

        for tb in tablesInDatabase:
            if tb["TABLE_NAME"] not in tbList:
                thisTableName = tb["TABLE_NAME"]
                print "`%(thisTableName)s` added to `tcs_helper_catalogue_tables_info` database table" % locals()
                sqlQuery = u"""
                    INSERT INTO tcs_helper_catalogue_tables_info (
                            id,
                            table_name
                        )
                        VALUES (
                            %(highestId)s,
                            "%(thisTableName)s"
                    )""" % locals()
                writequery(
                    log=self.log,
                    sqlQuery=sqlQuery,
                    dbConn=self.cataloguesDbConn,
                )
                highestId += 1

        self.log.debug(
            'completed the ``_update_tcs_helper_catalogue_tables_info_with_new_tables`` method')
        return None


Instruction:
Implement a function in Python 3 to
clean up the columns of the tcs_helper_catalogue_tables_info table_name and views_name.

Response:
def _clean_up_columns(
            self):
        """clean up columns

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug('starting the ``_clean_up_columns`` method')

        sqlQueries = [
            "update tcs_helper_catalogue_tables_info set old_table_name = table_name where old_table_name is null;",
            "update tcs_helper_catalogue_tables_info set version_number = 'stream' where table_name like '%%stream' and version_number is null;",
            """update tcs_helper_catalogue_tables_info set in_ned = 0 where table_name like '%%stream' and in_ned is null;""",
            """update tcs_helper_catalogue_tables_info set vizier_link = 0 where table_name like '%%stream' and vizier_link is null;""",
            "update tcs_helper_catalogue_views_info set old_view_name = view_name where old_view_name is null;",
        ]

        for sqlQuery in sqlQueries:
            writequery(
                log=self.log,
                sqlQuery=sqlQuery,
                dbConn=self.cataloguesDbConn,
            )

        # VIEW OBJECT TYPES
        sqlQuery = u"""
            SELECT view_name FROM tcs_helper_catalogue_views_info where legacy_view = 0 and object_type is null;
        """ % locals()
        rows = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )

        for row in rows:
            view_name = row["view_name"]
            object_type = view_name.replace("tcs_view_", "").split("_")[0]

            sqlQuery = u"""
                update tcs_helper_catalogue_views_info set object_type = "%(object_type)s" where view_name = "%(view_name)s"
            """ % locals()
            writequery(
                log=self.log,
                sqlQuery=sqlQuery,
                dbConn=self.cataloguesDbConn,
            )

        # MASTER TABLE ID FOR VIEWS
        sqlQuery = u"""
            SELECT view_name FROM tcs_helper_catalogue_views_info where legacy_view = 0 and table_id is null;
        """ % locals()
        rows = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )

        for row in rows:
            view_name = row["view_name"]
            table_name = view_name.replace("tcs_view_", "").split("_")[1:]
            table_name = ("_").join(table_name)
            table_name = "tcs_cat_%(table_name)s" % locals()

            sqlQuery = u"""
                update tcs_helper_catalogue_views_info set table_id = (select id from tcs_helper_catalogue_tables_info where table_name = "%(table_name)s") where view_name = "%(view_name)s"
            """ % locals()
            writequery(
                log=self.log,
                sqlQuery=sqlQuery,
                dbConn=self.cataloguesDbConn,
            )

        self.log.debug('completed the ``_clean_up_columns`` method')
        return None


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _update_tcs_helper_catalogue_views_info_with_new_views(
            self):
        """ update tcs helper catalogue tables info with new tables

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``_update_tcs_helper_catalogue_views_info_with_new_views`` method')

        sqlQuery = u"""
            SELECT max(id) as thisId FROM tcs_helper_catalogue_views_info;
        """ % locals()
        thisId = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )
        try:
            highestId = thisId[0]["thisId"] + 1
        except:
            highestId = 1

        sqlQuery = u"""
            SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE='VIEW' AND TABLE_SCHEMA like '%%catalogues%%' and TABLE_NAME like "tcs_view%%" and TABLE_NAME not like "%%helper%%";
        """ % locals()
        tablesInDatabase = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )

        sqlQuery = u"""
            SELECT view_name FROM tcs_helper_catalogue_views_info;
        """ % locals()
        tableList = readquery(
            log=self.log,
            sqlQuery=sqlQuery,
            dbConn=self.cataloguesDbConn,
            quiet=False
        )
        tbList = []
        for tb in tableList:
            tbList.append(tb["view_name"])

        for tb in tablesInDatabase:
            if tb["TABLE_NAME"] not in tbList:
                thisViewName = tb["TABLE_NAME"]
                print "`%(thisViewName)s` added to `tcs_helper_catalogue_views_info` database table" % locals()
                sqlQuery = u"""
                    INSERT INTO tcs_helper_catalogue_views_info (
                            id,
                            view_name
                        )
                        VALUES (
                            %(highestId)s,
                            "%(thisViewName)s"
                    )""" % locals()
                writequery(
                    log=self.log,
                    sqlQuery=sqlQuery,
                    dbConn=self.cataloguesDbConn,
                )
                highestId += 1

        self.log.debug(
            'completed the ``_update_tcs_helper_catalogue_views_info_with_new_views`` method')
        return None

Response: update tcs helper catalogue tables info with new tables


Instruction:
Given the following Python 3 function, write the documentation
def restart_listener(self, topics):
        '''Restart listener after configuration update.
        '''
        if self.listener is not None:
            if self.listener.running:
                self.stop()
        self.__init__(topics=topics)

Response: Restart listener after configuration update.


Instruction:
Can you write a function in Python 3 where it
stops the listener thread.

Response:
def stop(self):
        '''Stop listener.'''
        self.logger.debug("Stopping listener.")
        self.listener.stop()
        if self.thread is not None:
            self.thread.join()
            self.thread = None
        self.logger.debug("Listener stopped.")


Instruction:
Can you tell what is the following Python 3 function doing
def create_subscriber(self):
        '''Create a subscriber instance using specified addresses and
        message types.
        '''
        if self.subscriber is None:
            if self.topics:
                self.subscriber = NSSubscriber(self.services, self.topics,
                                               addr_listener=True,
                                               addresses=self.addresses,
                                               nameserver=self.nameserver)
                self.recv = self.subscriber.start().recv

Response: Create a subscriber instance using specified addresses and message types.


Instruction:
Make a summary of the following Python 3 code
def stop(self):
        '''Stop subscriber and delete the instance
        '''
        self.running = False
        time.sleep(1)
        if self.subscriber is not None:
            self.subscriber.stop()
            self.subscriber = None

Response: Stop subscriber and delete the instance


Instruction:
Can you write a function in Python 3 where it
gives a dictionary of all version data available determine the current version

Response:
def get_current_semver(data):
    """Given a dictionary of all version data available, determine the current version"""
    # get the not-none values from data
    known = {
        key: data.get(alias)
        for key, alias in config._forward_aliases.items()
        if data.get(alias) is not None
    }

    # prefer the strict field, if available
    potentials = [
        known.pop(Constants.VERSION_STRICT_FIELD, None),
        known.pop(Constants.VERSION_FIELD, None),
    ]

    from_components = [known.get(k) for k in SemVerSigFig._fields if k in known]
    if len(from_components) == 3:
        potentials.append(".".join(from_components))

    versions = set()
    for potential in potentials:
        if not potential:
            continue
        match = re_semver.match(potential)
        if match:
            parts = match.groupdict()
            parts.pop("tail")
            versions.add(SemVer(**parts))

    if len(versions) > 1:
        raise ValueError("conflicting versions within project: %s" % versions)

    if not versions:
        _LOG.debug("key pairs found: \n%r", known)
        raise ValueError("could not find existing semver")
    return versions.pop()


Instruction:
Write a Python 3 script for
defining how to increment semver based on which significant figure is triggered

Response:
def make_new_semver(current_semver, all_triggers, **overrides):
    """Defines how to increment semver based on which significant figure is triggered"""
    new_semver = {}
    bumped = False
    for sig_fig in SemVerSigFig:  # iterate sig figs in order of significance
        value = getattr(current_semver, sig_fig)
        override = overrides.get(sig_fig)
        if override is not None:
            new_semver[sig_fig] = override
            if int(override) > int(value):
                bumped = True
        elif bumped:
            new_semver[sig_fig] = "0"
        elif sig_fig in all_triggers:
            new_semver[sig_fig] = str(int(value) + 1)
            bumped = True
        else:
            new_semver[sig_fig] = value
    return SemVer(**new_semver)


Instruction:
Write a Python 3 script for
parsing the output of a traceroute execution into a tree structure.

Response:
def loads(data):
    """Parser entry point. Parses the output of a traceroute execution"""
    data += "\n_EOS_" # Append EOS token. Helps to match last RE_HOP

    # Get headers
    match_dest = RE_HEADER.search(data)
    dest_name = match_dest.group(1)
    dest_ip = match_dest.group(2)

    # The Traceroute is the root of the tree.
    traceroute = Traceroute(dest_name, dest_ip)

    # Get hops
    matches_hop = RE_HOP.findall(data)

    for match_hop in matches_hop:
        # Initialize a hop
        idx = int(match_hop[0])
        if match_hop[1]:
            asn = int(match_hop[1])
        else:
            asn = None
        hop = Hop(idx, asn)

        # Parse probes data: <name> | <(IP)> | <rtt> | 'ms' | '*'
        probes_data = match_hop[2].split()
        # Get rid of 'ms': <name> | <(IP)> | <rtt> | '*'
        probes_data = filter(lambda s: s.lower() != 'ms', probes_data)

        i = 0
        while i < len(probes_data):
            # For each hop parse probes
            name = None
            ip = None
            rtt = None
            anno = ''

            # RTT check comes first because RE_PROBE_NAME can confuse rtt with an IP as name
            # The regex RE_PROBE_NAME can be improved
            if RE_PROBE_RTT.match(probes_data[i]):
                # Matched rtt, so name and IP have been parsed before
                rtt = float(probes_data[i])
                i += 1
            elif RE_PROBE_NAME.match(probes_data[i]):
                # Matched a name, so next elements are IP and rtt
                name = probes_data[i]
                ip = probes_data[i+1].strip('()')
                rtt = float(probes_data[i+2])
                i += 3
            elif RE_PROBE_TIMEOUT.match(probes_data[i]):
                # Its a timeout, so maybe name and IP have been parsed before
                # or maybe not. But it's Hop job to deal with it
                rtt = None
                i += 1
            else:
                ext = "i: %d\nprobes_data: %s\nname: %s\nip: %s\nrtt: %s\nanno: %s" % (i, probes_data, name, ip, rtt, anno)
                raise ParseError("Parse error \n%s" % ext)
            # Check for annotation
            try:
                if RE_PROBE_ANNOTATION.match(probes_data[i]):
                    anno = probes_data[i]
                    i += 1
            except IndexError:
                pass

            probe = Probe(name, ip, rtt, anno)
            hop.add_probe(probe)

        traceroute.add_hop(hop)

    return traceroute


Instruction:
Write a Python 3 function for
adding a Probe instance to this hop s results.

Response:
def add_probe(self, probe):
        """Adds a Probe instance to this hop's results."""
        if self.probes:
            probe_last = self.probes[-1]
            if not probe.ip:
                probe.ip = probe_last.ip
                probe.name = probe_last.name
        self.probes.append(probe)


Instruction:
How would you code a function in Python 3 to
function decorator to make function synchronized on self. _lock.

Response:
def synchronized(func, *args, **kwargs):
    """Function decorator to make function synchronized on ``self._lock``.

    If the first argument to the function (hopefully self) does not have a _lock
    attribute, then this decorator does nothing.
    """
    if not (args and hasattr(args[0], '_lock')):
        return func(*args, **kwargs)
    with args[0]._lock: # pylint: disable=W0212
        return func(*args, **kwargs)


Instruction:
Given the following Python 3 function, write the documentation
def normalize_job_id(job_id):
	if not isinstance(job_id, uuid.UUID):
		job_id = uuid.UUID(job_id)
	return job_id

Response: Normalizes a value to a job id.


Instruction:
Can you implement a function in Python 3 that
returns a : py : class : datetime. datetime instance representing the current time.

Response:
def now(self):
		"""
		Return a :py:class:`datetime.datetime` instance representing the current time.

		:rtype: :py:class:`datetime.datetime`
		"""
		if self.use_utc:
			return datetime.datetime.utcnow()
		else:
			return datetime.datetime.now()


Instruction:
Write a Python 3 function that can
start the JobManager thread.

Response:
def start(self):
		"""
		Start the JobManager thread.
		"""
		if self._thread_running.is_set():
			raise RuntimeError('the JobManager has already been started')
		self._thread.start()
		self._thread_running.wait()
		return


Instruction:
How would you implement a function in Python 3 that
stops the JobManager thread.

Response:
def stop(self):
		"""
		Stop the JobManager thread.
		"""
		self.logger.debug('stopping the job manager')
		self._thread_running.clear()
		self._thread_shutdown.wait()
		self._job_lock.acquire()
		self.logger.debug('waiting on ' + str(len(self._jobs)) + ' job threads')
		for job_desc in self._jobs.values():
			if job_desc['job'] is None:
				continue
			if not job_desc['job'].is_alive():
				continue
			job_desc['job'].join()

		# the job lock must be released before the thread can be joined because the thread routine acquires it before
		# checking if it should exit, see https://github.com/zeroSteiner/smoke-zephyr/issues/4 for more details
		self._job_lock.release()
		self._thread.join()

		self.logger.info('the job manager has been stopped')
		return


Instruction:
Write a Python 3 script for
adding a job and run it once immediately.

Response:
def job_run(self, callback, parameters=None):
		"""
		Add a job and run it once immediately.

		:param function callback: The function to run asynchronously.
		:param parameters: The parameters to be provided to the callback.
		:type parameters: list, tuple
		:return: The job id.
		:rtype: :py:class:`uuid.UUID`
		"""
		if not self._thread_running.is_set():
			raise RuntimeError('the JobManager is not running')
		parameters = (parameters or ())
		if not isinstance(parameters, (list, tuple)):
			parameters = (parameters,)
		job_desc = {}
		job_desc['job'] = JobRun(callback, parameters)
		job_desc['last_run'] = None
		job_desc['run_every'] = datetime.timedelta(0, 1)
		job_desc['callback'] = callback
		job_desc['parameters'] = parameters
		job_desc['enabled'] = True
		job_desc['tolerate_exceptions'] = False
		job_desc['run_count'] = 0
		job_desc['expiration'] = 0
		job_id = uuid.uuid4()
		self.logger.info('adding new job with id: ' + str(job_id) + ' and callback function: ' + callback.__name__)
		with self._job_lock:
			self._jobs[job_id] = job_desc
			self._job_execute(job_id)
		return job_id


Instruction:
Write a Python 3 function for
adding a job to the job manager.

Response:
def job_add(self, callback, parameters=None, hours=0, minutes=0, seconds=0, tolerate_exceptions=True, expiration=None):
		"""
		Add a job to the job manager.

		:param function callback: The function to run asynchronously.
		:param parameters: The parameters to be provided to the callback.
		:type parameters: list, tuple
		:param int hours: Number of hours to sleep between running the callback.
		:param int minutes: Number of minutes to sleep between running the callback.
		:param int seconds: Number of seconds to sleep between running the callback.
		:param bool tolerate_execptions: Whether to continue running a job after it has thrown an exception.
		:param expiration: When to expire and remove the job. If an integer
			is provided, the job will be executed that many times.  If a
			datetime or timedelta instance is provided, then the job will
			be removed after the specified time.
		:type expiration: int, :py:class:`datetime.timedelta`, :py:class:`datetime.datetime`
		:return: The job id.
		:rtype: :py:class:`uuid.UUID`
		"""
		if not self._thread_running.is_set():
			raise RuntimeError('the JobManager is not running')
		parameters = (parameters or ())
		if not isinstance(parameters, (list, tuple)):
			parameters = (parameters,)
		job_desc = {}
		job_desc['job'] = JobRun(callback, parameters)
		job_desc['last_run'] = None
		job_desc['run_every'] = datetime.timedelta(0, ((hours * 60 * 60) + (minutes * 60) + seconds))
		job_desc['callback'] = callback
		job_desc['parameters'] = parameters
		job_desc['enabled'] = True
		job_desc['tolerate_exceptions'] = tolerate_exceptions
		job_desc['run_count'] = 0
		if isinstance(expiration, int):
			job_desc['expiration'] = expiration
		elif isinstance(expiration, datetime.timedelta):
			job_desc['expiration'] = self.now() + expiration
		elif isinstance(expiration, datetime.datetime):
			job_desc['expiration'] = expiration
		else:
			job_desc['expiration'] = None
		job_id = uuid.uuid4()
		self.logger.info('adding new job with id: ' + str(job_id) + ' and callback function: ' + callback.__name__)
		with self._job_lock:
			self._jobs[job_id] = job_desc
		return job_id


Instruction:
Write a Python 3 function for
returning the number of enabled jobs.

Response:
def job_count_enabled(self):
		"""
		Return the number of enabled jobs.

		:return: The number of jobs that are enabled.
		:rtype: int
		"""
		enabled = 0
		for job_desc in self._jobs.values():
			if job_desc['enabled']:
				enabled += 1
		return enabled


Instruction:
Given the following Python 3 function, write the documentation
def job_enable(self, job_id):
		job_id = normalize_job_id(job_id)
		with self._job_lock:
			job_desc = self._jobs[job_id]
			job_desc['enabled'] = True

Response:
Enable a job.

		:param job_id: Job identifier to enable.
		:type job_id: :py:class:`uuid.UUID`


Instruction:
Make a summary of the following Python 3 code
def job_disable(self, job_id):
		job_id = normalize_job_id(job_id)
		with self._job_lock:
			job_desc = self._jobs[job_id]
			job_desc['enabled'] = False

Response:
Disable a job. Disabled jobs will not be executed.

		:param job_id: Job identifier to disable.
		:type job_id: :py:class:`uuid.UUID`


Instruction:
Can you create a Python 3 function that
deletes a job. :param job_id: Job identifier to delete. :type job_id: :py:class:`uuid.UUID` :param bool wait: If the job is currently running, wait for it to complete before deleting it.

Response:
def job_delete(self, job_id, wait=True):
		"""
		Delete a job.

		:param job_id: Job identifier to delete.
		:type job_id: :py:class:`uuid.UUID`
		:param bool wait: If the job is currently running, wait for it to complete before deleting it.
		"""
		job_id = normalize_job_id(job_id)
		self.logger.info('deleting job with id: ' + str(job_id) + ' and callback function: ' + self._jobs[job_id]['callback'].__name__)
		job_desc = self._jobs[job_id]
		with self._job_lock:
			job_desc['enabled'] = False
			if wait and self.job_is_running(job_id):
				job_desc['job'].join()
			del self._jobs[job_id]


Instruction:
How would you code a function in Python 3 to
check if a job is enabled.

Response:
def job_is_enabled(self, job_id):
		"""
		Check if a job is enabled.

		:param job_id: Job identifier to check the status of.
		:type job_id: :py:class:`uuid.UUID`
		:rtype: bool
		"""
		job_id = normalize_job_id(job_id)
		job_desc = self._jobs[job_id]
		return job_desc['enabled']


Instruction:
Can you write a function in Python 3 where it
checks if a job is currently running. False is returned if the job does not exist.

Response:
def job_is_running(self, job_id):
		"""
		Check if a job is currently running. False is returned if the job does
		not exist.

		:param job_id: Job identifier to check the status of.
		:type job_id: :py:class:`uuid.UUID`
		:rtype: bool
		"""
		job_id = normalize_job_id(job_id)
		if job_id not in self._jobs:
			return False
		job_desc = self._jobs[job_id]
		if job_desc['job']:
			return job_desc['job'].is_alive()
		return False


Instruction:
How would you explain what the following Python 3 function does
def bin_b64_type(arg):
	try:
		arg = base64.standard_b64decode(arg)
	except (binascii.Error, TypeError):
		raise argparse.ArgumentTypeError("{0} is invalid base64 data".format(repr(arg)))
	return arg

Response: An argparse type representing binary data encoded in base64.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def bin_hex_type(arg):
	if re.match(r'^[a-f0-9]{2}(:[a-f0-9]{2})+$', arg, re.I):
		arg = arg.replace(':', '')
	elif re.match(r'^(\\x[a-f0-9]{2})+$', arg, re.I):
		arg = arg.replace('\\x', '')
	try:
		arg = binascii.a2b_hex(arg)
	except (binascii.Error, TypeError):
		raise argparse.ArgumentTypeError("{0} is invalid hex data".format(repr(arg)))
	return arg

Response: An argparse type representing binary data encoded in hex.


Instruction:
Can you tell what is the following Python 3 function doing
def dir_type(arg):
	if not os.path.isdir(arg):
		raise argparse.ArgumentTypeError("{0} is not a valid directory".format(repr(arg)))
	return arg

Response: An argparse type representing a valid directory.


Instruction:
Can you tell what is the following Python 3 function doing
def email_type(arg):
	if not is_valid_email_address(arg):
		raise argparse.ArgumentTypeError("{0} is not a valid email address".format(repr(arg)))
	return arg

Response: An argparse type representing an email address.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def log_level_type(arg):
	if not arg.upper() in ('NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'):
		raise argparse.ArgumentTypeError("{0} is not a valid log level".format(repr(arg)))
	return getattr(logging, arg.upper())

Response: An argparse type representing a logging level.


Instruction:
Make a summary of the following Python 3 code
def port_type(arg):
	error_msg = "{0} is not a valid port".format(repr(arg))
	try:
		arg = ast.literal_eval(arg)
	except ValueError:
		raise argparse.ArgumentTypeError(error_msg)
	if arg < 0 or arg > 65535:
		raise argparse.ArgumentTypeError(error_msg)
	return arg

Response: An argparse type representing a tcp or udp port number.


Instruction:
Make a summary of the following Python 3 code
def timespan_type(arg):
	try:
		arg = parse_timespan(arg)
	except ValueError:
		raise argparse.ArgumentTypeError("{0} is not a valid time span".format(repr(arg)))
	return arg

Response: An argparse type representing a timespan such as 6h for 6 hours.


Instruction:
Can you tell what is the following Python 3 function doing
def search(self):
        self.log.debug('starting the ``search`` method')

        # ACCOUNT FOR TYPE OF SEARCH
        sqlWhere = False
        magnitudeLimitFilter = self.magnitudeLimitFilter
        disCols = ["zColName",
                   "distanceColName", "semiMajorColName"]
        sqlWhere = ""

        if self.physicalSearch == True:
            for d in disCols:
                colName = self.colMaps[self.tableName][d]
                if colName:
                    sqlWhere += " or `%(colName)s` is not null" % locals()
                if len(sqlWhere):
                    sqlWhere = " and (" + sqlWhere[4:] + ")"

        if self.upperMagnitudeLimit != False and self.upperMagnitudeLimit and not self.lowerMagnitudeLimit:
            mag = self.upperMagnitudeLimit
            sqlWhere += " and `%(magnitudeLimitFilter)s` > %(mag)s" % locals()
        if self.lowerMagnitudeLimit != False and self.lowerMagnitudeLimit and not self.upperMagnitudeLimit:
            mag = self.lowerMagnitudeLimit
            sqlWhere += " and `%(magnitudeLimitFilter)s` < %(mag)s" % locals()
        # THE GENERAL (INBETWEEN) CASE
        if self.lowerMagnitudeLimit != False and self.lowerMagnitudeLimit and self.upperMagnitudeLimit != False and self.upperMagnitudeLimit:
            upperMagnitudeLimit = self.upperMagnitudeLimit
            lowerMagnitudeLimit = self.lowerMagnitudeLimit
            sqlWhere += " and ((`%(magnitudeLimitFilter)s` > %(upperMagnitudeLimit)s and `%(magnitudeLimitFilter)s` < %(lowerMagnitudeLimit)s) or `%(magnitudeLimitFilter)s` is null)" % locals()

        if sqlWhere and " and" == sqlWhere[0:4]:
            sqlWhere = sqlWhere[5:]

        # THE COLUMN MAP LIFTED FROM ``tcs_helper_catalogue_tables_info` TABLE
        # IN CATALOGUE DATABASE (COLUMN NAMES ENDDING WITH 'ColName')
        columns = {}
        for k, v in self.colMaps[self.tableName].iteritems():
            name = k.replace("ColName", "")
            if "colname" in k.lower() and v:
                columns[k] = "`%(v)s` as `%(name)s`" % locals()
        columns = ", ".join(columns.values())

        cs = hmptyConesearch(
            log=self.log,
            dbConn=self.dbConn,
            tableName=self.tableName,
            columns=columns,
            ra=self.ra,
            dec=self.dec,
            radiusArcsec=self.radius,
            separations=True,
            distinct=False,
            sqlWhere=sqlWhere,
            closest=self.nearestOnly,
            raCol="ra",
            decCol="dec"
        )
        matchIndies, matches = cs.search()

        # MATCH ARE NOT NECESSARILY UNIQUE IF MANY TRANSIENT MATCH ONE SOURCE
        uniqueMatchDicts = []
        uniqueMatchDicts[:] = [copy.copy(d) for d in matches.list]

        self.log.debug('completed the ``search`` method')
        return matchIndies, uniqueMatchDicts

Response: search the transient sources for the specified transient sources and return the list of transient sources that match the criteria


Instruction:
How would you code a function in Python 3 to
perform an angular crossmatch against a given catalogue

Response:
def angular_crossmatch_against_catalogue(
        self,
        objectList,
        searchPara={},
        search_name="",
        brightnessFilter=False,
        physicalSearch=False,
        classificationType=False
    ):
        """*perform an angular separation crossmatch against a given catalogue in the database and annotate the crossmatch with some value added parameters (distances, physical separations, sub-type of transient etc)*

        **Key Arguments:**
            - ``objectList`` -- the list of transient locations to match against the crossmatch catalogue
            - ``searchPara`` -- the search parameters for this individual search as lifted from the search algorithm in the sherlock settings file
            - ``search_name`` -- the name of the search as given in the sherlock settings file
            - ``brightnessFilter`` -- is this search to be constrained by magnitude of the catalogue sources? Default *False*. [bright|faint|general]
            - ``physicalSearch`` -- is this angular search a sub-part of a physical separation search
            - ``classificationType`` -- synonym, association or annotation. Default *False*

         **Return:**
            - matchedObjects -- any sources matched against the object

        **Usage:**

            Take a list of transients from somewhere

            .. code-block:: python

                transients = [
                    {'ps1_designation': u'PS1-14aef',
                     'name': u'4L3Piiq',
                     'detection_list_id': 2,
                     'local_comments': u'',
                     'ra': 0.02548233704918263,
                     'followup_id': 2065412L,
                     'dec': -4.284933417540423,
                     'id': 1000006110041705700L,
                     'object_classification': 0L
                     },

                    {'ps1_designation': u'PS1-13dcr',
                     'name': u'3I3Phzx',
                     'detection_list_id': 2,
                     'local_comments': u'',
                     'ra': 4.754236999477372,
                     'followup_id': 1140386L,
                     'dec': 28.276703631398625,
                     'id': 1001901011281636100L,
                     'object_classification': 0L
                     },

                    {'ps1_designation': u'PS1-13dhc',
                     'name': u'3I3Pixd',
                     'detection_list_id': 2,
                     'local_comments': u'',
                     'ra': 1.3324973428505413,
                     'followup_id': 1202386L,
                     'dec': 32.98869220595689,
                     'id': 1000519791325919200L,
                     'object_classification': 0L
                     }
                ]

            Then run the ``angular_crossmatch_against_catalogue`` method to crossmatch against the catalogues and return results:

            .. code-block:: python

                # ANGULAR CONESEARCH ON CATALOGUE
                search_name = "ned_d spec sn"
                searchPara = self.settings["search algorithm"][search_name]
                matchedObjects = xmatcher.angular_crossmatch_against_catalogue(
                    objectList=transients,
                    searchPara=searchPara,
                    search_name=search_name
                )

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``angular_crossmatch_against_catalogue`` method')

        self.log.info("STARTING %s SEARCH" %
                      (search_name,))

        start_time = time.time()

        # DEFAULTS

        # print search_name, classificationType

        magnitudeLimitFilter = None
        upperMagnitudeLimit = False
        lowerMagnitudeLimit = False

        catalogueName = searchPara["database table"]

        if not "mag column" in searchPara:
            searchPara["mag column"] = None

        if brightnessFilter:
            if "mag column" in searchPara and searchPara["mag column"]:
                magnitudeLimitFilter = self.colMaps[
                    catalogueName][searchPara["mag column"] + "ColName"]
            theseSearchPara = searchPara[brightnessFilter]
        else:
            theseSearchPara = searchPara

        # EXTRACT PARAMETERS FROM ARGUMENTS & SETTINGS FILE
        if classificationType == "synonym":
            radius = self.settings["synonym radius arcsec"]
            matchedType = theseSearchPara["synonym"]
        elif classificationType == "association":
            radius = theseSearchPara["angular radius arcsec"]
            matchedType = theseSearchPara["association"]
        elif classificationType == "annotation":
            radius = theseSearchPara["angular radius arcsec"]
            matchedType = theseSearchPara["annotation"]

        if brightnessFilter == "faint":
            upperMagnitudeLimit = theseSearchPara["mag limit"]
        elif brightnessFilter == "bright":
            lowerMagnitudeLimit = theseSearchPara["mag limit"]
        elif brightnessFilter == "general":
            if "faint" in searchPara:
                lowerMagnitudeLimit = searchPara["faint"]["mag limit"]
            if "bright" in searchPara:
                upperMagnitudeLimit = searchPara["bright"]["mag limit"]

        # VARIABLES
        matchedObjects = []
        matchSubset = []

        transRAs = []
        transRAs[:] = [t['ra'] for t in objectList]
        transDecs = []
        transDecs[:] = [t['dec'] for t in objectList]

        if len(transRAs) == 0:
            return []

        cs = catalogue_conesearch(
            log=self.log,
            ra=transRAs,
            dec=transDecs,
            radiusArcsec=radius,
            colMaps=self.colMaps,
            tableName=catalogueName,
            dbConn=self.dbConn,
            nearestOnly=False,
            physicalSearch=physicalSearch,
            upperMagnitudeLimit=upperMagnitudeLimit,
            lowerMagnitudeLimit=lowerMagnitudeLimit,
            magnitudeLimitFilter=magnitudeLimitFilter
        )

        # catalogueMatches ARE ORDERED BY ANGULAR SEPARATION
        indices, catalogueMatches = cs.search()
        count = 1
        annotatedcatalogueMatches = []

        for i, xm in zip(indices, catalogueMatches):

            # CALCULATE PHYSICAL PARAMETERS ... IF WE CAN
            if "cmSepArcsec" in xm:
                xm["separationArcsec"] = xm["cmSepArcsec"]
                # CALCULATE SEPARATION IN ARCSEC

                calculator = separations(
                    log=self.log,
                    ra1=objectList[i]["ra"],
                    dec1=objectList[i]["dec"],
                    ra2=xm["ra"],
                    dec2=xm["dec"]
                )
                angularSeparation, north, east = calculator.get()

                xm["northSeparationArcsec"] = north
                xm["eastSeparationArcsec"] = east
                del xm["cmSepArcsec"]

            xm["association_type"] = matchedType
            xm["catalogue_view_name"] = catalogueName
            xm["transient_object_id"] = objectList[i]["id"]
            xm["catalogue_table_name"] = self.colMaps[
                catalogueName]["description"]
            xm["catalogue_table_id"] = self.colMaps[
                catalogueName]["table_id"]
            xm["catalogue_view_id"] = self.colMaps[
                catalogueName]["id"]
            if classificationType == "synonym":
                xm["classificationReliability"] = 1
            elif classificationType == "association":
                xm["classificationReliability"] = 2
            elif classificationType == "annotation":
                xm["classificationReliability"] = 3

            xm = self._annotate_crossmatch_with_value_added_parameters(
                crossmatchDict=xm,
                catalogueName=catalogueName,
                searchPara=theseSearchPara,
                search_name=search_name
            )
            annotatedcatalogueMatches.append(xm)

        catalogueMatches = annotatedcatalogueMatches

        # IF BRIGHT STAR SEARCH
        if brightnessFilter == "bright" and "star" in search_name:
            catalogueMatches = self._bright_star_match(
                matchedObjects=catalogueMatches,
                catalogueName=catalogueName,
                lowerMagnitudeLimit=lowerMagnitudeLimit,
                magnitudeLimitFilter=searchPara["mag column"]
            )

        if brightnessFilter == "general" and "galaxy" in search_name and "galaxy-like" not in search_name and "physical radius kpc" not in theseSearchPara:
            catalogueMatches = self._galaxy_association_cuts(
                matchedObjects=catalogueMatches,
                catalogueName=catalogueName,
                lowerMagnitudeLimit=lowerMagnitudeLimit,
                upperMagnitudeLimit=upperMagnitudeLimit,
                magnitudeLimitFilter=searchPara["mag column"]
            )

        if "match nearest source only" in theseSearchPara and theseSearchPara["match nearest source only"] == True and len(catalogueMatches):
            nearestMatches = []
            transList = []
            for c in catalogueMatches:
                if c["transient_object_id"] not in transList:
                    transList.append(c["transient_object_id"])
                    nearestMatches.append(c)
            catalogueMatches = nearestMatches

        self.log.debug(
            'completed the ``angular_crossmatch_against_catalogue`` method')

        self.log.debug("FINISHED %s SEARCH IN %0.5f s" %
                       (search_name, time.time() - start_time,))

        return catalogueMatches


Instruction:
How would you explain what the following Python 3 function does
def _annotate_crossmatch_with_value_added_parameters(
            self,
            crossmatchDict,
            catalogueName,
            searchPara,
            search_name):
        """*annotate each crossmatch with physical parameters such are distances etc*

        **Key Arguments:**
            - ``crossmatchDict`` -- the crossmatch dictionary
            - ``catalogueName`` -- the name of the catalogue the crossmatch results from
            - ``searchPara`` -- the search parameters for this individual search as lifted from the search algorithm in the sherlock settings file
            - ``search_name`` -- the name of the search as given in the sherlock settings file

        **Return:**
            - ``crossmatchDict`` -- the annotated crossmatch dictionary

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``_annotate_crossmatch_with_value_added_parameters`` method')

        redshift = None
        z = None
        scale = None
        distance = None
        distance_modulus = None
        major_axis_arcsec = None
        direct_distance = None
        direct_distance_scale = None
        direct_distance_modulus = None

        # IF THERE'S A REDSHIFT, CALCULATE PHYSICAL PARAMETERS
        if 'z' in crossmatchDict:
            # THE CATALOGUE HAS A REDSHIFT COLUMN
            redshift = crossmatchDict['z']
        elif 'photoZ' in crossmatchDict:
            redshift = crossmatchDict['photoZ']
        if redshift and redshift > 0.0:
            # CALCULATE DISTANCE MODULUS, ETC
            c = converter(log=self.log)
            dists = c.redshift_to_distance(
                z=redshift,
                WM=0.3,
                WV=0.7,
                H0=70.0
            )

            if dists:
                z = dists['z']
                scale = dists["da_scale"]
                distance = dists["dl_mpc"]
                distance_modulus = dists["dmod"]
        # ADD MAJOR AXIS VALUE
        if "or within semi major axis" in searchPara and searchPara["or within semi major axis"] == True and "semiMajor" in crossmatchDict and crossmatchDict["semiMajor"]:
            major_axis_arcsec = crossmatchDict[
                "semiMajor"] * self.colMaps[catalogueName]["semiMajorToArcsec"]

        if "semiMajor" in crossmatchDict:
            del crossmatchDict["semiMajor"]
        # ADD DISTANCE VALUES
        if "distance" in crossmatchDict and crossmatchDict["distance"]:
            direct_distance = crossmatchDict["distance"]
            direct_distance_scale = direct_distance / 206.264806
            direct_distance_modulus = 5 * \
                math.log10(direct_distance * 1e6) - 5
        # crossmatchDict['z'] = z
        crossmatchDict['scale'] = scale
        crossmatchDict['distance'] = distance
        crossmatchDict['distance_modulus'] = distance_modulus
        crossmatchDict['major_axis_arcsec'] = major_axis_arcsec
        crossmatchDict['direct_distance'] = direct_distance
        crossmatchDict['direct_distance_scale'] = direct_distance_scale
        crossmatchDict['direct_distance_modulus'] = direct_distance_modulus

        crossmatchDict['catalogue_object_type'] = self.colMaps[
            catalogueName]["object_type"]
        crossmatchDict["search_name"] = search_name
        crossmatchDict["raDeg"] = crossmatchDict["ra"]

        crossmatchDict["decDeg"] = crossmatchDict["dec"]
        del crossmatchDict["ra"]
        del crossmatchDict["dec"]
        crossmatchDict["original_search_radius_arcsec"] = searchPara[
            "angular radius arcsec"]

        physical_separation_kpc = None
        # CALCULATE MOST ACCURATE PHYSICAL SEPARATION
        if crossmatchDict["direct_distance_scale"]:
            physical_separation_kpc = crossmatchDict[
                "direct_distance_scale"] * crossmatchDict["separationArcsec"]
        elif crossmatchDict["scale"]:
            physical_separation_kpc = crossmatchDict[
                "scale"] * crossmatchDict["separationArcsec"]

        crossmatchDict["physical_separation_kpc"] = physical_separation_kpc

        self.log.debug(
            'completed the ``_annotate_crossmatch_with_value_added_parameters`` method')
        return crossmatchDict

Response: This method is used to annotate each crossmatch with physical parameters.


Instruction:
How would you implement a function in Python 3 that
performs a bright star match on the matched objects

Response:
def _bright_star_match(
            self,
            matchedObjects,
            catalogueName,
            magnitudeLimitFilter,
            lowerMagnitudeLimit):
        """*perform a bright star match on the crossmatch results if required by the catalogue search*

        **Key Arguments:**
            - ``matchedObjects`` -- the list of matched sources from the catalogue crossmatch
            - ``catalogueName`` -- the name of the catalogue the crossmatch results from
            - ``magnitudeLimitFilter`` -- the name of the column containing the magnitude to filter on
            - ``lowerMagnitudeLimit`` -- the lower magnitude limit to match bright stars against

        **Return:**
            - ``brightStarMatches`` -- the trimmed matched sources (bright stars associations only)

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug('starting the ``_bright_star_match`` method')

        import decimal
        decimal.getcontext().prec = 10

        # MATCH BRIGHT STAR ASSOCIATIONS
        brightStarMatches = []
        for row in matchedObjects:
            mag = decimal.Decimal(row[magnitudeLimitFilter])
            if mag and mag < lowerMagnitudeLimit:
                sep = decimal.Decimal(row["separationArcsec"])
                if sep < decimal.Decimal(decimal.Decimal(10)**(-decimal.Decimal(0.2) * mag + decimal.Decimal(3.7))) and sep < 20.:
                    brightStarMatches.append(row)

        self.log.debug('completed the ``_bright_star_match`` method')
        return brightStarMatches


Instruction:
Write a Python 3 script for
performing a bright star match on the matched objects and return a list of galaxies that match the given values.

Response:
def _galaxy_association_cuts(
            self,
            matchedObjects,
            catalogueName,
            magnitudeLimitFilter,
            upperMagnitudeLimit,
            lowerMagnitudeLimit):
        """*perform a bright star match on the crossmatch results if required by the catalogue search*

        **Key Arguments:**
            - ``matchedObjects`` -- the list of matched sources from the catalogue crossmatch
            - ``catalogueName`` -- the name of the catalogue the crossmatch results from
            - ``magnitudeLimitFilter`` -- the name of the column containing the magnitude to filter on
            - ``lowerMagnitudeLimit`` -- the lower magnitude limit to match general galaxies against
            - ``upperMagnitudeLimit`` -- the upper magnitude limit to match general galaxies against

        **Return:**
            - ``galaxyMatches`` -- the trimmed matched sources (associated galaxies only)

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug('starting the ``_galaxy_association_cuts`` method')

        import decimal
        decimal.getcontext().prec = 10

        # MATCH BRIGHT STAR ASSOCIATIONS
        galaxyMatches = []
        for row in matchedObjects:
            if not magnitudeLimitFilter or row[magnitudeLimitFilter] == None:
                galaxyMatches.append(row)
            else:
                mag = decimal.Decimal(row[magnitudeLimitFilter])
                if mag and mag < lowerMagnitudeLimit and mag > upperMagnitudeLimit:
                    sep = decimal.Decimal(row["separationArcsec"])
                    if sep < decimal.Decimal(decimal.Decimal(10)**(decimal.Decimal((decimal.Decimal(25.) - mag) / decimal.Decimal(6.)))):
                        galaxyMatches.append(row)

        self.log.debug('completed the ``_galaxy_association_cuts`` method')
        return galaxyMatches


Instruction:
Write a Python 3 script for
performing an physical separation crossmatch against a given catalogue

Response:
def physical_separation_crossmatch_against_catalogue(
        self,
        objectList,
        searchPara,
        search_name,
        brightnessFilter=False,
        classificationType=False
    ):
        """*perform an physical separation crossmatch against a given catalogue in the database*

        This search is basically the same as the angular separation search except extra filtering is done to exclude sources outside the physical search radius (matched sources require distance info to calulate physical separations)

        **Key Arguments:**
            - ``objectList`` -- transients to be crossmatched
            - ``searchPara`` -- parameters of the search (from settings file)
            - ``search_name`` -- the name of the search
            - ``brightnessFilter`` -- is this search to be constrained by magnitude of the catalogue sources? Default *False*. [bright|faint|general]
            - ``classificationType`` -- synonym, association or annotation. Default *False*

        **Return:**
            - matchedObjects -- any sources matched against the object

        To run a physical separation crossmatch, run in a similar way to the angular separation crossmatch:

        **Usage:**

            .. code-block:: python

                search_name = "ned spec sn"
                searchPara = self.settings["search algorithm"][search_name]
                matchedObjects = xmatcher.physical_separation_crossmatch_against_catalogue(
                    objectList=transients,
                    searchPara=searchPara,
                    search_name=search_name
                )

        .. todo ::

            - update key arguments values and definitions with defaults
            - update return values and definitions
            - update usage examples and text
            - update docstring text
            - check sublime snippet exists
            - clip any useful text to docs mindmap
            - regenerate the docs and check redendering of this docstring
        """
        self.log.debug(
            'starting the ``physical_separation_crossmatch_against_catalogue`` method')

        start_time = time.time()

        bf = brightnessFilter
        # SETUP PARAMETERS
        tableName = searchPara["database table"]
        if bf:
            angularRadius = searchPara[bf]["angular radius arcsec"]
            physicalRadius = searchPara[bf]["physical radius kpc"]
            matchedType = searchPara[bf][classificationType]
            if "match nearest source only" in searchPara[bf]:
                nearestOnly = searchPara[bf]["match nearest source only"]
            else:
                nearestOnly = False
        else:
            angularRadius = searchPara["angular radius arcsec"]
            physicalRadius = searchPara["physical radius kpc"]
            matchedType = searchPara[classificationType]
            if "match nearest source only" in searchPara:
                nearestOnly = searchPara["match nearest source only"]
            else:
                nearestOnly = False

        matchedObjects = []
        matchSubset = []

        # RETURN ALL ANGULAR MATCHES BEFORE RETURNING NEAREST PHYSICAL SEARCH
        tmpSearchPara = dict(searchPara)
        tmpSearchPara["match nearest source only"] = False

        # ANGULAR CONESEARCH ON CATALOGUE - RETURN ALL MATCHES
        catalogueMatches = self.angular_crossmatch_against_catalogue(
            objectList=objectList,
            searchPara=tmpSearchPara,
            search_name=search_name,
            physicalSearch=True,
            brightnessFilter=brightnessFilter,
            classificationType=classificationType
        )

        # OK - WE HAVE SOME ANGULAR SEPARATION MATCHES. NOW SEARCH THROUGH THESE FOR MATCHES WITH
        # A PHYSICAL SEPARATION WITHIN THE PHYSICAL RADIUS.
        if catalogueMatches:
            for row in catalogueMatches:
                thisMatch = False
                physical_separation_kpc = row["physical_separation_kpc"]
                newsearch_name = search_name

                # FIRST CHECK FOR MAJOR AXIS MEASUREMENT
                # BYPASS NED FAULTY AXES MEASUREMENTS:
                # https://gist.github.com/search?utf8=%E2%9C%93&q=user%3Athespacedoctor+ned
                if row["major_axis_arcsec"] and ("ned" not in search_name or (row["unkMag"] and row["unkMag"] < 20.)):
                    if row["separationArcsec"] < row["major_axis_arcsec"] * self.settings["galaxy radius stetch factor"]:
                        thisMatch = True
                        newsearch_name = newsearch_name + \
                            " (within %s * major axis)" % (
                                self.settings["galaxy radius stetch factor"],)
                        newAngularSep = row[
                            "major_axis_arcsec"] * self.settings["galaxy radius stetch factor"]
                    else:
                        continue
                # NOW CHECK FOR A DIRECT DISTANCE MEASUREMENT
                elif row["direct_distance_scale"] and physical_separation_kpc < physicalRadius:
                    if row["separationArcsec"] > 300.:
                        continue
                    thisMatch = True
                    newsearch_name = newsearch_name + " (direct distance)"
                    newAngularSep = physicalRadius / \
                        row["direct_distance_scale"]
                # NEW CHECK FOR A REDSHIFT DISTANCE
                elif row["scale"] and physical_separation_kpc < physicalRadius:
                    thisMatch = True
                    newsearch_name = newsearch_name + " (redshift distance)"
                    newAngularSep = physicalRadius / row["scale"]

                if thisMatch == True:
                    row["physical_separation_kpc"] = physical_separation_kpc
                    row["original_search_radius_arcsec"] = newAngularSep
                    if physical_separation_kpc:
                        self.log.debug(
                            "\t\tPhysical separation = %.2f kpc" % (physical_separation_kpc,))
                    row["search_name"] = newsearch_name
                    matchSubset.append(row)

        if matchSubset:

            from operator import itemgetter
            matchSubset = sorted(matchSubset, key=itemgetter(
                'physical_separation_kpc'), reverse=False)

            if nearestOnly == True:
                theseMatches = matchSubset[0]
            else:
                theseMatches = matchSubset

            matchedObjects = matchSubset
        self.log.debug(
            'completed the ``physical_separation_crossmatch_against_catalogue`` method')

        self.log.debug("FINISHED %s SEARCH IN %0.5f s" %
                       (search_name, time.time() - start_time,))

        return matchedObjects


Instruction:
Can you implement a function in Python 3 that
gets the host s ip number.

Response:
def get_own_ip():
    """Get the host's ip number.
    """
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        sock.connect(("8.8.8.8", 80))
    except socket.gaierror:
        ip_ = "127.0.0.1"
    else:
        ip_ = sock.getsockname()[0]
    finally:
        sock.close()
    return ip_


Instruction:
Write a Python 3 script for
sending a message to the related user.

Response:
def send(self, msg):
        """Send the given message.
        """
        with self._pub_lock:
            self.publish.send_string(msg)
        return self


Instruction:
Explain what the following Python 3 code does
def heartbeat(self, min_interval=0):
        if not self._heartbeat:
            self._heartbeat = _PublisherHeartbeat(self)
        self._heartbeat(min_interval)

Response: Send a heartbeat... but only if min_interval * seconds have passed since last beat.


Instruction:
Here you have a function in Python 3, explain what it does
def light_curve(self, t, texp=0.0, tol=1e-8, maxdepth=4):
        t = np.atleast_1d(t)
        return CythonSolver().simple_light_curve(self._get_params(),
                                                 t, texp, tol, maxdepth)

Response: Simple light curve function.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def light_curve_gradient(self, t, texp=0.0, tol=1e-8, maxdepth=4):
        t = np.atleast_1d(t)
        f, df = CythonSolver().simple_gradient(self._get_params(),
                                               t, texp, tol, maxdepth)

        return f, df

Response: Calculates the gradient of the light curve at a list of times using the current object s model.


Instruction:
Explain what the following Python 3 code does
def configure_stream_logger(logger='', level=None, formatter='%(levelname)-8s %(message)s'):
	level = level or logging.WARNING
	if isinstance(level, str):
		level = getattr(logging, level, None)
		if level is None:
			raise ValueError('invalid log level: ' + level)
	root_logger = logging.getLogger('')
	for handler in root_logger.handlers:
		root_logger.removeHandler(handler)

	logging.getLogger(logger).setLevel(logging.DEBUG)
	console_log_handler = logging.StreamHandler()
	console_log_handler.setLevel(level)
	if isinstance(formatter, str):
		formatter = logging.Formatter(formatter)
	elif not isinstance(formatter, logging.Formatter):
		raise TypeError('formatter must be an instance of logging.Formatter')
	console_log_handler.setFormatter(formatter)
	logging.getLogger(logger).addHandler(console_log_handler)
	logging.captureWarnings(True)
	return console_log_handler

Response: Configure the default stream handler for logging messages to the console.


Instruction:
How would you implement a function in Python 3 that
downloads a file from a url and save it to disk.

Response:
def download(url, filename=None):
	"""
	Download a file from a url and save it to disk.

	:param str url: The URL to fetch the file from.
	:param str filename: The destination file to write the data to.
	"""
	# requirements os, shutil, urllib.parse, urllib.request
	if not filename:
		url_parts = urllib.parse.urlparse(url)
		filename = os.path.basename(url_parts.path)
	url_h = urllib.request.urlopen(url)
	with open(filename, 'wb') as file_h:
		shutil.copyfileobj(url_h, file_h)
	url_h.close()
	return


Instruction:
How would you implement a function in Python 3 that
takes a number of bytes and convert it to a human readable number.

Response:
def format_bytes_size(val):
	"""
	Take a number of bytes and convert it to a human readable number.

	:param int val: The number of bytes to format.
	:return: The size in a human readable format.
	:rtype: str
	"""
	if not val:
		return '0 bytes'
	for sz_name in ['bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB']:
		if val < 1024.0:
			return "{0:.2f} {1}".format(val, sz_name)
		val /= 1024.0
	raise OverflowError()


Instruction:
Can you generate the documentation for the following Python 3 function
def grep(expression, file, flags=0, invert=False):
	# requirements = re
	if isinstance(file, str):
		file = open(file)
	lines = []
	for line in file:
		if bool(re.search(expression, line, flags=flags)) ^ invert:
			lines.append(line)
	return lines

Response: Search a file and return a list of all the lines that match a regular expression.


Instruction:
Create a Python 3 function to
open a URI in a platform intelligent way.

Response:
def open_uri(uri):
	"""
	Open a URI in a platform intelligent way. On Windows this will use
	'cmd.exe /c start' and on Linux this will use gvfs-open or xdg-open
	depending on which is available. If no suitable application can be
	found to open the URI, a RuntimeError will be raised.

	.. versionadded:: 1.3.0

	:param str uri: The URI to open.
	"""
	close_fds = True
	startupinfo = None
	proc_args = []
	if sys.platform.startswith('win'):
		proc_args.append(which('cmd.exe'))
		proc_args.append('/c')
		proc_args.append('start')
		uri = uri.replace('&', '^&')
		close_fds = False
		startupinfo = subprocess.STARTUPINFO()
		startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
		startupinfo.wShowWindow = subprocess.SW_HIDE
	elif which('gvfs-open'):
		proc_args.append(which('gvfs-open'))
	elif which('xdg-open'):
		proc_args.append(which('xdg-open'))
	else:
		raise RuntimeError('could not find suitable application to open uri')
	proc_args.append(uri)
	proc_h = subprocess.Popen(proc_args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=close_fds, startupinfo=startupinfo)
	return proc_h.wait() == 0


Instruction:
Can you write a function in Python 3 where it
converts a string from snake_case to CamelCase.

Response:
def parse_case_snake_to_camel(snake, upper_first=True):
	"""
	Convert a string from snake_case to CamelCase.

	:param str snake: The snake_case string to convert.
	:param bool upper_first: Whether or not to capitalize the first
		character of the string.
	:return: The CamelCase version of string.
	:rtype: str
	"""
	snake = snake.split('_')
	first_part = snake[0]
	if upper_first:
		first_part = first_part.title()
	return first_part + ''.join(word.title() for word in snake[1:])


Instruction:
Here you have a function in Python 3, explain what it does
def parse_server(server, default_port):
	server = server.rsplit(':', 1)
	host = server[0]
	if host.startswith('[') and host.endswith(']'):
		host = host[1:-1]
	if len(server) == 1:
		return (host, default_port)
	port = server[1]
	if not port:
		port = default_port
	else:
		port = int(port)
	return (host, port)

Response: parse_server - Convert a server string to a tuple suitable for passing to connect for the connection.


Instruction:
Given the following Python 3 function, write the documentation
def parse_timespan(timedef):
	if isinstance(timedef, int):
		return timedef
	converter_order = ('w', 'd', 'h', 'm', 's')
	converters = {
		'w': 604800,
		'd': 86400,
		'h': 3600,
		'm': 60,
		's': 1
	}
	timedef = timedef.lower()
	if timedef.isdigit():
		return int(timedef)
	elif len(timedef) == 0:
		return 0
	seconds = -1
	for spec in converter_order:
		timedef = timedef.split(spec)
		if len(timedef) == 1:
			timedef = timedef[0]
			continue
		elif len(timedef) > 2 or not timedef[0].isdigit():
			seconds = -1
			break
		adjustment = converters[spec]
		seconds = max(seconds, 0)
		seconds += (int(timedef[0]) * adjustment)
		timedef = timedef[1]
		if not len(timedef):
			break
	if seconds < 0:
		raise ValueError('invalid time format')
	return seconds

Response: parse_timespan - Convert a string timespan definition to seconds.


Instruction:
Create a Python 3 function for
parsing a string into a slug format suitable for use in URLs and other characters restricted applications.

Response:
def parse_to_slug(words, maxlen=24):
	"""
	Parse a string into a slug format suitable for use in URLs and other
	character restricted applications. Only utf-8 strings are supported at this
	time.

	:param str words: The words to parse.
	:param int maxlen: The maximum length of the slug.
	:return: The parsed words as a slug.
	:rtype: str
	"""
	slug = ''
	maxlen = min(maxlen, len(words))
	for c in words:
		if len(slug) == maxlen:
			break
		c = ord(c)
		if c == 0x27:
			continue
		elif c >= 0x30 and c <= 0x39:
			slug += chr(c)
		elif c >= 0x41 and c <= 0x5a:
			slug += chr(c + 0x20)
		elif c >= 0x61 and c <= 0x7a:
			slug += chr(c)
		elif len(slug) and slug[-1] != '-':
			slug += '-'
	if len(slug) and slug[-1] == '-':
		slug = slug[:-1]
	return slug