Instruction:
Write a Python 3 script for
returning the EPSG ID as int if it exists.

Response:
def srid(self):
        """Returns the EPSG ID as int if it exists."""
        epsg_id = (self.GetAuthorityCode('PROJCS') or
                   self.GetAuthorityCode('GEOGCS'))
        try:
            return int(epsg_id)
        except TypeError:
            return


Instruction:
Can you generate a brief explanation for the following Python 3 code
def main():
    args = get_args()
    ret_code = args.target(args)
    _logger.debug('Exiting with code %d', ret_code)
    sys.exit(ret_code)

Response: Entry point for the CLI.


Instruction:
Write a Python 3 script to
update the content of a single file.

Response:
def update_file(url, filename):
    """Update the content of a single file."""
    resp = urlopen(url)
    if resp.code != 200:
        raise Exception('GET {} failed.'.format(url))
    with open(_get_package_path(filename), 'w') as fp:
        for l in resp:
            if not l.startswith(b'#'):
                fp.write(l.decode('utf8'))
    print('Updated {}'.format(filename))


Instruction:
Can you tell what is the following Python 3 function doing
def available_drivers():
    drivers = {}
    for i in range(gdal.GetDriverCount()):
        d = gdal.GetDriver(i)
        drivers[d.ShortName] = d.GetMetadata()
    return drivers

Response:
Returns a dictionary of GDAL Driver metadata keyed by the
    ShortName attribute.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def driver_for_path(path, drivers=None):
    ext = (os.path.splitext(path)[1][1:] or path).lower()
    drivers = drivers or ImageDriver.registry if ext else {}
    for name, meta in drivers.items():
        if ext == meta.get('DMD_EXTENSION', '').lower():
            return ImageDriver(name)
    return None

Response: Returns the gdal. Driver for a path.


Instruction:
Given the following Python 3 function, write the documentation
def geom_to_array(geom, size, affine):
    driver = ImageDriver('MEM')
    rast = driver.raster(driver.ShortName, size)
    rast.affine = affine
    rast.sref = geom.GetSpatialReference()
    with MemoryLayer.from_records([(1, geom)]) as ml:
        status = gdal.RasterizeLayer(rast.ds, (1,), ml.layer, burn_values=(1,))
    arr = rast.array()
    rast.close()
    return arr

Response: Converts an OGR polygon to a 2D NumPy array.


Instruction:
Here you have a function in Python 3, explain what it does
def rasterize(layer, rast):
    driver = ImageDriver('MEM')
    r2 = driver.raster(driver.ShortName, rast.size)
    r2.affine = rast.affine
    sref = rast.sref
    if not sref.srid:
        sref = SpatialReference(4326)
    r2.sref = sref
    ml = MemoryLayer(sref, layer.GetGeomType())
    ml.load(layer)
    status = gdal.RasterizeLayer(
        r2.ds, (1,), ml.layer, options=['ATTRIBUTE=%s' % ml.id])
    ml.close()
    return r2

Response: Returns a Raster from a Layer features.


Instruction:
Here you have a function in Python 3, explain what it does
def open(path, mode=gdalconst.GA_ReadOnly):
    path = getattr(path, 'name', path)
    try:
        return Raster(vsiprefix(path), mode)
    except AttributeError:
        try:
            imgdata = path.read()
        except AttributeError:
            raise TypeError('Not a file-like object providing read()')
        else:
            imgio = MemFileIO(delete=False)
            gdal.FileFromMemBuffer(imgio.name, imgdata)
            return Raster(imgio, mode)
    raise ValueError('Failed to open raster from "%r"' % path)

Response: Returns a Raster instance from a local or remote path as str or file - like object.


Instruction:
Make a summary of the following Python 3 code
def frombytes(data, size, bandtype=gdal.GDT_Byte):
    r = ImageDriver('MEM').raster('', size, bandtype)
    r.frombytes(data)
    return r

Response: Returns an in - memory raster initialized from a byte buffer.


Instruction:
Can you implement a function in Python 3 that
converts image pixel coordinates to georeferenced x / y

Response:
def project(self, coords):
        """Convert image pixel/line coordinates to georeferenced x/y, return a
        generator of two-tuples.

        Arguments:
        coords -- input coordinates as iterable containing two-tuples/lists
        such as ((0, 0), (10, 10))
        """
        geotransform = self.tuple
        for x, y in coords:
            geo_x = geotransform[0] + geotransform[1] * x + geotransform[2] * y
            geo_y = geotransform[3] + geotransform[4] * x + geotransform[5] * y
            # Move the coordinate to the center of the pixel.
            geo_x += geotransform[1] / 2.0
            geo_y += geotransform[5] / 2.0
            yield geo_x, geo_y


Instruction:
Can you generate the documentation for the following Python 3 function
def transform(self, coords):
        # Use local vars for better performance here.
        origin_x, origin_y = self.origin
        sx, sy = self.scale
        return [(int(math.floor((x - origin_x) / sx)),
                 int(math.floor((y - origin_y) / sy)))
                for x, y in coords]

Response: Transform from projection coordinates ( Xp Yp ) space to pixel / line raster space based on the provided geotransformation.


Instruction:
Implement a function in Python 3 to
copy a source Raster instance to a destination Raster instance.

Response:
def copy(self, source, dest):
        """Returns a copied Raster instance.

        Arguments:
        source -- the source Raster instance or filepath as str
        dest -- destination filepath as str
        """
        if not self.copyable:
            raise IOError('Driver does not support raster copying')
        if not isinstance(source, Raster):
            source = Raster(source)
            should_close = True
        else:
            should_close = False
        if source.name == dest:
            raise ValueError(
                'Input and output are the same location: %s' % source.name)
        settings = driverdict_tolist(self.settings)
        ds = self.CreateCopy(dest, source.ds, self.strictmode,
                             options=settings)
        if should_close:
            source.close()
        return Raster(ds)


Instruction:
Create a Python 3 function for
calling Driver. Create with optionally provided creation options as dict falls back to driver specific defaults.

Response:
def Create(self, *args, **kwargs):
        """Calls Driver.Create() with optionally provided creation options as
        dict, or falls back to driver specific defaults.
        """
        if not self.writable:
            raise IOError('Driver does not support raster creation')
        options = kwargs.pop('options', {})
        kwargs['options'] = driverdict_tolist(options or self.settings)
        return self._driver.Create(*args, **kwargs)


Instruction:
Write a Python 3 script to
return a dict of driver specific raster creation options.

Response:
def options(self):
        """Returns a dict of driver specific raster creation options.

        See GDAL format docs at http://www.gdal.org/formats_list.html
        """
        if self._options is None:
            try:
                elem = ET.fromstring(
                    self.info.get('DMD_CREATIONOPTIONLIST', ''))
            except ET.ParseError:
                elem = []
            opts = {}
            for child in elem:
                choices = [val.text for val in child]
                if choices:
                    child.attrib.update(choices=choices)
                opts[child.attrib.pop('name')] = child.attrib
            self._options = opts
        return self._options


Instruction:
Write a Python 3 script for
creating a new raster file.

Response:
def raster(self, path, size, bandtype=gdal.GDT_Byte):
        """Returns a new Raster instance.

        gdal.Driver.Create() does not support all formats.

        Arguments:
        path -- file object or path as str
        size -- two or three-tuple of (xsize, ysize, bandcount)
        bandtype -- GDAL pixel data type
        """
        path = getattr(path, 'name', path)
        try:
            is_multiband = len(size) > 2
            nx, ny, nbands = size if is_multiband else size + (1,)
        except (TypeError, ValueError) as exc:
            exc.args = ('Size must be 2 or 3-item sequence',)
            raise
        if nx < 1 or ny < 1:
            raise ValueError('Invalid raster size %s' % (size,))
        # Do not write to a non-empty file.
        if not self._is_empty(path):
            raise IOError('%s already exists, open with Raster()' % path)
        ds = self.Create(path, nx, ny, nbands, bandtype)
        if not ds:
            raise ValueError(
                'Could not create %s using %s' % (path, str(self)))
        return Raster(ds)


Instruction:
Implement a function in Python 3 to
set the affine transformation.

Response:
def SetGeoTransform(self, affine):
        """Sets the affine transformation.

        Intercepts the gdal.Dataset call to ensure use as a property setter.

        Arguments:
        affine -- AffineTransform or six-tuple of geotransformation values
        """
        if isinstance(affine, collections.Sequence):
            affine = AffineTransform(*affine)
        self._affine = affine
        self.ds.SetGeoTransform(affine)


Instruction:
Explain what the following Python 3 code does
def array(self, envelope=()):
        args = ()
        if envelope:
            args = self.get_offset(envelope)
        return self.ds.ReadAsArray(*args)

Response: Returns an NDArray optionally subset by spatial envelope.


Instruction:
Write a Python 3 function that can
return the minimum bounding rectangle as a tuple of min X max Y min X max Y.

Response:
def envelope(self):
        """Returns the minimum bounding rectangle as a tuple of min X, min Y,
        max X, max Y.
        """
        if self._envelope is None:
            origin = self.affine.origin
            ur_x = origin[0] + self.ds.RasterXSize * self.affine.scale[0]
            ll_y = origin[1] + self.ds.RasterYSize * self.affine.scale[1]
            self._envelope = Envelope(origin[0], ll_y, ur_x, origin[1])
        return self._envelope


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_offset(self, envelope):
        if isinstance(envelope, collections.Sequence):
            envelope = Envelope(envelope)
        if not (self.envelope.contains(envelope) or
                self.envelope.intersects(envelope)):
            raise ValueError('Envelope does not intersect with this extent')
        coords = self.affine.transform((envelope.ul, envelope.lr))
        nxy = [(min(dest, size) - origin) or 1
               for size, origin, dest in zip(self.size, *coords)]
        return coords[0] + tuple(nxy)

Response: Returns a 4 - tuple pixel window offset tuple or Envelope.


Instruction:
Write a Python 3 function that can
return the underlying ImageDriver instance.

Response:
def driver(self):
        """Returns the underlying ImageDriver instance."""
        if self._driver is None:
            self._driver = ImageDriver(self.ds.GetDriver())
        return self._driver


Instruction:
Implement a Python 3 function for
deriving new Raster instances.

Response:
def new(self, size=(), affine=None):
        """Derive new Raster instances.

        Keyword args:
        size -- tuple of image size (width, height)
        affine -- AffineTransform or six-tuple of geotransformation values
        """
        size = size or self.size + (len(self),)
        band = self.ds.GetRasterBand(1)
        driver = ImageDriver('MEM')
        rcopy = driver.raster(driver.ShortName, size, band.DataType)
        rcopy.sref = self.GetProjection()
        rcopy.affine = affine or tuple(self.affine)
        colors = band.GetColorTable()
        for outband in rcopy:
            if self.nodata is not None:
                outband.SetNoDataValue(self.nodata)
            if colors:
                outband.SetColorTable(colors)
        return rcopy


Instruction:
Write a Python 3 function that can
return a MaskedArray using nodata values.

Response:
def masked_array(self, geometry=None):
        """Returns a MaskedArray using nodata values.

        Keyword args:
        geometry -- any geometry, envelope, or coordinate extent tuple
        """
        if geometry is None:
            return self._masked_array()
        geom = transform(geometry, self.sref)
        env = Envelope.from_geom(geom).intersect(self.envelope)
        arr = self._masked_array(env)
        if geom.GetGeometryType() != ogr.wkbPoint:
            dims = self.get_offset(env)[2:]
            affine = AffineTransform(*tuple(self.affine))
            affine.origin = env.ul
            mask = ~np.ma.make_mask(geom_to_array(geom, dims, affine))
            arr.mask = arr.mask | mask
        return arr


Instruction:
Make a summary of the following Python 3 code
def nodata(self):
        if self._nodata is None:
            self._nodata = self[0].GetNoDataValue()
        return self._nodata

Response:
Returns read only property for band nodata value assuming single
        band rasters for now.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def ReadRaster(self, *args, **kwargs):
        args = args or (0, 0, self.ds.RasterXSize, self.ds.RasterYSize)
        return self.ds.ReadRaster(*args, **kwargs)

Response: Reads a raster from the dataset.


Instruction:
Given the following Python 3 function, write the documentation
def resample(self, size, interpolation=gdalconst.GRA_NearestNeighbour):
        # Find the scaling factor for pixel size.
        factors = (size[0] / float(self.RasterXSize),
                   size[1] / float(self.RasterYSize))
        affine = AffineTransform(*tuple(self.affine))
        affine.scale = (affine.scale[0] / factors[0],
                        affine.scale[1] / factors[1])
        dest = self.new(size, affine)
        # Uses self and dest projection when set to None
        gdal.ReprojectImage(self.ds, dest.ds, None, None, interpolation)
        return dest

Response: Resample the image to the specified size.


Instruction:
Can you write a function in Python 3 where it
saves this instance to the path and format provided.

Response:
def save(self, to, driver=None):
        """Save this instance to the path and format provided.

        Arguments:
        to -- output path as str, file, or MemFileIO instance
        Keyword args:
        driver -- GDAL driver name as string or ImageDriver
        """
        path = getattr(to, 'name', to)
        if not driver and hasattr(path, 'encode'):
            driver = driver_for_path(path, self.driver.filter_copyable())
        elif hasattr(driver, 'encode'):
            driver = ImageDriver(driver)
        if driver is None or not driver.copyable:
            raise ValueError('Copy supporting driver not found for %s' % path)
        driver.copy(self, path).close()


Instruction:
Here you have a function in Python 3, explain what it does
def SetProjection(self, sref):
        if not hasattr(sref, 'ExportToWkt'):
            sref = SpatialReference(sref)
        self._sref = sref
        self.ds.SetProjection(sref.ExportToWkt())

Response: Sets the spatial reference to use as a property setter.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def shape(self):
        shp = (self.ds.RasterYSize, self.ds.RasterXSize, self.ds.RasterCount)
        return shp[:2] if shp[2] <= 1 else shp

Response: Returns a tuple of row column band count if multidimensional.


Instruction:
Can you tell what is the following Python 3 function doing
def warp(self, to_sref, dest=None, interpolation=gdalconst.GRA_NearestNeighbour):
        if not hasattr(to_sref, 'ExportToWkt'):
            to_sref = SpatialReference(to_sref)
        dest_wkt = to_sref.ExportToWkt()
        dtype = self[0].DataType
        err_thresh = 0.125
        # Determine new values for destination raster dimensions and
        # geotransform.
        vrt = gdal.AutoCreateWarpedVRT(self.ds, None, dest_wkt,
                                       interpolation, err_thresh)
        if vrt is None:
            raise ValueError('Could not warp %s to %s' % (self, dest_wkt))
        warpsize = (vrt.RasterXSize, vrt.RasterYSize, len(self))
        warptrans = vrt.GetGeoTransform()
        vrt = None
        if dest is None:
            imgio = MemFileIO()
            rwarp = self.driver.raster(imgio, warpsize, dtype)
            imgio.close()
        else:
            rwarp = self.driver.raster(dest, warpsize, dtype)
        rwarp.SetGeoTransform(warptrans)
        rwarp.SetProjection(to_sref)
        if self.nodata is not None:
            for band in rwarp:
                band.SetNoDataValue(self.nodata)
                band = None
        # Uses self and rwarp projection when set to None
        gdal.ReprojectImage(self.ds, rwarp.ds, None, None, interpolation)
        return rwarp

Response: Warp the image to a new location.


Instruction:
Write a Python 3 script to
calculate the ideal conversion ratio for the given alphabet.

Response:
def calc_chunklen(alph_len):
    '''
    computes the ideal conversion ratio for the given alphabet.
    A ratio is considered ideal when the number of bits in one output
    encoding chunk that don't add up to one input encoding chunk is minimal.
    '''
    binlen, enclen = min([
                          (i, i*8 / math.log(alph_len, 2))
                          for i in range(1, 7)
                         ], key=lambda k: k[1] % 1)

    return binlen, int(enclen)


Instruction:
Here you have a function in Python 3, explain what it does
def lookup_alphabet(charset):
    '''
    retrieves a named charset or treats the input as a custom alphabet and use that
    '''
    if charset in PRESETS:
        return PRESETS[charset]
    if len(charset) < 16:
        _logger.warning('very small alphabet in use, possibly a failed lookup?')
    return charset

Response: Returns a named alphabet or treats the input as a custom alphabet and use that alphabet if it is not already there.


Instruction:
Here you have a function in Python 3, explain what it does
def _encode_chunk(self, data, index):
        '''
        gets a chunk from the input data, converts it to a number and
        encodes that number
        '''
        chunk = self._get_chunk(data, index)
        return self._encode_long(self._chunk_to_long(chunk))

Response: Encodes a single chunk from the input data converts it to a number and returns that number.


Instruction:
Can you generate the documentation for the following Python 3 function
def _encode_long(self, val):
        '''
        encodes an integer of 8*self.chunklen[0] bits using the specified
        alphabet
        '''
        return ''.join([
                self.alphabet[(val//len(self.alphabet)**i) % len(self.alphabet)]
                for i in reversed(range(self.chunklen[1]))
            ])

Response:
Encodes an integer of 8 * self. chunklen [ 0 ] bits using the specified
 clf alphabet


Instruction:
Create a Python 3 function for
parsing a chunk of bytes to integer using big - endian representation

Response:
def _chunk_to_long(self, chunk):
        '''
        parses a chunk of bytes to integer using big-endian representation
        '''
        return sum([
                256**(self.chunklen[0]-1-i) * ord_byte(chunk[i])
                for i in range(self.chunklen[0])
            ])


Instruction:
Given the following Python 3 function, write the documentation
def _get_chunk(self, data, index):
        '''
        partition the data into chunks and retrieve the chunk at the given index
        '''
        return data[index*self.chunklen[0]:(index+1)*self.chunklen[0]]

Response: partition the data into chunks and retrieve the chunk at the given index


Instruction:
Here you have a function in Python 3, explain what it does
def memoize(func):
    cache = {}

    @wraps(func)
    def inner(filename):
        if filename not in cache:
            cache[filename] = func(filename)
        return cache[filename]
    return inner

Response: Cache result of function call.


Instruction:
Create a Python 3 function for
getting a list of patterns from a file and make a regular expression.

Response:
def _regexp(filename):
    """Get a list of patterns from a file and make a regular expression."""
    lines = _get_resource_content(filename).decode('utf-8').splitlines()
    return re.compile('|'.join(lines))


Instruction:
Create a Python 3 function to
normalize the given date format.

Response:
def normalize_date_format(date):
    '''
    Dates can be defined in many ways, but zipline use
    aware datetime objects only. Plus, the software work
    with utc timezone so we convert it.
    '''
    if isinstance(date, int):
        # This is probably epoch time
        date = time.strftime('%Y-%m-%d %H:%M:%S',
                             time.localtime(date))

    # assert isinstance(date, str) or isinstance(date, unicode)
    if isinstance(date, str) or isinstance(date, unicode):
        date = dateutil.parser.parse(date)
    if not date.tzinfo:
        local_tz = pytz.timezone(_detect_timezone())
        local_dt = local_tz.localize(date, is_dst=None)
        # TODO I'm not sure why and when I need to add a date to make it right
        date = local_dt.astimezone(pytz.utc) + pd.datetools.day

    return date


Instruction:
Write a Python 3 script to
detect timezone as set by the system

Response:
def _detect_timezone():
    '''
    Get timezone as set by the system
    '''
    default_timezone = 'America/New_York'
    locale_code = locale.getdefaultlocale()
    return default_timezone if not locale_code[0] else \
        str(pytz.country_timezones[locale_code[0][-2:]][0])


Instruction:
Can you create a Python 3 function that
returns the full url for the given resource.

Response:
def api_url(full_version, resource):
    '''
    >>> # Harmonize api endpoints
    >>> # __version__ should be like major.minor.fix
    >>> from my_app import __version__
    >>> api_url(__version__, '/some/endpoint')
    /v0/some/endpoint
    '''
    return '/v{}/{}'.format(dna.utils.Version(full_version).major, resource)


Instruction:
Implement a function in Python 3 to
wrap api endpoints with more details

Response:
def api_doc(full_version, resource, method='GET', **kwargs):
    '''
    >>> # Wrap api endpoints with more details
    >>> api_doc('/resource', secure=True, key='value')
    GET /resource?secure=true&key=value
    '''
    doc = '{} {}'.format(method, api_url(full_version, resource))
    params = '&'.join(['{}={}'.format(k, v) for k, v in kwargs.iteritems()])
    if params:
        doc = '?'.join([doc, params])
    return doc


Instruction:
Create a Python 3 function to
return the model properties as a dict

Response:
def to_dict(self):
        """
        Returns the model properties as a dict
        """
        result = {}

        for attr, _ in iteritems(self.swagger_types):
            value = getattr(self, attr)
            if isinstance(value, list):
                result[attr] = list(map(
                    lambda x: x.to_dict() if hasattr(x, "to_dict") else x,
                    value
                ))
            elif hasattr(value, "to_dict"):
                result[attr] = value.to_dict()
            else:
                result[attr] = value

        return result


Instruction:
Here you have a function in Python 3, explain what it does
def activate_pdb_hook():
    ''' Catch exceptions with a prompt for post-mortem analyzis'''
    def debug_exception(type_exception, value, tb):
        import pdb
        pdb.post_mortem(tb)

    import sys
    sys.excepthook = debug_exception

Response: Activate the exception hook in sys. excepthook.


Instruction:
Explain what the following Python 3 code does
def listcoins(self):
        '''
        Use this function to list all coins with their data which are available on cryptocoincharts.
        Usage: http://api.cryptocoincharts.info/listCoins
        '''
        url = self.API_PATH + 'listCoins'

        json_data = json.loads(self._getdata(url))

        coins = []
        for entry in json_data:
            coin = Coin()
            coin.id = entry['id']
            coin.name = entry['name']
            coin.website = entry['website']
            coin.price_btc = entry['price_btc']
            coin.volume_btc = entry['volume_btc']
            coins.append(coin)

        return coins

Response: This function returns a list of all coins with their data which are available on cryptocoincharts.


Instruction:
Write a Python 3 function that can
use this function to query price and volume data for ONE trading pair. A list with all coin currencies can be found by using the listcoins method. A example pair: currency1_currency2 = "doge_btc" Usage: http://api.cryptocoincharts.info/tradingPair/[currency1_currency2]

Response:
def tradingpair(self, pair):
        '''
        Use this function to query price and volume data for ONE trading pair.
        A list with all coin currencies can be found by using the listcoins method.
        A example pair: currency1_currency2 = "doge_btc"
        Usage: http://api.cryptocoincharts.info/tradingPair/[currency1_currency2]
        '''
        url = self.API_PATH + 'tradingPair/' + pair

        json_data = json.loads(self._getdata(url))

        tradingpair = TradingPair()
        tradingpair.id = json_data['id']
        tradingpair.price = json_data['price']
        tradingpair.price_before_24h = json_data['price_before_24h']
        tradingpair.volume_first = json_data['volume_first']
        tradingpair.volume_second = json_data['volume_second']
        tradingpair.volume_btc = json_data['volume_btc']
        tradingpair.best_market = json_data['best_market']
        tradingpair.latest_trade = json_data['latest_trade']

        return tradingpair


Instruction:
Can you create a Python 3 function that
uses this function to query price and volume data for MANY trading pairs. Usage: http://api.cryptocoincharts.info/tradingPairs/[currency1_currency2,currency2_currency3,...] A example pair: currency1_currency2 = "doge_btc" currency2_currency3 = "btc_eur" http://api.cryptocoincharts.info/tradingPairs/"doge_btc,btc_eur"

Response:
def tradingpairs(self, pairs):
        '''
        Use this function to query price and volume data for MANY trading pairs.
        Usage: http://api.cryptocoincharts.info/tradingPairs/[currency1_currency2,currency2_currency3,...]
               A example pair: currency1_currency2 = "doge_btc"
                               currency2_currency3 = "btc_eur"
               http://api.cryptocoincharts.info/tradingPairs/"doge_btc,btc_eur"
        '''
        url = self.API_PATH + 'tradingPairs/'
        data = { 'pairs':pairs }

        json_data = json.loads(self._getdata(url, data))

        tradingpairs = []
        for entry in json_data:
            tradingpair = TradingPair()
            tradingpair.id = entry['id']
            tradingpair.price = entry['price']
            tradingpair.price_before_24h = entry['price_before_24h']
            tradingpair.volume_first = entry['volume_first']
            tradingpair.volume_second = entry['volume_second']
            tradingpair.volume_btc = entry['volume_btc']
            tradingpair.best_market = entry['best_market']
            tradingpair.latest_trade = entry['latest_trade']
            tradingpairs.append(tradingpair)

        return tradingpairs


Instruction:
How would you explain what the following Python 3 function does
def _getdata(self, url, data = ""):
        '''
        Wrapper method
        '''
        request = Request(url)

        if data != "":
            request = Request(url, urlencode(data))

        try:
            response = urlopen(request)
        except HTTPError as e:
            print('The Server couldn\'t fulfill the request.')
            print('Error code: ', e.code)
        except URLError as e:
            print('We failed to reach a server.')
            print('Reason: ', e.code)
        else:
            # Everything is fine.
            return response.read()

Response: Wrapper method for getdata


Instruction:
Create a Python 3 function for
handling the jobs from the master and executes them and returns a response.

Response:
async def handle_jobs(job_handler, host, port, *, loop):
    """
    Connects to the remote master and continuously receives calls, executes
    them, then returns a response until interrupted.
    """

    try:

        try:
            reader, writer = await asyncio.open_connection(host, port, loop=loop)
        except OSError:
            logging.error("worker could not connect to server")
            return

        while True:

            try:
                call_encoded = await reader.readuntil(b"\n")
            except (asyncio.IncompleteReadError, ConnectionResetError):
                break
            logging.debug("worker got call")
            call_json = call_encoded.decode("utf-8")
            call = json.loads(call_json)

            response = job_handler(call)

            response_json = json.dumps(response) + "\n"
            response_encoded = response_json.encode("utf-8")
            writer.write(response_encoded)
            logging.debug("worker returned response")

    except KeyboardInterrupt:

        pass


Instruction:
Implement a function in Python 3 to
start an asyncio event loop to connect to the master and run jobs.

Response:
def worker_main(job_handler, host, port):
    """
    Starts an asyncio event loop to connect to the master and run jobs.
    """

    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(None)
    loop.run_until_complete(handle_jobs(job_handler, host, port, loop=loop))
    loop.close()


Instruction:
Can you generate the documentation for the following Python 3 function
def run_worker_pool(job_handler, host="localhost", port=48484,
                      *, max_workers=None):
    """
    Runs a pool of workers which connect to a remote HighFive master and begin
    executing calls.
    """

    if max_workers is None:
        max_workers = multiprocessing.cpu_count()

    processes = []
    for _ in range(max_workers):
        p = multiprocessing.Process(target=worker_main,
                args=(job_handler, host, port))
        p.start()
        processes.append(p)

    logger.debug("workers started")

    for p in processes:
        p.join()

    logger.debug("all workers completed")

Response: Runs a pool of workers which connect to a remote HighFive master and begin executing calls.


Instruction:
Create a Python 3 function to
set the classification of this CompanyDetailCompany.

Response:
def classification(self, classification):
        """
        Sets the classification of this CompanyDetailCompany.
        Classification of Company

        :param classification: The classification of this CompanyDetailCompany.
        :type: str
        """
        allowed_values = ["Public Limited Indian Non-Government Company", "Private Limited Indian Non-Government Company", "One Person Company", "Private Limited Foreign Company Incorporated in India", "Public Limited Foreign Company Incorporated in India", "Union Government Company", "State Government Company", "Guarantee & Association Public", "Guarantee & Association Private", "Not For Profit Company", "Unlimited Liabilities Public", "Unlimited Liabilities Private", "Undefined"]
        if classification not in allowed_values:
            raise ValueError(
                "Invalid value for `classification`, must be one of {0}"
                .format(allowed_values)
            )
        self._classification = classification


Instruction:
How would you explain what the following Python 3 function does
def _send_message(self, msg):
        LWLink.the_queue.put_nowait(msg)
        if LWLink.thread is None or not LWLink.thread.isAlive():
            LWLink.thread = Thread(target=self._send_queue)
            LWLink.thread.start()

Response: Add a message to the queue and start processing the queue.


Instruction:
Write a Python 3 script for
turning on the light.

Response:
def turn_on_light(self, device_id, name):
        """Create the message to turn light on."""
        msg = "!%sFdP32|Turn On|%s" % (device_id, name)
        self._send_message(msg)


Instruction:
Explain what the following Python 3 code does
def turn_on_switch(self, device_id, name):
        msg = "!%sF1|Turn On|%s" % (device_id, name)
        self._send_message(msg)

Response: Turn on switch on.


Instruction:
Implement a Python 3 function for
turning on with the given brightness.

Response:
def turn_on_with_brightness(self, device_id, name, brightness):
        """Scale brightness from 0..255 to 1..32."""
        brightness_value = round((brightness * 31) / 255) + 1
        # F1 = Light on and F0 = light off. FdP[0..32] is brightness. 32 is
        # full. We want that when turning the light on.
        msg = "!%sFdP%d|Lights %d|%s" % (
            device_id, brightness_value, brightness_value, name)
        self._send_message(msg)


Instruction:
Can you create a Python 3 function that
turns light or switch off.

Response:
def turn_off(self, device_id, name):
        """Create the message to turn light or switch off."""
        msg = "!%sF0|Turn Off|%s" % (device_id, name)
        self._send_message(msg)


Instruction:
Given the following Python 3 function, write the documentation
def _send_queue(self):
        while not LWLink.the_queue.empty():
            self._send_reliable_message(LWLink.the_queue.get_nowait())

Response: Process the queue and send the messages.


Instruction:
How would you explain what the following Python 3 function does
def _send_reliable_message(self, msg):
        result = False
        max_retries = 15
        trans_id = next(LWLink.transaction_id)
        msg = "%d,%s" % (trans_id, msg)
        try:
            with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) \
                    as write_sock, \
                    socket.socket(socket.AF_INET, socket.SOCK_DGRAM) \
                    as read_sock:
                write_sock.setsockopt(
                    socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
                read_sock.setsockopt(socket.SOL_SOCKET,
                                     socket.SO_BROADCAST, 1)
                read_sock.settimeout(self.SOCKET_TIMEOUT)
                read_sock.bind(('0.0.0.0', self.RX_PORT))
                while max_retries:
                    max_retries -= 1
                    write_sock.sendto(msg.encode(
                        'UTF-8'), (LWLink.link_ip, self.TX_PORT))
                    result = False
                    while True:
                        response, dummy = read_sock.recvfrom(1024)
                        response = response.decode('UTF-8')
                        if "Not yet registered." in response:
                            _LOGGER.error("Not yet registered")
                            self.register()
                            result = True
                            break

                        if response.startswith("%d,OK" % trans_id):
                            result = True
                            break
                        if response.startswith("%d,ERR" % trans_id):
                            _LOGGER.error(response)
                            break

                        _LOGGER.info(response)

                    if result:
                        break

                    time.sleep(0.25)

        except socket.timeout:
            _LOGGER.error("LW broker timeout!")
            return result

        except Exception as ex:
            _LOGGER.error(ex)
            raise

        if result:
            _LOGGER.info("LW broker OK!")
        else:
            _LOGGER.error("LW broker fail!")
        return result

Response: Send a message to LightwaveRF hub.


Instruction:
Can you create a Python 3 function that
creates a CMPH capable adapter for the given object.

Response:
def create_adapter(cmph, ffi, obj):
    """ Generates a wrapped adapter for the given object

    Parameters
    ----------
    obj : list, buffer, array, or file

    Raises
    ------
    ValueError
        If presented with an object that cannot be adapted

    Returns
    -------
    CMPH capable adapter
    """

    # if arraylike and fixed unit size
    # if file
    # if buffer

    if is_file_location(obj):
        # The FP is captured for GC reasons inside the dtor closure
        # pylint: disable=invalid-name
        fd = open(obj)
        adapter = cmph.cmph_io_nlfile_adapter(fd)

        def dtor():
            cmph.cmph_io_nlfile_adapter_destroy(adapter)
            fd.close()

        # pylint: enable=invalid-name
        return _AdapterCxt(adapter, dtor)
    elif is_file(obj):
        adapter = cmph.cmph_io_nlfile_adapter(obj)
        dtor = lambda: cmph.cmph_io_nlfile_adapter_destroy(adapter)
        return _AdapterCxt(adapter, dtor)
    elif isinstance(obj, Sequence):
        if len(obj) == 0:
            raise ValueError("An empty sequence is already a perfect hash!")
        return _create_pyobj_adapter(cmph, ffi, obj)
    else:
        raise ValueError("data cannot have a cmph wrapper generated")


Instruction:
Explain what the following Python 3 code does
def nature(self, nature):
        allowed_values = ["STANDALONE"]
        if nature not in allowed_values:
            raise ValueError(
                "Invalid value for `nature`, must be one of {0}"
                .format(allowed_values)
            )
        self._nature = nature

Response: Sets the nature of this YearlyFinancials.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def computed_displaywidth():
    '''Figure out a reasonable default with. Use os.environ['COLUMNS'] if possible,
    and failing that use 80.
    '''
    try:
        width = int(os.environ['COLUMNS'])
    except (KeyError, ValueError):
        width = get_terminal_size().columns

    return width or 80

Response: Figure out a reasonable default with. Use os. environ. COLUMNS or get_terminal_size. columns otherwise.


Instruction:
Explain what the following Python 3 code does
def columnize(array, displaywidth=80, colsep = '  ',
              arrange_vertical=True, ljust=True, lineprefix='',
              opts={}):
    """Return a list of strings as a compact set of columns arranged
    horizontally or vertically.

    For example, for a line width of 4 characters (arranged vertically):
        ['1', '2,', '3', '4'] => '1  3\n2  4\n'

    or arranged horizontally:
        ['1', '2,', '3', '4'] => '1  2\n3  4\n'

    Each column is only as wide as necessary.  By default, columns are
    separated by two spaces - one was not legible enough. Set "colsep"
    to adjust the string separate columns. Set `displaywidth' to set
    the line width.

    Normally, consecutive items go down from the top to bottom from
    the left-most column to the right-most. If "arrange_vertical" is
    set false, consecutive items will go across, left to right, top to
    bottom."""
    if not isinstance(array, (list, tuple)):
        raise TypeError((
            'array needs to be an instance of a list or a tuple'))

    o = {}
    if len(opts.keys()) > 0:
        for key in default_opts.keys():
            o[key] = get_option(key, opts)
            pass
        if o['arrange_array']:
            o['array_prefix'] = '['
            o['lineprefix']   = ' '
            o['linesuffix']   = ",\n"
            o['array_suffix'] = "]\n"
            o['colsep']       = ', '
            o['arrange_vertical'] = False
            pass

    else:
        o = default_opts.copy()
        o['displaywidth']     = displaywidth
        o['colsep']           = colsep
        o['arrange_vertical'] = arrange_vertical
        o['ljust']            = ljust
        o['lineprefix']       = lineprefix
        pass

    # if o['ljust'] is None:
    #     o['ljust'] = !(list.all?{|datum| datum.kind_of?(Numeric)})
    #     pass

    if o['colfmt']:
        array = [(o['colfmt'] % i) for i in array]
    else:
        array = [str(i) for i in array]
        pass

    # Some degenerate cases
    size = len(array)
    if 0 == size:
        return "<empty>\n"
    elif size == 1:
        return '%s%s%s\n' % (o['array_prefix'], str(array[0]),
                             o['array_suffix'])

    o['displaywidth'] = max(4, o['displaywidth'] - len(o['lineprefix']))
    if o['arrange_vertical']:
        array_index = lambda nrows, row, col: nrows*col + row
        # Try every row count from 1 upwards
        for nrows in range(1, size+1):
            ncols = (size+nrows-1) // nrows
            colwidths = []
            totwidth = -len(o['colsep'])
            for col in range(ncols):
                # get max column width for this column
                colwidth = 0
                for row in range(nrows):
                    i = array_index(nrows, row, col)
                    if i >= size: break
                    x = array[i]
                    colwidth = max(colwidth, len(x))
                    pass
                colwidths.append(colwidth)
                totwidth += colwidth + len(o['colsep'])
                if totwidth > o['displaywidth']:
                    break
                pass
            if totwidth <= o['displaywidth']:
                break
            pass
        # The smallest number of rows computed and the
        # max widths for each column has been obtained.
        # Now we just have to format each of the
        # rows.
        s = ''
        for row in range(nrows):
            texts = []
            for col in range(ncols):
                i = row + nrows*col
                if i >= size:
                    x = ""
                else:
                    x = array[i]
                texts.append(x)
            while texts and not texts[-1]:
                del texts[-1]
            for col in range(len(texts)):
                if o['ljust']:
                    texts[col] = texts[col].ljust(colwidths[col])
                else:
                    texts[col] = texts[col].rjust(colwidths[col])
                    pass
                pass
            s += "%s%s%s" % (o['lineprefix'], str(o['colsep'].join(texts)),
                             o['linesuffix'])
            pass
        return s
    else:
        array_index = lambda ncols, row, col: ncols*(row-1) + col
        # Try every column count from size downwards
        colwidths = []
        for ncols in range(size, 0, -1):
            # Try every row count from 1 upwards
            min_rows = (size+ncols-1) // ncols
            nrows = min_rows -1
            while nrows < size:
                nrows += 1
                rounded_size = nrows * ncols
                colwidths = []
                totwidth  = -len(o['colsep'])
                for col in range(ncols):
                    # get max column width for this column
                    colwidth  = 0
                    for row in range(1, nrows+1):
                        i = array_index(ncols, row, col)
                        if i >= rounded_size: break
                        elif i < size:
                            x = array[i]
                            colwidth = max(colwidth, len(x))
                            pass
                        pass
                    colwidths.append(colwidth)
                    totwidth += colwidth + len(o['colsep'])
                    if totwidth >= o['displaywidth']:
                        break
                    pass
                if totwidth <= o['displaywidth'] and i >= rounded_size-1:
                    # Found the right nrows and ncols
                    # print "right nrows and ncols"
                    nrows  = row
                    break
                elif totwidth >= o['displaywidth']:
                    # print "reduce ncols", ncols
                    # Need to reduce ncols
                    break
                pass
            if totwidth <= o['displaywidth'] and i >= rounded_size-1:
                break
            pass
        # The smallest number of rows computed and the
        # max widths for each column has been obtained.
        # Now we just have to format each of the
        # rows.
        s = ''
        if len(o['array_prefix']) != 0:
            prefix = o['array_prefix']
        else:
            prefix = o['lineprefix']
            pass
        for row in range(1, nrows+1):
            texts = []
            for col in range(ncols):
                i = array_index(ncols, row, col)
                if i >= size:
                    break
                else: x = array[i]
                texts.append(x)
                pass
            for col in range(len(texts)):
                if o['ljust']:
                    texts[col] = texts[col].ljust(colwidths[col])
                else:
                    texts[col] = texts[col].rjust(colwidths[col])
                    pass
                pass
            s += "%s%s%s" % (prefix, str(o['colsep'].join(texts)),
                             o['linesuffix'])
            prefix = o['lineprefix']
            pass
        if o['arrange_array']:
            colsep = o['colsep'].rstrip()
            colsep_pos = -(len(colsep)+1)
            if s[colsep_pos:] == colsep + "\n":
                s = s[:colsep_pos] + o['array_suffix'] + "\n"
                pass
            pass
        else:
            s += o['array_suffix']
            pass
        return s
    pass

Response: Return a list of strings as a compact set of columns.


Instruction:
Create a Python 3 function for
generating a new Minimal Perfect Hash (MPH) Parameters ---------- data : list, array-like, file-like The input that is used to generate the minimal perfect hash. Be aware, in most cases the input is expected to be distinct, and many of the algorithms benefit from the input being sorted. algorithm : string, optional {chd_ph (default), chd, bmz, bmz8, chm, brz, fch, bdz, bdz_ph} The algorithm to use in generating MPH's, choice of: chd / chd_ph - Compress Hash and Displace (default) (http://cmph.sourceforge.net/chd.html) - It is the fastest algorithm to build PHFs and MPHFs in linear time. - It generates the most compact PHFs and MPHFs we know of. - It can generate PHFs with a load factor up to 99 %. - It can be used to generate t-perfect hash functions. A t-perfect hash function allows at most t collisions in a given bin. It is a well-known fact that modern memories are organized as blocks which constitute transfer unit. Example of such blocks are cache lines for internal memory or sectors for hard disks. Thus, it can be very useful for devices that carry out I/O operations in blocks. - It is a two level scheme. It uses a first level hash function to split the key set in buckets of average size determined by a parameter b in the range [1,32]. In the second level it uses displacement values to resolve the collisions that have given rise to the buckets. - It can generate MPHFs that can be stored in approximately 2.07 bits per key. - For a load factor equal to the maximum one that is achieved by the BDZ algorithm (81 %), the resulting PHFs are stored in approximately 1.40 bits per key. bdz - BDZ / BPZ algorithm (http://cmph.sourceforge.net/bdz.html) - It is very simple and efficient. It outperforms all others except CHD. - It constructs both PHFs and MPHFs in linear time. - The maximum load factor one can achieve for a PHF is 1/1.23. - It is based on acyclic random 3-graphs. A 3-graph is a generalization of a graph where each edge connects 3 vertices instead of only 2. - The resulting MPHFs are not order preserving. - The resulting MPHFs can be stored in only (2 + x)cn bits, where c should be larger than or equal to 1.23 and x is a constant larger than 0 (actually, x = 1/b and b is a parameter that should be larger than 2). For c = 1.23 and b = 8, the resulting functions are stored in approximately 2.6 bits per key. - For its maximum load factor (81 %), the resulting PHFs are stored in approximately 1.95 bits per key. bmz - Botelho, Menoti and Ziviani algorithm: (http://cmph.sourceforge.net/bdz.html) - Constructs MPHFs in linear time. - It is based on cyclic random graphs. This makes it faster than the CHM algorithm. - The resulting MPHFs are not order preserving. - The resulting MPHFs are more compact than the ones generated by the CHM algorithm and can be stored in 4cn bytes, where c is in the range [0.93,1.15]. brz - BRZ algorithm: (http://cmph.sourceforge.net/brz.html) - A very fast external memory based algorithm for constructing minimal perfect hash functions for sets in the order of billions of keys. - It works in linear time. - The resulting MPHFs are not order preserving. - The resulting MPHFs can be stored using less than 8.0 bits per key. chm - Czech, Havas and Majewski algorithm: (http://cmph.sourceforge.net/chm.html) - Construct minimal MPHFs in linear time. - It is based on acyclic random graphs - The resulting MPHFs are order preserving. - The resulting MPHFs are stored in 4cn bytes, where c is greater than 2. fch - Fox, Chen and Heath algorithm: (http://cmph.sourceforge.net/chm.html) - Construct minimal perfect hash functions that require less than 4 bits per key to be stored. - The resulting MPHFs are very compact and very efficient at evaluation time - The algorithm is only efficient for small sets. - It is used as internal algorithm in the BRZ algorithm to efficiently solve larger problems and even so to generate MPHFs that require approximately 4.1 bits per key to be stored. For that, you just need to set the parameters -a to brz and -c to a value larger than or equal to 2.6. hash_fns : list {jenkins (default), count} optional Internal hash functions to use inside MPH generation functions, can be multiple fns as a list. chd_keys_per_bin : int [1 to 128], optional Set the number of keys per bin for a t-perfect hashing function. A t-perfect hash function allows at most t collisions in a given bin. This parameter applies only to the `chd` and `chd_ph` algorithms. Its value should be an integer in the range [1, 128]. Default is 1 chd_load_factor : float, optional The load factor used in the `chd_ph` algorithm fch_bits_per_key : int, optional The number of bits per key required in the FCH algorithm num_graph_vertices : int, optional The number of vertices in the graph for the algorithms BMZ and CHM brz_memory_size : int (default 8), optional Main memory availability (in MB) used in BRZ algorithm Default is 8Mb brz_temp_dir : string, optional Temporary directory used in BRZ algorithm brz_max_keys_per_bucket : int [64 to 175] (default 128), optional Used to make the maximal number of keys in a bucket lower than 256. In this case its value should be an integer in the range [64,175]. Default is 128. bdz_precomputed_rank : int [3 to 10] (default 7), optional For BDZ it is used to determine the size of some precomputed rank information and its value should be an integer in the range [3,10]. Default is 7. The larger is this value, the more compact are the resulting functions and the slower are them at evaluation time. chd_avg_keys_per_bucket : int [1 to 32] (default 4), optional For CHD and CHD_PH it is used to set the average number of keys per bucket and its value should be an integer in the range [1,32]. Default is 4. The larger is this value, the slower is the construction of the functions. Returns ------- MPH A wrapper object that represents a minimal perfect hash in memory Raises ------ ValueError If arguments presented are incomplete, or incompatable RuntimeError If the MPH generation fails

Response:
def generate_hash(data, algorithm='chd_ph', hash_fns=(), chd_keys_per_bin=1,
                  chd_load_factor=None, fch_bits_per_key=None,
                  num_graph_vertices=None, brz_memory_size=8,
                  brz_temp_dir=None, brz_max_keys_per_bucket=128,
                  bdz_precomputed_rank=7, chd_avg_keys_per_bucket=4):
    """
    Generates a new Minimal Perfect Hash (MPH)

    Parameters
    ----------
    data : list, array-like, file-like
        The input that is used to generate the minimal perfect hash.

        Be aware, in most cases the input is expected to be distinct, and
        many of the algorithms benefit from the input being sorted.

    algorithm : string, optional
        {chd_ph (default), chd, bmz, bmz8, chm, brz, fch, bdz, bdz_ph}
        The algorithm to use in generating MPH's, choice of:

        chd / chd_ph - Compress Hash and Displace (default)
            (http://cmph.sourceforge.net/chd.html)
            - It is the fastest algorithm to build PHFs and MPHFs in linear
            time.
            - It generates the most compact PHFs and MPHFs we know of.
            - It can generate PHFs with a load factor up to 99 %.
            - It can be used to generate t-perfect hash functions. A
            t-perfect hash function allows at most t collisions in a given
            bin. It is a well-known fact that modern memories are
            organized as blocks which constitute transfer unit. Example of
            such blocks are cache lines for internal memory or sectors for
            hard disks. Thus, it can be very useful for devices that
            carry out I/O operations in blocks.
            - It is a two level scheme. It uses a first level hash function
            to split the key set in buckets of average size determined by
            a parameter b in the range [1,32]. In the second level it uses
            displacement values to resolve the collisions that have given
            rise to the buckets.
            - It can generate MPHFs that can be stored in approximately 2.07
            bits per key.
            - For a load factor equal to the maximum one that is achieved by
            the BDZ algorithm (81 %), the resulting PHFs are stored in
            approximately 1.40 bits per key.

        bdz - BDZ / BPZ algorithm
            (http://cmph.sourceforge.net/bdz.html)
            - It is very simple and efficient. It outperforms all others
            except CHD.
            - It constructs both PHFs and MPHFs in linear time.
            - The maximum load factor one can achieve for a PHF is 1/1.23.
            - It is based on acyclic random 3-graphs. A 3-graph is a
            generalization of a graph where each edge connects 3 vertices
            instead of only 2.
            - The resulting MPHFs are not order preserving.
            - The resulting MPHFs can be stored in only (2 + x)cn bits,
            where c should be larger than or equal to 1.23 and x is a
            constant larger than 0 (actually, x = 1/b and b is a parameter
            that should be larger than 2). For c = 1.23 and b = 8, the
            resulting functions are stored in approximately 2.6 bits per
            key.
            - For its maximum load factor (81 %), the resulting PHFs are
            stored in approximately 1.95 bits per key.

        bmz - Botelho, Menoti and Ziviani algorithm:
            (http://cmph.sourceforge.net/bdz.html)
            - Constructs MPHFs in linear time.
            - It is based on cyclic random graphs. This makes it faster than
            the CHM algorithm.
            - The resulting MPHFs are not order preserving.
            - The resulting MPHFs are more compact than the ones generated by
            the CHM algorithm and can be stored in 4cn bytes, where c is in
            the range [0.93,1.15].

        brz - BRZ algorithm:
            (http://cmph.sourceforge.net/brz.html)
            - A very fast external memory based algorithm for constructing
            minimal perfect hash functions for sets in the order of
            billions of keys.
            - It works in linear time.
            - The resulting MPHFs are not order preserving.
            - The resulting MPHFs can be stored using less than 8.0 bits per
            key.

        chm - Czech, Havas and Majewski algorithm:
            (http://cmph.sourceforge.net/chm.html)
            - Construct minimal MPHFs in linear time.
            - It is based on acyclic random graphs
            - The resulting MPHFs are order preserving.
            - The resulting MPHFs are stored in 4cn bytes, where c is greater
            than 2.

        fch - Fox, Chen and Heath algorithm:
            (http://cmph.sourceforge.net/chm.html)
            - Construct minimal perfect hash functions that require less than
            4 bits per key to be stored.
            - The resulting MPHFs are very compact and very efficient at
            evaluation time
            - The algorithm is only efficient for small sets.
            - It is used as internal algorithm in the BRZ algorithm to
            efficiently solve larger problems and even so to generate MPHFs
            that require approximately 4.1 bits per key to be stored. For
            that, you just need to set the parameters -a to brz and -c to a
            value larger than or equal to 2.6.

    hash_fns : list {jenkins (default), count} optional
        Internal hash functions to use inside MPH generation functions,
        can be multiple fns as a list.

    chd_keys_per_bin : int [1 to 128], optional
        Set the number of keys per bin for a t-perfect hashing function. A
        t-perfect hash function allows at most t collisions in a given bin.
        This parameter applies only to the `chd` and `chd_ph` algorithms.
        Its value should be an integer in the range [1, 128].

        Default is 1

    chd_load_factor : float, optional
        The load factor used in the `chd_ph` algorithm

    fch_bits_per_key : int, optional
        The number of bits per key required in the FCH algorithm

    num_graph_vertices : int, optional
        The number of vertices in the graph for the algorithms BMZ and CHM

    brz_memory_size : int (default 8), optional
        Main memory availability (in MB) used in BRZ algorithm

        Default is 8Mb

    brz_temp_dir : string, optional
        Temporary directory used in BRZ algorithm

    brz_max_keys_per_bucket : int [64 to 175] (default 128), optional
        Used to make the maximal number of keys in a bucket lower than 256.
        In this case its value should be an integer in the range [64,175].
        Default is 128.

    bdz_precomputed_rank : int [3 to 10] (default 7), optional
        For BDZ it is used to determine the size of some precomputed rank
        information and its value should be an integer in the range [3,10].

        Default is 7.

        The larger is this value, the more compact are the resulting
        functions and the slower are them at evaluation time.

    chd_avg_keys_per_bucket : int [1 to 32] (default 4), optional
        For CHD and CHD_PH it is used to set the average number of keys per
        bucket and its value should be an integer in the range [1,32].

        Default is 4.

        The larger is this value, the slower is the construction of the
        functions.

    Returns
    -------
    MPH
        A wrapper object that represents a minimal perfect hash in memory

    Raises
    ------
    ValueError
        If arguments presented are incomplete, or incompatable
    RuntimeError
        If the MPH generation fails
    """

    cfg = _cfg(algorithm, hash_fns, chd_keys_per_bin, chd_load_factor,
               fch_bits_per_key, num_graph_vertices, brz_memory_size,
               brz_temp_dir, brz_max_keys_per_bucket, bdz_precomputed_rank,
               chd_avg_keys_per_bucket)

    with create_adapter(_cmph, ffi, data) as source:
        with _create_config(source, cfg) as config:
            _mph = _cmph.cmph_new(config)
            if not _mph:
                raise RuntimeError("MPH generation failed")
            return MPH(_mph)


Instruction:
Can you generate the documentation for the following Python 3 function
def load_hash(existing_mph):
    if is_file_location(existing_mph):
        with open(abspath(existing_mph)) as hash_table:
            _mph = _cmph.cmph_load(hash_table)
    elif is_file(existing_mph):
        _mph = _cmph.cmph_load(existing_mph)

    if not _mph:
        raise IOError("Unable to load an MPH from the given source")

    return MPH(_mph)

Response: Loads a minimal perfect hash from an existing MPH file.


Instruction:
Make a summary of the following Python 3 code
def save(self, output):
        assert self._mph, "There is no MPH ?"
        if isinstance(output, six.string_types):
            with open(abspath(output), 'w') as out:
                _cmph.cmph_dump(self._mph, out)
        else:
            _cmph.cmph_dump(self._mph, output)

Response: Saves the MPH to a file - like object.


Instruction:
Make a summary of the following Python 3 code
def lookup(self, key):
        assert self._mph
        key = convert_to_bytes(key)
        box = ffi.new('char[]', key)
        try:
            result = _cmph.cmph_search(self._mph, box, len(key))
            return result
        finally:
            del box

Response: Returns the hash code for the given item in the cache.


Instruction:
How would you code a function in Python 3 to
update values of configuration section with dict.

Response:
def update_(self, sct_dict, conf_arg=True):
        """Update values of configuration section with dict.

        Args:
            sct_dict (dict): dict indexed with option names. Undefined
                options are discarded.
            conf_arg (bool): if True, only options that can be set in a config
                file are updated.
        """
        for opt, val in sct_dict.items():
            if opt not in self.def_:
                continue
            if not conf_arg or self.def_[opt].conf_arg:
                self[opt] = val


Instruction:
How would you code a function in Python 3 to
restore default values of options in this section.

Response:
def reset_(self):
        """Restore default values of options in this section."""
        for opt, meta in self.defaults_():
            self[opt] = meta.default


Instruction:
How would you explain what the following Python 3 function does
def from_dict_(cls, conf_dict):
        return cls(**{name: Section(**opts)
                      for name, opts in conf_dict.items()})

Response: Use a dictionary to create a ConfigurationManager.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def set_config_files_(self, *config_files):
        self._config_files = tuple(pathlib.Path(path) for path in config_files)

Response: Set the list of config files.


Instruction:
How would you explain what the following Python 3 function does
def opt_vals_(self):
        for sct, opt in self.options_():
            yield sct, opt, self[sct][opt]

Response: Iterator over sections option names and option values.


Instruction:
Explain what the following Python 3 code does
def defaults_(self):
        for sct, opt in self.options_():
            yield sct, opt, self[sct].def_[opt]

Response: Iterator over sections option names and option metadata.


Instruction:
Implement a Python 3 function for
creating config file in self. config_files_ [ index ].

Response:
def create_config_(self, index=0, update=False):
        """Create config file.

        Create config file in :attr:`config_files_[index]`.

        Parameters:
            index(int): index of config file.
            update (bool): if set to True and :attr:`config_files_` already
                exists, its content is read and all the options it sets are
                kept in the produced config file.
        """
        if not self.config_files_[index:]:
            return
        path = self.config_files_[index]
        if not path.parent.exists():
            path.parent.mkdir(parents=True)
        conf_dict = {}
        for section in self.sections_():
            conf_opts = [o for o, m in self[section].defaults_() if m.conf_arg]
            if not conf_opts:
                continue
            conf_dict[section] = {}
            for opt in conf_opts:
                conf_dict[section][opt] = (self[section][opt] if update else
                                           self[section].def_[opt].default)
        with path.open('w') as cfile:
            toml.dump(conf_dict, cfile)


Instruction:
Implement a Python 3 function for
updating values of configuration options with dict.

Response:
def update_(self, conf_dict, conf_arg=True):
        """Update values of configuration options with dict.

        Args:
            conf_dict (dict): dict of dict indexed with section and option
                names.
            conf_arg (bool): if True, only options that can be set in a config
                file are updated.
        """
        for section, secdict in conf_dict.items():
            self[section].update_(secdict, conf_arg)


Instruction:
Can you tell what is the following Python 3 function doing
def read_config_(self, cfile):
        if not cfile.exists():
            return {}
        try:
            conf_dict = toml.load(str(cfile))
        except toml.TomlDecodeError:
            return None
        self.update_(conf_dict)
        return conf_dict

Response: Read a config file and set config values accordingly.


Instruction:
Can you implement a function in Python 3 that
reads config files and set config values accordingly.

Response:
def read_configs_(self):
        """Read config files and set config values accordingly.

        Returns:
            (dict, list, list): respectively content of files, list of
            missing/empty files and list of files for which a parsing error
            arised.
        """
        if not self.config_files_:
            return {}, [], []
        content = {section: {} for section in self}
        empty_files = []
        faulty_files = []
        for cfile in self.config_files_:
            conf_dict = self.read_config_(cfile)
            if conf_dict is None:
                faulty_files.append(cfile)
                continue
            elif not conf_dict:
                empty_files.append(cfile)
                continue
            for section, secdict in conf_dict.items():
                content[section].update(secdict)
        return content, empty_files, faulty_files


Instruction:
Create a Python 3 function for
listing of cli strings for a given option.

Response:
def _names(section, option):
    """List of cli strings for a given option."""
    meta = section.def_[option]
    action = meta.cmd_kwargs.get('action')
    if action is internal.Switch:
        names = ['-{}'.format(option), '+{}'.format(option)]
        if meta.shortname is not None:
            names.append('-{}'.format(meta.shortname))
            names.append('+{}'.format(meta.shortname))
    else:
        names = ['--{}'.format(option)]
        if meta.shortname is not None:
            names.append('-{}'.format(meta.shortname))
    return names


Instruction:
Make a summary of the following Python 3 code
def sections_list(self, cmd=None):
        sections = list(self.common.sections)
        if not cmd:
            if self.bare is not None:
                sections.extend(self.bare.sections)
                return sections
            return []
        sections.extend(self.subcmds[cmd].sections)
        if cmd in self._conf:
            sections.append(cmd)
        return sections

Response: Returns a list of config sections used by a command.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _cmd_opts_solver(self, cmd_name):
        sections = self.sections_list(cmd_name)
        cmd_dict = self._opt_cmds[cmd_name] if cmd_name else self._opt_bare
        for sct in reversed(sections):
            for opt, opt_meta in self._conf[sct].def_.items():
                if not opt_meta.cmd_arg:
                    continue
                if opt not in cmd_dict:
                    cmd_dict[opt] = sct
                else:
                    warnings.warn(
                        'Command <{0}>: {1}.{2} shadowed by {3}.{2}'.format(
                            cmd_name, sct, opt, cmd_dict[opt]),
                        error.LoamWarning, stacklevel=4)

Response: Scan options related to one command and enrich _opt_cmds.


Instruction:
Can you create a Python 3 function that
adds options to a parser.

Response:
def _add_options_to_parser(self, opts_dict, parser):
        """Add options to a parser."""
        store_bool = ('store_true', 'store_false')
        for opt, sct in opts_dict.items():
            meta = self._conf[sct].def_[opt]
            kwargs = copy.deepcopy(meta.cmd_kwargs)
            action = kwargs.get('action')
            if action is internal.Switch:
                kwargs.update(nargs=0)
            elif meta.default is not None and action not in store_bool:
                kwargs.setdefault('type', type(meta.default))
            kwargs.update(help=meta.help)
            kwargs.setdefault('default', self._conf[sct][opt])
            parser.add_argument(*_names(self._conf[sct], opt), **kwargs)


Instruction:
Write a Python 3 script for
building command line argument parser.

Response:
def _build_parser(self):
        """Build command line argument parser.

        Returns:
            :class:`argparse.ArgumentParser`: the command line argument parser.
            You probably won't need to use it directly. To parse command line
            arguments and update the :class:`ConfigurationManager` instance
            accordingly, use the :meth:`parse_args` method.
        """
        main_parser = argparse.ArgumentParser(description=self.common.help,
                                              prefix_chars='-+')

        self._add_options_to_parser(self._opt_bare, main_parser)
        main_parser.set_defaults(**self.common.defaults)
        if self.bare is not None:
            main_parser.set_defaults(**self.bare.defaults)

        subparsers = main_parser.add_subparsers(dest='loam_sub_name')
        for cmd_name, meta in self.subcmds.items():
            kwargs = {'prefix_chars': '+-', 'help': meta.help}
            dummy_parser = subparsers.add_parser(cmd_name, **kwargs)
            self._add_options_to_parser(self._opt_cmds[cmd_name], dummy_parser)
            dummy_parser.set_defaults(**meta.defaults)

        return main_parser


Instruction:
Make a summary of the following Python 3 code
def parse_args(self, arglist=None):
        args = self._parser.parse_args(args=arglist)
        sub_cmd = args.loam_sub_name
        if sub_cmd is None:
            for opt, sct in self._opt_bare.items():
                self._conf[sct][opt] = getattr(args, opt, None)
        else:
            for opt, sct in self._opt_cmds[sub_cmd].items():
                self._conf[sct][opt] = getattr(args, opt, None)
        return args

Response: Parse arguments and update options accordingly.


Instruction:
Implement a function in Python 3 to
write zsh command for a given command.

Response:
def _zsh_comp_command(self, zcf, cmd, grouping, add_help=True):
        """Write zsh _arguments compdef for a given command.

        Args:
            zcf (file): zsh compdef file.
            cmd (str): command name, set to None or '' for bare command.
            grouping (bool): group options (zsh>=5.4).
            add_help (bool): add an help option.
        """
        if add_help:
            if grouping:
                print("+ '(help)'", end=BLK, file=zcf)
            print("'--help[show help message]'", end=BLK, file=zcf)
            print("'-h[show help message]'", end=BLK, file=zcf)
        # could deal with duplicate by iterating in reverse and keep set of
        # already defined opts.
        no_comp = ('store_true', 'store_false')
        cmd_dict = self._opt_cmds[cmd] if cmd else self._opt_bare
        for opt, sct in cmd_dict.items():
            meta = self._conf[sct].def_[opt]
            if meta.cmd_kwargs.get('action') == 'append':
                grpfmt, optfmt = "+ '{}'", "'*{}[{}]{}'"
                if meta.comprule is None:
                    meta.comprule = ''
            else:
                grpfmt, optfmt = "+ '({})'", "'{}[{}]{}'"
            if meta.cmd_kwargs.get('action') in no_comp \
               or meta.cmd_kwargs.get('nargs') == 0:
                meta.comprule = None
            if meta.comprule is None:
                compstr = ''
            elif meta.comprule == '':
                optfmt = optfmt.split('[')
                optfmt = optfmt[0] + '=[' + optfmt[1]
                compstr = ': :( )'
            else:
                optfmt = optfmt.split('[')
                optfmt = optfmt[0] + '=[' + optfmt[1]
                compstr = ': :{}'.format(meta.comprule)
            if grouping:
                print(grpfmt.format(opt), end=BLK, file=zcf)
            for name in _names(self._conf[sct], opt):
                print(optfmt.format(name, meta.help.replace("'", "'\"'\"'"),
                                    compstr), end=BLK, file=zcf)


Instruction:
Can you generate the documentation for the following Python 3 function
def zsh_complete(self, path, cmd, *cmds, sourceable=False):
        grouping = internal.zsh_version() >= (5, 4)
        path = pathlib.Path(path)
        firstline = ['#compdef', cmd]
        firstline.extend(cmds)
        subcmds = list(self.subcmds.keys())
        with path.open('w') as zcf:
            print(*firstline, end='\n\n', file=zcf)
            # main function
            print('function _{} {{'.format(cmd), file=zcf)
            print('local line', file=zcf)
            print('_arguments -C', end=BLK, file=zcf)
            if subcmds:
                # list of subcommands and their description
                substrs = ["{}\\:'{}'".format(sub, self.subcmds[sub].help)
                           for sub in subcmds]
                print('"1:Commands:(({}))"'.format(' '.join(substrs)),
                      end=BLK, file=zcf)
            self._zsh_comp_command(zcf, None, grouping)
            if subcmds:
                print("'*::arg:->args'", file=zcf)
                print('case $line[1] in', file=zcf)
                for sub in subcmds:
                    print('{sub}) _{cmd}_{sub} ;;'.format(sub=sub, cmd=cmd),
                          file=zcf)
                print('esac', file=zcf)
            print('}', file=zcf)
            # all subcommand completion handlers
            for sub in subcmds:
                print('\nfunction _{}_{} {{'.format(cmd, sub), file=zcf)
                print('_arguments', end=BLK, file=zcf)
                self._zsh_comp_command(zcf, sub, grouping)
                print('}', file=zcf)
            if sourceable:
                print('\ncompdef _{0} {0}'.format(cmd), *cmds, file=zcf)

Response: Write a zsh compdef script.


Instruction:
Write a Python 3 function that can
build a list of all CLI options for a given command.

Response:
def _bash_comp_command(self, cmd, add_help=True):
        """Build a list of all options for a given command.

        Args:
            cmd (str): command name, set to None or '' for bare command.
            add_help (bool): add an help option.

        Returns:
            list of str: list of CLI options strings.
        """
        out = ['-h', '--help'] if add_help else []
        cmd_dict = self._opt_cmds[cmd] if cmd else self._opt_bare
        for opt, sct in cmd_dict:
            out.extend(_names(self._conf[sct], opt))
        return out


Instruction:
Explain what the following Python 3 code does
def bash_complete(self, path, cmd, *cmds):
        path = pathlib.Path(path)
        subcmds = list(self.subcmds.keys())
        with path.open('w') as bcf:
            # main function
            print('_{}() {{'.format(cmd), file=bcf)
            print('COMPREPLY=()', file=bcf)
            print(r'local cur=${COMP_WORDS[COMP_CWORD]}', end='\n\n', file=bcf)
            optstr = ' '.join(self._bash_comp_command(None))
            print(r'local options="{}"'.format(optstr), end='\n\n', file=bcf)
            if subcmds:
                print('local commands="{}"'.format(' '.join(subcmds)),
                      file=bcf)
                print('declare -A suboptions', file=bcf)
            for sub in subcmds:
                optstr = ' '.join(self._bash_comp_command(sub))
                print('suboptions[{}]="{}"'.format(sub, optstr), file=bcf)
            condstr = 'if'
            for sub in subcmds:
                print(condstr, r'[[ "${COMP_LINE}" == *"', sub, '"* ]] ; then',
                      file=bcf)
                print(r'COMPREPLY=( `compgen -W "${suboptions[', sub,
                      r']}" -- ${cur}` )', sep='', file=bcf)
                condstr = 'elif'
            print(condstr, r'[[ ${cur} == -* ]] ; then', file=bcf)
            print(r'COMPREPLY=( `compgen -W "${options}" -- ${cur}`)',
                  file=bcf)
            if subcmds:
                print(r'else', file=bcf)
                print(r'COMPREPLY=( `compgen -W "${commands}" -- ${cur}`)',
                      file=bcf)
            print('fi', file=bcf)
            print('}', end='\n\n', file=bcf)
            print('complete -F _{0} {0}'.format(cmd), *cmds, file=bcf)

Response: Write bash complete script.


Instruction:
Create a Python 3 function for
trying to guess zsh version return tuple of 0 on failure

Response:
def zsh_version():
    """Try to guess zsh version, return (0, 0) on failure."""
    try:
        out = subprocess.check_output(shlex.split('zsh --version'))
    except (FileNotFoundError, subprocess.CalledProcessError):
        return (0, 0)
    match = re.search(br'[0-9]+\.[0-9]+', out)
    return tuple(map(int, match.group(0).split(b'.'))) if match else (0, 0)


Instruction:
Can you write a function in Python 3 where it
starts a new HighFive master at the given host and port and returns it.

Response:
async def start_master(host="", port=48484, *, loop=None):
    """
    Starts a new HighFive master at the given host and port, and returns it.
    """

    loop = loop if loop is not None else asyncio.get_event_loop()

    manager = jobs.JobManager(loop=loop)
    workers = set()
    server = await loop.create_server(
            lambda: WorkerProtocol(manager, workers), host, port)
    return Master(server, manager, workers, loop=loop)


Instruction:
Implement a Python 3 function for
calling when a new connection is made.

Response:
def connection_made(self, transport):
        """
        Called when a remote worker connection has been found. Finishes setting
        up the protocol object.
        """

        if self._manager.is_closed():
            logger.debug("worker tried to connect while manager was closed")
            return

        logger.debug("new worker connected")

        self._transport = transport
        self._buffer = bytearray()
        self._worker = Worker(self._transport, self._manager)
        self._workers.add(self._worker)


Instruction:
Write a Python 3 script to
call when a chunk of data is received from the remote worker.

Response:
def data_received(self, data):
        """
        Called when a chunk of data is received from the remote worker. These
        chunks are stored in a buffer. When a complete line is found in the
        buffer, it removed and sent to line_received().
        """

        self._buffer.extend(data)
        while True:
            i = self._buffer.find(b"\n")
            if i == -1:
                break
            line = self._buffer[:i+1]
            self._buffer = self._buffer[i+1:]
            self.line_received(line)


Instruction:
Write a Python 3 script to
call when a complete line is found from the remote worker.

Response:
def line_received(self, line):
        """
        Called when a complete line is found from the remote worker. Decodes
        a response object from the line, then passes it to the worker object.
        """

        response = json.loads(line.decode("utf-8"))
        self._worker.response_received(response)


Instruction:
How would you implement a function in Python 3 that
calls when the connection to the remote worker is lost. Closes the worker and removes the worker from the list of available workers.

Response:
def connection_lost(self, exc):
        """
        Called when the connection to the remote worker is broken. Closes the
        worker.
        """

        logger.debug("worker connection lost")

        self._worker.close()
        self._workers.remove(self._worker)


Instruction:
Can you generate the documentation for the following Python 3 function
def _job_loaded(self, job):

        logger.debug("worker {} found a job".format(id(self)))

        if self._closed:
            self._manager.return_job(job)
            return

        self._job = job
        call_obj = self._job.get_call()
        call = (json.dumps(call_obj) + "\n").encode("utf-8")
        self._transport.write(call)

Response:
Called when a job has been loaded. Sends the job s
        RPC to the remote worker.


Instruction:
Can you generate the documentation for the following Python 3 function
def response_received(self, response):

        if self._closed:
            return

        assert self._job is not None

        logger.debug("worker {} got response".format(id(self)))
        result = self._job.get_result(response)
        self._manager.add_result(self._job, result)

        self._load_job()

Response: Called when a response to a job RPC has been received. Decodes the response and reports the result to the job manager.


Instruction:
Implement a Python 3 function for
closing the worker. No more jobs will be handled by the worker, and any running job is immediately returned to the job manager.

Response:
def close(self):
        """
        Closes the worker. No more jobs will be handled by the worker, and any
        running job is immediately returned to the job manager.
        """

        if self._closed:
            return

        self._closed = True

        if self._job is not None:
            self._manager.return_job(self._job)
            self._job = None


Instruction:
Can you generate the documentation for the following Python 3 function
def run(self, job_list):

        if self._closed:
            raise RuntimeError("master is closed")

        return self._manager.add_job_set(job_list)

Response: Runs a job set which consists of the jobs in an iterable job list.


Instruction:
Can you implement a function in Python 3 that
closes the HighFive master.

Response:
def close(self):
        """
        Starts closing the HighFive master. The server will be closed and
        all queued job sets will be cancelled.
        """

        if self._closed:
            return

        self._closed = True

        self._server.close()
        self._manager.close()
        for worker in self._workers:
            worker.close()


Instruction:
Implement a function in Python 3 to
call when a state change has occurred.

Response:
def _change(self):
        """
        Called when a state change has occurred. Waiters are notified that a
        change has occurred.
        """

        for waiter in self._waiters:
            if not waiter.done():
                waiter.set_result(None)
        self._waiters = []


Instruction:
Explain what the following Python 3 code does
def add(self, result):

        assert not self._complete

        self._results.append(result)
        self._change()

Response: Adds a new result.


Instruction:
Can you generate the documentation for the following Python 3 function
async def wait_changed(self):

        if not self.is_complete():
            waiter = self._loop.create_future()
            self._waiters.append(waiter)
            await waiter

Response: Waits until the result set has changed.


Instruction:
Write a Python 3 script for
loading the next available job from the job iterator and increments the active job count.

Response:
def _load_job(self):
        """
        If there is still a job in the job iterator, loads it and increments
        the active job count.
        """

        try:
            next_job = next(self._jobs)
        except StopIteration:
            self._on_deck = None
        else:
            if not isinstance(next_job, Job):
                next_job = DefaultJob(next_job)
            self._on_deck = next_job
            self._active_jobs += 1


Instruction:
Given the following Python 3 function, write the documentation
def _done(self):

        self._results.complete()
        waiters = self._waiters
        for waiter in waiters:
            waiter.set_result(None)
        self._manager.job_set_done(self)

Response: Mark the job set as complete and notifies all waiting tasks.


Instruction:
Write a Python 3 function that can
get a job from the job set if one is available.

Response:
def get_job(self):
        """
        Gets a job from the job set if one is queued. The jobs_available()
        method should be consulted first to determine if a job can be obtained
        from a call to this method. If no jobs are available, an IndexError is
        raised.
        """

        if len(self._return_queue) > 0:
            return self._return_queue.popleft()
        elif self._on_deck is not None:
            job = self._on_deck
            self._load_job()
            return job
        else:
            raise IndexError("no jobs available")


Instruction:
Given the following Python 3 function, write the documentation
def add_result(self, result):

        if self._active_jobs == 0:
            return

        self._results.add(result)
        self._active_jobs -= 1
        if self._active_jobs == 0:
            self._done()

Response: Adds the result of a completed job to the result list and then decrements the active job count.


Instruction:
Can you tell what is the following Python 3 function doing
def cancel(self):

        if self._active_jobs == 0:
            return

        self._jobs = iter(())
        self._on_deck = None
        self._return_queue.clear()
        self._active_jobs = 0

        self._done()

Response: Cancels the job set.


Instruction:
Can you write a function in Python 3 where it
waits until all jobs in the job set are finished. Returns immediately if the job set is already finished.

Response:
async def wait_done(self):
        """
        Waits until the job set is finished. Returns immediately if the job set
        is already finished.
        """

        if self._active_jobs > 0:
            future = self._loop.create_future()
            self._waiters.append(future)
            await future


Instruction:
Can you write a function in Python 3 where it
distributes jobs from the active job set to any waiting get_job callbacks.

Response:
def _distribute_jobs(self):
        """
        Distributes jobs from the active job set to any waiting get_job
        callbacks.
        """

        while (self._active_js.job_available()
                and len(self._ready_callbacks) > 0):
            job = self._active_js.get_job()
            self._job_sources[job] = self._active_js
            callback = self._ready_callbacks.popleft()
            callback(job)


Instruction:
Can you create a Python 3 function that
adds a job set to the manager s queue.

Response:
def add_job_set(self, job_list):
        """
        Adds a job set to the manager's queue. If there is no job set running,
        it is activated immediately. A new job set handle is returned.
        """

        assert not self._closed

        results = Results(loop=self._loop)
        js = JobSet(job_list, results, self, loop=self._loop)
        if not js.is_done():
            if self._active_js is None:
                self._active_js = js
                logger.debug("activated job set")
                self._distribute_jobs()
            else:
                self._js_queue.append(js)
        else:
            logger.debug("new job set has no jobs")
        return JobSetHandle(js, results)


Instruction:
Explain what the following Python 3 code does
def get_job(self, callback):

        assert not self._closed

        if self._active_js is None or not self._active_js.job_available():
            self._ready_callbacks.append(callback)
        else:
            job = self._active_js.get_job()
            self._job_sources[job] = self._active_js
            callback(job)

Response: Calls the given callback function when a job becomes available.


Instruction:
Explain what the following Python 3 code does
def return_job(self, job):

        if self._closed:
            return

        js = self._job_sources[job]
        if len(self._ready_callbacks) > 0:
            callback = self._ready_callbacks.popleft()
            callback(job)
        else:
            del self._job_sources[job]
            js.return_job(job)

Response: Returns a job to its source job set to be run later.


Instruction:
Can you implement a function in Python 3 that
adds the result of a job to the results list of the job s source job set.

Response:
def add_result(self, job, result):
        """
        Adds the result of a job to the results list of the job's source job
        set.
        """

        if self._closed:
            return

        js = self._job_sources[job]
        del self._job_sources[job]
        js.add_result(result)


Instruction:
Make a summary of the following Python 3 code
def job_set_done(self, js):

        if self._closed:
            return

        if self._active_js != js:
            return

        try:
            while self._active_js.is_done():
                logger.debug("job set done")
                self._active_js = self._js_queue.popleft()
                logger.debug("activated job set")
        except IndexError:
            self._active_js = None
        else:
            self._distribute_jobs()

Response: Called when a job set has been completed or cancelled.


Instruction:
Can you tell what is the following Python 3 function doing
def close(self):

        if self._closed:
            return

        self._closed = True
        if self._active_js is not None:
            self._active_js.cancel()
        for js in self._js_queue:
            js.cancel()

Response: Closes the job manager.


Instruction:
Make a summary of the following Python 3 code
def _uniquify(_list):
    seen = set()
    result = []
    for x in _list:
        if x not in seen:
            result.append(x)
            seen.add(x)
    return result

Response: Remove duplicates in a list.


Instruction:
Can you write a function in Python 3 where it
returns True if the regex matches the object.

Response:
def _match_regex(regex, obj):
    """
    Returns true if the regex matches the object, or a string in the object
    if it is some sort of container.

    :param regex: A regex.
    :type regex: ``regex``
    :param obj: An arbitrary object.
    :type object: ``object``

    :rtype: ``bool``
    """
    if isinstance(obj, six.string_types):
        return len(regex.findall(obj)) > 0
    elif isinstance(obj, dict):
        return _match_regex(regex, obj.values())
    elif hasattr(obj, '__iter__'):
        # Object is a list or some other iterable.
        return any(_match_regex(regex, s)
                   for s in obj if isinstance(s, six.string_types))
    else:
        return False


Instruction:
Write a Python 3 script for
returning a list of host entries that match the given filters and exclude.

Response:
def get_entries(latest, filters, exclude, limit=None):
    """
    Lists all available instances.

    :param latest: If true, ignores the cache and grabs the latest list.
    :type latest: ``bool``
    :param filters: Filters to apply to results. A result will only be shown
                    if it includes all text in all filters.
    :type filters: [``str``]
    :param exclude: The opposite of filters. Results will be rejected if they
                    include any of these strings.
    :type exclude: [``str``]
    :param limit: Maximum number of entries to show (default no maximum).
    :type limit: ``int`` or ``NoneType``

    :return: A list of host entries.
    :rtype: ``list`` of :py:class:`HostEntry`
    """
    entry_list = _list_all_latest() if latest is True or not _is_valid_cache()\
                 else _list_all_cached()
    filtered = filter_entries(entry_list, filters, exclude)
    if limit is not None:
        return filtered[:limit]
    else:
        return filtered


Instruction:
Implement a function in Python 3 to
use the environment variable to get the current region

Response:
def get_region():
    """Use the environment to get the current region"""
    global _REGION
    if _REGION is None:
        region_name = os.getenv("AWS_DEFAULT_REGION") or "us-east-1"
        region_dict = {r.name: r for r in boto.regioninfo.get_regions("ec2")}
        if region_name not in region_dict:
            raise ValueError("No such EC2 region: {}. Check AWS_DEFAULT_REGION "
                             "environment variable".format(region_name))
        _REGION = region_dict[region_name]
    return _REGION


Instruction:
Write a Python 3 script for
returning if the cache is valid.

Response:
def _is_valid_cache():
    """
    Returns if the cache is valid (exists and modified within the interval).
    :return: Whether the cache is valid.
    :rtype: ``bool``
    """
    if not os.path.exists(get_cache_location()):
        return False
    modified = os.path.getmtime(get_cache_location())
    modified = time.ctime(modified)
    modified = datetime.strptime(modified, '%a %b %d %H:%M:%S %Y')
    return datetime.now() - modified <= CACHE_EXPIRATION_INTERVAL


Instruction:
Can you generate the documentation for the following Python 3 function
def _list_all_cached():
    with open(get_cache_location()) as f:
        contents = f.read()
        objects = json.loads(contents)
        return [HostEntry.from_dict(obj) for obj in objects]

Response: Reads the description cache and returns each instance s information.


Instruction:
Here you have a function in Python 3, explain what it does
def filter_entries(entries, filters, exclude):
    filtered = [entry
                for entry in entries
                if all(entry.matches(f) for f in filters)
                and not any(entry.matches(e) for e in exclude)]
    return filtered

Response: Filters a list of host entries according to the given filters.


Instruction:
Can you implement a function in Python 3 that
gets the public dns name of name if it exists.

Response:
def get_host(name):
    """
    Prints the public dns name of `name`, if it exists.

    :param name: The instance name.
    :type name: ``str``
    """
    f = {'instance-state-name': 'running', 'tag:Name': name}
    ec2 = boto.connect_ec2(region=get_region())
    rs = ec2.get_all_instances(filters=f)
    if len(rs) == 0:
        raise Exception('Host "%s" not found' % name)
    print(rs[0].instances[0].public_dns_name)


Instruction:
Explain what the following Python 3 code does
def from_dict(cls, entry_dict):
        return cls(
            name=entry_dict["name"],
            instance_type=entry_dict["instance_type"],
            hostname=entry_dict["hostname"],
            private_ip=entry_dict["private_ip"],
            public_ip=entry_dict["public_ip"],
            stack_name=entry_dict["stack_name"],
            stack_id=entry_dict["stack_id"],
            logical_id=entry_dict["logical_id"],
            security_groups=entry_dict["security_groups"],
            tags=entry_dict["tags"],
            ami_id=entry_dict["ami_id"],
            launch_time=entry_dict["launch_time"],
            instance_id=entry_dict["instance_id"]
        )

Response: Deserialize a HostEntry object from a dictionary.


Instruction:
Implement a function in Python 3 to
give an attribute name, looks it up on the entry. Names that start with ``tags.`` are looked up in the ``tags`` dictionary. :param attr: Name of attribute to look up. :type attr: ``str`` :param convert_to_str: Convert result to a string. :type convert_to_str: ``bool`` :rtype: ``object``

Response:
def _get_attrib(self, attr, convert_to_str=False):
        """
        Given an attribute name, looks it up on the entry. Names that
        start with ``tags.`` are looked up in the ``tags`` dictionary.

        :param attr: Name of attribute to look up.
        :type attr: ``str``
        :param convert_to_str: Convert result to a string.
        :type convert_to_str: ``bool``

        :rtype: ``object``
        """
        if attr.startswith('tags.'):
            tag = attr[len('tags.'):]
            if tag in self.tags and self.tags[tag] != '':
                return self.tags[tag]
            elif convert_to_str is True:
                return '<not set>'
            else:
                return self.tags.get(tag)
        elif not hasattr(self, attr):
            raise AttributeError('Invalid attribute: {0}. Perhaps you meant '
                                 '{1}?'.format(red(attr),
                                               green('tags.' + attr)))
        else:
            result = getattr(self, attr)
            if convert_to_str is True and not result:
                return '<none>'
            elif convert_to_str is True and isinstance(result, list):
                return ', '.join(result)
            elif convert_to_str is True:
                return str(result)
            else:
                return result


Instruction:
Implement a Python 3 function for
listing all of the attributes that are found on an instance of this class.

Response:
def list_attributes(cls):
        """
        Lists all of the attributes to be found on an instance of this class.
        It creates a "fake instance" by passing in `None` to all of the
        ``__init__`` arguments, then returns all of the attributes of that
        instance.

        :return: A list of instance attributes of this class.
        :rtype: ``list`` of ``str``
        """
        fake_args = [None for _ in inspect.getargspec(cls.__init__).args[1:]]
        fake_instance = cls(*fake_args)
        return vars(fake_instance).keys()


Instruction:
Write a Python 3 function that can
sort a list of entries by the given attribute.

Response:
def sort_by(cls, entries, attribute):
        """
        Sorts a list of entries by the given attribute.
        """
        def key(entry):
            return entry._get_attrib(attribute, convert_to_str=True)
        return sorted(entries, key=key)


Instruction:
Here you have a function in Python 3, explain what it does
def repr_as_line(self, additional_columns=None, only_show=None, sep=','):
        additional_columns = additional_columns or []
        if only_show is not None:
            columns = _uniquify(only_show)
        else:
            columns = _uniquify(self.DEFAULT_COLUMNS + additional_columns)
        to_display = [self._get_attrib(c, convert_to_str=True) for c in columns]
        return sep.join(to_display)

Response: Returns a string representation of the host as a single line.