Instruction:
Here you have a function in Python 3, explain what it does
def find_longest_match(self, alo, ahi, blo, bhi):
        besti, bestj, bestsize = _cdifflib.find_longest_match(self, alo, ahi, blo, bhi)
        return _Match(besti, bestj, bestsize)

Response: Find the longest matching block in a [ alo ahi bhi.


Instruction:
Make a summary of the following Python 3 code
def set_seq1(self, a):
        if a is self.a:
            return
        self.a = a
        if not isinstance(self.a, list):
            self.a = list(self.a)
        # Types must be hashable to work in the c layer.  This will raise if
        # list items are *not* hashable.
        [hash(x) for x in self.a]

Response:
Same as SequenceMatcher. set_seq1 but check for non - list inputs
        implementation.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _tostream(parser, obj, stream, skipprepack = False):
    if hasattr(parser, 'tostream'):
        return parser.tostream(obj, stream, skipprepack)
    else:
        data = parser.tobytes(obj, skipprepack)
        cls = type(parser)
        if cls not in _deprecated_parsers:
            _deprecated_parsers.add(cls)
            warnings.warn("Parser %r does not have 'tostream' interfaces" % (cls,), UserWarning)
        return stream.write(data)

Response: Internal helper function to write a Tostream object to a file - like object.


Instruction:
Can you write a function in Python 3 where it
converts bytes in a dump value to str allowing json encode

Response:
def _to_str(dumped_val, encoding='utf-8', ordered=True):
    """
    Convert bytes in a dump value to str, allowing json encode
    """
    _dict = OrderedDict if ordered else dict
    if isinstance(dumped_val, dict):
        return OrderedDict((k, _to_str(v, encoding)) for k,v in dumped_val.items())
    elif isinstance(dumped_val, (list, tuple)):
        return [_to_str(v, encoding) for v in dumped_val]
    elif isinstance(dumped_val, bytes):
        try:
            d = dumped_val.decode('utf-8')
        except Exception:
            d = repr(dumped_val)
        return d
    else:
        return dumped_val


Instruction:
Write a Python 3 function that can
dump a value into a JSON - friendly format.

Response:
def dump(val, humanread = True, dumpextra = False, typeinfo = DUMPTYPE_FLAT, ordered=True,
         tostr=False, encoding='utf-8'):
    '''
    Convert a parsed NamedStruct (probably with additional NamedStruct as fields) into a
    JSON-friendly format, with only Python primitives (dictionaries, lists, bytes, integers etc.)
    Then you may use json.dumps, or pprint to further process the result.

    :param val: parsed result, may contain NamedStruct

    :param humanread: if True (default), convert raw data into readable format with type-defined formatters.
            For example, enumerators are converted into names, IP addresses are converted into dotted formats, etc.

    :param dumpextra: if True, dump "extra" data in '_extra' field. False (default) to ignore them.

    :param typeinfo: Add struct type information in the dump result. May be the following values:

      DUMPTYPE_FLAT ('flat')
        add a field '_type' for the type information (default)

      DUMPTYPE_KEY ('key')
        convert the value to dictionary like: {'<struc_type>': value}

      DUMPTYPE_NONE ('none')
        do not add type information

    :param tostr: if True, convert all bytes to str

    :param encoding: if tostr=`True`, first try to decode bytes in `encoding`. If failed, use `repr()` instead.

    :returns: "dump" format of val, suitable for JSON-encode or print.
    '''
    dumped = _dump(val, humanread, dumpextra, typeinfo, ordered)
    if tostr:
        dumped = _to_str(dumped, encoding, ordered)
    return dumped


Instruction:
Can you create a Python 3 function that
returns a function which gets size from specified fields with limits.

Response:
def sizefromlen(limit, *properties):
    '''
    Factory to generate a function which get size from specified field with limits.
    Often used in nstruct "size" parameter.

    To retrieve size without limit, simply use lambda expression: lambda x: x.header.length

    :param limit: the maximum size limit, if the acquired value if larger then the limit, BadLenError is raised
            to protect against serious result like memory overflow or dead loop.

    :param properties: the name of the specified fields. Specify more than one string to form a property path,
            like: sizefromlen(256, 'header', 'length') -> s.header.length

    :returns: a function which takes a NamedStruct as parameter, and returns the length value from specified
            property path.
    '''
    def func(namedstruct):
        v = namedstruct._target
        for p in properties:
            v = getattr(v, p)
        if v > limit:
            raise BadLenError('Struct length exceeds limit ' + str(limit))
        return v
    return func


Instruction:
Can you generate a brief explanation for the following Python 3 code
def packsize(*properties):
    '''
    Revert to sizefromlen, store the struct size (len(struct)) to specified property path. The size includes
    padding. To store the size without padding, use packrealsize() instead. Often used in nstruct "prepack"
    parameter.

    :param properties: specified field name, same as sizefromlen.

    :returns: a function which takes a NamedStruct as parameter, and pack the size to specified field.

    '''
    def func(namedstruct):
        v = namedstruct._target
        for p in properties[:-1]:
            v = getattr(v, p)
        setattr(v, properties[-1], len(namedstruct))
    return func

Response: Returns a function that packs the size of a named struct to the specified property path.


Instruction:
Can you generate the documentation for the following Python 3 function
def packrealsize(*properties):
    '''
    Revert to sizefromlen, pack the struct real size (struct._realsize()) to specified property path.
    Unlike packsize, the size without padding is stored. Often used in nstruct "prepack" parameter.

    :param properties: specified field name, same as sizefromlen.

    :returns: a function which takes a NamedStruct as parameter, and pack the size to specified field.

    '''
    def func(namedstruct):
        v = namedstruct._target
        for p in properties[:-1]:
            v = getattr(v, p)
        setattr(v, properties[-1], namedstruct._realsize())
    return func

Response: Returns a function that packs the real size of the namedstruct to the specified property path.


Instruction:
Implement a function in Python 3 to
pack a value into a property path.

Response:
def packvalue(value, *properties):
    '''
    Store a specified value to specified property path. Often used in nstruct "init" parameter.

    :param value: a fixed value

    :param properties: specified field name, same as sizefromlen.

    :returns: a function which takes a NamedStruct as parameter, and store the value to property path.

    '''
    def func(namedstruct):
        v = namedstruct._target
        for p in properties[:-1]:
            v = getattr(v, p)
        setattr(v, properties[-1], value)
    return func


Instruction:
How would you implement a function in Python 3 that
stores a evaluated value to specified property path. Often used in nstruct "prepack" parameter. :param func: a function which takes a NamedStruct as parameter and returns a value, often a lambda expression :param properties: specified field name, same as sizefromlen. :returns: a function which takes a NamedStruct as parameter, and store the return value of func to property path.

Response:
def packexpr(func, *properties):
    '''
    Store a evaluated value to specified property path. Often used in nstruct "prepack" parameter.

    :param func: a function which takes a NamedStruct as parameter and returns a value, often a lambda expression

    :param properties: specified field name, same as sizefromlen.

    :returns: a function which takes a NamedStruct as parameter, and store the return value of func to property path.
    '''
    def func2(namedstruct):
        v = namedstruct._target
        for p in properties[:-1]:
            v = getattr(v, p)
        setattr(v, properties[-1], func(namedstruct))
    return func2


Instruction:
Can you create a Python 3 function that
creates indices for all the embedded structs. For parser internal use.

Response:
def _create_embedded_indices(self):
        '''
        Create indices for all the embedded structs. For parser internal use.
        '''
        try:
            _set(self, '_embedded_indices', dict((k,(self,v)) for k,v in getattr(self._parser.typedef, 'inline_names', {}).items()))
        except AttributeError:
            _set(self, '_embedded_indices', {})


Instruction:
Make a summary of the following Python 3 code
def _unpack(self, data):
        '''
        Unpack a struct from bytes. For parser internal use.
        '''
        #self._logger.log(logging.DEBUG, 'unpacking %r', self)
        current = self
        while current is not None:
            data = current._parser.unpack(data, current)
            last = current
            current = getattr(current, '_sub', None)
        _set(last, '_extra', data)

Response: Unpack a struct from bytes. For parser internal use.


Instruction:
How would you code a function in Python 3 to
pack the current structure into a stream. For parser internal use.

Response:
def _packto(self, stream):
        '''
        Pack current struct into stream. For parser internal use.

        :param stream: a buffered stream (File or BytesIO)

        :return: packed bytes length
        '''
        #self._logger.log(logging.DEBUG, 'packing %r', self)
        total_size = 0
        current = self
        while current is not None:
            total_size += current._parser.packto(current, stream)
            last = current
            current = getattr(current, '_sub', None)
        if hasattr(last, '_extra'):
            _extra = last._extra
            total_size += stream.write(_extra)
        return total_size


Instruction:
Create a Python 3 function to
convert the named struct to bytes.

Response:
def _tobytes(self, skipprepack = False):
        '''
        Convert the struct to bytes. This is the standard way to convert a NamedStruct to bytes.

        :param skipprepack: if True, the prepack stage is skipped. For parser internal use.

        :returns: converted bytes
        '''
        stream = BytesIO()
        self._tostream(stream, skipprepack)
        return stream.getvalue()


Instruction:
Create a Python 3 function to
convert the named struct into a bytes stream.

Response:
def _tostream(self, stream, skipprepack= False):
        '''
        Convert the struct into a bytes stream. This is the standard way to convert a NamedStruct to bytes.

        :param stream: a list of bytes to get the result

        :param skipprepack: if True, the prepack stage is skipped. For parser internal use.

        :returns: total appended size
        '''
        if not skipprepack:
            self._prepack()
        datasize = self._packto(stream)
        paddingSize = self._parser.paddingsize2(datasize)
        if paddingSize > datasize:
            stream.write(b'\x00' * (paddingSize - datasize))
        return paddingSize


Instruction:
How would you explain what the following Python 3 function does
def _realsize(self):
        '''
        Get the struct size without padding (or the "real size")

        :returns: the "real size" in bytes

        '''
        current = self
        size= 0
        while current is not None:
            size += current._parser.sizeof(current)
            last = current
            current = getattr(current, '_sub', None)
        size += len(getattr(last, '_extra', b''))
        return size

Response: Get the struct size without padding or the real size


Instruction:
Can you tell what is the following Python 3 function doing
def _subclass(self, parser):
        '''
        Create sub-classed struct from extra data, with specified parser. For parser internal use.

        :param parser: parser of subclass
        '''
        _set(self, '_sub', parser._create(memoryview(getattr(self, '_extra', b'')), self._target))
        try:
            object.__delattr__(self, '_extra')
        except:
            pass

Response: Create sub - classed struct from extra data with specified parser. For parser internal use.


Instruction:
Can you create a Python 3 function that
extends the base class with a new subclass.

Response:
def _extend(self, newsub):
        '''
        Append a subclass (extension) after the base class. For parser internal use.
        '''
        current = self
        while hasattr(current, '_sub'):
            current = current._sub
        _set(current, '_sub', newsub)
        try:
            object.__delattr__(self, '_extra')
        except:
            pass


Instruction:
Create a Python 3 function for
returning current type of this struct

Response:
def _gettype(self):
        '''
        Return current type of this struct

        :returns: a typedef object (e.g. nstruct)

        '''
        current = self
        lastname = getattr(current._parser, 'typedef', None)
        while hasattr(current, '_sub'):
            current = current._sub
            tn = getattr(current._parser, 'typedef', None)
            if tn is not None:
                lastname = tn
        return lastname


Instruction:
How would you implement a function in Python 3 that
sets the _extra field in the struct which stands for the additional data after the defined fields.

Response:
def _setextra(self, extradata):
        '''
        Set the _extra field in the struct, which stands for the additional ("extra") data after the
        defined fields.
        '''
        current = self
        while hasattr(current, '_sub'):
            current = current._sub
        _set(current, '_extra', extradata)


Instruction:
Implement a Python 3 function for
getting the extra data of this struct.

Response:
def _getextra(self):
        '''
        Get the extra data of this struct.
        '''
        current = self
        while hasattr(current, '_sub'):
            current = current._sub
        return getattr(current, '_extra', None)


Instruction:
Write a Python 3 function for
replacing the embedded struct with a newly - created struct of another type.

Response:
def _replace_embedded_type(self, name, newtype):
        '''
        Replace the embedded struct to a newly-created struct of another type (usually based on the
        original type). The attributes of the old struct is NOT preserved.

        :param name: either the original type, or the name of the original type. It is always the type
                     used in type definitions, even if it is already replaced once or more.

        :param newtype: the new type to replace
        '''
        if hasattr(name, 'readablename'):
            name = name.readablename
        t,i = self._target._embedded_indices[name]
        t._seqs[i] = newtype.parser().new(self._target)


Instruction:
Can you generate the documentation for the following Python 3 function
def _get_embedded(self, name):
        '''
        Return an embedded struct object to calculate the size or use _tobytes(True) to convert just the
        embedded parts.

        :param name: either the original type, or the name of the original type. It is always the type
                     used in type definitions, even if it is already replaced once or more.

        :returns: an embedded struct
        '''
        if hasattr(name, 'readablename'):
            name = name.readablename
        t,i = self._target._embedded_indices[name]
        return t._seqs[i]

Response: Returns an embedded struct object for the specified type.


Instruction:
Write a Python 3 script for
registering a type with the specified name.

Response:
def _registerPickleType(name, typedef):
        '''
        Register a type with the specified name. After registration, NamedStruct with this type
        (and any sub-types) can be successfully pickled and transfered.
        '''
        NamedStruct._pickleNames[typedef] = name
        NamedStruct._pickleTypes[name] = typedef


Instruction:
Can you write a function in Python 3 where it
creates indices for all the embedded structs. For parser internal use.

Response:
def _create_embedded_indices(self):
        '''
        Create indices for all the embedded structs. For parser internal use.
        '''
        try:
            self._target._embedded_indices.update(((k,(self,v)) for k,v in getattr(self._parser.typedef, 'inline_names', {}).items()))
        except AttributeError:
            pass


Instruction:
Can you write a function in Python 3 where it
tries to parse the named struct from bytes sequence.

Response:
def parse(self, buffer, inlineparent = None):
        '''
        Try to parse the struct from bytes sequence. The bytes sequence is taken from a streaming source.

        :param buffer: bytes sequence to be parsed from.

        :param inlineparent: if specified, this struct is embedded in another struct.

        :returns: None if the buffer does not have enough data for this struct (e.g. incomplete read
                from socket); (struct, size) else, where struct is the parsed result (usually a NamedStruct object)
                and size is the used bytes length, so you can start another parse from buffer[size:].
        '''
        if self.base is not None:
            return self.base.parse(buffer, inlineparent)
        r = self._parse(buffer, inlineparent)
        if r is None:
            return None
        (s, size) = r
        self.subclass(s)
        return (s, (size + self.padding - 1) // self.padding * self.padding)


Instruction:
Implement a function in Python 3 to
create an empty struct of this type.

Response:
def new(self, inlineparent = None):
        '''
        Create an empty struct of this type. "initfunc" is called on the created struct to initialize it.

        :param inlineparent: if specified, this struct is embedded into another struct "inlineparent"

        :returns: a created struct (usually a NamedStruct object)
        '''
        if self.base is not None:
            s = self.base.new(inlineparent)
            s._extend(self._new(s._target))
        else:
            s = self._new(inlineparent)
        if self.initfunc is not None:
            self.initfunc(s)
        return s


Instruction:
Can you tell what is the following Python 3 function doing
def create(self, data, inlineparent = None):
        '''
        Create a struct and use all bytes of data. Different from parse(), this takes all data,
        store unused bytes in "extra" data of the struct. Some types like variable-length array
        may have different parse result with create() and parse().

        :param data: bytes of a packed struct.

        :param inlineparent: if specified, this struct is embedded in another struct "inlineparent"

        :returns: a created NamedStruct object.
        '''
        if self.base is not None:
            return self.base.create(data, inlineparent)
        c = self._create(data, inlineparent)
        self.subclass(c)
        return c

Response: Create a new NamedStruct and use all bytes of data.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def paddingsize(self, namedstruct):
        '''
        Return the size of the padded struct (including the "real" size and the padding bytes)

        :param namedstruct: a NamedStruct object of this type.

        :returns: size including both data and padding.
        '''
        if self.base is not None:
            return self.base.paddingsize(namedstruct)
        realsize = namedstruct._realsize()
        return (realsize + self.padding - 1) // self.padding * self.padding

Response: Returns the size of the padded struct including the real size and the padding bytes.


Instruction:
Can you generate the documentation for the following Python 3 function
def paddingsize2(self, realsize):
        '''
        Return a padded size from realsize, for NamedStruct internal use.
        '''
        if self.base is not None:
            return self.base.paddingsize2(realsize)
        return (realsize + self.padding - 1) // self.padding * self.padding

Response: Return a padded size from realsize for NamedStruct internal use.


Instruction:
Write a Python 3 function that can
convert a NamedStruct to packed bytes append the bytes to the stream

Response:
def tostream(self, namedstruct, stream, skipprepack = False):
        '''
        Convert a NamedStruct to packed bytes, append the bytes to the stream

        :param namedstruct: a NamedStruct object of this type to pack.

        :param skipprepack: if True, the prepack stage is skipped.

        :param stream: a buffered stream

        :return: appended bytes size
        '''
        return namedstruct._tostream(stream, skipprepack)


Instruction:
Can you tell what is the following Python 3 function doing
def prepack(self, namedstruct, skip_self=False, skip_sub=False):
        '''
        Run prepack
        '''
        if not skip_self and self.prepackfunc is not None:
            self.prepackfunc(namedstruct)

Response: Run prepackfunc on the namedstruct.


Instruction:
Create a Python 3 function to
pack a named struct to a buffered stream

Response:
def packto(self, namedstruct, stream):
        """
        Pack a struct to a stream

        :param namedstruct: struct to pack

        :param stream: a buffered stream

        :return: appended bytes size
        """
        # Default implementation
        data = self.pack(namedstruct)
        return stream.write(data)


Instruction:
Implement a Python 3 function for
unpacking the struct from specified bytes.

Response:
def unpack(self, data, namedstruct):
        '''
        Unpack the struct from specified bytes. If the struct is sub-classed, definitions from the sub type
        is not unpacked.

        :param data: bytes of the struct, including fields of sub type and "extra" data.

        :param namedstruct: a NamedStruct object of this type

        :returns: unused bytes from data, which forms data of the sub type and "extra" data.
        '''
        try:
            result = self.struct.unpack(data[0:self.struct.size])
        except struct.error as exc:
            raise BadFormatError(exc)
        start = 0
        t = namedstruct._target
        for p in self.properties:
            if len(p) > 1:
                if isinstance(result[start], bytes):
                    v = [r.rstrip(b'\x00') for r in result[start:start + p[1]]]
                else:
                    v = list(result[start:start + p[1]])
                start += p[1]
            else:
                v = result[start]
                if isinstance(v, bytes):
                    v = v.rstrip(b'\x00')
                start += 1
            setin = t
            for sp in p[0][0:-1]:
                if not hasattr(setin, sp):
                    setin2 = InlineStruct(namedstruct._target)
                    setattr(setin, sp, setin2)
                    setin = setin2
                else:
                    setin = getattr(setin, sp)
            setattr(setin, p[0][-1], v)
        return data[self.struct.size:]


Instruction:
Can you implement a function in Python 3 that
packs the struct and return the packed bytes.

Response:
def pack(self, namedstruct):
        '''
        Pack the struct and return the packed bytes.

        :param namedstruct: a NamedStruct of this type.

        :returns: packed bytes, only contains fields of definitions in this type, not the sub type and "extra" data.
        '''
        elements = []
        t = namedstruct._target
        for p in self.properties:
            v = t
            for sp in p[0]:
                v = getattr(v, sp)
            if len(p) > 1:
                elements.extend(v[0:p[1]])
            else:
                elements.append(v)
        return self.struct.pack(*elements)


Instruction:
How would you explain what the following Python 3 function does
def parse(self, buffer, inlineparent = None):
        '''
        Compatible to Parser.parse()
        '''
        if len(buffer) < self.struct.size:
            return None
        try:
            return (self.struct.unpack(buffer[:self.struct.size])[0], self.struct.size)
        except struct.error as exc:
            raise BadFormatError(exc)

Response: Parses the ISO - 8601 tag from the given buffer.


Instruction:
Given the following Python 3 function, write the documentation
def parse(self, buffer, inlineparent = None):
        '''
        Compatible to Parser.parse()
        '''
        size = 0
        v = []
        for i in range(0, self.size):  # @UnusedVariable
            r = self.innerparser.parse(buffer[size:], None)
            if r is None:
                return None
            v.append(r[0])
            size += r[1]
        return (v, size)

Response: Compatibility to Parser. parse


Instruction:
How would you explain what the following Python 3 function does
def new(self, inlineparent = None):
        '''
        Compatible to Parser.new()
        '''
        v = list(range(0, self.size))
        for i in range(0, self.size):
            v[i] = self.innerparser.new()
        return v

Response: Compatibility to Parser. new


Instruction:
Can you generate a brief explanation for the following Python 3 code
def create(self, data, inlineparent = None):
        '''
        Compatible to Parser.create()
        '''
        if self.size > 0:
            r = self.parse(data)
            if r is None:
                raise ParseError('data is not enough to create an array of size ' + self.size)
            else:
                return r[0]
        else:
            v = []
            start = 0
            while start < len(data):
                r = self.innerparser.parse(data[start:], None)
                if r is None:
                    break
                v.append(r[0])
                start += r[1]
            return v

Response: Compatibility to Parser. create


Instruction:
Create a Python 3 function for
converting the object to bytes.

Response:
def tobytes(self, prim, skipprepack = False):
        '''
        Compatible to Parser.tobytes()
        '''
        stream = BytesIO()
        self.tostream(prim, stream, skipprepack=skipprepack)
        return stream.getvalue()


Instruction:
Given the following Python 3 function, write the documentation
def create(self, data, inlineparent = None):
        '''
        Compatible to Parser.create()
        '''
        if self.cstr:
            return _copy(data).rstrip(b'\x00')
        else:
            return _copy(data)

Response: Create a new instance of the class.


Instruction:
Create a Python 3 function for
getting the parser for this type. Create the parser on first call.

Response:
def parser(self):
        '''
        Get parser for this type. Create the parser on first call.
        '''
        if not hasattr(self, '_parser'):
            self._parser = self._compile()
        return self._parser


Instruction:
Create a Python 3 function for
creating a new object of this type with the specified arguments and keyword arguments set on the new object.

Response:
def new(self, *args, **kwargs):
        '''
        Create a new object of this type. It is also available as __call__, so you can create a new object
        just like creating a class instance: a = mytype(a=1,b=2)

        :param args: Replace the embedded struct type. Each argument is a tuple (name, newtype).
                     It is equivalent to call _replace_embedded_type with *name* and *newtype*
                     one by one. Both the "directly" embedded struct and the embedded struct inside another
                     embedded struct can be set. If you want to replace an embedded struct in a
                     replaced struct type, make sure the outer struct is replaced first. The embeded
                     struct type must have a *name* to be replaced by specify *name* option.

        :param kwargs: extra key-value arguments, each one will be set on the new object, to set value
                       to the fields conveniently.

        :returns: a new object, with the specified "kwargs" set.
        '''
        obj = self.parser().new()
        for k,v in args:
            obj._replace_embedded_type(k,v)
        for k,v in kwargs.items():
            setattr(obj, k, v)
        return obj


Instruction:
Can you generate a brief explanation for the following Python 3 code
def getName(self, value, defaultName = None):
        '''
        Get the enumerate name of a specified value.
        :param value: the enumerate value
        :param defaultName: returns if the enumerate value is not defined
        :returns: the corresponding enumerate value or *defaultName* if not found
        '''
        for k,v in self._values.items():
            if v == value:
                return k
        return defaultName

Response: Get the enumerate name of a specified value.


Instruction:
Write a Python 3 script to
import all the enumerate values from this enumerate to gs

Response:
def importAll(self, gs):
        '''
        Import all the enumerate values from this enumerate to *gs*
        :param gs: usually globals(), a dictionary. At lease __setitem__ should be implemented if not a dictionary.
        '''
        for k,v in self._values.items():
            gs[k] = v


Instruction:
Can you generate a brief explanation for the following Python 3 code
def extend(self, namespace = None, name = None, **kwargs):
        '''
        Create a new enumerate with current values merged with new enumerate values
        :param namespace: same as __init__
        :param name: same as __init__
        :param kwargs: same as __init__
        :returns: a new enumerate type
        '''
        if name is None:
            name = self._readablename
        d = dict(self._values)
        d.update(kwargs)
        return enum(name, namespace, self, self._bitwise, **d)

Response: Create a new enumerate with current values merged with new enumerate values


Instruction:
How would you code a function in Python 3 to
format a enumerate value to enumerate names if possible. Used to generate human readable a dump result.

Response:
def formatter(self, value):
        '''
        Format a enumerate value to enumerate names if possible. Used to generate human readable
        dump result.
        '''
        if not self._bitwise:
            n = self.getName(value)
            if n is None:
                return value
            else:
                return n
        else:
            names = []
            for k,v in sorted(self._values.items(), key=lambda x: x[1], reverse=True):
                if (v & value) == v:
                    names.append(k)
                    value = value ^ v
            names.reverse()
            if value != 0:
                names.append(hex(value))
            if not names:
                return 0
            return ' '.join(names)


Instruction:
Write a Python 3 script to
pack a struct to a stream

Response:
def packto(self, namedstruct, stream):
        """
        Pack a struct to a stream
        """
        if hasattr(namedstruct, self.name):
            return _tostream(self.basetypeparser, getattr(namedstruct, self.name), stream, True)
        else:
            return 0


Instruction:
How would you implement a function in Python 3 that
runs prepack on the namedstruct.

Response:
def prepack(self, namedstruct, skip_self=False, skip_sub=False):
        '''
        Run prepack
        '''
        if not skip_sub and hasattr(namedstruct, self.name) and hasattr(self.basetypeparser, 'fullprepack'):
            self.basetypeparser.fullprepack(getattr(namedstruct, self.name))
        Parser.prepack(self, namedstruct, skip_self, skip_sub)


Instruction:
Can you tell what is the following Python 3 function doing
def prepack(self, namedstruct, skip_self=False, skip_sub=False):
        '''
        Run prepack
        '''
        if not skip_sub and hasattr(self.innertypeparser, 'fullprepack'):
            for v in getattr(namedstruct, self.name):
                self.innertypeparser.fullprepack(v)
        Parser.prepack(self, namedstruct, skip_self, skip_sub)

Response: Run prepack on the namedstruct.


Instruction:
Can you generate the documentation for the following Python 3 function
def prepack(self, namedstruct, skip_self=False, skip_sub=False):
        '''
        Run prepack
        '''
        if not skip_sub and self.header is not None and hasattr(self.header, 'fullprepack'):
            self.header.fullprepack(namedstruct._seqs[0])
        Parser.prepack(self, namedstruct, skip_self, skip_sub)

Response: Run prepack on the namedstruct.


Instruction:
Explain what the following Python 3 code does
def _checkresponse(response):

        results_error = 'No results were found matching your query'
        auth_error = 'The token or API key is not valid, please contact Josh Clark at joshua.m.clark@utah.edu to ' \
                     'resolve this'
        rule_error = 'This request violates a rule of the API. Please check the guidelines for formatting a data ' \
                     'request and try again'
        catch_error = 'Something went wrong. Check all your calls and try again'

        if response['SUMMARY']['RESPONSE_CODE'] == 1:
            return response
        elif response['SUMMARY']['RESPONSE_CODE'] == 2:
            if response['SUMMARY']['NUMBER_OF_OBJECTS'] == 0:
                return None
            raise MesoPyError(results_error)
        elif response['SUMMARY']['RESPONSE_CODE'] == 200:
            raise MesoPyError(auth_error)
        elif response['SUMMARY']['RESPONSE_CODE'] == 400:
            raise MesoPyError(rule_error)
        elif response['SUMMARY']['RESPONSE_CODE'] == -1:
            format_error = response['SUMMARY']['RESPONSE_MESSAGE']
            raise MesoPyError(format_error)
        else:
            raise MesoPyError(catch_error)

Response: Checks the response from the API and returns the data requested by the other methods.


Instruction:
Given the following Python 3 function, write the documentation
def _get_response(self, endpoint, request_dict):
        http_error = 'Could not connect to the API. This could be because you have no internet connection, a parameter' \
                     ' was input incorrectly, or the API is currently down. Please try again.'

        json_error = 'Could not retrieve JSON values. Try again with a shorter date range.'

        # For python 3.4
        try:
            qsp = urllib.parse.urlencode(request_dict, doseq=True)
            resp = urllib.request.urlopen(self.base_url + endpoint + '?' + qsp).read()

        # For python 2.7
        except AttributeError or NameError:
            try:
                qsp = urllib.urlencode(request_dict, doseq=True)
                resp = urllib2.urlopen(self.base_url + endpoint + '?' + qsp).read()
            except urllib2.URLError:
                raise MesoPyError(http_error)
        except urllib.error.URLError:
            raise MesoPyError(http_error)

        try:
            json_data = json.loads(resp.decode('utf-8'))
        except ValueError:
            raise MesoPyError(json_error)

        return self._checkresponse(json_data)

Response: Returns a dictionary of data requested by each function.


Instruction:
Can you generate the documentation for the following Python 3 function
def _check_geo_param(self, arg_list):

        geo_func = lambda a, b: any(i in b for i in a)
        check = geo_func(self.geo_criteria, arg_list)
        if check is False:
            raise MesoPyError('No stations or geographic search criteria specified. Please provide one of the '
                              'following: stid, state, county, country, radius, bbox, cwa, nwsfirezone, gacc, subgacc')

Response: r Checks each function call to make sure that the user has provided at least one of the following geographic parameters.


Instruction:
Given the following Python 3 function, write the documentation
def timeseries(self, start, end, **kwargs):

        self._check_geo_param(kwargs)
        kwargs['start'] = start
        kwargs['end'] = end
        kwargs['token'] = self.token

        return self._get_response('stations/timeseries', kwargs)

Response: r Returns a time series of observations at a user specified location for a specified time.


Instruction:
Can you generate the documentation for the following Python 3 function
def climate_stats(self, startclim, endclim, type, **kwargs):

        self._check_geo_param(kwargs)
        kwargs['type'] = type
        kwargs['startclim'] = startclim
        kwargs['endclim'] = endclim
        kwargs['token'] = self.token

        return self._get_response('stations/climatology', kwargs)

Response: r Returns a dictionary of aggregated yearly climate statistics for a specified time series.


Instruction:
How would you explain what the following Python 3 function does
def metadata(self, **kwargs):

        self._check_geo_param(kwargs)
        kwargs['token'] = self.token

        return self._get_response('stations/metadata', kwargs)

Response: r Returns the metadata for a given MesoWest station or station.


Instruction:
Can you tell what is the following Python 3 function doing
def latency(self, start, end, **kwargs):

        self._check_geo_param(kwargs)
        kwargs['start'] = start
        kwargs['end'] = end
        kwargs['token'] = self.token

        return self._get_response('stations/latency', kwargs)

Response: r Returns the latency values for a given station based on a start and end date.


Instruction:
Can you tell what is the following Python 3 function doing
def avg(self, key=None):
        count = self.count()

        if count:
            return self.sum(key) / count

Response: Get the average value of a given key.


Instruction:
Can you generate the documentation for the following Python 3 function
def chunk(self, size):
        chunks = self._chunk(size)

        return self.__class__(list(map(self.__class__, chunks)))

Response: Returns a new collection with the given size.


Instruction:
Can you tell what is the following Python 3 function doing
def _chunk(self, size):
        items = self.items

        return [items[i:i + size] for i in range(0, len(items), size)]

Response: Returns a list of size items from the underlying collection.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def contains(self, key, value=None):
        if value is not None:
            return self.contains(lambda x: data_get(x, key) == value)

        if self._use_as_callable(key):
            return self.first(key) is not None

        return key in self.items

Response: Returns True if the element with the given key is in the collection False otherwise.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def collapse(self):
        results = []

        items = self.items

        for values in items:
            if isinstance(values, BaseCollection):
                values = values.all()

            results += values

        return self.__class__(results)

Response: Collapse the collection items into a single element


Instruction:
Can you tell what is the following Python 3 function doing
def each(self, callback):
        items = self.items

        for item in items:
            if callback(item) is False:
                break

        return self

Response: Execute a callback over each item in the collection.


Instruction:
How would you implement a function in Python 3 that
creates a new collection consisting of every n - th element.

Response:
def every(self, step, offset=0):
        """
        Create a new collection consisting of every n-th element.

        :param step: The step size
        :type step: int

        :param offset: The start offset
        :type offset: int

        :rtype: Collection
        """
        new = []

        for position, item in enumerate(self.items):
            if position % step == offset:
                new.append(item)

        return self.__class__(new)


Instruction:
Can you implement a function in Python 3 that
returns a new Collection with all items except the specified keys.

Response:
def without(self, *keys):
        """
        Get all items except for those with the specified keys.

        :param keys: The keys to remove
        :type keys: tuple

        :rtype: Collection
        """
        items = copy(self.items)

        keys = reversed(sorted(keys))

        for key in keys:
            del items[key]

        return self.__class__(items)


Instruction:
How would you explain what the following Python 3 function does
def only(self, *keys):
        items = []

        for key, value in enumerate(self.items):
            if key in keys:
                items.append(value)

        return self.__class__(items)

Response: Returns a new Collection with only the specified keys.


Instruction:
Can you generate the documentation for the following Python 3 function
def filter(self, callback=None):
        if callback:
            return self.__class__(list(filter(callback, self.items)))

        return self.__class__(list(filter(None, self.items)))

Response: Run a filter over each of the items.


Instruction:
Create a Python 3 function for
filtering items by the given key value pair.

Response:
def where(self, key, value):
        """
        Filter items by the given key value pair.

        :param key: The key to filter by
        :type key: str

        :param value: The value to filter by
        :type value: mixed

        :rtype: Collection
        """
        return self.filter(lambda item: data_get(item, key) == value)


Instruction:
Write a Python 3 function for
getting the first item of the collection.

Response:
def first(self, callback=None, default=None):
        """
        Get the first item of the collection.

        :param default: The default value
        :type default: mixed
        """
        if callback is not None:
            for val in self.items:
                if callback(val):
                    return val

            return value(default)

        if len(self.items) > 0:
            return self.items[0]
        else:
            return default


Instruction:
How would you implement a function in Python 3 that
gets a flattened list of the items in the collection.

Response:
def flatten(self):
        """
        Get a flattened list of the items in the collection.

        :rtype: Collection
        """

        def _flatten(d):
            if isinstance(d, dict):
                for v in d.values():
                    for nested_v in _flatten(v):
                        yield nested_v
            elif isinstance(d, list):
                for list_v in d:
                    for nested_v in _flatten(list_v):
                        yield nested_v
            else:
                yield d

        return self.__class__(list(_flatten(self.items)))


Instruction:
How would you explain what the following Python 3 function does
def forget(self, *keys):
        keys = reversed(sorted(keys))

        for key in keys:
            del self[key]

        return self

Response: Removes an item from the collection by the given keys.


Instruction:
Can you generate the documentation for the following Python 3 function
def get(self, key, default=None):
        try:
            return self.items[key]
        except IndexError:
            return value(default)

Response: Get an element of the collection.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def implode(self, value, glue=''):
        first = self.first()

        if not isinstance(first, (basestring)):
            return glue.join(self.pluck(value).all())

        return value.join(self.items)

Response: Returns the string representation of the key as a string.


Instruction:
Can you tell what is the following Python 3 function doing
def last(self, callback=None, default=None):
        if callback is not None:
            for val in reversed(self.items):
                if callback(val):
                    return val

            return value(default)

        if len(self.items) > 0:
            return self.items[-1]
        else:
            return default

Response: Get the last item of the collection.


Instruction:
Create a Python 3 function to
return a list with the values of a given key.

Response:
def pluck(self, value, key=None):
        """
        Get a list with the values of a given key.

        :rtype: Collection
        """
        if key:
            return dict(map(lambda x: (data_get(x, key), data_get(x, value)), self.items))
        else:
            results = list(map(lambda x: data_get(x, value), self.items))

        return self.__class__(results)


Instruction:
Can you create a Python 3 function that
runs a map over each of the items in the collection.

Response:
def map(self, callback):
        """
        Run a map over each of the item.

        :param callback: The map function
        :type callback: callable

        :rtype: Collection
        """
        return self.__class__(list(map(callback, self.items)))


Instruction:
Can you tell what is the following Python 3 function doing
def max(self, key=None):

        def _max(result, item):
            val = data_get(item, key)

            if result is None or val > result:
                return value

            return result

        return self.reduce(_max)

Response: Get the max value of a given key.


Instruction:
Here you have a function in Python 3, explain what it does
def min(self, key=None):

        def _min(result, item):
            val = data_get(item, key)

            if result is None or val < result:
                return value

            return result

        return self.reduce(_min)

Response: Get the minimum value of a given key.


Instruction:
Here you have a function in Python 3, explain what it does
def for_page(self, page, per_page):
        start = (page - 1) * per_page

        return self[start:start + per_page]

Response: Paginate the collection by slicing it into a smaller collection.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def pull(self, key, default=None):
        val = self.get(key, default)

        self.forget(key)

        return val

Response: Pulls an item from the collection.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def reject(self, callback):
        if self._use_as_callable(callback):
            return self.filter(lambda item: not callback(item))

        return self.filter(lambda item: item != callback)

Response: Create a collection of all elements that do not pass a given truth test.


Instruction:
Can you tell what is the following Python 3 function doing
def sort(self, callback=None):
        items = self.items

        if callback:
            return self.__class__(sorted(items, key=callback))
        else:
            return self.__class__(sorted(items))

Response: Sort through each item in the collection with a callback.


Instruction:
Write a Python 3 script to
get the sum of the given values.

Response:
def sum(self, callback=None):
        """
        Get the sum of the given values.

        :param callback: The callback
        :type callback: callable or string or None

        :rtype: mixed
        """
        if callback is None:
            return sum(self.items)

        callback = self._value_retriever(callback)

        return self.reduce(lambda result, item: (result or 0) + callback(item))


Instruction:
Explain what the following Python 3 code does
def unique(self, key=None):
        if key is None:
            seen = set()
            seen_add = seen.add

            return self.__class__([x for x in self.items if not (x in seen or seen_add(x))])

        key = self._value_retriever(key)

        exists = []

        def _check(item):
            id_ = key(item)
            if id_ in exists:
                return True

            exists.append(id_)

        return self.reject(_check)

Response: Return only unique items from the collection list.


Instruction:
Can you implement a function in Python 3 that
zips the collection together with one or more arrays.

Response:
def zip(self, *items):
        """
        Zip the collection together with one or more arrays.

        :param items: The items to zip
        :type items: list

        :rtype: Collection
        """
        return self.__class__(list(zip(self.items, *items)))


Instruction:
Given the following Python 3 function, write the documentation
def merge(self, items):
        if isinstance(items, BaseCollection):
            items = items.all()

        if not isinstance(items, list):
            raise ValueError('Unable to merge uncompatible types')

        self._items += items

        return self

Response: Merge the items with the given items.


Instruction:
Can you implement a function in Python 3 that
transforms each item in the collection using a callback.

Response:
def transform(self, callback):
        """
        Transform each item in the collection using a callback.

        :param callback: The callback
        :type callback: callable

        :rtype: Collection
        """
        self._items = self.map(callback).all()

        return self


Instruction:
How would you code a function in Python 3 to
get a value retrieving callback.

Response:
def _value_retriever(self, value):
        """
        Get a value retrieving callback.

        :type value: mixed

        :rtype: callable
        """
        if self._use_as_callable(value):
            return value

        return lambda item: data_get(item, value)


Instruction:
Can you generate the documentation for the following Python 3 function
def serialize(self):

        def _serialize(value):
            if hasattr(value, 'serialize'):
                return value.serialize()
            elif hasattr(value, 'to_dict'):
                return value.to_dict()
            else:
                return value

        return list(map(_serialize, self.items))

Response: Returns the collection of items as a serialized object.


Instruction:
Write a Python 3 function that can
step 4 ( 6 for invariant. Process contract ( validator )

Response:
def validate(self, *args, **kwargs):
        """
        Step 4 (6 for invariant). Process contract (validator)
        """
        # Schemes validation interface
        if is_scheme(self.validator):
            params = getcallargs(self.function, *args, **kwargs)
            params.update(kwargs)
            validator = self.validator(data=params, request=None)
            if validator.is_valid():
                return
            raise self.exception(validator.errors)

        # Simple validation interface
        if hasattr(self.validator, 'is_valid'):
            validator = self.validator(*args, **kwargs)
            # is valid
            if validator.is_valid():
                return
            # is invalid
            if hasattr(validator, 'errors'):
                raise self.exception(validator.errors)
            if hasattr(validator, '_errors'):
                raise self.exception(validator._errors)
            raise self.exception

        validation_result = self.validator(*args, **kwargs)
        # is invalid (validator return error message)
        if isinstance(validation_result, string_types):
            raise self.exception(validation_result)
        # is valid (truely result)
        if validation_result:
            return
        # is invalid (falsy result)
        raise self.exception


Instruction:
Can you generate the documentation for the following Python 3 function
def patched_function(self, *args, **kwargs):
        self.validate(*args, **kwargs)
        return self.function(*args, **kwargs)

Response: Step 3. Wrapped function calling.


Instruction:
Can you write a function in Python 3 where it
steps 3. Wrapped function calling.

Response:
def patched_function(self, *args, **kwargs):
        """
        Step 3. Wrapped function calling.
        """
        result = self.function(*args, **kwargs)
        self.validate(result)
        return result


Instruction:
Can you implement a function in Python 3 that
performs validation of the object.

Response:
def _validate(self):
        """
        Step 5 (1st flow) or Step 4 (2nd flow). Process contract for object.
        """
        # disable methods matching before validation
        self._disable_patching = True
        # validation by Invariant.validate
        self._validate_base(self)
        # enable methods matching after validation
        self._disable_patching = False


Instruction:
Can you generate the documentation for the following Python 3 function
def _patched_method(self, method, *args, **kwargs):
        self._validate()
        result = method(*args, **kwargs)
        self._validate()
        return result

Response: Patched method to validate the result of the method.


Instruction:
Explain what the following Python 3 code does
def buildvrt(input_file_list, output_file,
             relative=True, **kwargs):
    """Build a VRT

    See also: https://www.gdal.org/gdalbuildvrt.html

    You can find the possible BuildVRTOptions (**kwargs**) here:
    https://github.com/nextgis/pygdal/blob/78a793057d2162c292af4f6b240e19da5d5e52e2/2.1.0/osgeo/gdal.py#L1051

    Arguments:
        input_file_list {list of str or Path objects} -- List of input files.
        output_file {str or Path object} -- Output file (VRT).

    Keyword Arguments:
        relative {bool} -- If ``True``, the ``input_file_list`` paths are converted to relative
            paths (relative to the output file) and the VRT works even if the data is moved somewhere else -
            given that the relative location of theVRT and the input files does not chance!
        **kwargs {} -- BuildVRTOptions - see function description for a link to .

    Returns:
        [int] -- If successful, 0 is returned as exit code.
    """

    # create destination directory
    if not Path(output_file).parent.exists():
        Path(output_file).parent.mkdir(parents=True, exist_ok=True)

    # make sure we have absolute paths and strings since BuildVRT does not like something else
    input_file_list = [str(Path(p).absolute()) for p in input_file_list]
    output_file = str(Path(output_file).absolute())

    vrt_options = gdal.BuildVRTOptions(**kwargs)

    vrt = gdal.BuildVRT(output_file,
                        input_file_list,
                        options=vrt_options)
    vrt = None

    # if needed, create the input file paths relative to the output vrt path
    # and replace them in the vrt.
    # if desired, fix the paths and the relativeToVRT tag in the VRT
    if relative:
        input_file_list_relative = [relpath(p, Path(output_file).parent) for p in input_file_list]

        with open(output_file, 'r') as file:
            # read a list of lines into data
            lines = file.readlines()

        new_lines = []
        counter = -1
        for line in lines:
            # sometimes it is relative by default
            # maybe when all files contain the parent directory of the output file (?)
            if "relativeToVRT=\"1\"" in line:
                counter += 1
            elif "relativeToVRT=\"0\"" in line:
                counter += 1
                input_file = str(input_file_list[counter])
                input_file_relative = str(input_file_list_relative[counter])
                if input_file not in line:
                    raise Exception(f"Expect path {input_file} not part of line {line}.")
                line = line.replace(input_file,
                                    input_file_relative)
                line = line.replace("relativeToVRT=\"0\"",
                                    "relativeToVRT=\"1\"")
            else:
                pass
            new_lines.append(line)

        with open(output_file, 'w') as file:
            file.writelines(new_lines)
    return 0

Response: Builds a VRT from a list of input files and outputs a new VRT.


Instruction:
Here you have a function in Python 3, explain what it does
def reproject_on_template_raster(src_file, dst_file, template_file, resampling="near", compress=None, overwrite=False):

    if not overwrite and Path(dst_file).exists():
        print("Processing skipped. Destination file exists.")
        return 0

    GDAL_RESAMPLING_ALGORITHMS = {
        "bilinear": "GRA_Bilinear",
        "cubic": "GRA_Cubic",
        "cubicspline": "GRA_CubicSpline",
        "lanczos": "GRA_Lanczos",
        "average": "GRA_Average",
        "mode": "GRA_Mode",
        "max": "GRA_Max",
        "min": "GRA_Min",
        "med": "GRA_Med",
        "near": "GRA_NearestNeighbour",
        "q1": "GRA_Q1",
        "q3": "GRA_Q3"
    }

    compressions = ["lzw", "packbits", "deflate"]

    if resampling not in GDAL_RESAMPLING_ALGORITHMS.keys():
        raise ValueError(f"'resampling must be one of {', '.join(GDAL_RESAMPLING_ALGORITHMS.keys())}")

    if compress is None:
        options = []
    else:
        if compress.lower() not in compressions:
            raise ValueError(f"'compress must be one of {', '.join(compressions)}")
        else:
            options = [f'COMPRESS={compress.upper()}']

    # Source
    src = gdal.Open(src_file, gdalconst.GA_ReadOnly)
    src_band = src.GetRasterBand(1)
    src_proj = src.GetProjection()

    # We want a section of source that matches this:
    match_ds = gdal.Open(template_file, gdalconst.GA_ReadOnly)
    match_proj = match_ds.GetProjection()
    match_geotrans = match_ds.GetGeoTransform()
    wide = match_ds.RasterXSize
    high = match_ds.RasterYSize

    # Output / destination
    Path(dst_file).parent.mkdir(parents=True, exist_ok=True)
    dst = gdal.GetDriverByName('GTiff').Create(dst_file, wide, high, 1, src_band.DataType, options=options)
    dst.SetGeoTransform( match_geotrans )
    dst.SetProjection( match_proj)

    # Do the work
    gdal.ReprojectImage(src, dst, src_proj, match_proj,
                        getattr(gdalconst, GDAL_RESAMPLING_ALGORITHMS[resampling]))

    del dst # Flush
    return 0

Response: Reproject a one - band raster to fit the projection extend pixel size etc. of a template raster.


Instruction:
Given the following Python 3 function, write the documentation
def rasterize(src_vector: str,
              burn_attribute: str,
              src_raster_template: str,
              dst_rasterized: str,
              gdal_dtype: int = 4):
    """Rasterize the values of a spatial vector file.

    Arguments:
        src_vector {str}} -- A OGR vector file (e.g. GeoPackage, ESRI Shapefile) path containing the
            data to be rasterized.
        burn_attribute {str} -- The attribute of the vector data to be burned in the raster.
        src_raster_template {str} -- Path to a GDAL raster file to be used as template for the
            rasterized data.
        dst_rasterized {str} -- Path of the destination file.
        gdal_dtype {int} -- Numeric GDAL data type, defaults to 4 which is UInt32.
            See https://github.com/mapbox/rasterio/blob/master/rasterio/dtypes.py for useful look-up
            tables.
    Returns:
        None
    """

    data = gdal.Open(str(src_raster_template),  # str for the case that a Path instance arrives here
                     gdalconst.GA_ReadOnly)
    geo_transform = data.GetGeoTransform()
    #source_layer = data.GetLayer()
    # x_max = x_min + geo_transform[1] * data.RasterXSize
    # y_min = y_max + geo_transform[5] * data.RasterYSize
    x_res = data.RasterXSize
    y_res = data.RasterYSize
    mb_v = ogr.Open(src_vector)
    mb_l = mb_v.GetLayer()
    target_ds = gdal.GetDriverByName('GTiff').Create(dst_rasterized,
                                                     x_res, y_res, 1,
                                                     gdal_dtype)  # gdal.GDT_Byte
    # import osr
    target_ds.SetGeoTransform((geo_transform[0],  # x_min
                               geo_transform[1],  # pixel_width
                               0,
                               geo_transform[3],  # y_max
                               0,
                               geo_transform[5]  # pixel_height
                               ))
    prj = data.GetProjection()
    # srs = osr.SpatialReference(wkt=prj)  # Where was this needed?
    target_ds.SetProjection(prj)
    band = target_ds.GetRasterBand(1)
    # NoData_value = 0
    # band.SetNoDataValue(NoData_value)
    band.FlushCache()
    gdal.RasterizeLayer(target_ds, [1], mb_l, options=[f"ATTRIBUTE={burn_attribute}"])

    target_ds = None

Response: Rasterize the values of a spatial vector file.


Instruction:
Can you tell what is the following Python 3 function doing
def calc_distance_to_border(polygons, template_raster, dst_raster, overwrite=False,
                            keep_interim_files=False):
    """Calculate the distance of each raster cell (in and outside the polygons) to the next polygon border.

    Arguments:
        polygons {str} -- Filename to a geopandas-readable file with polygon features.
        template_raster {[type]} -- Filename to a rasterio-readable file.
        dst_raster {[type]} -- Destination filename for the distance to polygon border raster file (tif).

    Keyword Arguments:
        overwrite {bool} -- Overwrite files if they exists? (default: {False})
        keep_interim_files {bool} -- Keep the interim line vector and raster files (default: {True})

    Returns:
        [type] -- [description]
    """
    if Path(dst_raster).exists() and not overwrite:
        print(f"Returning 0 - File exists: {dst_raster}")
        return 0

    with rasterio.open(template_raster) as tmp:
        crs = tmp.crs

    dst_raster = Path(dst_raster)
    dst_raster.parent.mkdir(exist_ok=True, parents=True)

    tempdir = Path(tempfile.mkdtemp(prefix=f"TEMPDIR_{dst_raster.stem}_", dir=dst_raster.parent))
    interim_file_lines_vector = tempdir / "interim_sample_vector_dataset_lines.shp"
    interim_file_lines_raster = tempdir / "interim_sample_vector_dataset_lines.tif"

    exit_code = convert_polygons_to_lines(polygons,
                                          interim_file_lines_vector,
                                          crs=crs,
                                          add_allone_col=True)

    rasterize(src_vector=str(interim_file_lines_vector),
              burn_attribute="ALLONE",
              src_raster_template=str(template_raster),
              dst_rasterized=str(interim_file_lines_raster),
              gdal_dtype=1)

    cmd = f"{PROXIMITY_PATH} " \
          f"{str(Path(interim_file_lines_raster).absolute())} " \
          f"{str(Path(dst_raster).absolute())} " \
          f"-ot Float32 -distunits PIXEL -values 1 -maxdist 255"
    subprocess.check_call(cmd, shell=True)

    if not keep_interim_files:
        shutil.rmtree(tempdir)
    else:
        print(f"Interim files are in {tempdir}")
    return 0

Response: Calculate the distance of each raster cell to the next polygon border.


Instruction:
Here you have a function in Python 3, explain what it does
def convert_polygons_to_lines(src_polygons, dst_lines, crs=None, add_allone_col=False):
    gdf = gpd.read_file(src_polygons)
    geom_coords = gdf["geometry"] # featureset.get(5)["geometry"]["coordinates"]
    lines = []
    row_ids = []
    for i_row, pol in tqdm(enumerate(geom_coords), total=len(geom_coords)):
        boundary = pol.boundary
        if boundary.type == 'MultiLineString':
            for line in boundary:
                lines.append(line)
                row_ids.append(i_row)
        else:
            lines.append(boundary)
            row_ids.append(i_row)

    gdf_lines = gdf.drop("geometry", axis=1).iloc[row_ids, :]
    gdf_lines["Coordinates"] = lines
    gdf_lines = gpd.GeoDataFrame(gdf_lines, geometry='Coordinates', crs=gdf.crs)
    if crs is not None:
        gdf_lines = gdf_lines.to_crs(crs)
    if add_allone_col:
        gdf_lines["ALLONE"] = 1
    Path(dst_lines).parent.mkdir(exist_ok=True, parents=True)
    gdf_lines.to_file(dst_lines)
    return 0

Response: Convert polygons to lines.


Instruction:
Make a summary of the following Python 3 code
def dtype_checker_df(df, dtype, return_=None):
    dtype_range = dtype_ranges[dtype]
    df_out_of_range = (df < dtype_range[0]) | (df > dtype_range[1]) | (~np.isfinite(df))
    if df_out_of_range.any().any():
        if return_== "colsums":
            df_out_of_range = df_out_of_range.apply(sum, axis=0) # column
        elif return_== "rowsums":
            df_out_of_range = df_out_of_range.apply(sum, axis=1) # row
        elif return_== "all":
            df_out_of_range = df_out_of_range
        else:
            df_out_of_range = 1
    else:
        df_out_of_range = 0
    return df_out_of_range

Response: Checks if the values of a given datatype are out of range.


Instruction:
Write a Python 3 function for
getting positions of rows matching specific band and date.

Response:
def get_df_ilocs(self, band, date):
        """Get positions of rows matching specific band(s) and date(s).

        The method supports three typical queries:

        * one band and one date (both given as strings)

        * one band and of several dates (band given as strings, date as list of strings)

        * several band and of one date (date given as strings, band as list of strings)

        Arguments:
            band {str or list} -- Band(s) for which to derive the iloc index.
            date {str or list} -- Date(s) for which to derive the iloc index.

        Returns:
            int or list -- Integer (if band and date are str) or list of iloc indices.
        """

        df = self.df_layers.copy()
        df["index"] = range(df.shape[0])

        idx_layers = []
        if isinstance(band, str) and isinstance(date, str):
            idx_layers = df[(df["date"] == date) &  (df["band"] == band)]["index"].values[0]
        if isinstance(band, list) and isinstance(date, str):
            for b in band:
                idx = df[(df["date"] == date) &  (df["band"] == b)]["index"].values[0]
                idx_layers.append(idx)
        elif isinstance(band, str) and isinstance(date, list):
            for d in date:
                idx = df[(df["band"] == band) &  (df["date"] == d)]["index"].values[0]
                idx_layers.append(idx)
        return idx_layers


Instruction:
How would you code a function in Python 3 to
get the spatial bounds of the chunk.

Response:
def _get_spatial_bounds(self):
        """Get the spatial bounds of the chunk."""
        # This should be a MultiRasterIO method
        with rasterio.open(self._mrio._get_template_for_given_resolution(self._mrio.dst_res, "path")) as src_layer:
            pass # later we need src_layer for src_layer.window_transform(win)
        win_transform = src_layer.window_transform(self._window)
        bounds = rasterio.windows.bounds(window=self._window,
                                         transform=win_transform,
                                         height=0, width=0)
        return bounds


Instruction:
Make a summary of the following Python 3 code
def convert_data_to_ndarray(self):
        if self._data_structure != "DataFrame":
            raise Exception(f"Data is not a DataFrame but {self._data_structure}.")
        self._data = self._convert_to_ndarray(self._data)
        self._update_data_structure()
        return self

Response: Converts the data from dataframe to ndarray format.


Instruction:
Here you have a function in Python 3, explain what it does
def _convert_to_ndarray(self, data):
        if data.__class__.__name__ != "DataFrame":
            raise Exception(f"data is not a DataFrame but {data.__class__.__name__}.")
        shape_ndarray = (self._height, self._width, data.shape[1])
        data_ndarray = data.values.reshape(shape_ndarray)
        return data_ndarray

Response: Converts data from dataframe to ndarray format.


Instruction:
Implement a Python 3 function for
writing results to disc.

Response:
def write_dataframe(self, result, dst_paths, nodata=None, compress='lzw'):
        """Write results (dataframe) to disc."""
        result = self._convert_to_ndarray(result)
        self.write_ndarray(result, dst_paths, nodata=nodata, compress=compress)


Instruction:
Can you tell what is the following Python 3 function doing
def write_ndarray(self, result, dst_paths, nodata=None, compress='lzw'):

        assert len(dst_paths) == result.shape[2]
        assert result.shape[0] == self._height
        assert result.shape[1] == self._width
        assert result.shape[2] == len(dst_paths)
        with rasterio.open(self._mrio._get_template_for_given_resolution(self._mrio.dst_res, "path")) as src_layer:
            pass # later we need src_layer for src_layer.window_transform(win)
        for i, pth in enumerate(dst_paths):
            dst_path_chunk = self.get_chunk_path_from_layer_path(pth, self.ji)

            result_layer_i = result[:, :, [i]]
            assert result_layer_i.shape[2] == 1
            kwargs = self._mrio._get_template_for_given_resolution(
                res=self._mrio.dst_res, return_="meta").copy()
            kwargs.update({"driver": "GTiff",
                           "compress": compress,
                           "nodata": nodata,
                           "height": self._height,
                           "width": self._width,
                           "dtype": result_layer_i.dtype,
                           "transform": src_layer.window_transform(self._window)})
            with rasterio.open(dst_path_chunk, "w", **kwargs) as dst:
                dst.write(result_layer_i[:, :, 0], 1)

Response: Write the result to disc.


Instruction:
Write a Python 3 script to
get a robust data range i. e. 2nd and 98th percentile for vmin vmax parameters.

Response:
def robust_data_range(arr, robust=False, vmin=None, vmax=None):
        """Get a robust data range, i.e. 2nd and 98th percentile for vmin, vmax parameters."""
        # from the seaborn code
        # https://github.com/mwaskom/seaborn/blob/3a3ec75befab52c02650c62772a90f8c23046038/seaborn/matrix.py#L201

        def _get_vmin_vmax(arr2d, vmin=None, vmax=None):
            if vmin is None:
                vmin = np.percentile(arr2d, 2) if robust else arr2d.min()
            if vmax is None:
                vmax = np.percentile(arr2d, 98) if robust else arr2d.max()
            return vmin, vmax

        if len(arr.shape) == 3 and vmin is None and vmax is None:
            vmin = []
            vmax = []
            for i in range(arr.shape[2]):
                arr_i = arr[:, :, i]
                vmin_i, vmax_i = _get_vmin_vmax(arr_i, vmin=None, vmax=None)
                vmin.append(vmin_i)
                vmax.append(vmax_i)
        else:
            vmin, vmax = _get_vmin_vmax(arr, vmin=vmin, vmax=vmax)
        return vmin, vmax


Instruction:
Write a Python 3 script to
create a EOCubeChunk object from an EOCube object.

Response:
def from_eocube(eocube, ji):
        """Create a EOCubeChunk object from an EOCube object."""
        eocubewin = EOCubeChunk(ji, eocube.df_layers, eocube.chunksize, eocube.wdir)
        return eocubewin


Instruction:
Create a Python 3 function to
get a EOCubeChunk object

Response:
def get_chunk(self, ji):
        """Get a EOCubeChunk"""
        return EOCubeSceneCollectionChunk(ji=ji,
                                          df_layers=self.df_layers,
                                          chunksize=self.chunksize,
                                          variables=self.variables,
                                          qa=self.qa,
                                          qa_valid=self.qa_valid,
                                          wdir=self.wdir)


Instruction:
Write a Python 3 script to
read and masks the data and converts it in one dataframe per variable.

Response:
def read_data_by_variable(self, mask=True):
        """Reads and masks (if desired) the data and converts it in one dataframe per variable."""
        def print_elapsed_time(start, last_stopped, prefix):
            # print(f"{prefix} - Elapsed time [s] since start / last stopped: \
            #     {(int(time.time() - start_time))} / {(int(time.time() - last_stopped))}")
            return time.time()
        start_time = time.time()
        last_stopped = time.time()
        last_stopped = print_elapsed_time(start_time, last_stopped, "Starting chunk function")

        verbose = False

        self.read_data()
        last_stopped = print_elapsed_time(start_time, last_stopped, "Data read")

        # 2.
        sc_chunk = self.convert_data_to_dataframe()
        last_stopped = print_elapsed_time(start_time, last_stopped, "Data converted to df")

        # 3.B.
        if mask:
            # 3.A.
            ilocs_qa = np.where((self.df_layers["band"] == self.qa).values)[0]
            df_qa = self.data.iloc[:, ilocs_qa]
            df_qa.columns = self.df_layers["date"].iloc[ilocs_qa]
            df_clearsky = df_qa.isin(self.qa_valid)
            last_stopped = print_elapsed_time(start_time, last_stopped, "Clearsky df created")

            return_bands = self.variables
        else:
            return_bands = self.variables + [self.qa]

        dfs_variables = {}
        for var in return_bands:
            if verbose:
                print("VARIABLE:", var)
            ilocs_var = np.where((self.df_layers["band"] == var).values)[0]
            df_var = self.data.iloc[:, ilocs_var]
            df_var.columns = self.df_layers["date"].iloc[ilocs_var]
            if mask:
                df_var = df_var.where(df_clearsky, other=np.nan)
            dfs_variables[var] = df_var
        last_stopped = print_elapsed_time(start_time, last_stopped, "Clearsky df created")
        self._data = dfs_variables
        return self


Instruction:
How would you code a function in Python 3 to
get a specific sampledata for a specific dataset.

Response:
def get_dataset(dataset="s2l1c"):
    """Get a specific sampledata to play around.

    So far the following sampledata exist:

    * 's2l1c': One Sentinel-2 Level 1C scene with a reference dataset.
    * 'lsts': A time series of 105 Landsat scenes each with the bands b3 (red), b4 (nir), b5 (swir1) and fmask.

    Keyword Arguments:
        dataset {str} -- The name of the dataset (default: {'s2l1c'}).

    Returns:
        [dict] -- A dictionary with paths and information about the sampledata.
    """
    if dataset == "s2l1c":
        search_string = os.path.join(DIR_DATA, dataset, "**", "*_B??.jp2")
        files = glob.glob(search_string, recursive=True)
        if not files:
            raise IOError(f"Could not find raster files of the s2l1c dataset. Search string: {search_string}")
        basename_splitted = [pth.replace(".jp2", "").split("_")[-2:] for pth in files]
        dset = {"raster_files": files,
                "raster_bands": [ele[1] for ele in basename_splitted],
                "raster_times": [ele[0] for ele in basename_splitted],
                "vector_file": os.path.join(DIR_DATA, "s2l1c", "s2l1c_ref.gpkg"),
                "vector_file_osm": os.path.join(DIR_DATA, "s2l1c", "gis_osm_landuse-water_a_free_1_area-10000-to-500000.gpkg")}

    elif dataset == "lsts":
        search_string = os.path.join(DIR_DATA, dataset, "**", "*.tif")
        files = glob.glob(search_string, recursive=True)
        if not files:
            raise IOError(f"Could not find raster files of the lsts dataset. Search string: {search_string}")
        basename_splitted = [os.path.basename(pth).replace(".tif", "").split("_") for pth in files]
        dset = {"raster_files": files,
                "raster_bands": [ele[1] for ele in basename_splitted],
                "raster_times": [ele[0][9:16] for ele in basename_splitted]}

    # If you want to add a new dataset here, do not forget to do all of the following steps:
    # 1) add the dataset in the eo-box/sampledata/eobox/sampledata/data/<name of new dataset>
    # 2) write the code here to get the paths of the data and eventually some additional information
    # 3) write a test to make sure you get the data
    # 4) add the new dataset to package_data in eo-box/sampledata/eobox/setup.py
    # 5) add the new dataset to package_data in eo-box/sampledata/MANIFEST.in
    # 4) change the version number in eo-box/sampledata/eobox/sampledata/__init__.py to '<current>.<current+1>.0'

    return dset


Instruction:
Can you implement a function in Python 3 that
creates rasterio. windows. Window instances with given size which fully cover a raster.

Response:
def windows_from_blocksize(blocksize_xy, width, height):
    """Create rasterio.windows.Window instances with given size which fully cover a raster.

    Arguments:
        blocksize_xy {int or list of two int} -- [description]
        width {int} -- With of the raster for which to create the windows.
        height {int} -- Heigth of the raster for which to create the windows.

    Returns:
        list -- List of windows according to the following format
            ``[[<row-index>, <column index>], rasterio.windows.Window(<col_off>, <row_off>, <width>, <height>)]``.
    """

    # checks the blocksize input
    value_error_msg = "'blocksize must be an integer or a list of two integers.'"
    if isinstance(blocksize_xy, int):
        blockxsize, blockysize = (blocksize_xy, blocksize_xy)
    elif isinstance(blocksize_xy, list):
        if len(blocksize_xy) != 2:
            raise ValueError(value_error_msg)
        else:
            if not all([isinstance(blocksize_xy[0], int), isinstance(blocksize_xy[1], int)]):
                raise ValueError(value_error_msg)
            blockxsize, blockysize = blocksize_xy
    else:
        raise ValueError(value_error_msg)

    # create the col_off and row_off elements for all windows
    n_cols = int(np.ceil(width / blockxsize))
    n_rows = int(np.ceil(height / blockysize))
    col = list(range(n_cols)) * n_rows
    col_off = np.array(col) * blockxsize
    row = np.repeat(list(range(n_rows)), n_cols)
    row_off = row * blockysize

    # create the windows
    # if necessary, reduce the width and/or height of the border windows
    blocksize_wins = []
    for ridx, roff, cidx, coff, in zip(row, row_off, col, col_off):
        if coff + blockxsize > width:
            bxsize = width - coff
        else:
            bxsize = blockxsize
        if roff + blockysize > height:
            bysize = height - roff
        else:
            bysize = blockysize
        blocksize_wins.append([[ridx, cidx], rasterio.windows.Window(coff, roff, bxsize, bysize)])
    return blocksize_wins


Instruction:
How would you explain what the following Python 3 function does
def _get_dst_resolution(self, dst_res=None):
        if dst_res is None:
            dst_res = min(self._res_indices.keys())
        return dst_res

Response: Get default resolution i. e. the highest resolution or smallest cell size.


Instruction:
Can you implement a function in Python 3 that
loads windows for chunks - wise processing from raster internal tiling.

Response:
def block_windows(self, res=None):  # setter and getter ?
        """Load windows for chunks-wise processing from raster internal tiling (first raster of given resolution).

        Arguments:
            res {numeric} -- Resolution determining the raster (1st of resolution group) from which to take the tiling.
        """

        if res is None:
            res = max(self._res_indices.keys())
        self._windows_res = res
        a_file_index_given_res = self._res_indices[res][0]
        with rasterio.open(self._layer_files[a_file_index_given_res]) as src:
            wins_of_first_dst_res_layer = tuple(src.block_windows())
        self.windows = np.array([win[1] for win in wins_of_first_dst_res_layer])
        self.windows_row = np.array([win[0][0] for win in wins_of_first_dst_res_layer])
        self.windows_col = np.array([win[0][1] for win in wins_of_first_dst_res_layer])


Instruction:
Create a Python 3 function to
create rasterio. windows. Window instances with given size which fully cover the raster.

Response:
def windows_from_blocksize(self, blocksize_xy=512):
        """Create rasterio.windows.Window instances with given size which fully cover the raster.

        Arguments:
            blocksize_xy {int or list of two int} -- Size of the window. If one integer is given it defines
                the width and height of the window. If a list of two integers if given the first defines the
                width and the second the height.

        Returns:
            None -- But the attributes ``windows``, ``windows_row`` and ``windows_col`` are updated.
        """
        meta = self._get_template_for_given_resolution(self.dst_res, "meta")
        width = meta["width"]
        height = meta["height"]
        blocksize_wins = windows_from_blocksize(blocksize_xy, width, height)

        self.windows = np.array([win[1] for win in blocksize_wins])
        self.windows_row = np.array([win[0][0] for win in blocksize_wins])
        self.windows_col = np.array([win[0][1] for win in blocksize_wins])
        return self


Instruction:
Here you have a function in Python 3, explain what it does
def _get_template_for_given_resolution(self, res, return_):
        path = self._layer_files[self._res_indices[res][0]]
        if return_ == "path":
            return_value = path
        else:
            with rasterio.open(str(path)) as src:
                if return_ == "meta":
                    return_value = src.meta
                elif return_ == "windows":
                    return_value = tuple(src.block_windows())
                else:
                    raise ValueError("'return_' must be 'path', meta' or 'windows'.")
        return return_value

Response: Given specified resolution ('res') return template layer path meta or windows.


Instruction:
Create a Python 3 function for
getting Windows ( W ) W - row W - col and W - index of windows e. g. loaded with block_windows as a dataframe.

Response:
def windows_df(self):
        """Get Windows (W) W-row, W-col and W-index of windows e.g. loaded with :meth:`block_windows` as a dataframe.

        Returns:
            [dataframe] -- A dataframe with the window information and indices (row, col, index).
        """

        import pandas as pd
        if self.windows is None:
            raise Exception("You need to call the block_windows or windows before.")
        df_wins = []
        for row, col, win in zip(self.windows_row, self.windows_col, self.windows):
            df_wins.append(pd.DataFrame({"row":[row], "col":[col], "Window":[win]}))
        df_wins = pd.concat(df_wins).set_index(["row", "col"])
        df_wins["window_index"] = range(df_wins.shape[0])
        df_wins = df_wins.sort_index()
        return df_wins


Instruction:
How would you explain what the following Python 3 function does
def ji_windows(self, ij_win):  # what can be given to ij_win NOT intuitive/right name by now!!!
        ji_windows = {}
        transform_src = self._layer_meta[self._res_indices[self._windows_res][0]]["transform"]
        for res in self._res_indices:
            transform_dst = self._layer_meta[self._res_indices[res][0]]["transform"]
            ji_windows[res] = window_from_window(window_src=self.windows[ij_win],
                                                 transform_src=transform_src,
                                                 transform_dst=transform_dst)
        return ji_windows

Response: Returns a dictionary of all windows for a given window.


Instruction:
Here you have a function in Python 3, explain what it does
def get_arrays(self, ji_win):
        if isinstance(ji_win, dict):
            ji_windows = ji_win
        else:
            ji_windows = self.ji_windows(ji_win)

        arrays = []
        for filename, res in zip(self._layer_files, self._layer_resolution):
            with rasterio.open(filename) as src:
                arr = src.read(1, window=ji_windows[res])
            arrays.append(arr)
        if self.dst_res is not None:
            arrays = self._resample(arrays=arrays, ji_windows=ji_windows)
        return arrays

Response: Get the data of the a window given the ji_windows derived with ji_windows.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _resample(self, arrays, ji_windows):
        # get a destination array template
        win_dst = ji_windows[self.dst_res]
        aff_dst = self._layer_meta[self._res_indices[self.dst_res][0]]["transform"]
        arrays_dst = list()
        for i, array in enumerate(arrays):
            arr_dst = np.zeros((int(win_dst.height), int(win_dst.width)))
            if self._layer_resolution[i] > self.dst_res:
                resampling = getattr(Resampling, self.upsampler)
            elif self._layer_resolution[i] < self.dst_res:
                resampling = getattr(Resampling, self.downsampler)
            else:
                arrays_dst.append(array.copy())
                continue
            reproject(array, arr_dst,  # arr_dst[0, :, :, i],
                      src_transform=self._layer_meta[i]["transform"],
                      dst_transform=aff_dst,
                      src_crs=self._layer_meta[0]["crs"],
                      dst_crs=self._layer_meta[0]["crs"],
                      resampling=resampling)
            arrays_dst.append(arr_dst.copy())
        arrays_dst = np.stack(arrays_dst, axis=2) # n_images x n x m x 10 would be the synergise format
        return arrays_dst

Response: Resample all arrays with potentially different resolutions to a common resolution.


Instruction:
Here you have a function in Python 3, explain what it does
def _process_windows_merge_stack(self, func, **kwargs):
        ji_results = self._process_windows(func, **kwargs)
        for idx_layer in range(len(ji_results[0])):  # this is the number of output layers
            for j in np.unique(self.windows_row):
                win_indices_j = np.where(self.windows_row == j)[0]
                layer_merged_j = np.hstack([ji_results[idx][idx_layer] for idx in win_indices_j])
                if j == 0:
                    layer_merged = layer_merged_j
                else:
                    layer_merged = np.vstack([layer_merged, layer_merged_j])
            if idx_layer == 0:
                layers_merged = layer_merged
            else:
                layers_merged = np.stack([layers_merged, layer_merged], axis=2)
        return layers_merged

Response: Load ( resampled ) array of all windows apply custom function on it merge and stack results to one array.


Instruction:
Implement a Python 3 function for
loading ( resampled ) array of all windows and apply custom function on it.

Response:
def _process_windows(self, func, **kwargs):
        """Load (resampled) array of all windows and apply custom function on it."""
        ji_results = []
        for ji_win in range(len(self.windows)):
            ji_results.append(self._process_window(ji_win, func, **kwargs))
        return ji_results


Instruction:
Implement a function in Python 3 to
process a window and apply custom function on it.

Response:
def _process_window(self, ji_win, func, **kwargs):
        """Load (resampled) array of window ji_win and apply custom function on it. """
        arr = self.get_arrays(ji_win)
        result = func(arr, **kwargs)
        return result


Instruction:
How would you explain what the following Python 3 function does
def get_window_from_xy(self, xy):
        a_transform = self._get_template_for_given_resolution(res=self.dst_res, return_="meta")["transform"]
        row, col = transform.rowcol(a_transform, xy[0], xy[1])
        ij_containing_xy = None
        for ji, win in enumerate(self.windows):
            (row_start, row_end), (col_start, col_end) = rasterio.windows.toranges(win)
            # print(row, col, row_start, row_end, col_start, col_end)
            if ((col >= col_start) & (col < col_end)) & ((row >= row_start) & (row < row_end)):
                ij_containing_xy = ji
                break
        if ij_containing_xy is None:
            raise ValueError("The given 'xy' value is not contained in any window.")
        return ij_containing_xy

Response: Get the window index given a coordinate ( raster CRS ).


Instruction:
Implement a Python 3 function for
extracting values from a single band raster file for pixels overlapping with a vector dataset.

Response:
def extract(src_vector: str,
            burn_attribute: str,
            src_raster: list,
            dst_names: list,
            dst_dir: str,
            src_raster_template: str = None,
            gdal_dtype: int = 4,
            n_jobs: int = 1):
    """Extract values from list of single band raster for pixels overlapping with a vector data.

    The extracted data will be stored in the ``dst_dir`` by using the ``dst_names`` for the
    filename. If a file with a given name already exists the raster will be skipped.

    Arguments:
        src_vector {str} -- Filename of the vector dataset. Currently it must have the same CRS as
            the raster.
        burn_attribute {str} -- Name of the attribute column in the ``src_vector`` dataset to be
            stored with the extracted data. This should usually be a unique ID for the features
            (points, lines, polygons) in the vector dataset.
        src_raster {list} -- List of filenames of the single band raster files from which to
            extract.
        dst_names {list} -- List corresponding to ``src_raster`` names used to store and later
            identify the extracted to.
        dst_dir {str} -- Directory to store the data to.

    Keyword Arguments:
        src_raster_template {str} -- A template raster to be used for rasterizing the vectorfile.
            Usually the first element of ``src_raster``. (default: {None})
        gdal_dtype {int} -- Numeric GDAL data type, defaults to 4 which is UInt32.
            See https://github.com/mapbox/rasterio/blob/master/rasterio/dtypes.py for useful look-up
            tables.

    Returns:
        [int] -- If successful, 0 is returned as exit code.
    """
    if src_raster_template is None:
        src_raster_template = src_raster[0]
    path_rasterized = os.path.join(dst_dir, f"burn_attribute_rasterized_{burn_attribute}.tif")
    paths_extracted_aux = {ele: os.path.join(dst_dir, f"{ele}.npy") \
                           for ele in [f"aux_vector_{burn_attribute}",
                                       "aux_coord_x",
                                       "aux_coord_y"]}
    paths_extracted_raster = {}
    for path, name in zip(src_raster, dst_names):
        dst = f"{os.path.join(dst_dir, name)}.npy"
        if not os.path.exists(dst):
            paths_extracted_raster[path] = dst

    if not os.path.exists(dst_dir):
        os.makedirs(dst_dir)
    # if it does not already exist, here we first create the rasterized data
    if not os.path.exists(path_rasterized):
        if src_raster_template is None:
            src_raster_template = src_raster[0]
        # print("Rasterizing vector attribute.")
        rasterize(src_vector=src_vector,
                  burn_attribute=burn_attribute,
                  src_raster_template=src_raster_template,
                  dst_rasterized=path_rasterized,
                  gdal_dtype=gdal_dtype)

    # if any of the destination files do not exist we need the locations of the pixels to be
    # extracted in form of a numpy array bool (mask_arr) that fits the rasters from which we will
    # extract below
    if not (all([os.path.exists(path) for path in paths_extracted_aux.values()]) and \
            all([os.path.exists(path) for path in paths_extracted_raster.values()])):
        # print("Creating mask array for pixels to be extracted.")
        mask_arr = _get_mask_array(path_rasterized, paths_extracted_aux, burn_attribute)
    else:
        return 0

    # create the pixel coordinates if they do not exist
    if not all([os.path.exists(paths_extracted_aux["aux_coord_x"]),
                os.path.exists(paths_extracted_aux["aux_coord_y"])]):
        _create_and_save_coords(path_rasterized, paths_extracted_aux, mask_arr)

    # lets extract the raster values in case of sequential processing
    # or remove existing raster layers to prepare parallel processing
    if n_jobs == 1:
        for path_src, path_dst in tqdm(paths_extracted_raster.items(),
                                       total=len(paths_extracted_raster)):
            _extract_and_save_one_layer(path_src, path_dst, mask_arr)
    else:
        import multiprocessing as mp
        if n_jobs == -1:
            n_jobs = mp.cpu_count()
        pool = mp.Pool(processes=n_jobs)
        _ = [pool.apply_async(_extract_and_save_one_layer,
                              args=(src, dst, mask_arr)) for \
                                  src, dst in paths_extracted_raster.items()]
        pool.close()
        pool.join()
    return 0


Instruction:
Can you implement a function in Python 3 that
loads data extracted and stored by extract.

Response:
def load_extracted(src_dir: str,
                   patterns="*.npy",
                   vars_in_cols: bool = True,
                   index: pd.Series = None):
    """Load data extracted and stored by :py:func:`extract`

    Arguments:
        src_dir {str} -- The directory where the data is stored.

    Keyword Arguments:
        patterns {str, or list of str} -- A pattern (str) or list of patterns (list)
            to identify the variables to be loaded.
            The default loads all variables, i.e. all .npy files. (default: {'*.npy'})
        vars_in_cols {bool} -- Return the variables in columns (``True``) or rows ``False``
            (default: {True})
        index {pd.Series} -- A boolean pandas Series which indicates with ``True`` which samples to
            load.

    Returns:
        pandas.DataFrame -- A dataframe with the data.
    """
    def _load(path, index):
        if index is None:
            arr = np.load(str(path))
        else:
            arr = np.load(str(path), mmap_mode="r")[index]
        return arr

    src_dir = Path(src_dir)
    paths = []
    if isinstance(patterns, str):
        patterns = [patterns]
    for pat in patterns:
        paths += src_dir.glob(pat)

    if vars_in_cols:
        df_data = {}
        for path in paths:
            df_data[path.stem] = _load(path, index)
            df_data = pd.DataFrame(df_data)
            if index is not None:
                df_data.index = index.index[index]
    else:
        df_data = []
        for path in paths:
            arr = _load(path, index)
            df_data.append(pd.DataFrame(np.expand_dims(arr, 0), index=[path.stem]))
        df_data = pd.concat(df_data)
        if index is not None:
            df_data.columns = index.index[index]
    return df_data


Instruction:
Can you generate the documentation for the following Python 3 function
def extrema(self, x0, y0, w, h):
        minimum = 9223372036854775807
        maximum = 0
        for y in range(y0, y0 + h):
            for x in range(x0, x0 + w):
                value = self[x, y]
                if value != self.filler:
                    minimum = min(minimum, value)
                    maximum = max(maximum, value)
        return minimum, maximum

Response: Returns the minimum and maximum values contained in a given area.


Instruction:
Explain what the following Python 3 code does
def rgb2short(rgb):
    incs = (0x00, 0x5f, 0x87, 0xaf, 0xd7, 0xff)
    # Break 6-char RGB code into 3 integer vals.
    parts = [ int(h, 16) for h in re.split(r'(..)(..)(..)', rgb)[1:4] ]
    res = []
    for part in parts:
        i = 0
        while i < len(incs)-1:
            s, b = incs[i], incs[i+1]  # smaller, bigger
            if s <= part <= b:
                s1 = abs(s - part)
                b1 = abs(b - part)
                if s1 < b1: closest = s
                else: closest = b
                res.append(closest)
                break
            i += 1
    #print '***', res
    res = ''.join([ ('%02.x' % i) for i in res ])
    equiv = RGB2SHORT_DICT[ res ]
    #print '***', res, equiv
    return equiv, res

Response: Find the closest xterm - 256 approximation to the given RGB value.


Instruction:
Can you create a Python 3 function that
sets the cursor to the desired position.

Response:
def set_cursor(self, x, y):
        """
        Sets the cursor to the desired position.

        :param x: X position
        :param y: Y position
        """
        curses.curs_set(1)
        self.screen.move(y, x)


Instruction:
Can you generate the documentation for the following Python 3 function
def put(self, x, y, text, fg, bg):
        if x < self.width and y < self.height:
            try:
                self.screen.addstr(y, x, symbols.encode(text), self.pairs[fg, bg])
            except curses.error:
                # Ignore out of bounds error
                pass

Response: Writes a string at the desired coordinates using the provided colors.


Instruction:
Write a Python 3 function that can
wait for an event to happen and returns a string related to the event.

Response:
def poll_event(self):
        """
        Waits for an event to happen and returns a string related to the event.

        If the event is a normal (letter) key press, the letter is returned (case sensitive)

        :return: Event type
        """
        # Flush all inputs before this one that were done since last poll
        curses.flushinp()

        ch = self.screen.getch()

        if ch == 27:
            return EVENT_ESC
        elif ch == -1 or ch == curses.KEY_RESIZE:
            return EVENT_RESIZE
        elif ch == 10 or ch == curses.KEY_ENTER:
            return EVENT_ENTER
        elif ch == 127 or ch == curses.KEY_BACKSPACE:
            return EVENT_BACKSPACE
        elif ch == curses.KEY_UP:
            return EVENT_UP
        elif ch == curses.KEY_DOWN:
            return EVENT_DOWN
        elif ch == curses.KEY_LEFT:
            return EVENT_LEFT
        elif ch == curses.KEY_RIGHT:
            return EVENT_RIGHT
        elif ch == 3:
            return EVENT_CTRL_C
        elif 0 <= ch < 256:
            return chr(ch)
        else:
            return EVENT_UNHANDLED


Instruction:
Write a Python 3 function for
computing the capture parameters.

Response:
def compute_capture(args):
    x, y, w, h, params = args
    """Callable function for the multiprocessing pool."""
    return x, y, mandelbrot_capture(x, y, w, h, params)