Instruction:
Here you have a function in Python 3, explain what it does
async def run_task(self) -> None:
        '''Initialize the queue and spawn extra worker tasks if this if the
        first task.  Then wait for work items to enter the task queue, and
        execute the `run()` method with the current work item.'''

        while self.running:
            try:
                item = self.QUEUE.get_nowait()

                Log.debug('%s processing work item', self.name)
                await self.run(item)

                Log.debug('%s completed work item', self.name)
                self.QUEUE.task_done()

            except asyncio.QueueEmpty:
                if self.OPEN:
                    await self.sleep(0.05)

                else:
                    Log.debug('%s queue closed and empty, stopping', self.name)
                    return

            except CancelledError:
                Log.debug('%s cancelled, dropping work item')
                self.QUEUE.task_done()
                raise

            except Exception:
                Log.exception('%s failed work item', self.name)
                self.QUEUE.task_done()

Response: Initialize the queue and spawn extra worker tasks.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def schedule_saved(sender, instance, **kwargs):
    from contentstore.tasks import sync_schedule

    sync_schedule.delay(str(instance.id))

Response: Called when a new object is saved in the scheduler.


Instruction:
Implement a Python 3 function for
deactivating the celery task that ensures that this schedule is deactivated

Response:
def schedule_deleted(sender, instance, **kwargs):
    """
    Fires off the celery task to ensure that this schedule is deactivated

    Arguments:
        sender {class} -- The model class, always Schedule
        instance {Schedule} --
            The instance of the schedule that we want to deactivate
    """
    from contentstore.tasks import deactivate_schedule

    deactivate_schedule.delay(str(instance.scheduler_schedule_id))


Instruction:
Can you write a function in Python 3 where it
executes the task inside the asyncio event loop.

Response:
async def run_task(self) -> None:
        '''Execute the task inside the asyncio event loop.  Track the time it
        takes to run, and log when it starts/stops.  After `INTERVAL` seconds,
        if/once the task has finished running, run it again until `stop()`
        is called.'''

        while self.running:
            try:
                Log.debug('executing periodic task %s', self.name)
                before = self.time()
                await self.run()
                total = self.time() - before
                Log.debug('finished periodic task %s in %.1f seconds',
                          self.name, total)

                sleep = self.INTERVAL - total
                if sleep > 0:
                    await self.sleep(sleep)

            except CancelledError:
                Log.debug('cancelled periodic task %s', self.name)
                raise

            except Exception:
                Log.exception('exception in periodic task %s', self.name)


Instruction:
Given the following Python 3 function, write the documentation
def group_by(keys, values=None, reduction=None, axis=0):
    g = GroupBy(keys, axis)
    if values is None:
        return g
    groups = g.split(values)
    if reduction is None:
        return g.unique, groups
    return [(key, reduction(group)) for key, group in zip(g.unique, groups)]

Response: construct a GroupBy object on the given keys optionally performing the given reduction on the given values


Instruction:
Explain what the following Python 3 code does
def split_iterable_as_iterable(self, values):
        values = iter(enumerate(values))
        cache = dict()
        def get_value(ti):
            try:
                return cache.pop(ti)
            except:
                while True:
                    i, v = next(values)
                    if i==ti:
                        return v
                    cache[i] = v
        s = iter(self.index.sorter)
        for c in self.count:
            yield (get_value(i) for i in itertools.islice(s, int(c)))

Response:
Splits an iterable into lists of items in the order of the keys
           .


Instruction:
How would you code a function in Python 3 to
split an iterable into key - group tuples without regard for the ordering of self. index. unique .

Response:
def split_iterable_as_unordered_iterable(self, values):
        """Group iterable into iterables, without regard for the ordering of self.index.unique
        key-group tuples are yielded as soon as they are complete

        Parameters
        ----------
        values : iterable of length equal to keys
            iterable of values to be grouped

        Yields
        ------
        tuple of key, and a list of corresponding items in values

        Notes
        -----
        This approach is lazy, insofar as grouped values are close in their iterable
        """
        from collections import defaultdict
        cache = defaultdict(list)
        count = self.count
        unique = self.unique
        key = (lambda i: unique[i]) if isinstance(unique, np.ndarray) else (lambda i: tuple(c[i] for c in unique))
        for i,v in zip(self.inverse, values):
            cache[i].append(v)
            if len(cache[i]) == count[i]:
                yield key(i), cache.pop(i)


Instruction:
Make a summary of the following Python 3 code
def split_sequence_as_iterable(self, values):
        print(self.count)
        s = iter(self.index.sorter)
        for c in self.count:
            yield (values[i] for i in itertools.islice(s, int(c)))

Response:
Splits the given values into iterables of items in the same length as the keys
           .


Instruction:
Create a Python 3 function for
splitting the array into a single array by means of reshaping

Response:
def split_array_as_array(self, values):
        """Group ndarray into ndarray by means of reshaping

        Parameters
        ----------
        values : ndarray_like, [index.size, ...]

        Returns
        -------
        ndarray, [groups, group_size, ...]
            values grouped by key

        Raises
        ------
        AssertionError
            This operation is only possible if index.uniform==True
        """
        if not self.index.uniform:
            raise ValueError("Array can only be split as array if all groups have the same size")
        values = np.asarray(values)
        values = values[self.index.sorter]
        return values.reshape(self.groups, -1, *values.shape[1:])


Instruction:
Here you have a function in Python 3, explain what it does
def split_array_as_list(self, values):
        values = np.asarray(values)
        values = values[self.index.sorter]
        return np.split(values, self.index.slices[1:-1], axis=0)

Response: Group values as a list of arrays or a jagged - array


Instruction:
Write a Python 3 function for
reducing the values over identical key groups using the given ufunc operator over the first axis.

Response:
def reduce(self, values, operator=np.add, axis=0, dtype=None):
        """Reduce the values over identical key groups, using the given ufunc
        reduction is over the first axis, which should have elements corresponding to the keys
        all other axes are treated indepenently for the sake of this reduction

        Parameters
        ----------
        values : ndarray, [keys, ...]
            values to perform reduction over
        operator : numpy.ufunc
            a numpy ufunc, such as np.add or np.sum
        axis : int, optional
            the axis to reduce over
        dtype : output dtype

        Returns
        -------
        ndarray, [groups, ...]
        values reduced by operator over the key-groups
        """
        values = np.take(values, self.index.sorter, axis=axis)
        return operator.reduceat(values, self.index.start, axis=axis, dtype=dtype)


Instruction:
Here you have a function in Python 3, explain what it does
def sum(self, values, axis=0, dtype=None):
        values = np.asarray(values)
        return self.unique, self.reduce(values, axis=axis, dtype=dtype)

Response: compute the sum over each group


Instruction:
Can you implement a function in Python 3 that
computes the product over each group

Response:
def prod(self, values, axis=0, dtype=None):
        """compute the product over each group

        Parameters
        ----------
        values : array_like, [keys, ...]
            values to multiply per group
        axis : int, optional
            alternative reduction axis for values
        dtype : output dtype

        Returns
        -------
        unique: ndarray, [groups]
            unique keys
        reduced : ndarray, [groups, ...]
            value array, reduced over groups
        """
        values = np.asarray(values)
        return self.unique, self.reduce(values, axis=axis, dtype=dtype, operator=np.multiply)


Instruction:
Can you generate the documentation for the following Python 3 function
def mean(self, values, axis=0, weights=None, dtype=None):
        values = np.asarray(values)
        if weights is None:
            result = self.reduce(values, axis=axis, dtype=dtype)
            shape = [1] * values.ndim
            shape[axis] = self.groups
            weights = self.count.reshape(shape)
        else:
            weights = np.asarray(weights)
            result = self.reduce(values * weights, axis=axis, dtype=dtype)
            weights = self.reduce(weights, axis=axis, dtype=dtype)
        return self.unique, result / weights

Response: compute the mean over each group


Instruction:
Can you tell what is the following Python 3 function doing
def var(self, values, axis=0, weights=None, dtype=None):
        values = np.asarray(values)
        unique, mean = self.mean(values, axis, weights, dtype)
        err = values - mean.take(self.inverse, axis)

        if weights is None:
            shape = [1] * values.ndim
            shape[axis] = self.groups
            group_weights = self.count.reshape(shape)
            var = self.reduce(err ** 2, axis=axis, dtype=dtype)
        else:
            weights = np.asarray(weights)
            group_weights = self.reduce(weights, axis=axis, dtype=dtype)
            var = self.reduce(weights * err ** 2, axis=axis, dtype=dtype)

        return unique, var / group_weights

Response: compute the variance over each group


Instruction:
Write a Python 3 script to
return the standard deviation over each group

Response:
def std(self, values, axis=0, weights=None, dtype=None):
        """standard deviation over each group

        Parameters
        ----------
        values : array_like, [keys, ...]
            values to take standard deviation of per group
        axis : int, optional
            alternative reduction axis for values

        Returns
        -------
        unique: ndarray, [groups]
            unique keys
        reduced : ndarray, [groups, ...]
            value array, reduced over groups
        """
        unique, var = self.var(values, axis, weights, dtype)
        return unique, np.sqrt(var)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def median(self, values, axis=0, average=True):
        mid_2 = self.index.start + self.index.stop
        hi = (mid_2    ) // 2
        lo = (mid_2 - 1) // 2

        #need this indirection for lex-index compatibility
        sorted_group_rank_per_key = self.index.sorted_group_rank_per_key

        def median1d(slc):
            #place values at correct keys; preconditions the upcoming lexsort
            slc    = slc[self.index.sorter]
            #refine value sorting within each keygroup
            sorter = np.lexsort((slc, sorted_group_rank_per_key))
            slc    = slc[sorter]
            return (slc[lo]+slc[hi]) / 2 if average else slc[hi]

        values = np.asarray(values)
        if values.ndim>1:   #is trying to skip apply_along_axis somewhat premature optimization?
            values = np.apply_along_axis(median1d, axis, values)
        else:
            values = median1d(values)
        return self.unique, values

Response: compute the median value over each group


Instruction:
How would you explain what the following Python 3 function does
def mode(self, values, weights=None):
        if weights is None:
            unique, weights = npi.count((self.index.sorted_group_rank_per_key, values))
        else:
            unique, weights = npi.group_by((self.index.sorted_group_rank_per_key, values)).sum(weights)

        x, bin = npi.group_by(unique[0]).argmax(weights)
        return x, unique[1][bin]

Response: compute the mode of each entry in the set


Instruction:
Can you generate the documentation for the following Python 3 function
def min(self, values, axis=0):
        values = np.asarray(values)
        return self.unique, self.reduce(values, np.minimum, axis)

Response: return the minimum of unique entries within each group


Instruction:
How would you code a function in Python 3 to
return the maximum within each group

Response:
def max(self, values, axis=0):
        """return the maximum within each group

        Parameters
        ----------
        values : array_like, [keys, ...]
            values to take maximum of per group
        axis : int, optional
            alternative reduction axis for values

        Returns
        -------
        unique: ndarray, [groups]
            unique keys
        reduced : ndarray, [groups, ...]
            value array, reduced over groups
        """
        values = np.asarray(values)
        return self.unique, self.reduce(values, np.maximum, axis)


Instruction:
How would you code a function in Python 3 to
return values at first occurance of its associated key

Response:
def first(self, values, axis=0):
        """return values at first occurance of its associated key

        Parameters
        ----------
        values : array_like, [keys, ...]
            values to pick the first value of per group
        axis : int, optional
            alternative reduction axis for values

        Returns
        -------
        unique: ndarray, [groups]
            unique keys
        reduced : ndarray, [groups, ...]
            value array, reduced over groups
        """
        values = np.asarray(values)
        return self.unique, np.take(values, self.index.sorter[self.index.start], axis)


Instruction:
Explain what the following Python 3 code does
def last(self, values, axis=0):
        values = np.asarray(values)
        return self.unique, np.take(values, self.index.sorter[self.index.stop-1], axis)

Response: return values at last occurance of its associated key


Instruction:
Explain what the following Python 3 code does
def any(self, values, axis=0):
        values = np.asarray(values)
        if not values.dtype == np.bool:
            values = values != 0
        return self.unique, self.reduce(values, axis=axis) > 0

Response: compute if any item evaluates to true in each group


Instruction:
How would you explain what the following Python 3 function does
def all(self, values, axis=0):
        values = np.asarray(values)
        return self.unique, self.reduce(values, axis=axis, operator=np.multiply) != 0

Response: compute if all items evaluates to true in each group


Instruction:
Write a Python 3 function for
returning the index into values corresponding to the minimum value of the group

Response:
def argmin(self, values):
        """return the index into values corresponding to the minimum value of the group

        Parameters
        ----------
        values : array_like, [keys]
            values to pick the argmin of per group

        Returns
        -------
        unique: ndarray, [groups]
            unique keys
        argmin : ndarray, [groups]
            index into value array, representing the argmin per group
        """
        keys, minima = self.min(values)
        minima = minima[self.inverse]
        # select the first occurence of the minimum in each group
        index = as_index((self.inverse, values == minima))
        return keys, index.sorter[index.start[-self.groups:]]


Instruction:
Create a Python 3 function to
return the index into values corresponding to the maximum value of the group

Response:
def argmax(self, values):
        """return the index into values corresponding to the maximum value of the group

        Parameters
        ----------
        values : array_like, [keys]
            values to pick the argmax of per group

        Returns
        -------
        unique: ndarray, [groups]
            unique keys
        argmax : ndarray, [groups]
            index into value array, representing the argmax per group
        """
        keys, maxima = self.max(values)
        maxima = maxima[self.inverse]
        # select the first occurence of the maximum in each group
        index = as_index((self.inverse, values == maxima))
        return keys, index.sorter[index.start[-self.groups:]]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def as_index(keys, axis=semantics.axis_default, base=False, stable=True, lex_as_struct=False):
    if isinstance(keys, Index):
        if type(keys) is BaseIndex and base==False:
            keys = keys.keys    #need to upcast to an indirectly sorted index type
        else:
            return keys         #already done here
    if isinstance(keys, tuple):
        if lex_as_struct:
            keys = as_struct_array(*keys)
        else:
            return LexIndex(keys, stable)

    try:
        keys = np.asarray(keys)
    except:
        raise TypeError('Given object does not form a valid set of keys')
    if axis is None:
        keys = keys.flatten()
    if keys.ndim==1:
        if base:
            return BaseIndex(keys)
        else:
            return Index(keys, stable=stable)
    else:
        return ObjectIndex(keys, axis, stable=stable)

Response: Converts a given set of keys to an index.


Instruction:
Can you create a Python 3 function that
returns index array that maps unique values back to original space. unique [ inverse ] == keys

Response:
def inverse(self):
        """return index array that maps unique values back to original space. unique[inverse]==keys"""
        inv = np.empty(self.size, np.int)
        inv[self.sorter] = self.sorted_group_rank_per_key
        return inv


Instruction:
Can you tell what is the following Python 3 function doing
def rank(self):
        r = np.empty(self.size, np.int)
        r[self.sorter] = np.arange(self.size)
        return r

Response: how high in sorted list each key is. inverse permutation of sorter


Instruction:
Can you create a Python 3 function that
returns a tuple of unique key columns

Response:
def unique(self):
        """returns a tuple of unique key columns"""
        return tuple(
            (array_as_typed(s, k.dtype, k.shape) if k.ndim>1 else s)[self.start]
                for s, k in zip(self.sorted, self._keys))


Instruction:
How would you code a function in Python 3 to
pack a sequence of columns into a recarray containing the input columns as struct fields

Response:
def as_struct_array(*columns):
    """pack a sequence of columns into a recarray

    Parameters
    ----------
    columns : sequence of key objects

    Returns
    -------
    data : recarray
        recarray containing the input columns as struct fields
    """
    columns = [np.asarray(c) for c in columns]
    rows = len(columns[0])

    names = ['f'+str(i) for i in range(len(columns))]
    dtype = [(names[i], c.dtype, c.shape[1:]) for i, c in enumerate(columns)]
    data = np.empty(rows, dtype)
    for i, c in enumerate(columns):
        data[names[i]] = c
    return data


Instruction:
Write a Python 3 function for
casting the given axis of an array to a void object

Response:
def axis_as_object(arr, axis=-1):
    """cast the given axis of an array to a void object
    if the axis to be cast is contiguous, a view is returned, otherwise a copy is made
    this is useful for efficiently sorting by the content of an axis, for instance

    Parameters
    ----------
    arr : ndarray
        array to view as void object type
    axis : int
        axis to view as a void object type

    Returns
    -------
    ndarray
        array with the given axis viewed as a void object
    """
    shape = arr.shape
    # make axis to be viewed as a void object as contiguous items
    arr = np.ascontiguousarray(np.rollaxis(arr, axis, arr.ndim))
    # number of bytes in each void object
    nbytes = arr.dtype.itemsize * shape[axis]
    # void type with the correct number of bytes
    voidtype = np.dtype((np.void, nbytes))
    # return the view as such, with the reduced shape
    return arr.view(voidtype).reshape(np.delete(shape, axis))


Instruction:
Can you create a Python 3 function that
casts an array of void objects to a typed axis

Response:
def object_as_axis(arr, dtype, axis=-1):
    """
    cast an array of void objects to a typed axis

    Parameters
    ----------
    arr : ndarray, [ndim], void
        array of type np.void
    dtype : numpy dtype object
        the output dtype to cast the input array to
    axis : int
        position to insert the newly formed axis into

    Returns
    -------
    ndarray, [ndim+1], dtype
        output array cast to given dtype
    """
    # view the void objects as typed elements
    arr = arr.view(dtype).reshape(arr.shape + (-1,))
    # put the axis in the specified location
    return np.rollaxis(arr, -1, axis)


Instruction:
Write a Python 3 function for
counting the number of times each key occurs in the input set

Response:
def count(keys, axis=semantics.axis_default):
    """count the number of times each key occurs in the input set

    Arguments
    ---------
    keys : indexable object

    Returns
    -------
    unique : ndarray, [groups, ...]
        unique keys
    count : ndarray, [groups], int
        the number of times each key occurs in the input set

    Notes
    -----
    Can be seen as numpy work-alike of collections.Counter
    Alternatively, as sparse equivalent of count_table
    """
    index = as_index(keys, axis, base=True)
    return index.unique, index.count


Instruction:
Write a Python 3 function that can
count the number of times each key occurs in the input set

Response:
def count_table(*keys):
    """count the number of times each key occurs in the input set

    Arguments
    ---------
    keys : tuple of indexable objects, each having the same number of items

    Returns
    -------
    unique : tuple of ndarray, [groups, ...]
        unique keys for each input item
        they form the axes labels of the table
    table : ndarray, [keys[0].groups, ... keys[n].groups], int
        the number of times each key-combination occurs in the input set

    Notes
    -----
    Equivalent to R's pivot table or pandas 'crosstab'
    Alternatively, dense equivalent of the count function
    Should we add weights option?
    Or better yet; what about general reductions over key-grids?
    """
    indices  = [as_index(k, axis=0) for k in keys]
    uniques  = [i.unique  for i in indices]
    inverses = [i.inverse for i in indices]
    shape    = [i.groups  for i in indices]
    table = np.zeros(shape, np.int)
    np.add.at(table, inverses, 1)
    return tuple(uniques), table


Instruction:
Create a Python 3 function to
perform binning over the given axes of the keys

Response:
def binning(keys, start, end, count, axes=None):
    """Perform binning over the given axes of the keys

    Parameters
    ----------
    keys : indexable or tuple of indexable

    Examples
    --------
    binning(np.random.rand(100), 0, 1, 10)
    """
    if isinstance(keys, tuple):
        n_keys = len(keys)
    else:
        n_keys = 1

    bins = np.linspace(start, end, count+1, endpoint=True)
    idx = np.searchsorted(bins, keys)
    if axes is None:
        axes = [-1]


Instruction:
How would you implement a function in Python 3 that
returns the multiplicity of each key or how often it occurs in the set

Response:
def multiplicity(keys, axis=semantics.axis_default):
    """return the multiplicity of each key, or how often it occurs in the set

    Parameters
    ----------
    keys : indexable object

    Returns
    -------
    ndarray, [keys.size], int
        the number of times each input item occurs in the set
    """
    index = as_index(keys, axis)
    return index.count[index.inverse]


Instruction:
How would you code a function in Python 3 to
return the rank of the object in the pecking order.

Response:
def rank(keys, axis=semantics.axis_default):
    """where each item is in the pecking order.

    Parameters
    ----------
    keys : indexable object

    Returns
    -------
    ndarray, [keys.size], int
        unique integers, ranking the sorting order

    Notes
    -----
    we should have that index.sorted[index.rank] == keys
    """
    index = as_index(keys, axis)
    return index.rank


Instruction:
Explain what the following Python 3 code does
def mode(keys, axis=semantics.axis_default, weights=None, return_indices=False):
    index = as_index(keys, axis)
    if weights is None:
        unique, weights = count(index)
    else:
        unique, weights = group_by(index).sum(weights)
    bin = np.argmax(weights)
    _mode = unique[bin]     # FIXME: replace with index.take for lexindex compatibility?
    if return_indices:
        indices = index.sorter[index.start[bin]: index.stop[bin]]
        return _mode, indices
    else:
        return _mode

Response: compute the mode or most frequent occuring key in a set


Instruction:
Explain what the following Python 3 code does
def incidence(boundary):
    return GroupBy(boundary).split(np.arange(boundary.size) // boundary.shape[1])

Response: compute indidence info matrix for a given boundary info between simplices


Instruction:
Write a Python 3 function that can
return true if all keys are unique

Response:
def all_unique(keys, axis=semantics.axis_default):
    """Returns true if all keys are unique"""
    index = as_index(keys, axis)
    return index.groups == index.size


Instruction:
Explain what the following Python 3 code does
def any_unique(keys, axis=semantics.axis_default):
    index = as_index(keys, axis)
    return np.any(index.count == 1)

Response: returns true if any of the keys is unique


Instruction:
Explain what the following Python 3 code does
def all_equal(keys, axis=semantics.axis_default):
    index = as_index(keys, axis)
    return index.groups == 1

Response: returns true of all keys are equal


Instruction:
Can you implement a function in Python 3 that
returns true if all keys have equal multiplicity

Response:
def is_uniform(keys, axis=semantics.axis_default):
    """returns true if all keys have equal multiplicity"""
    index = as_index(keys, axis)
    return index.uniform


Instruction:
How would you implement a function in Python 3 that
returns tuples of inverse indices of the given keys

Response:
def get_inverses(self, keys):
        """
        Returns
        -------
        Tuple of inverse indices
        """
        return tuple([as_index(k, axis=0).inverse for k in keys])


Instruction:
How would you explain what the following Python 3 function does
def unique(self, values):
        _, count = self.count()
        if not np.array_equiv(count, 1):
            raise ValueError("Not every entry in the table is assigned a unique value")
        return self.sum(values)

Response: Place each entry in a table while asserting that each entry occurs once


Instruction:
Can you write a function in Python 3 where it
computes the set of unique keys within a node - level object

Response:
def unique(keys, axis=semantics.axis_default, return_index=False, return_inverse=False, return_count=False):
    """compute the set of unique keys

    Parameters
    ----------
    keys : indexable key object
        keys object to find unique keys within
    axis : int
        if keys is a multi-dimensional array, the axis to regard as the sequence of key objects
    return_index : bool
        if True, return indexes such that keys[index] == unique
    return_inverse : bool
        if True, return the indices such that unique[inverse] == keys
    return_count : bool
        if True, return the number of times each unique key occurs in the input

    Notes
    -----
    The kwargs are there to provide a backwards compatible interface to numpy.unique, but arguably,
    it is cleaner to call index and its properties directly, should more than unique values be desired as output
    """
    stable = return_index or return_inverse
    index = as_index(keys, axis, base = not stable, stable = stable)

    ret = index.unique,
    if return_index:
        ret = ret + (index.index,)
    if return_inverse:
        ret = ret + (index.inverse,)
    if return_count:
        ret = ret + (index.count,)
    return ret[0] if len(ret) == 1 else ret


Instruction:
Implement a Python 3 function for
returning a boolean array indicating if each element of that is contained in this.

Response:
def contains(this, that, axis=semantics.axis_default):
    """Returns bool for each element of `that`, indicating if it is contained in `this`

    Parameters
    ----------
    this : indexable key sequence
        sequence of items to test against
    that : indexable key sequence
        sequence of items to test for

    Returns
    -------
    ndarray, [that.size], bool
        returns a bool for each element in `that`, indicating if it is contained in `this`

    Notes
    -----
    Reads as 'this contains that'
    Similar to 'that in this', but with different performance characteristics
    """
    this = as_index(this, axis=axis, lex_as_struct=True, base=True)
    that = as_index(that, axis=axis, lex_as_struct=True)

    left = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='left')
    right = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='right')

    flags = np.zeros(that.size + 1, dtype=np.int)
    np.add.at(flags, left, 1)
    np.add.at(flags, right, -1)

    return np.cumsum(flags)[:-1].astype(np.bool)[that.rank]


Instruction:
Implement a Python 3 function for
returning a bool for each element of this indicating if it is present in that.

Response:
def in_(this, that, axis=semantics.axis_default):
    """Returns bool for each element of `this`, indicating if it is present in `that`

    Parameters
    ----------
    this : indexable key sequence
        sequence of items to test for
    that : indexable key sequence
        sequence of items to test against

    Returns
    -------
    ndarray, [that.size], bool
        returns a bool for each element in `this`, indicating if it is present in `that`

    Notes
    -----
    Reads as 'this in that'
    Similar to 'that contains this', but with different performance characteristics
    """
    this = as_index(this, axis=axis, lex_as_struct=True, base=True)
    that = as_index(that, axis=axis, lex_as_struct=True)

    left = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='left')
    right = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='right')

    return left != right


Instruction:
Here you have a function in Python 3, explain what it does
def indices(this, that, axis=semantics.axis_default, missing='raise'):
    this = as_index(this, axis=axis, lex_as_struct=True)
    # use this for getting this.keys and that.keys organized the same way;
    # sorting is superfluous though. make sorting a cached property?
    # should we be working with cached properties generally?
    # or we should use sorted values, if searchsorted can exploit this knowledge?
    that = as_index(that, axis=axis, base=True, lex_as_struct=True)

    # use raw private keys here, rather than public unpacked keys
    insertion = np.searchsorted(this._keys, that._keys, sorter=this.sorter, side='left')
    indices = np.take(this.sorter, insertion, mode='clip')

    if missing != 'ignore':
        invalid = this._keys[indices] != that._keys
        if missing == 'raise':
            if np.any(invalid):
                raise KeyError('Not all keys in `that` are present in `this`')
        elif missing == 'mask':
            indices = np.ma.masked_array(indices, invalid)
        else:
            indices[invalid] = missing
    return indices

Response: Find indices such that this [ indices ] == that [ indices ]


Instruction:
Can you tell what is the following Python 3 function doing
def remap(input, keys, values, missing='ignore', inplace=False):
    input = np.asarray(input)   # FIXME: currently instances of Index are not allowed
    values = np.asarray(values)
    if missing == 'ignore':
        idx = indices(keys, input, missing='mask')
        mask = np.logical_not(idx.mask)
        idx = idx.data
    elif missing == 'raise':
        idx = indices(keys, input, missing='raise')
        mask = Ellipsis
    else:
        raise ValueError("'missing' should be either 'ignore' or 'raise'")
    output = input if inplace else input.copy()
    output[mask] = values[idx[mask]]
    return output

Response: Given an array of keys and values remaps its entries corresponding to keys to values.


Instruction:
Create a Python 3 function to
concatenate indexable objects. Parameters ---------- sets : iterable of indexable objects Returns ------- indexable object handles both arrays and tuples of arrays

Response:
def _set_concatenate(sets):
    """concatenate indexable objects.

    Parameters
    ----------
    sets : iterable of indexable objects

    Returns
    -------
    indexable object

    handles both arrays and tuples of arrays
    """
    def con(set):
        # if not all():
        #     raise ValueError('concatenated keys must have the same dtype')
        try:
            return np.concatenate([s for s in sets if len(s)])
        except ValueError:
            return set[0]

    if any(not isinstance(s, tuple) for s in sets):
        #assume all arrays
        return con(sets)
    else:
        #assume all tuples
        return tuple(con(s) for s in zip(*sets))


Instruction:
Implement a function in Python 3 to
return the elements which occur n times over the sequence of sets

Response:
def _set_count(sets, n, **kwargs):
    """return the elements which occur n times over the sequence of sets

    Parameters
    ----------
    sets : iterable of indexable objects
    n : int
        number of sets the element should occur in

    Returns
    -------
    indexable
        indexable with all elements that occured in n of the sets

    Notes
    -----
    used by both exclusive and intersection
    """
    sets = _set_preprocess(sets, **kwargs)
    i = as_index(_set_concatenate(sets), axis=0, base=True)
    # FIXME : this does not work for lex-keys
    return i.unique[i.count == n]


Instruction:
Given the following Python 3 function, write the documentation
def union(*sets, **kwargs):
    sets = _set_preprocess(sets, **kwargs)
    return as_index( _set_concatenate(sets), axis=0, base=True).unique

Response:
all unique items which occur in any one of the sets
    Returns a new indexable object containing the union of all unique items in any of the sets


Instruction:
How would you explain what the following Python 3 function does
def difference(*sets, **kwargs):
    head, tail = sets[0], sets[1:]
    idx = as_index(head, **kwargs)
    lhs = idx.unique
    rhs = [intersection(idx, s, **kwargs) for s in tail]
    return exclusive(lhs, *rhs, axis=0, assume_unique=True)

Response: returns the difference of all items in the set in the head set and all other items in the tail set


Instruction:
How would you code a function in Python 3 to
generate a random id for div chart.

Response:
def _generate_div_id_chart(prefix="chart_id", digits=8):
    """Generate a random id for div chart.
    """
    choices = (random.randrange(0, 52) for _ in range(digits))
    return prefix + "".join((string.ascii_letters[x] for x in choices))


Instruction:
Write a Python 3 script for
displaying you DataFrame with Highcharts.

Response:
def display_charts(df, chart_type="default", render_to=None, **kwargs):
    """Display you DataFrame with Highcharts.

    df: DataFrame
    chart_type: str
        'default' or 'stock'
    render_to: str
        div id for plotting your data
    """
    if chart_type not in ("default", "stock"):
        raise ValueError("Wrong chart_type: accept 'default' or 'stock'.")
    chart_id = render_to if render_to is not None else _generate_div_id_chart()
    json_data = serialize(df, render_to=chart_id, chart_type=chart_type,
                          **kwargs)
    content = """<div id="{chart_id}"</div>
    <script type="text/javascript">{data}</script>"""
    return display(HTML(content.format(chart_id=chart_id,
                                       data=json_data)))


Instruction:
Can you generate the documentation for the following Python 3 function
def _series_data_filter(data):
    data = copy.deepcopy(data)
    if "series" in data:
        for series in data["series"]:
            series["data"] = "[...]"
    return data

Response: Filter the data for the series key in the list stored under the key series by "[...]".


Instruction:
Can you generate a brief explanation for the following Python 3 code
def pretty_params(data, indent=2):
    data_to_print = _series_data_filter(data)
    print(json.dumps(data_to_print, indent=indent))

Response: Pretty print your Highcharts params into a JSON.


Instruction:
Here you have a function in Python 3, explain what it does
def get_additional_params(self, **params):

        # TODO: Move these params to their own vertical if needed.
        polling_params = [
            'locationschema',
            'carrierschema',
            'sorttype',
            'sortorder',
            'originairports',
            'destinationairports',
            'stops',
            'outbounddeparttime',
            'outbounddepartstarttime',
            'outbounddepartendtime',
            'inbounddeparttime',
            'inbounddepartstarttime',
            'inbounddepartendtime',
            'duration',
            'includecarriers',
            'excludecarriers'
        ]

        additional_params = dict(
            (key, value) for key, value in params.items()
            if key in polling_params
        )

        return additional_params

Response: Get the additional params needed for polling.


Instruction:
Can you tell what is the following Python 3 function doing
def get_result(self, errors=GRACEFUL, **params):
        additional_params = self.get_additional_params(**params)
        return self.poll_session(
            self.create_session(**params),
            errors=errors,
            **additional_params
        )

Response: Get all results by creating and polling the session.


Instruction:
Implement a Python 3 function for
making a request to the specified service URL and returns the response.

Response:
def make_request(self, service_url, method='get', headers=None, data=None,
                     callback=None, errors=GRACEFUL, **params):
        """
        Reusable method for performing requests.

        :param service_url - URL to request
        :param method - request method, default is 'get'
        :param headers - request headers
        :param data - post data
        :param callback - callback to be applied to response,
                          default callback will parse response as json object.
        :param errors - specifies communication errors handling mode,
                        possible values are:
                         * strict (default) - throw an error as soon as one
                                              occurred
                         * graceful - ignore certain errors, e.g. EmptyResponse
                         * ignore - ignore all errors and return a result in
                                    any case.
                                    NOTE that it DOES NOT mean that
                                    no exceptions can be raised from
                                    this method, it mostly ignores
                                    communication related errors.
                         * None or empty string equals to default
        :param params - additional query parameters for request
        """
        error_modes = (STRICT, GRACEFUL, IGNORE)
        error_mode = errors or GRACEFUL
        if error_mode.lower() not in error_modes:
            raise ValueError(
                'Possible values for errors argument are: %s' %
                ', '.join(error_modes)
            )

        if callback is None:
            callback = self._default_resp_callback

        if 'apikey' not in service_url.lower():
            params.update({
                'apiKey': self.api_key
            })

        request = getattr(requests, method.lower())

        log.debug('* Request URL: %s' % service_url)
        log.debug('* Request method: %s' % method)
        log.debug('* Request query params: %s' % params)
        log.debug('* Request headers: %s' % headers)

        r = request(service_url, headers=headers, data=data, params=params)
        try:
            r.raise_for_status()
            return callback(r)
        except Exception as e:
            return self._with_error_handling(r, e, error_mode,
                                             self.response_format)


Instruction:
How would you explain what the following Python 3 function does
def get_markets(self, market):
        url = "{url}/{market}".format(url=self.MARKET_SERVICE_URL,
                                      market=market)
        return self.make_request(url, headers=self._headers())

Response: Get the list of markets available for a given market


Instruction:
Make a summary of the following Python 3 code
def location_autosuggest(self, **params):
        service_url = "{url}/{params_path}".format(
            url=self.LOCATION_AUTOSUGGEST_URL,
            params_path=self._construct_params(
                params, self.LOCATION_AUTOSUGGEST_PARAMS)
        )
        return self.make_request(
            service_url,
            headers=self._headers(),
            **params
        )

Response: This method returns a response object containing the location of the object.


Instruction:
How would you code a function in Python 3 to
poll the URL :param poll_url - URL to poll, should be returned by 'create_session' call :param initial_delay - specifies how many seconds to wait before the first poll :param delay - specifies how many seconds to wait between the polls :param tries - number of polls to perform :param errors - errors handling mode, see corresponding parameter in 'make_request' method :param params - additional query params for each poll request

Response:
def poll_session(self, poll_url, initial_delay=2, delay=1, tries=20,
                     errors=GRACEFUL, **params):
        """
        Poll the URL
        :param poll_url - URL to poll,
                          should be returned by 'create_session' call
        :param initial_delay - specifies how many seconds to wait before
                               the first poll
        :param delay - specifies how many seconds to wait between the polls
        :param tries - number of polls to perform
        :param errors - errors handling mode,
                        see corresponding parameter in 'make_request' method
        :param params - additional query params for each poll request
        """
        time.sleep(initial_delay)
        poll_response = None
        for n in range(tries):
            poll_response = self.make_request(
                poll_url,
                headers=self._headers(),
                errors=errors, **params
            )

            if self.is_poll_complete(poll_response):
                return poll_response
            else:
                time.sleep(delay)

        if STRICT == errors:
            raise ExceededRetries(
                "Failed to poll within {0} tries.".format(tries))
        else:
            return poll_response


Instruction:
Create a Python 3 function to
construct params list from required_keys and opt_keys.

Response:
def _construct_params(params, required_keys, opt_keys=None):
        """
        Construct params list in order of given keys.
        """
        try:
            params_list = [params.pop(key) for key in required_keys]
        except KeyError as e:
            raise MissingParameter(
                'Missing expected request parameter: %s' % e)
        if opt_keys:
            params_list.extend([params.pop(key)
                                for key in opt_keys if key in params])
        return '/'.join(str(p) for p in params_list)


Instruction:
How would you explain what the following Python 3 function does
def create_session(self, **params):
        return self.make_request(self.PRICING_SESSION_URL,
                                 method='post',
                                 headers=self._session_headers(),
                                 callback=lambda resp: resp.headers[
                                     'location'],
                                 data=params)

Response: Create the session for the current user


Instruction:
Can you tell what is the following Python 3 function doing
def request_booking_details(self, poll_url, **params):
        return self.make_request("%s/booking" % poll_url,
                                 method='put',
                                 headers=self._headers(),
                                 callback=lambda resp: resp.headers[
                                     'location'],
                                 **params)

Response: Request for booking details for a specific resource.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_cheapest_price_by_date(self, **params):
        service_url = "{url}/{params_path}".format(
            url=self.BROWSE_DATES_SERVICE_URL,
            params_path=self._construct_params(
                params, self._REQ_PARAMS, self._OPT_PARAMS)
        )

        return self.make_request(
            service_url,
            headers=self._headers(),
            **params
        )

Response: This method returns the cheapest price for the current date.


Instruction:
How would you explain what the following Python 3 function does
def get_cheapest_price_by_route(self, **params):
        service_url = "{url}/{params_path}".format(
            url=self.BROWSE_ROUTES_SERVICE_URL,
            params_path=self._construct_params(
                params, self._REQ_PARAMS, self._OPT_PARAMS)

        )
        return self.make_request(
            service_url,
            headers=self._headers(),
            **params
        )

Response: This method returns the cheapest price by route.


Instruction:
Given the following Python 3 function, write the documentation
def get_cheapest_quotes(self, **params):
        service_url = "{url}/{params_path}".format(
            url=self.BROWSE_QUOTES_SERVICE_URL,
            params_path=self._construct_params(
                params, self._REQ_PARAMS, self._OPT_PARAMS)
        )
        return self.make_request(
            service_url,
            headers=self._headers(),
            **params
        )

Response: This method returns the cheapest quotes for the current locale.


Instruction:
Can you write a function in Python 3 where it
creates the session for the user.

Response:
def create_session(self, **params):
        """
        Create the session
        date format: YYYY-MM-DDThh:mm
        location: ISO code
        """

        required_keys = ('market', 'currency', 'locale', 'pickupplace',
                         'dropoffplace', 'pickupdatetime', 'dropoffdatetime',
                         'driverage')

        service_url = "{url}/{params_path}".format(
            url=self.PRICING_SESSION_URL,
            params_path=self._construct_params(params, required_keys)
        )

        poll_path = self.make_request(service_url,
                                      headers=self._session_headers(),
                                      callback=lambda resp: resp.headers[
                                          'location'],
                                      userip=params['userip'])

        return "{url}{path}".format(url=self.API_HOST, path=poll_path)


Instruction:
How would you implement a function in Python 3 that
formats keyboard /command field.

Response:
def keyboard_field(value, args=None):
    """
    Format keyboard /command field.
    """
    qs = QueryDict(args)
    per_line = qs.get('per_line', 1)
    field = qs.get("field", "slug")
    command = qs.get("command")
    convert = lambda element: "/" + command + " " + str(getattr(element, field))
    group = lambda flat, size: [flat[i:i+size] for i in range(0, len(flat), size)]
    grouped = group(value, int(per_line))
    new_list = []
    for line in grouped:
        new_list.append([convert(e) for e in line])
    return str(new_list).encode('utf-8')


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_queryset(self):
        if self.queryset is None:
            if self.model:
                return self.model._default_manager.all()
            else:
                raise ImproperlyConfigured(
                    "%(cls)s is missing a QuerySet. Define "
                    "%(cls)s.model, %(cls)s.queryset, or override "
                    "%(cls)s.get_queryset()." % {
                        'cls': self.__class__.__name__
                    }
                )
        return self.queryset.all()

Response: Returns the QuerySet that will be used to look up the object.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def login_required(view_func):
    @wraps(view_func)
    def wrapper(bot, update, **kwargs):
        chat = Chat.objects.get(id=update.message.chat.id)
        if chat.is_authenticated():
            return view_func(bot, update, **kwargs)
        from telegrambot.bot_views.login import LoginBotView
        login_command_view = LoginBotView.as_command_view()
        bot_model = Bot.objects.get(token=bot.token)
        kwargs['link'] = reverse('telegrambot:auth', kwargs={'bot': bot_model.user_api.username})
        return login_command_view(bot, update, **kwargs)
    return wrapper

Response: Decorator for command views that checks that the chat is authenticated and sends message with link for authenticated if necessary.


Instruction:
Here you have a function in Python 3, explain what it does
def data_structure_builder(func):
    @wraps(func)
    def ds_builder_wrapper(function, *args, **kwargs):
        try:
            function = DSBuilder(function)
        except NoBuilder:
            pass
        return func(function, *args, **kwargs)

    return ds_builder_wrapper

Response: Decorator to handle automatic data structure creation for pipe - utils.


Instruction:
How would you explain what the following Python 3 function does
def regex_condition(func):
    @wraps(func)
    def regex_condition_wrapper(condition, *args, **kwargs):
        if isinstance(condition, string_types):
            condition = maybe | partial(re.match, condition)
        return func(condition, *args, **kwargs)
    return regex_condition_wrapper

Response: Decorator that turns a condition string instead of a function into a regex - matching function.


Instruction:
Implement a function in Python 3 to
sort an incoming sequence by using the given function as key.

Response:
def sort_by(function):
    """
    Sorts an incoming sequence by using the given `function` as key.

    >>> range(10) > sort_by(-X)
    [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

    Supports automatic data-structure creation::

        users > sort_by([X.last_name, X.first_name])

    There is also a shortcut for ``sort_by(X)`` called ``sort``:

    >>> [4, 5, 8, -3, 0] > sort
    [-3, 0, 4, 5, 8]

    And (as of ``0.2.3``) a shortcut for reversing the sort:

    >>> 'asdfaSfa' > sort_by(X.lower()).descending
    ['s', 'S', 'f', 'f', 'd', 'a', 'a', 'a']
    """
    f = partial(sorted, key=function)
    f.attrs = {'descending': _descending_sort_by(function)}
    return f


Instruction:
Write a Python 3 function that can
take the first count items from the input.

Response:
def take_first(count):
    """
    Assumes an iterable on the input, returns an iterable with first `count`
    items from the input (or possibly less, if there isn't that many).

    >>> range(9000) > where(X % 100 == 0) | take_first(5) | tuple
    (0, 100, 200, 300, 400)

    """
    def _take_first(iterable):
        return islice(iterable, count)
    return pipe | set_name('take_first(%s)' % count, _take_first)


Instruction:
Make a summary of the following Python 3 code
def drop_first(count):
    def _drop_first(iterable):
        g = (x for x in range(1, count + 1))
        return dropwhile(
            lambda i: unless(StopIteration, lambda: next(g))(), iterable)
    return pipe | set_name('drop_first(%s)' % count, _drop_first)

Response:
Returns an iterable with identical items
            except for the first count items.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def unless(exception_class_or_tuple, func, *args, **kwargs):
    @pipe_util
    @auto_string_formatter
    @data_structure_builder
    def construct_unless(function):
        # a wrapper so we can re-use the decorators
        def _unless(*args, **kwargs):
            try:
                return function(*args, **kwargs)
            except exception_class_or_tuple:
                pass
        return _unless

    name = lambda: 'unless(%s, %s)' % (exception_class_or_tuple, ', '.join(
        filter(None, (get_name(func), repr_args(*args, **kwargs)))))

    return set_name(name, construct_unless(func, *args, **kwargs))

Response: A decorator that returns a new object that will not be caught while executing func.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def group_by(function):
    def _group_by(seq):
        result = {}
        for item in seq:
            result.setdefault(function(item), []).append(item)
        return dict_items(result)

    return _group_by

Response: Groups input sequence by function.


Instruction:
Given the following Python 3 function, write the documentation
def take_until(condition):
    f = partial(takewhile, pipe | condition | operator.not_)
    f.attrs = {'including': take_until_including(condition)}
    return f

Response: take_until returns a list of nodes where condition is true


Instruction:
Make a summary of the following Python 3 code
def take_until_including(condition):
    def take_until_including_(interable):
        for i in interable:
            if not condition(i):
                yield i
            else:
                yield i
                break
    return take_until_including_

Response: Return a generator that yields the elements of the list of items in the list that satisfy the condition.


Instruction:
Make a summary of the following Python 3 code
def xpartial(func, *xargs, **xkwargs):
    any_x = any(isinstance(a, XObject) for a in xargs + tuple(xkwargs.values()))
    use = lambda x, value: (~x)(value) if isinstance(x, XObject) else x

    @wraps(func, assigned=filter(partial(hasattr, func), WRAPPER_ASSIGNMENTS))
    def xpartially_applied(*func_args, **func_kwargs):
        if any_x:
            if not func_args:
                raise ValueError('Function "%s" partially applied with an '
                    'X placeholder but called with no positional arguments.'
                    % get_name(func))
            first = func_args[0]
            rest = func_args[1:]
            args = tuple(use(x, first) for x in xargs) + rest
            kwargs = dict((k, use(x, first)) for k, x in dict_items(xkwargs))
            kwargs.update(func_kwargs)
        else:
            args = xargs + func_args
            kwargs = dict(xkwargs, **func_kwargs)
        return func(*args, **kwargs)

    name = lambda: '%s(%s)' % (get_name(func), repr_args(*xargs, **xkwargs))
    return set_name(name, xpartially_applied)

Response: A function that returns a new object which is a partial of the function func with the given arguments and keyword arguments.


Instruction:
Create a Python 3 function for
returning the ecliptic longitude of the sun at the given UTC time.

Response:
def sun_ecliptic_longitude(utc_time):
    """Ecliptic longitude of the sun at *utc_time*.
    """
    jdate = jdays2000(utc_time) / 36525.0
    # mean anomaly, rad
    m_a = np.deg2rad(357.52910 +
                     35999.05030 * jdate -
                     0.0001559 * jdate * jdate -
                     0.00000048 * jdate * jdate * jdate)
    # mean longitude, deg
    l_0 = 280.46645 + 36000.76983 * jdate + 0.0003032 * jdate * jdate
    d_l = ((1.914600 - 0.004817 * jdate - 0.000014 * jdate * jdate) * np.sin(m_a) +
           (0.019993 - 0.000101 * jdate) * np.sin(2 * m_a) + 0.000290 * np.sin(3 * m_a))
    # true longitude, deg
    l__ = l_0 + d_l
    return np.deg2rad(l__)


Instruction:
How would you explain what the following Python 3 function does
def sun_ra_dec(utc_time):
    jdate = jdays2000(utc_time) / 36525.0
    eps = np.deg2rad(23.0 + 26.0 / 60.0 + 21.448 / 3600.0 -
                     (46.8150 * jdate + 0.00059 * jdate * jdate -
                      0.001813 * jdate * jdate * jdate) / 3600)
    eclon = sun_ecliptic_longitude(utc_time)
    x__ = np.cos(eclon)
    y__ = np.cos(eps) * np.sin(eclon)
    z__ = np.sin(eps) * np.sin(eclon)
    r__ = np.sqrt(1.0 - z__ * z__)
    # sun declination
    declination = np.arctan2(z__, r__)
    # right ascension
    right_ascension = 2 * np.arctan2(y__, (x__ + r__))
    return right_ascension, declination

Response: Right ascension and declination of the sun at *utc_time*.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_alt_az(utc_time, lon, lat):
    lon = np.deg2rad(lon)
    lat = np.deg2rad(lat)

    ra_, dec = sun_ra_dec(utc_time)
    h__ = _local_hour_angle(utc_time, lon, ra_)
    return (np.arcsin(np.sin(lat) * np.sin(dec) +
                      np.cos(lat) * np.cos(dec) * np.cos(h__)),
            np.arctan2(-np.sin(h__), (np.cos(lat) * np.tan(dec) -
                                      np.sin(lat) * np.cos(h__))))

Response: Return sun altitude and azimuth from the given time.


Instruction:
Can you tell what is the following Python 3 function doing
def cos_zen(utc_time, lon, lat):
    lon = np.deg2rad(lon)
    lat = np.deg2rad(lat)

    r_a, dec = sun_ra_dec(utc_time)
    h__ = _local_hour_angle(utc_time, lon, r_a)
    return (np.sin(lat) * np.sin(dec) + np.cos(lat) * np.cos(dec) * np.cos(h__))

Response: Cosine of the sun - zenith angle for lon and lat at the UTC time.


Instruction:
Create a Python 3 function for
returning sun - zonal angle at a given longitude and latitude.

Response:
def sun_zenith_angle(utc_time, lon, lat):
    """Sun-zenith angle for *lon*, *lat* at *utc_time*.
    lon,lat in degrees.
    The angle returned is given in degrees
    """
    return np.rad2deg(np.arccos(cos_zen(utc_time, lon, lat)))


Instruction:
Can you create a Python 3 function that
calculates the sun earth distance correction relative to 1 AU.

Response:
def sun_earth_distance_correction(utc_time):
    """Calculate the sun earth distance correction, relative to 1 AU.
    """
    year = 365.256363004
    # This is computed from
    # http://curious.astro.cornell.edu/question.php?number=582
    # AU = 149597870700.0
    # a = 149598261000.0
    # theta = (jdays2000(utc_time) - 2) * (2 * np.pi) / year
    # e = 0.01671123
    # r = a*(1-e*e)/(1+e * np.cos(theta))
    # corr_me = (r / AU) ** 2

    # from known software.
    corr = 1 - 0.0334 * np.cos(2 * np.pi * (jdays2000(utc_time) - 2) / year)

    return corr


Instruction:
Make a summary of the following Python 3 code
def observer_position(time, lon, lat, alt):

    lon = np.deg2rad(lon)
    lat = np.deg2rad(lat)

    theta = (gmst(time) + lon) % (2 * np.pi)
    c = 1 / np.sqrt(1 + F * (F - 2) * np.sin(lat)**2)
    sq = c * (1 - F)**2

    achcp = (A * c + alt) * np.cos(lat)
    x = achcp * np.cos(theta)  # kilometers
    y = achcp * np.sin(theta)
    z = (A * sq + alt) * np.sin(lat)

    vx = -MFACTOR * y  # kilometers/second
    vy = MFACTOR * x
    vz = 0

    return (x, y, z), (vx, vy, vz)

Response: Calculate observer ECI position.


Instruction:
Can you implement a function in Python 3 that
calculates the observer look angle to a satellite.

Response:
def get_observer_look(sat_lon, sat_lat, sat_alt, utc_time, lon, lat, alt):
    """Calculate observers look angle to a satellite.
    http://celestrak.com/columns/v02n02/

    utc_time: Observation time (datetime object)
    lon: Longitude of observer position on ground in degrees east
    lat: Latitude of observer position on ground in degrees north
    alt: Altitude above sea-level (geoid) of observer position on ground in km

    Return: (Azimuth, Elevation)
    """
    (pos_x, pos_y, pos_z), (vel_x, vel_y, vel_z) = astronomy.observer_position(
        utc_time, sat_lon, sat_lat, sat_alt)

    (opos_x, opos_y, opos_z), (ovel_x, ovel_y, ovel_z) = \
        astronomy.observer_position(utc_time, lon, lat, alt)

    lon = np.deg2rad(lon)
    lat = np.deg2rad(lat)

    theta = (astronomy.gmst(utc_time) + lon) % (2 * np.pi)

    rx = pos_x - opos_x
    ry = pos_y - opos_y
    rz = pos_z - opos_z

    sin_lat = np.sin(lat)
    cos_lat = np.cos(lat)
    sin_theta = np.sin(theta)
    cos_theta = np.cos(theta)

    top_s = sin_lat * cos_theta * rx + \
        sin_lat * sin_theta * ry - cos_lat * rz
    top_e = -sin_theta * rx + cos_theta * ry
    top_z = cos_lat * cos_theta * rx + \
        cos_lat * sin_theta * ry + sin_lat * rz

    az_ = np.arctan(-top_e / top_s)

    if has_xarray and isinstance(az_, xr.DataArray):
        az_data = az_.data
    else:
        az_data = az_

    if has_dask and isinstance(az_data, da.Array):
        az_data = da.where(top_s > 0, az_data + np.pi, az_data)
        az_data = da.where(az_data < 0, az_data + 2 * np.pi, az_data)
    else:
        az_data[np.where(top_s > 0)] += np.pi
        az_data[np.where(az_data < 0)] += 2 * np.pi

    if has_xarray and isinstance(az_, xr.DataArray):
        az_.data = az_data
    else:
        az_ = az_data

    rg_ = np.sqrt(rx * rx + ry * ry + rz * rz)
    el_ = np.arcsin(top_z / rg_)

    return np.rad2deg(az_), np.rad2deg(el_)


Instruction:
How would you code a function in Python 3 to
calculate time of last ascending node relative to the timecluster.

Response:
def get_last_an_time(self, utc_time):
        """Calculate time of last ascending node relative to the
        specified time
        """

        # Propagate backwards to ascending node
        dt = np.timedelta64(10, 'm')
        t_old = utc_time
        t_new = t_old - dt
        pos0, vel0 = self.get_position(t_old, normalize=False)
        pos1, vel1 = self.get_position(t_new, normalize=False)
        while not (pos0[2] > 0 and pos1[2] < 0):
            pos0 = pos1
            t_old = t_new
            t_new = t_old - dt
            pos1, vel1 = self.get_position(t_new, normalize=False)

        # Return if z within 1 km of an
        if np.abs(pos0[2]) < 1:
            return t_old
        elif np.abs(pos1[2]) < 1:
            return t_new

        # Bisect to z within 1 km
        while np.abs(pos1[2]) > 1:
            # pos0, vel0 = pos1, vel1
            dt = (t_old - t_new) / 2
            t_mid = t_old - dt
            pos1, vel1 = self.get_position(t_mid, normalize=False)
            if pos1[2] > 0:
                t_old = t_mid
            else:
                t_new = t_mid

        return t_mid


Instruction:
Explain what the following Python 3 code does
def get_position(self, utc_time, normalize=True):

        kep = self._sgdp4.propagate(utc_time)
        pos, vel = kep2xyz(kep)

        if normalize:
            pos /= XKMPER
            vel /= XKMPER * XMNPDA / SECDAY

        return pos, vel

Response: Get the cartesian position and velocity from the satellite.


Instruction:
Make a summary of the following Python 3 code
def get_lonlatalt(self, utc_time):
        (pos_x, pos_y, pos_z), (vel_x, vel_y, vel_z) = self.get_position(
            utc_time, normalize=True)

        lon = ((np.arctan2(pos_y * XKMPER, pos_x * XKMPER) - astronomy.gmst(utc_time))
               % (2 * np.pi))

        lon = np.where(lon > np.pi, lon - np.pi * 2, lon)
        lon = np.where(lon <= -np.pi, lon + np.pi * 2, lon)

        r = np.sqrt(pos_x ** 2 + pos_y ** 2)
        lat = np.arctan2(pos_z, r)
        e2 = F * (2 - F)
        while True:
            lat2 = lat
            c = 1 / (np.sqrt(1 - e2 * (np.sin(lat2) ** 2)))
            lat = np.arctan2(pos_z + c * e2 * np.sin(lat2), r)
            if np.all(abs(lat - lat2) < 1e-10):
                break
        alt = r / np.cos(lat) - c
        alt *= A
        return np.rad2deg(lon), np.rad2deg(lat), alt

Response: Calculate lon lat and alt of the satellite.


Instruction:
How would you explain what the following Python 3 function does
def get_observer_look(self, utc_time, lon, lat, alt):

        utc_time = dt2np(utc_time)
        (pos_x, pos_y, pos_z), (vel_x, vel_y, vel_z) = self.get_position(
            utc_time, normalize=False)
        (opos_x, opos_y, opos_z), (ovel_x, ovel_y, ovel_z) = \
            astronomy.observer_position(utc_time, lon, lat, alt)

        lon = np.deg2rad(lon)
        lat = np.deg2rad(lat)

        theta = (astronomy.gmst(utc_time) + lon) % (2 * np.pi)

        rx = pos_x - opos_x
        ry = pos_y - opos_y
        rz = pos_z - opos_z

        sin_lat = np.sin(lat)
        cos_lat = np.cos(lat)
        sin_theta = np.sin(theta)
        cos_theta = np.cos(theta)

        top_s = sin_lat * cos_theta * rx + \
            sin_lat * sin_theta * ry - cos_lat * rz
        top_e = -sin_theta * rx + cos_theta * ry
        top_z = cos_lat * cos_theta * rx + \
            cos_lat * sin_theta * ry + sin_lat * rz

        az_ = np.arctan(-top_e / top_s)

        az_ = np.where(top_s > 0, az_ + np.pi, az_)
        az_ = np.where(az_ < 0, az_ + 2 * np.pi, az_)

        rg_ = np.sqrt(rx * rx + ry * ry + rz * rz)
        el_ = np.arcsin(top_z / rg_)

        return np.rad2deg(az_), np.rad2deg(el_)

Response: Calculate the observer look angle to a satellite.


Instruction:
Write a Python 3 script to
calculate the orbit number at the specified time.

Response:
def get_orbit_number(self, utc_time, tbus_style=False):
        """Calculate orbit number at specified time.
        Optionally use TBUS-style orbit numbering (TLE orbit number + 1)
        """
        utc_time = np.datetime64(utc_time)
        try:
            dt = astronomy._days(utc_time - self.orbit_elements.an_time)
            orbit_period = astronomy._days(self.orbit_elements.an_period)
        except AttributeError:
            pos_epoch, vel_epoch = self.get_position(self.tle.epoch,
                                                     normalize=False)
            if np.abs(pos_epoch[2]) > 1 or not vel_epoch[2] > 0:
                # Epoch not at ascending node
                self.orbit_elements.an_time = self.get_last_an_time(
                    self.tle.epoch)
            else:
                # Epoch at ascending node (z < 1 km) and positive v_z
                self.orbit_elements.an_time = self.tle.epoch

            self.orbit_elements.an_period = self.orbit_elements.an_time - \
                self.get_last_an_time(self.orbit_elements.an_time
                                      - np.timedelta64(10, 'm'))

            dt = astronomy._days(utc_time - self.orbit_elements.an_time)
            orbit_period = astronomy._days(self.orbit_elements.an_period)

        orbit = int(self.tle.orbit + dt / orbit_period +
                    self.tle.mean_motion_derivative * dt**2 +
                    self.tle.mean_motion_sec_derivative * dt**3)

        if tbus_style:
            orbit += 1
        return orbit


Instruction:
Given the following Python 3 function, write the documentation
def get_next_passes(self, utc_time, length, lon, lat, alt, tol=0.001, horizon=0):

        def elevation(minutes):
            """Compute the elevation."""
            return self.get_observer_look(utc_time +
                                          timedelta(
                                              minutes=np.float64(minutes)),
                                          lon, lat, alt)[1] - horizon

        def elevation_inv(minutes):
            """Compute the inverse of elevation."""
            return -elevation(minutes)

        def get_root(fun, start, end, tol=0.01):
            """Root finding scheme"""
            x_0 = end
            x_1 = start
            fx_0 = fun(end)
            fx_1 = fun(start)
            if abs(fx_0) < abs(fx_1):
                fx_0, fx_1 = fx_1, fx_0
                x_0, x_1 = x_1, x_0

            x_n = optimize.brentq(fun, x_0, x_1)
            return x_n

        def get_max_parab(fun, start, end, tol=0.01):
            """Successive parabolic interpolation."""
            a = float(start)
            c = float(end)
            b = (a + c) / 2.0

            f_a = fun(a)
            f_b = fun(b)
            f_c = fun(c)

            x = b

            while True:
                x = x - 0.5 * (((b - a) ** 2 * (f_b - f_c)
                                - (b - c) ** 2 * (f_b - f_a)) /
                               ((b - a) * (f_b - f_c) - (b - c) * (f_b - f_a)))
                if np.isnan(x):
                    return b
                if abs(b - x) <= tol:
                    return x

                a, b, c = (a + x) / 2.0, x, (x + c) / 2.0
                f_a, f_b, f_c = fun(a), fun(b), fun(c)

        # every minute
        times = utc_time + np.array([timedelta(minutes=minutes)
                                     for minutes in range(length * 60)])
        elev = self.get_observer_look(times, lon, lat, alt)[1] - horizon
        zcs = np.where(np.diff(np.sign(elev)))[0]
        res = []
        risetime = None
        falltime = None
        for guess in zcs:
            horizon_mins = get_root(
                elevation, guess, guess + 1.0, tol=tol / 60.0)
            horizon_time = utc_time + timedelta(minutes=horizon_mins)
            if elev[guess] < 0:
                risetime = horizon_time
                risemins = horizon_mins
                falltime = None
            else:
                falltime = horizon_time
                fallmins = horizon_mins
                if risetime:
                    int_start = max(0, int(np.floor(risemins)))
                    int_end = min(len(elev), int(np.ceil(fallmins) + 1))
                    middle = int_start + np.argmax(elev[int_start:int_end])
                    highest = utc_time + \
                        timedelta(minutes=get_max_parab(
                            elevation_inv,
                            max(risemins, middle - 1), min(fallmins, middle + 1),
                            tol=tol / 60.0
                        ))
                    res += [(risetime, falltime, highest)]
                risetime = None
        return res

Response: Calculate the next passes for a given observer.


Instruction:
Can you generate the documentation for the following Python 3 function
def _get_time_at_horizon(self, utc_time, obslon, obslat, **kwargs):
        warnings.warn("_get_time_at_horizon is replaced with get_next_passes",
                      DeprecationWarning)
        if "precision" in kwargs:
            precision = kwargs['precision']
        else:
            precision = timedelta(seconds=0.001)
        if "max_iterations" in kwargs:
            nmax_iter = kwargs["max_iterations"]
        else:
            nmax_iter = 100

        sec_step = 0.5
        t_step = timedelta(seconds=sec_step / 2.0)

        # Local derivative:
        def fprime(timex):
            el0 = self.get_observer_look(timex - t_step,
                                         obslon, obslat, 0.0)[1]
            el1 = self.get_observer_look(timex + t_step,
                                         obslon, obslat, 0.0)[1]
            return el0, (abs(el1) - abs(el0)) / sec_step

        tx0 = utc_time - timedelta(seconds=1.0)
        tx1 = utc_time
        idx = 0
        # eps = 500.
        eps = 100.
        while abs(tx1 - tx0) > precision and idx < nmax_iter:
            tx0 = tx1
            fpr = fprime(tx0)
            # When the elevation is high the scale is high, and when
            # the elevation is low the scale is low
            # var_scale = np.abs(np.sin(fpr[0] * np.pi/180.))
            # var_scale = np.sqrt(var_scale)
            var_scale = np.abs(fpr[0])
            tx1 = tx0 - timedelta(seconds=(eps * var_scale * fpr[1]))
            idx = idx + 1
            # print idx, tx0, tx1, var_scale, fpr
            if abs(tx1 - utc_time) < precision and idx < 2:
                tx1 = tx1 + timedelta(seconds=1.0)

        if abs(tx1 - tx0) <= precision and idx < nmax_iter:
            return tx1
        else:
            return None

Response: Get the time closest in time to utc_time when the satellite is at the horizon relative to the position of an observer on the base of the satellite.


Instruction:
How would you explain what the following Python 3 function does
def subpoint(query_point, a=A, b=B):
    x, y, z = query_point

    lat = geodetic_lat(query_point)
    lon = np.arctan2(y, x)
    e2_ = (a * a - b * b) / (a * a)
    n__ = a / np.sqrt(1 - e2_ * np.sin(lat)**2)
    nx_ = n__ * np.cos(lat) * np.cos(lon)
    ny_ = n__ * np.cos(lat) * np.sin(lon)
    nz_ = (1 - e2_) * n__ * np.sin(lat)

    return np.stack([nx_, ny_, nz_], axis=0)

Response: Get the point on the ellipsoid under the query_point.


Instruction:
Can you create a Python 3 function that
rotates vector around axis by angle.

Response:
def qrotate(vector, axis, angle):
    """Rotate *vector* around *axis* by *angle* (in radians).

    *vector* is a matrix of column vectors, as is *axis*.
    This function uses quaternion rotation.
    """
    n_axis = axis / vnorm(axis)
    sin_angle = np.expand_dims(np.sin(angle / 2), 0)
    if np.ndim(n_axis) == 1:
        n_axis = np.expand_dims(n_axis, 1)
        p__ = np.dot(n_axis, sin_angle)[:, np.newaxis]
    else:
        p__ = n_axis * sin_angle

    q__ = Quaternion(np.cos(angle / 2), p__)
    shape = vector.shape
    return np.einsum("kj, ikj->ij",
                     vector.reshape((3, -1)),
                     q__.rotation_matrix()[:3, :3]).reshape(shape)


Instruction:
Can you generate the documentation for the following Python 3 function
def compute_pixels(orb, sgeom, times, rpy=(0.0, 0.0, 0.0)):
    if isinstance(orb, (list, tuple)):
        tle1, tle2 = orb
        orb = Orbital("mysatellite", line1=tle1, line2=tle2)

    # get position and velocity for each time of each pixel
    pos, vel = orb.get_position(times, normalize=False)

    # now, get the vectors pointing to each pixel
    vectors = sgeom.vectors(pos, vel, *rpy)

    # compute intersection of lines (directed by vectors and passing through
    # (0, 0, 0)) and ellipsoid. Derived from:
    # http://en.wikipedia.org/wiki/Line%E2%80%93sphere_intersection

    # do the computation between line and ellipsoid (WGS 84)
    # NB: AAPP uses GRS 80...
    centre = -pos
    a__ = 6378.137  # km
    # b__ = 6356.75231414 # km, GRS80
    b__ = 6356.752314245  # km, WGS84
    radius = np.array([[1 / a__, 1 / a__, 1 / b__]]).T
    shape = vectors.shape

    xr_ = vectors.reshape([3, -1]) * radius
    cr_ = centre.reshape([3, -1]) * radius
    ldotc = np.einsum("ij,ij->j", xr_, cr_)
    lsq = np.einsum("ij,ij->j", xr_, xr_)
    csq = np.einsum("ij,ij->j", cr_, cr_)

    d1_ = (ldotc - np.sqrt(ldotc ** 2 - csq * lsq + lsq)) / lsq

    # return the actual pixel positions
    return vectors * d1_.reshape(shape[1:]) - centre

Response: Compute the cartesian coordinates of the pixels in instrument scan.


Instruction:
Can you implement a function in Python 3 that
norms of a matrix of vectors stacked along the axis * dimension.

Response:
def mnorm(m, axis=None):
    """norm of a matrix of vectors stacked along the *axis* dimension."""
    if axis is None:
        axis = np.ndim(m) - 1
    return np.sqrt((m**2).sum(axis))


Instruction:
Write a Python 3 function for
getting the unit vectors pointing to the different pixels.

Response:
def vectors(self, pos, vel, roll=0.0, pitch=0.0, yaw=0.0):
        """Get unit vectors pointing to the different pixels.

        *pos* and *vel* are column vectors, or matrices of column
        vectors. Returns vectors as stacked rows.
        """
        # TODO: yaw steering mode !

        # Fake nadir: This is the intersection point between the satellite
        # looking down at the centre of the ellipsoid and the surface of the
        # ellipsoid. Nadir on the other hand is the point which vertical goes
        # through the satellite...
        # nadir = -pos / vnorm(pos)

        nadir = subpoint(-pos)
        nadir /= vnorm(nadir)

        # x is along track (roll)
        x = vel / vnorm(vel)

        # y is cross track (pitch)
        y = np.cross(nadir, vel, 0, 0, 0)
        y /= vnorm(y)

        # rotate first around x
        x_rotated = qrotate(nadir, x, self.fovs[0] + roll)
        # then around y
        xy_rotated = qrotate(x_rotated, y, self.fovs[1] + pitch)
        # then around z
        return qrotate(xy_rotated, nadir, yaw)


Instruction:
Explain what the following Python 3 code does
def avhrr(scans_nb, scan_points,
          scan_angle=55.37, frequency=1 / 6.0, apply_offset=True):
    """Definition of the avhrr instrument.

    Source: NOAA KLM User's Guide, Appendix J
    http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/html/j/app-j.htm
    """
    # build the avhrr instrument (scan angles)
    avhrr_inst = np.vstack(((scan_points / 1023.5 - 1)
                            * np.deg2rad(-scan_angle),
                            np.zeros((len(scan_points),))))

    avhrr_inst = np.tile(
        avhrr_inst[:, np.newaxis, :], [1, np.int(scans_nb), 1])

    # building the corresponding times array
    # times = (np.tile(scan_points * 0.000025 + 0.0025415, [scans_nb, 1])
    #         + np.expand_dims(offset, 1))

    times = np.tile(scan_points * 0.000025, [np.int(scans_nb), 1])
    if apply_offset:
        offset = np.arange(np.int(scans_nb)) * frequency
        times += np.expand_dims(offset, 1)

    return ScanGeometry(avhrr_inst, times)

Response: Build the avhrr instrument for the given number of scans.


Instruction:
How would you explain what the following Python 3 function does
def avhrr_gac(scan_times, scan_points,
              scan_angle=55.37, frequency=0.5):
    """Definition of the avhrr instrument, gac version

    Source: NOAA KLM User's Guide, Appendix J
    http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/html/j/app-j.htm
    """
    try:
        offset = np.array([(t - scan_times[0]).seconds +
                           (t - scan_times[0]).microseconds / 1000000.0 for t in scan_times])
    except TypeError:
        offset = np.arange(scan_times) * frequency
    scans_nb = len(offset)

    avhrr_inst = np.vstack(((scan_points / 1023.5 - 1)
                            * np.deg2rad(-scan_angle),
                            np.zeros((len(scan_points),))))

    avhrr_inst = np.tile(
        avhrr_inst[:, np.newaxis, :], [1, np.int(scans_nb), 1])
    # building the corresponding times array
    times = (np.tile(scan_points * 0.000025, [scans_nb, 1])
             + np.expand_dims(offset, 1))
    return ScanGeometry(avhrr_inst, times)

Response: Return the avhrr instrument and gac version of the avhrr archive.


Instruction:
Can you create a Python 3 function that
describes the VIIRS instrument geometry I - band by default.

Response:
def viirs(scans_nb, scan_indices=slice(0, None),
          chn_pixels=6400, scan_lines=32, scan_step=1):
    """Describe VIIRS instrument geometry, I-band by default.
    VIIRS scans several lines simultaneously (there are 16 detectors for each
    M-band, 32 detectors for each I-band) so the scan angles (and times) are
    two-dimensional arrays, contrary to AVHRR for example.

    scan_step: The increment in number of scans. E.g. if scan_step is 100 and
               the number of scans (scans_nb) is 10 then these 10 scans are
               distributed over the swath so that between each scan there are
               99 emtpy (excluded) scans

    """

    entire_width = np.arange(chn_pixels)
    scan_points = entire_width[scan_indices.astype('int')]
    scan_pixels = len(scan_points)

    ''' initial angle 55.84 deg replaced with 56.28 deg found in
    VIIRS User's Guide from NESDIS, version 1.2 (09/10/2013).
    Ref : NOAA Technical Report NESDIS 142.
    Seems to be better (not quantified)'''
    across_track = \
        (scan_points / (chn_pixels / 2. - 0.5) - 1) * np.deg2rad(-56.28)
    y_max_angle = np.arctan2(11.87 / 2, 824.0)
    along_track = \
        -(np.arange(scan_lines) / (scan_lines / 2. - 0.5) - 1) * \
        y_max_angle
    scan = np.dstack((np.tile(across_track, (scan_lines, 1)).T,
                      np.tile(along_track, (scan_pixels, 1))))
    npp = np.tile(scan, [scans_nb, 1]).T

    # from the timestamp in the filenames, a granule takes 1:25.400 to record
    # (85.4 seconds) so 1.779166667 would be the duration of 1 scanline (48
    # scans per granule) dividing the duration of a single scan by a width of
    # 6400 pixels results in 0.0002779947917 seconds for each column of 32
    # pixels in the scanline

    # the individual times per pixel are probably wrong, unless the scanning
    # behaves the same as for AVHRR, The VIIRS sensor rotates to allow internal
    # calibration before each scanline. This would imply that the scanline
    # always moves in the same direction.  more info @
    # http://www.eoportal.org/directory/pres_NPOESSNationalPolarorbitingOperationalEnvironmentalSatelliteSystem.html

    SEC_EACH_SCANCOLUMN = 0.0002779947917
    sec_scan_duration = 1.779166667
    times = np.tile(scan_points * SEC_EACH_SCANCOLUMN,
                    [np.int(scans_nb*scan_lines), 1])
    offset = np.repeat(np.arange(scans_nb) *
                       sec_scan_duration*scan_step, scan_lines)
    times += np.expand_dims(offset, 1)

    # build the scan geometry object
    return ScanGeometry(npp, times)


Instruction:
Can you create a Python 3 function that
describes HIRS 4 instrument geometry.

Response:
def hirs4(scans_nb, edges_only=False):
    """Describe HIRS/4 instrument geometry.

    See:
    - https://www.eumetsat.int/website/home/Satellites/CurrentSatellites/Metop/MetopDesign/HIRS/index.html
    - https://www1.ncdc.noaa.gov/pub/data/satellite/publications/podguides/
          N-15%20thru%20N-19/pdf/0.0%20NOAA%20KLM%20Users%20Guide.pdf
      (NOAA KLM Users Guide –August 2014 Revision)

    Parameters:
       scans_nb | int -  number of scan lines

     Keywords:
     * edges_only - use only edge pixels

    Returns:
       pyorbital.geoloc.ScanGeometry object

    """

    scan_len = 56  # 56 samples per scan
    scan_rate = 6.4  # single scan, seconds
    scan_angle = -49.5  # swath, degrees
    sampling_interval = abs(scan_rate) / scan_len  # single view, seconds

    if edges_only:
        scan_points = np.array([0, scan_len - 1])
    else:
        scan_points = np.arange(0, scan_len)

    # build the instrument (scan angles)
    samples = np.vstack(((scan_points / (scan_len * 0.5 - 0.5) - 1)
                         * np.deg2rad(scan_angle),
                         np.zeros((len(scan_points),))))
    samples = np.tile(samples[:, np.newaxis, :], [1, np.int(scans_nb), 1])

    # building the corresponding times array
    offset = np.arange(scans_nb) * scan_rate
    times = (np.tile(scan_points * sampling_interval, [np.int(scans_nb), 1])
             + np.expand_dims(offset, 1))

    # build the scan geometry object
    return ScanGeometry(samples, times)


Instruction:
Explain what the following Python 3 code does
def olci(scans_nb, scan_points=None):

    if scan_points is None:
        scan_len = 4000  # samples per scan
        scan_points = np.arange(4000)
    else:
        scan_len = len(scan_points)
    # scan_rate = 0.044  # single scan, seconds
    scan_angle_west = 46.5  # swath, degrees
    scan_angle_east = -22.1  # swath, degrees
    # sampling_interval = 18e-3  # single view, seconds
    # build the olci instrument scan line angles
    scanline_angles = np.linspace(np.deg2rad(scan_angle_west),
                                  np.deg2rad(scan_angle_east), scan_len)
    inst = np.vstack((scanline_angles, np.zeros(scan_len,)))

    inst = np.tile(inst[:, np.newaxis, :], [1, np.int(scans_nb), 1])

    # building the corresponding times array
    # times = (np.tile(scan_points * 0.000025 + 0.0025415, [scans_nb, 1])
    #         + np.expand_dims(offset, 1))

    times = np.tile(np.zeros_like(scanline_angles), [np.int(scans_nb), 1])
    # if apply_offset:
    #     offset = np.arange(np.int(scans_nb)) * frequency
    #     times += np.expand_dims(offset, 1)

    return ScanGeometry(inst, times)

Response: Return a ScanGeometry object for the OLCI instrument.


Instruction:
Write a Python 3 script for
reading platform numbers from the PPP_CONFIG_DIR.

Response:
def read_platform_numbers(in_upper=False, num_as_int=False):
    """Read platform numbers from $PPP_CONFIG_DIR/platforms.txt if available."""
    out_dict = {}
    os.getenv('PPP_CONFIG_DIR', PKG_CONFIG_DIR)
    platform_file = None
    if 'PPP_CONFIG_DIR' in os.environ:
        platform_file = os.path.join(os.environ['PPP_CONFIG_DIR'], 'platforms.txt')
    if not platform_file or not os.path.isfile(platform_file):
        platform_file = os.path.join(PKG_CONFIG_DIR, 'platforms.txt')

    try:
        fid = open(platform_file, 'r')
    except IOError:
        LOGGER.error("Platform file %s not found.", platform_file)
        return out_dict
    for row in fid:
        # skip comment lines
        if not row.startswith('#'):
            parts = row.split()
            if len(parts) < 2:
                continue
            if in_upper:
                parts[0] = parts[0].upper()
            if num_as_int:
                parts[1] = int(parts[1])
            out_dict[parts[0]] = parts[1]
    fid.close()

    return out_dict


Instruction:
How would you code a function in Python 3 to
read TLE from tle_file line1 and line2.

Response:
def read(platform, tle_file=None, line1=None, line2=None):
    """Read TLE for `platform` from `tle_file`

    File is read from `line1` to `line2`, from the newest file provided in the
    TLES pattern, or from internet if none is provided.
    """
    return Tle(platform, tle_file=tle_file, line1=line1, line2=line2)


Instruction:
Can you implement a function in Python 3 that
fetches TLE from internet and save it to destination.

Response:
def fetch(destination):
    """Fetch TLE from internet and save it to `destination`."""
    with io.open(destination, mode="w", encoding="utf-8") as dest:
        for url in TLE_URLS:
            response = urlopen(url)
            dest.write(response.read().decode("utf-8"))


Instruction:
Write a Python 3 script for
performing the checksum for the current TLE.

Response:
def _checksum(self):
        """Performs the checksum for the current TLE."""
        for line in [self._line1, self._line2]:
            check = 0
            for char in line[:-1]:
                if char.isdigit():
                    check += int(char)
                if char == "-":
                    check += 1

            if (check % 10) != int(line[-1]):
                raise ChecksumError(self._platform + " " + line)


Instruction:
Implement a function in Python 3 to
parse the TLE data from the Aleph file.

Response:
def _parse_tle(self):
        """Parse values from TLE data."""

        def _read_tle_decimal(rep):
            """Convert *rep* to decimal value."""
            if rep[0] in ["-", " ", "+"]:
                digits = rep[1:-2].strip()
                val = rep[0] + "." + digits + "e" + rep[-2:]
            else:
                digits = rep[:-2].strip()
                val = "." + digits + "e" + rep[-2:]

            return float(val)

        self.satnumber = self._line1[2:7]
        self.classification = self._line1[7]
        self.id_launch_year = self._line1[9:11]
        self.id_launch_number = self._line1[11:14]
        self.id_launch_piece = self._line1[14:17]
        self.epoch_year = self._line1[18:20]
        self.epoch_day = float(self._line1[20:32])
        self.epoch = \
            np.datetime64(datetime.datetime.strptime(self.epoch_year, "%y") +
                          datetime.timedelta(days=self.epoch_day - 1), 'us')
        self.mean_motion_derivative = float(self._line1[33:43])
        self.mean_motion_sec_derivative = _read_tle_decimal(self._line1[44:52])
        self.bstar = _read_tle_decimal(self._line1[53:61])
        try:
            self.ephemeris_type = int(self._line1[62])
        except ValueError:
            self.ephemeris_type = 0
        self.element_number = int(self._line1[64:68])

        self.inclination = float(self._line2[8:16])
        self.right_ascension = float(self._line2[17:25])
        self.excentricity = int(self._line2[26:33]) * 10 ** -7
        self.arg_perigee = float(self._line2[34:42])
        self.mean_anomaly = float(self._line2[43:51])
        self.mean_motion = float(self._line2[52:63])
        self.orbit = int(self._line2[63:68])


Instruction:
Here you have a function in Python 3, explain what it does
def fix_chrome_mac_platform(platform):
    ver = platform.split('OS X ')[1]
    build_range = range(*MACOSX_CHROME_BUILD_RANGE[ver])
    build = choice(build_range)
    mac_ver = ver.replace('.', '_') + '_' + str(build)
    return 'Macintosh; Intel Mac OS X %s' % mac_ver

Response: Fix Chrome on Mac OS X 10. 11. 6 platform.


Instruction:
Can you write a function in Python 3 where it
builds random system components for given os_id and navigator_id.

Response:
def build_system_components(device_type, os_id, navigator_id):
    """
    For given os_id build random platform and oscpu
    components

    Returns dict {platform_version, platform, ua_platform, oscpu}

    platform_version is OS name used in different places
    ua_platform goes to navigator.platform
    platform is used in building navigator.userAgent
    oscpu goes to navigator.oscpu
    """

    if os_id == 'win':
        platform_version = choice(OS_PLATFORM['win'])
        cpu = choice(OS_CPU['win'])
        if cpu:
            platform = '%s; %s' % (platform_version, cpu)
        else:
            platform = platform_version
        res = {
            'platform_version': platform_version,
            'platform': platform,
            'ua_platform': platform,
            'oscpu': platform,
        }
    elif os_id == 'linux':
        cpu = choice(OS_CPU['linux'])
        platform_version = choice(OS_PLATFORM['linux'])
        platform = '%s %s' % (platform_version, cpu)
        res = {
            'platform_version': platform_version,
            'platform': platform,
            'ua_platform': platform,
            'oscpu': 'Linux %s' % cpu,
        }
    elif os_id == 'mac':
        cpu = choice(OS_CPU['mac'])
        platform_version = choice(OS_PLATFORM['mac'])
        platform = platform_version
        if navigator_id == 'chrome':
            platform = fix_chrome_mac_platform(platform)
        res = {
            'platform_version': platform_version,
            'platform': 'MacIntel',
            'ua_platform': platform,
            'oscpu': 'Intel Mac OS X %s' % platform.split(' ')[-1],
        }
    elif os_id == 'android':
        assert navigator_id in ('firefox', 'chrome')
        assert device_type in ('smartphone', 'tablet')
        platform_version = choice(OS_PLATFORM['android'])
        if navigator_id == 'firefox':
            if device_type == 'smartphone':
                ua_platform = '%s; Mobile' % platform_version
            elif device_type == 'tablet':
                ua_platform = '%s; Tablet' % platform_version
        elif navigator_id == 'chrome':
            device_id = choice(SMARTPHONE_DEV_IDS)
            ua_platform = 'Linux; %s; %s' % (platform_version, device_id)
        oscpu = 'Linux %s' % choice(OS_CPU['android'])
        res = {
            'platform_version': platform_version,
            'ua_platform': ua_platform,
            'platform': oscpu,
            'oscpu': oscpu,
        }
    return res


Instruction:
Implement a function in Python 3 to
build app components for given navigator_id

Response:
def build_app_components(os_id, navigator_id):
    """
    For given navigator_id build app features

    Returns dict {name, product_sub, vendor, build_version, build_id}
    """

    if navigator_id == 'firefox':
        build_version, build_id = get_firefox_build()
        if os_id in ('win', 'linux', 'mac'):
            geckotrail = '20100101'
        else:
            geckotrail = build_version
        res = {
            'name': 'Netscape',
            'product_sub': '20100101',
            'vendor': '',
            'build_version': build_version,
            'build_id': build_id,
            'geckotrail': geckotrail,
        }
    elif navigator_id == 'chrome':
        res = {
            'name': 'Netscape',
            'product_sub': '20030107',
            'vendor': 'Google Inc.',
            'build_version': get_chrome_build(),
            'build_id': None,
        }
    elif navigator_id == 'ie':
        num_ver, build_version, trident_version = get_ie_build()
        if num_ver >= 11:
            app_name = 'Netscape'
        else:
            app_name = 'Microsoft Internet Explorer'
        res = {
            'name': app_name,
            'product_sub': None,
            'vendor': '',
            'build_version': build_version,
            'build_id': None,
            'trident_version': trident_version,
        }
    return res


Instruction:
Can you tell what is the following Python 3 function doing
def get_option_choices(opt_name, opt_value, default_value, all_choices):

    choices = []
    if isinstance(opt_value, six.string_types):
        choices = [opt_value]
    elif isinstance(opt_value, (list, tuple)):
        choices = list(opt_value)
    elif opt_value is None:
        choices = default_value
    else:
        raise InvalidOption('Option %s has invalid'
                            ' value: %s' % (opt_name, opt_value))
    if 'all' in choices:
        choices = all_choices
    for item in choices:
        if item not in all_choices:
            raise InvalidOption('Choices of option %s contains invalid'
                                ' item: %s' % (opt_name, item))
    return choices

Response: Generates possible choices for the option opt_name opt_value with default value as default_value as all_choices


Instruction:
Can you create a Python 3 function that
selects one random pair of config ids from the given os and navigator filters.

Response:
def pick_config_ids(device_type, os, navigator):
    """
    Select one random pair (device_type, os_id, navigator_id) from
    all possible combinations matching the given os and
    navigator filters.

    :param os: allowed os(es)
    :type os: string or list/tuple or None
    :param navigator: allowed browser engine(s)
    :type navigator: string or list/tuple or None
    :param device_type: limit possible oses by device type
    :type device_type: list/tuple or None, possible values:
        "desktop", "smartphone", "tablet", "all"
    """

    if os is None:
        default_dev_types = ['desktop']
    else:
        default_dev_types = list(DEVICE_TYPE_OS.keys())
    dev_type_choices = get_option_choices(
        'device_type', device_type, default_dev_types,
        list(DEVICE_TYPE_OS.keys())
    )
    os_choices = get_option_choices('os', os, list(OS_NAVIGATOR.keys()),
                                    list(OS_NAVIGATOR.keys()))
    nav_choices = get_option_choices('navigator', navigator,
                                     list(NAVIGATOR_OS.keys()),
                                     list(NAVIGATOR_OS.keys()))

    variants = []
    for dev, os, nav in product(dev_type_choices, os_choices,
                                nav_choices):

        if (os in DEVICE_TYPE_OS[dev]
                and nav in DEVICE_TYPE_NAVIGATOR[dev]
                and nav in OS_NAVIGATOR[os]):
            variants.append((dev, os, nav))
    if not variants:
        raise InvalidOption('Options device_type, os and navigator'
                            ' conflicts with each other')
    device_type, os_id, navigator_id = choice(variants)

    assert os_id in OS_PLATFORM
    assert navigator_id in NAVIGATOR_OS
    assert device_type in DEVICE_TYPE_OS

    return device_type, os_id, navigator_id


Instruction:
Write a Python 3 function that can
generate a web navigator s config.

Response:
def generate_navigator(os=None, navigator=None, platform=None,
                       device_type=None):
    """
    Generates web navigator's config

    :param os: limit list of oses for generation
    :type os: string or list/tuple or None
    :param navigator: limit list of browser engines for generation
    :type navigator: string or list/tuple or None
    :param device_type: limit possible oses by device type
    :type device_type: list/tuple or None, possible values:
        "desktop", "smartphone", "tablet", "all"

    :return: User-Agent config
    :rtype: dict with keys (os, name, platform, oscpu, build_version,
                            build_id, app_version, app_name, app_code_name,
                            product, product_sub, vendor, vendor_sub,
                            user_agent)
    :raises InvalidOption: if could not generate user-agent for
        any combination of allowed platforms and navigators
    :raise InvalidOption: if any of passed options is invalid
    """

    if platform is not None:
        os = platform
        warn('The `platform` option is deprecated.'
             ' Use `os` option instead.', stacklevel=3)
    device_type, os_id, navigator_id = (
        pick_config_ids(device_type, os, navigator)
    )
    system = build_system_components(
        device_type, os_id, navigator_id)
    app = build_app_components(os_id, navigator_id)
    ua_template = choose_ua_template(
        device_type, navigator_id, app)
    user_agent = ua_template.format(system=system, app=app)
    app_version = build_navigator_app_version(
        os_id, navigator_id, system['platform_version'], user_agent)
    return {
        # ids
        'os_id': os_id,
        'navigator_id': navigator_id,
        # system components
        'platform': system['platform'],
        'oscpu': system['oscpu'],
        # app components
        'build_version': app['build_version'],
        'build_id': app['build_id'],
        'app_version': app_version,
        'app_name': app['name'],
        'app_code_name': 'Mozilla',
        'product': 'Gecko',
        'product_sub': app['product_sub'],
        'vendor': app['vendor'],
        'vendor_sub': '',
        # compiled user agent
        'user_agent': user_agent,
    }