Instruction:
Create a Python 3 function to
log the given |IoSequence| to a suitable instance of and a |InfoArray| containing all of the information about the resulting object structure.

Response:
def log(self, sequence, infoarray) -> None:
        """Pass the given |IoSequence| to a suitable instance of
        a |NetCDFVariableBase| subclass.

        When writing data, the second argument should be an |InfoArray|.
        When reading data, this argument is ignored. Simply pass |None|.

        (1) We prepare some devices handling some sequences by applying
        function |prepare_io_example_1|.  We limit our attention to the
        returned elements, which handle the more diverse sequences:

        >>> from hydpy.core.examples import prepare_io_example_1
        >>> nodes, (element1, element2, element3) = prepare_io_example_1()

        (2) We define some shortcuts for the sequences used in the
        following examples:

        >>> nied1 = element1.model.sequences.inputs.nied
        >>> nied2 = element2.model.sequences.inputs.nied
        >>> nkor2 = element2.model.sequences.fluxes.nkor
        >>> nkor3 = element3.model.sequences.fluxes.nkor

        (3) We define a function that logs these example sequences
        to a given |NetCDFFile| object and prints some information
        about the resulting object structure.  Note that sequence
        `nkor2` is logged twice, the first time with its original
        time series data, the second time with averaged values:

        >>> from hydpy import classname
        >>> def test(ncfile):
        ...     ncfile.log(nied1, nied1.series)
        ...     ncfile.log(nied2, nied2.series)
        ...     ncfile.log(nkor2, nkor2.series)
        ...     ncfile.log(nkor2, nkor2.average_series())
        ...     ncfile.log(nkor3, nkor3.average_series())
        ...     for name, variable in ncfile.variables.items():
        ...         print(name, classname(variable), variable.subdevicenames)

        (4) We prepare a |NetCDFFile| object with both options
        `flatten` and `isolate` being disabled:

        >>> from hydpy.core.netcdftools import NetCDFFile
        >>> ncfile = NetCDFFile(
        ...     'model', flatten=False, isolate=False, timeaxis=1, dirpath='')

        (5) We log all test sequences results in two |NetCDFVariableDeep|
        and one |NetCDFVariableAgg| objects.  To keep both NetCDF variables
        related to |lland_fluxes.NKor| distinguishable, the name
        `flux_nkor_mean` includes information about the kind of aggregation
        performed:

        >>> test(ncfile)
        input_nied NetCDFVariableDeep ('element1', 'element2')
        flux_nkor NetCDFVariableDeep ('element2',)
        flux_nkor_mean NetCDFVariableAgg ('element2', 'element3')

        (6) We confirm that the |NetCDFVariableBase| objects received
        the required information:

        >>> ncfile.flux_nkor.element2.sequence.descr_device
        'element2'
        >>> ncfile.flux_nkor.element2.array
        InfoArray([[ 16.,  17.],
                   [ 18.,  19.],
                   [ 20.,  21.],
                   [ 22.,  23.]])
        >>> ncfile.flux_nkor_mean.element2.sequence.descr_device
        'element2'
        >>> ncfile.flux_nkor_mean.element2.array
        InfoArray([ 16.5,  18.5,  20.5,  22.5])

        (7) We again prepare a |NetCDFFile| object, but now with both
        options `flatten` and `isolate` being enabled.  To log test
        sequences with their original time series data does now trigger
        the initialisation of class |NetCDFVariableFlat|.  When passing
        aggregated data, nothing changes:

        >>> ncfile = NetCDFFile(
        ...     'model', flatten=True, isolate=True, timeaxis=1, dirpath='')
        >>> test(ncfile)
        input_nied NetCDFVariableFlat ('element1', 'element2')
        flux_nkor NetCDFVariableFlat ('element2_0', 'element2_1')
        flux_nkor_mean NetCDFVariableAgg ('element2', 'element3')
        >>> ncfile.flux_nkor.element2.sequence.descr_device
        'element2'
        >>> ncfile.flux_nkor.element2.array
        InfoArray([[ 16.,  17.],
                   [ 18.,  19.],
                   [ 20.,  21.],
                   [ 22.,  23.]])
        >>> ncfile.flux_nkor_mean.element2.sequence.descr_device
        'element2'
        >>> ncfile.flux_nkor_mean.element2.array
        InfoArray([ 16.5,  18.5,  20.5,  22.5])

        (8) We technically confirm that the `isolate` argument is passed
        to the constructor of subclasses of |NetCDFVariableBase| correctly:

        >>> from unittest.mock import patch
        >>> with patch('hydpy.core.netcdftools.NetCDFVariableFlat') as mock:
        ...     ncfile = NetCDFFile(
        ...         'model', flatten=True, isolate=False, timeaxis=0,
        ...         dirpath='')
        ...     ncfile.log(nied1, nied1.series)
        ...     mock.assert_called_once_with(
        ...         name='input_nied', timeaxis=0, isolate=False)
        """
        aggregated = ((infoarray is not None) and
                      (infoarray.info['type'] != 'unmodified'))
        descr = sequence.descr_sequence
        if aggregated:
            descr = '_'.join([descr, infoarray.info['type']])
        if descr in self.variables:
            var_ = self.variables[descr]
        else:
            if aggregated:
                cls = NetCDFVariableAgg
            elif self._flatten:
                cls = NetCDFVariableFlat
            else:
                cls = NetCDFVariableDeep
            var_ = cls(name=descr,
                       isolate=self._isolate,
                       timeaxis=self._timeaxis)
            self.variables[descr] = var_
        var_.log(sequence, infoarray)


Instruction:
How would you implement a function in Python 3 that
opens an existing NetCDF file temporarily and call method Quick. read of all handled |NetCDFVariableBase| objects.

Response:
def read(self) -> None:
        """Open an existing NetCDF file temporarily and call method
        |NetCDFVariableDeep.read| of all handled |NetCDFVariableBase|
        objects."""
        try:
            with netcdf4.Dataset(self.filepath, "r") as ncfile:
                timegrid = query_timegrid(ncfile)
                for variable in self.variables.values():
                    variable.read(ncfile, timegrid)
        except BaseException:
            objecttools.augment_excmessage(
                f'While trying to read data from NetCDF file `{self.filepath}`')


Instruction:
Can you generate a brief explanation for the following Python 3 code
def write(self, timeunit, timepoints) -> None:
        with netcdf4.Dataset(self.filepath, "w") as ncfile:
            ncfile.Conventions = 'CF-1.6'
            self._insert_timepoints(ncfile, timepoints, timeunit)
            for variable in self.variables.values():
                variable.write(ncfile)

Response: Write |NetCDFVariableBase. write| of all handled |NetCDFVariableBase| objects.


Instruction:
How would you implement a function in Python 3 that
gets the index of the object with a specialized error message.

Response:
def get_index(self, name_subdevice) -> int:
        """Item access to the wrapped |dict| object with a specialized
        error message."""
        try:
            return self.dict_[name_subdevice]
        except KeyError:
            raise OSError(
                'No data for sequence `%s` and (sub)device `%s` '
                'in NetCDF file `%s` available.'
                % (self.name_sequence,
                   name_subdevice,
                   self.name_ncfile))


Instruction:
Create a Python 3 function for
logging the given |IOSequence| object either for reading or writing or a modified time series.

Response:
def log(self, sequence, infoarray) -> None:
        """Log the given |IOSequence| object either for reading or writing
        data.

        The optional `array` argument allows for passing alternative data
        in an |InfoArray| object replacing the series of the |IOSequence|
        object, which is useful for writing modified (e.g. spatially
        averaged) time series.

        Logged time series data is available via attribute access:

        >>> from hydpy.core.netcdftools import NetCDFVariableBase
        >>> from hydpy import make_abc_testable
        >>> NCVar = make_abc_testable(NetCDFVariableBase)
        >>> ncvar = NCVar('flux_nkor', isolate=True, timeaxis=1)
        >>> from hydpy.core.examples import prepare_io_example_1
        >>> nodes, elements = prepare_io_example_1()
        >>> nkor = elements.element1.model.sequences.fluxes.nkor
        >>> ncvar.log(nkor, nkor.series)
        >>> 'element1' in dir(ncvar)
        True
        >>> ncvar.element1.sequence is nkor
        True
        >>> 'element2' in dir(ncvar)
        False
        >>> ncvar.element2
        Traceback (most recent call last):
        ...
        AttributeError: The NetCDFVariable object `flux_nkor` does \
neither handle time series data under the (sub)device name `element2` \
nor does it define a member named `element2`.
        """
        descr_device = sequence.descr_device
        self.sequences[descr_device] = sequence
        self.arrays[descr_device] = infoarray


Instruction:
Can you generate a brief explanation for the following Python 3 code
def insert_subdevices(self, ncfile) -> None:
        prefix = self.prefix
        nmb_subdevices = '%s%s' % (prefix, dimmapping['nmb_subdevices'])
        nmb_characters = '%s%s' % (prefix, dimmapping['nmb_characters'])
        subdevices = '%s%s' % (prefix, varmapping['subdevices'])
        statchars = str2chars(self.subdevicenames)
        create_dimension(ncfile, nmb_subdevices, statchars.shape[0])
        create_dimension(ncfile, nmb_characters, statchars.shape[1])
        create_variable(
            ncfile, subdevices, 'S1', (nmb_subdevices, nmb_characters))
        ncfile[subdevices][:, :] = statchars

Response:
Insert a variable of the names of the subdevices of the logged
        sequences into the given NetCDF file.


Instruction:
Write a Python 3 script to
query the names of the subdevices of the logged sequences from the given NetCDF file.

Response:
def query_subdevices(self, ncfile) -> List[str]:
        """Query the names of the (sub)devices of the logged sequences
        from the given NetCDF file

        (1) We apply function |NetCDFVariableBase.query_subdevices| on
        an empty NetCDF file.  The error message shows that the method
        tries to query the (sub)device names both under the assumptions
        that variables have been isolated or not:

        >>> from hydpy.core.netcdftools import NetCDFVariableBase
        >>> from hydpy import make_abc_testable, TestIO
        >>> from hydpy.core.netcdftools import netcdf4
        >>> with TestIO():
        ...     ncfile = netcdf4.Dataset('model.nc', 'w')
        >>> Var = make_abc_testable(NetCDFVariableBase)
        >>> Var.subdevicenames = 'element1', 'element_2'
        >>> var = Var('flux_prec', isolate=False, timeaxis=1)
        >>> var.query_subdevices(ncfile)
        Traceback (most recent call last):
        ...
        OSError: NetCDF file `model.nc` does neither contain a variable \
named `flux_prec_station_id` nor `station_id` for defining the \
coordinate locations of variable `flux_prec`.

        (2) After inserting the (sub)device name, they can be queried
        and returned:

        >>> var.insert_subdevices(ncfile)
        >>> Var('flux_prec', isolate=False, timeaxis=1).query_subdevices(ncfile)
        ['element1', 'element_2']
        >>> Var('flux_prec', isolate=True, timeaxis=1).query_subdevices(ncfile)
        ['element1', 'element_2']

        >>> ncfile.close()
        """
        tests = ['%s%s' % (prefix, varmapping['subdevices'])
                 for prefix in ('%s_' % self.name, '')]
        for subdevices in tests:
            try:
                chars = ncfile[subdevices][:]
                break
            except (IndexError, KeyError):
                pass
        else:
            raise IOError(
                'NetCDF file `%s` does neither contain a variable '
                'named `%s` nor `%s` for defining the coordinate '
                'locations of variable `%s`.'
                % (get_filepath(ncfile), tests[0], tests[1], self.name))
        return chars2str(chars)


Instruction:
Create a Python 3 function for
querying |NetCDFVariableBase. query_subdevices| and return |Subdevice2Index| object that maps the sub device names to their position within the given NetCDF file.

Response:
def query_subdevice2index(self, ncfile) -> Subdevice2Index:
        """Return a |Subdevice2Index| that maps the (sub)device names to
        their position within the given NetCDF file.

        Method |NetCDFVariableBase.query_subdevice2index| is based on
        |NetCDFVariableBase.query_subdevices|.  The returned
        |Subdevice2Index| object remembers the NetCDF file the
        (sub)device names stem from, allowing for clear error messages:

        >>> from hydpy.core.netcdftools import NetCDFVariableBase, str2chars
        >>> from hydpy import make_abc_testable, TestIO
        >>> from hydpy.core.netcdftools import netcdf4
        >>> with TestIO():
        ...     ncfile = netcdf4.Dataset('model.nc', 'w')
        >>> Var = make_abc_testable(NetCDFVariableBase)
        >>> Var.subdevicenames = [
        ...     'element3', 'element1', 'element1_1', 'element2']
        >>> var = Var('flux_prec', isolate=True, timeaxis=1)
        >>> var.insert_subdevices(ncfile)
        >>> subdevice2index = var.query_subdevice2index(ncfile)
        >>> subdevice2index.get_index('element1_1')
        2
        >>> subdevice2index.get_index('element3')
        0
        >>> subdevice2index.get_index('element5')
        Traceback (most recent call last):
        ...
        OSError: No data for sequence `flux_prec` and (sub)device \
`element5` in NetCDF file `model.nc` available.

        Additionally, |NetCDFVariableBase.query_subdevice2index|
        checks for duplicates:

        >>> ncfile['station_id'][:] = str2chars(
        ...     ['element3', 'element1', 'element1_1', 'element1'])
        >>> var.query_subdevice2index(ncfile)
        Traceback (most recent call last):
        ...
        OSError: The NetCDF file `model.nc` contains duplicate (sub)device \
names for variable `flux_prec` (the first found duplicate is `element1`).

        >>> ncfile.close()
        """
        subdevices = self.query_subdevices(ncfile)
        self._test_duplicate_exists(ncfile, subdevices)
        subdev2index = {subdev: idx for (idx, subdev) in enumerate(subdevices)}
        return Subdevice2Index(subdev2index, self.name, get_filepath(ncfile))


Instruction:
Make a summary of the following Python 3 code
def sort_timeplaceentries(self, timeentry, placeentry) -> Tuple[Any, Any]:
        if self._timeaxis:
            return placeentry, timeentry
        return timeentry, placeentry

Response: Return a |tuple| containing the given timeentry and placeentry sorted in agreement with the currently selected timeaxis.


Instruction:
Implement a function in Python 3 to
return a |tuple| for indexing a complete time series of a certain location available in |NetCDFVariableBase. array|.

Response:
def get_timeplaceslice(self, placeindex) -> \
            Union[Tuple[slice, int], Tuple[int, slice]]:
        """Return a |tuple| for indexing a complete time series of a certain
        location available in |NetCDFVariableBase.array|.

        >>> from hydpy.core.netcdftools import NetCDFVariableBase
        >>> from hydpy import make_abc_testable
        >>> NCVar = make_abc_testable(NetCDFVariableBase)
        >>> ncvar = NCVar('flux_nkor', isolate=True, timeaxis=1)
        >>> ncvar.get_timeplaceslice(2)
        (2, slice(None, None, None))
        >>> ncvar = NetCDFVariableDeep('test', isolate=False, timeaxis=0)
        >>> ncvar.get_timeplaceslice(2)
        (slice(None, None, None), 2)
        """
        return self.sort_timeplaceentries(slice(None), int(placeindex))


Instruction:
Here you have a function in Python 3, explain what it does
def subdevicenames(self) -> Tuple[str, ...]:
        self: NetCDFVariableBase
        return tuple(self.sequences.keys())

Response: A |tuple| containing the device names.


Instruction:
Given the following Python 3 function, write the documentation
def write(self, ncfile) -> None:
        self: NetCDFVariableBase
        self.insert_subdevices(ncfile)
        dimensions = self.dimensions
        array = self.array
        for dimension, length in zip(dimensions[2:], array.shape[2:]):
            create_dimension(ncfile, dimension, length)
        create_variable(ncfile, self.name, 'f8', dimensions)
        ncfile[self.name][:] = array

Response: Write the data to the given NetCDF file.


Instruction:
Make a summary of the following Python 3 code
def get_slices(self, idx, shape) -> Tuple[IntOrSlice, ...]:
        slices = list(self.get_timeplaceslice(idx))
        for length in shape:
            slices.append(slice(0, length))
        return tuple(slices)

Response:
Return a |tuple| of one int and some |slice| objects to
        accesses all values of a certain device within the given |NetCDFVariableDeep. array|.


Instruction:
Can you create a Python 3 function that
requires shape of |NetCDFVariableDeep. array|.

Response:
def shape(self) -> Tuple[int, ...]:
        """Required shape of |NetCDFVariableDeep.array|.

        For the default configuration, the first axis corresponds to the
        number of devices, and the second one to the number of timesteps.
        We show this for the 0-dimensional input sequence |lland_inputs.Nied|:

        >>> from hydpy.core.examples import prepare_io_example_1
        >>> nodes, elements = prepare_io_example_1()
        >>> from hydpy.core.netcdftools import NetCDFVariableDeep
        >>> ncvar = NetCDFVariableDeep('input_nied', isolate=False, timeaxis=1)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.inputs.nied, None)
        >>> ncvar.shape
        (3, 4)

        For higher dimensional sequences, each new entry corresponds
        to the maximum number of fields the respective sequences require.
        In the next example, we select the 1-dimensional sequence
        |lland_fluxes.NKor|.  The maximum number 3 (last value of the
        returned |tuple|) is due to the third element defining three
        hydrological response units:

        >>> ncvar = NetCDFVariableDeep('flux_nkor', isolate=False, timeaxis=1)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.fluxes.nkor, None)
        >>> ncvar.shape
        (3, 4, 3)

        When using the first axis for time (`timeaxis=0`) the order of the
        first two |tuple| entries turns:

        >>> ncvar = NetCDFVariableDeep('flux_nkor', isolate=False, timeaxis=0)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.fluxes.nkor, None)
        >>> ncvar.shape
        (4, 3, 3)
        """
        nmb_place = len(self.sequences)
        nmb_time = len(hydpy.pub.timegrids.init)
        nmb_others = collections.deque()
        for sequence in self.sequences.values():
            nmb_others.append(sequence.shape)
        nmb_others_max = tuple(numpy.max(nmb_others, axis=0))
        return self.sort_timeplaceentries(nmb_time, nmb_place) + nmb_others_max


Instruction:
Given the following Python 3 function, write the documentation
def dimensions(self) -> Tuple[str, ...]:
        nmb_timepoints = dimmapping['nmb_timepoints']
        nmb_subdevices = '%s%s' % (self.prefix, dimmapping['nmb_subdevices'])
        dimensions = list(self.sort_timeplaceentries(
            nmb_timepoints, nmb_subdevices))
        for idx in range(list(self.sequences.values())[0].NDIM):
            dimensions.append('%saxis%d' % (self.prefix, idx + 3))
        return tuple(dimensions)

Response: The dimension names of the NetCDF variable.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def read(self, ncfile, timegrid_data) -> None:
        array = query_array(ncfile, self.name)
        subdev2index = self.query_subdevice2index(ncfile)
        for subdevice, sequence in self.sequences.items():
            idx = subdev2index.get_index(subdevice)
            values = array[self.get_slices(idx, sequence.shape)]
            sequence.series = sequence.adjust_series(
                timegrid_data, values)

Response: Read the data from the given NetCDF file.


Instruction:
Create a Python 3 function to
require shape of |NetCDFVariableAgg. array|.

Response:
def shape(self) -> Tuple[int, int]:
        """Required shape of |NetCDFVariableAgg.array|.

        For the default configuration, the first axis corresponds to the
        number of devices, and the second one to the number of timesteps.
        We show this for the 1-dimensional input sequence |lland_fluxes.NKor|:

        >>> from hydpy.core.examples import prepare_io_example_1
        >>> nodes, elements = prepare_io_example_1()
        >>> from hydpy.core.netcdftools import NetCDFVariableAgg
        >>> ncvar = NetCDFVariableAgg('flux_nkor', isolate=False, timeaxis=1)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.fluxes.nkor, None)
        >>> ncvar.shape
        (3, 4)

        When using the first axis as the "timeaxis", the order of |tuple|
        entries turns:

        >>> ncvar = NetCDFVariableAgg('flux_nkor', isolate=False, timeaxis=0)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.fluxes.nkor, None)
        >>> ncvar.shape
        (4, 3)
        """
        return self.sort_timeplaceentries(
            len(hydpy.pub.timegrids.init), len(self.sequences))


Instruction:
How would you implement a function in Python 3 that
requires shape of |NetCDFVariableFlat. array|.

Response:
def shape(self) -> Tuple[int, int]:
        """Required shape of |NetCDFVariableFlat.array|.

        For 0-dimensional sequences like |lland_inputs.Nied| and for the
        default configuration (`timeaxis=1`), the first axis corresponds
        to the number of devices, and the second one two the number of
        timesteps:

        >>> from hydpy.core.examples import prepare_io_example_1
        >>> nodes, elements = prepare_io_example_1()
        >>> from hydpy.core.netcdftools import NetCDFVariableFlat
        >>> ncvar = NetCDFVariableFlat('input_nied', isolate=False, timeaxis=1)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.inputs.nied, None)
        >>> ncvar.shape
        (3, 4)

        For higher dimensional sequences, the first axis corresponds
        to "subdevices", e.g. hydrological response units within
        different elements.  The 1-dimensional sequence |lland_fluxes.NKor|
        is logged for three elements with one, two, and three response
        units respectively, making up a sum of six subdevices:

        >>> ncvar = NetCDFVariableFlat('flux_nkor', isolate=False, timeaxis=1)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.fluxes.nkor, None)
        >>> ncvar.shape
        (6, 4)

        When using the first axis as the "timeaxis", the order of |tuple|
        entries turns:

        >>> ncvar = NetCDFVariableFlat('flux_nkor', isolate=False, timeaxis=0)
        >>> for element in elements:
        ...     ncvar.log(element.model.sequences.fluxes.nkor, None)
        >>> ncvar.shape
        (4, 6)
        """
        return self.sort_timeplaceentries(
            len(hydpy.pub.timegrids.init),
            sum(len(seq) for seq in self.sequences.values()))


Instruction:
Can you generate a brief explanation for the following Python 3 code
def subdevicenames(self) -> Tuple[str, ...]:
        stats: List[str] = collections.deque()
        for devicename, seq in self.sequences.items():
            if seq.NDIM:
                temp = devicename + '_'
                for prod in self._product(seq.shape):
                    stats.append(temp + '_'.join(str(idx) for idx in prod))
            else:
                stats.append(devicename)
        return tuple(stats)

Response: A |tuple| containing the subdevice names of the |NetCDFVariableFlat| object.


Instruction:
How would you explain what the following Python 3 function does
def read(self, ncfile, timegrid_data) -> None:
        array = query_array(ncfile, self.name)
        idxs: Tuple[Any] = (slice(None),)
        subdev2index = self.query_subdevice2index(ncfile)
        for devicename, seq in self.sequences.items():
            if seq.NDIM:
                if self._timeaxis:
                    subshape = (array.shape[1],) + seq.shape
                else:
                    subshape = (array.shape[0],) + seq.shape
                subarray = numpy.empty(subshape)
                temp = devicename + '_'
                for prod in self._product(seq.shape):
                    station = temp + '_'.join(str(idx) for idx in prod)
                    idx0 = subdev2index.get_index(station)
                    subarray[idxs+prod] = array[self.get_timeplaceslice(idx0)]
            else:
                idx = subdev2index.get_index(devicename)
                subarray = array[self.get_timeplaceslice(idx)]
            seq.series = seq.adjust_series(timegrid_data, subarray)

Response: Read the data from the given NetCDF file.


Instruction:
Make a summary of the following Python 3 code
def write(self, ncfile) -> None:
        self.insert_subdevices(ncfile)
        create_variable(ncfile, self.name, 'f8', self.dimensions)
        ncfile[self.name][:] = self.array

Response: Write the data to the given NetCDF file.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def update(self):
        maxdt = self.subpars.pars.control.maxdt
        seconds = self.simulationstep.seconds
        self.value = numpy.ceil(seconds/maxdt)

Response: Update the internal state of the internal state of the current entry.


Instruction:
How would you implement a function in Python 3 that
updates the internal state of the object with the current values from the auxilary term.

Response:
def update(self):
        """Calulate the auxilary term.

        >>> from hydpy.models.llake import *
        >>> parameterstep('1d')
        >>> simulationstep('12h')
        >>> n(3)
        >>> v(0., 1e5, 1e6)
        >>> q(_1=[0., 1., 2.], _7=[0., 2., 5.])
        >>> maxdt('12h')
        >>> derived.seconds.update()
        >>> derived.nmbsubsteps.update()
        >>> derived.vq.update()
        >>> derived.vq
        vq(toy_1_1_0_0_0=[0.0, 243200.0, 2086400.0],
           toy_7_1_0_0_0=[0.0, 286400.0, 2216000.0])
        """
        con = self.subpars.pars.control
        der = self.subpars
        for (toy, qs) in con.q:
            setattr(self, str(toy), 2.*con.v+der.seconds/der.nmbsubsteps*qs)
        self.refresh()


Instruction:
How would you explain what the following Python 3 function does
def prepare_io_example_1() -> Tuple[devicetools.Nodes, devicetools.Elements]:
    # noinspection PyUnresolvedReferences
    """Prepare an IO example configuration.

    >>> from hydpy.core.examples import prepare_io_example_1
    >>> nodes, elements = prepare_io_example_1()

    (1) Prepares a short initialisation period of five days:

    >>> from hydpy import pub
    >>> pub.timegrids
    Timegrids(Timegrid('2000-01-01 00:00:00',
                       '2000-01-05 00:00:00',
                       '1d'))

    (2) Prepares a plain IO testing directory structure:

    >>> pub.sequencemanager.inputdirpath
    'inputpath'
    >>> pub.sequencemanager.fluxdirpath
    'outputpath'
    >>> pub.sequencemanager.statedirpath
    'outputpath'
    >>> pub.sequencemanager.nodedirpath
    'nodepath'
    >>> import os
    >>> from hydpy import TestIO
    >>> with TestIO():
    ...     print(sorted(filename for filename in os.listdir('.')
    ...                  if not filename.startswith('_')))
    ['inputpath', 'nodepath', 'outputpath']

    (3) Returns three |Element| objects handling either application model
    |lland_v1| or |lland_v2|, and two |Node| objects handling variables
    `Q` and `T`:

    >>> for element in elements:
    ...     print(element.name, element.model)
    element1 lland_v1
    element2 lland_v1
    element3 lland_v2
    >>> for node in nodes:
    ...     print(node.name, node.variable)
    node1 Q
    node2 T

    (4) Prepares the time series data of the input sequence
    |lland_inputs.Nied|, flux sequence |lland_fluxes.NKor|, and state
    sequence |lland_states.BoWa| for each model instance, and |Sim| for
    each node instance (all values are different), e.g.:

    >>> nied1 = elements.element1.model.sequences.inputs.nied
    >>> nied1.series
    InfoArray([ 0.,  1.,  2.,  3.])
    >>> nkor1 = elements.element1.model.sequences.fluxes.nkor
    >>> nkor1.series
    InfoArray([[ 12.],
               [ 13.],
               [ 14.],
               [ 15.]])
    >>> bowa3 = elements.element3.model.sequences.states.bowa
    >>> bowa3.series
    InfoArray([[ 48.,  49.,  50.],
               [ 51.,  52.,  53.],
               [ 54.,  55.,  56.],
               [ 57.,  58.,  59.]])
    >>> sim2 = nodes.node2.sequences.sim
    >>> sim2.series
    InfoArray([ 64.,  65.,  66.,  67.])

    (5) All sequences carry |numpy.ndarray| objects with (deep) copies
    of the time series data for testing:

    >>> import numpy
    >>> (numpy.all(nied1.series == nied1.testarray) and
    ...  numpy.all(nkor1.series == nkor1.testarray) and
    ...  numpy.all(bowa3.series == bowa3.testarray) and
    ...  numpy.all(sim2.series == sim2.testarray))
    InfoArray(True, dtype=bool)
    >>> bowa3.series[1, 2] = -999.0
    >>> numpy.all(bowa3.series == bowa3.testarray)
    InfoArray(False, dtype=bool)
    """
    from hydpy import TestIO
    TestIO.clear()
    from hydpy.core.filetools import SequenceManager
    hydpy.pub.sequencemanager = SequenceManager()
    with TestIO():
        hydpy.pub.sequencemanager.inputdirpath = 'inputpath'
        hydpy.pub.sequencemanager.fluxdirpath = 'outputpath'
        hydpy.pub.sequencemanager.statedirpath = 'outputpath'
        hydpy.pub.sequencemanager.nodedirpath = 'nodepath'

    hydpy.pub.timegrids = '2000-01-01', '2000-01-05', '1d'

    from hydpy import Node, Nodes, Element, Elements, prepare_model
    node1 = Node('node1')
    node2 = Node('node2', variable='T')
    nodes = Nodes(node1, node2)
    element1 = Element('element1', outlets=node1)
    element2 = Element('element2', outlets=node1)
    element3 = Element('element3', outlets=node1)
    elements = Elements(element1, element2, element3)

    from hydpy.models import lland_v1, lland_v2
    element1.model = prepare_model(lland_v1)
    element2.model = prepare_model(lland_v1)
    element3.model = prepare_model(lland_v2)

    from hydpy.models.lland import ACKER
    for idx, element in enumerate(elements):
        parameters = element.model.parameters
        parameters.control.nhru(idx+1)
        parameters.control.lnk(ACKER)
        parameters.derived.absfhru(10.0)

    with hydpy.pub.options.printprogress(False):
        nodes.prepare_simseries()
        elements.prepare_inputseries()
        elements.prepare_fluxseries()
        elements.prepare_stateseries()

    def init_values(seq, value1_):
        value2_ = value1_ + len(seq.series.flatten())
        values_ = numpy.arange(value1_, value2_, dtype=float)
        seq.testarray = values_.reshape(seq.seriesshape)
        seq.series = seq.testarray.copy()
        return value2_

    import numpy
    value1 = 0
    for subname, seqname in zip(['inputs', 'fluxes', 'states'],
                                ['nied', 'nkor', 'bowa']):
        for element in elements:
            subseqs = getattr(element.model.sequences, subname)
            value1 = init_values(getattr(subseqs, seqname), value1)
    for node in nodes:
        value1 = init_values(node.sequences.sim, value1)

    return nodes, elements

Response: Prepare an IO example configuration.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def prepare_full_example_1() -> None:
    testtools.TestIO.clear()
    shutil.copytree(
        os.path.join(data.__path__[0], 'LahnH'),
        os.path.join(iotesting.__path__[0], 'LahnH'))
    seqpath = os.path.join(iotesting.__path__[0], 'LahnH', 'series')
    for folder in ('output', 'node', 'temp'):
        os.makedirs(os.path.join(seqpath, folder))

Response: Prepare the complete LahnH project for testing.


Instruction:
Can you tell what is the following Python 3 function doing
def prepare_full_example_2(lastdate='1996-01-05') -> (
        hydpytools.HydPy, hydpy.pub, testtools.TestIO):
    """Prepare the complete `LahnH` project for testing.

    |prepare_full_example_2| calls |prepare_full_example_1|, but also
    returns a readily prepared |HydPy| instance, as well as module
    |pub| and class |TestIO|, for convenience:

    >>> from hydpy.core.examples import prepare_full_example_2
    >>> hp, pub, TestIO = prepare_full_example_2()
    >>> hp.nodes
    Nodes("dill", "lahn_1", "lahn_2", "lahn_3")
    >>> hp.elements
    Elements("land_dill", "land_lahn_1", "land_lahn_2", "land_lahn_3",
             "stream_dill_lahn_2", "stream_lahn_1_lahn_2",
             "stream_lahn_2_lahn_3")
    >>> pub.timegrids
    Timegrids(Timegrid('1996-01-01 00:00:00',
                       '1996-01-05 00:00:00',
                       '1d'))
    >>> from hydpy import classname
    >>> classname(TestIO)
    'TestIO'

    The last date of the initialisation period is configurable:

    >>> hp, pub, TestIO = prepare_full_example_2('1996-02-01')
    >>> pub.timegrids
    Timegrids(Timegrid('1996-01-01 00:00:00',
                       '1996-02-01 00:00:00',
                       '1d'))
    """
    prepare_full_example_1()
    with testtools.TestIO():
        hp = hydpytools.HydPy('LahnH')
        hydpy.pub.timegrids = '1996-01-01', lastdate, '1d'
        hp.prepare_everything()
    return hp, hydpy.pub, testtools.TestIO

Response: Prepare the complete LahnH project for testing.


Instruction:
Given the following Python 3 function, write the documentation
def get_postalcodes_around_radius(self, pc, radius):
        postalcodes = self.get(pc)
        if postalcodes is None:
            raise PostalCodeNotFoundException("Could not find postal code you're searching for.")
        else:
            pc = postalcodes[0]

        radius = float(radius)

        '''
        Bounding box calculations updated from pyzipcode
        '''
        earth_radius  = 6371
        dlat = radius / earth_radius
        dlon = asin(sin(dlat) / cos(radians(pc.latitude)))
        lat_delta = degrees(dlat)
        lon_delta = degrees(dlon)

        if lat_delta < 0:
            lat_range = (pc.latitude + lat_delta, pc.latitude - lat_delta)
        else:
            lat_range = (pc.latitude - lat_delta, pc.latitude + lat_delta)

        long_range  = (pc.longitude - lat_delta, pc.longitude + lon_delta)

        return format_result(self.conn_manager.query(PC_RANGE_QUERY % (
            long_range[0], long_range[1],
            lat_range[0], lat_range[1]
        )))

Response: Returns a list of postal codes around a radius.


Instruction:
Make a summary of the following Python 3 code
def get_all_player_ids(ids="shots"):
    url = "http://stats.nba.com/stats/commonallplayers?IsOnlyCurrentSeason=0&LeagueID=00&Season=2015-16"

    # get the web page
    response = requests.get(url, headers=HEADERS)
    response.raise_for_status()
    # access 'resultSets', which is a list containing the dict with all the data
    # The 'header' key accesses the headers
    headers = response.json()['resultSets'][0]['headers']
    # The 'rowSet' key contains the player data along with their IDs
    players = response.json()['resultSets'][0]['rowSet']
    # Create dataframe with proper numeric types
    df = pd.DataFrame(players, columns=headers)

    # Dealing with different means of converision for pandas 0.17.0 or 0.17.1
    # and 0.15.0 or loweer
    if '0.17' in pd.__version__:
        # alternative to convert_objects() to numeric to get rid of warning
        # as convert_objects() is deprecated in pandas 0.17.0+
        df = df.apply(pd.to_numeric, args=('ignore',))
    else:
        df = df.convert_objects(convert_numeric=True)

    if ids == "shots":
        df = df.query("(FROM_YEAR >= 2001) or (TO_YEAR >= 2001)")
        df = df.reset_index(drop=True)
        # just keep the player ids and names
        df = df.iloc[:, 0:2]
        return df
    if ids == "all_players":
        df = df.iloc[:, 0:2]
        return df
    if ids == "all_data":
        return df
    else:
        er = "Invalid 'ids' value. It must be 'shots', 'all_shots', or 'all_data'."
        raise ValueError(er)

Response: Returns a pandas DataFrame containing the player IDs used in the stats. nba. com API.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_player_id(player):
    players_df = get_all_player_ids("all_data")
    player = players_df[players_df.DISPLAY_LAST_COMMA_FIRST == player]
    # if there are no plyaers by the given name, raise an a error
    if len(player) == 0:
        er = "Invalid player name passed or there is no player with that name."
        raise ValueError(er)
    player_id = player.PERSON_ID.values
    return player_id

Response: Returns the player ID associated with the given player name.


Instruction:
Write a Python 3 function that can
return a pandas DataFrame with all Team IDs

Response:
def get_all_team_ids():
    """Returns a pandas DataFrame with all Team IDs"""
    df = get_all_player_ids("all_data")
    df = pd.DataFrame({"TEAM_NAME": df.TEAM_NAME.unique(),
                       "TEAM_ID": df.TEAM_ID.unique()})
    return df


Instruction:
How would you code a function in Python 3 to
return the team ID associated with the team name that is passed in.

Response:
def get_team_id(team_name):
    """ Returns the team ID associated with the team name that is passed in.

    Parameters
    ----------
    team_name : str
        The team name whose ID we want.  NOTE: Only pass in the team name
        (e.g. "Lakers"), not the city, or city and team name, or the team
        abbreviation.

    Returns
    -------
    team_id : int
        The team ID associated with the team name.

    """
    df = get_all_team_ids()
    df = df[df.TEAM_NAME == team_name]
    if len(df) == 0:
        er = "Invalid team name or there is no team with that name."
        raise ValueError(er)
    team_id = df.TEAM_ID.iloc[0]
    return team_id


Instruction:
Can you generate the documentation for the following Python 3 function
def get_player_img(player_id):
    url = "http://stats.nba.com/media/players/230x185/"+str(player_id)+".png"
    img_file = str(player_id) + ".png"
    pic = urlretrieve(url, img_file)
    player_img = plt.imread(pic[0])
    return player_img

Response: Returns the image of the player from stats. nba. com.


Instruction:
Write a Python 3 script for
returning team game logs as a pandas DataFrame

Response:
def get_game_logs(self):
        """Returns team game logs as a pandas DataFrame"""
        logs = self.response.json()['resultSets'][0]['rowSet']
        headers = self.response.json()['resultSets'][0]['headers']
        df = pd.DataFrame(logs, columns=headers)
        df.GAME_DATE = pd.to_datetime(df.GAME_DATE)
        return df


Instruction:
Explain what the following Python 3 code does
def get_game_id(self, date):
        df = self.get_game_logs()
        game_id = df[df.GAME_DATE == date].Game_ID.values[0]
        return game_id

Response: Returns the Game ID associated with the date that is passed in.


Instruction:
Explain what the following Python 3 code does
def update_params(self, parameters):
        self.url_paramaters.update(parameters)
        self.response = requests.get(self.base_url, params=self.url_paramaters,
                                     headers=HEADERS)
        # raise error if status code is not 200
        self.response.raise_for_status()
        return self

Response:
Pass in a dictionary to update url parameters for NBA stats API
            API parameters.


Instruction:
Write a Python 3 script for
returning the shot chart data as a pandas DataFrame.

Response:
def get_shots(self):
        """Returns the shot chart data as a pandas DataFrame."""
        shots = self.response.json()['resultSets'][0]['rowSet']
        headers = self.response.json()['resultSets'][0]['headers']
        return pd.DataFrame(shots, columns=headers)


Instruction:
Implement a Python 3 function for
connecting to the NATS server.

Response:
def connect(self):
        """
        Connect will attempt to connect to the NATS server. The url can
        contain username/password semantics.
        """
        self._build_socket()
        self._connect_socket()
        self._build_file_socket()
        self._send_connect_msg()


Instruction:
Given the following Python 3 function, write the documentation
def subscribe(self, subject, callback, queue=''):
        s = Subscription(
            sid=self._next_sid,
            subject=subject,
            queue=queue,
            callback=callback,
            connetion=self
        )

        self._subscriptions[s.sid] = s
        self._send('SUB %s %s %d' % (s.subject, s.queue, s.sid))
        self._next_sid += 1

        return s

Response: Subscribe to a subject.


Instruction:
Given the following Python 3 function, write the documentation
def publish(self, subject, msg, reply=None):
        if msg is None:
            msg = ''

        if reply is None:
            command = 'PUB %s %d' % (subject, len(msg))
        else:
            command = 'PUB %s %s %d' % (subject, reply, len(msg))

        self._send(command)
        self._send(msg)

Response: Publishes the data argument to the subject.


Instruction:
Implement a Python 3 function for
publishing a message with an implicit inbox listener as the reply.

Response:
def request(self, subject, callback, msg=None):
        """
        ublish a message with an implicit inbox listener as the reply.
        Message is optional.

        Args:
            subject (string): a string with the subject
            callback (function): callback to be called
            msg (string=None): payload string
        """
        inbox = self._build_inbox()
        s = self.subscribe(inbox, callback)
        self.unsubscribe(s, 1)
        self.publish(subject, msg, inbox)

        return s


Instruction:
Create a Python 3 function to
wait for a given number of seconds from any subject.

Response:
def wait(self, duration=None, count=0):
        """
        Publish publishes the data argument to the given subject.

        Args:
            duration (float): will wait for the given number of seconds
            count (count): stop of wait after n messages from any subject
        """
        start = time.time()
        total = 0
        while True:
            type, result = self._recv(MSG, PING, OK)
            if type is MSG:
                total += 1
                if self._handle_msg(result) is False:
                    break

                if count and total >= count:
                    break

            elif type is PING:
                self._handle_ping()

            if duration and time.time() - start > duration:
                break


Instruction:
How would you explain what the following Python 3 function does
def draw_court(ax=None, color='gray', lw=1, outer_lines=False):
    if ax is None:
        ax = plt.gca()

    # Create the various parts of an NBA basketball court

    # Create the basketball hoop
    hoop = Circle((0, 0), radius=7.5, linewidth=lw, color=color, fill=False)

    # Create backboard
    backboard = Rectangle((-30, -12.5), 60, 0, linewidth=lw, color=color)

    # The paint
    # Create the outer box 0f the paint, width=16ft, height=19ft
    outer_box = Rectangle((-80, -47.5), 160, 190, linewidth=lw, color=color,
                          fill=False)
    # Create the inner box of the paint, widt=12ft, height=19ft
    inner_box = Rectangle((-60, -47.5), 120, 190, linewidth=lw, color=color,
                          fill=False)

    # Create free throw top arc
    top_free_throw = Arc((0, 142.5), 120, 120, theta1=0, theta2=180,
                         linewidth=lw, color=color, fill=False)
    # Create free throw bottom arc
    bottom_free_throw = Arc((0, 142.5), 120, 120, theta1=180, theta2=0,
                            linewidth=lw, color=color, linestyle='dashed')
    # Restricted Zone, it is an arc with 4ft radius from center of the hoop
    restricted = Arc((0, 0), 80, 80, theta1=0, theta2=180, linewidth=lw,
                     color=color)

    # Three point line
    # Create the right side 3pt lines, it's 14ft long before it arcs
    corner_three_a = Rectangle((-220, -47.5), 0, 140, linewidth=lw,
                               color=color)
    # Create the right side 3pt lines, it's 14ft long before it arcs
    corner_three_b = Rectangle((220, -47.5), 0, 140, linewidth=lw, color=color)
    # 3pt arc - center of arc will be the hoop, arc is 23'9" away from hoop
    three_arc = Arc((0, 0), 475, 475, theta1=22, theta2=158, linewidth=lw,
                    color=color)

    # Center Court
    center_outer_arc = Arc((0, 422.5), 120, 120, theta1=180, theta2=0,
                           linewidth=lw, color=color)
    center_inner_arc = Arc((0, 422.5), 40, 40, theta1=180, theta2=0,
                           linewidth=lw, color=color)

    # List of the court elements to be plotted onto the axes
    court_elements = [hoop, backboard, outer_box, inner_box, top_free_throw,
                      bottom_free_throw, restricted, corner_three_a,
                      corner_three_b, three_arc, center_outer_arc,
                      center_inner_arc]

    if outer_lines:
        # Draw the half court line, baseline and side out bound lines
        outer_lines = Rectangle((-250, -47.5), 500, 470, linewidth=lw,
                                color=color, fill=False)
        court_elements.append(outer_lines)

    # Add the court elements onto the axes
    for element in court_elements:
        ax.add_patch(element)

    return ax

Response: Returns an axes with a basketball court drawn onto to it.


Instruction:
Create a Python 3 function for
creating a scatter plot of the player shots.

Response:
def shot_chart(x, y, kind="scatter", title="", color="b", cmap=None,
               xlim=(-250, 250), ylim=(422.5, -47.5),
               court_color="gray", court_lw=1, outer_lines=False,
               flip_court=False, kde_shade=True, gridsize=None, ax=None,
               despine=False, **kwargs):
    """
    Returns an Axes object with player shots plotted.

    Parameters
    ----------

    x, y : strings or vector
        The x and y coordinates of the shots taken. They can be passed in as
        vectors (such as a pandas Series) or as columns from the pandas
        DataFrame passed into ``data``.
    data : DataFrame, optional
        DataFrame containing shots where ``x`` and ``y`` represent the
        shot location coordinates.
    kind : { "scatter", "kde", "hex" }, optional
        The kind of shot chart to create.
    title : str, optional
        The title for the plot.
    color : matplotlib color, optional
        Color used to plot the shots
    cmap : matplotlib Colormap object or name, optional
        Colormap for the range of data values. If one isn't provided, the
        colormap is derived from the valuue passed to ``color``. Used for KDE
        and Hexbin plots.
    {x, y}lim : two-tuples, optional
        The axis limits of the plot.
    court_color : matplotlib color, optional
        The color of the court lines.
    court_lw : float, optional
        The linewidth the of the court lines.
    outer_lines : boolean, optional
        If ``True`` the out of bound lines are drawn in as a matplotlib
        Rectangle.
    flip_court : boolean, optional
        If ``True`` orients the hoop towards the bottom of the plot.  Default
        is ``False``, which orients the court where the hoop is towards the top
        of the plot.
    kde_shade : boolean, optional
        Default is ``True``, which shades in the KDE contours.
    gridsize : int, optional
        Number of hexagons in the x-direction.  The default is calculated using
        the Freedman-Diaconis method.
    ax : Axes, optional
        The Axes object to plot the court onto.
    despine : boolean, optional
        If ``True``, removes the spines.
    kwargs : key, value pairs
        Keyword arguments for matplotlib Collection properties or seaborn plots.

    Returns
    -------
     ax : Axes
        The Axes object with the shot chart plotted on it.

    """

    if ax is None:
        ax = plt.gca()

    if cmap is None:
        cmap = sns.light_palette(color, as_cmap=True)

    if not flip_court:
        ax.set_xlim(xlim)
        ax.set_ylim(ylim)
    else:
        ax.set_xlim(xlim[::-1])
        ax.set_ylim(ylim[::-1])

    ax.tick_params(labelbottom="off", labelleft="off")
    ax.set_title(title, fontsize=18)

    draw_court(ax, color=court_color, lw=court_lw, outer_lines=outer_lines)

    if kind == "scatter":
        ax.scatter(x, y, c=color, **kwargs)

    elif kind == "kde":
        sns.kdeplot(x, y, shade=kde_shade, cmap=cmap, ax=ax, **kwargs)
        ax.set_xlabel('')
        ax.set_ylabel('')

    elif kind == "hex":
        if gridsize is None:
            # Get the number of bins for hexbin using Freedman-Diaconis rule
            # This is idea was taken from seaborn, which got the calculation
            # from http://stats.stackexchange.com/questions/798/
            from seaborn.distributions import _freedman_diaconis_bins
            x_bin = _freedman_diaconis_bins(x)
            y_bin = _freedman_diaconis_bins(y)
            gridsize = int(np.mean([x_bin, y_bin]))

        ax.hexbin(x, y, gridsize=gridsize, cmap=cmap, **kwargs)

    else:
        raise ValueError("kind must be 'scatter', 'kde', or 'hex'.")

    # Set the spines to match the rest of court lines, makes outer_lines
    # somewhate unnecessary
    for spine in ax.spines:
        ax.spines[spine].set_lw(court_lw)
        ax.spines[spine].set_color(court_color)

    if despine:
        ax.spines["top"].set_visible(False)
        ax.spines["bottom"].set_visible(False)
        ax.spines["right"].set_visible(False)
        ax.spines["left"].set_visible(False)

    return ax


Instruction:
Create a Python 3 function for
returning a JointGrid object containing the shot chart for the given location.

Response:
def shot_chart_jointgrid(x, y, data=None, joint_type="scatter", title="",
                         joint_color="b", cmap=None,  xlim=(-250, 250),
                         ylim=(422.5, -47.5), court_color="gray", court_lw=1,
                         outer_lines=False, flip_court=False,
                         joint_kde_shade=True, gridsize=None,
                         marginals_color="b", marginals_type="both",
                         marginals_kde_shade=True, size=(12, 11), space=0,
                         despine=False, joint_kws=None, marginal_kws=None,
                         **kwargs):

    """
    Returns a JointGrid object containing the shot chart.

    This function allows for more flexibility in customizing your shot chart
    than the ``shot_chart_jointplot`` function.

    Parameters
    ----------

    x, y : strings or vector
        The x and y coordinates of the shots taken. They can be passed in as
        vectors (such as a pandas Series) or as columns from the pandas
        DataFrame passed into ``data``.
    data : DataFrame, optional
        DataFrame containing shots where ``x`` and ``y`` represent the shot
        location coordinates.
    joint_type : { "scatter", "kde", "hex" }, optional
        The type of shot chart for the joint plot.
    title : str, optional
        The title for the plot.
    joint_color : matplotlib color, optional
        Color used to plot the shots on the joint plot.
    cmap : matplotlib Colormap object or name, optional
        Colormap for the range of data values. If one isn't provided, the
        colormap is derived from the value passed to ``color``. Used for KDE
        and Hexbin joint plots.
    {x, y}lim : two-tuples, optional
        The axis limits of the plot.  The defaults represent the out of bounds
        lines and half court line.
    court_color : matplotlib color, optional
        The color of the court lines.
    court_lw : float, optional
        The linewidth the of the court lines.
    outer_lines : boolean, optional
        If ``True`` the out of bound lines are drawn in as a matplotlib
        Rectangle.
    flip_court : boolean, optional
        If ``True`` orients the hoop towards the bottom of the plot. Default is
        ``False``, which orients the court where the hoop is towards the top of
        the plot.
    joint_kde_shade : boolean, optional
        Default is ``True``, which shades in the KDE contours on the joint plot.
    gridsize : int, optional
        Number of hexagons in the x-direction. The default is calculated using
        the Freedman-Diaconis method.
    marginals_color : matplotlib color, optional
        Color used to plot the shots on the marginal plots.
    marginals_type : { "both", "hist", "kde"}, optional
        The type of plot for the marginal plots.
    marginals_kde_shade : boolean, optional
        Default is ``True``, which shades in the KDE contours on the marginal
        plots.
    size : tuple, optional
        The width and height of the plot in inches.
    space : numeric, optional
        The space between the joint and marginal plots.
    despine : boolean, optional
        If ``True``, removes the spines.
    {joint, marginal}_kws : dicts
        Additional kewyord arguments for joint and marginal plot components.
    kwargs : key, value pairs
        Keyword arguments for matplotlib Collection properties or seaborn plots.

    Returns
    -------
     grid : JointGrid
        The JointGrid object with the shot chart plotted on it.

    """

    # The joint_kws and marginal_kws idea was taken from seaborn
    # Create the default empty kwargs for joint and marginal plots
    if joint_kws is None:
        joint_kws = {}
    joint_kws.update(kwargs)

    if marginal_kws is None:
        marginal_kws = {}

    # If a colormap is not provided, then it is based off of the joint_color
    if cmap is None:
        cmap = sns.light_palette(joint_color, as_cmap=True)

    # Flip the court so that the hoop is by the bottom of the plot
    if flip_court:
        xlim = xlim[::-1]
        ylim = ylim[::-1]

    # Create the JointGrid to draw the shot chart plots onto
    grid = sns.JointGrid(x=x, y=y, data=data, xlim=xlim, ylim=ylim,
                         space=space)

    # Joint Plot
    # Create the main plot of the joint shot chart
    if joint_type == "scatter":
        grid = grid.plot_joint(plt.scatter, color=joint_color, **joint_kws)

    elif joint_type == "kde":
        grid = grid.plot_joint(sns.kdeplot, cmap=cmap,
                               shade=joint_kde_shade, **joint_kws)

    elif joint_type == "hex":
        if gridsize is None:
            # Get the number of bins for hexbin using Freedman-Diaconis rule
            # This is idea was taken from seaborn, which got the calculation
            # from http://stats.stackexchange.com/questions/798/
            from seaborn.distributions import _freedman_diaconis_bins
            x_bin = _freedman_diaconis_bins(x)
            y_bin = _freedman_diaconis_bins(y)
            gridsize = int(np.mean([x_bin, y_bin]))

        grid = grid.plot_joint(plt.hexbin, gridsize=gridsize, cmap=cmap,
                               **joint_kws)

    else:
        raise ValueError("joint_type must be 'scatter', 'kde', or 'hex'.")

    # Marginal plots
    # Create the plots on the axis of the main plot of the joint shot chart.
    if marginals_type == "both":
        grid = grid.plot_marginals(sns.distplot, color=marginals_color,
                                   **marginal_kws)

    elif marginals_type == "hist":
        grid = grid.plot_marginals(sns.distplot, color=marginals_color,
                                   kde=False, **marginal_kws)

    elif marginals_type == "kde":
        grid = grid.plot_marginals(sns.kdeplot, color=marginals_color,
                                   shade=marginals_kde_shade, **marginal_kws)

    else:
        raise ValueError("marginals_type must be 'both', 'hist', or 'kde'.")

    # Set the size of the joint shot chart
    grid.fig.set_size_inches(size)

    # Extract the the first axes, which is the main plot of the
    # joint shot chart, and draw the court onto it
    ax = grid.fig.get_axes()[0]
    draw_court(ax, color=court_color, lw=court_lw, outer_lines=outer_lines)

    # Get rid of the axis labels
    grid.set_axis_labels(xlabel="", ylabel="")
    # Get rid of all tick labels
    ax.tick_params(labelbottom="off", labelleft="off")
    # Set the title above the top marginal plot
    ax.set_title(title, y=1.2, fontsize=18)

    # Set the spines to match the rest of court lines, makes outer_lines
    # somewhate unnecessary
    for spine in ax.spines:
        ax.spines[spine].set_lw(court_lw)
        ax.spines[spine].set_color(court_color)
        # set the marginal spines to be the same as the rest of the spines
        grid.ax_marg_x.spines[spine].set_lw(court_lw)
        grid.ax_marg_x.spines[spine].set_color(court_color)
        grid.ax_marg_y.spines[spine].set_lw(court_lw)
        grid.ax_marg_y.spines[spine].set_color(court_color)

    if despine:
        ax.spines["top"].set_visible(False)
        ax.spines["bottom"].set_visible(False)
        ax.spines["right"].set_visible(False)
        ax.spines["left"].set_visible(False)

    return grid


Instruction:
Can you generate the documentation for the following Python 3 function
def shot_chart_jointplot(x, y, data=None, kind="scatter", title="", color="b",
                         cmap=None, xlim=(-250, 250), ylim=(422.5, -47.5),
                         court_color="gray", court_lw=1, outer_lines=False,
                         flip_court=False, size=(12, 11), space=0,
                         despine=False, joint_kws=None, marginal_kws=None,
                         **kwargs):
    """
    Returns a seaborn JointGrid using sns.jointplot

    Parameters
    ----------

    x, y : strings or vector
        The x and y coordinates of the shots taken. They can be passed in as
        vectors (such as a pandas Series) or as column names from the pandas
        DataFrame passed into ``data``.
    data : DataFrame, optional
        DataFrame containing shots where ``x`` and ``y`` represent the
        shot location coordinates.
    kind : { "scatter", "kde", "hex" }, optional
        The kind of shot chart to create.
    title : str, optional
        The title for the plot.
    color : matplotlib color, optional
        Color used to plot the shots
    cmap : matplotlib Colormap object or name, optional
        Colormap for the range of data values. If one isn't provided, the
        colormap is derived from the valuue passed to ``color``. Used for KDE
        and Hexbin joint plots.
    {x, y}lim : two-tuples, optional
        The axis limits of the plot.  The defaults represent the out of bounds
        lines and half court line.
    court_color : matplotlib color, optional
        The color of the court lines.
    court_lw : float, optional
        The linewidth the of the court lines.
    outer_lines : boolean, optional
        If ``True`` the out of bound lines are drawn in as a matplotlib
        Rectangle.
    flip_court : boolean, optional
        If ``True`` orients the hoop towards the bottom of the plot.  Default
        is ``False``, which orients the court where the hoop is towards the top
        of the plot.
    gridsize : int, optional
        Number of hexagons in the x-direction.  The default is calculated using
        the Freedman-Diaconis method.
    size : tuple, optional
        The width and height of the plot in inches.
    space : numeric, optional
        The space between the joint and marginal plots.
    {joint, marginal}_kws : dicts
        Additional kewyord arguments for joint and marginal plot components.
    kwargs : key, value pairs
        Keyword arguments for matplotlib Collection properties or seaborn plots.

    Returns
    -------
     grid : JointGrid
        The JointGrid object with the shot chart plotted on it.

   """

    # If a colormap is not provided, then it is based off of the color
    if cmap is None:
        cmap = sns.light_palette(color, as_cmap=True)

    if kind not in ["scatter", "kde", "hex"]:
        raise ValueError("kind must be 'scatter', 'kde', or 'hex'.")

    grid = sns.jointplot(x=x, y=y, data=data, stat_func=None, kind=kind,
                         space=0, color=color, cmap=cmap, joint_kws=joint_kws,
                         marginal_kws=marginal_kws, **kwargs)

    grid.fig.set_size_inches(size)

    # A joint plot has 3 Axes, the first one called ax_joint
    # is the one we want to draw our court onto and adjust some other settings
    ax = grid.ax_joint

    if not flip_court:
        ax.set_xlim(xlim)
        ax.set_ylim(ylim)
    else:
        ax.set_xlim(xlim[::-1])
        ax.set_ylim(ylim[::-1])

    draw_court(ax, color=court_color, lw=court_lw, outer_lines=outer_lines)

    # Get rid of axis labels and tick marks
    ax.set_xlabel('')
    ax.set_ylabel('')
    ax.tick_params(labelbottom='off', labelleft='off')

    # Add a title
    ax.set_title(title, y=1.2, fontsize=18)

    # Set the spines to match the rest of court lines, makes outer_lines
    # somewhate unnecessary
    for spine in ax.spines:
        ax.spines[spine].set_lw(court_lw)
        ax.spines[spine].set_color(court_color)
        # set the margin joint spines to be same as the rest of the plot
        grid.ax_marg_x.spines[spine].set_lw(court_lw)
        grid.ax_marg_x.spines[spine].set_color(court_color)
        grid.ax_marg_y.spines[spine].set_lw(court_lw)
        grid.ax_marg_y.spines[spine].set_color(court_color)

    if despine:
        ax.spines["top"].set_visible(False)
        ax.spines["bottom"].set_visible(False)
        ax.spines["right"].set_visible(False)
        ax.spines["left"].set_visible(False)

    return grid

Response: Creates a seaborn JointGrid from a set of shots taken from a pandas DataFrame or a pandas Series.


Instruction:
Create a Python 3 function for
plotting a heatmap for the court and court number of times shot.

Response:
def heatmap(x, y, z, title="", cmap=plt.cm.YlOrRd, bins=20,
            xlim=(-250, 250), ylim=(422.5, -47.5),
            facecolor='lightgray', facecolor_alpha=0.4,
            court_color="black", court_lw=0.5, outer_lines=False,
            flip_court=False, ax=None, **kwargs):

    """
    Returns an AxesImage object that contains a heatmap.

    TODO: Redo some code and explain parameters
    """

    # Bin the FGA (x, y) and Calculcate the mean number of times shot was
    # made (z) within each bin
    # mean is the calculated FG percentage for each bin
    mean, xedges, yedges, binnumber = binned_statistic_2d(x=x, y=y,
                                                          values=z,
                                                          statistic='mean',
                                                          bins=bins)

    if ax is None:
        ax = plt.gca()

    if not flip_court:
        ax.set_xlim(xlim)
        ax.set_ylim(ylim)
    else:
        ax.set_xlim(xlim[::-1])
        ax.set_ylim(ylim[::-1])

    ax.tick_params(labelbottom="off", labelleft="off")
    ax.set_title(title, fontsize=18)

    ax.patch.set_facecolor(facecolor)
    ax.patch.set_alpha(facecolor_alpha)

    draw_court(ax, color=court_color, lw=court_lw, outer_lines=outer_lines)

    heatmap = ax.imshow(mean.T, origin='lower', extent=[xedges[0], xedges[-1],
                        yedges[0], yedges[-1]], interpolation='nearest',
                        cmap=cmap)

    return heatmap


Instruction:
Can you implement a function in Python 3 that
returns a Bokeh figure with the basketball court lines drawn onto it

Response:
def bokeh_draw_court(figure, line_color='gray', line_width=1):
    """Returns a figure with the basketball court lines drawn onto it

    This function draws a court based on the x and y-axis values that the NBA
    stats API provides for the shot chart data.  For example the center of the
    hoop is located at the (0,0) coordinate.  Twenty-two feet from the left of
    the center of the hoop in is represented by the (-220,0) coordinates.
    So one foot equals +/-10 units on the x and y-axis.

    Parameters
    ----------
    figure : Bokeh figure object
        The Axes object to plot the court onto.
    line_color : str, optional
        The color of the court lines. Can be a a Hex value.
    line_width : float, optional
        The linewidth the of the court lines in pixels.

    Returns
    -------
    figure : Figure
        The Figure object with the court on it.

    """

    # hoop
    figure.circle(x=0, y=0, radius=7.5, fill_alpha=0,
                  line_color=line_color, line_width=line_width)

    # backboard
    figure.line(x=range(-30, 31), y=-12.5, line_color=line_color)

    # The paint
    # outerbox
    figure.rect(x=0, y=47.5, width=160, height=190, fill_alpha=0,
                line_color=line_color, line_width=line_width)
    # innerbox
    # left inner box line
    figure.line(x=-60, y=np.arange(-47.5, 143.5), line_color=line_color,
                line_width=line_width)
    # right inner box line
    figure.line(x=60, y=np.arange(-47.5, 143.5), line_color=line_color,
                line_width=line_width)

    # Restricted Zone
    figure.arc(x=0, y=0, radius=40, start_angle=pi, end_angle=0,
               line_color=line_color, line_width=line_width)

    # top free throw arc
    figure.arc(x=0, y=142.5, radius=60, start_angle=pi, end_angle=0,
               line_color=line_color)

    # bottome free throw arc
    figure.arc(x=0, y=142.5, radius=60, start_angle=0, end_angle=pi,
               line_color=line_color, line_dash="dashed")

    # Three point line
    # corner three point lines
    figure.line(x=-220, y=np.arange(-47.5, 92.5), line_color=line_color,
                line_width=line_width)
    figure.line(x=220, y=np.arange(-47.5, 92.5), line_color=line_color,
                line_width=line_width)
    # # three point arc
    figure.arc(x=0, y=0, radius=237.5, start_angle=3.528, end_angle=-0.3863,
               line_color=line_color, line_width=line_width)

    # add center court
    # outer center arc
    figure.arc(x=0, y=422.5, radius=60, start_angle=0, end_angle=pi,
               line_color=line_color, line_width=line_width)
    # inner center arct
    figure.arc(x=0, y=422.5, radius=20, start_angle=0, end_angle=pi,
               line_color=line_color, line_width=line_width)

    # outer lines, consistting of half court lines and out of bounds lines
    figure.rect(x=0, y=187.5, width=500, height=470, fill_alpha=0,
                line_color=line_color, line_width=line_width)

    return figure


Instruction:
Create a Python 3 function for
creating a figure that shows the shot chart on it.

Response:
def bokeh_shot_chart(data, x="LOC_X", y="LOC_Y", fill_color="#1f77b4",
                     scatter_size=10, fill_alpha=0.4, line_alpha=0.4,
                     court_line_color='gray', court_line_width=1,
                     hover_tool=False, tooltips=None, **kwargs):

    # TODO: Settings for hover tooltip
    """
    Returns a figure with both FGA and basketball court lines drawn onto it.

    This function expects data to be a ColumnDataSource with the x and y values
    named "LOC_X" and "LOC_Y".  Otherwise specify x and y.

    Parameters
    ----------

    data : DataFrame
        The DataFrame that contains the shot chart data.
    x, y : str, optional
        The x and y coordinates of the shots taken.
    fill_color : str, optional
        The fill color of the shots. Can be a a Hex value.
    scatter_size : int, optional
        The size of the dots for the scatter plot.
    fill_alpha : float, optional
        Alpha value for the shots. Must be a floating point value between 0
        (transparent) to 1 (opaque).
    line_alpha : float, optiona
        Alpha value for the outer lines of the plotted shots. Must be a
        floating point value between 0 (transparent) to 1 (opaque).
    court_line_color : str, optional
        The color of the court lines. Can be a a Hex value.
    court_line_width : float, optional
        The linewidth the of the court lines in pixels.
    hover_tool : boolean, optional
        If ``True``, creates hover tooltip for the plot.
    tooltips : List of tuples, optional
        Provides the information for the the hover tooltip.

    Returns
    -------
    fig : Figure
        The Figure object with the shot chart plotted on it.

    """
    source = ColumnDataSource(data)

    fig = figure(width=700, height=658, x_range=[-250, 250],
                 y_range=[422.5, -47.5], min_border=0, x_axis_type=None,
                 y_axis_type=None, outline_line_color="black", **kwargs)

    fig.scatter(x, y, source=source, size=scatter_size, color=fill_color,
                alpha=fill_alpha, line_alpha=line_alpha)

    bokeh_draw_court(fig, line_color=court_line_color,
                     line_width=court_line_width)

    if hover_tool:
        hover = HoverTool(renderers=[fig.renderers[0]], tooltips=tooltips)
        fig.add_tools(hover)

    return fig


Instruction:
Can you create a Python 3 function that
updates Cluster Centers and Sse centers for each cluster.

Response:
def _update_centers(X, membs, n_clusters, distance):
    """ Update Cluster Centers:
           calculate the mean of feature vectors for each cluster.

        distance can be a string or callable.
    """
    centers = np.empty(shape=(n_clusters, X.shape[1]), dtype=float)
    sse = np.empty(shape=n_clusters, dtype=float)
    for clust_id in range(n_clusters):
        memb_ids = np.where(membs == clust_id)[0]
        X_clust = X[memb_ids,:]

        dist = np.empty(shape=memb_ids.shape[0], dtype=float)
        for i,x in enumerate(X_clust):
           dist[i] = np.sum(scipy.spatial.distance.cdist(X_clust, np.array([x]), distance))

        inx_min = np.argmin(dist)
        centers[clust_id,:] = X_clust[inx_min,:]
        sse[clust_id] = dist[inx_min]
    return(centers, sse)


Instruction:
Write a Python 3 function that can
run a single trial of k - medoids clustering on dataset X and given number of clusters n_clusters and distance distance.

Response:
def _kmedoids_run(X, n_clusters, distance, max_iter, tol, rng):
    """ Run a single trial of k-medoids clustering
        on dataset X, and given number of clusters
    """
    membs = np.empty(shape=X.shape[0], dtype=int)
    centers = kmeans._kmeans_init(X, n_clusters, method='', rng=rng)

    sse_last = 9999.9
    n_iter = 0
    for it in range(1,max_iter):
        membs = kmeans._assign_clusters(X, centers)
        centers,sse_arr = _update_centers(X, membs, n_clusters, distance)
        sse_total = np.sum(sse_arr)
        if np.abs(sse_total - sse_last) < tol:
            n_iter = it
            break
        sse_last = sse_total

    return(centers, membs, sse_total, sse_arr, n_iter)


Instruction:
Explain what the following Python 3 code does
def fit(self, X):
        self.centers_, self.labels_, self.sse_arr_, self.n_iter_ = \
              _kmedoids(X, self.n_clusters, self.distance, self.max_iter, self.n_trials, self.tol, self.rng)

Response:
Apply KMeans Clustering
 self. n_clusters clustering clustering


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _kernelized_dist2centers(K, n_clusters, wmemb, kernel_dist):
    n_samples = K.shape[0]

    for j in range(n_clusters):
        memb_j = np.where(wmemb == j)[0]
        size_j = memb_j.shape[0]

        K_sub_j = K[memb_j][:, memb_j]

        kernel_dist[:,j] = 1 + np.sum(K_sub_j) /(size_j*size_j)
        kernel_dist[:,j] -= 2 * np.sum(K[:, memb_j], axis=1) / size_j

    return

Response:
Compute the distance in transformed feature space to
         cluster centers.


Instruction:
How would you implement a function in Python 3 that
computes the log probability of a multivariate density of a set of means and covars.

Response:
def _log_multivariate_density(X, means, covars):
    """
      Class conditional density:
        P(x | mu, Sigma) = 1/((2pi)^d/2 * |Sigma|^1/2) * exp(-1/2 * (x-mu)^T * Sigma^-1 * (x-mu))

      log of class conditional density:
        log P(x | mu, Sigma) = -1/2*(d*log(2pi) + log(|Sigma|) + (x-mu)^T * Sigma^-1 * (x-mu))
    """
    n_samples, n_dim = X.shape
    n_components = means.shape[0]

    assert(means.shape[0] == covars.shape[0])

    log_proba = np.empty(shape=(n_samples, n_components), dtype=float)
    for i, (mu, cov) in enumerate(zip(means, covars)):
        try:
            cov_chol = scipy.linalg.cholesky(cov, lower=True)
        except scipy.linalg.LinAlgError:
            try:
                cov_chol = scipy.linalg.cholesky(cov + Lambda*np.eye(n_dim), lower=True)
            except:
                raise ValueError("Triangular Matrix Decomposition not performed!\n")

        cov_log_det = 2 * np.sum(np.log(np.diagonal(cov_chol)))

        try:
            cov_solve = scipy.linalg.solve_triangular(cov_chol, (X - mu).T, lower=True).T
        except:
            raise ValueError("Solve_triangular not perormed!\n")

        log_proba[:, i] = -0.5 * (np.sum(cov_solve ** 2, axis=1) + \
                         n_dim * np.log(2 * np.pi) + cov_log_det)

    return(log_proba)


Instruction:
Implement a function in Python 3 to
compute the log likelihood of a mixture parameter given data.

Response:
def _log_likelihood_per_sample(X, means, covars):
    """
      Theta = (theta_1, theta_2, ... theta_M)
      Likelihood of mixture parameters given data: L(Theta | X) = product_i P(x_i | Theta)
      log likelihood: log L(Theta | X) = sum_i log(P(x_i | Theta))

      and note that p(x_i | Theta) = sum_j prior_j * p(x_i | theta_j)

      Probability of sample x being generated from component i:
         P(w_i | x) = P(x|w_i) * P(w_i) / P(X)
           where P(X) = sum_i P(x|w_i) * P(w_i)

       Here post_proba = P/(w_i | x)
        and log_likelihood = log(P(x|w_i))
    """

    logden = _log_multivariate_density(X, means, covars)

    logden_max = logden.max(axis=1)
    log_likelihood = np.log(np.sum(np.exp(logden.T - logden_max) + Epsilon, axis=0))
    log_likelihood += logden_max

    post_proba = np.exp(logden - log_likelihood[:, np.newaxis])

    return (log_likelihood, post_proba)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _validate_params(priors, means, covars):

    for i,(p,m,cv) in enumerate(zip(priors, means, covars)):
        if np.any(np.isinf(p)) or np.any(np.isnan(p)):
            raise ValueError("Component %d of priors is not valid " % i)

        if np.any(np.isinf(m)) or np.any(np.isnan(m)):
            raise ValueError("Component %d of means is not valid " % i)

        if np.any(np.isinf(cv)) or np.any(np.isnan(cv)):
            raise ValueError("Component %d of covars is not valid " % i)

        if (not np.allclose(cv, cv.T) or np.any(scipy.linalg.eigvalsh(cv) <= 0)):
            raise ValueError("Component %d of covars must be positive-definite" % i)

Response:
Validate parameters of the M. L. paramateres
   .


Instruction:
Write a Python 3 script to
update class parameters as below: priors: P(w_i) = sum_x P(w_i | x) ==> Then normalize to get in [0,1] Class means: center_w_i = sum_x P(w_i|x)*x / sum_i sum_x P(w_i|x)

Response:
def _maximization_step(X, posteriors):
    """
      Update class parameters as below:
        priors: P(w_i) = sum_x P(w_i | x) ==> Then normalize to get in [0,1]
        Class means: center_w_i = sum_x P(w_i|x)*x / sum_i sum_x P(w_i|x)
    """

    ### Prior probabilities or class weights
    sum_post_proba = np.sum(posteriors, axis=0)
    prior_proba = sum_post_proba / (sum_post_proba.sum() + Epsilon)

    ### means
    means = np.dot(posteriors.T, X) / (sum_post_proba[:, np.newaxis] + Epsilon)

    ### covariance matrices
    n_components = posteriors.shape[1]
    n_features = X.shape[1]
    covars = np.empty(shape=(n_components, n_features, n_features), dtype=float)

    for i in range(n_components):
        post_i = posteriors[:, i]
        mean_i = means[i]
        diff_i = X - mean_i

        with np.errstate(under='ignore'):
            covar_i = np.dot(post_i * diff_i.T, diff_i) / (post_i.sum() + Epsilon)
        covars[i] = covar_i + Lambda * np.eye(n_features)

    _validate_params(prior_proba, means, covars)
    return(prior_proba, means, covars)


Instruction:
Here you have a function in Python 3, explain what it does
def fit(self, X):
        params_dict = _fit_gmm_params(X=X, n_mixtures=self.n_clusters, \
                        n_init=self.n_trials, init_method=self.init_method, \
                        n_iter=self.max_iter, tol=self.tol)
        self.priors_ = params_dict['priors']
        self.means_  = params_dict['means']
        self.covars_ = params_dict['covars']

        self.converged = True
        self.labels_ = self.predict(X)

Response: Fit mixture - density parameters with EM algorithm


Instruction:
Can you tell what is the following Python 3 function doing
def _kmeans_init(X, n_clusters, method='balanced', rng=None):
    n_samples = X.shape[0]
    if rng is None:
        cent_idx = np.random.choice(n_samples, replace=False, size=n_clusters)
    else:
        #print('Generate random centers using RNG')
        cent_idx = rng.choice(n_samples, replace=False, size=n_clusters)

    centers = X[cent_idx,:]
    mean_X = np.mean(X, axis=0)

    if method == 'balanced':
        centers[n_clusters-1] = n_clusters*mean_X - np.sum(centers[:(n_clusters-1)], axis=0)

    return (centers)

Response: Initialize k = n_clusters centroids randomly


Instruction:
Make a summary of the following Python 3 code
def _assign_clusters(X, centers):
    dist2cents = scipy.spatial.distance.cdist(X, centers, metric='seuclidean')
    membs = np.argmin(dist2cents, axis=1)

    return(membs)

Response: Assign each point to the closet cluster center


Instruction:
Here you have a function in Python 3, explain what it does
def _cal_dist2center(X, center):
    dmemb2cen = scipy.spatial.distance.cdist(X, center.reshape(1,X.shape[1]), metric='seuclidean')
    return(np.sum(dmemb2cen))

Response: Calculate the SSE to the cluster center


Instruction:
Explain what the following Python 3 code does
def _update_centers(X, membs, n_clusters):
    centers = np.empty(shape=(n_clusters, X.shape[1]), dtype=float)
    sse = np.empty(shape=n_clusters, dtype=float)
    for clust_id in range(n_clusters):
        memb_ids = np.where(membs == clust_id)[0]

        if memb_ids.shape[0] == 0:
            memb_ids = np.random.choice(X.shape[0], size=1)
            #print("Empty cluster replaced with ", memb_ids)
        centers[clust_id,:] = np.mean(X[memb_ids,:], axis=0)

        sse[clust_id] = _cal_dist2center(X[memb_ids,:], centers[clust_id,:])
    return(centers, sse)

Response:
Update Cluster Centers:
           calculate the mean of feature vectors for each cluster


Instruction:
Create a Python 3 function to
run a single trial of k - means clustering on dataset X and given number of clusters

Response:
def _kmeans_run(X, n_clusters, max_iter, tol):
    """ Run a single trial of k-means clustering
        on dataset X, and given number of clusters
    """
    membs = np.empty(shape=X.shape[0], dtype=int)
    centers = _kmeans_init(X, n_clusters)

    sse_last = 9999.9
    n_iter = 0
    for it in range(1,max_iter):
        membs = _assign_clusters(X, centers)
        centers,sse_arr = _update_centers(X, membs, n_clusters)
        sse_total = np.sum(sse_arr)
        if np.abs(sse_total - sse_last) < tol:
            n_iter = it
            break
        sse_last = sse_total

    return(centers, membs, sse_total, sse_arr, n_iter)


Instruction:
Create a Python 3 function for
running multiple trials of k - means clustering and k - means cluster labels and outputt he best centers and cluster labels

Response:
def _kmeans(X, n_clusters, max_iter, n_trials, tol):
    """ Run multiple trials of k-means clustering,
        and outputt he best centers, and cluster labels
    """
    n_samples, n_features = X.shape[0], X.shape[1]

    centers_best = np.empty(shape=(n_clusters,n_features), dtype=float)
    labels_best  = np.empty(shape=n_samples, dtype=int)
    for i in range(n_trials):
        centers, labels, sse_tot, sse_arr, n_iter  = _kmeans_run(X, n_clusters, max_iter, tol)
        if i==0:
            sse_tot_best = sse_tot
            sse_arr_best = sse_arr
            n_iter_best = n_iter
            centers_best = centers.copy()
            labels_best  = labels.copy()
        if sse_tot < sse_tot_best:
            sse_tot_best = sse_tot
            sse_arr_best = sse_arr
            n_iter_best = n_iter
            centers_best = centers.copy()
            labels_best  = labels.copy()

    return(centers_best, labels_best, sse_arr_best, n_iter_best)


Instruction:
Can you tell what is the following Python 3 function doing
def fit(self, X):
        self.centers_, self.labels_, self.sse_arr_, self.n_iter_ = \
              _kmeans(X, self.n_clusters, self.max_iter, self.n_trials, self.tol)

Response:
Apply KMeans Clustering
              X: dataset with feature vectors


Instruction:
Can you tell what is the following Python 3 function doing
def _cut_tree(tree, n_clusters, membs):
    ## starting from root,
    ## a node is added to the cut_set or
    ## its children are added to node_set
    assert(n_clusters >= 2)
    assert(n_clusters <= len(tree.leaves()))

    cut_centers = dict() #np.empty(shape=(n_clusters, ndim), dtype=float)

    for i in range(n_clusters-1):
        if i==0:
            search_set = set(tree.children(0))
            node_set,cut_set = set(), set()
        else:
            search_set = node_set.union(cut_set)
            node_set,cut_set = set(), set()

        if i+2 == n_clusters:
            cut_set = search_set
        else:
            for _ in range(len(search_set)):
                n = search_set.pop()

                if n.data['ilev'] is None or n.data['ilev']>i+2:
                    cut_set.add(n)
                else:
                    nid = n.identifier
                    if n.data['ilev']-2==i:
                        node_set = node_set.union(set(tree.children(nid)))

    conv_membs = membs.copy()
    for node in cut_set:
        nid = node.identifier
        label = node.data['label']
        cut_centers[label] = node.data['center']
        sub_leaves = tree.leaves(nid)
        for leaf in sub_leaves:
            indx = np.where(conv_membs == leaf)[0]
            conv_membs[indx] = nid

    return(conv_membs, cut_centers)

Response: Cut the tree to get desired number of clusters as n_desired < n_clusters


Instruction:
Explain what the following Python 3 code does
def _add_tree_node(tree, label, ilev,  X=None, size=None, center=None, sse=None, parent=None):
    if size is None:
        size = X.shape[0]
    if (center is None):
        center = np.mean(X, axis=0)
    if (sse is None):
        sse = _kmeans._cal_dist2center(X, center)

    center = list(center)
    datadict = {
        'size'  : size,
        'center': center,
        'label' : label,
        'sse'   : sse,
        'ilev'  : None
    }
    if (parent is None):
        tree.create_node(label, label, data=datadict)
    else:
        tree.create_node(label, label, parent=parent, data=datadict)
        tree.get_node(parent).data['ilev'] = ilev

    return(tree)

Response: Add a node to the tree


Instruction:
How would you code a function in Python 3 to
bisect Kmeans clustering to reach n_clusters number of clusters

Response:
def _bisect_kmeans(X, n_clusters, n_trials, max_iter, tol):
    """ Apply Bisecting Kmeans clustering
        to reach n_clusters number of clusters
    """
    membs = np.empty(shape=X.shape[0], dtype=int)
    centers = dict() #np.empty(shape=(n_clusters,X.shape[1]), dtype=float)
    sse_arr = dict() #-1.0*np.ones(shape=n_clusters, dtype=float)

    ## data structure to store cluster hierarchies
    tree = treelib.Tree()
    tree = _add_tree_node(tree, 0, ilev=0, X=X)

    km = _kmeans.KMeans(n_clusters=2, n_trials=n_trials, max_iter=max_iter, tol=tol)
    for i in range(1,n_clusters):
        sel_clust_id,sel_memb_ids = _select_cluster_2_split(membs, tree)
        X_sub = X[sel_memb_ids,:]
        km.fit(X_sub)

        #print("Bisecting Step %d    :"%i, sel_clust_id, km.sse_arr_, km.centers_)
        ## Updating the clusters & properties
        #sse_arr[[sel_clust_id,i]] = km.sse_arr_
        #centers[[sel_clust_id,i]] = km.centers_
        tree = _add_tree_node(tree, 2*i-1, i, \
                              size=np.sum(km.labels_ == 0), center=km.centers_[0], \
                              sse=km.sse_arr_[0], parent= sel_clust_id)
        tree = _add_tree_node(tree, 2*i,   i, \
                             size=np.sum(km.labels_ == 1), center=km.centers_[1], \
                             sse=km.sse_arr_[1], parent= sel_clust_id)

        pred_labels = km.labels_
        pred_labels[np.where(pred_labels == 1)[0]] = 2*i
        pred_labels[np.where(pred_labels == 0)[0]] = 2*i - 1
        #if sel_clust_id == 1:
        #    pred_labels[np.where(pred_labels == 0)[0]] = sel_clust_id
        #    pred_labels[np.where(pred_labels == 1)[0]] = i
        #else:
        #    pred_labels[np.where(pred_labels == 1)[0]] = i
        #    pred_labels[np.where(pred_labels == 0)[0]] = sel_clust_id

        membs[sel_memb_ids] = pred_labels

    for n in tree.leaves():
        label = n.data['label']
        centers[label] = n.data['center']
        sse_arr[label] = n.data['sse']

    return(centers, membs, sse_arr, tree)


Instruction:
Can you create a Python 3 function that
returns a LaTeX ready table of model comparisons.

Response:
def comparison_table(self, caption=None, label="tab:model_comp", hlines=True,
                         aic=True, bic=True, dic=True, sort="bic", descending=True):  # pragma: no cover
        """
        Return a LaTeX ready table of model comparisons.

        Parameters
        ----------
        caption : str, optional
            The table caption to insert.
        label : str, optional
            The table label to insert.
        hlines : bool, optional
            Whether to insert hlines in the table or not.
        aic : bool, optional
            Whether to include a column for AICc or not.
        bic : bool, optional
            Whether to include a column for BIC or not.
        dic : bool, optional
            Whether to include a column for DIC or not.
        sort : str, optional
            How to sort the models. Should be one of "bic", "aic" or "dic".
        descending : bool, optional
            The sort order.

        Returns
        -------
        str
            A LaTeX table to be copied into your document.
        """

        if sort == "bic":
            assert bic, "You cannot sort by BIC if you turn it off"
        if sort == "aic":
            assert aic, "You cannot sort by AIC if you turn it off"
        if sort == "dic":
            assert dic, "You cannot sort by DIC if you turn it off"

        if caption is None:
            caption = ""
        if label is None:
            label = ""

        base_string = get_latex_table_frame(caption, label)
        end_text = " \\\\ \n"
        num_cols = 1 + (1 if aic else 0) + (1 if bic else 0)
        column_text = "c" * (num_cols + 1)
        center_text = ""
        hline_text = "\\hline\n"
        if hlines:
            center_text += hline_text
        center_text += "\tModel" + (" & AIC" if aic else "") + (" & BIC " if bic else "") \
                       + (" & DIC " if dic else "") + end_text
        if hlines:
            center_text += "\t" + hline_text
        if aic:
            aics = self.aic()
        else:
            aics = np.zeros(len(self.parent.chains))
        if bic:
            bics = self.bic()
        else:
            bics = np.zeros(len(self.parent.chains))
        if dic:
            dics = self.dic()
        else:
            dics = np.zeros(len(self.parent.chains))

        if sort == "bic":
            to_sort = bics
        elif sort == "aic":
            to_sort = aics
        elif sort == "dic":
            to_sort = dics
        else:
            raise ValueError("sort %s not recognised, must be dic, aic or dic" % sort)

        good = [i for i, t in enumerate(to_sort) if t is not None]
        names = [self.parent.chains[g].name for g in good]
        aics = [aics[g] for g in good]
        bics = [bics[g] for g in good]
        to_sort = bics if sort == "bic" else aics

        indexes = np.argsort(to_sort)

        if descending:
            indexes = indexes[::-1]

        for i in indexes:
            line = "\t" + names[i]
            if aic:
                line += "  &  %5.1f  " % aics[i]
            if bic:
                line += "  &  %5.1f  " % bics[i]
            if dic:
                line += "  &  %5.1f  " % dics[i]
            line += end_text
            center_text += line
        if hlines:
            center_text += "\t" + hline_text

        return base_string % (column_text, center_text)


Instruction:
Write a Python 3 script for
estimating un - normalised probability density at target points.

Response:
def evaluate(self, data):
        """ Estimate un-normalised probability density at target points

        Parameters
        ----------
        data : np.ndarray
            A `(num_targets, num_dim)` array of points to investigate.

        Returns
        -------
        np.ndarray
            A `(num_targets)` length array of estimates

        Returns array of probability densities
        """
        if len(data.shape) == 1 and self.num_dim == 1:
            data = np.atleast_2d(data).T

        _d = np.dot(data - self.mean, self.A)

        # Get all points within range of kernels
        neighbors = self.tree.query_ball_point(_d, self.sigma * self.truncation)
        out = []
        for i, n in enumerate(neighbors):
            if len(n) >= self.nmin:
                diff = self.d[n, :] - _d[i]
                distsq = np.sum(diff * diff, axis=1)
            else:
                # If too few points get nmin closest
                dist, n = self.tree.query(_d[i], k=self.nmin)
                distsq = dist * dist
            out.append(np.sum(self.weights[n] * np.exp(self.sigma_fact * distsq)))
        return np.array(out)