Instruction:
Here you have a function in Python 3, explain what it does
def save_array(self, array, name=None, partname=None, rootpath='.',
                   raw=False, as_int=True):
        """
        Standard array saving routine

        Parameters
        -----------
        array : array
            Array to save to file
        name : str, optional
            Default 'array.tif'. Filename of array to save. Over-writes
            partname.
        partname : str, optional
            Part of the filename to save (with the coordinates appended)
        rootpath : str, optional
            Default '.'. Which directory to save file
        raw : bool, optional
            Default False. If true will save a .npz of the array. If false,
            will save a geotiff
        as_int : bool, optional
            Default True. If true will save array as an integer array (
            excellent compression). If false will save as float array.
        """
        if name is None and partname is not None:
            fnl_file = self.get_full_fn(partname, rootpath)
            tmp_file = os.path.join(rootpath, partname,
                                    self.get_fn(partname + '_tmp'))
        elif name is not None:
            fnl_file = name
            tmp_file = fnl_file + '_tmp.tiff'
        else:
            fnl_file = 'array.tif'
        if not raw:
            s_file = self.elev.clone_traits()
            s_file.raster_data = np.ma.masked_array(array)
            count = 10
            while count > 0 and (s_file.raster_data.mask.sum() > 0 \
                    or np.isnan(s_file.raster_data).sum() > 0):
                s_file.inpaint()
                count -= 1

            s_file.export_to_geotiff(tmp_file)

            if as_int:
                cmd = "gdalwarp -multi -wm 2000 -co BIGTIFF=YES -of GTiff -co compress=lzw -ot Int16 -co TILED=YES -wo OPTIMIZE_SIZE=YES -r near -t_srs %s %s %s" \
                    % (self.save_projection, tmp_file, fnl_file)
            else:
                cmd = "gdalwarp -multi -wm 2000 -co BIGTIFF=YES -of GTiff -co compress=lzw -co TILED=YES -wo OPTIMIZE_SIZE=YES -r near -t_srs %s %s %s" \
                    % (self.save_projection, tmp_file, fnl_file)
            print "<<"*4, cmd, ">>"*4
            subprocess.call(cmd)
            os.remove(tmp_file)
        else:
            np.savez_compressed(fnl_file, array)

Response: Standard array saving routine.


Instruction:
Can you create a Python 3 function that
saves the upstream contributing area to a file

Response:
def save_uca(self, rootpath, raw=False, as_int=False):
        """ Saves the upstream contributing area to a file
        """
        self.save_array(self.uca, None, 'uca', rootpath, raw, as_int=as_int)


Instruction:
Create a Python 3 function to
save the topographic wetness index to a file.

Response:
def save_twi(self, rootpath, raw=False, as_int=True):
        """ Saves the topographic wetness index to a file
        """
        self.twi = np.ma.masked_array(self.twi, mask=self.twi <= 0,
                                      fill_value=-9999)
        #  self.twi = self.twi.filled()
        self.twi[self.flats] = 0
        self.twi.mask[self.flats] = True
        # self.twi = self.flats
        self.save_array(self.twi, None, 'twi', rootpath, raw, as_int=as_int)


Instruction:
Given the following Python 3 function, write the documentation
def save_slope(self, rootpath, raw=False, as_int=False):
        self.save_array(self.mag, None, 'mag', rootpath, raw, as_int=as_int)

Response: Saves the magnitude of the slope to a file.


Instruction:
Implement a function in Python 3 to
save the direction of the slope to a file

Response:
def save_direction(self, rootpath, raw=False, as_int=False):
        """ Saves the direction of the slope to a file
        """
        self.save_array(self.direction, None, 'ang', rootpath, raw, as_int=as_int)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def save_outputs(self, rootpath='.', raw=False):
        self.save_twi(rootpath, raw)
        self.save_uca(rootpath, raw)
        self.save_slope(rootpath, raw)
        self.save_direction(rootpath, raw)

Response: Saves TWI UCA magnitude and direction of slope to files.


Instruction:
How would you explain what the following Python 3 function does
def load_array(self, fn, name):

        if os.path.exists(fn + '.npz'):
            array = np.load(fn + '.npz')
            try:
                setattr(self, name, array['arr_0'])
            except Exception, e:
                print e
            finally:
                array.close()

        else:
            raise RuntimeError("File %s does not exist." % (fn + '.npz'))

Response: Loads an array from a file and sets the attribute name to the value of the attribute name.


Instruction:
Write a Python 3 script to
calculate the edges of the array that gives the nominal size of the array and the overlap of the chunks.

Response:
def _get_chunk_edges(self, NN, chunk_size, chunk_overlap):
        """
        Given the size of the array, calculate and array that gives the
        edges of chunks of nominal size, with specified overlap
        Parameters
        ----------
        NN : int
            Size of array
        chunk_size : int
            Nominal size of chunks (chunk_size < NN)
        chunk_overlap : int
            Number of pixels chunks will overlap
        Returns
        -------
        start_id : array
            The starting id of a chunk. start_id[i] gives the starting id of
            the i'th chunk
        end_id : array
            The ending id of a chunk. end_id[i] gives the ending id of
            the i'th chunk
        """
        left_edge = np.arange(0, NN - chunk_overlap, chunk_size)
        left_edge[1:] -= chunk_overlap
        right_edge = np.arange(0, NN - chunk_overlap, chunk_size)
        right_edge[:-1] = right_edge[1:] + chunk_overlap
        right_edge[-1] = NN
        right_edge = np.minimum(right_edge, NN)
        return left_edge, right_edge


Instruction:
Can you implement a function in Python 3 that
assigns data from a chunk to a full array.

Response:
def _assign_chunk(self, data, arr1, arr2, te, be, le, re, ovr, add=False):
        """
        Assign data from a chunk to the full array. The data in overlap regions
        will not be assigned to the full array

        Parameters
        -----------
        data : array
            Unused array (except for shape) that has size of full tile
        arr1 : array
            Full size array to which data will be assigned
        arr2 : array
            Chunk-sized array from which data will be assigned
        te : int
            Top edge id
        be : int
            Bottom edge id
        le : int
            Left edge id
        re : int
            Right edge id
        ovr : int
            The number of pixels in the overlap
        add : bool, optional
            Default False. If true, the data in arr2 will be added to arr1,
            otherwise data in arr2 will overwrite data in arr1
        """

        if te == 0:
            i1 = 0
        else:
            i1 = ovr

        if be == data.shape[0]:
            i2 = 0
            i2b = None
        else:
            i2 = -ovr
            i2b = -ovr

        if le == 0:
            j1 = 0
        else:
            j1 = ovr

        if re == data.shape[1]:
            j2 = 0
            j2b = None
        else:
            j2 = -ovr
            j2b = -ovr

        if add:
            arr1[te+i1:be+i2, le+j1:re+j2] += arr2[i1:i2b, j1:j2b]
        else:
            arr1[te+i1:be+i2, le+j1:re+j2] = arr2[i1:i2b, j1:j2b]


Instruction:
Can you write a function in Python 3 where it
calculates the magnitude and direction of slopes and fills self. data

Response:
def calc_slopes_directions(self, plotflag=False):
        """
        Calculates the magnitude and direction of slopes and fills
        self.mag, self.direction
        """

        # TODO minimum filter behavior with nans?

        # fill/interpolate flats first
        if self.fill_flats:
            data = np.ma.filled(self.data.astype('float64'), np.nan)
            filled = data.copy()

            edge = np.ones_like(data, bool)
            edge[1:-1, 1:-1] = False

            if self.fill_flats_below_sea: sea_mask = data != 0
            else: sea_mask = data > 0
            flat = (spndi.minimum_filter(data, (3, 3)) >= data) & sea_mask

            flats, n = spndi.label(flat, structure=FLATS_KERNEL3)
            objs = spndi.find_objects(flats)

            for i, _obj in enumerate(objs):
                obj = grow_obj(_obj, data.shape)
                self._fill_flat(data[obj], filled[obj], flats[obj]==i+1, edge[obj])

            self.data = np.ma.masked_array(filled, mask=np.isnan(filled)).astype(self.data.dtype)

        # %% Calculate the slopes and directions based on the 8 sections from
        # Tarboton http://www.neng.usu.edu/cee/faculty/dtarb/96wr03137.pdf
        if self.data.shape[0] <= self.chunk_size_slp_dir and \
                self.data.shape[1] <= self.chunk_size_slp_dir:
            print "starting slope/direction calculation"
            self.mag, self.direction = self._slopes_directions(
                self.data, self.dX, self.dY, 'tarboton')
            # Find the flat regions. This is mostly simple (look for mag < 0),
            # but the downstream pixel at the edge of a flat will have a
            # calcuable angle which will not be accurate. We have to also find
            # these edges and set their magnitude to -1 (that is, the flat_id)

            self.find_flats()
        else:
            self.direction = np.full(self.data.shape, FLAT_ID_INT, 'float64')
            self.mag = np.full(self.data.shape, FLAT_ID_INT, 'float64')
            self.flats = np.zeros(self.data.shape, bool)
            top_edge, bottom_edge = \
                self._get_chunk_edges(self.data.shape[0],
                                      self.chunk_size_slp_dir,
                                      self.chunk_overlap_slp_dir)
            left_edge, right_edge = \
                self._get_chunk_edges(self.data.shape[1],
                                      self.chunk_size_slp_dir,
                                      self.chunk_overlap_slp_dir)
            ovr = self.chunk_overlap_slp_dir
            count = 1
            for te, be in zip(top_edge, bottom_edge):
                for le, re in zip(left_edge, right_edge):
                    print "starting slope/direction calculation for chunk", \
                        count, "[%d:%d, %d:%d]" % (te, be, le, re)
                    count += 1
                    mag, direction = \
                        self._slopes_directions(self.data[te:be, le:re],
                                                self.dX[te:be-1],
                                                self.dY[te:be-1])

                    flats = self._find_flats_edges(self.data[te:be, le:re],
                                                   mag, direction)

                    direction[flats] = FLAT_ID_INT
                    mag[flats] = FLAT_ID_INT
                    self._assign_chunk(self.data, self.mag, mag,
                                       te, be, le, re, ovr)
                    self._assign_chunk(self.data, self.direction, direction,
                                       te, be, le, re, ovr)
                    self._assign_chunk(self.data, self.flats, flats,
                                       te, be, le, re, ovr)

        if plotflag:
            self._plot_debug_slopes_directions()

        gc.collect()  # Just in case
        return self.mag, self.direction


Instruction:
Explain what the following Python 3 code does
def _slopes_directions(self, data, dX, dY, method='tarboton'):
        # %%
        if method == 'tarboton':
            return self._tarboton_slopes_directions(data, dX, dY)
        elif method == 'central':
            return self._central_slopes_directions(data, dX, dY)

Response: Wrapper to pick between various algorithms


Instruction:
Given the following Python 3 function, write the documentation
def _tarboton_slopes_directions(self, data, dX, dY):

        return _tarboton_slopes_directions(data, dX, dY,
                                           self.facets, self.ang_adj)

Response: Calculate the slopes and directions based on Tarboton s section 3. 1. 1.


Instruction:
Given the following Python 3 function, write the documentation
def _central_slopes_directions(self, data, dX, dY):
        shp = np.array(data.shape) - 1

        direction = np.full(data.shape, FLAT_ID_INT, 'float64')
        mag = np.full(direction, FLAT_ID_INT, 'float64')

        ind = 0
        d1, d2, theta = _get_d1_d2(dX, dY, ind, [0, 1], [1, 1], shp)
        s2 = (data[0:-2, 1:-1] - data[2:, 1:-1]) / d2
        s1 = -(data[1:-1, 0:-2] - data[1:-1, 2:]) / d1
        direction[1:-1, 1:-1] = np.arctan2(s2, s1) + np.pi
        mag = np.sqrt(s1**2 + s2**2)

        return mag, direction

Response: Calculates the magnitude and direction of slopes using central difference


Instruction:
How would you code a function in Python 3 to
find the flats that are adjacent to the downstream side of the flat.

Response:
def _find_flats_edges(self, data, mag, direction):
        """
        Extend flats 1 square downstream
        Flats on the downstream side of the flat might find a valid angle,
        but that doesn't mean that it's a correct angle. We have to find
        these and then set them equal to a flat
        """

        i12 = np.arange(data.size).reshape(data.shape)

        flat = mag == FLAT_ID_INT
        flats, n = spndi.label(flat, structure=FLATS_KERNEL3)
        objs = spndi.find_objects(flats)

        f = flat.ravel()
        d = data.ravel()
        for i, _obj in enumerate(objs):
            region = flats[_obj] == i+1
            I = i12[_obj][region]
            J = get_adjacent_index(I, data.shape, data.size)
            f[J] = d[J] == d[I[0]]

        flat = f.reshape(data.shape)
        return flat


Instruction:
Given the following Python 3 function, write the documentation
def calc_uca(self, plotflag=False, edge_init_data=None, uca_init=None):
        if self.direction is None:
            self.calc_slopes_directions()

        # Initialize the upstream area
        uca_edge_init = np.zeros(self.data.shape, 'float64')
        uca_edge_done = np.zeros(self.data.shape, bool)
        uca_edge_todo = np.zeros(self.data.shape, bool)
        edge_init_done, edge_init_todo = None, None
        if edge_init_data is not None:
            edge_init_data, edge_init_done, edge_init_todo = edge_init_data
            slices = {'left': [slice(None), slice(0, 1)],
                      'right': [slice(None), slice(-1, None)],
                      'top': [slice(0, 1), slice(None)],
                      'bottom': [slice(-1, None), slice(None)]}
            for key, val in slices.iteritems():
                # To initialize and edge it needs to have data and be finished
                uca_edge_done[val] += \
                    edge_init_done[key].reshape(uca_edge_init[val].shape)
                uca_edge_init[val] = \
                    edge_init_data[key].reshape(uca_edge_init[val].shape)
                uca_edge_init[val][~uca_edge_done[val]] = 0
                uca_edge_todo[val] += \
                    edge_init_todo[key].reshape(uca_edge_init[val].shape)

        if uca_init is None:
            self.uca = np.full(self.data.shape, FLAT_ID_INT, 'float64')
        else:
            self.uca = uca_init.astype('float64')

        if self.data.shape[0] <= self.chunk_size_uca and \
                self.data.shape[1] <= self.chunk_size_uca:
            if uca_init is None:
                print "Starting uca calculation"
                res = self._calc_uca_chunk(self.data, self.dX, self.dY,
                                           self.direction, self.mag,
                                           self.flats,
                                           area_edges=uca_edge_init,
                                           plotflag=plotflag)
                self.edge_todo = res[1]
                self.edge_done = res[2]
                self.uca = res[0]
            else:
                print "Starting edge resolution round: ",
                # last return value will be None: edge_
                area, e2doi, edone, _ = \
                    self._calc_uca_chunk_update(self.data, self.dX, self.dY,
                                                self.direction, self.mag,
                                                self.flats,
                                                area_edges=uca_edge_init,
                                                edge_todo=uca_edge_todo,
                                                edge_done=uca_edge_done)
                self.uca += area
                self.edge_todo = e2doi
                self.edge_done = edone

        else:
            top_edge, bottom_edge = \
                self._get_chunk_edges(self.data.shape[0], self.chunk_size_uca,
                                      self.chunk_overlap_uca)
            left_edge, right_edge = \
                self._get_chunk_edges(self.data.shape[1], self.chunk_size_uca,
                                      self.chunk_overlap_uca)
            ovr = self.chunk_overlap_uca

            # Initialize the edge_todo and done arrays
            edge_todo = np.zeros(self.data.shape, bool)
            edge_todo_tile = np.zeros(self.data.shape, bool)
            edge_not_done_tile = np.zeros(self.data.shape, bool)
            edge_done = np.zeros(self.data.shape, bool)

            tile_edge = TileEdge(top_edge, bottom_edge, left_edge,
                                 right_edge, ovr,
                                 self.elev.grid_coordinates.x_axis,
                                 self.elev.grid_coordinates.y_axis, self.data)
            count = 1

            # Mask out the edges because we're just trying to resolve the
            # internal edge conflicts
            self.data.mask[:, 0] = True
            self.data.mask[:, -1] = True
            self.data.mask[0, :] = True
            self.data.mask[-1, :] = True

            # if 1:  # uca_init == None:
            print "Starting uca calculation for chunk: ",
            # %%
            for te, be in zip(top_edge, bottom_edge):
                for le, re in zip(left_edge, right_edge):
                    print count, "[%d:%d, %d:%d]" % (te, be, le, re),
                    count += 1
                    area, e2doi, edone, e2doi_no_mask, e2o_no_mask = \
                        self._calc_uca_chunk(self.data[te:be, le:re],
                                             self.dX[te:be-1],
                                             self.dY[te:be-1],
                                             self.direction[te:be, le:re],
                                             self.mag[te:be, le:re],
                                             self.flats[te:be, le:re],
                                             area_edges=uca_edge_init[te:be, le:re],
                                             plotflag=plotflag,
                                             edge_todo_i_no_mask=uca_edge_todo[te:be, le:re])
                    self._assign_chunk(self.data, self.uca, area,
                                       te, be, le, re, ovr)
                    edge_todo[te:be, le:re] += e2doi
                    edge_not_done_tile[te:be, le:re] += e2o_no_mask
                    # if this tile is on the edge of the domain, we actually
                    # want to keep the edge information
                    # UPDATE: I don't think we actually need this here as it
                    # will be handled by chunk update ???
                    self._assign_chunk(self.data, edge_todo_tile, e2doi_no_mask,
                                       te, be, le, re, ovr)
#                    if te == top_edge[0] or be == bottom_edge[-1] \
#                            or le == left_edge[0] or re == right_edge[-1]:
#                        edge_todo_tile[te:be, le:re] = e2doi
                    self._assign_chunk(self.data, edge_done, edone,
                                       te, be, le, re, ovr)
                    tile_edge.set_all_neighbors_data(self.uca,
                                                     edge_done,
                                                     (te, be, le, re))
                    tile_edge.set_sides((te, be, le, re), e2doi, 'todo',
                                        local=True)
            # %%
            print '..Done'
            # This needs to be much more sophisticated because we have to
            # follow the tile's edge value through the interior.
            # Since we have to do that anyway, we might as well recompute
            # the UCA from scratch. So the above branch does that. The branch
            # below would be more efficient if we can get it working.
            if 0:  # else:
                # need to populate tile_edge somehow
                edge_todo_tile = uca_edge_todo & ~uca_edge_done
                edge_not_done_tile = edge_todo_tile.copy()
                for te, be in zip(top_edge, bottom_edge):
                    for le, re in zip(
                            [left_edge[0], left_edge[-1]],
                            [right_edge[0], right_edge[-1]]):
                        e2doi = uca_edge_todo[te:be, le:re]
                        tiledata = uca_edge_init[te:be, le:re]
                        tile_edge.set_sides((te, be, le, re), tiledata, 'data',
                                            local=True)
                        tiledone = uca_edge_done[te:be, le:re]
                        tile_edge.set_sides((te, be, le, re), tiledone, 'done',
                                            local=True)
                for te, be in zip([top_edge[0], top_edge[-1]],
                                  [bottom_edge[0], bottom_edge[-1]]):
                    for le, re in zip(left_edge, right_edge):
                        e2doi = uca_edge_todo[te:be, le:re]
                        tiledata = uca_edge_init[te:be, le:re]
                        tile_edge.set_sides((te, be, le, re), tiledata, 'data',
                                            local=True)
                        tiledone = uca_edge_done[te:be, le:re]
                        tile_edge.set_sides((te, be, le, re), tiledone, 'done',
                                            local=True)

            if not self.resolve_edges:
                # This branch is probably horribly broken (but it might have
                # always been that way)
                self.tile_edge = tile_edge
                self.edge_todo = edge_todo
                self.edge_done = edge_done
                return self.uca

            # ## RESOLVING EDGES ## #

            # Get a good starting tile for the iteration
            i = tile_edge.find_best_candidate()
            tile_edge.fix_shapes()
#            dbug = np.zeros_like(self.uca)
            print "Starting edge resolution round: ",
            count = 0
            i_old = -1
            while i is not None and i != i_old:
                count += 1
                print count, '(%d) .' % i,
                # %%
                te, be, le, re = tile_edge.coords[i]
                data, dX, dY, direction, mag, flats = \
                    [self.data[te:be, le:re],
                     self.dX[te:be-1], self.dY[te:be-1],
                     self.direction[te:be, le:re],
                     self.mag[te:be, le:re], self.flats[te:be, le:re]]
                area, e2doi, edone, e2doi_tile = self._calc_uca_chunk_update(
                    data, dX, dY, direction, mag, flats, tile_edge, i,
                    edge_todo=edge_not_done_tile[te:be, le:re])
                self._assign_chunk(self.data, self.uca, area,
                                   te, be, le, re, ovr, add=True)
                self._assign_chunk(self.data, edge_done, edone,
                                   te, be, le, re, ovr)
                tile_edge.set_all_neighbors_data(self.uca,
                                                 edge_done, (te, be, le, re))

                try:
                    edge_not_done_tile[te:be, le:re] += e2doi_tile
                except:
                    import ipdb; ipdb.set_trace() # BREAKPOINT
                tile_edge.set_sides((te, be, le, re), e2doi, 'todo',
                                    local=True)
                i_old = i
                i = tile_edge.find_best_candidate()
                # Debugging plots below. Feel free to uncomment for debugging
#                def drawgrid():
#                    ax = gca();
#                    ax.set_xticks(np.linspace(-0.5, 63.5, 9))
#                    ax.set_yticks(np.linspace(-0.5, 63.5, 9))
#                    grid(lw=2, ls='-', c=(0.5, 0.5, 0.5))
#                figure(1);clf();imshow((self.uca), interpolation='none');colorbar(); title("uca" + str(i_old) + " " + str(i));drawgrid()
#                figure(2);clf();imshow(area, interpolation='none');colorbar(); title("local area" + str(i_old) + " " + str(i))
##                edge_todo[:] = 0
##                edge_todo = tile_edge.fill_array(edge_todo, 'todo', add=True)
#                figure(3);clf();imshow(edge_todo*1.0 + edge_done*2.0, interpolation='none');colorbar(); title("todo" + str(i_old) + " " + str(i));clim(0, 3)
#                edge_todo[:] = 0
#                edge_todo = tile_edge.fill_array(edge_todo, 'coulddo', add=True)
#                figure(3);clf();imshow(edge_todo*1.0 + edge_done*2.0, interpolation='none');colorbar(); title("todo" + str(i_old) + " " + str(i));clim(0, 3);drawgrid()
#                figure(4);clf();imshow(tile_edge.percent_done, interpolation='none');colorbar(); title("percent done" + str(i_old) + " " + str(i));clim(0, 1)
#                dbug[:] = 0
#                dbug = tile_edge.fill_array(dbug, 'coulddo', maximize=False)
#                dbug[dbug > 0] -= (self.uca - ref_area)[dbug > 0]
#                figure(5);clf();imshow(dbug, interpolation='none');colorbar(); title("data diff" + str(i_old) + " " + str(i));drawgrid()
#                dbug = (self.uca - area1)
#                figure(6);clf();imshow(np.log10(np.abs(dbug)), interpolation='none');colorbar(); title("uca diff" + str(i_old) + " " + str(i));drawgrid()
                # %%
            self.tile_edge = tile_edge
            self.edge_todo = edge_todo_tile
            self.edge_done = ~edge_not_done_tile
        print '..Done'

        # Fix the very last pixel on the edges
        self.fix_edge_pixels(edge_init_data, edge_init_done, edge_init_todo)

        gc.collect()  # Just in case
        return self.uca

Response: Calculates the upstream contributing area of the key - value entry.


Instruction:
Explain what the following Python 3 code does
def fix_edge_pixels(self, edge_init_data, edge_init_done, edge_init_todo):
        data, dX, dY, direction, flats = \
            self.data, self.dX, self.dY, self.direction, self.flats
        sides = ['left', 'right', 'top', 'bottom']
        slices_o = [[slice(None), slice(1, 2)], [slice(None), slice(-2, -1)],
                    [slice(1, 2), slice(None)], [slice(-2, -1), slice(None)]]
        slices_d = [[slice(None), slice(0, 1)], [slice(None), slice(-1, None)],
                    [slice(0, 1), slice(None)], [slice(-1, None), slice(None)]]

        # The first set of edges will have contributions from two nodes whereas
        # the second set of edges will only have contributinos from one node
        indices = {'left': [[3, 4], [2, 5]], 'right': [[0, 7], [1, 6]],
                   'top': [[1, 2], [0, 3]], 'bottom': [[5, 6], [4, 7]]}

        # Figure out which section the drainage goes towards, and what
        # proportion goes to the straight-sided (as opposed to diagonal) node.

        for side, slice_o, slice_d in zip(sides, slices_o, slices_d):
            section, proportion = \
                self._calc_uca_section_proportion(data[slice_o],
                                                  dX[slice_o[0]],
                                                  dY[slice_o[0]],
                                                  direction[slice_o],
                                                  flats[slice_o])
            # self-initialize:
            if side in ['left', 'right']:
                self.uca[slice_d] = \
                    np.concatenate(([dX[slice_d[0]][0] * dY[slice_d[0]][0]],
                                    dX[slice_d[0]] * dY[slice_d[0]]))\
                    .reshape(self.uca[slice_d].shape)
            else:
                self.uca[slice_d] = dX[slice_d[0]][0] * dY[slice_d[0]][0]
            for e in range(2):
                for i in indices[side][e]:
                    ed = self.facets[i][2]
                    ids = section == i
                    if e == 0:
                        self.uca[slice_d][ids] += self.uca[slice_o][ids] \
                            * proportion[ids]
                    self.uca[slice_d][ids] += \
                        np.roll(np.roll(self.uca[slice_o] * (1 - proportion),
                                        ed[0], 0),
                                ed[1], 1)[ids]
                    if e == 1:
                        self.uca[slice_d][ids] += \
                            np.roll(np.roll(self.uca[slice_o] * (proportion),
                                            ed[0], 0),
                                    ed[1], 1)[ids]

            # Finally, add the edge data from adjacent tiles
            if edge_init_done is not None:
                ids = edge_init_done[side]  # > 0
                if side in ['left', 'right']:
                    self.uca[slice_d][ids, :] = \
                        edge_init_data[side][ids][:, None]
                else:
                    self.uca[slice_d][:, ids] = edge_init_data[side][ids]

Response: This function fixes the pixels on the very edge of the tile.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _calc_uca_chunk_update(self, data, dX, dY, direction, mag, flats,
                               tile_edge=None, i=None,
                               area_edges=None, edge_todo=None, edge_done=None,
                               plotflag=False):
        """
        Calculates the upstream contributing area due to contributions from
        the edges only.
        """
        # %%

        sides = ['left', 'right', 'top', 'bottom']
        slices = [[slice(None), slice(0, 1)], [slice(None), slice(-1, None)],
                  [slice(0, 1), slice(None)], [slice(-1, None), slice(None)]]
        # Figure out which section the drainage goes towards, and what
        # proportion goes to the straight-sided (as opposed to diagonal) node.

        section, proportion = self._calc_uca_section_proportion(
            data, dX, dY, direction, flats)

        # Build the drainage or adjacency matrix
        A = self._mk_adjacency_matrix(section, proportion, flats, data, mag, dX, dY)
        if CYTHON:
            B = A
            C = A.tocsr()
        if not CYTHON:
            A = A.tocoo()

        ids = np.zeros(data.shape, bool)
        area = np.zeros(data.shape, 'float64')
        # Set the ids to the edges that are now done, and initialize the
        # edge area
        if tile_edge is not None:
            if edge_todo is not None:
                edge_todo_tile = edge_todo
            else:
                edge_todo_tile = None
            edge_todo = np.zeros(data.shape, bool)
            for side, slice0 in zip(sides, slices):
                edge = getattr(tile_edge, side).ravel()[i]
                ids[slice0] = edge.done & edge.coulddo
                # only add area from the finished edges
                area[slice0] = edge.data * edge.done * edge.coulddo
                edge_todo[slice0] = edge.todo & ~edge.done
        elif area_edges is not None and edge_todo is not None \
                and edge_done is not None:
            area[:, 0] = area_edges[:, 0]
            area[:, -1] = area_edges[:, -1]
            area[-1, :] = area_edges[-1, :]
            area[0, :] = area_edges[0, :]

            # Initialize starting ids
            ids = edge_done & edge_todo
            edge_todo = edge_todo & ~edge_done
            edge_todo_tile = None
        else:
            raise RuntimeError("Need to specify either tile_edge or area_edges"
                               "in _calc_uca_chunk_update")

        ids = ids.ravel()
        ids0 = ids.copy()

        area[flats] = np.nan
        edge_done = ~edge_todo

        edge_todo_i = edge_todo.copy()
        ids_old = np.zeros_like(ids)

        # I need this to keep track of when I have to add the area, and when
        # I have to replace the area.
        ids_i = np.arange(ids.size)

        done = np.ones(data.shape, bool)
        done.ravel()[ids] = False

        # Now we have to advance done through the mesh to figure out which
        # contributions matter (i.e. what's done already)
        def drain_pixels_done(ids, arr, rows_A, cols_A):
            ids_old = ids.copy()
            ids_old[:] = False
            # If I use ids.sum() > 0 then I might get stuck in circular
            # references.
            while (ids - ids_old).sum() > 0:
                # %%
                print "x",
                ids_old = ids.copy()
                ids_todo = ids_i[ids.ravel()]
                ids[:] = False
                for id_todo in ids_todo:
                    rows = cols_A == id_todo
                    rows_id = rows_A[rows]
                    ids[rows_id] += arr.ravel()[rows_id] is True
                    arr.ravel()[rows_id] = False  # Set second arrival new id

            return arr

        if CYTHON:
            a = cyutils.drain_connections(
                done.ravel(), ids, B.indptr, B.indices, set_to=False)
            done = a.reshape(done.shape).astype(bool)
        else:
            done = drain_pixels_done(ids, done, A.row, A.col)

        done[data.mask] = True  # deal with no-data values
        #
        ids = ids0.copy()
        # Set all the edges to "done" for ids0. This ensures that no edges
        # will ever be updated, whether they are done or not.
        ids0 = ids0.reshape(data.shape)
        ids0[:, 0] = True
        ids0[:, -1] = True
        ids0[0, :] = True
        ids0[-1, :] = True
        ids0 = ids0.ravel()

        ids_old[:] = 0

        # %%
        def drain_area(ids, area, done, rows_A, cols_A, data_A,
                       edge_todo_tile):
            ids_old = ids.copy()
            ids_old[:] = False
            # If I use ids.sum() > 0 then I might get stuck in
            # circular references.
            while (ids - ids_old).sum() > 0:
                # %%
                print "o",
                ids_old = ids.copy()
                done.ravel()[ids] = True
                ids_todo = ids_i[ids.ravel()]
                ids[:] = False
                for id_todo in ids_todo:
                    rows = cols_A == id_todo
                    rows_id = rows_A[rows]
                    factor = data_A[rows]
                    # not allowed to modify edge values
                    edge_filter_ids = ~ids0[rows_id]
                    factor = factor[edge_filter_ids]
                    rows_id = rows_id[edge_filter_ids]
                    area.ravel()[rows_id] += area.ravel()[id_todo] * factor
                    if edge_todo_tile is not None:
                        edge_todo_tile.ravel()[rows_id] += \
                            edge_todo_tile.ravel()[id_todo] * factor
                    # Figure out of this cell that just received a contribution
                    # should give its contribution to what's next... i.e. make
                    # sure all inputs have been added together
                    for row_id in rows_id:  # this is the 'waiting' part
                        cols = cols_A[rows_A == row_id]
                        ids[row_id] += (~(done.ravel()[cols])).sum() == 0
#                        for col in cols:
#                            print 'row', row_id, 'col', col, 'done', done.ravel()[col]
                # Follow the drainage along. New candidates are cells that
                # just changed
                #ids = (track_id_old.ravel() == -1) \
                #        & (track_id_old.ravel() != track_id.ravel())
#                done.ravel()[ids] = True
    #            figure(7);clf();imshow(ids.reshape(mag.shape), interpolation='none')
    #            figure(8);clf();imshow(area, interpolation='none');colorbar()
    #            figure(9);clf();imshow(done, interpolation='none');colorbar()
    #            figure(10);clf();imshow(a + area - b, interpolation='none');colorbar()
                #%%
    #            self._plot_connectivity(A, data=data)
            return area, done, edge_todo_tile

        if CYTHON:
            if edge_todo_tile is not None:
                a, b, c, d = cyutils.drain_area(area.ravel(),
                                                done.ravel(),
                                                ids,
                                                B.indptr, B.indices, B.data,
                                                C.indptr, C.indices,
                                                area.shape[0], area.shape[1],
                                                edge_todo_tile.astype('float64').ravel(),
                                                skip_edge=True)
                edge_todo_tile = c.reshape(edge_todo_tile.shape)
            else:
                a, b, c, d = cyutils.drain_area(area.ravel(),
                                                done.ravel(),
                                                ids,
                                                B.indptr, B.indices, B.data,
                                                C.indptr, C.indices,
                                                area.shape[0], area.shape[1],
                                                skip_edge=True)
            area = a.reshape(area.shape)
            done = b.reshape(done.shape)

        else:
            area, done, edge_todo_tile = \
                drain_area(ids, area, done, A.row, A.col, A.data,
                           edge_todo_tile)

        # Rather unfortunately, we still have to follow through the boolean
        # edge_todo matrix...
        ids = edge_todo.copy().ravel()
        # %%

        def drain_pixels_todo(ids, arr, rows_A, cols_A):
            ids_old = ids.copy()
            ids_old[:] = False
            # If I use ids.sum() > 0 then I might get stuck in
            # circular references.
            while (ids - ids_old).sum() > 0:
                # %%
                print "x",
                ids_old = ids.copy()
#                edge_todo_old = arr.copy()
                ids_todo = ids_i[ids.ravel()]
                ids[:] = False
                for id_todo in ids_todo:
                    rows = cols_A == id_todo
                    rows_id = rows_A[rows]
                    ids[rows_id] += arr.ravel()[rows_id] == False
                    arr.ravel()[rows_id] = True  # Set new id of second arrival

#                #Follow the drainage along. New candidates are cells that just changed
#                ids = (edge_todo_old.ravel() != arr.ravel())
            return arr

        if CYTHON:
            a = cyutils.drain_connections(edge_todo.ravel(),
                                          ids, B.indptr, B.indices,
                                          set_to=True)
            edge_todo = a.reshape(edge_todo.shape).astype(bool)
        else:
            edge_todo = drain_pixels_todo(ids, edge_todo, A.row, A.col)

        area[flats] = np.nan
        edge_done = ~edge_todo

        return area, edge_todo_i, edge_done, edge_todo_tile

Response: Calculates the upstream contributing area due to contributions from the edges only.


Instruction:
Can you write a function in Python 3 where it
calculates the upstream contributing area for the interior and the upstream contributing area for the interior and the upstream contributing area for the interior and the upstream contributing area for the interior and the upstream contributing area for the interior.

Response:
def _calc_uca_chunk(self, data, dX, dY, direction, mag, flats,
                        area_edges, plotflag=False, edge_todo_i_no_mask=True):
        """
        Calculates the upstream contributing area for the interior, and
        includes edge contributions if they are provided through area_edges.
        """
        # %%
        # Figure out which section the drainage goes towards, and what
        # proportion goes to the straight-sided (as opposed to diagonal) node.
        section, proportion = self._calc_uca_section_proportion(
            data, dX, dY, direction, flats)

        # Build the drainage or adjacency matrix
        A = self._mk_adjacency_matrix(section, proportion, flats, data, mag, dX, dY)
        if CYTHON:
            B = A.tocsr()

        colsum = np.array(A.sum(1)).ravel()
        ids = colsum == 0  # If no one drains into me

        area = (dX * dY)
        # Record minimum area
        min_area = np.nanmin(area)
        self.twi_min_area = min(self.twi_min_area, min_area)

        area = np.concatenate((area[0:1], area)).reshape(area.size+1, 1)
        area = area.repeat(data.shape[1], 1)
        # Set the edge areas to zero, will add those contributions later
        area[:, 0] = area_edges[:, 0]
        area[:, -1] = area_edges[:, -1]
        area[-1, :] = area_edges[-1, :]
        area[0, :] = area_edges[0, :]
        # These edges are done, they have been drained already
        ids[area_edges.ravel() > 0] = True

        done = np.zeros(data.shape, bool)
        done.ravel()[ids] = True
        # deal with no-data values
        done[1:-1, 1:-1] = done[1:-1, 1:-1] | data.mask[1:-1, 1:-1]

        # Check the inlet edges
        edge_todo = np.zeros_like(done)
        ids_ed = np.arange(data.size).reshape(data.shape)
        # left
        edge_todo[:, 0] = (A[:, ids_ed[:, 0]].sum(0) > 0) \
            & (area_edges[:, 0] == 0)
        edge_todo[:, -1] = (A[:, ids_ed[:, -1]].sum(0) > 0) \
            & (area_edges[:, -1] == 0)
        edge_todo[0, :] = (A[:, ids_ed[0, :]].sum(0) > 0) \
            & (area_edges[0, :] == 0)
        edge_todo[-1, :] = (A[:, ids_ed[-1, :]].sum(0) > 0) \
            & (area_edges[-1, :] == 0)

        # Will do the tile-level doneness
        edge_todo_i_no_mask = edge_todo.copy() & edge_todo_i_no_mask
        edge_todo_no_mask = edge_todo_i_no_mask.copy()  # tile-level doneness
        edge_todo[data.mask] = False  # Don't do masked areas
        # Initialize done edges
        edge_todo_i = edge_todo.copy()
        ids_old = np.zeros_like(ids)
        # %%
        count = 1

        if CYTHON:
            area_ = area.ravel()
            done_ = done.ravel()
            edge_todo_ = edge_todo.astype('float64').ravel()
            edge_todo_no_mask_ = edge_todo_no_mask.astype('float64').ravel()
        data_ = data.ravel()

        while (np.any(~done) and count < self.circular_ref_maxcount):
            print ".",
            count += 1
            if CYTHON:
                area_, done_, edge_todo_, edge_todo_no_mask_ = cyutils.drain_area(area_,
                    done_, ids,
                    A.indptr, A.indices, A.data, B.indptr, B.indices,
                    area.shape[0], area.shape[1],
                    edge_todo_, edge_todo_no_mask_)
            else:
                # If I use ids.sum() > 0 then I might get stuck in
                # circular references.
                while (ids - ids_old).sum() > 0:
                    # %%
                    ids_old = ids.copy()
                    ids, area, done, edge_todo = \
                        self._drain_step(A, ids, area, done, edge_todo)
    #                figure(1);clf();imshow(area, interpolation='none');colorbar()
    #                figure(2);clf();imshow(ids.reshape(area.shape), interpolation='none');colorbar()
    #                figure(3);clf();imshow(done, interpolation='none');colorbar()
                done_ = done.ravel()
            #%%
            ids[:] = False
            max_elev = (data_ * (~done_)).max()
            ids[((data_ * (~done_) - max_elev) / max_elev > -0.01)] = True

        if CYTHON:
            area = area_.reshape(area.shape)
            done = done_.reshape(done.shape)
            edge_todo = edge_todo_.reshape(edge_todo.shape).astype(bool)
            edge_todo_no_mask = edge_todo_no_mask_.reshape(edge_todo_no_mask.shape).astype(bool)

        area[flats] = np.nan

        edge_done = ~edge_todo
        edge_done[data.mask] = True  # Don't do masked areas

        if self.apply_uca_limit_edges:
            # 2x because of bifurcations (maybe should be more than 2x, but
            # should be ok
            edge_done[area > self.uca_saturation_limit * 2 * min_area] = True

        # %%
        if plotflag:
            # TODO DTYPE
            self._plot_connectivity(A, (done.astype('float64') is False)
                                    + flats.astype('float64') * 2, [0, 3])
        return area, edge_todo_i, edge_done, edge_todo_i_no_mask, edge_todo_no_mask


Instruction:
Here you have a function in Python 3, explain what it does
def _drain_step(self, A, ids, area, done, edge_todo):
        # Only drain to cells that have a contribution
        A_todo = A[:, ids.ravel()]
        colsum = np.array(A_todo.sum(1)).ravel()
        # Only touch cells that actually receive a contribution
        # during this stage
        ids_new = colsum != 0
        # Is it possible that I may drain twice from my own cell?
        # -- No, I don't think so...
        # Is it possible that other cells may drain into me in
        # multiple iterations -- yes
        # Then say I check for when I'm done ensures that I don't drain until
        # everyone has drained into me
        area.ravel()[ids_new] += (A_todo[ids_new, :]
                                  * (area.ravel()[ids].ravel()))
        edge_todo.ravel()[ids_new] += (A_todo[ids_new, :]
                                       * (edge_todo.ravel()[ids].ravel()))
        # Figure out what's left to do.
        done.ravel()[ids] = True

        colsum = A * (~done.ravel())
        ids = colsum == 0
        # Figure out the new-undrained ids
        ids = ids & (~done.ravel())

        return ids, area, done, edge_todo

Response: Do a single step of the upstream contributing area calculation.


Instruction:
Can you tell what is the following Python 3 function doing
def _calc_uca_section_proportion(self, data, dX, dY, direction, flats):
        shp = np.array(data.shape) - 1

        facets = self.facets
        adjust = self.ang_adj[:, 1]

        d1, d2, theta = _get_d1_d2(dX, dY, 0, facets[0][1], facets[0][2], shp)
        if dX.size > 1:
            theta = np.row_stack((theta[0, :], theta, theta[-1, :]))
        # Which quadrant am I in?
        section = ((direction / np.pi * 2.0) // 1).astype('int8') # TODO DTYPE
        # Gets me in the quadrant
        quadrant = (direction - np.pi / 2.0 * section)
        proportion = np.full_like(quadrant, np.nan)
        # Now which section within the quadrant
        section = section * 2 \
            + (quadrant > theta.repeat(data.shape[1], 1)) * (section % 2 == 0) \
            + (quadrant > (np.pi/2 - theta.repeat(data.shape[1], 1))) \
            * (section % 2 == 1)  # greater than because of ties resolution b4

        # %%    Calculate proportion
        # As a side note, it's crazy to me how:
        #    _get_d1_d2 needs to use indices 0, 3, 4, 7,
        #    section uses even/odd (i.e. % 2)
        #    proportion uses indices (0, 1, 4, 5) {ALl of them different! ARG!}
        I1 = (section == 0) | (section == 1) | (section == 4) | (section == 5)
    #    I1 = section % 2 == 0
        I = I1 & (quadrant <= theta.repeat(data.shape[1], 1))
        proportion[I] = quadrant[I] / theta.repeat(data.shape[1], 1)[I]
        I = I1 & (quadrant > theta.repeat(data.shape[1], 1))
        proportion[I] = (quadrant[I] - theta.repeat(data.shape[1], 1)[I]) \
            / (np.pi / 2 - theta.repeat(data.shape[1], 1)[I])
        I = (~I1) & (quadrant <= (np.pi / 2 - theta.repeat(data.shape[1], 1)))
        proportion[I] = (quadrant[I]) \
            / (np.pi / 2 - theta.repeat(data.shape[1], 1)[I])
        I = (~I1) & (quadrant > (np.pi / 2 - theta.repeat(data.shape[1], 1)))
        proportion[I] = (quadrant[I] - (np.pi / 2 - theta.repeat(data.shape[1], 1)[I])) \
            / (theta.repeat(data.shape[1], 1)[I])
        # %%Finish Proportion Calculation
        section[flats] = FLAT_ID_INT
        proportion[flats] = FLAT_ID

        section[section == 8] = 0  # Fence-post error correction
        proportion = (1 + adjust[section]) / 2.0 - adjust[section] * proportion

        return section, proportion

Response: Calculate the uca section proportion of the drainage


Instruction:
Can you tell what is the following Python 3 function doing
def _mk_adjacency_matrix(self, section, proportion, flats, elev, mag, dX, dY):
        shp = section.shape
        mat_data = np.row_stack((proportion, 1 - proportion))
        NN = np.prod(shp)
        i12 = np.arange(NN).reshape(shp)
        j1 = - np.ones_like(i12)
        j2 = - np.ones_like(i12)

        # make the connectivity for the non-flats/pits
        j1, j2 = self._mk_connectivity(section, i12, j1, j2)
        j = np.row_stack((j1, j2))
        i = np.row_stack((i12, i12))

        # connectivity for flats/pits
        if self.drain_pits:
            pit_i, pit_j, pit_prop, flats, mag = \
                self._mk_connectivity_pits(i12, flats, elev, mag, dX, dY)

            j = np.concatenate([j.ravel(), pit_j]).astype('int64')
            i = np.concatenate([i.ravel(), pit_i]).astype('int64')
            mat_data = np.concatenate([mat_data.ravel(), pit_prop])

        elif self.drain_flats:
            j1, j2, mat_data, flat_i, flat_j, flat_prop = \
                self._mk_connectivity_flats(
                    i12, j1, j2, mat_data, flats, elev, mag)

            j = np.concatenate([j.ravel(), flat_j]).astype('int64')
            i = np.concatenate([i.ravel(), flat_j]).astype('int64')
            mat_data = np.concatenate([mat_data.ravel(), flat_prop])

        # This prevents no-data values, remove connections when not present,
        # and makes sure that floating point precision errors do not
        # create circular references where a lower elevation cell drains
        # to a higher elevation cell
        I = ~np.isnan(mat_data) & (j != -1) & (mat_data > 1e-8) \
            & (elev.ravel()[j] <= elev.ravel()[i])

        mat_data = mat_data[I]
        j = j[I]
        i = i[I]

        # %%Make the matrix and initialize
        # What is A? The row i area receives area contributions from the
        # entries in its columns. If all the entries in my columns have
        #  drained, then I can drain.
        A = sps.csc_matrix((mat_data.ravel(),
                            np.row_stack((j.ravel(), i.ravel()))),
                           shape=(NN, NN))
        normalize = np.array(A.sum(0) + 1e-16).squeeze()
        A = np.dot(A, sps.diags(1/normalize, 0))

        return A

Response: Calculates the adjacency matrix for the internal state of the logarithmic entry.


Instruction:
Explain what the following Python 3 code does
def _mk_connectivity_flats(self, i12, j1, j2, mat_data, flats, elev, mag):
        nn, mm = flats.shape
        NN = np.prod(flats.shape)
        # Label the flats
        assigned, n_flats = spndi.label(flats, FLATS_KERNEL3)

        flat_ids, flat_coords, flat_labelsf = _get_flat_ids(assigned)
        flat_j = [None] * n_flats
        flat_prop = [None] * n_flats
        flat_i = [None] * n_flats

        # Temporary array to find the flats
        edges = np.zeros_like(flats)
        # %% Calcute the flat drainage
        warn_flats = []
        for ii in xrange(n_flats):
            ids_flats = flat_ids[flat_coords[ii]:flat_coords[ii+1]]
            edges[:] = 0
            j = ids_flats % mm
            i = ids_flats // mm
            for iii in [-1, 0, 1]:
                for jjj in [-1, 0, 1]:
                    i_2 = i + iii
                    j_2 = j + jjj
                    ids_tmp = (i_2 >= 0) & (j_2 >= 0) & (i_2 < nn) & (j_2 < mm)
                    edges[i_2[ids_tmp], j_2[ids_tmp]] += \
                        FLATS_KERNEL3[iii+1, jjj+1]
            edges.ravel()[ids_flats] = 0
            ids_edge = np.argwhere(edges.ravel()).squeeze()

            flat_elev_loc = elev.ravel()[ids_flats]
            # It is possble for the edges to merge 2 flats, so we need to
            # take the lower elevation to avoid large circular regions
            flat_elev = flat_elev_loc.min()

            loc_elev = elev.ravel()[ids_edge]
            # Filter out any elevations larger than the flat elevation
            # TODO: Figure out if this should be <= or <
            I_filt = loc_elev < flat_elev
            try:
                loc_elev = loc_elev[I_filt]
                loc_slope = mag.ravel()[ids_edge][I_filt]
            except: # If this is fully masked out (i.e. inside a no-data area)
                loc_elev = np.array([])
                loc_slope = np.array([])

            loc_dx = self.dX.mean()

            # Now I have to figure out if I should just use the minimum or
            # distribute amongst many pixels on the flat boundary
            n = len(loc_slope)
            if n == 0:  # Flat does not have anywhere to drain
                # Let's see if the flat goes to the edge. If yes, we'll just
                # distribute the area along the edge.
                ids_flat_on_edge = ((ids_flats % mag.shape[1]) == 0) | \
                    ((ids_flats % mag.shape[1]) == (mag.shape[1] - 1)) | \
                    (ids_flats <= mag.shape[1]) | \
                    (ids_flats >= (mag.shape[1] * (mag.shape[0] - 1)))
                if ids_flat_on_edge.sum() == 0:
                    warn_flats.append(ii)
                    continue

                drain_ids = ids_flats[ids_flat_on_edge]
                loc_proportions = mag.ravel()[ids_flats[ids_flat_on_edge]]
                loc_proportions /= loc_proportions.sum()

                ids_flats = ids_flats[~ids_flat_on_edge]
                # This flat is entirely on the edge of the image
                if len(ids_flats) == 0:
                    # therefore, whatever drains into it is done.
                    continue
                flat_elev_loc = flat_elev_loc[~ids_flat_on_edge]
            else:  # Flat has a place to drain to
                min_edges = np.zeros(loc_slope.shape, bool)
                min_edges[np.argmin(loc_slope)] = True
                # Add to the min edges any edge that is within an error
                # tolerance as small as the minimum
                min_edges = (loc_slope + loc_slope * loc_dx / 2) \
                    >= loc_slope[min_edges]

                drain_ids = ids_edge[I_filt][min_edges]

                loc_proportions = loc_slope[min_edges]
                loc_proportions /= loc_proportions.sum()

            # Now distribute the connectivity amongst the chosen elevations
            # proportional to their slopes

            # First, let all the the ids in the flats drain to 1
            # flat id (for ease)
            one_id = np.zeros(ids_flats.size, bool)
            one_id[np.argmin(flat_elev_loc)] = True

            j1.ravel()[ids_flats[~one_id]] = ids_flats[one_id]
            mat_data.ravel()[ids_flats[~one_id]] = 1
            # Negative indices will be eliminated before making the matix
            j2.ravel()[ids_flats[~one_id]] = -1
            mat_data.ravel()[ids_flats[~one_id] + NN] = 0

            # Now drain the 1 flat to the drains
            j1.ravel()[ids_flats[one_id]] = drain_ids[0]
            mat_data.ravel()[ids_flats[one_id]] = loc_proportions[0]
            if len(drain_ids) > 1:
                j2.ravel()[ids_flats[one_id]] = drain_ids[1]
                mat_data.ravel()[ids_flats[one_id] + NN] = loc_proportions[1]

            if len(loc_proportions > 2):
                flat_j[ii] = drain_ids[2:]
                flat_prop[ii] = loc_proportions[2:]
                flat_i[ii] = np.ones(drain_ids[2:].size, 'int64') * ids_flats[one_id]
        try:
            flat_j = np.concatenate([fj for fj in flat_j if fj is not None])
            flat_prop = \
                np.concatenate([fp for fp in flat_prop if fp is not None])
            flat_i = np.concatenate([fi for fi in flat_i if fi is not None])
        except:
            flat_j = np.array([], 'int64')
            flat_prop = np.array([], 'float64')
            flat_i = np.array([], 'int64')

        if len(warn_flats) > 0:
            warnings.warn("Warning %d flats had no place" % len(warn_flats) +
                          " to drain to --> these are pits (check pit-remove"
                          "algorithm).")
        return j1, j2, mat_data, flat_i, flat_j, flat_prop

Response: This function calculates the connectivity between two flat regions.


Instruction:
Write a Python 3 function that can
calculate the topographic wetness index and saves the result in self. twi.

Response:
def calc_twi(self):
        """
        Calculates the topographic wetness index and saves the result in
        self.twi.

        Returns
        -------
        twi : array
            Array giving the topographic wetness index at each pixel
        """
        if self.uca is None:
            self.calc_uca()
            gc.collect()  # Just in case
        min_area = self.twi_min_area
        min_slope = self.twi_min_slope
        twi = self.uca.copy()
        if self.apply_twi_limits_on_uca:
            twi[twi > self.uca_saturation_limit * min_area] = \
                self.uca_saturation_limit * min_area
        gc.collect()  # Just in case
        twi = np.log((twi) / (self.mag + min_slope))
        # apply the cap
        if self.apply_twi_limits:
            twi_sat_value = \
                np.log(self.uca_saturation_limit * min_area / min_slope)
            twi[twi > twi_sat_value] = twi_sat_value
        # multiply by 10 for better integer resolution when storing
        self.twi = twi * 10

        gc.collect()  # Just in case
        return twi


Instruction:
Make a summary of the following Python 3 code
def _plot_connectivity(self, A, data=None, lims=[None, None]):

        if data is None:
            data = self.data

        B = A.tocoo()
        self._plot_connectivity_helper(B.col, B.row, B.data, data, lims)

Response: A debug function used to plot the adjacency matrix A.


Instruction:
How would you explain what the following Python 3 function does
def _plot_connectivity_helper(self, ii, ji, mat_datai, data, lims=[1, 8]):
        from matplotlib.pyplot import quiver, colorbar, clim,  matshow
        I = ~np.isnan(mat_datai) & (ji != -1) & (mat_datai >= 0)
        mat_data = mat_datai[I]
        j = ji[I]
        i = ii[I]
        x = i.astype(float) % data.shape[1]
        y = i.astype(float) // data.shape[1]
        x1 = (j.astype(float) % data.shape[1]).ravel()
        y1 = (j.astype(float) // data.shape[1]).ravel()
        nx = (x1 - x)
        ny = (y1 - y)
        matshow(data, cmap='gist_rainbow'); colorbar(); clim(lims)
        quiver(x, y, nx, ny, mat_data.ravel(), angles='xy', scale_units='xy',
               scale=1, cmap='bone')
        colorbar(); clim([0, 1])

Response: A debug function used to plot the adjacency matrix and connectivity matrix.


Instruction:
Make a summary of the following Python 3 code
def _plot_debug_slopes_directions(self):
        # %%
        from matplotlib.pyplot import matshow, colorbar, clim, title

        matshow(self.direction / np.pi * 180); colorbar(); clim(0, 360)
        title('Direction')

        mag2, direction2 = self._central_slopes_directions()
        matshow(direction2 / np.pi * 180.0); colorbar(); clim(0, 360)
        title('Direction (central difference)')

        matshow(self.mag); colorbar()
        title('Magnitude')
        matshow(mag2); colorbar(); title("Magnitude (Central difference)")

        # %%
        # Compare to Taudem
        filename = self.file_name
        os.chdir('testtiff')
        try:
            os.remove('test_ang.tif')
            os.remove('test_slp.tif')
        except:
            pass
        cmd = ('dinfflowdir -fel "%s" -ang "%s" -slp "%s"' %
               (os.path.split(filename)[-1], 'test_ang.tif', 'test_slp.tif'))
        taudem._run(cmd)

        td_file = GdalReader(file_name='test_ang.tif')
        td_ang, = td_file.raster_layers
        td_file2 = GdalReader(file_name='test_slp.tif')
        td_mag, = td_file2.raster_layers
        os.chdir('..')

        matshow(td_ang.raster_data / np.pi*180); clim(0, 360); colorbar()
        title('Taudem direction')
        matshow(td_mag.raster_data); colorbar()
        title('Taudem magnitude')

        matshow(self.data); colorbar()
        title('The test data (elevation)')

        diff = (td_ang.raster_data - self.direction) / np.pi * 180.0
        diff[np.abs(diff) > 300] = np.nan
        matshow(diff); colorbar(); clim([-1, 1])
        title('Taudem direction - calculated Direction')

        # normalize magnitudes
        mag2 = td_mag.raster_data
        mag2 /= np.nanmax(mag2)
        mag = self.mag.copy()
        mag /= np.nanmax(mag)
        matshow(mag - mag2); colorbar()
        title('Taudem magnitude - calculated magnitude')
        del td_file
        del td_file2
        del td_ang
        del td_mag

Response: A debug function to plot the direction calculated in various ways.


Instruction:
Explain what the following Python 3 code does
def clean(ctx, dry_run=False):
    basedir = ctx.sphinx.destdir or "build/docs"
    cleanup_dirs([basedir], dry_run=dry_run)

Response: Cleanup generated document artifacts.


Instruction:
Write a Python 3 function for
building docs with sphinx - build

Response:
def build(ctx, builder="html", options=""):
    """Build docs with sphinx-build"""
    sourcedir = ctx.config.sphinx.sourcedir
    destdir = Path(ctx.config.sphinx.destdir or "build")/builder
    destdir = destdir.abspath()
    with cd(sourcedir):
        destdir_relative = Path(".").relpathto(destdir)
        command = "sphinx-build {opts} -b {builder} {sourcedir} {destdir}" \
                    .format(builder=builder, sourcedir=".",
                            destdir=destdir_relative, opts=options)
        ctx.run(command)


Instruction:
Can you tell what is the following Python 3 function doing
def browse(ctx):
    page_html = Path(ctx.config.sphinx.destdir)/"html"/"index.html"
    if not page_html.exists():
        build(ctx, builder="html")
    assert page_html.exists()
    open_cmd = "open"   # -- WORKS ON: MACOSX
    if sys.platform.startswith("win"):
        open_cmd = "start"
    ctx.run("{open} {page_html}".format(open=open_cmd, page_html=page_html))

Response: Open documentation in web browser.


Instruction:
Can you tell what is the following Python 3 function doing
def save(ctx, dest="docs.html", format="html"):
    print("STEP: Generate docs in HTML format")
    build(ctx, builder=format)

    print("STEP: Save docs under %s/" % dest)
    source_dir = Path(ctx.config.sphinx.destdir)/format
    Path(dest).rmtree_p()
    source_dir.copytree(dest)

    # -- POST-PROCESSING: Polish up.
    for part in [ ".buildinfo", ".doctrees" ]:
        partpath = Path(dest)/part
        if partpath.isdir():
            partpath.rmtree_p()
        elif partpath.exists():
            partpath.remove_p()

Response: Save docs under destination directory.


Instruction:
Write a Python 3 function for
finding the tile neighbors based on filenames.

Response:
def find_neighbors(neighbors, coords, I, source_files, f, sides):
    """Find the tile neighbors based on filenames

    Parameters
    -----------
    neighbors : dict
        Dictionary that stores the neighbors. Format is
        neighbors["source_file_name"]["side"] = "neighbor_source_file_name"
    coords : list
        List of coordinates determined from the filename.
        See :py:func:`utils.parse_fn`
    I : array
        Sort index. Different sorting schemes will speed up when neighbors
        are found
    source_files : list
        List of strings of source file names
    f : callable
        Function that determines if two tiles are neighbors based on their
        coordinates. f(c1, c2) returns True if tiles are neighbors
    sides : list
        List of 2 strings that give the "side" where tiles are neighbors.

    Returns
    -------
    neighbors : dict
        Dictionary of neighbors

    Notes
    -------
    For example, if Tile1 is to the left of Tile2, then
    neighbors['Tile1']['right'] = 'Tile2'
    neighbors['Tile2']['left'] = 'Tile1'
    """
    for i, c1 in enumerate(coords):
        me = source_files[I[i]]
        # If the left neighbor has already been found...
        if neighbors[me][sides[0]] != '':
            continue
        # could try coords[i:] (+ fixes) for speed if it becomes a problem
        for j, c2 in enumerate(coords):
            if f(c1, c2):
                # then tiles are neighbors neighbors
                neigh = source_files[I[j]]
                neighbors[me][sides[0]] = neigh
                neighbors[neigh][sides[1]] = me
                break
    return neighbors


Instruction:
Can you implement a function in Python 3 that
sets the data and done arrays for the neighbors of the given elevation filename.

Response:
def set_neighbor_data(self, elev_fn, dem_proc, interp=None):
        """
        From the elevation filename, we can figure out and load the data and
        done arrays.
        """
        if interp is None:
            interp = self.build_interpolator(dem_proc)
        opp = {'top': 'bottom', 'left': 'right'}
        for key in self.neighbors[elev_fn].keys():
            tile = self.neighbors[elev_fn][key]
            if tile == '':
                continue
            oppkey = key
            for me, neigh in opp.iteritems():
                if me in key:
                    oppkey = oppkey.replace(me, neigh)
                else:
                    oppkey = oppkey.replace(neigh, me)
            opp_edge = self.neighbors[tile][oppkey]
            if opp_edge == '':
                continue

            interp.values = dem_proc.uca[::-1, :]
#            interp.values[:, 0] = np.ravel(dem_proc.uca)  # for other interp.
            # for the top-left tile we have to set the bottom and right edges
            # of that tile, so two edges for those tiles
            for key_ed in oppkey.split('-'):
                self.edges[tile][key_ed].set_data('data', interp)

            interp.values = dem_proc.edge_done[::-1, :].astype(float)
#            interp.values[:, 0] = np.ravel(dem_proc.edge_done)
            for key_ed in oppkey.split('-'):
                self.edges[tile][key_ed].set_data('done', interp)


Instruction:
Write a Python 3 script for
updating the todo of the edge

Response:
def update_edge_todo(self, elev_fn, dem_proc):
        """
        Can figure out how to update the todo based on the elev filename
        """
        for key in self.edges[elev_fn].keys():
            self.edges[elev_fn][key].set_data('todo', data=dem_proc.edge_todo)


Instruction:
Write a Python 3 function for
updating the edges of the node with the given elev_fn and dem_proc.

Response:
def update_edges(self, elev_fn, dem_proc):
        """
        After finishing a calculation, this will update the neighbors and the
        todo for that tile
        """
        interp = self.build_interpolator(dem_proc)
        self.update_edge_todo(elev_fn, dem_proc)
        self.set_neighbor_data(elev_fn, dem_proc, interp)


Instruction:
Explain what the following Python 3 code does
def get_edge_init_data(self, fn, save_path=None):

        edge_init_data = {key: self.edges[fn][key].get('data') for key in
                          self.edges[fn].keys()}
        edge_init_done = {key: self.edges[fn][key].get('done') for key in
                          self.edges[fn].keys()}
        edge_init_todo = {key: self.edges[fn][key].get('todo') for key in
                          self.edges[fn].keys()}
        return edge_init_data, edge_init_done, edge_init_todo

Response: Creates the initialization data from the edge structure


Instruction:
Explain what the following Python 3 code does
def find_best_candidate(self, elev_source_files=None):
        self.fill_percent_done()
        i_b = np.argmax(self.percent_done.values())
        if self.percent_done.values()[i_b] <= 0:
            return None

        # check for ties
        I = np.array(self.percent_done.values()) == \
            self.percent_done.values()[i_b]
        if I.sum() == 1:
            pass  # no ties
        else:
            I2 = np.argmax(np.array(self.max_elev.values())[I])
            i_b = I.nonzero()[0][I2]

            # Make sure the apples are still apples
            assert(np.array(self.max_elev.keys())[I][I2]
                   == np.array(self.percent_done.keys())[I][I2])

        if elev_source_files is not None:
            fn = self.percent_done.keys()[i_b]
            lckfn = _get_lockfile_name(fn)
            if os.path.exists(lckfn):  # another process is working on it
                # Find a different Candidate
                i_alt = np.argsort(self.percent_done.values())[::-1]
                for i in i_alt:
                    fn = self.percent_done.keys()[i]
                    lckfn = _get_lockfile_name(fn)
                    if not os.path.exists(lckfn):
                        break
            # Get and return the index
            i_b = elev_source_files.index(fn)

        return i_b

Response: Find the best candidate for the current thread based on the updated edge information.


Instruction:
Explain what the following Python 3 code does
def process_twi(self, index=None, do_edges=False, skip_uca_twi=False):
        if index is not None:
            elev_source_files = [self.elev_source_files[index]]
        else:
            elev_source_files = self.elev_source_files
        for i, esfile in enumerate(elev_source_files):
            try:
                fn, status = self.calculate_twi(esfile,
                                                save_path=self.save_path,
                                                do_edges=do_edges,
                                                skip_uca_twi=skip_uca_twi)
                if index is None:
                    self.twi_status[i] = status
                else:
                    self.twi_status[index] = status
            except:
                lckfn = _get_lockfile_name(esfile)
                try:
                    os.remove(lckfn)
                except:
                    pass
                traceback.print_exc()
                print traceback.format_exc()
                if index is None:
                    self.twi_status[i] = "Error " + traceback.format_exc()
                else:
                    self.twi_status[index] = "Error " + traceback.format_exc()

Response: Processes the TWI and returns the TWI status of the UCA and TWI.


Instruction:
Can you tell what is the following Python 3 function doing
def calculate_twi(self, esfile, save_path, use_cache=True, do_edges=False,
                      skip_uca_twi=False):
        """
        Calculates twi for supplied elevation file

        Parameters
        -----------
        esfile : str
            Path to elevation file to be processed
        save_path: str
            Root path to location where TWI will be saved. TWI will be saved in
            a subdirectory 'twi'.
        use_cache : bool (optional)
            Default True. If a temporary file exists (from a previous run),
            the cached file will be used. Otherwise, if False, existing files
            will be recomputed
        do_edges : bool (optional)
            See :py:func:`process_twi` for details on this argument.
        skip_uca_twi : bool (optional)
            Skips the calculation of the UCA and TWI (only calculates the
            magnitude and direction)
        """
        if os.path.exists(os.path.join(save_path, 'tile_edge.pkl')) and \
                self.tile_edge is None:
            with open(os.path.join(save_path, 'tile_edge.pkl'), 'r') as fid:
                self.tile_edge = cPickle.load(fid)
        elif self.tile_edge is None:
            self.tile_edge = TileEdgeFile(self.elev_source_files, save_path)
            with open(os.path.join(save_path, 'tile_edge.pkl'), 'wb') as fid:
                cPickle.dump(self.tile_edge, fid)

        status = 'Success'  # optimism
        # Check if file is locked
        lckfn = _get_lockfile_name(esfile)
        coords = parse_fn(esfile)
        fn = get_fn_from_coords(coords, 'twi')
        print '*'*79
        if skip_uca_twi:
            print '*'*10, fn, 'Slope Calculation starting...:', '*'*10
        else:
            print '*'*10, fn, 'TWI Calculation starting...:', '*'*10
        print '*'*79
        if os.path.exists(lckfn):  # another process is working on it
            print fn, 'is locked'
            return fn, "Locked"
        else:  # lock this tile
            fid = file(lckfn, 'w')
            fid.close()

        dem_proc = DEMProcessor(esfile)
        # check if the slope already exists for the file. If yes, we should
        # move on to the next tile without doing anything else
        if skip_uca_twi \
                and os.path.exists(dem_proc.get_full_fn('mag', save_path)
                                   + '.npz') \
                and os.path.exists(dem_proc.get_full_fn('ang', save_path)
                                   + '.npz'):
            print dem_proc.get_full_fn('mag', save_path) + '.npz', 'already exists'
            print dem_proc.get_full_fn('ang', save_path) + '.npz', 'already exists'
            # remove lock file
            os.remove(lckfn)
            return fn, 'Cached: Slope'
        # check if the twi already exists for the file. If not in the edge
        # resolution round, we should move on to the next tile
        if os.path.exists(dem_proc.get_full_fn('twi', save_path)) \
                and (do_edges is False):
            print dem_proc.get_full_fn('twi', save_path), 'already exists'
            # remove lock file
            os.remove(lckfn)
            return fn, 'Cached'

        # only calculate the slopes and direction if they do not exist in cache
        fn_ang = dem_proc.get_full_fn('ang', save_path)
        fn_mag = dem_proc.get_full_fn('mag', save_path)
        if os.path.exists(fn_ang + '.npz') and os.path.exists(fn_mag + '.npz')\
                and not self.overwrite_cache:
            dem_proc.load_direction(fn_ang)
            dem_proc.load_slope(fn_mag)
            dem_proc.find_flats()
        else:
            if os.path.exists(fn_ang + '.npz') and os.path_exists(fn_mag + '.npz')\
                    and self.overwrite_cache:
                os.remove(fn_ang)
                os.remove(fn_mag)
            dem_proc.calc_slopes_directions()
            dem_proc.save_slope(save_path, raw=True)
            dem_proc.save_direction(save_path, raw=True)
        if self._DEBUG:
            dem_proc.save_slope(save_path, as_int=False)
            dem_proc.save_direction(save_path, as_int=False)

        if skip_uca_twi:
            # remove lock file
            os.remove(lckfn)
            return fn, status + ":mag-dir-only"

        fn_uca = dem_proc.get_full_fn('uca', save_path)
        fn_uca_ec = dem_proc.get_full_fn('uca_edge_corrected', save_path)
        fn_twi = dem_proc.get_full_fn('twi', save_path)

        # check if edge structure exists for this tile and initialize
        edge_init_data, edge_init_done, edge_init_todo = \
            self.tile_edge.get_edge_init_data(esfile, save_path)

        # Check if uca data exists (if yes, we are in the
        # edge-resolution round)
        uca_init = None
        if os.path.exists(fn_uca + '.npz'):
            if os.path.exists(fn_uca_ec + '.npz'):
                dem_proc.load_uca(fn_uca_ec)
            else:
                dem_proc.load_uca(fn_uca)
            uca_init = dem_proc.uca

        if do_edges or uca_init is None:
            dem_proc.calc_uca(uca_init=uca_init,
                              edge_init_data=[edge_init_data, edge_init_done,
                                              edge_init_todo])

            if uca_init is None:
                dem_proc.save_uca(save_path, raw=True)
                if self._DEBUG:
                    # Also save a geotiff for debugging
                    dem_proc.save_uca(save_path, as_int=False)
            else:
                if os.path.exists(fn_uca_ec):
                    os.remove(fn_uca_ec)
                dem_proc.save_array(dem_proc.uca, None, 'uca_edge_corrected',
                                    save_path, raw=True)
                if self._DEBUG:
                    dem_proc.save_array(dem_proc.uca, None, 'uca_edge_corrected',
                                        save_path, as_int=False)
            # Saving Edge Data, and updating edges
            self.tile_edge.update_edges(esfile, dem_proc)

        dem_proc.calc_twi()
        if os.path.exists(fn_twi):
            os.remove(fn_twi)
        dem_proc.save_twi(save_path, raw=False)

        # clean up for in case
        gc.collect()

        # remove lock file
        os.remove(lckfn)
        # Save last-used dem_proc for debugging purposes
        if self._DEBUG:
            self.dem_proc = dem_proc
        return fn, status

Response: Calculates the TWI for the supplied elevation file and saves it in the specified path.


Instruction:
Write a Python 3 function that can
process the command for all the elevation sources in the source directory.

Response:
def process_command(self, command, save_name='custom', index=None):
        """
        Processes the hillshading

        Parameters
        -----------
        index : int/slice (optional)
            Default: None - process all tiles in source directory. Otherwise,
            will only process the index/indices of the files as listed in
            self.elev_source_files

        """
        if index is not None:
            elev_source_files = [self.elev_source_files[index]]
        else:
            elev_source_files = self.elev_source_files
        save_root = os.path.join(self.save_path, save_name)
        if not os.path.exists(save_root):
            os.makedirs(save_root)

        for i, esfile in enumerate(elev_source_files):
            try:
                status = 'Success'  # optimism
                # Check if file is locked
                lckfn = _get_lockfile_name(esfile)
                coords = parse_fn(esfile)
                fn = get_fn_from_coords(coords, save_name)
                fn = os.path.join(save_root, fn)
                if os.path.exists(lckfn):  # another process is working on it
                    print fn, 'is locked'
                    status = 'locked'
                elif os.path.exists(fn):
                    print fn, 'already exists'
                    status = 'cached'
                else:  # lock this tile
                    print fn, '... calculating ', save_name
                    fid = file(lckfn, 'w')
                    fid.close()

                    # Calculate the custom process for this tile
                    status = command(esfile, fn)

                    os.remove(lckfn)

                if index is None:
                    self.custom_status[i] = status
                else:
                    self.custom_status[index] = status
            except:
                lckfn = _get_lockfile_name(esfile)
                try:
                    os.remove(lckfn)
                except:
                    pass
                traceback.print_exc()
                print traceback.format_exc()
                if index is None:
                    self.custom_status[i] = "Error " + traceback.format_exc()
                else:
                    self.custom_status[index] = "Error " + traceback.format_exc()


Instruction:
Write a Python 3 function that can
give a list of file paths for elevation files, this function will rename those files to the format required by the pyDEM package. This assumes a .tif extension. Parameters ----------- files : list A list of strings of the paths to the elevation files that will be renamed name : str (optional) Default = None. A suffix to the filename. For example <filename>_suffix.tif Notes ------ The files are renamed in the same directory as the original file locations

Response:
def rename_files(files, name=None):
    """
    Given a list of file paths for elevation files, this function will rename
    those files to the format required by the pyDEM package.

    This assumes a .tif extension.

    Parameters
    -----------
    files : list
        A list of strings of the paths to the elevation files that will be
        renamed
    name : str (optional)
        Default = None. A suffix to the filename. For example
        <filename>_suffix.tif

    Notes
    ------
    The files are renamed in the same directory as the original file locations
    """
    for fil in files:
        elev_file = GdalReader(file_name=fil)
        elev, = elev_file.raster_layers
        fn = get_fn(elev, name)
        del elev_file
        del elev
        fn = os.path.join(os.path.split(fil)[0], fn)
        os.rename(fil, fn)
        print "Renamed", fil, "to", fn


Instruction:
Write a Python 3 function that can
determine the standard filename for a given GeoTIFF Layer.

Response:
def get_fn(elev, name=None):
    """
    Determines the standard filename for a given GeoTIFF Layer.

    Parameters
    -----------
    elev : GdalReader.raster_layer
        A raster layer from the GdalReader object.
    name : str (optional)
        An optional suffix to the filename.
    Returns
    -------
    fn : str
        The standard <filename>_<name>.tif with suffix (if supplied)
    """
    gcs = elev.grid_coordinates
    coords = [gcs.LLC.lat, gcs.LLC.lon, gcs.URC.lat, gcs.URC.lon]
    return get_fn_from_coords(coords, name)


Instruction:
Implement a Python 3 function for
giving a set of coordinates returns the standard filename.

Response:
def get_fn_from_coords(coords, name=None):
    """ Given a set of coordinates, returns the standard filename.

    Parameters
    -----------
    coords : list
        [LLC.lat, LLC.lon, URC.lat, URC.lon]
    name : str (optional)
        An optional suffix to the filename.

    Returns
    -------
    fn : str
        The standard <filename>_<name>.tif with suffix (if supplied)
    """
    NS1 = ["S", "N"][coords[0] > 0]
    EW1 = ["W", "E"][coords[1] > 0]
    NS2 = ["S", "N"][coords[2] > 0]
    EW2 = ["W", "E"][coords[3] > 0]
    new_name = "%s%0.3g%s%0.3g_%s%0.3g%s%0.3g" % \
        (NS1, coords[0], EW1, coords[1], NS2, coords[2], EW2, coords[3])
    if name is not None:
        new_name += '_' + name
    return new_name.replace('.', 'o') + '.tif'


Instruction:
Can you implement a function in Python 3 that
extracts the change in x and y coordinates from the geotiff layer. Presently only supports WGS - 84 files.

Response:
def mk_dx_dy_from_geotif_layer(geotif):
    """
    Extracts the change in x and y coordinates from the geotiff file. Presently
    only supports WGS-84 files.
    """
    ELLIPSOID_MAP = {'WGS84': 'WGS-84'}
    ellipsoid = ELLIPSOID_MAP[geotif.grid_coordinates.wkt]
    d = distance(ellipsoid=ellipsoid)
    dx = geotif.grid_coordinates.x_axis
    dy = geotif.grid_coordinates.y_axis
    dX = np.zeros((dy.shape[0]-1))
    for j in xrange(len(dX)):
        dX[j] = d.measure((dy[j+1], dx[1]), (dy[j+1], dx[0])) * 1000  # km2m
    dY = np.zeros((dy.shape[0]-1))
    for i in xrange(len(dY)):
        dY[i] = d.measure((dy[i], 0), (dy[i+1], 0)) * 1000  # km2m
    return dX, dY


Instruction:
How would you implement a function in Python 3 that
creates a new geotiff file object and return a handle to it and driver

Response:
def mk_geotiff_obj(raster, fn, bands=1, gdal_data_type=gdal.GDT_Float32,
                   lat=[46, 45], lon=[-73, -72]):
    """
    Creates a new geotiff file objects using the WGS84 coordinate system, saves
    it to disk, and returns a handle to the python file object and driver

    Parameters
    ------------
    raster : array
        Numpy array of the raster data to be added to the object
    fn : str
        Name of the geotiff file
    bands : int (optional)
        See :py:func:`gdal.GetDriverByName('Gtiff').Create
    gdal_data : gdal.GDT_<type>
        Gdal data type (see gdal.GDT_...)
    lat : list
        northern lat, southern lat
    lon : list
        [western lon, eastern lon]
    """
    NNi, NNj = raster.shape
    driver = gdal.GetDriverByName('GTiff')
    obj = driver.Create(fn, NNj, NNi, bands, gdal_data_type)
    pixel_height = -np.abs(lat[0] - lat[1]) / (NNi - 1.0)
    pixel_width = np.abs(lon[0] - lon[1]) / (NNj - 1.0)
    obj.SetGeoTransform([lon[0], pixel_width, 0, lat[0], 0, pixel_height])
    srs = osr.SpatialReference()
    srs.SetWellKnownGeogCS('WGS84')
    obj.SetProjection(srs.ExportToWkt())
    obj.GetRasterBand(1).WriteArray(raster)
    return obj, driver


Instruction:
Can you tell what is the following Python 3 function doing
def sortrows(a, i=0, index_out=False, recurse=True):
    I = np.argsort(a[:, i])
    a = a[I, :]
    # We recursively call sortrows to make sure it is sorted best by every
    # column
    if recurse & (len(a[0]) > i + 1):
        for b in np.unique(a[:, i]):
            ids = a[:, i] == b
            colids = range(i) + range(i+1, len(a[0]))
            a[np.ix_(ids, colids)], I2 = sortrows(a[np.ix_(ids, colids)],
                                                  0, True, True)
            I[ids] = I[np.nonzero(ids)[0][I2]]

    if index_out:
        return a, I
    else:
        return a

Response: Sorts array a in descending order by column i.


Instruction:
How would you implement a function in Python 3 that
returns indices of 2d adjacent to those in I. Helper function for get_border.

Response:
def get_adjacent_index(I, shape, size):
    """
    Find indices 2d-adjacent to those in I. Helper function for get_border*.

    Parameters
    ----------
    I : np.ndarray(dtype=int)
        indices in the flattened region
    shape : tuple(int, int)
        region shape
    size : int
        region size (technically computable from shape)

    Returns
    -------
    J : np.ndarray(dtype=int)
        indices orthogonally and diagonally adjacent to I

    """

    m, n = shape
    In = I % n
    bL = In != 0
    bR = In != n-1

    J = np.concatenate([
        # orthonally adjacent
        I - n,
        I[bL] - 1,
        I[bR] + 1,
        I + n,

        # diagonally adjacent
        I[bL] - n-1,
        I[bR] - n+1,
        I[bL] + n-1,
        I[bR] + n+1])

    # remove indices outside the array
    J = J[(J>=0) & (J<size)]

    return J


Instruction:
Implement a Python 3 function for
returning the indices of the border of the region I.

Response:
def get_border_index(I, shape, size):
    """
    Get flattened indices for the border of the region I.

    Parameters
    ----------
    I : np.ndarray(dtype=int)
        indices in the flattened region.
    size : int
        region size (technically computable from shape argument)
    shape : tuple(int, int)
        region shape

    Returns
    -------
    J : np.ndarray(dtype=int)
        indices orthogonally and diagonally bordering I
    """

    J = get_adjacent_index(I, shape, size)

    # instead of setdiff?
    # border = np.zeros(size)
    # border[J] = 1
    # border[I] = 0
    # J, = np.where(border)

    return np.setdiff1d(J, I)


Instruction:
Explain what the following Python 3 code does
def get_border_mask(region):

    # common special case (for efficiency)
    internal = region[1:-1, 1:-1]
    if internal.all() and internal.any():
        return ~region

    I, = np.where(region.ravel())
    J = get_adjacent_index(I, region.shape, region.size)

    border = np.zeros(region.size, dtype='bool')
    border[J] = 1
    border[I] = 0
    border = border.reshape(region.shape)

    return border

Response: Returns a boolean array mask of the border of the region.


Instruction:
How would you explain what the following Python 3 function does
def get_distance(region, src):

    dmax = float(region.size)
    d = np.full(region.shape, dmax)
    d[src] = 0
    for n in range(region.size):
        d_orth = minimum_filter(d, footprint=_ORTH2) + 1
        d_diag = minimum_filter(d, (3, 3)) + _SQRT2
        d_adj = np.minimum(d_orth[region], d_diag[region])
        d[region] = np.minimum(d_adj, d[region])
        if (d[region] < dmax).all():
            break
    return d

Response: Compute within - region distances from the nearest src pixel.


Instruction:
Given the following Python 3 function, write the documentation
def grow_slice(slc, size):

    return slice(max(0, slc.start-1), min(size, slc.stop+1))

Response: Grow a slice object by 1 in each direction without overreaching the list.


Instruction:
Can you write a function in Python 3 where it
checks if a 2d object is on the edge of the array.

Response:
def is_edge(obj, shape):
    """
    Check if a 2d object is on the edge of the array.

    Parameters
    ----------
    obj : tuple(slice, slice)
        Pair of slices (e.g. from scipy.ndimage.measurements.find_objects)
    shape : tuple(int, int)
        Array shape.

    Returns
    -------
    b : boolean
        True if the object touches any edge of the array, else False.
    """

    if obj[0].start == 0: return True
    if obj[1].start == 0: return True
    if obj[0].stop == shape[0]: return True
    if obj[1].stop == shape[1]: return True
    return False


Instruction:
Write a Python 3 script to
find an approximate centroid for a region that is within the region.

Response:
def find_centroid(region):
    """
    Finds an approximate centroid for a region that is within the region.

    Parameters
    ----------
    region : np.ndarray(shape=(m, n), dtype='bool')
        mask of the region.

    Returns
    -------
    i, j : tuple(int, int)
        2d index within the region nearest the center of mass.
    """

    x, y = center_of_mass(region)
    w = np.argwhere(region)
    i, j = w[np.argmin(np.linalg.norm(w - (x, y), axis=1))]
    return i, j


Instruction:
Can you implement a function in Python 3 that
resets the object at its initial state.

Response:
def clear(self):
        """Resets the object at its initial (empty) state."""
        self._deque.clear()
        self._total_length = 0
        self._has_view = False


Instruction:
Implement a Python 3 function for
serializing the write buffer into a single string ( bytes.

Response:
def _tobytes(self):
        """Serializes the write buffer into a single string (bytes).

        Returns:
            a string (bytes) object.
        """
        if not self._has_view:
            # fast path optimization
            if len(self._deque) == 0:
                return b""
            elif len(self._deque) == 1:
                # no copy
                return self._deque[0]
            else:
                return b"".join(self._deque)
        else:
            tmp = [x.tobytes() if isinstance(x, memoryview) else x
                   for x in self._deque]
            return b"".join(tmp)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def pop_chunk(self, chunk_max_size):
        if self._total_length < chunk_max_size:
            # fastpath (the whole queue fit in a single chunk)
            res = self._tobytes()
            self.clear()
            return res
        first_iteration = True
        while True:
            try:
                data = self._deque.popleft()
                data_length = len(data)
                self._total_length -= data_length
                if first_iteration:
                    # first iteration
                    if data_length == chunk_max_size:
                        # we are lucky !
                        return data
                    elif data_length > chunk_max_size:
                        # we have enough data at first iteration
                        # => fast path optimization
                        view = self._get_pointer_or_memoryview(data,
                                                               data_length)
                        self.appendleft(view[chunk_max_size:])
                        return view[:chunk_max_size]
                    else:
                        # no single iteration fast path optimization :-(
                        # let's use a WriteBuffer to build the result chunk
                        chunk_write_buffer = WriteBuffer()
                else:
                    # not first iteration
                    if chunk_write_buffer._total_length + data_length \
                       > chunk_max_size:
                        view = self._get_pointer_or_memoryview(data,
                                                               data_length)
                        limit = chunk_max_size - \
                            chunk_write_buffer._total_length - data_length
                        self.appendleft(view[limit:])
                        data = view[:limit]
                chunk_write_buffer.append(data)
                if chunk_write_buffer._total_length >= chunk_max_size:
                    break
            except IndexError:
                # the buffer is empty (so no memoryview inside)
                self._has_view = False
                break
            first_iteration = False
        return chunk_write_buffer._tobytes()

Response: Pops a chunk of the given max size.


Instruction:
Can you generate the documentation for the following Python 3 function
def with_name(self, name):
        if not self.name:
            raise ValueError("%r has an empty name" % (self,))
        return self._from_parsed_parts(self._drv, self._root,
                                       self._parts[:-1] + [name])

Response: Return a new path with the file name changed.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def with_suffix(self, suffix):
        # XXX if suffix is None, should the current suffix be removed?
        drv, root, parts = self._flavour.parse_parts((suffix,))
        if drv or root or len(parts) != 1:
            raise ValueError("Invalid suffix %r" % (suffix))
        suffix = parts[0]
        if not suffix.startswith('.'):
            raise ValueError("Invalid suffix %r" % (suffix))
        name = self.name
        if not name:
            raise ValueError("%r has an empty name" % (self,))
        old_suffix = self.suffix
        if not old_suffix:
            name = name + suffix
        else:
            name = name[:-len(old_suffix)] + suffix
        return self._from_parsed_parts(self._drv, self._root,
                                       self._parts[:-1] + [name])

Response: Return a new path with the file suffix changed or added.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _raw_open(self, flags, mode=0o777):
        return self._accessor.open(self, flags, mode)

Response: Open the file pointed by this path and return a file descriptor.


Instruction:
Write a Python 3 script for
iterating over the files in this directory. Does not yield any result for the special paths....

Response:
def iterdir(self):
        """Iterate over the files in this directory.  Does not yield any
        result for the special paths '.' and '..'.
        """
        for name in self._accessor.listdir(self):
            if name in ('.', '..'):
                # Yielding a path object for these makes little sense
                continue
            yield self._make_child_relpath(name)


Instruction:
Can you generate the documentation for the following Python 3 function
def absolute(self):
        # XXX untested yet!
        if self.is_absolute():
            return self
        # FIXME this must defer to the specific flavour (and, under Windows,
        # use nt._getfullpathname())
        obj = self._from_parts([os.getcwd()] + self._parts, init=False)
        obj._init(template=self)
        return obj

Response: Return an absolute version of this path.


Instruction:
Make a summary of the following Python 3 code
def resolve(self):
        s = self._flavour.resolve(self)
        if s is None:
            # No symlink resolution => for consistency, raise an error if
            # the path doesn't exist or is forbidden
            self.stat()
            s = str(self.absolute())
        # Now we have no symlinks in the path, it's safe to normalize it.
        normed = self._flavour.pathmod.normpath(s)
        obj = self._from_parts((normed,), init=False)
        obj._init(template=self)
        return obj

Response: Resolve the path and return a new object.


Instruction:
Can you write a function in Python 3 where it
opens the file pointed by this path and return a file object.

Response:
def open(self, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None):
        """
        Open the file pointed by this path and return a file object, as
        the built-in open() function does.
        """
        if sys.version_info >= (3, 3):
            return io.open(str(self), mode, buffering, encoding, errors, newline,
                           opener=self._opener)
        else:
            return io.open(str(self), mode, buffering, encoding, errors, newline)


Instruction:
Can you generate the documentation for the following Python 3 function
def replace(self, target):
        if sys.version_info < (3, 3):
            raise NotImplementedError("replace() is only available "
                                      "with Python 3.3 and later")
        self._accessor.replace(self, target)

Response: Rename this path to the given path clobbering the existing one.


Instruction:
How would you explain what the following Python 3 function does
def symlink_to(self, target, target_is_directory=False):
        self._accessor.symlink(target, self, target_is_directory)

Response: Make this path a symlink pointing to the given path.


Instruction:
Create a Python 3 function to
return True if this path is a symbolic link.

Response:
def is_symlink(self):
        """
        Whether this path is a symbolic link.
        """
        try:
            return S_ISLNK(self.lstat().st_mode)
        except OSError as e:
            if e.errno != ENOENT:
                raise
            # Path doesn't exist
            return False


Instruction:
Given the following Python 3 function, write the documentation
def is_block_device(self):
        try:
            return S_ISBLK(self.stat().st_mode)
        except OSError as e:
            if e.errno != ENOENT:
                raise
            # Path doesn't exist or is a broken symlink
            # (see https://bitbucket.org/pitrou/pathlib/issue/12/)
            return False

Response: Returns True if this path is a block device.