Instruction:
Explain what the following Python 3 code does
def get(self):
        personalization = {}

        for key in ['tos', 'ccs', 'bccs']:
            value = getattr(self, key)
            if value:
                personalization[key[:-1]] = value

        for key in ['subject', 'send_at', 'dynamic_template_data']:
            value = getattr(self, key)
            if value:
                personalization[key] = value

        for prop_name in ['headers', 'substitutions', 'custom_args']:
            prop = getattr(self, prop_name)
            if prop:
                obj = {}
                for key in prop:
                    obj.update(key)
                    personalization[prop_name] = obj

        return personalization

Response: Returns a JSON - ready representation of this Personalization.


Instruction:
Can you create a Python 3 function that
returns a JSON - ready representation of this CustomArg.

Response:
def get(self):
        """
        Get a JSON-ready representation of this CustomArg.

        :returns: This CustomArg, ready for use in a request body.
        :rtype: dict
        """
        custom_arg = {}
        if self.key is not None and self.value is not None:
            custom_arg[self.key] = self.value
        return custom_arg


Instruction:
Create a Python 3 function for
returning a JSON - ready representation of this Section.

Response:
def get(self):
        """
        Get a JSON-ready representation of this Section.

        :returns: This Section, ready for use in a request body.
        :rtype: dict
        """
        section = {}
        if self.key is not None and self.value is not None:
            section[self.key] = self.value
        return section


Instruction:
Can you implement a function in Python 3 that
returns a JSON - ready representation of this ClickTracking.

Response:
def get(self):
        """
        Get a JSON-ready representation of this ClickTracking.

        :returns: This ClickTracking, ready for use in a request body.
        :rtype: dict
        """
        click_tracking = {}
        if self.enable is not None:
            click_tracking["enable"] = self.enable

        if self.enable_text is not None:
            click_tracking["enable_text"] = self.enable_text
        return click_tracking


Instruction:
How would you code a function in Python 3 to
get a JSON - ready representation of this SubscriptionTracking.

Response:
def get(self):
        """
        Get a JSON-ready representation of this SubscriptionTracking.

        :returns: This SubscriptionTracking, ready for use in a request body.
        :rtype: dict
        """
        subscription_tracking = {}
        if self.enable is not None:
            subscription_tracking["enable"] = self.enable

        if self.text is not None:
            subscription_tracking["text"] = self.text.get()

        if self.html is not None:
            subscription_tracking["html"] = self.html.get()

        if self.substitution_tag is not None:
            subscription_tracking["substitution_tag"] = \
                self.substitution_tag.get()
        return subscription_tracking


Instruction:
Create a Python 3 function to
get a JSON - ready representation of this TrackingSettings.

Response:
def get(self):
        """
        Get a JSON-ready representation of this TrackingSettings.

        :returns: This TrackingSettings, ready for use in a request body.
        :rtype: dict
        """
        tracking_settings = {}
        if self.click_tracking is not None:
            tracking_settings["click_tracking"] = self.click_tracking.get()
        if self.open_tracking is not None:
            tracking_settings["open_tracking"] = self.open_tracking.get()
        if self.subscription_tracking is not None:
            tracking_settings[
                "subscription_tracking"] = self.subscription_tracking.get()
        if self.ganalytics is not None:
            tracking_settings["ganalytics"] = self.ganalytics.get()
        return tracking_settings


Instruction:
How would you implement a function in Python 3 that
gets a JSON - ready representation of this OpenTracking.

Response:
def get(self):
        """
        Get a JSON-ready representation of this OpenTracking.

        :returns: This OpenTracking, ready for use in a request body.
        :rtype: dict
        """
        open_tracking = {}
        if self.enable is not None:
            open_tracking["enable"] = self.enable

        if self.substitution_tag is not None:
            open_tracking["substitution_tag"] = self.substitution_tag.get()
        return open_tracking


Instruction:
Given the following Python 3 function, write the documentation
def get(self):
        stats = {}
        if self.start_date is not None:
            stats["start_date"] = self.start_date
        if self.end_date is not None:
            stats["end_date"] = self.end_date
        if self.aggregated_by is not None:
            stats["aggregated_by"] = self.aggregated_by
        if self.sort_by_metric is not None:
            stats["sort_by_metric"] = self.sort_by_metric
        if self.sort_by_direction is not None:
            stats["sort_by_direction"] = self.sort_by_direction
        if self.limit is not None:
            stats["limit"] = self.limit
        if self.offset is not None:
            stats["offset"] = self.offset
        if self.categories is not None:
            stats['categories'] = [category.get() for category in
                                   self.categories]
        return stats

Response: Get the response stats dict from the object.


Instruction:
Make a summary of the following Python 3 code
def file_content(self, value):
        if isinstance(value, FileContent):
            self._file_content = value
        else:
            self._file_content = FileContent(value)

Response: Sets the file_content attribute of the object.


Instruction:
Write a Python 3 script to
set the file type of the content you are attaching.

Response:
def file_type(self, value):
        """The MIME type of the content you are attaching

        :param file_type: The MIME type of the content you are attaching
        :type file_type FileType, string, optional
        """
        if isinstance(value, FileType):
            self._file_type = value
        else:
            self._file_type = FileType(value)


Instruction:
Can you create a Python 3 function that
sets the content - disposition of the attachment.

Response:
def disposition(self, value):
        """The content-disposition of the attachment, specifying display style.

        Specifies how you would like the attachment to be displayed.
         - "inline" results in the attached file being displayed automatically
            within the message.
         - "attachment" results in the attached file requiring some action to
            display (e.g. opening or downloading the file).
        If unspecified, "attachment" is used. Must be one of the two choices.

        :param disposition: The content-disposition of the attachment,
                            specifying display style. Specifies how you would
                            like the attachment to be displayed.
                            - "inline" results in the attached file being
                              displayed automatically within the message.
                            - "attachment" results in the attached file
                              requiring some action to display (e.g. opening
                              or downloading the file).
                            If unspecified, "attachment" is used. Must be one
                            of the two choices.
        :type disposition: Disposition, string, optional
        """
        if isinstance(value, Disposition):
            self._disposition = value
        else:
            self._disposition = Disposition(value)


Instruction:
Implement a function in Python 3 to
set the content id for the attachment.

Response:
def content_id(self, value):
        """The content id for the attachment.

        This is used when the disposition is set to "inline" and the attachment
        is an image, allowing the file to be displayed within the email body.

        :param content_id: The content id for the attachment.
                           This is used when the Disposition is set to "inline"
                           and the attachment is an image, allowing the file to
                           be displayed within the email body.
        :type content_id: ContentId, string, optional
        """
        if isinstance(value, ContentId):
            self._content_id = value
        else:
            self._content_id = ContentId(value)


Instruction:
How would you implement a function in Python 3 that
gets a JSON - ready representation of this Attachment.

Response:
def get(self):
        """
        Get a JSON-ready representation of this Attachment.

        :returns: This Attachment, ready for use in a request body.
        :rtype: dict
        """
        attachment = {}
        if self.file_content is not None:
            attachment["content"] = self.file_content.get()

        if self.file_type is not None:
            attachment["type"] = self.file_type.get()

        if self.file_name is not None:
            attachment["filename"] = self.file_name.get()

        if self.disposition is not None:
            attachment["disposition"] = self.disposition.get()

        if self.content_id is not None:
            attachment["content_id"] = self.content_id.get()
        return attachment


Instruction:
Can you create a Python 3 function that
executes computation of blobs on input eopatch

Response:
def execute(self, eopatch):
        """ Execute computation of blobs on input eopatch

        :param eopatch: Input eopatch
        :type eopatch: eolearn.core.EOPatch
        :return: EOPatch instance with new key holding the blob image.
        :rtype: eolearn.core.EOPatch
        """
        for feature_type, feature_name, new_feature_name in self.feature:
            eopatch[feature_type][new_feature_name] = self._compute_blob(
                eopatch[feature_type][feature_name].astype(np.float64)).astype(np.float32)

        return eopatch


Instruction:
Can you tell what is the following Python 3 function doing
def _check_classifier(classifier):
        predict = getattr(classifier, "predict", None)
        if not callable(predict):
            raise ValueError('Classifier does not have predict method!')

        predict_proba = getattr(classifier, "predict_proba", None)
        if not callable(predict_proba):
            raise ValueError('Classifier does not have predict_proba method!')

Response: Check if the classifier implements predict and predict_proba methods.


Instruction:
Given the following Python 3 function, write the documentation
def _check_image(self, X):

        if (len(X.shape) < 3) or (len(X.shape) > 4):
            raise ValueError('Input has to have shape [n_samples, n_pixels_y, n_pixels_x] '
                             'or [n_samples, n_pixels_y, n_pixels_x, n_bands].')

        self._samples = X.shape[0]
        self._image_size = X.shape[1:3]

        if (self._image_size[0] % self.receptive_field[0]) or (self._image_size[0] % self.receptive_field[0]):
            raise ValueError('Image (%d,%d) and receptive fields (%d,%d) mismatch.\n'
                             'Resize your image to be divisible with receptive field.'
                             % (self._image_size[0], self._image_size[0], self.receptive_field[0],
                                self.receptive_field[1]))

Response: Check the size of the image and check that it is compatible with the classifier s receptive field.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def image_predict(self, X):
        self._check_image(X)

        new_shape = (X.shape[0] * X.shape[1] * X.shape[2],)

        if len(X.shape) == 4:
            new_shape += (X.shape[3],)

        pixels = X.reshape(new_shape)

        predictions = self.classifier.predict(self._transform_input(pixels))

        return predictions.reshape(X.shape[0], X.shape[1], X.shape[2])

Response: Predicts class label for the entire image.


Instruction:
How would you code a function in Python 3 to
predict class probabilities for the entire image.

Response:
def image_predict_proba(self, X):
        """
        Predicts class probabilities for the entire image.

        Parameters:
        -----------

        X: array, shape = [n_samples, n_pixels_x, n_pixels_y, n_bands]
            Array of training images

        y: array, shape = [n_samples] or [n_samples, n_pixels_x, n_pixels_y, n_classes]
            Target probabilities
        """
        self._check_image(X)

        new_shape = (X.shape[0] * X.shape[1] * X.shape[2],)

        if len(X.shape) == 4:
            new_shape += (X.shape[3],)

        pixels = X.reshape(new_shape)

        probabilities = self.classifier.predict_proba(self._transform_input(pixels))

        return probabilities.reshape(X.shape[0], X.shape[1], X.shape[2],
                                     probabilities.shape[1])


Instruction:
How would you explain what the following Python 3 function does
def _to_patches(self, X):

        window = self.receptive_field
        asteps = self.receptive_field

        if len(X.shape) == 4:
            window += (0,)
            asteps += (1,)

        image_view = rolling_window(X, window, asteps)

        new_shape = image_view.shape

        # this makes a copy of the array? can we do without reshaping?
        image_view = image_view.reshape((new_shape[0] * new_shape[1] * new_shape[2],) + new_shape[3:])

        if len(X.shape) == 4:
            image_view = np.moveaxis(image_view, 1, -1)

        return image_view, new_shape

Response: Reshapes input to patches of the size of the classifier s receptive field. For example :


Instruction:
Here you have a function in Python 3, explain what it does
def image_predict(self, X):
        self._check_image(X)

        patches, patches_shape = self._to_patches(X)

        predictions = self.classifier.predict(self._transform_input(patches))

        image_predictions = predictions.reshape(patches_shape[0:3])

        image_results = np.zeros((self._samples,) + self._image_size)

        nx, ny = self.receptive_field
        row_steps = self._image_size[0] // nx
        col_steps = self._image_size[1] // ny

        # how can this be optimised?
        for i, j, k in itertools.product(range(row_steps), range(col_steps), range(self._samples)):
            image_results[k, nx * i:nx * (i + 1), ny * j:ny * (j + 1)] = image_predictions[k, i, j]

        return image_results

Response: Predicts class label for the entire image.


Instruction:
Explain what the following Python 3 code does
def image_predict_proba(self, X):
        self._check_image(X)

        patches, patches_shape = self._to_patches(X)

        probabilities = self.classifier.predict_proba(self._transform_input(patches))

        image_probabilities = probabilities.reshape(patches_shape[0:3] + (probabilities.shape[1],))

        image_results = np.zeros((self._samples,) + self._image_size + (probabilities.shape[1],))

        nx, ny = self.receptive_field
        row_steps = self._image_size[0] // nx
        col_steps = self._image_size[1] // ny

        # how can this be optimised?
        for i, j, k in itertools.product(range(row_steps), range(col_steps), range(self._samples)):
            image_results[k, nx * i:nx * (i + 1), ny * j:ny * (j + 1), :] = image_probabilities[k, i, j, :]

        return image_results

Response: Predicts class probabilities for the entire image.


Instruction:
Can you generate the documentation for the following Python 3 function
def _to_patches(self, X):

        window = self.patch_size
        asteps = self.patch_size

        if len(X.shape) == 4:
            window += (0,)
            asteps += (1,)

        image_view = rolling_window(X, window, asteps)

        new_shape = image_view.shape

        return image_view, new_shape

Response: Reshapes input to patches of the size of the classifier s receptive field.


Instruction:
Make a summary of the following Python 3 code
def image_predict(self, X):
        self._check_image(X)

        if self.mode == 'majority_class':
            predictions = self.pixel_classifier.image_predict(X)

        elif self.mode == 'mean_prob':
            probabilities = self.image_predict_proba(X)
            predictions = (probabilities[..., self.target] > self.target_threshold).astype(np.int)

        patches, _ = self._to_patches(predictions)

        row_steps = self._image_size[0] // self.patch_size[0]
        col_steps = self._image_size[1] // self.patch_size[1]

        # how can this be optimised?
        for i, j, k in itertools.product(range(row_steps), range(col_steps), range(self._samples)):
            patches[k, i, j] = self._target(patches[k, i, j])

        return predictions

Response: Predicts class label for the entire image.


Instruction:
Given the following Python 3 function, write the documentation
def image_predict_proba(self, X):
        self._check_image(X)

        probabilities = self.pixel_classifier.image_predict_proba(X)

        patches, _ = self._to_patches(probabilities)

        row_steps = self._image_size[0] // self.patch_size[0]
        col_steps = self._image_size[1] // self.patch_size[1]

        ps = self.patch_size[0] * self.patch_size[1]

        # how can this be optimised?
        for i, j, k in itertools.product(range(row_steps), range(col_steps), range(self._samples)):
            patches[k, i, j, 0] = np.sum(patches[k, i, j, 0]) / ps
            patches[k, i, j, 1] = np.sum(patches[k, i, j, 1]) / ps

        return probabilities

Response: Predicts class probabilities for the entire image.


Instruction:
Create a Python 3 function for
taking an eopatch and applies the classification mask to the eopatch.

Response:
def execute(self, eopatch):
        """ Transforms [n, w, h, d] eopatch into a [n, w, h, 1] eopatch, adding it the classification mask.

            :param eopatch: An input EOPatch
            :type eopatch: EOPatch
            :return: Outputs EOPatch with n classification masks appended to out_feature_type with out_feature_name key
            :rtype: EOPatch
        """
        in_type, in_name = next(self.input_feature(eopatch))
        out_type, out_name = next(self.input_feature())

        eopatch[out_type][out_name] = self.classifier.image_predict(eopatch[in_type][in_name])

        return eopatch


Instruction:
Create a Python 3 function to
parse the execution arguments provided by user and returns a list of EOWorkflow objects.

Response:
def _parse_execution_args(execution_args):
        """ Parses execution arguments provided by user and raises an error if something is wrong
        """
        if not isinstance(execution_args, (list, tuple)):
            raise ValueError("Parameter 'execution_args' should be a list")

        return [EOWorkflow.parse_input_args(input_args) for input_args in execution_args]


Instruction:
Implement a Python 3 function for
running the executor with n workers.

Response:
def run(self, workers=1):
        """ Runs the executor with n workers.

        :param workers: Number of parallel processes used in the execution. Default is a single process. If set to
            `None` the number of workers will be the number of processors of the system.
        :type workers: int or None
        """
        self.report_folder = self._get_report_folder()
        if self.save_logs and not os.path.isdir(self.report_folder):
            os.mkdir(self.report_folder)

        execution_num = len(self.execution_args)
        log_paths = [self._get_log_filename(idx) if self.save_logs else None
                     for idx in range(execution_num)]

        processing_args = [(self.workflow, init_args, log_path) for init_args, log_path in zip(self.execution_args,
                                                                                log_paths)]

        with concurrent.futures.ProcessPoolExecutor(max_workers=workers) as executor:
            self.execution_stats = list(tqdm(executor.map(self._execute_workflow, processing_args),
                                             total=len(processing_args)))

        self.execution_logs = [None] * execution_num
        if self.save_logs:
            for idx, log_path in enumerate(log_paths):
                with open(log_path) as fin:
                    self.execution_logs[idx] = fin.read()


Instruction:
Can you tell what is the following Python 3 function doing
def _execute_workflow(cls, process_args):
        workflow, input_args, log_path = process_args

        if log_path:
            logger = logging.getLogger()
            logger.setLevel(logging.DEBUG)
            handler = cls._get_log_handler(log_path)
            logger.addHandler(handler)

        stats = {'start_time': dt.datetime.now()}
        try:
            _ = workflow.execute(input_args, monitor=True)
        except BaseException:
            stats['error'] = traceback.format_exc()
        stats['end_time'] = dt.datetime.now()

        if log_path:
            handler.close()
            logger.removeHandler(handler)

        return stats

Response: Handles a single execution of a workflow


Instruction:
Make a summary of the following Python 3 code
def make_report(self):
        if self.execution_stats is None:
            raise RuntimeError('Cannot produce a report without running the executor first, check EOExecutor.run '
                               'method')

        if os.environ.get('DISPLAY', '') == '':
            LOGGER.info('No display found, using non-interactive Agg backend')
            plt.switch_backend('Agg')

        dependency_graph = self._create_dependency_graph()
        task_descriptions = self._get_task_descriptions()

        formatter = HtmlFormatter(linenos=True)
        task_source = self._render_task_source(formatter)
        execution_stats = self._render_execution_errors(formatter)

        template = self._get_template()

        html = template.render(dependency_graph=dependency_graph,
                               task_descriptions=task_descriptions,
                               task_source=task_source,
                               execution_stats=execution_stats,
                               execution_logs=self.execution_logs,
                               code_css=formatter.get_style_defs())

        if not os.path.isdir(self.report_folder):
            os.mkdir(self.report_folder)

        with open(self.get_report_filename(), 'w') as fout:
            fout.write(html)

Response: Makes a html report and saves it into the same folder where logs are stored.


Instruction:
How would you implement a function in Python 3 that
actions to be taken when an event is triggered.

Response:
def onscroll(self, event):
        """Action to be taken when an event is triggered.

        Event is scroll of the mouse's wheel. This leads to changing the temporal frame displayed.

        :param event: Scroll of mouse wheel
        """
        if event.button == 'up':
            self.ind = (self.ind + 1) % self.slices
        else:
            self.ind = (self.ind - 1) % self.slices
        self.update()


Instruction:
Can you generate the documentation for the following Python 3 function
def update(self):
        if self.single_channel:
            self.im.set_data(self.data[self.ind, :, :])
        else:
            self.im.set_data(self.data[self.ind, :, :, :])
        self.ax.set_ylabel('time frame %s' % self.ind)
        self.im.axes.figure.canvas.draw()

Response: Updates image to be displayed with new time frame.


Instruction:
Can you create a Python 3 function that
returns array of data to be displayed and whether it is single channel or not.

Response:
def _get_data_to_display(self, eopatch):
        """Performs checks on dimensionality of data to makes it suitable for display.

        :param eopatch: Input eopatch
        :return: Array to display and whether it is single channel or not
        """
        feature_type, feature_name = next(self.feature(eopatch))

        image_seq = eopatch[feature_type][feature_name]
        single_channel = False
        if image_seq.ndim == 3:
            # single channel
            single_channel = True
        elif image_seq.ndim == 4:
            # If multi-channel make sure indexing is correct
            n_channels = image_seq.shape[-1]
            if n_channels == 1:
                image_seq = np.squeeze(image_seq, axis=-1)
                single_channel = True
            elif n_channels == 3:
                pass
            else:
                if (self.indices is not None) and ((len(self.indices) == 1) or (len(self.indices) == 3)):
                    image_seq = image_seq[..., self.indices]
                    if len(self.indices) == 1:
                        image_seq = np.squeeze(image_seq, axis=-1)
                        single_channel = True
                else:
                    raise ValueError("Specify valid indices for multi-channel EOPatch")
        else:
            raise ValueError("Unsupported format for EOPatch")
        return image_seq, single_channel


Instruction:
Can you implement a function in Python 3 that
shows data and scrolls through. Currently not working from Jupyter notebooks.

Response:
def execute(self, eopatch, title=None, colorbar=False):
        """Shows data and scrolls through. Currently not working from Jupyter notebooks.

        :param eopatch: Image sequence to display. Can be single channel, RGB or multi-channel. If multi-channel,
                        indices must be specified.
        :type eopatch: numpy array
        :param title: String to be displayed as title of the plot
        :type title: string
        :param colorbar: Whether to add colorbar to plot or not. Default is ``False``
        :type colorbar: bool
        :return: Same input eopatch
        """
        image_seq, single_channel = self._get_data_to_display(eopatch)
        # clip to positive values
        if image_seq.dtype is np.float:
            image_seq = np.clip(image_seq, 0, 1)
        elif image_seq.dtype is np.int:
            image_seq = np.clip(image_seq, 0, 255)
        # Call IndexTracker and visualise time frames
        fig, axis = plt.subplots(1, 1)
        tracker = IndexTracker(axis, image_seq,
                               single_channel=single_channel,
                               msg=title,
                               colorbar=colorbar)
        fig.canvas.mpl_connect('scroll_event', tracker.onscroll)
        plt.show()
        return eopatch


Instruction:
Make a summary of the following Python 3 code
def execute(self, eopatch):
        # pylint: disable=invalid-name
        amax_ndvi, amin_ndvi = eopatch.data_timeless[self.argmax_ndvi], eopatch.data_timeless[self.argmin_ndvi]
        amax_ndvi_slope, amin_ndvi_slope = eopatch.data_timeless[self.argmax_ndvi_slope], \
                                           eopatch.data_timeless[self.argmin_ndvi_slope]
        amax_red = eopatch.data_timeless[self.argmax_red]

        stf_idx = [amax_ndvi, amin_ndvi, amax_ndvi_slope, amin_ndvi_slope, amax_red]

        bands = eopatch.data[self.data_feature][..., self.indices]

        _, h, w, _ = bands.shape
        hh, ww = np.ogrid[:h, :w]
        stf = np.concatenate([bands[ii.squeeze(), hh, ww] for ii in stf_idx if ii is not None], axis=-1)

        eopatch.data_timeless[self.feats_feature] = stf

        return eopatch

Response: Execute the spatio - temporal features on the input eopatch.


Instruction:
Can you generate the documentation for the following Python 3 function
def execute(self, eopatch):
        if self.mask_data:
            valid_data_mask = eopatch.mask['VALID_DATA']
        else:
            valid_data_mask = eopatch.mask['IS_DATA']

        if self.data_index is None:
            data = eopatch.data[self.data_feature]
        else:
            data = eopatch.data[self.data_feature][..., self.data_index]

        madata = np.ma.array(data,
                             dtype=np.float32,
                             mask=~valid_data_mask.astype(np.bool))

        argmax_data = np.ma.MaskedArray.argmax(madata, axis=0)
        argmin_data = np.ma.MaskedArray.argmin(madata, axis=0)

        if argmax_data.ndim == 2:
            argmax_data = argmax_data.reshape(argmax_data.shape + (1,))

        if argmin_data.ndim == 2:
            argmin_data = argmin_data.reshape(argmin_data.shape + (1,))

        eopatch.data_timeless[self.amax_feature] = argmax_data
        eopatch.data_timeless[self.amin_feature] = argmin_data

        return eopatch

Response: Compute argmax and argmin of specified data_feature and data_index.


Instruction:
Can you write a function in Python 3 where it
executes the NDVI slope computation on the eopatch.

Response:
def execute(self, eopatch):
        """ Computation of NDVI slope using finite central differences

        This implementation loops through every spatial location, considers the valid NDVI values and approximates their
        first order derivative using central differences. The argument of min and max is added to the eopatch.

        The NDVI slope at date t is comuted as $(NDVI_{t+1}-NDVI_{t-1})/(date_{t+1}-date_{t-1})$.

        :param eopatch: Input eopatch
        :return: eopatch with NDVI slope argmin/argmax features
        """
        # pylint: disable=invalid-name
        if self.mask_data:
            valid_data_mask = eopatch.mask['VALID_DATA']
        else:
            valid_data_mask = eopatch.mask['IS_DATA']

        ndvi = np.ma.array(eopatch.data[self.data_feature],
                           dtype=np.float32,
                           mask=~valid_data_mask.astype(np.bool))

        all_dates = np.asarray([x.toordinal() for x in eopatch.timestamp])

        if ndvi.ndim == 4:
            h, w = ndvi.shape[1: 3]
        else:
            raise ValueError('{} feature has incorrect number of dimensions'.format(self.data_feature))

        argmax_ndvi_slope, argmin_ndvi_slope = np.zeros((h, w, 1), dtype=np.uint8), np.zeros((h, w, 1), dtype=np.uint8)

        for ih, iw in it.product(range(h), range(w)):

            ndvi_curve = ndvi[:, ih, iw, :]
            valid_idx = np.where(~ndvi.mask[:, ih, iw])[0]

            ndvi_curve = ndvi_curve[valid_idx]
            valid_dates = all_dates[valid_idx]

            ndvi_slope = np.convolve(ndvi_curve.squeeze(), [1, 0, -1], 'valid') / np.convolve(valid_dates, [1, 0, -1],
                                                                                'valid')

            # +1 to compensate for the 'valid' convolution which eliminates first and last
            argmax_ndvi_slope[ih, iw] = valid_idx[np.argmax(ndvi_slope) + 1]
            argmin_ndvi_slope[ih, iw] = valid_idx[np.argmin(ndvi_slope) + 1]

            del ndvi_curve, valid_idx, valid_dates, ndvi_slope

        eopatch.data_timeless[self.argmax_feature] = argmax_ndvi_slope
        eopatch.data_timeless[self.argmin_feature] = argmin_ndvi_slope

        return eopatch


Instruction:
Write a Python 3 script for
mapping the binary class data to the scalar raster_value.

Response:
def _map_from_binaries(self, eopatch, dst_shape, request_data):
        """
        Each request represents a binary class which will be mapped to the scalar `raster_value`
        """
        if self.feature_name in eopatch[self.feature_type]:
            raster = eopatch[self.feature_type][self.feature_name].squeeze()
        else:
            raster = np.ones(dst_shape, dtype=self.raster_dtype) * self.no_data_val

        new_raster = self._reproject(eopatch, self._to_binary_mask(request_data))

        # update raster
        raster[new_raster != 0] = new_raster[new_raster != 0]

        return raster


Instruction:
Can you write a function in Python 3 where it
maps from a multiclass to a real raster.

Response:
def _map_from_multiclass(self, eopatch, dst_shape, request_data):
        """
        `raster_value` is a dictionary specifying the intensity values for each class and the corresponding label value.

        A dictionary example for GLC30 LULC mapping is:
        raster_value = {'no_data': (0,[0,0,0,0]),
                        'cultivated land': (1,[193, 243, 249, 255]),
                        'forest': (2,[73, 119, 20, 255]),
                        'grassland': (3,[95, 208, 169, 255]),
                        'schrubland': (4,[112, 179, 62, 255]),
                        'water': (5,[154, 86, 1, 255]),
                        'wetland': (6,[244, 206, 126, 255]),
                        'thundra': (7,[50, 100, 100, 255]),
                        'artificial surface': (8,[20, 47, 147, 255]),
                        'bareland': (9,[202, 202, 202, 255]),
                        'snow and ice': (10,[251, 237, 211, 255])}
        """
        raster = np.ones(dst_shape, dtype=self.raster_dtype) * self.no_data_val

        for key in self.raster_value.keys():
            value, intensities = self.raster_value[key]
            raster[np.mean(np.abs(request_data - intensities), axis=-1) < self.mean_abs_difference] = value

        return self._reproject(eopatch, raster)


Instruction:
Can you tell what is the following Python 3 function doing
def execute(self, eopatch):
        data_arr = eopatch[FeatureType.MASK]['IS_DATA']
        _, height, width, _ = data_arr.shape

        request = self._get_wms_request(eopatch.bbox, width, height)

        request_data, = np.asarray(request.get_data())

        if isinstance(self.raster_value, dict):
            raster = self._map_from_multiclass(eopatch, (height, width), request_data)
        elif isinstance(self.raster_value, (int, float)):
            raster = self._map_from_binaries(eopatch, (height, width), request_data)
        else:
            raise ValueError("Unsupported raster value type")

        if self.feature_type is FeatureType.MASK_TIMELESS and raster.ndim == 2:
            raster = raster[..., np.newaxis]

        eopatch[self.feature_type][self.feature_name] = raster

        return eopatch

Response: Execute the WMS request and add requested feature to this EOPatch.


Instruction:
Can you tell what is the following Python 3 function doing
def _get_parameter(self, name, eopatch):
        if hasattr(self, name) and getattr(self, name) is not None:
            return getattr(self, name)
        if name == 'bbox' and eopatch.bbox:
            return eopatch.bbox
        if name in eopatch.meta_info:
            return eopatch.meta_info[name]
        if name == 'maxcc':
            return 1.0
        if name == 'time_difference':
            return dt.timedelta(seconds=-1)
        if name in ('size_x', 'size_y'):
            return None

        raise ValueError('Parameter {} was neither defined in initialization of {} nor is contained in '
                         'EOPatch'.format(name, self.__class__.__name__))

Response: Returns the value of the named parameter from the initialization parameters or from the EOPatch.


Instruction:
Make a summary of the following Python 3 code
def _prepare_request_data(self, eopatch, bbox, time_interval):

        service_type = ServiceType(self._get_parameter('service_type', eopatch))
        if time_interval is None:
            time_interval = self._get_parameter('time_interval', eopatch)
        if service_type is ServiceType.WMS:
            size_x_name, size_y_name = 'width', 'height'
        else:
            size_x_name, size_y_name = 'resx', 'resy'
        return {
            'layer': self.layer,
            'bbox': bbox if bbox is not None else self._get_parameter('bbox', eopatch),
            'time': time_interval,
            'time_difference': self._get_parameter('time_difference', eopatch),
            'maxcc': self._get_parameter('maxcc', eopatch),
            'image_format': self.image_format,
            'custom_url_params': self.custom_url_params,
            'data_source': self.data_source,
            'instance_id': self.instance_id,
            size_x_name: self._get_parameter('size_x', eopatch),
            size_y_name: self._get_parameter('size_y', eopatch)
        }, service_type

Response: Prepare the data for a DataRequest.


Instruction:
Given the following Python 3 function, write the documentation
def _add_data(self, eopatch, data):
        valid_mask = data[..., -1]
        data = data[..., :-1]

        if data.ndim == 3:
            data = data.reshape(data.shape + (1,))
        if not self.feature_type.is_time_dependent():
            if data.shape[0] > 1:
                raise ValueError('Cannot save time dependent data to time independent feature')
            data = data.squeeze(axis=0)
        if self.feature_type.is_discrete():
            data = data.astype(np.int32)

        eopatch[self.feature_type][self.feature_name] = data

        mask_feature_type, mask_feature_name = next(self.valid_data_mask_feature())

        max_value = self.image_format.get_expected_max_value()
        valid_data = (valid_mask == max_value).astype(np.bool).reshape(valid_mask.shape + (1,))

        if mask_feature_name not in eopatch[mask_feature_type]:
            eopatch[mask_feature_type][mask_feature_name] = valid_data

Response: Adds downloaded data to the EOPatch.


Instruction:
Create a Python 3 function to
add any missing metadata info to the EOPatch.

Response:
def _add_meta_info(self, eopatch, request_params, service_type):
        """ Adds any missing metadata info to EOPatch """

        for param, eoparam in zip(['time', 'time_difference', 'maxcc'], ['time_interval', 'time_difference', 'maxcc']):
            if eoparam not in eopatch.meta_info:
                eopatch.meta_info[eoparam] = request_params[param]

        if 'service_type' not in eopatch.meta_info:
            eopatch.meta_info['service_type'] = service_type.value

        for param in ['size_x', 'size_y']:
            if param not in eopatch.meta_info:
                eopatch.meta_info[param] = getattr(self, param)

        if eopatch.bbox is None:
            eopatch.bbox = request_params['bbox']


Instruction:
Explain what the following Python 3 code does
def execute(self, eopatch=None, bbox=None, time_interval=None):
        if eopatch is None:
            eopatch = EOPatch()

        request_params, service_type = self._prepare_request_data(eopatch, bbox, time_interval)
        request = {ServiceType.WMS: WmsRequest,
                   ServiceType.WCS: WcsRequest}[service_type](**request_params)

        request_dates = request.get_dates()

        if not eopatch.timestamp:
            eopatch.timestamp = request_dates

        download_frames = None
        if self.feature_type.is_time_dependent():
            download_frames = get_common_timestamps(request_dates, eopatch.timestamp)

        images = request.get_data(raise_download_errors=self.raise_download_errors, data_filter=download_frames)

        if not self.raise_download_errors:
            bad_data = [idx for idx, value in enumerate(images) if value is None]
            for idx in reversed(bad_data):
                LOGGER.warning('Data from %s could not be downloaded for %s!', str(request_dates[idx]), self.layer)
                del images[idx]
                del request_dates[idx]

            for removed_frame in eopatch.consolidate_timestamps(request_dates):
                LOGGER.warning('Removed data for frame %s from EOPatch '
                               'due to unavailability of %s!', str(removed_frame), self.layer)

        self._add_data(eopatch, np.asarray(images))
        self._add_meta_info(eopatch, request_params, service_type)
        return eopatch

Response: Executes the request and returns the result of the request.


Instruction:
Here you have a function in Python 3, explain what it does
def get_s2_pixel_cloud_detector(threshold=0.4, average_over=4, dilation_size=2, all_bands=True):
    return S2PixelCloudDetector(threshold=threshold,
                                average_over=average_over,
                                dilation_size=dilation_size,
                                all_bands=all_bands)

Response: Wrapper function for pixel - based S2 cloud detector


Instruction:
How would you explain what the following Python 3 function does
def _get_rescale_factors(self, reference_shape, meta_info):
        # Figure out resampling size
        height, width = reference_shape

        service_type = ServiceType(meta_info['service_type'])
        rescale = None
        if service_type == ServiceType.WMS:

            if (self.cm_size_x is None) and (self.cm_size_y is not None):
                rescale = (self.cm_size_y / height, self.cm_size_y / height)
            elif (self.cm_size_x is not None) and (self.cm_size_y is None):
                rescale = (self.cm_size_x / width, self.cm_size_x / width)
            else:
                rescale = (self.cm_size_y / height, self.cm_size_x / width)

        elif service_type == ServiceType.WCS:
            # Case where only one resolution for cloud masks is specified in WCS
            if self.cm_size_y is None:
                self.cm_size_y = self.cm_size_x
            elif self.cm_size_x is None:
                self.cm_size_x = self.cm_size_y

            hr_res_x, hr_res_y = int(meta_info['size_x'].strip('m')), int(meta_info['size_y'].strip('m'))
            lr_res_x, lr_res_y = int(self.cm_size_x.strip('m')), int(self.cm_size_y.strip('m'))
            rescale = (hr_res_y / lr_res_y, hr_res_x / lr_res_x)

        return rescale

Response: Compute the resampling factors for the array of high - resolution cloud masks and the size of the cloud masks.


Instruction:
Here you have a function in Python 3, explain what it does
def _upsampling(lr_array, rescale, reference_shape, interp='linear'):
        hr_shape = reference_shape + (1,)
        lr_shape = lr_array.shape + (1,)

        if rescale is None:
            return lr_array.reshape(lr_shape)

        out_array = scipy.ndimage.interpolation.zoom(lr_array.reshape(lr_shape),
                                                     (1.0,) + tuple(1 / x for x in rescale) + (1.0,),
                                                     output=lr_array.dtype, order=INTERP_METHODS.index(interp),
                                                     mode='nearest')

        # Padding and cropping might be needed to get to the reference shape
        out_shape = out_array.shape
        padding = tuple((0, np.max((h-o, 0))) for h, o in zip(hr_shape, out_shape))
        hr_array = np.pad(out_array, padding, 'edge')
        hr_array = hr_array[:, :hr_shape[1], :hr_shape[2], :]

        return hr_array

Response: Upsample the low - resolution array to the original high - resolution grid.


Instruction:
Create a Python 3 function to
make OGC request to create input for cloud detector classifier.

Response:
def _make_request(self, bbox, meta_info, timestamps):
        """ Make OGC request to create input for cloud detector classifier

        :param bbox: Bounding box
        :param meta_info: Meta-info dictionary of input eopatch
        :return: Requested data
        """
        service_type = ServiceType(meta_info['service_type'])

        # Raise error if resolutions are not specified
        if self.cm_size_x is None and self.cm_size_y is None:
            raise ValueError("Specify size_x and size_y for data request")

        # If WCS request, make sure both resolutions are set
        if service_type == ServiceType.WCS:
            if self.cm_size_y is None:
                self.cm_size_y = self.cm_size_x
            elif self.cm_size_x is None:
                self.cm_size_x = self.cm_size_y

        custom_url_params = {CustomUrlParam.SHOWLOGO: False,
                             CustomUrlParam.TRANSPARENT: False,
                             CustomUrlParam.EVALSCRIPT: self.model_evalscript}

        request = {ServiceType.WMS: self._get_wms_request,
                   ServiceType.WCS: self._get_wcs_request}[service_type](bbox,
                                                                         meta_info['time_interval'],
                                                                         self.cm_size_x,
                                                                         self.cm_size_y,
                                                                         meta_info['maxcc'],
                                                                         meta_info['time_difference'],
                                                                         custom_url_params)

        request_dates = request.get_dates()
        download_frames = get_common_timestamps(request_dates, timestamps)

        request_return = request.get_data(raise_download_errors=False, data_filter=download_frames)
        bad_data = [idx for idx, value in enumerate(request_return) if value is None]
        for idx in reversed(sorted(bad_data)):
            LOGGER.warning('Data from %s could not be downloaded for %s!', str(request_dates[idx]), self.data_feature)
            del request_return[idx]
            del request_dates[idx]

        return np.asarray(request_return), request_dates


Instruction:
Here you have a function in Python 3, explain what it does
def execute(self, eopatch):
        # Downsample or make request
        if not eopatch.data:
            raise ValueError('EOPatch must contain some data feature')
        if self.data_feature in eopatch.data:
            new_data, rescale = self._downscaling(eopatch.data[self.data_feature], eopatch.meta_info)
            reference_shape = eopatch.data[self.data_feature].shape[:3]
        else:
            new_data, new_dates = self._make_request(eopatch.bbox, eopatch.meta_info, eopatch.timestamp)
            removed_frames = eopatch.consolidate_timestamps(new_dates)
            for rm_frame in removed_frames:
                LOGGER.warning('Removed data for frame %s from '
                               'eopatch due to unavailability of %s!', str(rm_frame), self.data_feature)

            # Get reference shape from first item in data dictionary
            reference_shape = next(iter(eopatch.data.values())).shape[:3]
            rescale = self._get_rescale_factors(reference_shape[1:3], eopatch.meta_info)

        clf_probs_lr = self.classifier.get_cloud_probability_maps(new_data)
        clf_mask_lr = self.classifier.get_mask_from_prob(clf_probs_lr)

        # Add cloud mask as a feature to EOPatch
        clf_mask_hr = self._upsampling(clf_mask_lr, rescale, reference_shape, interp='nearest')
        eopatch.mask[self.cm_feature] = clf_mask_hr.astype(np.bool)

        # If the feature name for cloud probability maps is specified, add as feature
        if self.cprobs_feature is not None:
            clf_probs_hr = self._upsampling(clf_probs_lr, rescale, reference_shape, interp='linear')
            eopatch.data[self.cprobs_feature] = clf_probs_hr.astype(np.float32)

        return eopatch

Response: Execute the cloud binary mask and cloud probability map transformation on the input eopatch.


Instruction:
Write a Python 3 function that can
execute the given eopatches and returns the result.

Response:
def _execute_handling(self, *eopatches, **kwargs):
        """ Handles measuring execution time and error propagation
        """
        self.private_task_config.start_time = datetime.datetime.now()

        caught_exception = None
        try:
            return_value = self.execute(*eopatches, **kwargs)
        except BaseException as exception:
            caught_exception = exception, sys.exc_info()[2]

        if caught_exception is not None:  # Exception is not raised in except statement to prevent duplicated traceback
            exception, traceback = caught_exception
            raise type(exception)('During execution of task {}: {}'.format(self.__class__.__name__,
                                                                           exception)).with_traceback(traceback)

        self.private_task_config.end_time = datetime.datetime.now()
        return return_value


Instruction:
Create a Python 3 function for
parsing a list of features.

Response:
def _parse_features(features, new_names=False, rename_function=None, default_feature_type=None,
                        allowed_feature_types=None):
        """ See eolearn.core.utilities.FeatureParser class.
        """
        return FeatureParser(features, new_names=new_names, rename_function=rename_function,
                             default_feature_type=default_feature_type, allowed_feature_types=allowed_feature_types)


Instruction:
Create a Python 3 function for
functioning which will process README. md file and divide it into INTRO. md and INSTALL. md which will be used in documentation

Response:
def process_readme():
    """ Function which will process README.md file and divide it into INTRO.md and INSTALL.md, which will be used in
    documentation
    """
    with open('../../README.md', 'r') as file:
        readme = file.read()

    readme = readme.replace('# eo-learn', '# Introduction').replace('docs/source/', '')
    readme = readme.replace('**`', '**').replace('`**', '**')

    chapters = [[]]
    for line in readme.split('\n'):
        if line.strip().startswith('## '):
            chapters.append([])
        if line.startswith('<img'):
            line = '

'

        chapters[-1].append(line)

    chapters = ['\n'.join(chapter) for chapter in chapters]

    intro = '\n'.join([chapter for chapter in chapters if not (chapter.startswith('## Install') or
                                                               chapter.startswith('## Documentation'))])
    install = '\n'.join([chapter for chapter in chapters if chapter.startswith('## Install')])

    with open(os.path.join(MARKDOWNS_FOLDER, 'INTRO.md'), 'w') as file:
        file.write(intro)
    with open(os.path.join(MARKDOWNS_FOLDER, 'INSTALL.md'), 'w') as file:
        file.write(install)


Instruction:
Can you tell what is the following Python 3 function doing
def get_common_timestamps(source, target):
    remove_from_source = set(source).difference(target)
    remove_from_source_idxs = [source.index(rm_date) for rm_date in remove_from_source]
    return [idx for idx, _ in enumerate(source) if idx not in remove_from_source_idxs]

Response: Return indices of timestamps from source that are also found in target.


Instruction:
Write a Python 3 function for
comparing two lists or tuples using the deep_eq method.

Response:
def deep_eq(fst_obj, snd_obj):
    """Compares whether fst_obj and snd_obj are deeply equal.

    In case when both fst_obj and snd_obj are of type np.ndarray or either np.memmap, they are compared using
    np.array_equal(fst_obj, snd_obj). Otherwise, when they are lists or tuples, they are compared for length and then
    deep_eq is applied component-wise. When they are dict, they are compared for key set equality, and then deep_eq is
    applied value-wise. For all other data types that are not list, tuple, dict, or np.ndarray, the method falls back
    to the __eq__ method.

    Because np.ndarray is not a hashable object, it is impossible to form a set of numpy arrays, hence deep_eq works
    correctly.

    :param fst_obj: First object compared
    :param snd_obj: Second object compared
    :return: `True` if objects are deeply equal, `False` otherwise
    """
    # pylint: disable=too-many-return-statements
    if isinstance(fst_obj, (np.ndarray, np.memmap)):
        if not isinstance(snd_obj, (np.ndarray, np.memmap)):
            return False

        if fst_obj.dtype != snd_obj.dtype:
            return False

        fst_nan_mask = np.isnan(fst_obj)
        snd_nan_mask = np.isnan(snd_obj)

        return np.array_equal(fst_obj[~fst_nan_mask], snd_obj[~snd_nan_mask]) and \
            np.array_equal(fst_nan_mask, snd_nan_mask)

    if not isinstance(fst_obj, type(snd_obj)):
        return False

    if isinstance(fst_obj, np.ndarray):
        if fst_obj.dtype != snd_obj.dtype:
            return False
        fst_nan_mask = np.isnan(fst_obj)
        snd_nan_mask = np.isnan(snd_obj)
        return np.array_equal(fst_obj[~fst_nan_mask], snd_obj[~snd_nan_mask]) and \
            np.array_equal(fst_nan_mask, snd_nan_mask)

    if isinstance(fst_obj, (list, tuple)):
        if len(fst_obj) != len(snd_obj):
            return False

        for element_fst, element_snd in zip(fst_obj, snd_obj):
            if not deep_eq(element_fst, element_snd):
                return False
        return True

    if isinstance(fst_obj, dict):
        if fst_obj.keys() != snd_obj.keys():
            return False

        for key in fst_obj:
            if not deep_eq(fst_obj[key], snd_obj[key]):
                return False
        return True

    return fst_obj == snd_obj


Instruction:
Make a summary of the following Python 3 code
def negate_mask(mask):
    res = np.ones(mask.shape, dtype=np.int8)
    res[mask > 0] = 0

    return res

Response: Returns the negated mask.


Instruction:
How would you implement a function in Python 3 that
functions pads an image of shape (rows, columns, channels) with zeros. It pads an image so that the shape becomes (rows + padded_rows, columns + padded_columns, channels), where padded_rows = (int(rows/multiple_of[0]) + 1) * multiple_of[0] - rows Same rule is applied to columns. :type X: array of shape (rows, columns, channels) or (rows, columns) :param multiple_of: make X' rows and columns multiple of this tuple :type multiple_of: tuple (rows, columns) :param up_down_rule: Add padded rows evenly to the top/bottom of the image, or up (top) / down (bottom) only :type up_down_rule: up_down_rule: string, (even, up, down) :param up_down_rule: Add padded columns evenly to the left/right of the image, or left / right only :type up_down_rule: up_down_rule: string, (even, left, right) :param pad_value: Value to be assigned to padded rows and columns :type pad_value: int

Response:
def constant_pad(X, multiple_of, up_down_rule='even', left_right_rule='even', pad_value=0):
    """Function pads an image of shape (rows, columns, channels) with zeros.

    It pads an image so that the shape becomes (rows + padded_rows, columns + padded_columns, channels), where
    padded_rows = (int(rows/multiple_of[0]) + 1) * multiple_of[0] - rows

    Same rule is applied to columns.

    :type X: array of shape (rows, columns, channels) or (rows, columns)
    :param multiple_of: make X' rows and columns multiple of this tuple
    :type multiple_of: tuple (rows, columns)
    :param up_down_rule: Add padded rows evenly to the top/bottom of the image, or up (top) / down (bottom) only
    :type up_down_rule: up_down_rule: string, (even, up, down)
    :param up_down_rule: Add padded columns evenly to the left/right of the image, or left / right only
    :type up_down_rule: up_down_rule: string, (even, left, right)
    :param pad_value: Value to be assigned to padded rows and columns
    :type pad_value: int
    """
    # pylint: disable=invalid-name
    shape = X.shape

    row_padding, col_padding = 0, 0

    if shape[0] % multiple_of[0]:
        row_padding = (int(shape[0] / multiple_of[0]) + 1) * multiple_of[0] - shape[0]

    if shape[1] % multiple_of[1]:
        col_padding = (int(shape[1] / multiple_of[1]) + 1) * multiple_of[1] - shape[1]

    row_padding_up, row_padding_down, col_padding_left, col_padding_right = 0, 0, 0, 0

    if row_padding > 0:
        if up_down_rule == 'up':
            row_padding_up = row_padding
        elif up_down_rule == 'down':
            row_padding_down = row_padding
        elif up_down_rule == 'even':
            row_padding_up = int(row_padding / 2)
            row_padding_down = row_padding_up + (row_padding % 2)
        else:
            raise ValueError('Padding rule for rows not supported. Choose beteen even, down or up!')

    if col_padding > 0:
        if left_right_rule == 'left':
            col_padding_left = col_padding
        elif left_right_rule == 'right':
            col_padding_right = col_padding
        elif left_right_rule == 'even':
            col_padding_left = int(col_padding / 2)
            col_padding_right = col_padding_left + (col_padding % 2)
        else:
            raise ValueError('Padding rule for columns not supported. Choose beteen even, left or right!')

    return np.lib.pad(X, ((row_padding_up, row_padding_down), (col_padding_left, col_padding_right)),
                      'constant', constant_values=((pad_value, pad_value), (pad_value, pad_value)))


Instruction:
Create a Python 3 function for
taking a collection of features structured in a few ways and parses them into one way.

Response:
def _parse_features(features, new_names):
        """Takes a collection of features structured in a various ways and parses them into one way.

        If input format is not recognized it raises an error.

        :return: A collection of features
        :rtype: collections.OrderedDict(FeatureType: collections.OrderedDict(str: str or Ellipsis) or Ellipsis)
        :raises: ValueError
        """
        if isinstance(features, dict):
            return FeatureParser._parse_dict(features, new_names)

        if isinstance(features, list):
            return FeatureParser._parse_list(features, new_names)

        if isinstance(features, tuple):
            return FeatureParser._parse_tuple(features, new_names)

        if features is ...:
            return OrderedDict([(feature_type, ...) for feature_type in FeatureType])

        if isinstance(features, FeatureType):
            return OrderedDict([(features, ...)])

        if isinstance(features, str):
            return OrderedDict([(None, OrderedDict([(features, ...)]))])

        raise ValueError('Unknown format of input features: {}'.format(features))


Instruction:
Can you generate the documentation for the following Python 3 function
def _parse_dict(features, new_names):
        feature_collection = OrderedDict()
        for feature_type, feature_names in features.items():
            try:
                feature_type = FeatureType(feature_type)
            except ValueError:
                ValueError('Failed to parse {}, keys of the dictionary have to be instances '
                           'of {}'.format(features, FeatureType.__name__))

            feature_collection[feature_type] = feature_collection.get(feature_type, OrderedDict())

            if feature_names is ...:
                feature_collection[feature_type] = ...

            if feature_type.has_dict() and feature_collection[feature_type] is not ...:
                feature_collection[feature_type].update(FeatureParser._parse_feature_names(feature_names, new_names))

        return feature_collection

Response: Helping function of _parse_features that parses a list of features.


Instruction:
Implement a function in Python 3 to
help function of _parse_features that parses a list of features.

Response:
def _parse_list(features, new_names):
        """Helping function of `_parse_features` that parses a list."""
        feature_collection = OrderedDict()
        for feature in features:
            if isinstance(feature, FeatureType):
                feature_collection[feature] = ...

            elif isinstance(feature, (tuple, list)):
                for feature_type, feature_dict in FeatureParser._parse_tuple(feature, new_names).items():
                    feature_collection[feature_type] = feature_collection.get(feature_type, OrderedDict())

                    if feature_dict is ...:
                        feature_collection[feature_type] = ...

                    if feature_collection[feature_type] is not ...:
                        feature_collection[feature_type].update(feature_dict)
            else:
                raise ValueError('Failed to parse {}, expected a tuple'.format(feature))
        return feature_collection


Instruction:
Make a summary of the following Python 3 code
def _parse_tuple(features, new_names):
        name_idx = 1
        try:
            feature_type = FeatureType(features[0])
        except ValueError:
            feature_type = None
            name_idx = 0

        if feature_type and not feature_type.has_dict():
            return OrderedDict([(feature_type, ...)])
        return OrderedDict([(feature_type, FeatureParser._parse_names_tuple(features[name_idx:], new_names))])

Response: Helping function of _parse_features that parses a tuple.


Instruction:
Write a Python 3 script to
help function of _parse_features that parses a collection of feature names.

Response:
def _parse_feature_names(feature_names, new_names):
        """Helping function of `_parse_features` that parses a collection of feature names."""
        if isinstance(feature_names, set):
            return FeatureParser._parse_names_set(feature_names)

        if isinstance(feature_names, dict):
            return FeatureParser._parse_names_dict(feature_names)

        if isinstance(feature_names, (tuple, list)):
            return FeatureParser._parse_names_tuple(feature_names, new_names)

        raise ValueError('Failed to parse {}, expected dictionary, set or tuple'.format(feature_names))


Instruction:
Can you implement a function in Python 3 that
helps function of _parse_feature_names that parses a set of feature names.

Response:
def _parse_names_set(feature_names):
        """Helping function of `_parse_feature_names` that parses a set of feature names."""
        feature_collection = OrderedDict()
        for feature_name in feature_names:
            if isinstance(feature_name, str):
                feature_collection[feature_name] = ...
            else:
                raise ValueError('Failed to parse {}, expected string'.format(feature_name))
        return feature_collection


Instruction:
Can you write a function in Python 3 where it
helps function of _parse_feature_names that parses a dictionary of feature names.

Response:
def _parse_names_dict(feature_names):
        """Helping function of `_parse_feature_names` that parses a dictionary of feature names."""
        feature_collection = OrderedDict()
        for feature_name, new_feature_name in feature_names.items():
            if isinstance(feature_name, str) and (isinstance(new_feature_name, str) or
                                                  new_feature_name is ...):
                feature_collection[feature_name] = new_feature_name
            else:
                if not isinstance(feature_name, str):
                    raise ValueError('Failed to parse {}, expected string'.format(feature_name))
                else:
                    raise ValueError('Failed to parse {}, expected string or Ellipsis'.format(new_feature_name))
        return feature_collection


Instruction:
Here you have a function in Python 3, explain what it does
def _parse_names_tuple(feature_names, new_names):
        for feature in feature_names:
            if not isinstance(feature, str) and feature is not ...:
                raise ValueError('Failed to parse {}, expected a string'.format(feature))

        if feature_names[0] is ...:
            return ...

        if new_names:
            if len(feature_names) == 1:
                return OrderedDict([(feature_names[0], ...)])
            if len(feature_names) == 2:
                return OrderedDict([(feature_names[0], feature_names[1])])
            raise ValueError("Failed to parse {}, it should contain at most two strings".format(feature_names))

        if ... in feature_names:
            return ...
        return OrderedDict([(feature_name, ...) for feature_name in feature_names])

Response: Helping function of _parse_feature_names that parses a tuple or a list of feature names.


Instruction:
Create a Python 3 function for
checking that the feature types are a subset of the allowed feature types.

Response:
def _check_feature_types(self):
        """ Checks that feature types are a subset of allowed feature types. (`None` is handled

        :raises: ValueError
        """
        if self.default_feature_type is not None and self.default_feature_type not in self.allowed_feature_types:
            raise ValueError('Default feature type parameter must be one of the allowed feature types')

        for feature_type in self.feature_collection:
            if feature_type is not None and feature_type not in self.allowed_feature_types:
                raise ValueError('Feature type has to be one of {}, but {} found'.format(self.allowed_feature_types,
                                                                                feature_type))


Instruction:
Explain what the following Python 3 code does
def _find_feature_type(self, feature_name, eopatch):
        for feature_type in self.allowed_feature_types:
            if feature_type.has_dict() and feature_name in eopatch[feature_type]:
                return feature_type
        return None

Response: Finds the feature type for which the given feature name exists in the given EOPatch. Returns None if no such feature type exists.


Instruction:
Can you generate the documentation for the following Python 3 function
def _return_feature(self, feature_type, feature_name, new_feature_name=...):
        if self.new_names:
            return feature_type, feature_name, (self.rename_function(feature_name) if new_feature_name is ... else
                                                new_feature_name)
        return feature_type, feature_name

Response: Returns the feature type and name for a new feature.


Instruction:
Write a Python 3 script to
parse dependencies and adds names of task_names.

Response:
def _parse_dependencies(dependencies, task_names):
        """Parses dependencies and adds names of task_names.

        :param dependencies: Input of dependency parameter
        :type dependencies: list(tuple or Dependency)
        :param task_names: Human-readable names of tasks
        :type task_names: dict(EOTask: str) or None
        :return: List of dependencies
        :rtype: list(Dependency)
        """
        parsed_dependencies = [dep if isinstance(dep, Dependency) else Dependency(*dep) for dep in dependencies]
        for dep in parsed_dependencies:
            if task_names and dep.task in task_names:
                dep.set_name(task_names[dep.task])
        return parsed_dependencies


Instruction:
Explain what the following Python 3 code does
def _set_task_uuid(self, dependencies):
        uuid_dict = {}
        for dep in dependencies:
            task = dep.task
            if task.private_task_config.uuid in uuid_dict:
                raise ValueError('EOWorkflow cannot execute the same instance of EOTask multiple times')

            task.private_task_config.uuid = self.id_gen.next()
            uuid_dict[task.private_task_config.uuid] = dep

        return uuid_dict

Response: Adds universally unique user ids to each task of the workflow.


Instruction:
Can you write a function in Python 3 where it
creates a directed graph from the given list of Dependency objects

Response:
def create_dag(self, dependencies):
        """Creates a directed graph from dependencies.

        :param dependencies: A list of Dependency objects
        :type dependencies: list(Dependency)
        :return: A directed graph of the workflow
        :rtype: DirectedGraph
        """
        dag = DirectedGraph()
        for dep in dependencies:
            for vertex in dep.inputs:
                task_uuid = vertex.private_task_config.uuid
                if task_uuid not in self.uuid_dict:
                    raise ValueError('Task {}, which is an input of a task {}, is not part of the defined '
                                     'workflow'.format(vertex.__class__.__name__, dep.name))
                dag.add_edge(self.uuid_dict[task_uuid], dep)
            if not dep.inputs:
                dag.add_vertex(dep)
        return dag


Instruction:
Can you tell what is the following Python 3 function doing
def _schedule_dependencies(dag):
        in_degrees = dict(dag.get_indegrees())

        independent_vertices = collections.deque([vertex for vertex in dag if dag.get_indegree(vertex) == 0])
        topological_order = []
        while independent_vertices:
            v_vertex = independent_vertices.popleft()
            topological_order.append(v_vertex)

            for u_vertex in dag[v_vertex]:
                in_degrees[u_vertex] -= 1
                if in_degrees[u_vertex] == 0:
                    independent_vertices.append(u_vertex)

        if len(topological_order) != len(dag):
            raise CyclicDependencyError('Tasks do not form an acyclic graph')

        return topological_order

Response: Returns a list of dependencies that are in the topologically ordered order of the tasks in the dag.


Instruction:
Explain what the following Python 3 code does
def make_linear_workflow(*tasks, **kwargs):
        warnings.warn("Method 'make_linear_workflow' will soon be removed. Use LinearWorkflow class instead",
                      DeprecationWarning, stacklevel=2)

        return LinearWorkflow(*tasks, **kwargs)

Response: Factory method for creating linear workflows.


Instruction:
Can you create a Python 3 function that
executes the workflow. :param input_args: External input arguments to the workflow. They have to be in a form of a dictionary where each key is an EOTask used in the workflow and each value is a dictionary or a tuple of arguments. :type input_args: dict(EOTask: dict(str: object) or tuple(object)) :param monitor: If True workflow execution will be monitored :type monitor: bool :return: An immutable mapping containing results of terminal tasks :rtype: WorkflowResults

Response:
def execute(self, input_args=None, monitor=False):
        """Executes the workflow.

        :param input_args: External input arguments to the workflow. They have to be in a form of a dictionary where
            each key is an EOTask used in the workflow and each value is a dictionary or a tuple of arguments.
        :type input_args: dict(EOTask: dict(str: object) or tuple(object))
        :param monitor: If True workflow execution will be monitored
        :type monitor: bool
        :return: An immutable mapping containing results of terminal tasks
        :rtype: WorkflowResults
        """
        out_degs = dict(self.dag.get_outdegrees())

        input_args = self.parse_input_args(input_args)

        _, intermediate_results = self._execute_tasks(input_args=input_args, out_degs=out_degs, monitor=monitor)

        return WorkflowResults(intermediate_results)


Instruction:
Can you generate the documentation for the following Python 3 function
def parse_input_args(input_args):
        input_args = input_args if input_args else {}
        for task, args in input_args.items():
            if not isinstance(task, EOTask):
                raise ValueError('Invalid input argument {}, should be an instance of EOTask'.format(task))

            if not isinstance(args, (tuple, dict)):
                raise ValueError('Execution input arguments of each task should be a dictionary or a tuple, for task '
                                 '{} got arguments of type {}'.format(task.__class__.__name__, type(args)))

        return input_args

Response: Parses input arguments provided by user and raises an error if something is wrong.


Instruction:
Implement a Python 3 function for
executing all the tasks in the predetermined order.

Response:
def _execute_tasks(self, *, input_args, out_degs, monitor):
        """Executes tasks comprising the workflow in the predetermined order.

        :param input_args: External input arguments to the workflow.
        :type input_args: Dict
        :param out_degs: Dictionary mapping vertices (task IDs) to their out-degrees. (The out-degree equals the number
        of tasks that depend on this task.)
        :type out_degs: Dict
        :return: An immutable mapping containing results of terminal tasks
        :rtype: WorkflowResults
        """
        done_tasks = set()

        intermediate_results = {}

        for dep in self.ordered_dependencies:
            result = self._execute_task(dependency=dep,
                                        input_args=input_args,
                                        intermediate_results=intermediate_results,
                                        monitor=monitor)

            intermediate_results[dep] = result

            self._relax_dependencies(dependency=dep,
                                     out_degrees=out_degs,
                                     intermediate_results=intermediate_results)

        return done_tasks, intermediate_results


Instruction:
Can you tell what is the following Python 3 function doing
def _execute_task(self, *, dependency, input_args, intermediate_results, monitor):
        task = dependency.task
        inputs = tuple(intermediate_results[self.uuid_dict[input_task.private_task_config.uuid]]
                       for input_task in dependency.inputs)

        kw_inputs = input_args.get(task, {})
        if isinstance(kw_inputs, tuple):
            inputs += kw_inputs
            kw_inputs = {}

        LOGGER.debug("Computing %s(*%s, **%s)", str(task), str(inputs), str(kw_inputs))
        return task(*inputs, **kw_inputs, monitor=monitor)

Response: Executes a task of the workflow.


Instruction:
Write a Python 3 script to
relax the dependencies incurred by the task_id.

Response:
def _relax_dependencies(self, *, dependency, out_degrees, intermediate_results):
        """
        Relaxes dependencies incurred by ``task_id``. After the task with ID ``task_id`` has been successfully
        executed, all the tasks it depended on are upadted. If ``task_id`` was the last remaining dependency of a task
        ``t`` then ``t``'s result is removed from memory and, depending on ``remove_intermediate``, from disk.

        :param dependency: A workflow dependecy
        :type dependency: Dependency
        :param out_degrees: Out-degrees of tasks
        :type out_degrees: dict
        :param intermediate_results: The dictionary containing the intermediate results (needed by tasks that have yet
        to be executed) of the already-executed tasks
        :type intermediate_results: dict
        """
        current_task = dependency.task
        for input_task in dependency.inputs:
            dep = self.uuid_dict[input_task.private_task_config.uuid]
            out_degrees[dep] -= 1

            if out_degrees[dep] == 0:
                LOGGER.debug("Removing intermediate result for %s", str(current_task))
                del intermediate_results[dep]


Instruction:
Can you implement a function in Python 3 that
returns an ordered dictionary with key being task_name and an instance of a corresponding task from this workflow.

Response:
def get_tasks(self):
        """Returns an ordered dictionary {task_name: task} of all tasks within this workflow.

        :return: Ordered dictionary with key being task_name (str) and an instance of a corresponding task from this
        workflow
        :rtype: OrderedDict
        """
        tasks = collections.OrderedDict()
        for dep in self.ordered_dependencies:
            tasks[dep.name] = dep.task

        return tasks


Instruction:
Can you implement a function in Python 3 that
generates the DOT representation of the underlying computational graph.

Response:
def get_dot(self):
        """Generates the DOT description of the underlying computational graph.

        :return: The DOT representation of the computational graph
        :rtype: Digraph
        """
        dot = Digraph()

        dep_to_dot_name = self._get_dep_to_dot_name_mapping(self.ordered_dependencies)

        for dep in self.ordered_dependencies:
            for input_task in dep.inputs:
                dot.edge(dep_to_dot_name[self.uuid_dict[input_task.private_task_config.uuid]],
                         dep_to_dot_name[dep])
        return dot


Instruction:
Write a Python 3 script to
return a mapping between Dependency classes and names used in DOT graph.

Response:
def _get_dep_to_dot_name_mapping(dependencies):
        """ Creates mapping between Dependency classes and names used in DOT graph
        """
        dot_name_to_deps = {}
        for dep in dependencies:
            dot_name = dep.name

            if dot_name not in dot_name_to_deps:
                dot_name_to_deps[dot_name] = [dep]
            else:
                dot_name_to_deps[dot_name].append(dep)

        dep_to_dot_name = {}
        for dot_name, deps in dot_name_to_deps.items():
            if len(deps) == 1:
                dep_to_dot_name[deps[0]] = dot_name
                continue

            for idx, dep in enumerate(deps):
                dep_to_dot_name[dep] = dot_name + str(idx)

        return dep_to_dot_name


Instruction:
Write a Python 3 script for
visualizing the computational graph.

Response:
def dependency_graph(self, filename=None):
        """Visualize the computational graph.

        :param filename: Filename of the output image together with file extension. Supported formats: `png`, `jpg`,
            `pdf`, ... . Check `graphviz` Python package for more options
        :type filename: str
        :return: The DOT representation of the computational graph, with some more formatting
        :rtype: Digraph
        """
        dot = self.get_dot()
        dot.attr(rankdir='LR')  # Show graph from left to right

        if filename is not None:
            file_name, file_format = filename.rsplit('.', 1)

            dot.render(filename=file_name, format=file_format, cleanup=True)

        return dot


Instruction:
Create a Python 3 function for
returning a list of tasks that are unique.

Response:
def _make_tasks_unique(tasks):
        """If some tasks of the workflow are the same they are deep copied."""
        unique_tasks = []
        prev_tasks = set()

        for task in tasks:
            if task in prev_tasks:
                task = copy.deepcopy(task)
            unique_tasks.append(task)

        return unique_tasks


Instruction:
How would you implement a function in Python 3 that
returns the most appropriate morphological operation for the given morphological type.

Response:
def get_operation(cls, morph_type):
        """ Maps morphological operation type to function

        :param morph_type: Morphological operation type
        :type morph_type: MorphologicalOperations
        :return: function
        """
        return {
            cls.OPENING: skimage.morphology.opening,
            cls.CLOSING: skimage.morphology.closing,
            cls.DILATION: skimage.morphology.dilation,
            cls.EROSION: skimage.morphology.erosion,
            cls.MEDIAN: skimage.filters.rank.median
        }[morph_type]


Instruction:
Write a Python 3 script for
executing method takes an EOPatch and changes the specified feature and returns the modified EOPatch

Response:
def execute(self, eopatch):
        """ Execute method takes EOPatch and changes the specified feature
        """
        feature_type, feature_name = next(self.feature(eopatch))

        eopatch[feature_type][feature_name] = self.process(eopatch[feature_type][feature_name])

        return eopatch


Instruction:
Can you generate the documentation for the following Python 3 function
def process(self, raster):
        dim = len(raster.shape)
        if dim == 3:
            for dim in range(raster.shape[2]):
                raster[:, :, dim] = self.morph_operation(raster[:, :, dim], self.struct_elem)
        elif dim == 4:
            for time, dim in it.product(range(raster.shape[0]), range(raster.shape[3])):
                raster[time, :, :, dim] = self.morph_operation(raster[time, :, :, dim], self.struct_elem)
        else:
            raise ValueError('Invalid number of dimensions: {}'.format(dim))

        return raster

Response: Applies the morphological operation to the mask object


Instruction:
Can you create a Python 3 function that
returns a dictionary of out - degree

Response:
def get_outdegrees(self):
        """
        :return: dictionary of out-degrees, see get_outdegree
        """
        return {vertex: len(self.adj_dict[vertex]) for vertex in self.adj_dict}


Instruction:
How would you implement a function in Python 3 that
adds the edge u_vertex -> v_vertex to the graph if the edge is not already present. Returns True if the edge was added False otherwise.

Response:
def add_edge(self, u_vertex, v_vertex):
        """Adds the edge ``u_vertex -> v_vertex`` to the graph if the edge is not already present.

        :param u_vertex: Vertex
        :param v_vertex: Vertex
        :return: ``True`` if a new edge was added. ``False`` otherwise.
        """
        self._vertices.add(u_vertex)
        self._vertices.add(v_vertex)
        if not self.is_edge(u_vertex, v_vertex):
            self.indegrees[v_vertex] += 1
            self.adj_dict[u_vertex].append(v_vertex)
            return True

        return False


Instruction:
Explain what the following Python 3 code does
def del_edge(self, u_vertex, v_vertex):
        if self.is_edge(u_vertex, v_vertex):
            self.indegrees[v_vertex] -= 1
            self.adj_dict[u_vertex].remove(v_vertex)
            return True

        return False

Response: Removes the edge u_vertex -> v_vertex from the graph if the edge is present. Returns True if the edge was removed False otherwise.


Instruction:
How would you implement a function in Python 3 that
adds a new vertex to the graph if not already present. Returns True if the vertex was added and False otherwise.

Response:
def add_vertex(self, vertex):
        """Adds a new vertex to the graph if not present.

        :param vertex: Vertex
        :return: ``True`` if ``vertex`` added and not yet present. ``False`` otherwise.
        """
        if vertex not in self._vertices:
            self._vertices.add(vertex)
            return True

        return False


Instruction:
Can you create a Python 3 function that
removes the given vertex and all incident edges from the graph.

Response:
def del_vertex(self, vertex):
        """Removes the vertex ``vertex`` and all incident edges from the graph.

        **Note** that this is an expensive operation that should be avoided!

        Running time is O(V+E)

        :param vertex: Vertex
        :return: ``True`` if ``vertex`` was removed from the graph. ``False`` otherwise.
        """
        for v_vertex in self.adj_dict[vertex]:
            self.indegrees[v_vertex] -= 1

        for v_vertex in self.vertices():
            if vertex in self.adj_dict[v_vertex]:
                self.adj_dict[v_vertex].remove(vertex)

        if vertex in self._vertices:
            self._vertices.remove(vertex)
            return True

        return False


Instruction:
Implement a function in Python 3 to
return a DirectedGraph created from edges.

Response:
def from_edges(edges):
        """ Return DirectedGraph created from edges
        :param edges:
        :return: DirectedGraph
        """
        dag = DirectedGraph()
        for _u, _v in edges:
            dag.add_edge(_u, _v)
        return dag


Instruction:
How would you code a function in Python 3 to
return True if the directed graph dag contains a cycle. False otherwise.

Response:
def _is_cyclic(dag):
        """True if the directed graph dag contains a cycle. False otherwise.

        The algorithm is naive, running in O(V^2) time, and not intended for serious use! For production purposes on
        larger graphs consider implementing Tarjan's O(V+E)-time algorithm instead.

        :type dag: DirectedGraph
        """
        # pylint: disable=invalid-name
        vertices = dag.vertices()
        for w in vertices:
            stack = [w]
            seen = set()
            while stack:
                u = stack.pop()
                seen.add(u)
                for v in dag[u]:
                    if v == w:
                        return True
                    if v not in seen:
                        stack.append(v)
        return False


Instruction:
Explain what the following Python 3 code does
def execute(self, eopatch):
        feature_type, feature_name = next(self.valid_data_feature())
        eopatch[feature_type][feature_name] = self.predicate(eopatch)
        return eopatch

Response: Execute predicate on input eopatch and return the same eopatch instance with a mask. valid_data array computed according to the predicate


Instruction:
How would you implement a function in Python 3 that
executes the masking of the values of the feature and mask_feature.

Response:
def execute(self, eopatch):
        """ Mask values of `feature` according to the `mask_values` in `mask_feature`

        :param eopatch: `eopatch` to be processed
        :return: Same `eopatch` instance with masked `feature`
        """
        feature_type, feature_name, new_feature_name = next(self.feature(eopatch))
        mask_feature_type, mask_feature_name = next(self.mask_feature(eopatch))

        data = np.copy(eopatch[feature_type][feature_name])
        mask = eopatch[mask_feature_type][mask_feature_name]

        if not isinstance(self.mask_values, list):
            raise ValueError('Incorrect format or values of argument `mask_values`')

        for value in self.mask_values:
            data[mask.squeeze() == value] = self.no_data_value

        eopatch.add_feature(feature_type, new_feature_name, data)

        return eopatch


Instruction:
Write a Python 3 script for
returning the number of dimensions of the array.

Response:
def ndim(self):
        """If given FeatureType stores a dictionary of numpy.ndarrays it returns dimensions of such arrays."""
        if self.is_raster():
            return {
                FeatureType.DATA: 4,
                FeatureType.MASK: 4,
                FeatureType.SCALAR: 2,
                FeatureType.LABEL: 2,
                FeatureType.DATA_TIMELESS: 3,
                FeatureType.MASK_TIMELESS: 3,
                FeatureType.SCALAR_TIMELESS: 1,
                FeatureType.LABEL_TIMELESS: 1
            }[self]
        return None


Instruction:
Implement a Python 3 function for
returning the type of the data for the given FeatureType.

Response:
def type(self):
        """Returns type of the data for the given FeatureType."""
        if self is FeatureType.TIMESTAMP:
            return list
        if self is FeatureType.BBOX:
            return BBox
        return dict


Instruction:
Can you generate a brief explanation for the following Python 3 code
def split_by_extensions(filename):
        parts = filename.split('.')
        idx = len(parts) - 1
        while FileFormat.is_file_format(parts[idx]):
            parts[idx] = FileFormat(parts[idx])
            idx -= 1
        return ['.'.join(parts[:idx + 1])] + parts[idx + 1:]

Response: Splits the filename string by the extension of the file


Instruction:
Write a Python 3 script to
reset all counters for this locale.

Response:
def reset_counters(self):
        """
        Resets all counters, truth and classification masks.
        """
        self.truth_masks = None
        self.classification_masks = None
        self.pixel_truth_counts = None
        self.pixel_classification_counts = None
        self.pixel_truth_sum = None
        self.pixel_classification_sum = None
        self.val_df = None
        self.n_validation_sets = 0


Instruction:
Given the following Python 3 function, write the documentation
def _count_truth_pixels(self):
        pixel_count = np.array([[np.nonzero(mask)[0].shape[0] for mask in masktype]
                                for masktype in self.truth_masks])

        pixel_count = np.moveaxis(pixel_count, 0, -1)

        if self.pixel_truth_counts is None:
            self.pixel_truth_counts = np.copy(pixel_count)
        else:
            self.pixel_truth_counts = np.concatenate((self.pixel_truth_counts, pixel_count))

Response: Counts the number of pixels belonging to each truth class.


Instruction:
Can you tell what is the following Python 3 function doing
def _count_classified_pixels(self):
        class_values = self.class_dictionary.values()

        classification_count = np.array([[[np.count_nonzero(prediction[np.nonzero(mask)] == class_val)
                                           for prediction, mask in zip(self.classification_masks, masktype)]
                                          for masktype in self.truth_masks]
                                         for class_val in class_values])

        classification_count = np.moveaxis(classification_count, 0, -1)
        classification_count = np.moveaxis(classification_count, 0, -2)

        if self.pixel_classification_counts is None:
            self.pixel_classification_counts = np.copy(classification_count)
        else:
            self.pixel_classification_counts = np.concatenate((self.pixel_classification_counts, classification_count))

Response: Counts the number of pixels belonging to each classified class.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def add_validation_patch(self, patch):
        # 2. Convert 8-bit mask
        self._transform_truth(patch)

        # 3. Count truth labeled pixels
        self._count_truth_pixels()

        # 5. Perform classification
        self._classify(patch)

        # 6. Count pixel classified as class i
        self._count_classified_pixels()

        self.n_validation_sets = self.n_validation_sets + 1

Response: Adds a validation patch to the validation set.


Instruction:
How would you explain what the following Python 3 function does
def validate(self):
        self.pixel_truth_sum = np.sum(self.pixel_truth_counts, axis=0)
        self.pixel_classification_sum = np.sum(self.pixel_classification_counts, axis=0)

Response: Aggregate the results from all EOPatches.


Instruction:
Can you create a Python 3 function that
saves the current object to a pickle file.

Response:
def save(self, filename):
        """
        Save validator object to pickle.
        """
        with open(filename, 'wb') as output:
            pickle.dump(self, output, protocol=pickle.HIGHEST_PROTOCOL)


Instruction:
Can you implement a function in Python 3 that
returns pandas DataFrame containing pixel counts for all truth classes classified classes and file name of the input ArcGIS EODataSet.

Response:
def pandas_df(self):
        """
        Returns pandas DataFrame containing pixel counts for all truth classes,
        classified classes (for each truth class), and file name of the input
        EODataSet.

        The data frame thus contains

        N =  self.n_validation_sets rows

        and

        M = len(self.truth_classes) + len(self.truth_classes) * len (self.class_dictionary) + 1 columns
        """

        if self.val_df is not None:
            return self.val_df

        clf = self.pixel_classification_counts.reshape(self.pixel_classification_counts.shape[0],
                                                       self.pixel_classification_counts.shape[1] *
                                                       self.pixel_classification_counts.shape[2])

        combo = np.hstack((self.pixel_truth_counts, clf))

        columns = list(itertools.product(self.truth_classes, list(self.class_dictionary.keys())))
        columns = [(item[0] + '_as_' + item[1]).replace(" ", "_") for item in columns]
        truth_columns = ['truth_' + item.replace(" ", "_") for item in self.truth_classes]

        self.val_df = pd.DataFrame(combo, columns=truth_columns + columns)

        return self.val_df


Instruction:
Write a Python 3 script to
return the normalised confusion matrix for the log - level log - level.

Response:
def confusion_matrix(self):
        """
        Returns the normalised confusion matrix
        """
        confusion_matrix = self.pixel_classification_sum.astype(np.float)
        confusion_matrix = np.divide(confusion_matrix.T, self.pixel_truth_sum.T).T

        return confusion_matrix * 100.0


Instruction:
How would you code a function in Python 3 to
plot the confusion matrix.

Response:
def plot_confusion_matrix(self, normalised=True):
        """
        Plots the confusion matrix.
        """
        conf_matrix = self.confusion_matrix()

        if normalised:
            sns.heatmap(conf_matrix,
                        annot=True, annot_kws={"size": 12}, fmt='2.1f', cmap='YlGnBu', vmin=0.0,
                        vmax=100.0,
                        xticklabels=list(self.class_dictionary.keys()),
                        yticklabels=self.truth_classes)
        else:
            sns.heatmap(self.pixel_classification_counts,
                        annot=True, annot_kws={"size": 12}, fmt='2.1f', cmap='YlGnBu', vmin=0.0,
                        vmax=np.max(self.pixel_classification_counts),
                        xticklabels=list(self.class_dictionary.keys()),
                        yticklabels=self.truth_classes)


Instruction:
How would you explain what the following Python 3 function does
def summary(self, scoring):

        if scoring == 'class_confusion':
            print('*' * 50)
            print('  Confusion Matrix ')
            print('x-axis: ' + ' | '.join(list(self.class_dictionary.keys())))
            print('y-axis: ' + ' | '.join(self.truth_classes))
            print(self.confusion_matrix())

Response: Prints out the summary of validation for giving scoring function.


Instruction:
Given the following Python 3 function, write the documentation
def execute(self, eopatch):
        feature_type, feature_name = next(self.feature(eopatch))

        good_idxs = self._get_filtered_indices(eopatch[feature_type][feature_name] if feature_name is not ... else
                                               eopatch[feature_type])

        for feature_type, feature_name in self.filter_features(eopatch):
            if feature_type.is_time_dependent():
                if feature_type.has_dict():
                    if feature_type.contains_ndarrays():
                        eopatch[feature_type][feature_name] = np.asarray([eopatch[feature_type][feature_name][idx] for
                                                                          idx in good_idxs])
                    # else:
                    #     NotImplemented
                else:
                    eopatch[feature_type] = [eopatch[feature_type][idx] for idx in good_idxs]

        self._update_other_data(eopatch)

        return eopatch

Response: Execute the filter on the given eopatch.


Instruction:
Can you create a Python 3 function that
executes computation of HoG features on input eopatch and return the resulting eopatch.

Response:
def execute(self, eopatch):
        """ Execute computation of HoG features on input eopatch

            :param eopatch: Input eopatch
            :type eopatch: eolearn.core.EOPatch
            :return: EOPatch instance with new keys holding the HoG features and HoG image for visualisation.
            :rtype: eolearn.core.EOPatch
        """
        for feature_type, feature_name, new_feature_name in self.feature:
            result = self._compute_hog(eopatch[feature_type][feature_name])
            eopatch[feature_type][new_feature_name] = result[0]
            if self.visualize:
                eopatch[feature_type][self.visualize_name] = result[1]

        return eopatch


Instruction:
Write a Python 3 script to
execute computation of local binary patterns on input eopatch.

Response:
def execute(self, eopatch):
        """ Execute computation of local binary patterns on input eopatch

            :param eopatch: Input eopatch
            :type eopatch: eolearn.core.EOPatch
            :return: EOPatch instance with new key holding the LBP image.
            :rtype: eolearn.core.EOPatch
        """
        for feature_type, feature_name, new_feature_name in self.feature:
            eopatch[feature_type][new_feature_name] = self._compute_lbp(eopatch[feature_type][feature_name])

        return eopatch


Instruction:
Make a summary of the following Python 3 code
def ransac(npts, model, n, k, t, d):
    iterations = 0
    bestfit = None
    besterr = np.inf
    # best_inlier_idxs = None
    while iterations < k:
        maybe_idxs, test_idxs = random_partition(n, npts)
        maybemodel = model.fit(maybe_idxs)
        test_err = model.score(test_idxs, maybemodel)
        also_idxs = test_idxs[test_err < t]  # select indices of rows with accepted points

        LOGGER.debug('test_err.min() %f', test_err.min() if test_err.size else None)
        LOGGER.debug('test_err.max() %f', test_err.max() if test_err.size else None)
        LOGGER.debug('numpy.mean(test_err) %f', np.mean(test_err) if test_err.size else None)
        LOGGER.debug('iteration %d, len(alsoinliers) = %d', iterations, len(also_idxs))

        if len(also_idxs) > d:
            betteridxs = np.concatenate((maybe_idxs, also_idxs))
            bettermodel = model.fit(betteridxs)
            better_errs = model.score(betteridxs, bettermodel)
            thiserr = np.mean(better_errs)
            if thiserr < besterr:
                bestfit = bettermodel
                besterr = thiserr
                # best_inlier_idxs = np.concatenate((maybe_idxs, also_idxs))
        iterations += 1
    return bestfit

Response: Fit model parameters to data using RANSAC algorithm.


Instruction:
Can you implement a function in Python 3 that
returns n random rows of data and also the other lenghts of data

Response:
def random_partition(n, n_data):
    """return n random rows of data (and also the other len(data)-n rows)"""
    all_idxs = np.arange(n_data)
    np.random.shuffle(all_idxs)
    idxs1 = all_idxs[:n]
    idxs2 = all_idxs[n:]
    return idxs1, idxs2


Instruction:
Can you create a Python 3 function that
estimates the rigid transformation matrix given a set of indices

Response:
def estimate_rigid_transformation(self, idx):
        """ Estimate rigid transformation given a set of indices

        :param idx: Array of indices used to estimate the transformation
        :return: Estimated transformation matrix
        """
        # Look at points of given indices only
        src_pts = self.src_pts[idx, :]
        trg_pts = self.trg_pts[idx, :]
        # Get centroid location
        src_centroid, trg_centroid = np.mean(src_pts, axis=0), np.mean(trg_pts, axis=0)
        # Centre sets of points
        src_pts_centr = src_pts - src_centroid
        trg_pts_centr = trg_pts - trg_centroid
        # SVD decomposition
        u, _, v = np.linalg.svd(np.matmul(src_pts_centr.transpose(), trg_pts_centr))
        e = np.eye(2)
        e[1, 1] = np.linalg.det(np.matmul(v.transpose(), u.transpose()))
        # Estimate rotation matrix
        r = np.matmul(v.transpose(), np.matmul(e, u.transpose()))
        # Estimate translation
        t = trg_centroid - np.matmul(r, src_centroid.transpose()).transpose()
        # Fill in transformation matrix and return
        warp_matrix = np.zeros((2, 3))
        warp_matrix[:2, :2] = r
        warp_matrix[:, 2] = t
        return warp_matrix