Instruction:
Here you have a function in Python 3, explain what it does
def date_requester_generator(
        df: pd.DataFrame,
        date_column: str,
        frequency: str,
        date_column_format: str = None,
        format: str = '%Y-%m-%d',
        granularities: Dict[str, str] = None,
        others_format: Dict[str, str] = None,
        times_delta: Dict[str, str] = None
) -> pd.DataFrame:
    """
    From a dataset containing dates in a column, return a dataset
    with at least 3 columns :
    - "DATE" : Label of date
    - "DATETIME" : Date in datetime dtype
    - "GRANULARITY" : Granularity of date

    ---

    ### Parameters

    *mandatory :*
    - `date_column` (*str*): name of column containing the date in the dataframe
    - `frequency` (*str*): see [pandas doc](
    http://pandas.pydata.org/pandas-docs/stable/timeseries.html#offset-aliases)

    *optional :*
    - `date_column_format` (*str*): format of the date in date_column
    - `format` (*str*): format of the date (see [pandas doc](
        https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))
        By default, the format is set to `'%d/%m/%Y'`
        **WARNING**: only use if `granularities` is None.
    - `granularities` (*dict*):
        - key (*str*): name of the granularity
        - value (*str*): Format of the granularity e.g. '%d/%m/%Y' (see [pandas doc](
            https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))
    - `others_format` (*dict*) : Add new columns for each key
        - key (*str*) : name of the column
        - value (*str*): format of the granularity e.g. '%d/%m/%Y' (see [pandas doc](
            https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))
    - `times_delta` (*dict*) : Add new columns for each key
        - key (*str*) : name of the column
        - value (*str*): time delta (e.g. '+1 day', '+3 day', '-4 month')

    ---

    ### Example

    **Input**

    date | kpi
    :---:|:-----:
    2018-01-01 | 1
    2018-01-05 | 2
    2018-01-04 | 3
    2018-01-03 | 4
    2018-01-02 | 5

    ```cson
    date_requester_generator:
      date_column: 'date'
      frequency: 'D'
      granularities:
        'day': '%d/%m/%Y'
        'Semaine': '%W'
      others_format:
        'year': '%Y'
    ```

    **Ouput**

    DATE  |  DATETIME  | GRANULARITY | year
    :---------:|:----------:|:-----------:|:---:
    01/01/2018 | 2018-01-01 |         day | 2018
    02/01/2018 | 2018-01-02 |         day | 2018
    03/01/2018 | 2018-01-03 |         day | 2018
    04/01/2018 | 2018-01-04 |         day | 2018
    05/01/2018 | 2018-01-05 |         day | 2018
    01 | 2018-01-01 |     Semaine | 2018
    """

    start_date = pd.to_datetime(df[date_column], format=date_column_format).min()
    end_date = pd.to_datetime(df[date_column], format=date_column_format).max()

    granularities = granularities or {'date': format}
    others_format = others_format or {}
    times_delta = times_delta or {}

    # Base DataFrame
    columns_list = ['DATE', 'DATETIME', 'GRANULARITY', *others_format, *times_delta]
    result_df = {col_name: [] for col_name in columns_list}

    # Generate the range
    date_range = pd.date_range(start=start_date, end=end_date, freq=frequency)

    for granularity_name, granularity_format in granularities.items():
        date_range_label = date_range.strftime(granularity_format)
        a = list(set(date_range_label))
        index_unique = list(set([a.index(x) for x in date_range_label]))
        date_range_datetime = date_range[index_unique]
        date_range_label = date_range_label.unique()

        result_df['DATE'] += list(date_range_label)
        result_df['DATETIME'] += list(date_range_datetime)
        result_df['GRANULARITY'] += [granularity_name] * len(date_range_label)

        for col_name, other_format in others_format.items():
            result_df[col_name] += list(date_range_datetime.strftime(other_format))

        for col_name, time_delta in times_delta.items():
            result_df[col_name] += list((date_range_datetime + pd.Timedelta(time_delta))
                                        .strftime(granularity_format))

    return pd.DataFrame(result_df)

Response: Generate a date requester for a given date column.


Instruction:
How would you code a function in Python 3 to
normalize symbolic date values in a valid date.

Response:
def _norm_date(datestr: str, date_fmt: str) -> date:
    """normalize symbolic date values (e.g. 'TODAY')

    Convert a symbolic value in a valid date.
    Currenlty known symbolic values are 'TODAY', 'YESTERDAY' and 'TOMORROW'.

    NOTE: This function will return `date` (not `datetime`) instances.

    Parameters:
        `datestr`: the date to parse, formatted as `date_fmt`
        `date_fmt`: expected output date format

    Returns:
        The interpreted date as a datetime.datetime object.
        If `datestr` doesn't match any of the known symbolic names, it just parses it.

    """
    try:
        days = {'TODAY': 0, 'YESTERDAY': -1, 'TOMORROW': 1}[datestr.upper()]
        return date.today() + pd.Timedelta(days=days)
    except KeyError:
        return datetime.strptime(datestr, date_fmt).date()


Instruction:
Explain what the following Python 3 code does
def add_offset(dateobj, hr_offset: str, sign: str):
    sign_coeff = 1 if sign == '+' else -1
    try:
        return dateobj + sign_coeff * pd.Timedelta(hr_offset)
    except ValueError:
        # pd.Timedelta could not parse the offset, let's try harder
        match = TIMEDELTA_RGX.match(hr_offset)
        if match is not None:
            groups = match.groupdict()
            unit = groups['unit'].lower()[0]
            num = sign_coeff * int(groups['num'])
            # is it a week ?
            if unit == 'w':
                return dateobj + num * timedelta(weeks=1)
            # or a month ?
            if unit == 'm':
                return add_months(dateobj, num)
            # or a year ?
            if unit == 'y':
                return add_years(dateobj, num)
        # we did what we could, just re-raise the original exception
        raise

Response: add a human readable offset to dateobj and return corresponding date.


Instruction:
Can you implement a function in Python 3 that
returns a date object with nb_months months added.

Response:
def add_months(dateobj, nb_months: int):
    """return `dateobj` + `nb_months`

    If landing date doesn't exist (e.g. february, 30th), return the last
    day of the landing month.

    >>> add_months(date(2018, 1, 1), 1)
    datetime.date(2018, 1, 1)
    >>> add_months(date(2018, 1, 1), -1)
    datetime.date(2017, 12, 1)
    >>> add_months(date(2018, 1, 1), 25)
    datetime.date(2020, 2, 1)
    >>> add_months(date(2018, 1, 1), -25)
    datetime.date(2015, 12, 1)
    >>> add_months(date(2018, 1, 31), 1)
    datetime.date(2018, 2, 28)
    """
    nb_years, nb_months = divmod(nb_months, 12)
    month = dateobj.month + nb_months
    if month > 12:
        nb_years += 1
        month -= 12
    year = dateobj.year + nb_years
    lastday = monthrange(year, month)[1]
    return dateobj.replace(year=year, month=month, day=min(lastday, dateobj.day))


Instruction:
Create a Python 3 function to
add nb_years to dateobj.

Response:
def add_years(dateobj, nb_years):
    """return `dateobj` + `nb_years`

    If landing date doesn't exist (e.g. february, 30th), return the last
    day of the landing month.

    >>> add_years(date(2018, 1, 1), 1)
    datetime.date(2019, 1, 1)
    >>> add_years(date(2018, 1, 1), -1)
    datetime.date(2017, 1, 1)
    >>> add_years(date(2020, 2, 29), 1)
    datetime.date(2021, 2, 28)
    >>> add_years(date(2020, 2, 29), -1)
    datetime.date(2019, 2, 28)
    """
    year = dateobj.year + nb_years
    lastday = monthrange(year, dateobj.month)[1]
    return dateobj.replace(year=year, day=min(lastday, dateobj.day))


Instruction:
Can you tell what is the following Python 3 function doing
def parse_date(datestr: str, date_fmt: str) -> date:
    rgx = re.compile(r'\((?P<date>.*)\)(\s*(?P<sign>[+-])(?P<offset>.*))?$')
    datestr = datestr.strip()
    match = rgx.match(datestr)
    # if regexp doesn't match, date must match the expected format
    if match is None:
        return _norm_date(datestr, date_fmt)
    datestr = match.group('date').strip()
    dateobj = _norm_date(datestr, date_fmt)
    offset = match.group('offset')
    if offset:
        return add_offset(dateobj, offset, match.group('sign'))
    return dateobj

Response: parse a date string and return corresponding date object.


Instruction:
Make a summary of the following Python 3 code
def filter_by_date(
    df,
    date_col: str,
    date_format: str = '%Y-%m-%d',
    start: str = None,
    stop: str = None,
    atdate: str = None
):
    """
    Filter dataframe your data by date.

    This function will interpret `start`, `stop` and `atdate` and build
    the corresponding date range. The caller must specify either:

    - `atdate`: keep all rows matching this date exactly,
    - `start`: keep all rows matching this date onwards.
    - `stop`: keep all rows matching dates before this one.
    - `start` and `stop`: keep all rows between `start` and `stop`,

    Any other combination will raise an error. The lower bound of the date range
    will be included, the upper bound will be excluded.

    When specified, `start`, `stop` and `atdate` values are expected to match the
    `date_format` format or a known symbolic value (i.e. 'TODAY', 'YESTERDAY' or 'TOMORROW').

    Additionally, the offset syntax "(date) + offset" is also supported (Mind
    the parenthesis around the date string). In that case, the offset must be
    one of the syntax supported by `pandas.Timedelta` (see [pandas doc](
    http://pandas.pydata.org/pandas-docs/stable/timedeltas.html))

    ---

    ### Parameters

    *mandatory :*
    - `date_col` (*str*): the name of the dataframe's column to filter on

    *optional :*
    - `date_format` (*str*): expected date format in column `date_col` (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior)
    - `start` (*str*): if specified, lower bound (included) of the date range
    - `stop` (*str*): if specified, upper bound (excluded) of the date range
    - `atdate` (*str*): if specified, the exact date we're filtering on
    """
    mask = None
    if start is None and stop is None and atdate is None:
        raise TypeError('either "start", "stop" or "atdate" must be specified')
    if start is not None and atdate is not None:
        raise TypeError('"start" and "atdate" are mutually exclusive')
    if stop is not None and atdate is not None:
        raise TypeError('"stop" and "atdate" are mutually exclusive')
    # add a new column that will hold actual date objects instead of strings.
    # This column is just temporary and will be removed before returning the
    # filtered dataframe.
    filtercol = str(uuid4())
    df[filtercol] = pd.to_datetime(df[date_col], format=date_format)
    if atdate is not None:
        mask = df[filtercol] == parse_date(atdate, date_format)
    elif start is not None and stop is not None:
        mask = ((df[filtercol] >= parse_date(start, date_format)) &
                (df[filtercol] < parse_date(stop, date_format)))
    elif stop is None:
        mask = df[filtercol] >= parse_date(start, date_format)
    elif start is None:
        mask = df[filtercol] < parse_date(stop, date_format)
    return df[mask].drop(filtercol, axis=1)

Response: Filter dataframe your data by date.


Instruction:
Write a Python 3 function for
replacing a column in a DataFrame.

Response:
def replace(df, column: str, new_column: str = None, **kwargs):
    """
    Change the label of a value or a columns within your data source.
    (Similar to `rename` but does not have the notion of locale)

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to modify.
    - `to_replace` (*dict*): keys of this dict are old values pointing on substitute.

    *optional :*
    - `new_column` (*str*): name of the output column. By default the `column` arguments is modified.

    **Note**: extra parameters can be used (see [pandas doc](
    https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.replace.html))

    ---

    ### Example

    **Input**

    article | rating
    :------:|:------:
    book    | 1
    puzzle  | 3
    food    | 5

    We want to split the ratings in three categories: "good", "average" and "poor".

    ```cson
    replace:
      column: "rating"
      new_column: "rating_category"  # create a new column with replaced data
      to_replace:
        1: "poor"
        2: "poor"
        3: "average"
        4: "good"
        5: "good"
    ```

    **Ouput**

    article | rating | rating_category
    :------:|:------:|:--------------:
    book    | 1      | poor
    puzzle  | 3      | average
    food    | 5      | good
    """
    new_column = new_column or column
    df.loc[:, new_column] = df[column].replace(**kwargs)
    return df


Instruction:
Implement a Python 3 function for
adding a column to the dataframe according to the groupby logic on group_cols --- ### Parameters *mandatory :* - `column` (*str*): name of the desired column you need percentage on *optional :* - `group_cols` (*list*): names of columns for the groupby logic - `new_column` (*str*): name of the output column. By default `column` will be overwritten. --- **Input** | gender | sport | number | |:------:|:----------:|:------:| | male | bicycle | 17 | | female | basketball | 17 | | male | basketball | 3 | | female | football | 7 | | female | running | 30 | | male | running | 20 | | male | football | 21 | | female | bicycle | 17 | ```cson percentage: new_column: 'number_percentage' column: 'number' group_cols: ['sport'] ``` **Output** | gender | sport | number | number_percentage | |:------:|:----------:|:------:|:-----------------:| | male | bicycle | 17 | 50.0 | | female | basketball | 17 | 85.0 | | male | basketball | 3 | 15.0 | | female | football | 7 | 25.0 | | female | running | 30 | 60.0 | | male | running | 20 | 40.0 | | male | football | 21 | 75.0 | | female | bicycle | 17 | 50.0 |

Response:
def percentage(
        df,
        column: str,
        group_cols: Union[str, List[str]] = None,
        new_column: str = None
):
    """
    Add a column to the dataframe according to the groupby logic on group_cols

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the desired column you need percentage on

    *optional :*
    - `group_cols` (*list*): names of columns for the groupby logic
    - `new_column` (*str*): name of the output column. By default `column` will be overwritten.

    ---

    **Input**

    | gender |    sport   | number |
    |:------:|:----------:|:------:|
    |  male  |   bicycle  |   17   |
    | female | basketball |   17   |
    |  male  | basketball |    3   |
    | female |  football  |    7   |
    | female |   running  |   30   |
    |  male  |   running  |   20   |
    |  male  |  football  |   21   |
    | female |   bicycle  |   17   |

    ```cson
    percentage:
      new_column: 'number_percentage'
      column: 'number'
      group_cols: ['sport']
    ```

    **Output**

    | gender |    sport   | number | number_percentage |
    |:------:|:----------:|:------:|:-----------------:|
    |  male  |   bicycle  |   17   |        50.0       |
    | female | basketball |   17   |        85.0       |
    |  male  | basketball |    3   |        15.0       |
    | female |  football  |    7   |        25.0       |
    | female |   running  |   30   |        60.0       |
    |  male  |   running  |   20   |        40.0       |
    |  male  |  football  |   21   |        75.0       |
    | female |   bicycle  |   17   |        50.0       |
    """
    new_column = new_column or column
    if group_cols is None:
        df[new_column] = 100. * df[column] / sum(df[column])
    else:
        df[new_column] = 100. * df[column] / df.groupby(group_cols)[column].transform(sum)
    return df


Instruction:
Implement a function in Python 3 to
optimize by SGD, AdaGrad, or AdaDelta.

Response:
def ada_family_core(params, gparams, learning_rate = 0.01, eps= 1e-6, rho=0.95, method="ADADELTA",
                        beta=0.0, gsum_regularization = 0.0001):
    """
    Optimize by SGD, AdaGrad, or AdaDelta.
    """

    _, _, _, args = inspect.getargvalues(inspect.currentframe())
    logging.info("ada_family_core: %s" % str(args.items()))
    free_parameters = []

    if method == "FINETUNING_ADAGRAD":
        method = "ADAGRAD"
        gsum_regularization = 0

    oneMinusBeta = 1 - beta

    gsums   = [theano.shared(np.zeros_like(param.get_value(borrow=True), dtype=FLOATX), name="gsum_%s" % param.name) if (method == 'ADADELTA' or method == 'ADAGRAD') else None for param in params]
    xsums   = [theano.shared(np.zeros_like(param.get_value(borrow=True), dtype=FLOATX), name="xsum_%s" % param.name) if method == 'ADADELTA' else None for param in params]

    # Fix for AdaGrad, init gsum to 1
    if method == 'ADAGRAD':
        for gsum in gsums:
            gsum.set_value(gsum.get_value() ** 0)

    updates = OrderedDict()
    # Updates
    for gparam, param, gsum, xsum in zip(gparams, params, gsums, xsums):

        if method == 'ADADELTA':
            updates[gsum] = rho * gsum + (1. - rho) * (gparam **2)
            dparam = -T.sqrt((xsum + eps) / (updates[gsum] + eps)) * gparam
            updates[xsum] =rho * xsum + (1. - rho) * (dparam **2)
            updates[param] = param * oneMinusBeta + dparam
        elif method == 'ADAGRAD':
            updates[gsum] = gsum + (gparam **2) - gsum_regularization * gsum
            updates[param] =  param * oneMinusBeta - learning_rate * (gparam / (T.sqrt(updates[gsum] + eps)))

        else:
            updates[param] = param * oneMinusBeta - gparam * learning_rate
    # Add free parameters
    if method == 'ADADELTA':
        free_parameters.extend(gsums + xsums)
    elif method == 'ADAGRAD':
        free_parameters.extend(gsums)
    # Check dtype
    for k in updates:
        if updates[k].dtype != FLOATX:
            updates[k] = updates[k].astype(FLOATX)
    return updates.items(), free_parameters


Instruction:
Can you tell what is the following Python 3 function doing
def optimize_updates(params, gradients, config=None, shapes=None):
    if config and isinstance(config, dict):
        config = TrainerConfig(config)

    # Clipping
    if config:
        clip_value = config.get("gradient_clipping", None)

        if clip_value:
            clip_constant = T.constant(clip_value, dtype=FLOATX)

            if config.avoid_compute_embed_norm:
                grad_norm = multiple_l2_norm([t[1] for t in zip(params, gradients) if not t[0].name.startswith("W_embed")])
            else:
                grad_norm = multiple_l2_norm(gradients)
            isnan = T.or_(T.isnan(grad_norm), T.isinf(grad_norm))
            multiplier = ifelse(grad_norm < clip_constant,
                                T.constant(1., dtype=FLOATX), clip_constant / (grad_norm + EPSILON))

            # Clip
            clipped_gradients = []
            for param, g in zip(params, gradients):
                g = multiplier * g
                if config.avoid_nan:
                    g = T.switch(isnan, np.float32(0.1) * param, g)
                if config.gradient_tolerance:
                    g = ifelse(grad_norm > config.gradient_tolerance, T.zeros_like(g) + EPSILON, g)
                clipped_gradients.append(g)

            gradients = clipped_gradients
    # Regularization
    if config and config.weight_l2:
        regularized_gradients = []
        for param, grad in zip(params, gradients):
            grad = grad + (2 * config.weight_l2 * param)
            regularized_gradients.append(grad)
        gradients = regularized_gradients

    # Avoid nan but not computing the norm
    # This is not recommended
    if config and config.avoid_nan and not config.gradient_clipping:
        logging.info("avoid NaN gradients")
        new_gradients = []
        for grad in gradients:
            new_grad = ifelse(T.isnan(grad).any(), T.zeros_like(grad) + EPSILON, grad)
            new_gradients.append(new_grad)
        gradients = new_gradients

    # Find method
    method = "SGD"
    if config:
        method = config.get("method", method).upper()
    # Get Function
    func = None
    if method in ["SGD", "ADAGRAD", "ADADELTA", "FINETUNING_ADAGRAD"]:
        from cores.ada_family import ada_family_core
        func = ada_family_core
    elif method == "ADAM":
        from cores.adam import adam_core
        func = adam_core
    elif method == "RMSPROP":
        from cores.rmsprop import rmsprop_core
        func = rmsprop_core
    elif method == "MOMENTUM":
        from cores.momentum import momentum_core
        func = momentum_core

    if not func:
        raise NotImplementedError("method '%s' is not supported" % method)

    logging.info("optimize method=%s parameters=%s" % (method, str(params)))

    free_parameters = []
    return_vals = wrap_core(func, config, params, gradients)
    if type(return_vals) == list and type(return_vals[0]) == list:
        updates, free_parameters = return_vals
    else:
        updates = return_vals

    # No free param recording
    if config and not config.record_free_params:
        free_parameters = []

    # Weight bound
    if config.weight_bound:
        logging.info("apply weight bound of %.2f" % config.weight_bound)
        new_updates = []
        for param, update_value in updates:
            bounded_value = (update_value * (T.abs_(update_value) <= config.weight_bound) +
                             config.weight_bound * (update_value > config.weight_bound) +
                             -config.weight_bound * (update_value < -config.weight_bound))
            new_updates.append((param, bounded_value))
        updates = new_updates
    return updates, free_parameters

Response: General optimization function for Theano.


Instruction:
Can you generate the documentation for the following Python 3 function
def optimize_function(params, config=None):
    gs = [dim_to_var(p.ndim) for p in params]
    updates, _ = optimize_updates(params, gs, config)
    return theano.function(gs, [], updates=updates)

Response: Create a optimizing function receives gradients.


Instruction:
Implement a function in Python 3 to
return updates in the training.

Response:
def _learning_updates(self):
        """
        Return updates in the training.
        """
        params = self.training_params()
        gradients = self.get_gradients(params)
        return self.optimization_updates(params, gradients)


Instruction:
Can you create a Python 3 function that
gets parameters to be optimized.

Response:
def training_params(self):
        """
        Get parameters to be optimized.
        """
        params = self.network.parameters
        # Freeze parameters
        if self.config.fixed_parameters:
            logging.info("fixed parameters: %s" % ", ".join(map(str, self.config.fixed_parameters)))
            params = [p for p in params if p not in self.config.fixed_parameters]
        return params


Instruction:
How would you code a function in Python 3 to
return updates from optimization.

Response:
def optimization_updates(self, params, gradients):
        """
        Return updates from optimization.
        """
        updates, free_parameters = optimize_updates(params, gradients, self.config)
        self.network.free_parameters.extend(free_parameters)
        logging.info("Added %d free parameters for optimization" % len(free_parameters))
        return updates


Instruction:
How would you code a function in Python 3 to
get the learning function.

Response:
def learning_function(self):
        """
        Get the learning function.
        :param func:
        :return:
        """
        network_updates = list(self.network.updates) + list(self.network.training_updates)
        learning_updates = list(self._learning_updates())
        update_list = network_updates + learning_updates

        logging.info("network updates: %s" % " ".join(map(str, [x[0] for x in network_updates])))
        logging.info("learning updates: %s" % " ".join(map(str, [x[0] for x in learning_updates])))

        variables = self.network.input_variables + self.network.target_variables
        givens = None
        return theano.function(
            variables,
            map(lambda v: theano.Out(v, borrow=True), self.training_variables),
            updates=update_list, allow_input_downcast=True,
            mode=self.config.get("theano_mode", None),
            givens=givens)


Instruction:
Can you implement a function in Python 3 that
functions to compute the GLIMPSE sensor from image and focus vector.

Response:
def _glimpse_sensor(self, x_t, l_p):
        """
        Parameters:
            x_t - 28x28 image
            l_p - 2x1 focus vector
        Returns:
            4x12 matrix
        """
        # Turn l_p to the left-top point of rectangle
        l_p = l_p * 14 + 14 - 2
        l_p = T.cast(T.round(l_p), "int32")

        l_p = l_p * (l_p >= 0)
        l_p = l_p * (l_p < 24) + (l_p >= 24) * 23
        l_p2 = l_p - 2
        l_p2 = l_p2 * (l_p2 >= 0)
        l_p2 = l_p2 * (l_p2 < 20) + (l_p2 >= 20) * 19
        l_p3 = l_p - 6
        l_p3 = l_p3 * (l_p3 >= 0)
        l_p3 = l_p3 * (l_p3 < 16) + (l_p3 >= 16) * 15
        glimpse_1 = x_t[l_p[0]: l_p[0] + 4][:, l_p[1]: l_p[1] + 4]
        glimpse_2 = x_t[l_p2[0]: l_p2[0] + 8][:, l_p2[1]: l_p2[1] + 8]
        glimpse_2 = theano.tensor.signal.downsample.max_pool_2d(glimpse_2, (2,2))
        glimpse_3 = x_t[l_p3[0]: l_p3[0] + 16][:, l_p3[1]: l_p3[1] + 16]
        glimpse_3 = theano.tensor.signal.downsample.max_pool_2d(glimpse_3, (4,4))
        return T.concatenate([glimpse_1, glimpse_2, glimpse_3])


Instruction:
How would you explain what the following Python 3 function does
def _refined_glimpse_sensor(self, x_t, l_p):
        # Turn l_p to the left-top point of rectangle
        l_p = l_p * 14 + 14 - 4
        l_p = T.cast(T.round(l_p), "int32")

        l_p = l_p * (l_p >= 0)
        l_p = l_p * (l_p < 21) + (l_p >= 21) * 20
        glimpse_1 = x_t[l_p[0]: l_p[0] + 7][:, l_p[1]: l_p[1] + 7]
        # glimpse_2 = theano.tensor.signal.downsample.max_pool_2d(x_t, (4,4))
        # return T.concatenate([glimpse_1, glimpse_2])
        return glimpse_1

Response: Refines the GLIMPSE sensor.


Instruction:
Given the following Python 3 function, write the documentation
def _glimpse_network(self, x_t, l_p):
        sensor_output = self._refined_glimpse_sensor(x_t, l_p)
        sensor_output = T.flatten(sensor_output)
        h_g = self._relu(T.dot(sensor_output, self.W_g0))
        h_l = self._relu(T.dot(l_p, self.W_g1))
        g = self._relu(T.dot(h_g, self.W_g2_hg) + T.dot(h_l, self.W_g2_hl))
        return g

Response: Function that computes the Glimse network for a single image and a single focus vector.


Instruction:
Explain what the following Python 3 code does
def _action_network(self, h_t):
        z = self._relu(T.dot(h_t, self.W_a) + self.B_a)
        return self._softmax(z)

Response: Function that computes the action network of the logarithmic state.


Instruction:
Given the following Python 3 function, write the documentation
def get_network(model=None, std=0.005, disable_reinforce=False, random_glimpse=False):
    network = NeuralClassifier(input_dim=28 * 28)
    network.stack_layer(FirstGlimpseLayer(std=std, disable_reinforce=disable_reinforce, random_glimpse=random_glimpse))
    if model and os.path.exists(model):
        network.load_params(model)
    return network

Response:
Get baseline model.
    Parameters:
        model - model path
    Returns:
        network


Instruction:
Can you implement a function in Python 3 that
computes first glimpse sensor.

Response:
def _first_glimpse_sensor(self, x_t):
        """
        Compute first glimpse position using down-sampled image.
        """
        downsampled_img = theano.tensor.signal.downsample.max_pool_2d(x_t, (4,4))
        downsampled_img = downsampled_img.flatten()
        first_l = T.dot(downsampled_img, self.W_f)
        if self.disable_reinforce:
            wf_grad = self.W_f
            if self.random_glimpse:
                first_l = self.srng.uniform((2,), low=-1.7, high=1.7)
        else:
            sampled_l_t = self._sample_gaussian(first_l, self.cov)
            sampled_pdf = self._multi_gaussian_pdf(disconnected_grad(sampled_l_t), first_l)
            wf_grad = T.grad(T.log(sampled_pdf), self.W_f)
            first_l = sampled_l_t
        return first_l, wf_grad


Instruction:
Can you write a function in Python 3 where it
functions to compute the core network.

Response:
def _core_network(self, l_p, h_p, x_t):
        """
        Parameters:
            x_t - 28x28 image
            l_p - 2x1 focus vector
            h_p - 256x1 vector
        Returns:
            h_t, 256x1 vector
        """
        g_t = self._glimpse_network(x_t, l_p)
        h_t = self._tanh(T.dot(g_t, self.W_h_g) + T.dot(h_p, self.W_h) + self.B_h)
        l_t = self._location_network(h_t)

        if not self.disable_reinforce:
            sampled_l_t = self._sample_gaussian(l_t, self.cov)
            sampled_pdf = self._multi_gaussian_pdf(disconnected_grad(sampled_l_t), l_t)
            wl_grad = T.grad(T.log(sampled_pdf), self.W_l)
        else:
            sampled_l_t = l_t
            wl_grad = self.W_l

        if self.random_glimpse and self.disable_reinforce:
            sampled_l_t = self.srng.uniform((2,), low=-1.7, high=1.7)

        a_t = self._action_network(h_t)

        return sampled_l_t, h_t, a_t, wl_grad


Instruction:
Write a Python 3 function for
preparing the internal layer and the internal layer classifiers.

Response:
def prepare(self):
        """
        All codes that create parameters should be put into 'setup' function.
        """
        self.output_dim = 10
        self.encoder = Chain(self.input_dim).stack(Dense(self.internal_layer_size, 'tanh'))
        self.decoder = Chain(self.internal_layer_size).stack(Dense(self.input_dim))
        self.classifier = Chain(self.internal_layer_size).stack(Dense(50, 'tanh'),
                                                      Dense(self.output_dim),
                                                      Softmax())

        self.register_inner_layers(self.encoder, self.decoder, self.classifier)

        self.target_input = T.ivector('target')
        self.register_external_inputs(self.target_input)


Instruction:
Write a Python 3 function for
computing the cost of a single tensor.

Response:
def compute_tensor(self, x):
        """
        Build the computation graph here.
        """
        internal_variable = self.encoder.compute_tensor(x)

        decoding_output = self.decoder.compute_tensor(internal_variable)

        classification_output = self.classifier.compute_tensor(internal_variable)

        auto_encoder_cost = AutoEncoderCost(decoding_output, x).get()

        classification_cost = CrossEntropyCost(classification_output, self.target_input).get()

        final_cost = 0.01 * auto_encoder_cost + classification_cost

        error_rate = ErrorRateCost(classification_output, self.target_input).get()

        self.register_monitors(("err", error_rate),
                               ("encoder_cost", auto_encoder_cost),
                               ("classify_cost", classification_cost))

        return final_cost


Instruction:
Can you generate the documentation for the following Python 3 function
def map(self, func):
        if self._train_set:
            self._train_set = map(func, self._train_set)
        if self._valid_set:
            self._valid_set = map(func, self._valid_set)
        if self._test_set:
            self._test_set = map(func, self._test_set)

Response: Process all data with given function.


Instruction:
Can you create a Python 3 function that
makes targets be one-hot vectors.

Response:
def vectorize_target(self, size):
        """
        Make targets be one-hot vectors.
        """
        if self._train_set:
            self._train_set = self._vectorize_set(self._train_set, size)
        if self._valid_set:
            self._valid_set = self._vectorize_set(self._valid_set, size)
        if self._test_set:
            self._test_set = self._vectorize_set(self._test_set, size)


Instruction:
Can you generate the documentation for the following Python 3 function
def train(self, train_set, valid_set=None, test_set=None, train_size=None):
        '''We train over mini-batches and evaluate periodically.'''
        iteration = 0
        while True:
            if not iteration % self.config.test_frequency and test_set:
                try:
                    self.test(iteration, test_set)
                except KeyboardInterrupt:
                    logging.info('interrupted!')
                    break

            if not iteration % self.validation_frequency and valid_set:
                try:
                    if not self.evaluate(iteration, valid_set):
                        logging.info('patience elapsed, bailing out')
                        break
                except KeyboardInterrupt:
                    logging.info('interrupted!')
                    break

            train_message = ""
            try:
                train_message = self.train_func(train_set)
            except KeyboardInterrupt:
                logging.info('interrupted!')
                break
            if not iteration % self.config.monitor_frequency:
                logging.info('monitor (iter=%i) %s', iteration + 1, train_message)

            iteration += 1
            if hasattr(self.network, "iteration_callback"):
                self.network.iteration_callback()

            yield train_message

        if valid_set:
            self.set_params(self.best_params)
        if test_set:
            self.test(0, test_set)

Response: Train over mini - batches and evaluate periodically.


Instruction:
Can you write a function in Python 3 where it
samples outputs from LM.

Response:
def sample(self, input, steps):
        """
        Sample outputs from LM.
        """
        inputs = [[onehot(self.input_dim, x) for x in input]]
        for _ in range(steps):
            target = self.compute(inputs)[0,-1].argmax()
            input.append(target)
            inputs[0].append(onehot(self.input_dim, target))
        return input


Instruction:
Can you tell what is the following Python 3 function doing
def compute_tensor(self, x):
        # Target class
        class_matrix = self.target_tensor // self.output_size
        class_vector = class_matrix.reshape((-1,))
        # Target index
        target_matrix = self.target_tensor % self.output_size
        target_vector = target_matrix.reshape((-1,))
        # Input matrix
        input_matrix = x.reshape((-1, self.input_dim))
        # Output matrix
        output_tensor3d = self.output_layer.compute_tensor(x)
        output_matrix = output_tensor3d.reshape((-1, self.class_size, self.output_size))
        arange_vec = self.arange_cache[:output_matrix.shape[0]]
        sub_output_matrix = output_matrix[arange_vec, class_vector]
        # Softmax
        softmax_output_matrix = self.softmax_layer.compute_tensor(sub_output_matrix)
        # Class prediction
        class_output_matrix = self.class_layer.compute_tensor(x)
        # Costs
        output_cost = LMCost(softmax_output_matrix, target_vector).get()
        class_cost = LMCost(class_output_matrix, class_matrix).get()
        final_cost = output_cost + class_cost

        return final_cost

Response: Compute the tensor for the given batch time and vec.


Instruction:
How would you implement a function in Python 3 that
computes the alignment weights based on the previous state and the precomputed values.

Response:
def compute_alignments(self, prev_state, precomputed_values, mask=None):
        """
        Compute the alignment weights based on the previous state.
        """

        WaSp = T.dot(prev_state, self.Wa)
        UaH = precomputed_values
        # For test time the UaH will be (time, output_dim)
        if UaH.ndim == 2:
            preact = WaSp[:, None, :] + UaH[None, :, :]
        else:
            preact = WaSp[:, None, :] + UaH
        act = T.activate(preact, 'tanh')
        align_scores = T.dot(act, self.Va)  # ~ (batch, time)
        if mask:
            mask = (1 - mask) * -99.00
            if align_scores.ndim == 3:
                align_scores += mask[None, :]
            else:
                align_scores += mask
        align_weights = T.nnet.softmax(align_scores)
        return align_weights


Instruction:
Can you implement a function in Python 3 that
computes the context vector with soft attention.

Response:
def compute_context_vector(self, prev_state, inputs, precomputed_values=None, mask=None):
        """
        Compute the context vector with soft attention.
        """
        precomputed_values = precomputed_values if precomputed_values else self.precompute(inputs)
        align_weights = self.compute_alignments(prev_state, precomputed_values, mask)
        context_vector = T.sum(align_weights[:, :, None] * inputs, axis=1)
        return context_vector


Instruction:
Implement a function in Python 3 to
train the model in multi - GPU environment.

Response:
def train(self, train_set, valid_set=None, test_set=None, train_size=None):
        """
        Train the model in multi-GPU environment.
        """
        from platoon.channel import Worker
        from platoon.param_sync import EASGD, ASGD
        server_port = self._port
        param_map = self.create_param_map()
        # Initialize the worker
        worker = Worker(control_port=server_port)
        if self.config.learning_rate:
            worker.send_req({'init_schedule': self._schedule_params})
        self.sync_hyperparams(worker.send_req('sync_hyperparams')['sync_hyperparams'])
        easgd_alpha = worker.send_req('get_easgd_alpha')
        if self._using_easgd:
            self.logger.info("using EASGD with alpha={}".format(easgd_alpha))
        else:
            self.logger.info("using ASGD rule")
        rule = EASGD(easgd_alpha) if self._using_easgd else ASGD()
        worker.init_shared_params(param_map.values(), param_sync_rule=rule)
        worker.send_req({
            "set_names": None,
            "training_names": self.training_names,
            "evaluation_names": self.evaluation_names
        })
        # Load all training batches, consume vast memory here
        self.logger.info("started process {}".format(os.getpid()))
        self.logger.info("(proc {}) load training data".format(os.getpid()))
        train_batches = list(train_set)
        network_callback = bool(self.network.training_callbacks)
        trainer_callback = bool(self._iter_controllers)
        # Start from valid, so the performance when a worked join can be known
        worker.copy_to_local()
        if valid_set:
            self._run_valid(self.epoch, valid_set, dry_run=True)
            self.fix_costs()
        worker.send_req({
            "valid_done": None,
            "valid_costs": self.last_run_costs,
            "auto_save": self.config.auto_save
        })
        worker.copy_to_local()
        # Begin the loop
        while True:
            resp = worker.send_req('next')
            if resp == 'stop':
                break
            elif resp == 'wait':
                time.sleep(1)
            elif resp == 'get_num_batches':
                worker.send_req({'get_num_batches_done': len(train_batches)})
            elif 'eval' in resp:
                self.best_cost = resp['best_valid_cost']
                worker.copy_to_local()
                valid_costs = None
                test_costs = None
                if valid_set:
                    self._run_valid(self.epoch, valid_set)
                    self.fix_costs()
                    valid_costs = self.last_run_costs
                if test_set:
                    self._run_test(self.epoch, test_set)
                    self.fix_costs()
                    test_costs = self.last_run_costs
                worker.send_req({
                    "eval_done": None,
                    "valid_costs": valid_costs,
                    "test_costs": test_costs,
                    "auto_save": self.config.auto_save
                })
            elif 'valid' in resp:
                self.best_cost = resp['best_valid_cost']
                worker.copy_to_local()
                if valid_set:
                    self._run_valid(self.epoch, valid_set, dry_run=True)
                    self.fix_costs()
                worker.send_req({
                    "valid_done": None,
                    "valid_costs": self.last_run_costs,
                    "auto_save": self.config.auto_save
                })
            elif 'train' in resp:
                batch_ids = resp['train']
                batch_costs = [[] for _ in self.training_names]
                for batch_id in batch_ids:
                    x = train_batches[batch_id]
                    cost_x = self.learn(*x)
                    for i, cost in enumerate(cost_x):
                        batch_costs[i].append(cost)
                    self.last_cost = cost_x[0]
                if network_callback:
                    self.network.training_callback()
                if trainer_callback:
                    for func in self._iter_controllers:
                        func(self)
                worker.sync_params(synchronous=True)
                worker.send_req({'train_done': None, 'costs': [float(np.mean(c)) for c in batch_costs]})
            elif 'sync_hyperparams' in resp:
                self.sync_hyperparams(resp['sync_hyperparams'])
        worker.close()
        return []


Instruction:
Given the following Python 3 function, write the documentation
def var(tensor_type, last_dim=0, test_shape=None):
    # Create tensor
    from deepy.core.neural_var import NeuralVariable
    from deepy.core.env import env
    from theano.tensor.var import TensorVariable
    if isinstance(tensor_type, NeuralVariable):
        var = tensor_type
        if last_dim != 0:
            var.output_dim = last_dim
    elif isinstance(tensor_type, TensorVariable):
        var = NeuralVariable(tensor_type, dim=last_dim)
    elif isinstance(tensor_type, str):
        theano_tensor = getattr(TT, tensor_type)()
        var = NeuralVariable(theano_tensor, dim=last_dim)
    else:
        raise Exception("tensor_type shall be a string or a NeuralVariable")
    # Set test value
    if test_shape:
        if type(test_shape) != list and type(test_shape) != tuple:
            # May be it's a value
            var.set_test_value(test_shape)
        else:
            test_val = env.numpy_rand.rand(*test_shape)
            if len(test_shape) > 0:
                test_val = test_val.astype(var.tensor.dtype)
            elif var.tensor.dtype.startswith("int"):
                test_val = 1
            var.set_test_value(test_val)
    else:
        # Create a general test_shape
        dims = [(d + 1) * 3 for d in range(var.tensor.ndim)]
        if var.dim() != 0:
            dims[-1] = var.dim()
        test_val = env.numpy_rand.rand(*dims)
        if len(dims) > 0:
            test_val = test_val.astype(var.tensor.dtype)
        elif var.tensor.dtype.startswith("int"):
            test_val = 1
        var.set_test_value(test_val)
    return var

Response: Wrap a Theano tensor into a NeuralVariable object for defining neural network.


Instruction:
Can you write a function in Python 3 where it
pads the dataset to given length in the left or right side.

Response:
def _pad(self, side, length):
        """
        Pad sequences to given length in the left or right side.
        """
        if self._train_set:
            self._train_set = pad_dataset(self._train_set, side, length)
        if self._valid_set:
            self._valid_set = pad_dataset(self._valid_set, side, length)
        if self._test_set:
            self._test_set = pad_dataset(self._test_set, side, length)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def pad_dataset(subset, side="right", length=-1):
    assert length == -1 or length > 0
    if type(subset[0][0][0]) in [float, int, np.int64, np.int32, np.float32]:
        return _pad_2d(subset, side, length)
    else:
        return _pad_3d(subset, side, length)

Response: Pads a dataset to specified length.


Instruction:
Create a Python 3 function for
preparing for one epoch.

Response:
def prepare_epoch(self):
        """
        Prepare for one epoch.
        Returns:
            bool: False if to stop the training.
        """
        self.epoch += 1
        if self.epoch >= self.epoch_start_halving and ((self.epoch - self.epoch_start_halving) % self._halving_freq == 0):
            self._lr *= 0.5
        self._current_iter = 0
        self._iters_from_last_valid = 0
        self._train_costs = []
        self.prepared_worker_pool.clear()
        self.batch_pool = range(self.num_train_batches)
        self.rand.shuffle(self.batch_pool)
        if self.epoch > self.end_at:
            self.log("Training is done, wait all workers to stop")
            return False
        else:
            self.log("start epoch {} with lr={}".format(self.epoch, self._lr))
            return True


Instruction:
Make a summary of the following Python 3 code
def handle_control(self, req, worker_id, req_info):
        if self.start_time is None: self.start_time = time.time()
        response = ""

        if req == 'next':
            if self.num_train_batches == 0:
                response = "get_num_batches"
            elif self._done:
                response = "stop"
                self.worker_is_done(worker_id)
            elif self._evaluating:
                response = 'wait'
            elif not self.batch_pool:
                # End of one iter
                if self._train_costs:
                    with self._lock:
                        sys.stdout.write("\r")
                        sys.stdout.flush()
                        mean_costs = []
                        for i in range(len(self._training_names)):
                            mean_costs.append(np.mean([c[i] for c in self._train_costs]))
                        self.log("train   (epoch={:2d}) {}".format(
                            self.epoch,
                            self.get_monitor_string(zip(self._training_names, mean_costs)))
                        )
                response = {'eval': None, 'best_valid_cost': self._best_valid_cost}
                self._evaluating = True
            else:
                # Continue training
                if worker_id not in self.prepared_worker_pool:
                    response = {"sync_hyperparams": self.feed_hyperparams()}
                    self.prepared_worker_pool.add(worker_id)
                elif self._iters_from_last_valid >= self._valid_freq:
                    response = {'valid': None, 'best_valid_cost': self._best_valid_cost}
                    self._iters_from_last_valid = 0
                else:
                    response = {"train": self.feed_batches()}
        elif 'eval_done' in req:
            with self._lock:
                self._evaluating = False
                sys.stdout.write("\r")
                sys.stdout.flush()
                if 'test_costs' in req and req['test_costs']:
                    self.log("test    (epoch={:2d}) {} (worker {})".format(
                        self.epoch,
                        self.get_monitor_string(req['test_costs']),
                        worker_id)
                    )
                if 'valid_costs' in req and req['test_costs']:
                    valid_J = req['valid_costs'][0][1]
                    if valid_J < self._best_valid_cost:
                        self._best_valid_cost = valid_J
                        star_str = "*"
                    else:
                        star_str = ""
                        self.log("valid   (epoch={:2d}) {} {} (worker {})".format(
                        self.epoch,
                        self.get_monitor_string(req['valid_costs']),
                        star_str,
                        worker_id))
                    # if star_str and 'auto_save' in req and req['auto_save']:
                    #     self.log("(worker {}) save the model to {}".format(
                    #         worker_id,
                    #         req['auto_save']
                    #     ))
                continue_training = self.prepare_epoch()
                self._epoch_start_time = time.time()
                if not continue_training:
                    self._done = True
                    self.log("training time {:.4f}s".format(time.time() - self.start_time))
                    response = "stop"
        elif 'valid_done' in req:
            with self._lock:
                sys.stdout.write("\r")
                sys.stdout.flush()
                if 'valid_costs' in req:
                    valid_J = req['valid_costs'][0][1]
                    if valid_J < self._best_valid_cost:
                        self._best_valid_cost = valid_J
                        star_str = "*"
                    else:
                        star_str = ""
                    self.log("valid   ( dryrun ) {} {} (worker {})".format(
                        self.get_monitor_string(req['valid_costs']),
                        star_str,
                        worker_id
                    ))
                    # if star_str and 'auto_save' in req and req['auto_save']:
                    #     self.log("(worker {}) save the model to {}".format(
                    #         worker_id,
                    #         req['auto_save']
                    #     ))
        elif 'train_done' in req:
            costs = req['costs']
            self._train_costs.append(costs)
            sys.stdout.write("\x1b[2K\r> %d%% | J=%.2f | %.1f batch/s" % (
                self._current_iter * 100 / self.num_train_batches,
                costs[0], float(len(self._train_costs) * self.sync_freq) / (time.time() - self._epoch_start_time)))
            sys.stdout.flush()
        elif 'get_num_batches_done' in req:
            self.num_train_batches = req['get_num_batches_done']
        elif 'get_easgd_alpha' in req:
            response = self._easgd_alpha
        elif 'sync_hyperparams' in req:
            response = {"sync_hyperparams": self.feed_hyperparams()}
        elif 'init_schedule' in req:
            with self._lock:
                sys.stdout.write("\r")
                sys.stdout.flush()
                self.log("worker {} connected".format(worker_id))
                if self.epoch == 0:
                    schedule_params = req['init_schedule']
                    sch_str = " ".join("{}={}".format(a, b) for (a, b) in schedule_params.items())
                    self.log("initialize the schedule with {}".format(sch_str))
                    for key, val in schedule_params.items():
                        if not val: continue
                        if key == 'learning_rate':
                            self._lr = val
                        elif key == 'start_halving_at':
                            self.epoch_start_halving = val
                        elif key == 'halving_freq':
                            self._halving_freq = val
                        elif key == 'end_at':
                            self.end_at = val
                        elif key == 'sync_freq':
                            self.sync_freq = val
                        elif key == 'valid_freq':
                            self._valid_freq = val

        elif 'set_names' in req:
            self._training_names = req['training_names']
            self._evaluation_names = req['evaluation_names']

        return response

Response: Handles a control request from a worker.


Instruction:
Can you write a function in Python 3 where it
compares to previous records and return whether the given cost is a new best.

Response:
def compare(self, cost_map):
        """
        Compare to previous records and return whether the given cost is a new best.
        :return: True if the given cost is a new best
        """
        cri_val = cost_map[self._criteria]
        if self._best_criteria is None:
            self._best_criteria = cri_val
            return True
        else:
            if self._smaller_is_better and cri_val < self._best_criteria:
                self._best_criteria = cri_val
                return True
            elif not self._smaller_is_better and cri_val > self._best_criteria:
                self._best_criteria = cri_val
                return True
            else:
                return False


Instruction:
Write a Python 3 script to
run the model with validation data and return the cost of the model.

Response:
def run(self, data_x):
        """
        Run the model with validation data and return costs.
        """
        output_vars = self.compute(*data_x)
        return self._extract_costs(output_vars)


Instruction:
Can you tell what is the following Python 3 function doing
def invoke(self):
        self._counter += 1
        if self._counter % self._freq == 0:
            cnt = 0.
            sum_map = defaultdict(float)
            for x in self._trainer.get_data(self._data_split):
                val_map = self.run(x)
                if not isinstance(val_map, dict):
                    raise Exception("Monitor.run must return a dict.")
                for k, val in val_map.items():
                    sum_map[k] += val
                cnt += 1
            for k in sum_map:
                sum_map[k] /= cnt
            new_best = self.compare(sum_map)
            self._trainer.report(sum_map, self._data_split, new_best=new_best)
            if new_best:
                self._trainer.save_checkpoint(self._save_path)

Response: This function is called after each iteration.


Instruction:
Create a Python 3 function to
create inner loop variables.

Response:
def _build_loop_vars(self):
        """
        Create inner loop variables.
        """
        from theano.tensor.var import TensorVariable
        from deepy.core.neural_var import NeuralVariable
        if not self._loop_vars:
            self._ordered_out_keys = self._outputs.keys()
            seq_keys = self._sequences.keys()
            filled_out_keys = [k for k in self._ordered_out_keys if self._outputs[k]]
            nonseq_keys = self._non_sequences.keys()
            dummy_tensors, self._scan_local_vars = get_dummy_args(
                sequences=[self._sequences[k].tensor for k in seq_keys],
                outputs_info=[self._outputs[k].tensor for k in self._ordered_out_keys],
                non_sequences=[self._non_sequences[k].tensor for k in nonseq_keys],
                **self._kwargs
            )
            dummy_map = dict(zip(seq_keys + filled_out_keys + nonseq_keys, dummy_tensors))
            arg_map = self._sequences.copy()
            arg_map.update(self._outputs)
            arg_map.update(self._non_sequences)
            self._loop_vars = LoopVars()
            for k, dummy_tensor in dummy_map.items():
                dummy_var = NeuralVariable(dummy_tensor, dim=arg_map[k].dim())
                self._loop_vars[k] = dummy_var


Instruction:
Can you tell what is the following Python 3 function doing
def get_outputs(self, *args):
        if args:
            output_vars = map(self._scan_outputs.get, args)
            if len(output_vars) == 1:
                return output_vars[0]
            else:
                return output_vars
        else:
            return self._scan_outputs

Response: Get the outputs of the loop.


Instruction:
Write a Python 3 script for
executing then_branch when training.

Response:
def iftrain(self, then_branch, else_branch):
        """
        Execute `then_branch` when training.
        """
        return ifelse(self._training_flag, then_branch, else_branch, name="iftrain")


Instruction:
Implement a Python 3 function for
switching training mode. :param flag: switch on training mode when flag is True.

Response:
def switch_training(self, flag):
        """
        Switch training mode.
        :param flag: switch on training mode when flag is True.
        """
        if self._is_training == flag: return
        self._is_training = flag
        if flag:
            self._training_flag.set_value(1)
        else:
            self._training_flag.set_value(0)


Instruction:
Explain what the following Python 3 code does
def nag_core(params, J, momentum=0.9, learning_rate=0.01):
    # TODO: this requires some refractorings.
    for param in params:
        step = theano.shared(np.zeros_like(param.get_value()), name=param.name + '_step')
        velocity = theano.shared(np.zeros_like(param.get_value()), name=param.name + '_vel')
        yield step, momentum * velocity
        yield param, param + step
        yield velocity, step - learning_rate * T.grad(J, param)
        yield param, param + velocity - step

Response: Generator for NAG core


Instruction:
Write a Python 3 script to
skip N batches in the training set

Response:
def skip(self, n_batches, n_epochs=0):
        """
        Skip N batches in the training.
        """
        logging.info("skip %d epochs and %d batches" % (n_epochs, n_batches))
        self._skip_batches = n_batches
        self._skip_epochs = n_epochs


Instruction:
How would you implement a function in Python 3 that
loads parameters for the training.

Response:
def load_params(self, path, exclude_free_params=False):
        """
        Load parameters for the training.
        This method can load free parameters and resume the training progress.
        """
        self.network.load_params(path, exclude_free_params=exclude_free_params)
        self.best_params = self.copy_params()
        # Resume the progress
        if self.network.train_logger.progress() > 0 or self.network.train_logger.epoch() > 0:
            self.skip(self.network.train_logger.progress(), self.network.train_logger.epoch() - 1)


Instruction:
Can you create a Python 3 function that
adds iteration callbacks function.

Response:
def add_iter_controllers(self, *controllers):
        """
        Add iteration callbacks function (receives an argument of the trainer).
        :param controllers: can be a `TrainingController` or a function.
        :type funcs: list of TrainingContoller
        """
        for controller in controllers:
            if isinstance(controller, TrainingController):
                controller.bind(self)
            self._iter_controllers.append(controller)


Instruction:
Can you create a Python 3 function that
adds epoch controllers function.

Response:
def add_epoch_controllers(self, *controllers):
        """
        Add epoch callbacks function.
        :param controllers: can be a `TrainingController` or a function.
        """
        for controller in controllers:
            if isinstance(controller, TrainingController):
                controller.bind(self)
            self._epoch_controllers.append(controller)


Instruction:
Explain what the following Python 3 code does
def train(self, train_set, valid_set=None, test_set=None, train_size=None):
        self._epoch = 0
        while True:
            if self._skip_epochs > 0:
                logging.info("skipping one epoch ...")
                self._skip_epochs -= 1
                self._epoch += 1
                yield None
                continue
            # Test
            if not self._epoch % self.config.test_frequency and test_set:
                try:
                    self._run_test(self._epoch, test_set)
                except KeyboardInterrupt:
                    logging.info('interrupted!')
                    break
            # Validate
            if not self._epoch % self.validation_frequency and valid_set:
                try:

                    if not self._run_valid(self._epoch, valid_set):
                        logging.info('patience elapsed, bailing out')
                        break
                except KeyboardInterrupt:
                    logging.info('interrupted!')
                    break
            # Train one step

            try:
                costs = self._run_train(self._epoch, train_set, train_size)
            except KeyboardInterrupt:
                logging.info('interrupted!')
                break
            # Check costs
            if np.isnan(costs[0][1]):
                logging.info("NaN detected in costs, rollback to last parameters")
                self.set_params(*self.checkpoint)
            else:
                self._epoch += 1
                self.network.epoch_callback()

            yield dict(costs)

        if valid_set and self.config.get("save_best_parameters", True):
            self.set_params(*self.best_params)
        if test_set:
            self._run_test(-1, test_set)

Response: Train the model and return the costs.


Instruction:
Here you have a function in Python 3, explain what it does
def _run_train(self, epoch, train_set, train_size=None):
        self.network.train_logger.record_epoch(epoch + 1)
        costs = self.train_step(train_set, train_size)
        if not epoch % self.config.monitor_frequency:
            self.report(dict(costs), "train", epoch)
        self.last_run_costs = costs
        return costs

Response: Runs one training iteration.


Instruction:
Here you have a function in Python 3, explain what it does
def _run_valid(self, epoch, valid_set, dry_run=False, save_path=None):
        costs = self.valid_step(valid_set)
        # this is the same as: (J_i - J_f) / J_i > min improvement
        _, J = costs[0]
        new_best = False
        if self.best_cost - J > self.best_cost * self.min_improvement:
            # save the best cost and parameters
            self.best_params = self.copy_params()
            new_best = True
            if not dry_run:
                self.best_cost = J
                self.best_epoch = epoch
            self.save_checkpoint(save_path)

        self.report(dict(costs), type="valid", epoch=0 if dry_run else epoch, new_best=new_best)
        self.last_run_costs = costs
        return epoch - self.best_epoch < self.patience

Response: Run one valid iteration return True if to continue training.


Instruction:
Can you generate the documentation for the following Python 3 function
def report(self, score_map, type="valid", epoch=-1, new_best=False):
        type_str = type
        if len(type_str) < 5:
            type_str += " " * (5 - len(type_str))
        info = " ".join("%s=%.2f" % el for el in score_map.items())
        current_epoch = epoch if epoch > 0 else self.current_epoch()
        epoch_str = "epoch={}".format(current_epoch + 1)
        if epoch < 0:
            epoch_str = "dryrun"
            sys.stdout.write("\r")
            sys.stdout.flush()
        marker = " *" if new_best else ""
        message = "{} ({}) {}{}".format(type_str, epoch_str, info, marker)
        self.network.train_logger.record(message)
        logging.info(message)

Response: Report the scores and record them in the log.


Instruction:
Can you create a Python 3 function that
gets the data from the specified split of data.

Response:
def get_data(self, data_split="train"):
        """
        Get specified split of data.
        """
        if data_split == 'train':
            return self._current_train_set
        elif data_split == 'valid':
            return self._current_valid_set
        elif data_split == 'test':
            return self._current_test_set
        else:
            return None


Instruction:
Can you tell what is the following Python 3 function doing
def run(self, train_set, valid_set=None, test_set=None, train_size=None, epoch_controllers=None):
        epoch_controllers = epoch_controllers if epoch_controllers else []
        epoch_controllers += self._epoch_controllers
        if isinstance(train_set, Dataset):
            dataset = train_set
            train_set = dataset.train_set()
            valid_set = dataset.valid_set()
            test_set = dataset.test_set()
            train_size = dataset.train_size()
        self._current_train_set = train_set
        self._current_valid_set = valid_set
        self._current_test_set = test_set
        if epoch_controllers:
            for controller in epoch_controllers:
                controller.bind(self)
        timer = Timer()
        for _ in self.train(train_set, valid_set=valid_set, test_set=test_set, train_size=train_size):
            if epoch_controllers:
                for controller in epoch_controllers:
                    controller.invoke()
            if self._ended:
                break
        if self._report_time:
            timer.report()

Response: Runs the training and test set.


Instruction:
Explain what the following Python 3 code does
def _cut_to_pieces(self, bunch_stack):
        stack_len = len(bunch_stack[0])
        for i in xrange(0, stack_len, self.fragment_length):
            yield np.array(map(lambda stack: stack[i: i + self.fragment_length], bunch_stack))

Response: Yields the pieces of the bunch_stack.


Instruction:
Write a Python 3 function for
padding the bunch_stack with zeros.

Response:
def _pad_zeros(self, bunch_stack):
        """
        :type bunch_stack: list of list
        """
        min_len = min(map(len, bunch_stack))
        for i in range(len(bunch_stack)):
            bunch_stack[i] = bunch_stack[i][:min_len]


Instruction:
How would you explain what the following Python 3 function does
def apply(self, func, dim=None):
        output_dim = dim if dim else self.output_dim
        return NeuralVariable(func(self.tensor), output_dim)

Response: Apply a function to tensors.


Instruction:
Make a summary of the following Python 3 code
def rprop_core(params, gradients, rprop_increase=1.01, rprop_decrease=0.99, rprop_min_step=0, rprop_max_step=100,
               learning_rate=0.01):
    """
    Rprop optimizer.
    See http://sci2s.ugr.es/keel/pdf/algorithm/articulo/2003-Neuro-Igel-IRprop+.pdf.
    """
    for param, grad in zip(params, gradients):
        grad_tm1 = theano.shared(np.zeros_like(param.get_value()), name=param.name + '_grad')
        step_tm1 = theano.shared(np.zeros_like(param.get_value()) + learning_rate, name=param.name+ '_step')

        test = grad * grad_tm1
        same = T.gt(test, 0)
        diff = T.lt(test, 0)
        step = T.minimum(rprop_max_step, T.maximum(rprop_min_step, step_tm1 * (
            T.eq(test, 0) +
            same * rprop_increase +
            diff * rprop_decrease)))
        grad = grad - diff * grad
        yield param, param - T.sgn(grad) * step
        yield grad_tm1, grad
        yield step_tm1, step

Response: Generator for core Rprop optimizer.


Instruction:
Can you generate the documentation for the following Python 3 function
def report(self):
        if self.logger:
            self.logger.info("accessed parameters:")
            for key in self.used_parameters:
                self.logger.info(" - %s %s" % (key, "(undefined)" if key in self.undefined_parameters else ""))

Response: Report usage of training parameters.


Instruction:
How would you explain what the following Python 3 function does
def new_block(self, *layers, **kwargs):
        from deepy.layers.block import Block
        block = Block(*layers, **kwargs)
        return block

Response: Create a new block with some layers and kwargs


Instruction:
Can you generate a brief explanation for the following Python 3 code
def var(self, tensor_type, last_dim=0, test_shape=None):
        from deepy.tensor import var
        return var(tensor_type, last_dim=last_dim, test_shape=test_shape)

Response: Returns the variable of the given type.


Instruction:
Can you generate the documentation for the following Python 3 function
def create_vars_from_data(self, dataset, split="train"):
        from deepy.core.neural_var import NeuralVariable
        vars = []
        if split == "valid":
            data_split = dataset.valid_set()
        elif split == "test":
            data_split = dataset.test_set()
        else:
            data_split = dataset.train_set()
        first_data_piece = list(data_split)[0]
        for i, numpy_tensor in enumerate(first_data_piece):
            if numpy_tensor.dtype == "int64":
                numpy_tensor = numpy_tensor.astype("int32")
            if numpy_tensor.dtype == "float64":
                numpy_tensor = numpy_tensor.astype(env.FLOATX)
            type_map = {
                0: "scalar",
                1: "vector",
                2: "matrix",
                3: "tensor3",
                4: "tensor4",
                5: "tensor5",
            }
            tensor_type = type_map[numpy_tensor.ndim] if numpy_tensor.ndim in type_map else type_map[0]
            if numpy_tensor.dtype.kind == "i":
                tensor_type = "i" + tensor_type
            theano_tensor = getattr(TT, tensor_type)("input_{}_{}".format(i + 1, tensor_type))
            last_dim = numpy_tensor.shape[-1]
            var = NeuralVariable(theano_tensor, dim=last_dim)
            var.set_test_value(numpy_tensor)
            vars.append(var)
        return vars

Response: Create vars given a dataset and set test values.


Instruction:
Can you generate the documentation for the following Python 3 function
def scan(self, func, sequences=None, outputs=None, non_sequences=None, block=None, **kwargs):
        results, updates = Scanner(func, sequences, outputs, non_sequences, neural_computation=True, **kwargs).compute()
        if block and updates:
            if type(updates) == dict:
                updates = updates.items()
            block.register_updates(*updates)
        return results

Response: A loop function that uses the same usage with the theano one.


Instruction:
Make a summary of the following Python 3 code
def loop(self, sequences=None, outputs=None, non_sequences=None, block=None, **kwargs):
        from loop import Loop
        return Loop(sequences, outputs, non_sequences, block, **kwargs)

Response:
Start a loop.
        Usage:
        ```
        with deepy.graph.loop(sequences={"x": x}, outputs={"o": None}) as vars:
            vars.o = vars.x + 1
        loop_outputs = deepy.graph.loop_outputs()
        result = loop_outputs.o
        ```


Instruction:
Can you tell what is the following Python 3 function doing
def get_trainer(self, model,  method='sgd', config=None, annealer=None, validator=None):
        from deepy.trainers import GeneralNeuralTrainer
        return GeneralNeuralTrainer(model, method=method, config=config, annealer=annealer, validator=validator)

Response: Get a trainer to optimize given model.


Instruction:
Given the following Python 3 function, write the documentation
def shared(self, value, name=None):
        if type(value) == int:
            final_value = np.array(value, dtype="int32")
        elif type(value) == float:
            final_value = np.array(value, dtype=env.FLOATX)
        else:
            final_value = value

        return theano.shared(final_value, name=name)

Response: Create a shared theano scalar value.


Instruction:
Can you tell what is the following Python 3 function doing
def fill_parameters(self, path, blocks, exclude_free_params=False, check_parameters=False):
        if not os.path.exists(path):
            raise Exception("model {} does not exist".format(path))
        # Decide which parameters to load
        normal_params = sum([nn.parameters for nn in blocks], [])
        all_params = sum([nn.all_parameters for nn in blocks], [])
        # Load parameters
        if path.endswith(".gz"):
            opener = gzip.open if path.lower().endswith('.gz') else open
            handle = opener(path, 'rb')
            saved_params = pickle.load(handle)
            handle.close()
            # Write parameters
            if len(all_params) != len(saved_params):
                logging.warning(
                    "parameters in the network: {}, parameters in the dumped model: {}".format(len(all_params),
                                                                                len(saved_params)))
            for target, source in zip(all_params, saved_params):
                if not exclude_free_params or target not in normal_params:
                    target.set_value(source)
        elif path.endswith(".npz"):
            arrs = np.load(path)
            # Write parameters
            if len(all_params) != len(arrs.keys()):
                logging.warning(
                    "parameters in the network: {}, parameters in the dumped model: {}".format(len(all_params),
                                                                                len(arrs.keys())))
            for target, idx in zip(all_params, range(len(arrs.keys()))):
                if not exclude_free_params or target not in normal_params:
                    source = arrs['arr_%d' % idx]
                    target.set_value(source)
        else:
            raise Exception("File format of %s is not supported, use '.gz' or '.npz' or '.uncompressed.gz'" % path)

Response: Load parameters from file to fill all blocks sequentially.


Instruction:
Implement a Python 3 function for
returning the size of training data.

Response:
def train_size(self):
        """
        Return size of training data. (optional)
        :rtype: number
        """
        train_set = self.train_set()
        if isinstance(train_set, collections.Iterable):
            return len(list(train_set))
        else:
            return None


Instruction:
Can you tell what is the following Python 3 function doing
def invoke(self):
        self._iter += 1
        if self._iter - max(self._trainer.best_iter, self._annealed_iter) >= self._patience:
            if self._annealed_times >= self._anneal_times:
                logging.info("ending")
                self._trainer.exit()
            else:
                self._trainer.set_params(*self._trainer.best_params)
                self._learning_rate.set_value(self._learning_rate.get_value() * 0.5)
                self._annealed_times += 1
                self._annealed_iter = self._iter
                logging.info("annealed learning rate to %f" % self._learning_rate.get_value())

Response: This method is called by the training thread when training is complete.


Instruction:
How would you implement a function in Python 3 that
runs it return whether to end training.

Response:
def invoke(self):
        """
        Run it, return whether to end training.
        """
        self._iter += 1
        logging.info("{} epochs left to run".format(self._patience - self._iter))
        if self._iter >= self._patience:
            self._trainer.exit()


Instruction:
Can you tell what is the following Python 3 function doing
def stack_reparameterization_layer(self, layer_size):
        self.rep_layer = ReparameterizationLayer(layer_size, sample=self.sample)
        self.stack_encoders(self.rep_layer)

Response: Perform reparameterization trick for latent variables.


Instruction:
Make a summary of the following Python 3 code
def stack_encoders(self, *layers):
        self.stack(*layers)
        self.encoding_layes.extend(layers)

Response: Stack encoding layers, this must be done before stacking decoding layers.


Instruction:
Make a summary of the following Python 3 code
def create_2d_gaussian(dim, sigma):

    # check if the dimension is odd
    if dim % 2 == 0:
        raise ValueError("Kernel dimension should be odd")

    # initialize the kernel
    kernel = np.zeros((dim, dim), dtype=np.float16)

    # calculate the center point
    center = dim/2

    # calculate the variance
    variance = sigma ** 2

    # calculate the normalization coefficeint
    coeff = 1. / (2 * variance)

    # create the kernel
    for x in range(0, dim):
        for y in range(0, dim):
            x_val = abs(x - center)
            y_val = abs(y - center)
            numerator = x_val**2 + y_val**2
            denom = 2*variance

            kernel[x,y] = coeff * np.exp(-1. * numerator/denom)

    return kernel/sum(sum(kernel))

Response: This function creates a 2d gaussian kernel with the standard deviation of the gaussian kernel denoted by sigma.


Instruction:
Here you have a function in Python 3, explain what it does
def stack_layer(self, layer, no_setup=False):
        if layer.name:
            layer.name += "%d" % (len(self.layers) + 1)
        if not self.layers:
            layer.init(self.input_dim, no_prepare=no_setup)
        else:
            layer.init(self.layers[-1].output_dim, no_prepare=no_setup)
        self._output = layer.compute_tensor(self._output)
        self._test_output = layer.compute_tensor(self._test_output)
        self._hidden_outputs.append(self._output)
        self.register_layer(layer)
        self.layers.append(layer)

Response: Stack a neural layer.


Instruction:
Write a Python 3 function for
registering the given layer so that it s param will be trained.

Response:
def register_layer(self, layer):
        """
        Register the layer so that it's param will be trained.
        But the output of the layer will not be stacked.
        """
        if type(layer) == Block:
            layer.fix()
        self.parameter_count += layer.parameter_count
        self.parameters.extend(layer.parameters)
        self.free_parameters.extend(layer.free_parameters)
        self.training_monitors.extend(layer.training_monitors)
        self.testing_monitors.extend(layer.testing_monitors)
        self.updates.extend(layer.updates)
        self.training_updates.extend(layer.training_updates)
        self.input_variables.extend(layer.external_inputs)
        self.target_variables.extend(layer.external_targets)

        self.training_callbacks.extend(layer.training_callbacks)
        self.testing_callbacks.extend(layer.testing_callbacks)
        self.epoch_callbacks.extend(layer.epoch_callbacks)


Instruction:
Explain what the following Python 3 code does
def monitor_layer_outputs(self):
        for layer, hidden in zip(self.layers, self._hidden_outputs):
            self.training_monitors.append(('mean(%s)' % (layer.name), abs(hidden).mean()))

Response: Monitoring the outputs of each layer.


Instruction:
Can you create a Python 3 function that
computes the network output.

Response:
def compute(self, *x):
        """
        Return network output.
        """
        self._compile()
        outs = self._compute(*x)
        if self._output_keys:
            return MapDict(dict(zip(self._output_keys, outs)))
        else:
            return outs


Instruction:
Can you generate a brief explanation for the following Python 3 code
def save_params(self, path, new_thread=False):
        save_logger.info(path)
        param_variables = self.all_parameters
        params = [p.get_value().copy() for p in param_variables]
        if new_thread:
            thread = Thread(target=save_network_params, args=(params, path))
            thread.start()
        else:
            save_network_params(params, path)
        self.train_logger.save(path)

Response: Save parameters to file.


Instruction:
Here you have a function in Python 3, explain what it does
def load_params(self, path, exclude_free_params=False):
        if not os.path.exists(path): return;
        logging.info("loading parameters from %s" % path)
        # Decide which parameters to load
        if exclude_free_params:
            params_to_load = self.parameters
        else:
            params_to_load = self.all_parameters
        # Load parameters
        if path.endswith(".gz"):
            opener = gzip.open if path.lower().endswith('.gz') else open
            handle = opener(path, 'rb')
            saved_params = pickle.load(handle)
            handle.close()
            # Write parameters
            for target, source in zip(params_to_load, saved_params):
                logging.info('%s: setting value %s', target.name, source.shape)
                target.set_value(source)
        elif path.endswith(".npz"):
            arrs = np.load(path)
            # Write parameters
            for target, idx in zip(params_to_load, range(len(arrs.keys()))):
                source = arrs['arr_%d' % idx]
                logging.info('%s: setting value %s', target.name, source.shape)
                target.set_value(source)
        else:
            raise Exception("File format of %s is not supported, use '.gz' or '.npz' or '.uncompressed.gz'" % path)

        self.train_logger.load(path)

Response: Load parameters from file.


Instruction:
Can you tell what is the following Python 3 function doing
def init(self, input_dim=0, input_dims=None, no_prepare=False):
        if self.initialized:
            return
        # configure input dimensions
        if input_dims:
            self.input_dims = input_dims
            self.input_dim = input_dims[0]
        else:
            self.input_dim = input_dim
            self.input_dims = [input_dims]
        # set default output dimension
        if self.output_dim == 0:
            self.output_dim = self.input_dim
        self.initialized = True
        # call prepare
        if not no_prepare:
            self.prepare()
        return self

Response:
Initialize the layer.
        :param no_prepare: avoid calling preparation function


Instruction:
How would you explain what the following Python 3 function does
def compute(self, *inputs, **kwargs):
        from deepy.core.neural_var import NeuralVariable
        from deepy.core.graph import graph
        if type(inputs[0]) != NeuralVariable:
            raise SystemError("The input of `compute` must be NeuralVar")

        dims = [t.dim() for t in inputs]
        if len(inputs) == 1:
            self.init(input_dim=dims[0])
        else:
            self.init(input_dims=dims)
        # Check block
        if self.parameters and not self._linked_block:
            self.belongs_to(graph.default_block())
        # convert kwargs
        train_kwargs, _, _ = convert_to_theano_var(kwargs)

        output = self.compute_tensor(*[t.tensor for t in inputs], **train_kwargs)

        if type(output) != list and type(output) != tuple:
            return NeuralVariable(output, dim=self.output_dim)
        else:
            return [NeuralVariable(*item) for item in zip(output, self.output_dims)]

Response: Compute based on NeuralVariable.


Instruction:
Make a summary of the following Python 3 code
def belongs_to(self, block):
        if self._linked_block:
            raise SystemError("The layer {} has already blonged to {}".format(self.name, self._linked_block.name))
        self._linked_block = block
        block.register_layer(self)
        return self

Response: This method is used to make sure that this layer is blonged to the given block or network.


Instruction:
Can you generate the documentation for the following Python 3 function
def register_updates(self, *updates):
        for key, node in updates:
            if key not in self._registered_updates:
                self.updates.append((key, node))
                self._registered_updates.add(key)

Response: Register updates that will be executed in each iteration.


Instruction:
Explain what the following Python 3 code does
def register_training_updates(self, *updates):
        for key, node in updates:
            if key not in self._registered_training_updates:
                self.training_updates.append((key, node))
                self._registered_training_updates.add(key)

Response: Register updates that will only be executed in training phase.


Instruction:
Make a summary of the following Python 3 code
def register_monitors(self, *monitors):
        for key, node in monitors:
            if key not in self._registered_monitors:
                node *= 1.0 # Avoid CudaNdarray
                self.training_monitors.append((key, node))
                self.testing_monitors.append((key, node))
                self._registered_monitors.add(key)

Response: Register monitors they should be tuple of name and Theano variable.


Instruction:
Can you implement a function in Python 3 that
gets the L2 norm of multiple tensors.

Response:
def multiple_l2_norm(tensors):
    """
    Get the L2 norm of multiple tensors.
    This function is taken from blocks.
    """
    # Another way for doing this, I don't know which one is fast
    # return T.sqrt(sum(T.sum(t ** 2) for t in tensors))
    flattened = [T.as_tensor_variable(t).flatten() for t in tensors]
    flattened = [(t if t.ndim > 0 else t.dimshuffle('x'))
                 for t in flattened]
    joined = T.join(0, *flattened)
    return T.sqrt(T.sqr(joined).sum())


Instruction:
Can you generate a brief explanation for the following Python 3 code
def dump_one(elt_to_pickle, file_obj):
        pickled_elt_str = dumps(elt_to_pickle)
        file_obj.write(pickled_elt_str)
        # record separator is a blank line
        # (since pickled_elt_str might contain its own newlines)
        file_obj.write('\n\n')

Response: Dumps one element to file_obj


Instruction:
Can you write a function in Python 3 where it
loads contents from file_obj returning a generator that yields one element at a time

Response:
def load(file_obj):
        """
        load contents from file_obj, returning a generator that yields one
        element at a time
        """
        cur_elt = []
        for line in file_obj:
          cur_elt.append(line)

          if line == '\n':
            pickled_elt_str = ''.join(cur_elt)
            cur_elt = []
            try:
                elt = loads(pickled_elt_str)
            except ValueError:
                continue

            yield elt


Instruction:
How would you code a function in Python 3 to
fix the block, register all the parameters of sub layers. :return:

Response:
def fix(self):
        """
        Fix the block, register all the parameters of sub layers.
        :return:
        """
        if not self.fixed:
            for layer in self.layers:
                if not layer.initialized:
                    raise Exception("All sub layers in a block must be initialized when fixing it.")
                self.register_inner_layers(layer)
            self.fixed = True


Instruction:
Implement a function in Python 3 to
register one connected layer.

Response:
def register_layer(self, layer):
        """
        Register one connected layer.
        :type layer: NeuralLayer
        """
        if self.fixed:
            raise Exception("After a block is fixed, no more layers can be registered.")
        self.layers.append(layer)


Instruction:
Write a Python 3 function for
loading parameters to the block.

Response:
def load_params(self, path, exclude_free_params=False):
        from deepy.core import graph
        """
        Load parameters to the block.
        """
        from deepy.core.comp_graph import ComputationalGraph
        model = graph.compile(blocks=[self])
        model.load_params(path, exclude_free_params=exclude_free_params)


Instruction:
How would you explain what the following Python 3 function does
def compute_step(self, state, lstm_cell=None, input=None, additional_inputs=None):
        if not self.initialized:
            input_dim = None
            if input and hasattr(input.tag, 'last_dim'):
                input_dim = input.tag.last_dim
            self.init(input_dim)

        input_map = self.merge_inputs(input, additional_inputs=additional_inputs)
        input_map.update({"state": state, "lstm_cell": lstm_cell})
        output_map = self.compute_new_state(input_map)
        outputs = [output_map.pop("state")]
        outputs += output_map.values()
        for tensor in outputs:
            tensor.tag.last_dim = self.hidden_size
        if len(outputs) == 1:
            return outputs[0]
        else:
            return outputs

Response: Compute one step in the RNN and GRU.


Instruction:
Implement a Python 3 function for
getting initial states for the cluster.

Response:
def get_initial_states(self, input_var, init_state=None):
        """
        :type input_var: T.var
        :rtype: dict
        """
        initial_states = {}
        for state in self.state_names:
            if state != "state" or not init_state:
                if self._input_type == 'sequence' and input_var.ndim == 2:
                    init_state = T.alloc(np.cast[env.FLOATX](0.), self.hidden_size)
                else:
                    init_state = T.alloc(np.cast[env.FLOATX](0.), input_var.shape[0], self.hidden_size)
            initial_states[state] = init_state
        return initial_states


Instruction:
Write a Python 3 function that can
return a dictionary of step inputs for the given input variable.

Response:
def get_step_inputs(self, input_var, states=None, mask=None, additional_inputs=None):
        """
        :type input_var: T.var
        :rtype: dict
        """
        step_inputs = {}
        if self._input_type == "sequence":
            if not additional_inputs:
                additional_inputs = []
            if mask:
                step_inputs['mask'] = mask.dimshuffle(1, 0)
            step_inputs.update(self.merge_inputs(input_var, additional_inputs=additional_inputs))
        else:
            # step_inputs["mask"] = mask.dimshuffle((1,0)) if mask else None
            if additional_inputs:
                step_inputs.update(self.merge_inputs(None, additional_inputs=additional_inputs))
        if states:
            for name in self.state_names:
                step_inputs[name] = states[name]

        return step_inputs


Instruction:
Write a Python 3 function for
converting a NeuralVariable or list or dict to theano vars.

Response:
def convert_to_theano_var(obj):
    """
    Convert neural vars to theano vars.
    :param obj: NeuralVariable or list or dict or tuple
    :return: theano var, test var, tensor found, neural var found
    """
    from deepy.core.neural_var import NeuralVariable
    if type(obj) == tuple:
        return tuple(convert_to_theano_var(list(obj)))
    if type(obj) == list:
        unpacked_list = map(convert_to_theano_var, obj)
        normal_list = []
        test_list = []
        theano_var_found = False
        neural_var_found = False
        for normal_var, tensor_found, neural_found in unpacked_list:
            normal_list.append(normal_var)
            if tensor_found: theano_var_found = True
            if neural_found: neural_var_found = True
        return normal_list, theano_var_found, neural_var_found
    elif type(obj) == dict:
        normal_map = {}
        theano_var_found = False
        neural_var_found = False
        for key in obj:
            normal_var, tensor_found, neural_found = convert_to_theano_var(obj[key])
            normal_map[key] = normal_var
            if tensor_found: theano_var_found = True
            if neural_found: neural_var_found = True
        return normal_map, theano_var_found, neural_var_found
    elif type(obj) == MapDict:
        normal_map = {}
        theano_var_found = False
        neural_var_found = False
        for key in obj:
            normal_var, tensor_found, neural_found = convert_to_theano_var(obj[key])
            normal_map[key] = normal_var
            if tensor_found: theano_var_found = True
            if neural_found: neural_var_found = True
        return MapDict(normal_map), theano_var_found, neural_var_found
    elif type(obj) == NeuralVariable:
        theano_tensor = obj.tensor
        theano_tensor.tag.last_dim = obj.dim()
        return theano_tensor, False, True
    elif type(obj) == TensorVariable:
        return obj, True, False
    elif type(obj) == slice:
        normal_args = []
        theano_var_found = False
        neural_var_found = False
        for arg in [obj.start, obj.stop, obj.step]:
            normal_var, tensor_found, neural_found = convert_to_theano_var(arg)
            normal_args.append(normal_var)
            if tensor_found: theano_var_found = True
            if neural_found: neural_var_found = True
        return slice(*normal_args), theano_var_found, neural_var_found
    else:
        return obj, False, False


Instruction:
Can you implement a function in Python 3 that
converts object and a test object into neural var.

Response:
def convert_to_neural_var(obj):
    """
    Convert object and a test object into neural var.
    :param obj: tensor or list or dict or tuple
    :param test_obj: NeuralVar or list or dict or tuple
    :return:
    """
    from theano.tensor.var import TensorVariable
    from deepy.core.neural_var import NeuralVariable
    if type(obj) == list:
        return [convert_to_neural_var(item) for item in obj]
    elif type(obj) == tuple:
        return tuple(convert_to_neural_var(list(obj)))
    elif type(obj) == dict:
        merged_map = {}
        for key in obj:
            merged_map[key] = convert_to_neural_var(obj[key])
        return merged_map
    elif type(obj) == MapDict:
        merged_map = {}
        for key in obj:
            merged_map[key] = convert_to_neural_var(obj[key])
        return MapDict(merged_map)
    elif type(obj) == TensorVariable:
        deepy_var = NeuralVariable(obj)
        if hasattr(obj, 'tag') and hasattr(obj.tag, 'last_dim'):
            deepy_var.output_dim = obj.tag.last_dim
        return deepy_var
    else:
        return obj


Instruction:
Can you tell what is the following Python 3 function doing
def neural_computation(original_func, prefer_tensor=False):

    def wrapper(*args, **kwargs):
        normal_args, tensor_found_in_args, neural_found_in_args = convert_to_theano_var(args)
        normal_kwargs, tensor_found_in_kwargs, neural_found_in_kwargs = convert_to_theano_var(kwargs)

        tensor_found = tensor_found_in_args or tensor_found_in_kwargs
        neural_found = neural_found_in_args or neural_found_in_kwargs

        if tensor_found and neural_found:
            raise Exception("Theano tensor variables can not be used together with neural variables.")

        normal_result = original_func(*normal_args, **normal_kwargs)

        if tensor_found or (not neural_found and prefer_tensor):
            # No neural variables are inputted, so output tensors
            return normal_result
        else:
            # Output neural variables, auto set output_dim
            result_var = convert_to_neural_var(normal_result)
            if (isinstance(normal_result, TensorVariable) and
                    hasattr(normal_result.tag, "test_value") and
                    hasattr(normal_result.tag.test_value, "shape") and
                    normal_result.tag.test_value.shape):
                result_var.output_dim = normal_result.tag.test_value.shape[-1]
            return result_var
    return wrapper

Response: Decorator for theano - based neural computation of a single neural variable.


Instruction:
How would you explain what the following Python 3 function does
def onehot_tensor(i_matrix, vocab_size):
    dim0, dim1 = i_matrix.shape
    i_vector = i_matrix.reshape((-1,))
    hot_matrix = T.extra_ops.to_one_hot(i_vector, vocab_size).reshape((dim0, dim1, vocab_size))
    return hot_matrix

Response: One - hot tensor for a batch x time series.