Instruction:
Given the following Python 3 function, write the documentation
def new_session(self, session):
        '''
        Clear out the current session on the remote and setup a new one.

        :return: A response from having expired the current session.
        :rtype: requests.Response
        '''
        response = self.__get('/Home/SessionExpire')
        self.session = update_session_headers(session)

        return response

Response: Clear out the current session on the remote and setup a new one.


Instruction:
Given the following Python 3 function, write the documentation
def reset_store(self):
        '''
        Clears out the current store and gets a cookie. Set the cross site
        request forgery token for each subsequent request.

        :return: A response having cleared the current store.
        :rtype: requests.Response
        '''
        response = self.__get('/Store/Reset')

        token = self.session.cookies['XSRF-TOKEN']
        self.session.headers.update({'X-XSRF-TOKEN': token})

        return response

Response: Clears out the current store and gets a cookie.


Instruction:
Make a summary of the following Python 3 code
def get_stores(self, search_term):
        '''
        Search for dominos pizza stores using a search term.

        :param string search: Search term.
        :return: A list of nearby stores matching the search term.
        :rtype: list
        '''
        params = {'SearchText': search_term}
        response = self.__get('/storefindermap/storesearch', params=params)

        return Stores(response.json())

Response: Search for dominos pizza stores using a search term.


Instruction:
Implement a Python 3 function for
setting the delivery system on the remote server.

Response:
def set_delivery_system(self, store, postcode, fulfilment_method=FULFILMENT_METHOD.DELIVERY):
        '''
        Set local cookies by initialising the delivery system on the remote.
        Requires a store ID and a delivery postcode.

        :param Store store: Store id.
        :param string postcode: A postcode.
        :return: A response having initialised the delivery system.
        :rtype: requests.Response
        '''
        method = 'delivery' if fulfilment_method == FULFILMENT_METHOD.DELIVERY else 'collection'

        params = {
            'fulfilmentMethod': method,
            'postcode': postcode,
            'storeid': store.store_id
        }

        return self.__post('/Journey/Initialize', json=params)


Instruction:
Implement a Python 3 function for
retrieving the menu from the selected store.

Response:
def get_menu(self, store):
        '''
        Retrieve the menu from the selected store.

        :param Store store: A store.
        :return: The store menu.
        :rtype: Menu
        '''
        params = {
            'collectionOnly': not store.delivery_available,
            'menuVersion': store.menu_version,
            'storeId': store.store_id,
        }

        response = self.__get('/ProductCatalog/GetStoreCatalog', params=params)
        return Menu(response.json())


Instruction:
How would you explain what the following Python 3 function does
def add_item_to_basket(self, item, variant=VARIANT.MEDIUM, quantity=1):
        '''
        Add an item to the current basket.

        :param Item item: Item from menu.
        :param int variant: Item SKU id. Ignored if the item is a side.
        :param int quantity: The quantity of item to be added.
        :return: A response having added an item to the current basket.
        :rtype: requests.Response
        '''
        item_type = item.type

        if item_type == 'Pizza':
            return self.add_pizza_to_basket(item, variant, quantity)
        elif item_type == 'Side':
            return self.add_side_to_basket(item, quantity)
        return None

Response: Add an item to the current basket.


Instruction:
How would you implement a function in Python 3 that
adds a pizza to the current basket.

Response:
def add_pizza_to_basket(self, item, variant=VARIANT.MEDIUM, quantity=1):
        '''
        Add a pizza to the current basket.

        :param Item item: Item from menu.
        :param int variant: Item SKU id. Some defaults are defined in the VARIANT enum.
        :param int quantity: The quantity of pizza to be added.
        :return: A response having added a pizza to the current basket.
        :rtype: requests.Response
        '''
        item_variant = item[variant]
        ingredients = item_variant['ingredients'].update([36, 42])

        params = {
            'stepId': 0,
            'quantity': quantity,
            'sizeId': variant,
            'productId': item.item_id,
            'ingredients': ingredients,
            'productIdHalfTwo': 0,
            'ingredientsHalfTwo': [],
            'recipeReferrer': 0
        }

        return self.__post('/Basket/AddPizza', json=params)


Instruction:
Given the following Python 3 function, write the documentation
def add_side_to_basket(self, item, quantity=1):
        '''
        Add a side to the current basket.

        :param Item item: Item from menu.
        :param int quantity: The quantity of side to be added.
        :return: A response having added a side to the current basket.
        :rtype: requests.Response
        '''
        item_variant = item[VARIANT.PERSONAL]

        params = {
            'productSkuId': item_variant['productSkuId'],
            'quantity': quantity,
            'ComplimentaryItems': []
        }

        return self.__post('/Basket/AddProduct', json=params)

Response: Add a side to the current basket.


Instruction:
How would you code a function in Python 3 to
remove an item from the current basket.

Response:
def remove_item_from_basket(self, idx):
        '''
        Remove an item from the current basket.

        :param int idx: Basket item id.
        :return: A response having removed an item from the current basket.
        :rtype: requests.Response
        '''
        params = {
            'basketItemId': idx,
            'wizardItemDelete': False
        }

        return self.__post('/Basket/RemoveBasketItem', json=params)


Instruction:
Implement a Python 3 function for
setting the payment method of the current node.

Response:
def set_payment_method(self, method=PAYMENT_METHOD.CASH_ON_DELIVERY):
        '''
        Select the payment method going to be used to make a purchase.

        :param int method: Payment method id.
        :return: A response having set the payment option.
        :rtype: requests.Response
        '''
        params = {'paymentMethod': method}
        return self.__post('/PaymentOptions/SetPaymentMethod', json=params)


Instruction:
Can you create a Python 3 function that
proceeds with payment using the payment method selected earlier. :return: A response having processes the payment. :rtype: requests.Response

Response:
def process_payment(self):
        '''
        Proceed with payment using the payment method selected earlier.

        :return: A response having processes the payment.
        :rtype: requests.Response
        '''
        params = {
            '__RequestVerificationToken': self.session.cookies,
            'method': 'submit'
        }

        return self.__post('/PaymentOptions/Proceed', json=params)


Instruction:
Write a Python 3 function that can
make a HTTP GET request to the Dominos UK API with the given parameters

Response:
def __get(self, path, **kargs):
        '''
        Make a HTTP GET request to the Dominos UK API with the given parameters
        for the current session.

        :param string path: The API endpoint path.
        :params list kargs: A list of arguments.
        :return: A response from the Dominos UK API.
        :rtype: response.Response
        '''
        return self.__call_api(self.session.get, path, **kargs)


Instruction:
Write a Python 3 script to
make a HTTP POST request to the Dominos UK API with the given path and parameters.

Response:
def __post(self, path, **kargs):
        '''
        Make a HTTP POST request to the Dominos UK API with the given
        parameters for the current session.

        :param string path: The API endpoint path.
        :params list kargs: A list of arguments.
        :return: A response from the Dominos UK API.
        :rtype: response.Response
        '''
        return self.__call_api(self.session.post, path, **kargs)


Instruction:
Write a Python 3 function for
making a HTTP request to the Dominos UK API with the given parameters for the current session.

Response:
def __call_api(self, verb, path, **kargs):
        '''
        Make a HTTP request to the Dominos UK API with the given parameters for
        the current session.

        :param verb func: HTTP method on the session.
        :param string path: The API endpoint path.
        :params list kargs: A list of arguments.
        :return: A response from the Dominos UK API.
        :rtype: response.Response
        '''
        response = verb(self.__url(path), **kargs)

        if response.status_code != 200:
            raise ApiError('{}: {}'.format(response.status_code, response))

        return response


Instruction:
Here you have a function in Python 3, explain what it does
def append_item(self, item):
        did_remove = self.remove_exit()
        item.menu = self
        self.items.append(item)
        if did_remove:
            self.add_exit()
        if self.screen:
            max_row, max_cols = self.screen.getmaxyx()
            if max_row < 6 + len(self.items):
                self.screen.resize(6 + len(self.items), max_cols)
            self.draw()

Response: Add an item to the end of the menu before the exit


Instruction:
Given the following Python 3 function, write the documentation
def add_exit(self):
        if self.items:
            if self.items[-1] is not self.exit_item:
                self.items.append(self.exit_item)
                return True
        return False

Response: Add the exit item if necessary. Used to make sure there aren t multiple exit items.


Instruction:
Write a Python 3 script to
redraw the menu and refreshes the screen. Should be called whenever something changes that needs to be redrawn.

Response:
def draw(self):
        """
        Redraws the menu and refreshes the screen. Should be called whenever something changes that needs to be redrawn.
        """

        self.screen.border(0)
        if self.title is not None:
            self.screen.addstr(2, 2, self.title, curses.A_STANDOUT)
        if self.subtitle is not None:
            self.screen.addstr(4, 2, self.subtitle, curses.A_BOLD)

        for index, item in enumerate(self.items):
            if self.current_option == index:
                text_style = self.highlight
            else:
                text_style = self.normal
            self.screen.addstr(5 + index, 4, item.show(index), text_style)

        screen_rows, screen_cols = CursesMenu.stdscr.getmaxyx()
        top_row = 0
        if 6 + len(self.items) > screen_rows:
            if screen_rows + self.current_option < 6 + len(self.items):
                top_row = self.current_option
            else:
                top_row = 6 + len(self.items) - screen_rows

        self.screen.refresh(top_row, 0, 0, 0, screen_rows - 1, screen_cols - 1)


Instruction:
Here you have a function in Python 3, explain what it does
def process_user_input(self):
        user_input = self.get_input()

        go_to_max = ord("9") if len(self.items) >= 9 else ord(str(len(self.items)))

        if ord('1') <= user_input <= go_to_max:
            self.go_to(user_input - ord('0') - 1)
        elif user_input == curses.KEY_DOWN:
            self.go_down()
        elif user_input == curses.KEY_UP:
            self.go_up()
        elif user_input == ord("\n"):
            self.select()

        return user_input

Response: Process the user input and decides what to do with it


Instruction:
Explain what the following Python 3 code does
def select(self):
        self.selected_option = self.current_option
        self.selected_item.set_up()
        self.selected_item.action()
        self.selected_item.clean_up()
        self.returned_value = self.selected_item.get_return()
        self.should_exit = self.selected_item.should_exit

        if not self.should_exit:
            self.draw()

Response: Select the current item and run it


Instruction:
Can you generate a brief explanation for the following Python 3 code
def action(self):
        commandline = "{0} {1}".format(self.command, " ".join(self.arguments))
        try:
            completed_process = subprocess.run(commandline, shell=True)
            self.exit_status = completed_process.returncode
        except AttributeError:
            self.exit_status = subprocess.call(commandline, shell=True)

Response: This method is used to run a command and set the exit_status attribute of the object.


Instruction:
Can you tell what is the following Python 3 function doing
def parse_old_menu(menu_data):
    menu_title = menu_data['title']
    menu = CursesMenu(menu_title)
    for item in menu_data["options"]:
        item_type = item["type"]
        item_title = item["title"]
        if item_type == menuItem.COMMAND:
            item_command = item["command"]
            menu.append_item(CommandItem(item_title, item_command, menu))
        elif item_type == menuItem.FUNCTION:
            item_function = item["function"]
            menu.append_item(FunctionItem(item_title, item_function, menu))
        elif item_type == menuItem.EXITMENU:
            menu.append_item(ExitItem(item_title, menu))
        elif item_type == menuItem.NUMBER:
            menu.append_item(SelectionItem(item_title, menu))
        elif item_type == menuItem.MENU:
            new_menu = parse_old_menu(item)
            menu.append_item(SubmenuItem(item_title, menu, new_menu))

    return menu

Response: Takes an old - style menuData dictionary and returns a CursesMenu object.


Instruction:
Make a summary of the following Python 3 code
def set_up(self):
        self.menu.pause()
        curses.def_prog_mode()
        self.menu.clear_screen()

Response: This method is called by the base class to set up the curses screen.


Instruction:
Make a summary of the following Python 3 code
def add_aggregation_columns(
        df, *,
        group_cols: Union[str, List[str]],
        aggregations: Dict[str, Agg]
):
    """
    Add new columns containing aggregations values on existing columns

    ---

    ### Parameters

    *mandatory :*
    - `group_cols` (*str* or *list*): columns used to aggregate the data
    - `aggregations` (*dict*): keys are name of new columns and values are aggregation functions
       Examples of aggregation functions : 'sum', 'max'
       Available aggregation functions are listed [here](
       https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html#aggregation)

    ---

    ### Example

    **Input**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 |
    |:------:|:----:|:-------:|:-------:|
    |    A   | 2017 |    10   |    3    |
    |    A   | 2017 |    20   |    1    |
    |    A   | 2018 |    10   |    5    |
    |    A   | 2018 |    30   |    4    |
    |    B   | 2017 |    60   |    4    |
    |    B   | 2017 |    40   |    3    |
    |    B   | 2018 |    50   |    7    |
    |    B   | 2018 |    60   |    6    |

    ```cson
    add_aggregation_columns:
      group_cols: ['ENTITY', 'YEAR']
      aggregations:
        sum_value1:
          VALUE_1: 'sum'  # sum of `VALUE_1` put in `sum_value1` column
        max_value1:
          VALUE_1: 'max'  # max of `VALUE_1` put in `max_value1` column
        mean_value2:
          VALUE_2: 'mean'  # mean of `VALUE_2` put in `mean_value2` column
    ]
    ```

    **Output**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 | sum_value1 | max_value1 | mean_value2 |
    |:------:|:----:|:-------:|:-------:|:----------:|:----------:|:-----------:|
    |    A   | 2017 |    10   |    3    |     30     |     20     |     2.0     |
    |    A   | 2017 |    20   |    1    |     30     |     20     |     2.0     |
    |    A   | 2018 |    10   |    5    |     40     |     30     |     4.5     |
    |    A   | 2018 |    30   |    4    |     40     |     30     |     4.5     |
    |    B   | 2017 |    60   |    4    |    100     |     60     |     3.5     |
    |    B   | 2017 |    40   |    3    |    100     |     60     |     3.5     |
    |    B   | 2018 |    50   |    7    |    110     |     60     |     6.5     |
    |    B   | 2018 |    60   |    6    |    110     |     60     |     6.5     |

    """
    group = df.groupby(group_cols)
    for new_col, aggs in aggregations.items():
        assert len(aggs) == 1
        (col, agg), *_ = aggs.items()
        df[new_col] = group[col].transform(agg)
    return df

Response: Adds new columns containing aggregations values on existing columns


Instruction:
Here you have a function in Python 3, explain what it does
def top(
        df,
        value: str,
        limit: int,
        order: str = 'asc',
        group: Union[str, List[str]] = None
):
    """
    Get the top or flop N results based on a column value for each specified group columns

    ---

    ### Parameters

    *mandatory :*
    - `value` (*str*): column name on which you will rank the results
    - `limit` (*int*): Number to specify the N results you want to retrieve.
        Use a positive number x to retrieve the first x results.
        Use a negative number -x to retrieve the last x results.

    *optional :*
    - `order` (*str*): `"asc"` or `"desc"` to sort by ascending ou descending order. By default : `"asc"`.
    - `group` (*str*, *list of str*): name(s) of columns on which you want to perform the group operation.

    ---

    ### Example

    **Input**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   lili   |    1     |  50  |
    |   lili   |    1     |  20  |
    |   toto   |    1     |  100  |
    |   toto   |    1     |  200  |
    |   toto   |    1     |  300  |
    |   lala   |    1     |  100  |
    |   lala   |    1     |  150  |
    |   lala   |    1     |  250  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |

    ```cson
    top:
      value: 'value'
      limit: 4
      order: 'asc'
    ```

    **Output**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   lala   |    1     |  250  |
    |   toto   |    1     |  300  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |
    """
    ascending = order != 'desc'
    limit = int(limit)
    filter_func = 'nlargest' if (limit > 0) ^ ascending else 'nsmallest'

    def _top(df):
        return getattr(df, filter_func)(abs(limit), value).sort_values(by=value,
                                                                       ascending=ascending)

    if group is None:
        df = _top(df)
    else:
        df = df.groupby(group).apply(_top)

    return df

Response: Return the top or flop N results for a specified column value


Instruction:
Make a summary of the following Python 3 code
def top_group(
        df,
        aggregate_by: List[str],
        value: str,
        limit: int,
        order: str = 'asc',
        function: str = 'sum',
        group: Union[str, List[str]] = None
):
    """
    Get the top or flop N results based on a function and a column value that agregates the input.
    The result is composed by all the original lines including only lines corresponding
    to the top groups

    ---

    ### Parameters

    *mandatory :*
    - `value` (*str*): Name of the column name on which you will rank the results.
    - `limit` (*int*): Number to specify the N results you want to retrieve from the sorted values.
        - Use a positive number x to retrieve the first x results.
        - Use a negative number -x to retrieve the last x results.
    - `aggregate_by` (*list of str*)): name(s) of columns you want to aggregate

    *optional :*
    - `order` (*str*): `"asc"` or `"desc"` to sort by ascending ou descending order. By default : `"asc"`.
    - `group` (*str*, *list of str*): name(s) of columns on which you want to perform the group operation.
    - `function` : Function to use to group over the group column

    ---

    ### Example

    **Input**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   lili   |    1     |  50  |
    |   lili   |    1     |  20  |
    |   toto   |    1     |  100  |
    |   toto   |    1     |  200  |
    |   toto   |    1     |  300  |
    |   lala   |    1     |  100  |
    |   lala   |    1     |  150  |
    |   lala   |    1     |  250  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |

    ```cson
    top_group:
      group: ["Category"]
      value: 'value'
      aggregate_by: ["variable"]
      limit: 2
      order: "desc"
    ```

    **Output**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   toto   |    1     |  100  |
    |   toto   |    1     |  200  |
    |   toto   |    1     |  300  |
    |   lala   |    1     |  100  |
    |   lala   |    1     |  150  |
    |   lala   |    1     |  250  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |
    """
    aggregate_by = aggregate_by or []
    group_top = group or []
    df2 = df.groupby(group_top + aggregate_by).agg(function).reset_index()
    df2 = top(df2, group=group, value=value, limit=limit, order=order).reset_index(drop=True)
    df2 = df2[group_top + aggregate_by]
    df = df2.merge(df, on=group_top + aggregate_by)

    return df

Response: Returns the top group of the result of a function on the given column value.


Instruction:
Can you write a function in Python 3 where it
converts string column into datetime column---

Response:
def convert_str_to_datetime(df, *, column: str, format: str):
    """
    Convert string column into datetime column

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to format
    - `format` (*str*): current format of the values (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))
    """
    df[column] = pd.to_datetime(df[column], format=format)
    return df


Instruction:
Can you write a function in Python 3 where it
converts datetime column into string

Response:
def convert_datetime_to_str(df, *, column: str, format: str, new_column: str = None):
    """
    Convert datetime column into string column

    ---

    ### Parameters

    *mandatory :*
    - column (*str*): name of the column to format
    - format (*str*): format of the result values (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))

    *optional :*
    - new_column (*str*): name of the output column. By default `column` is overwritten.
    """
    new_column = new_column or column
    df[new_column] = df[column].dt.strftime(format)
    return df


Instruction:
How would you explain what the following Python 3 function does
def change_date_format(
        df, *,
        column: str,
        output_format: str,
        input_format: str = None,
        new_column: str = None,
        new_time_zone=None
):
    """
    Convert the format of a date

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to change the format
    - `output_format` (*str*): format of the output values (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))

    *optional :*
    - `input_format` (*str*): format of the input values (by default let the parser detect it)
    - `new_column` (*str*): name of the output column  (by default overwrite `column`)
    - `new_time_zone` (*str*): name of new time zone (by default no time zone conversion is done)

    ---

    ### Example

    **Input**

    label   | date
    :------:|:----:
    France  | 2017-03-22
    Europe  | 2016-03-22

    ```cson
    change_date_format:
      column: 'date'
      input_format: '%Y-%m-%d'
      output_format: '%Y-%m'
    ```

    Output :

    label   | date
    :------:|:----:
    France  | 2017-03
    Europe  | 2016-03
    """
    new_column = new_column or column
    df[new_column] = (pd.to_datetime(df[column], format=input_format, utc=True)
                      .dt.tz_convert(new_time_zone)
                      .dt.strftime(output_format))
    return df

Response: Change the format of a date column in a DataFrame


Instruction:
Write a Python 3 script for
converting column s type into type

Response:
def cast(df, column: str, type: str, new_column=None):
    """
    Convert column's type into type

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to convert
    - `type` (*str*): output type. It can be :
        - `"int"` : integer type
        - `"float"` : general number type
        - `"str"` : text type

    *optional :*
    - `new_column` (*str*): name of the output column.
       By default the `column` arguments is modified.

    ---

    ### Example

    **Input**

    | Column 1 |  Column 2   |  Column 3  |
    |:-------:|:--------:|:--------:|
    |  'one'  |  '2014'  |   30.0   |
    |  'two'  |  2015.0  |    '1'   |
    |   3.1   |   2016   |    450   |

    ```cson
    postprocess: [
      cast:
        column: 'Column 1'
        type: 'str'
      cast:
        column: 'Column 2'
        type: 'int'
      cast:
        column: 'Column 3'
        type: 'float'
    ]
    ```

    **Output**

    | Column 1 |  Column 2  |  Column 3  |
    |:-------:|:------:|:--------:|
    |  'one'  |  2014  |   30.0   |
    |  'two'  |  2015  |    1.0   |
    |  '3.1'  |  2016  |  450.0   |
    """
    new_column = new_column or column
    df[new_column] = df[column].astype(type)
    return df


Instruction:
Make a summary of the following Python 3 code
def compute_evolution_by_frequency(
    df,
    id_cols: List[str],
    date_col: Union[str, Dict[str, str]],
    value_col: str,
    freq=1,
    method: str = 'abs',
    format: str = 'column',
    offseted_suffix: str = '_offseted',
    evolution_col_name: str = 'evolution_computed',
    missing_date_as_zero: bool = False,
    raise_duplicate_error: bool = True
):
    """
    This function answers the question: how has a value changed on a weekly, monthly, yearly basis ?

    ---

    ### Parameters

    *mandatory :*
    - `id_cols` (*list*): name of the columns used to create each group.
    - `date_col` (*str or dict*): either directly the name of the column containing the date or a dictionary with:
      - `selector` (*str*): the name of the column
      - `format` (*str*): the format of the date (see [pandas doc](
        https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))
    - `value_col` (*str*): name of the column containing the value to compare.

    *optional :*
    - `freq` (*int/pd.DateOffset/pd.Serie/dict*): the frequency at which we calculate evolutions
    - `method` (*str*): either `"abs"` for absolute values or `"pct"` for the evolution in percentage of previous value.
    - `offseted_suffix` (*str*): suffix of the offseted column. By default, `"_offseted"`.
    - `evolution_col_name` (*str*): name given to the evolution column. By default, `"evolution_computed"`.
    - `missing_date_as_zero` (*boolean*): add missing date with zero value.
    - `raise_duplicate_error` (*boolean*): raise an error when the dataset has duplicated values with the given `id_cols`.
    - `format` (*str*): `'df'` # Do not change it !!!

    ---

    ### Example

    **Input**

    |   id_cols |    value_col |    date_col|
    |:---------:|:------------:|:----------:|
    |         A |           20 |        2010|
    |           |            7 |        2011|
    |         B |          200 |        2010|
    |           |          220 |        2011|
    |         C |          100 |        2011|

    ```cson
    compute_evolution_by_frequency:
      id_cols: "id_cols"
      date_col: "date_col"
      value_col: "value_col"
    ```

    **Output**

    |   id_cols |    value_col |    date_col|  evolution|
    |:---------:|:------------:|:----------:|:---------:|
    |         A |           20 |        2010|       null|
    |           |            7 |        2011|        -13|
    |         B |          200 |        2010|       null|
    |           |          220 |        2011|         20|
    |         C |          100 |        2011|       null|
    """
    if missing_date_as_zero:
        how = 'outer'
        fillna = 0
    else:
        how = 'left'
        fillna = None

    return __compute_evolution(
        df=df,
        id_cols=id_cols,
        value_col=value_col,
        date_col=date_col,
        freq=freq,
        method=method,
        format=format,
        offseted_suffix=offseted_suffix,
        evolution_col_name=evolution_col_name,
        how=how,
        fillna=fillna,
        raise_duplicate_error=raise_duplicate_error
    )

Response: This function calculates the evolution of a group by frequency.


Instruction:
Make a summary of the following Python 3 code
def __compute_evolution(
    df,
    id_cols,
    value_col,
    date_col=None,
    freq=1,
    compare_to=None,
    method='abs',
    format='column',
    offseted_suffix='_offseted',
    evolution_col_name='evolution_computed',
    how='left',
    fillna=None,
    raise_duplicate_error=True
):
    """
    Compute an evolution column :
        - against a period distant from a fixed frequency.
        - against a part of the df

    Unfortunately, pandas doesn't allow .change() and .pct_change() to be
    executed with a MultiIndex.

    Args:
        df (pd.DataFrame):
        id_cols (list(str)):
        value_col (str):
        date_col (str/dict): default None
        freq (int/pd.DateOffset/pd.Serie): default 1
        compare_to (str): default None
        method (str): default ``'abs'`` can be also ``'pct'``
        format(str): default 'column' can be also 'df'
        offseted_suffix(str): default '_offseted'
        evolution_col_name(str): default 'evolution_computed'
        how(str): default 'left'
        fillna(str/int): default None
    """
    if date_col is not None:
        is_date_to_format = isinstance(date_col, dict) or (df[date_col].dtype == np.object)
        if is_date_to_format:
            if isinstance(date_col, dict):
                date_format = date_col.get('format', None)
                date_col = date_col['selector']
            else:
                date_format = None
            df['_'+date_col + '_copy_'] = pd.to_datetime(df[date_col], format=date_format)
            date_col = '_'+date_col + '_copy_'

        is_freq_dict = isinstance(freq, dict)
        if is_freq_dict:
            freq = pd.DateOffset(**{k: int(v) for k, v in freq.items()})

        check_params_columns_duplicate(id_cols + [value_col, date_col])
        # create df_offseted
        group_cols = id_cols + [date_col]
        df_offseted = df[group_cols + [value_col]].copy()
        df_offseted[date_col] += freq

        df_with_offseted_values = apply_merge(
            df, df_offseted, group_cols, how, offseted_suffix,
            raise_duplicate_error
        )
        if is_date_to_format:
            del df_with_offseted_values[date_col]

    elif compare_to is not None:
        # create df_offseted
        check_params_columns_duplicate(id_cols + [value_col])
        group_cols = id_cols
        df_offseted = df.query(compare_to).copy()
        df_offseted = df_offseted[group_cols + [value_col]]

        df_with_offseted_values = apply_merge(
            df, df_offseted, group_cols, how, offseted_suffix,
            raise_duplicate_error
        )

    apply_fillna(df_with_offseted_values, value_col, offseted_suffix, fillna)
    apply_method(df_with_offseted_values, evolution_col_name, value_col, offseted_suffix, method)
    return apply_format(df_with_offseted_values, evolution_col_name, format)

Response: Compute an evolution column for a single entry in the log.


Instruction:
Given the following Python 3 function, write the documentation
def rank(
        df,
        value_cols: Union[str, List[str]],
        group_cols: List[str] = None,
        rank_cols_names: List[str] = None,
        method='min',
        ascending: bool = True
):
    """
    This function creates rank columns based on numeric values to be ranked.

    ---

    ### Parameters

    *mandatory :*
    - `value_cols` (*list*): name(s) of the columns used

    *optional :*
    - `group_cols` (*list*): name(s) of the column(s) used to
      create each group inside which independent ranking needs to be applied
    - `rank_cols_names` (*list*): the names of the added ranking columns.
      If not filled, the ranking will be named after the value_cols with a '_rank' suffix
    - `method` (*str*): method to use when encountering equal values:
        - `'min'` (default): lowest rank in group
        - `'max'`: highest rank in group
        - `'average'`: average rank of group
        - `'first'`: ranks assigned in order the values appear in the series
        - `'dense'`: like 'min', but rank always increases by 1 between groups
    - `ascending` (*boolean*): whether the rank should be determined based on
       ascending (default) or descending order

    ---

    ### Example

    **Input**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 |
    | :---: | :---: | :---: | :---: |
    | A | 2017 | 10 | 3 |
    | A | 2017 | 20 | 1 |
    | A | 2018 | 10 | 5 |
    | A | 2018 | 30 | 4 |
    | B | 2017 | 60 | 4 |
    | B | 2017 | 40 | 3 |
    | B | 2018 | 50 | 7 |
    | B | 2018 | 50 | 6 |

    ```cson
    rank :
      value_cols: 'VALUE_1'
    ```

    **Output**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 | VALUE_1_rank
    | :---: | :---: | :---: | :---: | :---: |
    | A | 2017 | 10 | 3 | 1 |
    | A | 2017 | 20 | 1 | 3 |
    | A | 2018 | 10 | 5 | 1 |
    | A | 2018 | 30 | 4 | 4 |
    | B | 2017 | 60 | 4 | 8 |
    | B | 2017 | 40 | 3 | 5 |
    | B | 2018 | 50 | 7 | 6 |
    | B | 2018 | 50 | 6 | 6 |
    """

    value_cols = [value_cols] if not isinstance(value_cols, list) else value_cols
    for col in value_cols:
        if not np.issubdtype(df[col].dtype, np.number):
            raise TypeError(col + " specified in value_cols must be of numeric type")

    if rank_cols_names is None:
        rank_cols_names = [x + '_rank' for x in value_cols]

    if group_cols is None:
        df[rank_cols_names] = df[value_cols].rank(method=method, ascending=ascending)
    else:
        df[rank_cols_names] = (df.groupby(group_cols)[value_cols]
                                 .rank(method=method, ascending=ascending))

    if method != 'average':
        df[rank_cols_names] = df[rank_cols_names].astype('int')

    return df

Response: This function creates a new ranking column based on numeric values in the specified column.


Instruction:
Can you implement a function in Python 3 that
returns a line for each bars of a waterfall chart, totals, groups, subgroups. Compute the variation and variation rate for each line. --- ### Parameters *mandatory :* - `date` (*str*): name of the column that id the period of each lines - `value` (*str*): name of the column that contains the vaue for each lines - `start` (*dict*): - `label`: text displayed under the first master column - `id`: value in the date col that id lines for the first period - `end` (*dict*): - `label`: text displayed under the last master column - `id`: value in the date col that id lines for the second period *optional :* - `upperGroup` (*dict*): - `id`: name of the column that contains upperGroups unique IDs - `label`: not required, text displayed under each upperGroups bars, using ID when it's absent - `groupsOrder`: not required, order of upperGroups - `insideGroup` (*dict*): - `id`: name of the column that contains insideGroups unique IDs - `label`: not required, text displayed under each insideGroups bars, using ID when it's absent - `groupsOrder`: not required, order of insideGroups - `filters` (*list*): columns to filters on --- ### Example **Input** | product_id | played | date | ord | category_id | category_name | |:------------:|:--------:|:------:|:-----:|:-------------:|:---------------:| | super clap | 12 | t1 | 1 | clap | Clap | | clap clap | 1 | t1 | 10 | clap | Clap | | tac | 1 | t1 | 1 | snare | Snare | | super clap | 10 | t2 | 1 | clap | Clap | | tac | 100 | t2 | 1 | snare | Snare | | bom | 1 | t2 | 1 | tom | Tom | ```cson waterfall: upperGroup: id: 'category_id' label: 'category_name' insideGroup: id: 'product_id' groupsOrder: 'ord' date: 'date' value: 'played' start: label: 'Trimestre 1' id: 't1' end: label: 'Trimester 2' id: 't2' ``` **Output** | value | label | variation | groups | type | order | |:-------:|:-----------:|:-----------:|:--------:|:------:|:-------:| | 14 | Trimestre 1 | NaN | NaN | NaN | NaN | | -3 | Clap | -0.230769 | clap | parent | NaN | | -2 | super clap | -0.166667 | clap | child | 1 | | -1 | clap clap | -1 | clap | child | 10 | | 99 | Snare | 99 | snare | parent | NaN | | 99 | tac | 99 | snare | child | 1 | | 1 | Tom | inf | tom | parent | NaN | | 1 | bom | inf | tom | child | 1 | | 111 | Trimester 2 | NaN | NaN | NaN | NaN |

Response:
def waterfall(
        df,
        date: str,
        value: str,
        start: Dict[str, str],
        end: Dict[str, str],
        upperGroup: Dict[str, str],
        insideGroup: Dict[str, str] = None,
        filters: List[str] = None
):
    """
    Return a line for each bars of a waterfall chart, totals, groups, subgroups.
    Compute the variation and variation rate for each line.

    ---

    ### Parameters

    *mandatory :*
    - `date` (*str*): name of the column that id the period of each lines
    - `value` (*str*): name of the column that contains the vaue for each lines
    - `start` (*dict*):
        - `label`: text displayed under the first master column
        - `id`: value in the date col that id lines for the first period
    - `end` (*dict*):
        - `label`: text displayed under the last master column
        - `id`: value in the date col that id lines for the second period

    *optional :*
    - `upperGroup` (*dict*):
        - `id`: name of the column that contains upperGroups unique IDs
        - `label`: not required, text displayed under each upperGroups bars,
          using ID when it's absent
        - `groupsOrder`: not required, order of upperGroups
    - `insideGroup` (*dict*):
        - `id`: name of the column that contains insideGroups unique IDs
        - `label`: not required, text displayed under each insideGroups bars,
          using ID when it's absent
        - `groupsOrder`: not required, order of insideGroups
    - `filters` (*list*): columns to filters on

    ---

    ### Example

    **Input**

    | product_id   |   played | date   |   ord | category_id   | category_name   |
    |:------------:|:--------:|:------:|:-----:|:-------------:|:---------------:|
    | super clap   |       12 | t1     |     1 | clap          | Clap            |
    | clap clap    |        1 | t1     |    10 | clap          | Clap            |
    | tac          |        1 | t1     |     1 | snare         | Snare           |
    | super clap   |       10 | t2     |     1 | clap          | Clap            |
    | tac          |      100 | t2     |     1 | snare         | Snare           |
    | bom          |        1 | t2     |     1 | tom           | Tom             |

    ```cson
    waterfall:
      upperGroup:
        id: 'category_id'
        label: 'category_name'
      insideGroup:
        id: 'product_id'
        groupsOrder: 'ord'
      date: 'date'
      value: 'played'
      start:
        label: 'Trimestre 1'
        id: 't1'
      end:
        label: 'Trimester 2'
        id: 't2'
    ```

    **Output**

    |   value | label       |   variation | groups   | type   |   order |
    |:-------:|:-----------:|:-----------:|:--------:|:------:|:-------:|
    |      14 | Trimestre 1 |  NaN        | NaN      | NaN    |     NaN |
    |      -3 | Clap        |   -0.230769 | clap     | parent |     NaN |
    |      -2 | super clap  |   -0.166667 | clap     | child  |       1 |
    |      -1 | clap clap   |   -1        | clap     | child  |      10 |
    |      99 | Snare       |   99        | snare    | parent |     NaN |
    |      99 | tac         |   99        | snare    | child  |       1 |
    |       1 | Tom         |  inf        | tom      | parent |     NaN |
    |       1 | bom         |  inf        | tom      | child  |       1 |
    |     111 | Trimester 2 |  NaN        | NaN      | NaN    |     NaN |
    """

    if len(df) == 0:
        return df

    if filters is not None:
        if isinstance(filters, str):
            filters = [filters]

        def sub_waterfall(df):
            wa_df = waterfall(df, date, value, start, end, upperGroup, insideGroup)
            for filters_col in filters:
                wa_df[filters_col] = df[filters_col].values[0]
            return wa_df

        # filters df into a list of sub_df
        list_of_sub_df = [df[(df[filters].values == i).all(axis=1)]
                          for i in df[filters].drop_duplicates().values]

        return pd.concat([sub_waterfall(df) for df in list_of_sub_df], sort=False)

    groups = {
        'upperGroup': {
            'type': 'parent',
            'id':  'upperGroup',
            'order': {
                'by': ['upperGroup_order', 'groups'],
                'ascending': [True, True]
            },
            'obj': upperGroup
        }
    }
    if insideGroup is not None:
        groups['insideGroup'] = {
            'type': 'child',
            'id': 'insideGroup',
            'order': {
                'by': ['type', 'insideGroup_order', 'label'],
                'ascending': [False, True, True]
            },
            'obj': insideGroup
        }
    # prepare the dataframe with standard column names
    df = _compute_rename(df, date, value, groups)

    agg_conf = {'value': sum}
    agg_conf.update({f'{col}_label': 'first' for col in groups.keys()})
    agg_conf.update({f'{col}_order': 'first' for col in groups.keys()})
    df = df.groupby(list(groups.keys()) + ['date']).agg(agg_conf).reset_index()

    df_start, df_end = _compute_start_end(df, start, end)

    df = _compute_value_diff(df, start, end, groups)

    middle = _compute_upper_group(df)
    if insideGroup is not None:
        middle = pd.concat([middle, _compute_inside_group(df)])

    ret = _compute_order(df_start, df_end, middle, groups)

    return ret


Instruction:
How would you code a function in Python 3 to
compute two dataframes with value for start and end

Response:
def _compute_start_end(df, start, end):
    """
    Compute two dataframes with value for start and end
    Args:
        totals(dataframe):

    Returns: Dataframe, Dataframe

    """
    result = {}
    time_dict = {'start': start, 'end': end}
    totals = df.groupby('date').agg({'value': sum}).reset_index()
    for time_name, time in time_dict.items():
        if not totals[totals['date'] == time['id']].empty:
            value = totals.loc[
                totals['date'] == time['id'], 'value'
            ].values[0]
        else:
            value = 0
        result[time_name] = pd.DataFrame([{
            'value': value,
            'label': time['label'],
            'groups': time['label']
        }])
    return result['start'], result['end']


Instruction:
Implement a function in Python 3 to
compute diff value between start and end

Response:
def _compute_value_diff(df, start, end, groups):
    """
    Compute diff value between start and end
    Args:
        df(dataframe):

    Returns: Dataframe

    """
    start_values = df[df['date'] == start['id']].copy()
    end_values = df[df['date'] == end['id']].copy()

    merge_on = []
    for key, group in groups.items():
        merge_on = merge_on + [key, f'{key}_label', f'{key}_order']

    df = start_values.merge(end_values,
                            on=merge_on,
                            how='outer',
                            suffixes=('_start', '_end'), )

    # necessary before calculating variation
    df[['value_start', 'value_end']] = df[['value_start', 'value_end']].fillna(0)
    df['value'] = df['value_end'] - df['value_start']
    df.drop(['date_start', 'date_end', 'value_end'], axis=1, inplace=True)
    df.rename(columns={'upperGroup': 'groups'}, inplace=True)
    return df


Instruction:
How would you implement a function in Python 3 that
computes inside Group Args: df(dataframe): Returns: Dataframe

Response:
def _compute_inside_group(df):
    """
    Compute inside Group
    Args:
        df(dataframe):

    Returns: Dataframe

    """
    inside_group = df.copy()
    inside_group['type'] = 'child'
    inside_group['variation'] = inside_group['value'] / inside_group[
        'value_start']
    inside_group.drop(['upperGroup_label', 'insideGroup', 'value_start'],
                      axis=1, inplace=True)
    inside_group.rename(columns={'insideGroup_label': 'label'},
                        inplace=True)
    return inside_group


Instruction:
Can you generate the documentation for the following Python 3 function
def _compute_upper_group(df):
    upper_group = df.groupby(['groups']).agg({
        'value': sum,
        'value_start': sum,
        'upperGroup_label': 'first',
        'upperGroup_order': 'first'
    }).reset_index()
    upper_group['type'] = 'parent'
    upper_group['variation'] = upper_group['value'] / upper_group[
        'value_start']
    upper_group.drop(['value_start'], axis=1, inplace=True)
    upper_group.rename(columns={'upperGroup_label': 'label'}, inplace=True)
    return upper_group

Response: Compute the upper group of the items in the log file


Instruction:
Explain what the following Python 3 code does
def add(df, new_column, column_1, column_2):
    return _basic_math_operation(df, new_column, column_1, column_2, op='add')

Response: Add a column to another column.


Instruction:
How would you explain what the following Python 3 function does
def subtract(df, new_column, column_1, column_2):
    return _basic_math_operation(df, new_column, column_1, column_2, op='sub')

Response: Subtract a column from a DataFrame.


Instruction:
Explain what the following Python 3 code does
def multiply(df, new_column, column_1, column_2):
    return _basic_math_operation(df, new_column, column_1, column_2, op='mul')

Response: Multiplies a DataFrame by another column.


Instruction:
Given the following Python 3 function, write the documentation
def divide(df, new_column, column_1, column_2):
    return _basic_math_operation(df, new_column, column_1, column_2, op='truediv')

Response: Divide a dataframe by two columns.


Instruction:
Can you implement a function in Python 3 that
does mathematic operations on columns (add, subtract, multiply or divide) --- ### Parameters *mandatory:* - `new_column` (*str*): name of the output column - `formula` (*str*): Operation on column. Use name of column and special character: - `+` for addition - `-` for subtraction - `*` for multiplication - `/` for division **Note:** - your column name can contain spaces. - if your column name is a number, you must use a quote mark : `"` or `'` (cf. example) --- ### Examples **Input** | variable | valueA | valueB | My rate | |:--------:|:--------:|:-----:|:------:| | toto | 20 | 100 | 10 | | toto | 30 | 200 | 10 | | toto | 10 | 300 | 10 | ```cson formula: new_column: 'valueD' formula: '(valueB + valueA ) / My rate' ``` **Output** | variable | valueA | valueB | My rate | valueD | |:--------:|:--------:|:------:|:-------:|:-------:| | toto | 20 | 100 | 10 | 12 | | toto | 30 | 200 | 10 | 23 | | toto | 10 | 300 | 10 | 31 | --- **Input** | variable | 2018 | 2019 | |:--------:|:--------:|:-----:| | toto | 20 | 100 | | toto | 30 | 200 | | toto | 10 | 300 | ```cson formula: new_column: 'Evolution' formula: "'2019' - '2018'" ``` **Output** | variable | 2018 | 2019 | Evolution | |:--------:|:--------:|:-----:|:-----:| | toto | 20 | 100 | 80 | | toto | 30 | 200 | 170 | | toto | 10 | 300 | 290 |

Response:
def formula(df, *, new_column: str, formula: str):
    """
    Do mathematic operations on columns (add, subtract, multiply or divide)

    ---

    ### Parameters

    *mandatory:*
    - `new_column` (*str*): name of the output column
    - `formula` (*str*): Operation on column. Use name of column and special character:
        - `+` for addition
        - `-` for subtraction
        - `*` for multiplication
        - `/` for division

    **Note:**
    - your column name can contain spaces.
    - if your column name is a number, you must use a quote mark : `"` or `'` (cf. example)

    ---

    ### Examples

    **Input**

    | variable | valueA | valueB | My rate |
    |:--------:|:--------:|:-----:|:------:|
    |   toto   |    20    |  100  |   10   |
    |   toto   |    30    |  200  |   10   |
    |   toto   |    10    |  300  |   10   |

    ```cson
    formula:
      new_column: 'valueD'
      formula: '(valueB + valueA ) / My rate'
    ```

    **Output**

    | variable | valueA   | valueB |  My rate |  valueD |
    |:--------:|:--------:|:------:|:-------:|:-------:|
    |   toto   |    20    |   100  |    10   |     12  |
    |   toto   |    30    |   200  |    10   |     23  |
    |   toto   |    10    |   300  |    10   |     31  |

    ---

    **Input**

    | variable | 2018 | 2019 |
    |:--------:|:--------:|:-----:|
    |   toto   |    20    |  100  |
    |   toto   |    30    |  200  |
    |   toto   |    10    |  300  |

    ```cson
    formula:
      new_column: 'Evolution'
      formula: "'2019' - '2018'"
    ```

    **Output**

    | variable | 2018 | 2019 | Evolution |
    |:--------:|:--------:|:-----:|:-----:|
    |   toto   |    20    |  100  | 80 |
    |   toto   |    30    |  200  | 170 |
    |   toto   |    10    |  300  | 290 |

    """
    tokens = _parse_formula(formula)
    expression_splitted = []
    for t in tokens:
        # To use a column name with only digits, it has to be quoted!
        # Otherwise it is considered as a regular number
        if not t.quoted and (t in MATH_CHARACTERS or is_float(t)):
            expression_splitted.append(t)
        elif t in df.columns:
            expression_splitted.append(f'df["{t}"]')
        else:
            raise FormulaError(f'"{t}" is not a valid column name')
    expression = ''.join(expression_splitted)
    df[new_column] = eval(expression)
    return df


Instruction:
Explain what the following Python 3 code does
def round_values(df, *, column: str, decimals: int, new_column: str = None):
    new_column = new_column or column
    df[new_column] = df[column].round(decimals)
    return df

Response: Round each value of a column in a DataFrame to a specified number of decimals.


Instruction:
Explain what the following Python 3 code does
def absolute_values(df, *, column: str, new_column: str = None):
    new_column = new_column or column
    df[new_column] = abs(df[column])
    return df

Response: Returns the absolute numeric value of each element of a column in a DataFrame.


Instruction:
Make a summary of the following Python 3 code
def pivot(df, index: List[str], column: str, value: str, agg_function: str = 'mean'):
    if df.dtypes[value].type == np.object_:
        df = pd.pivot_table(df, index=index,
                            columns=column,
                            values=value,
                            aggfunc=lambda x: ' '.join(x))
    else:
        df = pd.pivot_table(df, index=index,
                            columns=column,
                            values=value,
                            aggfunc=agg_function)
    df = df.reset_index()
    return df

Response: Pivot the data in the DataFrame df.


Instruction:
How would you explain what the following Python 3 function does
def pivot_by_group(
        df,
        variable,
        value,
        new_columns,
        groups,
        id_cols=None
):
    """
    Pivot a dataframe by group of variables

    ---

    ### Parameters

    *mandatory :*
    * `variable` (*str*): name of the column used to create the groups.
    * `value` (*str*): name of the column containing the value to fill the pivoted df.
    * `new_columns` (*list of str*): names of the new columns.
    * `groups` (*dict*): names of the groups with their corresponding variables.
      **Warning**: the list of variables must have the same order as `new_columns`

    *optional :*
    * `id_cols` (*list of str*) : names of other columns to keep, default `None`.

    ---

    ### Example

    **Input**

    | type |  variable  | montant |
    |:----:|:----------:|:-------:|
    |   A  |    var1    |    5    |
    |   A  | var1_evol  |   0.3   |
    |   A  |    var2    |    6    |
    |   A  | var2_evol  |   0.2   |

    ```cson
    pivot_by_group :
      id_cols: ['type']
      variable: 'variable'
      value: 'montant'
      new_columns: ['value', 'variation']
      groups:
        'Group 1' : ['var1', 'var1_evol']
        'Group 2' : ['var2', 'var2_evol']
    ```

    **Ouput**

    | type |  variable  |  value  | variation |
    |:----:|:----------:|:-------:|:---------:|
    |   A  |   Group 1  |    5    |    0.3    |
    |   A  |   Group 2  |    6    |    0.2    |

    """
    if id_cols is None:
        index = [variable]
    else:
        index = [variable] + id_cols

    param = pd.DataFrame(groups, index=new_columns)
    temporary_colum = 'tmp'

    df[temporary_colum] = df[variable]
    for column in param.columns:
        df.loc[df[variable].isin(param[column]), variable] = column

    param = param.T
    for column in param.columns:
        df.loc[
            df[temporary_colum].isin(param[column]), temporary_colum] = column

    df = pivot(df, index, temporary_colum, value)
    return df

Response: Pivot a dataframe by group of variables.


Instruction:
Can you write a function in Python 3 where it
groups by group by columns.

Response:
def groupby(df, *, group_cols: Union[str, List[str]],
            aggregations: Dict[str, Union[str, List[str]]]):
    """
    Aggregate values by groups.

    ---

    ### Parameters

    *mandatory :*
    - `group_cols` (*list*): list of columns used to group data
    - `aggregations` (*dict*): dictionnary of values columns to group as keys and aggregation
      function to use as values (See the [list of aggregation functions](
      https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html#aggregation))

    ---

    ### Example

    **Input**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 |
    |:------:|:----:|:-------:|:-------:|
    |    A   | 2017 |    10   |    3    |
    |    A   | 2017 |    20   |    1    |
    |    A   | 2018 |    10   |    5    |
    |    A   | 2018 |    30   |    4    |
    |    B   | 2017 |    60   |    4    |
    |    B   | 2017 |    40   |    3    |
    |    B   | 2018 |    50   |    7    |
    |    B   | 2018 |    60   |    6    |

    ```cson
    groupby:
      group_cols: ['ENTITY', 'YEAR']
      aggregations:
        'VALUE_1': 'sum',
        'VALUE_2': 'mean'
    ```

    **Output**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 |
    |:------:|:----:|:-------:|:-------:|
    |    A   | 2017 |    30   |   2.0   |
    |    A   | 2018 |    40   |   4.5   |
    |    B   | 2017 |   100   |   3.5   |
    |    B   | 2018 |   110   |   6.5   |

    """
    df = df.groupby(group_cols, as_index=False).agg(aggregations)

    # When several aggregations are performed on the same column, pandas return
    # a multi-indexed dataframe, so we need to flatten the columns index to get
    # back to a unique level header
    if df.columns.nlevels == 2:
        level_0 = df.columns.get_level_values(0)
        level_1 = df.columns.get_level_values(1)
        new_columns = [(f'{x}_{y}' if x else y) for (x, y)
                       in zip(level_1, level_0)]
        df.columns = new_columns
    return df


Instruction:
Create a Python 3 function to
compute the cumulative sum of a column in a DataFrame.

Response:
def cumsum(df, new_column: str, column: str, index: list, date_column: str, date_format: str):
    """
    DEPRECATED - please use `compute_cumsum` instead
    """
    logging.getLogger(__name__).warning(f"DEPRECATED: use compute_cumsum")
    date_temp = '__date_temp__'
    if isinstance(index, str):
        index = [index]
    levels = list(range(0, len(index)))
    df[date_temp] = pd.to_datetime(df[date_column], format=date_format)
    reference_cols = [date_temp, date_column]
    df = df.groupby(index + reference_cols).sum()
    df[new_column] = df.groupby(level=levels)[column].cumsum()
    df.reset_index(inplace=True)
    del df[date_temp]

    return df


Instruction:
Can you generate a brief explanation for the following Python 3 code
def add_missing_row(
    df: pd.DataFrame,
    id_cols: List[str],
    reference_col: str,
    complete_index: Union[Dict[str, str], List[str]] = None,
    method: str = None,
    cols_to_keep: List[str] = None
) -> pd.DataFrame:
    """
    Add missing row to a df base on a reference column

    ---

    ### Parameters

    *mandatory :*
    - `id_cols` (*list of str*): names of the columns used to create each group
    - `reference_col` (*str*): name of the column used to identify missing rows

    *optional :*
    - `complete_index` (*list* or *dict*): [A, B, C] a list of values used to add missing rows.
      It can also be a dict to declare a date range.
      By default, use all values of reference_col.
    - `method` (*str*): by default all missing rows are added. The possible values are :
        - `"between"` : add missing rows having their value between min and max values for each group,
        - `"between_and_after"` : add missing rows having their value bigger than min value for each group.
        - `"between_and_before"` : add missing rows having their value smaller than max values for each group.
    - `cols_to_keep` (*list of str*): name of other columns to keep, linked to the reference_col.

    ---

    ### Example

    **Input**

    YEAR | MONTH | NAME
    :---:|:---:|:--:
    2017|1|A
    2017|2|A
    2017|3|A
    2017|1|B
    2017|3|B

    ```cson
    add_missing_row:
      id_cols: ['NAME']
      reference_col: 'MONTH'
    ```

    **Output**

    YEAR | MONTH | NAME
    :---:|:---:|:--:
    2017|1|A
    2017|2|A
    2017|3|A
    2017|1|B
    2017|2|B
    2017|3|B

    """
    if cols_to_keep is None:
        cols_for_index = [reference_col]
    else:
        cols_for_index = [reference_col] + cols_to_keep
    check_params_columns_duplicate(id_cols + cols_for_index)

    if method == 'between' or method == 'between_and_after':
        df['start'] = df.groupby(id_cols)[reference_col].transform(min)
        id_cols += ['start']
    if method == 'between' or method == 'between_and_before':
        df['end'] = df.groupby(id_cols)[reference_col].transform(max)
        id_cols += ['end']

    names = id_cols + cols_for_index
    new_df = df.set_index(names)
    index_values = df.groupby(id_cols).sum().index.values

    if complete_index is None:
        complete_index = df.groupby(cols_for_index).sum().index.values
    elif isinstance(complete_index, dict):
        if complete_index['type'] == 'date':
            freq = complete_index['freq']
            date_format = complete_index['format']
            start = complete_index['start']
            end = complete_index['end']
            if isinstance(freq, dict):
                freq = pd.DateOffset(**{k: int(v) for k, v in freq.items()})
            complete_index = pd.date_range(start=start, end=end, freq=freq)
            complete_index = complete_index.strftime(date_format)
        else:
            raise ParamsValueError(f'Unknown complete index type: '
                                   f'{complete_index["type"]}')

    if not isinstance(index_values[0], tuple):
        index_values = [(x,) for x in index_values]
    if not isinstance(complete_index[0], tuple):
        complete_index = [(x,) for x in complete_index]
    new_tuples_index = [x + y for x in index_values for y in complete_index]

    new_index = pd.MultiIndex.from_tuples(new_tuples_index, names=names)
    new_df = new_df.reindex(new_index).reset_index()

    if method == 'between' or method == 'between_and_after':
        new_df = new_df[new_df[reference_col] >= new_df['start']]
        del new_df['start']
    if method == 'between' or method == 'between_and_before':
        new_df = new_df[new_df[reference_col] <= new_df['end']]
        del new_df['end']

    return new_df

Response: Adds a missing row to a DataFrame.


Instruction:
Write a Python 3 function that can
extract the contents of a zip file into a dictionary.

Response:
def extract_zip(zip_file_path):
    """
    Returns:
        dict: Dict[str, DataFrame]
    """
    dfs = {}
    with zipfile.ZipFile(zip_file_path, mode='r') as z_file:
        names = z_file.namelist()
        for name in names:
            content = z_file.read(name)
            _, tmp_file_path = tempfile.mkstemp()
            try:
                with open(tmp_file_path, 'wb') as tmp_file:
                    tmp_file.write(content)

                dfs[name] = joblib.load(tmp_file_path)
            finally:
                shutil.rmtree(tmp_file_path, ignore_errors=True)
    return dfs


Instruction:
Implement a Python 3 function for
extracting the n term tables from the given data.

Response:
def extract(data):
    """
    Args:
        data (str | byte):

    Returns:
        dict: Dict[str, DataFrame]

    """
    _, tmp_file_path = tempfile.mkstemp()
    try:
        with open(tmp_file_path, 'wb') as tmp_file:
            tmp_file.write(data)

        if zipfile.is_zipfile(tmp_file_path):
            return extract_zip(tmp_file_path)
        else:
            raise DataSdkError('Unsupported file type')
    finally:
        shutil.rmtree(tmp_file_path, ignore_errors=True)


Instruction:
Can you generate the documentation for the following Python 3 function
def read_from_cache(self, domains=None):
        logger.info(f'Reading data from cache ({self.EXTRACTION_CACHE_PATH})')
        if domains is not None and isinstance(domains, list):
            dfs = {domain: self.read_entry(domain) for domain in domains}
        else:
            dfs = {name: self.read_entry(name)
                   for name in os.listdir(self.EXTRACTION_CACHE_PATH)}
        return dfs

Response: Reads the data from the cache.


Instruction:
Implement a function in Python 3 to
read a cache entry from the given file.

Response:
def read_entry(self, file_name):
        """
        Args:
            file_name (str):

        Returns:
            pd.DataFrame:
        """
        file_path = os.path.join(self.EXTRACTION_CACHE_PATH, file_name)
        logger.info(f'Reading cache entry: {file_path}')
        return joblib.load(file_path)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def write(self, dfs):
        if not os.path.exists(self.EXTRACTION_CACHE_PATH):
            os.makedirs(self.EXTRACTION_CACHE_PATH)

        for name, df in dfs.items():
            file_path = os.path.join(self.EXTRACTION_CACHE_PATH, name)
            joblib.dump(df, filename=file_path)
            logger.info(f'Cache entry added: {file_path}')

Response: Writes the data to the cache directory.


Instruction:
Can you tell what is the following Python 3 function doing
def compute_ffill_by_group(
        df,
        id_cols: List[str],
        reference_cols: List[str],
        value_col: str
):
    """
    Compute `ffill` with `groupby`
    Dedicated method as there is a performance issue with a simple groupby/fillna (2017/07)
    The method `ffill` propagates last valid value forward to next values.

    ---

    ### Parameters

    *mandatory :*
    - `id_cols` (*list of str*): names of columns used to create each group.
    - `reference_cols` (*list of str*): names of columns used to sort.
    - `value_col` (*str*): name of the columns to fill.

    ---

    ### Example

    **Input**

    name | rank | value
    :------:|:--------------:|:--------:
    A | 1 | 2
    A | 2 | 5
    A | 3 | null
    B | 1 | null
    B | 2 | 7

    ```cson
    compute_ffill_by_group:
      id_cols: ['name']
      reference_cols: ['rank']
      value_col: 'value'
    ```

    **Ouput**

    name | rank | value
    :------:|:--------------:|:--------:
    A | 1 | 2
    A | 2 | 5
    A | 3 | 5
    B | 1 | null
    B | 2 | 7
    """
    check_params_columns_duplicate(id_cols + reference_cols + [value_col])
    df = df.sort_values(by=id_cols + reference_cols)
    df = df.set_index(id_cols)
    df['fill'] = 1 - df[value_col].isnull().astype(int)
    df['fill'] = df.groupby(
        level=list(range(0, len(id_cols) - 1))
    )['fill'].cumsum()
    df[value_col] = df[value_col].ffill()
    df.loc[df['fill'] == 0, value_col] = None
    del df['fill']
    return df.reset_index()

Response: Compute the last valid value of a single record in a new tree tree.


Instruction:
Make a summary of the following Python 3 code
def two_values_melt(
    df,
    first_value_vars: List[str],
    second_value_vars: List[str],
    var_name: str,
    value_name: str
):
    """
    Transforms one or multiple columns into rows.
    Unlike melt function, two value columns can be returned by
    the function (e.g. an evolution column and a price column)

    ---

    ### Parameters

    *mandatory :*
    - `first_value_vars` (*list of str*): name of the columns corresponding to the first returned value column
    - `second_value_vars` (*list of str*): name of the columns corresponding to the second returned value column
    - `var_name` (*str*): name of the column containing values in first_value_vars
    - `value_name` (*str*): suffix of the two value columns (suffix_first / suffix_second)
    ---

    ### Example

    **Input**

    |    Region |      avg |       total |  evo_avg |   evo_total |
    |:---------:|:--------:|:-----------:|:--------:|:-----------:|
    |         A |        50|         100 |        1 |           4 |
    |         B |       40 |         250 |        2 |           5 |

    ```cson
    two_values_melt:
      first_value_vars: ["avg", "total"]
      second_value_vars: ["evo_avg", "evo_total"]
      var_name: "type"
      value_name: "value"
    ```

    **Output**

    |    Region |     type |  value_first |  value_second |
    |:---------:|:--------:|:------------:|:-------------:|
    |         A |       avg|           50 |             1 |
    |         A |     total|          100 |             4 |
    |         B |       avg|           40 |             2 |
    |         B |       avg|          250 |             5 |
    """
    value_name_first = value_name + '_first'
    value_name_second = value_name + '_second'

    # Melt on the first value columns
    melt_first_value = pd.melt(df,
                               id_vars=[col for col in list(df) if
                                        col not in first_value_vars],
                               value_vars=first_value_vars,
                               var_name=var_name,
                               value_name=value_name_first)
    melt_first_value.drop(second_value_vars, axis=1, inplace=True)

    # Melt on the second value columns
    melt_second_value = pd.melt(df,
                                id_vars=[col for col in list(df) if
                                         col not in second_value_vars],
                                value_vars=second_value_vars,
                                var_name=var_name,
                                value_name=value_name_second)

    # Since there are two value columns, there is no need to keep the
    # second_value_vars names. And it will make things easier for the merge.
    normalize_types = {k: v for k, v in zip(second_value_vars, first_value_vars)}
    melt_second_value.replace(normalize_types, inplace=True)
    melt_second_value.drop(first_value_vars, axis=1, inplace=True)

    on_cols = list(melt_first_value)
    on_cols.remove(value_name_first)
    return pd.merge(melt_first_value, melt_second_value, on=on_cols, how='outer')

Response: This function transforms one or multiple columns into two values.


Instruction:
Can you implement a function in Python 3 that
concatenates columns element - wise with a new column

Response:
def concat(
        df,
        *,
        columns: List[str],
        new_column: str,
        sep: str = None
):
    """
    Concatenate `columns` element-wise
    See [pandas doc](
    https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.cat.html) for more information

    ---

    ### Parameters

    *mandatory :*
    - `columns` (*list*): list of columns to concatenate (at least 2 columns)
    - `new_column` (*str*): the destination column

    *optional :*
    - `sep` (*str*): the separator
    """
    if len(columns) < 2:
        raise ValueError('The `columns` parameter needs to have at least 2 columns')
    first_col, *other_cols = columns
    df.loc[:, new_column] = df[first_col].astype(str).str.cat(df[other_cols].astype(str), sep=sep)
    return df


Instruction:
Here you have a function in Python 3, explain what it does
def contains(
        df,
        column: str,
        *,
        pat: str,
        new_column: str = None,
        case: bool = True,
        na: Any = None,
        regex: bool = True
):
    """
    Test if pattern or regex is contained within strings of `column`
    See [pandas doc](
    https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.contains.html) for more information

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): the column
    - `pat` (*str*): character sequence or regular expression.

    *optional :*
    - `new_column` (*str*): the destination column (if not set, `column` will be used)
    - `case` (*boolean*): if true, case sensitive.
    - `na`: fill value for missing values.
    - `regex` (*boolean*): default true
    """
    new_column = new_column or column
    df.loc[:, new_column] = df[column].str.contains(pat, case=case, na=na, regex=regex)
    return df

Response: Test if pattern or regex is contained within strings of column.


Instruction:
How would you explain what the following Python 3 function does
def repeat(
        df,
        column: str,
        *,
        times: int,
        new_column: str = None
):
    """
    Duplicate each string in `column` by indicated number of time
    See [pandas doc](
    https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.repeat.html) for more information

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): the column
    - `times` (*int*): times to repeat the string

    *optional :*
    - `new_column` (*str*): the destination column (if not set, `column` will be used)
    """
    new_column = new_column or column
    df.loc[:, new_column] = df[column].str.repeat(times)
    return df

Response: Duplicate each string in column by indicated number of time.


Instruction:
Can you create a Python 3 function that
replaces occurrences of pattern or regular expression in column with some other string.

Response:
def replace_pattern(
        df,
        column: str,
        *,
        pat: str,
        repl: str,
        new_column: str = None,
        case: bool = True,
        regex: bool = True
):
    """
    Replace occurrences of pattern/regex in `column` with some other string
    See [pandas doc](
    https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.replace.html) for more information

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): the column
    - `pat` (*str*): character sequence or regular expression
    - `repl` (*str*): replacement string

    *optional :*
    - `new_column` (*str*): the destination column (if not set, `column` will be used)
    - `case` (*boolean*): if true, case sensitive.
    - `regex` (*boolean*): default true
    """
    new_column = new_column or column
    df.loc[:, new_column] = df[column].str.replace(pat, repl, case=case, regex=regex)
    return df


Instruction:
How would you explain what the following Python 3 function does
def log_time(logger):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            start = time.time()
            result = func(*args, **kwargs)
            end = time.time()
            _log_time(logger, func.__name__, start, end)
            return result
        return wrapper
    return decorator

Response:
Decorator to log the execution time of a function
ridge


Instruction:
Can you generate the documentation for the following Python 3 function
def log(logger=None, start_message='Starting...', end_message='Done...'):
    def actual_log(f, real_logger=logger):
        logger = real_logger or _logger

        @wraps(f)
        def timed(*args, **kwargs):
            logger.info(f'{f.__name__} - {start_message}')
            start = time.time()
            res = f(*args, **kwargs)
            end = time.time()
            logger.info(f'{f.__name__} - {end_message} (took {end - start:.2f}s)')
            return res

        return timed

    if callable(logger):
        return actual_log(logger, real_logger=None)
    return actual_log

Response: A basic log decorator that logs the current state of the object.


Instruction:
Can you write a function in Python 3 where it
allows to apply a function f(df: DataFrame) -> DataFrame) on dfs by specifying the key E.g instead of writing: def process_domain1(dfs): df = dfs['domain1'] # actual process dfs['domain1'] = df return dfs You can write: @domain('domain1') def process_domain1(df): #actual process return df

Response:
def domain(domain_name):
    """
    Allow to apply a function f(df: DataFrame) -> DataFrame) on dfs by specifying the key
    E.g instead of writing:
        def process_domain1(dfs):
            df = dfs['domain1']
            # actual process
            dfs['domain1'] = df
            return dfs

    You can write:
        @domain('domain1')
        def process_domain1(df):
            #actual process
            return df
    """
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            dfs, *args = args
            if not isinstance(dfs, dict):
                raise TypeError(f'{dfs} is not a dict')
            df = dfs.pop(domain_name)
            df = func(df, *args, **kwargs)
            return {domain_name: df, **dfs}

        return wrapper

    return decorator


Instruction:
Here you have a function in Python 3, explain what it does
def cache(  # noqa: C901
    requires=None,
    disabled=False,
    applied_on_method=False,
    check_param=True,
    limit=None
):
    """ Avoid to recompute a function if its parameters and its source code doesnt have changed.

        Args:
            requires: list of dependencies (functions or function names)
            disabled (bool): disable the cache mecanism for this function (useful if you
                                 only want to use the dependency mecanism)
            applied_on_method (bool): ignore the first argument (useful to ignore "self")
            check_param (True, False or a str): the name of the parameter to check.
                                                    False to not check any of them.
                                                    True (default) to check all of them.
            limit (int or None): number of cache entries to keep (no limit by default)
    """
    if not requires:
        requires = []
    elif isinstance(requires, collections.Callable):
        requires = [requires]

    if not isinstance(check_param, (bool, str)):
        raise TypeError("'check_param' must be a str (name of the param to check) or a bool")
    if limit is not None and not isinstance(limit, int):
        raise TypeError("'limit' must be an int (number of cache entries to keep) or None")

    # We keep data in the function attributes so that this data
    # is not erased between two calls:
    if not hasattr(cache, 'funcs_references'):
        cache.funcs_references = {}  # dict of {function_name -> function_object (or None)}
    if not hasattr(cache, 'dependencies'):
        cache.dependencies = {}  # dict of {function_name -> [list of function names]}
    if not hasattr(cache, 'memories'):
        cache.memories = {}  # dict of {thread_id -> joblib.Memory object}

    def decorator(func):
        """ This code is executed when the augment module is read (when decorator is applied).
            Here we populate cache.funcs_references and cache.dependencies to use them later. """
        cache.funcs_references[func.__name__] = get_orig_function(func)
        dependencies_names = []
        for requirement in requires:
            if isinstance(requirement, collections.Callable):
                req_name = requirement.__name__
                cache.funcs_references[req_name] = get_orig_function(requirement)
            elif requirement not in cache.funcs_references:
                req_name = requirement
                cache.funcs_references[req_name] = None
            dependencies_names.append(req_name)

        cache.dependencies[func.__name__] = dependencies_names

        @wraps(func)
        def wrapper(*args, **kwargs):
            """ This code is executed when a decorated function is actually executed.
                It uses the previously built dependency tree (see above). """
            current_memory = cache.memories.get(current_thread().name)
            if disabled is True or current_memory is None:
                return func(*args, **kwargs)

            # if cache is enabled, we compute the md5 hash of the concatenated source codes
            # of all the dependencies.
            concatenated_source_code = ''
            dependencies = resolve_dependencies(func.__name__, cache.dependencies)
            for func_name in dependencies:
                function = cache.funcs_references[func_name]
                if function is None:
                    raise Exception(f"Can't get source code of function '{func_name}'")
                source_code = get_func_sourcecode(function)
                concatenated_source_code += source_code
            md5_hash = md5(str.encode(concatenated_source_code)).hexdigest()

            # Add extra parameters so that joblib checks they didnt have changed:
            tmp_extra_kwargs = {
                '__func_dependencies_hash__': md5_hash,
                '__original_func_name__': func.__name__,
            }

            if check_param is True:
                kwargs.update(tmp_extra_kwargs)

                if applied_on_method:
                    self_arg, args = args[0], args[1:]

                @wraps(func)
                def f(*args, **kwargs):
                    # delete the extra parameters that the underlying function doesnt expect:
                    for k in tmp_extra_kwargs.keys():
                        del kwargs[k]

                    if applied_on_method:
                        args = (self_arg,) + args
                    return func(*args, **kwargs)

                f = current_memory.cache(f)
                result = f(*args, **kwargs)
            else:
                if isinstance(check_param, str):
                    check_only_param_value = get_param_value_from_func_call(
                        param_name=check_param,
                        func=func,
                        call_args=args,
                        call_kwargs=kwargs,
                    )
                    tmp_extra_kwargs['__check_only__'] = check_only_param_value

                @wraps(func)
                def f(**tmp_extra_kwargs):
                    return func(*args, **kwargs)

                f = current_memory.cache(f)
                result = f(**tmp_extra_kwargs)

            if limit is not None:
                clean_cachedir_old_entries(f.store_backend, func.__name__, limit)

            return result

        return wrapper

    return decorator

Response: A decorator that creates a cache object for the given function.


Instruction:
Make a summary of the following Python 3 code
def setup_cachedir(cachedir, mmap_mode=None, bytes_limit=None):
    if not hasattr(cache, 'memories'):
        cache.memories = {}

    memory = joblib.Memory(
        location=cachedir,
        verbose=0,
        mmap_mode=mmap_mode,
        bytes_limit=bytes_limit,
    )
    cache.memories[current_thread().name] = memory
    return memory

Response: This function creates a joblib. Memory object in the cache object.


Instruction:
How would you explain what the following Python 3 function does
def melt(
        df,
        id: List[str],
        value: List[str],
        dropna=False
):
    """
    A melt will transform a dataset by creating a column "variable" and a column "value".
    This function is useful to transform a dataset into a format where one or more columns
    are identifier variables, while all other columns, considered measured
    variables (value_vars), are “unpivoted” to the row axis, leaving just two
    non-identifier columns, `"variable"` and `"value"`.

    ---

    ### Parameters

    *mandatory :*
    - `id` (*list of str*): names of the columns that must be kept in column.
    - `value` (*list of str*): names of the columns that will be transformed in long format (in rows).

    *optional :*
    - `dropna` (*boolean*): It allows you to drop missing values.

    ---

    ### Example

    **Input**

    | my_label | my_value | my_column_1 | my_column_2 | info_1 | info_2 | info_3 |
    |:--------:|:--------:|:-----------:|:-----------:|:------:|:------:|:------:|
    |   toto   |    10    |     S45     |    Lalaland |   10   |   20   |  None  |

    ```cson
    melt:
      id: ['my_label', 'my_value' 'my_column_1', 'my_colum_2']
        value: ['info_1', 'info_2', 'info_3']
        dropna: true
    ```

    **Ouput**

    | my_label | my_value | my_column_1 | my_column_2 | variable | value  |
    |:--------:|:--------:|:-----------:|:-----------:|:--------:|:------:|
    |   toto   |    10    |     S45     |    Lalaland |  info_1  |   10   |
    |   toto   |    10    |     S45     |    Lalaland |  info_2  |   20   |
    """
    df = df[(id + value)]
    df = pd.melt(df, id_vars=id, value_vars=value)
    if dropna:
        df = df.dropna(subset=['value'])

    return df

Response: A melt transform function for a given set of columns.


Instruction:
Explain what the following Python 3 code does
def rename(
        df,
        values: Dict[str, Dict[str, str]] = None,
        columns: Dict[str, Dict[str, str]] = None,
        locale: str = None
):
    """
    Replaces data values and column names according to the locale

    ---

    ### Parameters

    - `values` (optional: dict):
        - key: term to be replaced
        - value:
            - key: the locale e.g. 'en' or 'fr'
            - value: term's translation
    - `columns` (optional: dict):
        - key: columns name to be replaced
        - value:
            - key: the locale e.g. 'en' or 'fr'
            - value: column name's translation
    - `locale` (optional: str): the locale you want to use.
      By default the client locale is used.

    ---

    ### Example

    **Input**

    | label | value |
    |:----------------:|:-----:|
    | France | 100 |
    | Europe wo France | 500 |

    ```cson
    rename:
      values:
        'Europe wo France':
          'en': 'Europe excl. France'
          'fr': 'Europe excl. France'
      columns:
        'value':
          'en': 'revenue'
          'fr': 'revenue'
    ```

    **Output**

    | label | revenue |
    |:-------------------:|:-------:|
    | France | 100 |
    | Europe excl. France | 500 |

    """
    if values:
        to_replace = list(values.keys())
        value = [values[term][locale] for term in values]
        df = df.replace(to_replace=to_replace, value=value)
    if columns:
        _keys = list(columns.keys())
        _values = [column[locale] for column in columns.values()]
        columns = dict(list(zip(_keys, _values)))
        df = df.rename(columns=columns)
    return df

Response: Rename the names of the entries in the n - term table.


Instruction:
Create a Python 3 function to
compute the cumsum sum of a group of columns.

Response:
def compute_cumsum(
    df,
    id_cols: List[str],
    reference_cols: List[str],
    value_cols: List[str],
    new_value_cols: List[str] = None,
    cols_to_keep: List[str] = None
):
    """
    Compute cumsum for a group of columns.

    ---

    ### Parameters

    *mandatory :*
    - `id_cols` (*list*): the columns id to create each group
    - `reference_cols` (*list*): the columns to order the cumsum
    - `value_cols` (*list*): the columns to cumsum

    *optional :*
    - `new_value_cols` (*list*): the new columns with the result cumsum
    - `cols_to_keep` (*list*): other columns to keep in the dataset.
      This option can be used if there is only one row by group [id_cols + reference_cols]

    ---

    ### Example

    **Input**

    MONTH | DAY | NAME | VALUE | X
    :---:|:---:|:--:|:---:|:---:
     1   |   1 |   A  |  1 | lo
     2   |   1 |   A  |  1 | lo
     2   |  15 |   A  |  1 | la
     1   |  15 |   B  |  1 | la

    ```cson
    compute_cumsum:
      id_cols: ['NAME']
      reference_cols: ['MONTH', 'DAY']
      cumsum_cols: ['VALUE']
      cols_to_keep: ['X']
    ```

    **Output**

    NAME | MONTH | DAY | X | VALUE
    :---:|:---:|:--:|:---:|:---:
     A  |   1  |    1 | lo  |    1
     A  |   2  |    1 | la  |    2
     A  |   2  |   15 | lo  |    3
     B  |   1  |   15 | la  |    1
    """
    if cols_to_keep is None:
        cols_to_keep = []

    if new_value_cols is None:
        new_value_cols = value_cols
    if len(value_cols) != len(new_value_cols):
        raise ParamsValueError('`value_cols` and `new_value_cols` needs '
                               'to have the same number of elements')

    check_params_columns_duplicate(id_cols + reference_cols + cols_to_keep + value_cols)

    levels = list(range(0, len(id_cols)))

    df = df.groupby(id_cols + reference_cols + cols_to_keep).sum()
    df[new_value_cols] = df.groupby(level=levels)[value_cols].cumsum()

    return df.reset_index()


Instruction:
Here you have a function in Python 3, explain what it does
def combine_columns_aggregation(
        df,
        id_cols: List[str],
        cols_for_combination: Dict[str, str],
        agg_func: Union[str, List[str], Dict[str, str]] = 'sum'
):
    """
    Aggregates data to reproduce "All" category for requester

    ---

    ### Parameters

    *mandatory :*
    - `id_cols` (*list*): the columns id to group
    - `cols_for_combination` (*dict*): colums corresponding to
       the filters as key and their default value as value

    *optional :*
    - `agg_func` (*str*, *list* or *dict*): the function(s) to use for aggregating the data.
       Accepted combinations are:
       - string function name
       - list of functions and/or function names, e.g. [np.sum, 'mean']
       - dict of axis labels -> functions, function names or list of such.
    """
    requesters_cols = list(cols_for_combination.keys())
    requester_combination = [
        list(item) for i in range(0, len(requesters_cols) + 1)
        for item in itertools.combinations(requesters_cols, i)]
    dfs_result = []
    for comb in requester_combination:
        df_tmp = df.groupby(id_cols + comb).agg(agg_func).reset_index()
        for key in (set(cols_for_combination.keys()) - set(comb)):
            df_tmp[key] = cols_for_combination[key]
        dfs_result.append(df_tmp)

    return pd.concat(dfs_result, sort=False, ignore_index=True)

Response: Combine the columns of the given dataframe with the aggregation function.


Instruction:
Make a summary of the following Python 3 code
def get_param_value_from_func_call(param_name, func, call_args, call_kwargs):
    signature = inspect.signature(func)
    params_list = signature.parameters.keys()
    if param_name not in params_list:
        raise TypeError(f"'{param_name}' not found in {func.__name__}"
                        f"parameters list ([{params_list}])")
    call = signature.bind(*call_args, **call_kwargs)
    call.apply_defaults()
    return call.arguments[param_name]

Response: Get the value of a parameter from a function call.


Instruction:
Make a summary of the following Python 3 code
def get_func_sourcecode(func):

    def getsource(func):
        lines, lnum = getsourcelines(func)
        return ''.join(lines)

    def getsourcelines(func):
        lines, lnum = findsource(func)
        return inspect.getblock(lines[lnum:]), lnum + 1

    def findsource(func):
        file = getfile(func)  # file path
        module = inspect.getmodule(func, file)
        lines = linecache.getlines(file, module.__dict__)
        code = func.__code__
        lnum = code.co_firstlineno - 1
        pat = re.compile(r'^(\s*def\s)|(\s*async\s+def\s)|(.*(?<!\w)lambda(:|\s))|^(\s*@)')
        while lnum > 0:
            if pat.match(lines[lnum]):
                break
            lnum = lnum - 1  # pragma: no cover
        return lines, lnum

    def getfile(func):
        module = inspect.getmodule(func)
        return module.__file__

    try:
        return inspect.getsource(func)
    except Exception:
        return getsource(func)

Response: Try to get sourcecode using standard inspect. getsource.


Instruction:
Can you write a function in Python 3 where it
returns a slugified name

Response:
def slugify(name, separator='-'):
    """Returns a slugified name (we allow _ to be used)"""
    return _slugify(name, regex_pattern=re.compile('[^-_a-z0-9]+'), separator=separator)


Instruction:
Implement a Python 3 function for
resolving the dependencies for a given function name and a mapping of function dependencies.

Response:
def resolve_dependencies(func_name, dependencies):
    """ Given a function name and a mapping of function dependencies,
        returns a list of *all* the dependencies for this function. """

    def _resolve_deps(func_name, func_deps):
        """ Append dependencies recursively to func_deps (accumulator) """
        if func_name in func_deps:
            return

        func_deps.append(func_name)
        for dep in dependencies.get(func_name, []):
            _resolve_deps(dep, func_deps)

    func_deps = []
    _resolve_deps(func_name, func_deps)
    return sorted(func_deps)


Instruction:
Can you generate the documentation for the following Python 3 function
def clean_cachedir_old_entries(cachedir: StoreBackendBase, func_name: str, limit: int) -> int:
    if limit < 1:
        raise ValueError("'limit' must be greater or equal to 1")

    cache_entries = get_cachedir_entries(cachedir, func_name)
    cache_entries = sorted(cache_entries, key=lambda e: e.last_access, reverse=True)
    cache_entries_to_remove = cache_entries[limit:]
    for entry in cache_entries_to_remove:
        shutil.rmtree(entry.path, ignore_errors=True)

    return len(cache_entries_to_remove)

Response: Remove old entries from the cache


Instruction:
Make a summary of the following Python 3 code
def roll_up(
        df,
        levels: List[str],
        groupby_vars: List[str],
        extra_groupby_cols: List[str] = None,
        var_name: str = 'type',
        value_name: str = 'value',
        agg_func: str = 'sum',
        drop_levels: List[str] = None
):
    """
    Creates aggregates following a given hierarchy

    ---

    ### Parameters

    *mandatory :*
    - `levels` (*list of str*): name of the columns composing the hierarchy (from the top to the bottom level).
    - `groupby_vars` (*list of str*): name of the columns with value to aggregate.
    - `extra_groupby_cols` (*list of str*) optional: other columns used to group in each level.

    *optional :*
    - `var_name` (*str*) : name of the result variable column. By default, `“type”`.
    - `value_name` (*str*): name of the result value column. By default, `“value”`.
    - `agg_func` (*str*): name of the aggregation operation. By default, `“sum”`.
    - `drop_levels` (*list of str*): the names of the levels that you may want to discard from the output.

    ---

    ### Example

    **Input**

    |    Region |     City |  Population |
    |:---------:|:--------:|:-----------:|
    |       Idf |     Panam|         200 |
    |       Idf |   Antony |          50 |
    |      Nord |    Lille |          20 |

    ```cson
    roll_up:
      levels: ["Region", "City"]
      groupby_vars: "Population"
    ```

    **Output**

    |    Region |     City |  Population |    value |   type |
    |:---------:|:--------:|:-----------:|:--------:|:------:|
    |       Idf |     Panam|         200 |    Panam |   City |
    |       Idf |   Antony |          50 |   Antony |   City |
    |      Nord |    Lille |          20 |    Lille |   City |
    |       Idf |      Nan |         250 |      Idf | Region |
    |      Nord |      Nan |          20 |     Nord | Region |
    """
    dfs = list()
    groupby_cols_cpy = list(levels)
    levels_cpy = list(levels)
    levels_cpy.reverse()

    extra_groupby_cols = extra_groupby_cols or []
    drop_levels = drop_levels or []
    previous_level = None
    for top_level in levels_cpy:
        # Aggregation
        gb_df = getattr(
            df.groupby(groupby_cols_cpy + extra_groupby_cols)[groupby_vars],
            agg_func)().reset_index()

        # Melt-like columns
        gb_df[var_name] = top_level
        gb_df[value_name] = gb_df[top_level]
        dfs.append(gb_df)
        if previous_level in drop_levels:
            del dfs[-2]
        previous_level = top_level

        # Remove one level each time in the groupby: lowest level column needs
        # a groupby with every levels, the next level needs every one except
        # the lowest, etc. until the top level column that needs only itself
        # inside the groupby.
        groupby_cols_cpy.pop()
    return pd.concat(dfs, sort=False).reset_index()

Response: Creates an aggregate of the given levels and returns the resulting tree.


Instruction:
Given the following Python 3 function, write the documentation
def argmax(df, column: str, groups: Union[str, List[str]] = None):
    if groups is None:
        df = df[df[column] == df[column].max()].reset_index(drop=True)
    else:
        group_max = df.groupby(groups)[column].transform('max')
        df = (df
              .loc[df[column] == group_max, :]
              .drop_duplicates()
              .reset_index(drop=True)
              )
    return df

Response: Return the DataFrame with the maximum value in a column.


Instruction:
Can you implement a function in Python 3 that
returns the row of the dataframe corresponding to the minimal value in a column.

Response:
def argmin(df, column: str, groups: Union[str, List[str]] = None):
    """
    Keep the row of the data corresponding to the minimal value in a column

    ---

    ### Parameters

    *mandatory :*
    - `column` (str): name of the column containing the value you want to keep the minimum

    *optional :*
    - `groups` (*str or list(str)*): name of the column(s) used for 'groupby' logic
    (the function will return the argmax by group)
    ---

    ### Example

    **Input**

    | variable |   wave  |  year    | value |
    |:--------:|:-------:|:--------:|:-----:|
    |   toto   |  wave 1 |  2014    |  300  |
    |   toto   |  wave 1 |  2015    |  250  |
    |   toto   |  wave 1 |  2016    |  450  |

    ```cson
    argmin:
      column: 'year'
    ]
    ```

    **Output**

    | variable |   wave  |  year    | value |
    |:--------:|:-------:|:--------:|:-----:|
    |   toto   |  wave 1 |  2015    |  250  |
    """
    if groups is None:
        df = df[df[column] == df[column].min()].reset_index(drop=True)
    else:
        group_min = df.groupby(groups)[column].transform('min')
        df = (df
              .loc[df[column] == group_min, :]
              .drop_duplicates()
              .reset_index(drop=True)
              )
    return df


Instruction:
Can you tell what is the following Python 3 function doing
def fillna(df, column: str, value=None,  column_value=None):
    if column not in df.columns:
        df[column] = nan

    if value is not None and column_value is not None:
        raise ValueError('You cannot set both the parameters value and column_value')

    if value is not None:
        df[column] = df[column].fillna(value)

    if column_value is not None:
        if column_value not in df.columns:
            raise ValueError(f'"{column_value}" is not a valid column name')
        df[column] = df[column].fillna(df[column_value])

    return df

Response: Fill NaN values from a column in a DataFrame with a given value or a column_value.