Instruction:
Write a Python 3 script for
getting all git repositories within this environment

Response:
def find_repos(self, depth=10):
        '''Get all git repositories within this environment'''

        repos = []

        for root, subdirs, files in walk_dn(self.root, depth=depth):
            if 'modules' in root:
                continue
            if '.git' in subdirs:
                repos.append(root)

        return repos


Instruction:
Can you generate the documentation for the following Python 3 function
def clone(self, repo_path, destination, branch=None):
        '''Clone a repository to a destination relative to envrionment root'''

        logger.debug('Installing ' + repo_path)
        if not destination.startswith(self.env_path):
            destination = unipath(self.env_path, destination)

        if branch:
            return shell.run('git', 'clone', repo_path, '--branch', branch,
                             '--single-branch', '--recursive', destination)

        return shell.run('git', 'clone', '--recursive', repo_path, destination)

Response: Clone a repository to a destination relative to envrionment root


Instruction:
Write a Python 3 script to
clone a repository to a destination relative to envrionment root

Response:
def pull(self, repo_path, *args):
        '''Clone a repository to a destination relative to envrionment root'''

        logger.debug('Pulling ' + repo_path)
        if not repo_path.startswith(self.env_path):
            repo_path = unipath(self.env_path, repo_path)

        return shell.run('git', 'pull', *args, **{'cwd': repo_path})


Instruction:
Can you tell what is the following Python 3 function doing
def install(self, package):
        '''Install a python package using pip'''

        logger.debug('Installing ' + package)
        shell.run(self.pip_path, 'install',  package)

Response: Install a python package using pip


Instruction:
Implement a Python 3 function for
updating a python package using pip

Response:
def upgrade(self, package):
        '''Update a python package using pip'''

        logger.debug('Upgrading ' + package)
        shell.run(self.pip_path, 'install', '--upgrade', '--no-deps', package)
        shell.run(self.pip_path, 'install', package)


Instruction:
Write a Python 3 function that can
return the nb quantiles for datas in a dataframe

Response:
def df_quantile(df, nb=100):
    """Returns the nb quantiles for datas in a dataframe
    """
    quantiles = np.linspace(0, 1., nb)
    res = pd.DataFrame()
    for q in quantiles:
        res = res.append(df.quantile(q), ignore_index=True)
    return res


Instruction:
Create a Python 3 function for
computing the average along a 1D array like ma. mean but with a representativity coefficient

Response:
def mean(a, rep=0.75, **kwargs):
    """Compute the average along a 1D array like ma.mean,
    but with a representativity coefficient : if ma.count(a)/ma.size(a)>=rep,
    then the result is a masked value
    """
    return rfunc(a, ma.mean, rep, **kwargs)


Instruction:
Create a Python 3 function to
compute the max along a 1D array like ma. mean and a representativity coefficient coefficient.

Response:
def max(a, rep=0.75, **kwargs):
    """Compute the max along a 1D array like ma.mean,
    but with a representativity coefficient : if ma.count(a)/ma.size(a)>=rep,
    then the result is a masked value
    """
    return rfunc(a, ma.max, rep, **kwargs)


Instruction:
Can you generate the documentation for the following Python 3 function
def min(a, rep=0.75, **kwargs):
    return rfunc(a, ma.min, rep, **kwargs)

Response: Compute the minimum value along a 1D array like ma. mean but with a representativity coefficient coefficient.


Instruction:
Can you tell what is the following Python 3 function doing
def rfunc(a, rfunc=None, rep=0.75, **kwargs):
    if float(ma.count(a)) / ma.size(a) < rep:
        return ma.masked
    else:
        if rfunc is None:
            return a
        return rfunc(a, **kwargs)

Response:
Applies func on a if a comes with a representativity coefficient rep returns a masked array
    i. e. ma. count ( a ) > rep.


Instruction:
Explain what the following Python 3 code does
def rmse(a, b):
    return np.sqrt(np.square(a - b).mean())

Response: Returns the root mean square error betwwen a and b


Instruction:
Can you generate the documentation for the following Python 3 function
def nmse(a, b):
    return np.square(a - b).mean() / (a.mean() * b.mean())

Response: Returns the normalized mean square error of a and b


Instruction:
Can you generate the documentation for the following Python 3 function
def mfbe(a, b):
    return 2 * bias(a, b) / (a.mean() + b.mean())

Response: Returns the mean fractionalized bias error between two data sets.


Instruction:
Can you tell what is the following Python 3 function doing
def fa(a, b, alpha=2):
    return np.sum((a > b / alpha) & (a < b * alpha), dtype=float) / len(a) * 100

Response: Returns the factor of alpha when a > b


Instruction:
How would you implement a function in Python 3 that
returns the factor of exceedance

Response:
def foex(a, b):
    """Returns the factor of exceedance
    """
    return (np.sum(a > b, dtype=float) / len(a) - 0.5) * 100


Instruction:
Here you have a function in Python 3, explain what it does
def correlation(a, b):
    diff1 = a - a.mean()
    diff2 = b - b.mean()
    return (diff1 * diff2).mean() / (np.sqrt(np.square(diff1).mean() * np.square(diff2).mean()))

Response:
Computes the correlation between a and b says the Pearson s correlation
    coefficient R


Instruction:
How would you code a function in Python 3 to
figure of merit in time

Response:
def fmt(a, b):
    """Figure of merit in time
    """
    return 100 * np.min([a, b], axis=0).sum() / np.max([a, b], axis=0).sum()


Instruction:
Can you tell what is the following Python 3 function doing
def fullStats(a, b):

    stats = [
        ['bias', 'Bias', bias(a, b)],
        ['stderr', 'Standard Deviation Error', stderr(a, b)],
        ['mae', 'Mean Absolute Error', mae(a, b)],
        ['rmse', 'Root Mean Square Error', rmse(a, b)],
        ['nmse', 'Normalized Mean Square Error', nmse(a, b)],
        ['mfbe', 'Mean Fractionalized bias Error', mfbe(a, b)],
        ['fa2', 'Factor of Two', fa(a, b, 2)],
        ['foex', 'Factor of Exceedance', foex(a, b)],
        ['correlation', 'Correlation R', correlation(a, b)],
        ['determination', 'Coefficient of Determination r2', determination(a, b)],
        ['gmb', 'Geometric Mean Bias', gmb(a, b)],
        ['gmv', 'Geometric Mean Variance', gmv(a, b)],
        ['fmt', 'Figure of Merit in Time', fmt(a, b)]
    ]
    rec = np.rec.fromrecords(stats, names=('stat', 'description', 'result'))
    df = pd.DataFrame.from_records(rec, index='stat')
    return df

Response: Performs several stats on a against b typically a is the predictions array and b is the observations array and a is the observations array and b is the observations array


Instruction:
Can you generate a brief explanation for the following Python 3 code
def site_path(self):
        '''Path to environments site-packages'''

        if platform == 'win':
            return unipath(self.path, 'Lib', 'site-packages')

        py_ver = 'python{0}'.format(sys.version[:3])
        return unipath(self.path, 'lib', py_ver, 'site-packages')

Response: Path to site - packages


Instruction:
Can you create a Python 3 function that
activates the current python environment.

Response:
def _activate(self):
        '''
        Do some serious mangling to the current python environment...
        This is necessary to activate an environment via python.
        '''

        old_syspath = set(sys.path)
        site.addsitedir(self.site_path)
        site.addsitedir(self.bin_path)
        new_syspaths = set(sys.path) - old_syspath
        for path in new_syspaths:
            sys.path.remove(path)
            sys.path.insert(1, path)

        if not hasattr(sys, 'real_prefix'):
            sys.real_prefix = sys.prefix

        sys.prefix = self.path


Instruction:
Can you write a function in Python 3 where it
removes this environment and all of its children.

Response:
def remove(self):
        '''
        Remove this environment
        '''
        self.run_hook('preremove')
        utils.rmtree(self.path)
        self.run_hook('postremove')


Instruction:
Can you generate the documentation for the following Python 3 function
def command(self):
        '''Command used to launch this application module'''

        cmd = self.config.get('command', None)
        if cmd is None:
            return

        cmd = cmd[platform]
        return cmd['path'], cmd['args']

Response: Command used to launch this application module


Instruction:
Write a Python 3 script to
create a virtual environment.

Response:
def create(name_or_path=None, config=None):
    '''Create a virtual environment. You can pass either the name of a new
    environment to create in your CPENV_HOME directory OR specify a full path
    to create an environment outisde your CPENV_HOME.

    Create an environment in CPENV_HOME::

        >>> cpenv.create('myenv')

    Create an environment elsewhere::

        >>> cpenv.create('~/custom_location/myenv')

    :param name_or_path: Name or full path of environment
    :param config: Environment configuration including dependencies etc...
    '''

    # Get the real path of the environment
    if utils.is_system_path(name_or_path):
        path = unipath(name_or_path)
    else:
        path = unipath(get_home_path(), name_or_path)

    if os.path.exists(path):
        raise OSError('{} already exists'.format(path))

    env = VirtualEnvironment(path)
    utils.ensure_path_exists(env.path)

    if config:
        if utils.is_git_repo(config):
            Git('').clone(config, env.path)
        else:
            shutil.copy2(config, env.config_path)
    else:
        with open(env.config_path, 'w') as f:
            f.write(defaults.environment_config)

    utils.ensure_path_exists(env.hook_path)
    utils.ensure_path_exists(env.modules_path)

    env.run_hook('precreate')

    virtualenv.create_environment(env.path)
    if not utils.is_home_environment(env.path):
        EnvironmentCache.add(env)
        EnvironmentCache.save()

    try:
        env.update()
    except:
        utils.rmtree(path)
        logger.debug('Failed to update, rolling back...')
        raise
    else:
        env.run_hook('postcreate')

    return env


Instruction:
Create a Python 3 function for
removing an environment or module

Response:
def remove(name_or_path):
    '''Remove an environment or module

    :param name_or_path: name or path to environment or module
    '''

    r = resolve(name_or_path)
    r.resolved[0].remove()

    EnvironmentCache.discard(r.resolved[0])
    EnvironmentCache.save()


Instruction:
Implement a Python 3 function for
activating and launches a module

Response:
def launch(module_name, *args, **kwargs):
    '''Activates and launches a module

    :param module_name: name of module to launch
    '''

    r = resolve(module_name)
    r.activate()
    mod = r.resolved[0]
    mod.launch(*args, **kwargs)


Instruction:
Write a Python 3 function that can
deactivate an environment by restoring all env vars to a clean state stored prior to activating environments

Response:
def deactivate():
    '''Deactivates an environment by restoring all env vars to a clean state
    stored prior to activating environments
    '''

    if 'CPENV_ACTIVE' not in os.environ or 'CPENV_CLEAN_ENV' not in os.environ:
        raise EnvironmentError('Can not deactivate environment...')

    utils.restore_env_from_file(os.environ['CPENV_CLEAN_ENV'])


Instruction:
Create a Python 3 function for
returning your home path... CPENV_HOME env var OR ~/.cpenv

Response:
def get_home_path():
    ''':returns: your home path...CPENV_HOME env var OR ~/.cpenv'''

    home = unipath(os.environ.get('CPENV_HOME', '~/.cpenv'))
    home_modules = unipath(home, 'modules')
    if not os.path.exists(home):
        os.makedirs(home)
    if not os.path.exists(home_modules):
        os.makedirs(home_modules)
    return home


Instruction:
Write a Python 3 script for
returning a list of paths in CPENV_MODULES env var and CPENV_HOME / modules

Response:
def get_module_paths():
    ''':returns: paths in CPENV_MODULES env var and CPENV_HOME/modules'''

    module_paths = []

    cpenv_modules_path = os.environ.get('CPENV_MODULES', None)
    if cpenv_modules_path:
        module_paths.extend(cpenv_modules_path.split(os.pathsep))

    module_paths.append(unipath(get_home_path(), 'modules'))

    return module_paths


Instruction:
Can you write a function in Python 3 where it
returns a list of all known virtual environments as . This includes those in CPENV_HOME and any others that are cached. This includes those in CPENV_HOME and any others that are not cached.

Response:
def get_environments():
    '''Returns a list of all known virtual environments as
    :class:`VirtualEnvironment` instances. This includes those in CPENV_HOME
    and any others that are cached(created by the current user or activated
    once by full path.)
    '''

    environments = set()

    cwd = os.getcwd()
    for d in os.listdir(cwd):

        if d == 'environment.yml':
            environments.add(VirtualEnvironment(cwd))
            continue

        path = unipath(cwd, d)
        if utils.is_environment(path):
            environments.add(VirtualEnvironment(path))

    home = get_home_path()
    for d in os.listdir(home):

        path = unipath(home, d)
        if utils.is_environment(path):
            environments.add(VirtualEnvironment(path))

    for env in EnvironmentCache:
        environments.add(env)

    return sorted(list(environments), key=lambda x: x.name)


Instruction:
Implement a Python 3 function for
returning a list of available modules.

Response:
def get_modules():
    '''Returns a list of available modules.'''

    modules = set()

    cwd = os.getcwd()
    for d in os.listdir(cwd):

        if d == 'module.yml':
            modules.add(Module(cwd))

        path = unipath(cwd, d)
        if utils.is_module(path):
            modules.add(Module(cwd))

    module_paths = get_module_paths()
    for module_path in module_paths:
        for d in os.listdir(module_path):

            path = unipath(module_path, d)
            if utils.is_module(path):
                modules.add(Module(path))

    return sorted(list(modules), key=lambda x: x.name)


Instruction:
Can you implement a function in Python 3 that
returns a list of active : class:`Module` s or [ ]

Response:
def get_active_modules():
    ''':returns: a list of active :class:`Module` s or []'''

    modules = os.environ.get('CPENV_ACTIVE_MODULES', None)
    if modules:
        modules = modules.split(os.pathsep)
        return [Module(module) for module in modules]

    return []


Instruction:
Write a Python 3 function that can
add a module to CPENV_ACTIVE_MODULES environment variable

Response:
def add_active_module(module):
    '''Add a module to CPENV_ACTIVE_MODULES environment variable'''

    modules = set(get_active_modules())
    modules.add(module)
    new_modules_path = os.pathsep.join([m.path for m in modules])
    os.environ['CPENV_ACTIVE_MODULES'] = str(new_modules_path)


Instruction:
Write a Python 3 function for
removing a module from CPENV_ACTIVE_MODULES environment variable

Response:
def rem_active_module(module):
    '''Remove a module from CPENV_ACTIVE_MODULES environment variable'''

    modules = set(get_active_modules())
    modules.discard(module)
    new_modules_path = os.pathsep.join([m.path for m in modules])
    os.environ['CPENV_ACTIVE_MODULES'] = str(new_modules_path)


Instruction:
Can you create a Python 3 function that
formats a list of environments and modules for terminal output

Response:
def format_objects(objects, children=False, columns=None, header=True):
    '''Format a list of environments and modules for terminal output'''

    columns = columns or ('NAME', 'TYPE', 'PATH')
    objects = sorted(objects, key=_type_and_name)
    data = []
    for obj in objects:
        if isinstance(obj, cpenv.VirtualEnvironment):
            data.append(get_info(obj))
            modules = obj.get_modules()
            if children and modules:
                for mod in modules:
                    data.append(get_info(mod, indent=2, root=obj.path))
        else:
            data.append(get_info(obj))

    maxes = [len(max(col, key=len)) for col in zip(*data)]
    tmpl = '{:%d}  {:%d}  {:%d}' % tuple(maxes)
    lines = []
    if header:
        lines.append('\n' + bold_blue(tmpl.format(*columns)))

    for obj_data in data:
        lines.append(tmpl.format(*obj_data))

    return '\n'.join(lines)


Instruction:
Can you create a Python 3 function that
lists available environments and modules

Response:
def list_():
    '''List available environments and modules'''

    environments = cpenv.get_environments()
    modules = cpenv.get_modules()
    click.echo(format_objects(environments + modules, children=True))


Instruction:
Here you have a function in Python 3, explain what it does
def create(name_or_path, config):
    '''Create a new environment.'''

    if not name_or_path:
        ctx = click.get_current_context()
        click.echo(ctx.get_help())
        examples = (
            '\nExamples:\n'
            '    cpenv create my_env\n'
            '    cpenv create ./relative/path/to/my_env\n'
            '    cpenv create my_env --config ./relative/path/to/config\n'
            '    cpenv create my_env --config git@github.com:user/config.git\n'
        )
        click.echo(examples)
        return

    click.echo(
        blue('Creating a new virtual environment ' + name_or_path)
    )
    try:
        env = cpenv.create(name_or_path, config)
    except Exception as e:
        click.echo(bold_red('FAILED TO CREATE ENVIRONMENT!'))
        click.echo(e)
    else:
        click.echo(bold_green('Successfully created environment!'))
    click.echo(blue('Launching subshell'))

    cpenv.activate(env)
    shell.launch(env.name)

Response: Create a new virtual environment.


Instruction:
Implement a Python 3 function for
listing available environments and modules

Response:
def list_():
    '''List available environments and modules'''

    click.echo('Cached Environments')

    environments = list(EnvironmentCache)
    click.echo(format_objects(environments, children=False))


Instruction:
How would you explain what the following Python 3 function does
def add(path):
    '''Add an environment to the cache. Allows you to activate the environment
    by name instead of by full path'''

    click.echo('\nAdding {} to cache......'.format(path), nl=False)
    try:
        r = cpenv.resolve(path)
    except Exception as e:
        click.echo(bold_red('FAILED'))
        click.echo(e)
        return

    if isinstance(r.resolved[0], cpenv.VirtualEnvironment):
        EnvironmentCache.add(r.resolved[0])
        EnvironmentCache.save()
        click.echo(bold_green('OK!'))

Response: Add an environment to the cache. Allows you to activate the environment by name instead of by full path.


Instruction:
Write a Python 3 function that can
remove a cached environment.

Response:
def remove(path):
    '''Remove a cached environment. Removed paths will no longer be able to
    be activated by name'''

    r = cpenv.resolve(path)
    if isinstance(r.resolved[0], cpenv.VirtualEnvironment):
        EnvironmentCache.discard(r.resolved[0])
        EnvironmentCache.save()


Instruction:
Can you tell what is the following Python 3 function doing
def create(name_or_path, config):
    '''Create a new template module.

    You can also specify a filesystem path like "./modules/new_module"
    '''

    click.echo('Creating module {}...'.format(name_or_path), nl=False)
    try:
        module = cpenv.create_module(name_or_path, config)
    except Exception as e:
        click.echo(bold_red('FAILED'))
        raise
    else:
        click.echo(bold_green('OK!'))
        click.echo('Browse to your new module and make some changes.')
        click.echo("When you're ready add the module to an environment:")
        click.echo('    cpenv module add my_module ./path/to/my_module')
        click.echo('Or track your module on git and add it directly from the repo:')
        click.echo('    cpenv module add my_module git@github.com:user/my_module.git')

Response: Create a new template module.


Instruction:
Can you create a Python 3 function that
adds a module to an environment. PATH can be a git repository path or a filesystem path. branch can be a git repository path or a filesystem path. type can be local or shared

Response:
def add(name, path, branch, type):
    '''Add a module to an environment. PATH can be a git repository path or
    a filesystem path. '''

    if not name and not path:
        ctx = click.get_current_context()
        click.echo(ctx.get_help())
        examples = (
            '\nExamples:\n'
            '    cpenv module add my_module ./path/to/my_module\n'
            '    cpenv module add my_module git@github.com:user/my_module.git'
            '    cpenv module add my_module git@github.com:user/my_module.git --branch=master --type=shared'
        )
        click.echo(examples)
        return

    if not name:
        click.echo('Missing required argument: name')
        return

    if not path:
        click.echo('Missing required argument: path')

    env = cpenv.get_active_env()
    if type=='local':
        if not env:
            click.echo('\nActivate an environment to add a local module.\n')
            return

        if click.confirm('\nAdd {} to active env {}?'.format(name, env.name)):
            click.echo('Adding module...', nl=False)
            try:
                env.add_module(name, path, branch)
            except:
                click.echo(bold_red('FAILED'))
                raise
            else:
                click.echo(bold_green('OK!'))

        return

    module_paths = cpenv.get_module_paths()
    click.echo('\nAvailable module paths:\n')
    for i, mod_path in enumerate(module_paths):
        click.echo('    {}. {}'.format(i, mod_path))
    choice = click.prompt(
        'Where do you want to add your module?',
        type=int,
        default=0
    )
    module_root = module_paths[choice]
    module_path = utils.unipath(module_root, name)
    click.echo('Creating module {}...'.format(module_path), nl=False)
    try:
        cpenv.create_module(module_path, path, branch)
    except:
        click.echo(bold_red('FAILED'))
        raise
    else:
        click.echo(bold_green('OK!'))


Instruction:
Can you tell what is the following Python 3 function doing
def remove(name, local):
    '''Remove a module named NAME. Will remove the first resolved module named NAME. You can also specify a full path to a module. Use the --local option
    to ensure removal of modules local to the currently active environment.'''

    click.echo()
    if not local: # Use resolver to find module
        try:
            r = cpenv.resolve(name)
        except cpenv.ResolveError as e:
            click.echo(e)
            return

        obj = r.resolved[0]
    else: # Try to find module in active environment
        env = cpenv.get_active_env()
        if not env:
            click.echo('You must activate an env to remove local modules')
            return

        mod = env.get_module(name)
        if not mod:
            click.echo('Failed to resolve module: ' + name)
            return

        obj = mod

    if isinstance(obj, cpenv.VirtualEnvironment):
        click.echo('{} is an environment. Use `cpenv remove` instead.')
        return

    click.echo(format_objects([obj]))
    click.echo()

    user_confirmed = click.confirm(
        red('Are you sure you want to remove this module?')
    )
    if user_confirmed:
        click.echo('Attempting to remove...', nl=False)

        try:
            obj.remove()
        except Exception as e:
            click.echo(bold_red('FAILED'))
            click.echo(e)
        else:
            click.echo(bold_green('OK!'))

Response: Remove a module named NAME. Will remove the first resolved module named NAME.


Instruction:
Can you implement a function in Python 3 that
copies a global module to the active environment.

Response:
def localize(name):
    '''Copy a global module to the active environment.'''

    env = cpenv.get_active_env()
    if not env:
        click.echo('You need to activate an environment first.')
        return

    try:
        r = cpenv.resolve(name)
    except cpenv.ResolveError as e:
        click.echo('\n' + str(e))

    module = r.resolved[0]
    if isinstance(module, cpenv.VirtualEnvironment):
        click.echo('\nCan only localize a module not an environment')
        return

    active_modules = cpenv.get_active_modules()
    if module in active_modules:
        click.echo('\nCan not localize an active module.')
        return

    if module in env.get_modules():
        click.echo('\n{} is already local to {}'.format(module.name, env.name))
        return

    if click.confirm('\nAdd {} to env {}?'.format(module.name, env.name)):
        click.echo('Adding module...', nl=False)
        try:
            module = env.add_module(module.name, module.path)
        except:
            click.echo(bold_red('FAILED'))
            raise
        else:
            click.echo(bold_green('OK!'))

    click.echo('\nActivate the localize module:')
    click.echo('    cpenv activate {} {}'.format(env.name, module.name))


Instruction:
Write a Python 3 function for
resolving VirtualEnvironments with a relative or absolute path

Response:
def path_resolver(resolver, path):
    '''Resolves VirtualEnvironments with a relative or absolute path'''

    path = unipath(path)

    if is_environment(path):
        return VirtualEnvironment(path)

    raise ResolveError


Instruction:
Can you implement a function in Python 3 that
resolves VirtualEnvironments in CPENV_HOME

Response:
def home_resolver(resolver, path):
    '''Resolves VirtualEnvironments in CPENV_HOME'''

    from .api import get_home_path

    path = unipath(get_home_path(), path)

    if is_environment(path):
        return VirtualEnvironment(path)

    raise ResolveError


Instruction:
Create a Python 3 function to
resolve VirtualEnvironments in EnvironmentCache

Response:
def cache_resolver(resolver, path):
    '''Resolves VirtualEnvironments in EnvironmentCache'''

    env = resolver.cache.find(path)
    if env:
        return env

    raise ResolveError


Instruction:
How would you explain what the following Python 3 function does
def module_resolver(resolver, path):
    '''Resolves module in previously resolved environment.'''

    if resolver.resolved:

        if isinstance(resolver.resolved[0], VirtualEnvironment):
            env = resolver.resolved[0]
            mod = env.get_module(path)

            if mod:
                return mod

    raise ResolveError

Response: Resolves module in previously resolved environment.


Instruction:
Can you write a function in Python 3 where it
resolves modules in CPENV_MODULES path and CPENV_HOME / modules

Response:
def modules_path_resolver(resolver, path):
    '''Resolves modules in CPENV_MODULES path and CPENV_HOME/modules'''

    from .api import get_module_paths

    for module_dir in get_module_paths():
        mod_path = unipath(module_dir, path)

        if is_module(mod_path):
            return Module(mod_path)

    raise ResolveError


Instruction:
How would you code a function in Python 3 to
resolve modules in currently active environment.

Response:
def active_env_module_resolver(resolver, path):
    '''Resolves modules in currently active environment.'''

    from .api import get_active_env

    env = get_active_env()
    if not env:
        raise ResolveError

    mod = env.get_module(path)
    if not mod:
        raise ResolveError

    return mod


Instruction:
Can you generate the documentation for the following Python 3 function
def redirect_resolver(resolver, path):
    '''Resolves environment from .cpenv file...recursively walks up the tree
    in attempt to find a .cpenv file'''

    if not os.path.exists(path):
        raise ResolveError

    if os.path.isfile(path):
        path = os.path.dirname(path)

    for root, _, _ in walk_up(path):
        if is_redirecting(root):
            env_paths = redirect_to_env_paths(unipath(root, '.cpenv'))
            r = Resolver(*env_paths)
            return r.resolve()

    raise ResolveError

Response:
Resolves environment from. cpenv file... recursively walks up the tree
    in attempt to find a. cpenv file and returns a dict of environment paths to the environment paths that were redirected.


Instruction:
Make a summary of the following Python 3 code
def _engine_affinity(obj):
    from distob import engine
    this_engine = engine.eid
    if isinstance(obj, numbers.Number) or obj is None:
        return (this_engine, 0)
    elif hasattr(obj, '__engine_affinity__'):
        # This case includes Remote subclasses and DistArray
        return obj.__engine_affinity__
    else:
        return (this_engine, _rough_size(obj))

Response: Which engine or engines are preferred for processing this object


Instruction:
Write a Python 3 script to
move an input to a new location.

Response:
def _ufunc_move_input(obj, location, bshape):
    """Copy ufunc input `obj` to new engine location(s) unless obj is scalar.

    If the input is requested to be distributed to multiple engines, this
    function will also take care of broadcasting along the distributed axis.

    If the input obj is a scalar, it will be passed through unchanged.

    Args:
      obj (array_like or scalar): one of the inputs to a ufunc
      location (integer or tuple): If an integer, this specifies a single
        engine id to which an array input should be moved. If it is a tuple,
        location[0] is a list of engine ids for distributing the array input
        and location[1] an integer indicating which axis should be distributed.
      bshape (tuple): The shape to which the input will ultimately be broadcast

    Returns:
      array_like or RemoteArray or DistArray or scalar
    """
    if (not hasattr(type(obj), '__array_interface__') and
            not isinstance(obj, Remote) and
            (isinstance(obj, string_types) or
             not isinstance(obj, Sequence))):
        # then treat it as a scalar
        return obj
    from distob import engine
    this_engine = engine.eid
    if location == this_engine:
        # move obj to the local host, if not already here
        if isinstance(obj, Remote) or isinstance(obj, DistArray):
            return gather(obj)
        else:
            return obj
    elif isinstance(location, numbers.Integral):
        # move obj to a single remote engine
        if isinstance(obj, Remote) and obj._ref.id.engine == location:
            #print('no data movement needed!')
            return obj
        obj = gather(obj)
        return _directed_scatter(obj, axis=None, destination=location)
    else:
        # location is a tuple (list of engine ids, distaxis) indicating that
        # obj should be distributed.
        engine_ids, distaxis = location
        if not isinstance(obj, DistArray):
            gather(obj)
            if isinstance(obj, Sequence):
                obj = np.array(obj)
        if obj.ndim < len(bshape):
            ix = (np.newaxis,)*(len(bshape)-obj.ndim) + (slice(None),)*obj.ndim
            obj = obj[ix]
        if (isinstance(obj, DistArray) and distaxis == obj._distaxis and
                engine_ids == [ra._ref.id.engine for ra in obj._subarrays]):
            #print('no data movement needed!')
            return obj
        obj = gather(obj)
        if obj.shape[distaxis] == 1:
            # broadcast this axis across engines
            subarrays = [_directed_scatter(obj, None, m) for m in engine_ids]
            return DistArray(subarrays, distaxis)
        else:
            return _directed_scatter(obj, distaxis, destination=engine_ids)


Instruction:
Here you have a function in Python 3, explain what it does
def _ufunc_dispatch(ufunc, method, i, inputs, **kwargs):
    #__print_ufunc(ufunc, method, i, inputs, **kwargs)
    if 'out' in kwargs and kwargs['out'] is not None:
        raise Error('for distributed ufuncs `out=` is not yet implemented')
    nin = 2 if ufunc is np.dot else ufunc.nin
    if nin is 1 and method == '__call__':
        return vectorize(ufunc.__call__)(inputs[0], **kwargs)
    elif nin is 2 and method == '__call__':
        from distob import engine
        here = engine.eid
        # Choose best location for the computation, possibly distributed:
        locs, weights = zip(*[_engine_affinity(a) for a in inputs])
        # for DistArrays, adjust preferred distaxis to account for broadcasting
        bshape = _broadcast_shape(*inputs)
        locs = list(locs)
        for i, loc in enumerate(locs):
            if isinstance(loc, _TupleType):
                num_new_axes = len(bshape) - inputs[i].ndim
                if num_new_axes > 0:
                    locs[i] = (locs[i][0], locs[i][1] + num_new_axes)
        if ufunc is np.dot:
            locs = [here if isinstance(m, _TupleType) else m for m in locs]
        if locs[0] == locs[1]:
            location = locs[0]
        else:
            # TODO: More accurately penalize the increased data movement if we
            # choose to distribute an axis that requires broadcasting.
            smallest = 0 if weights[0] <= weights[1] else 1
            largest = 1 - smallest
            if locs[0] is here or locs[1] is here:
                location = here if weights[0] == weights[1] else locs[largest]
            else:
                # Both inputs are on remote engines. With the current
                # implementation, data on one remote engine can only be moved
                # to another remote engine via the client. Cost accordingly:
                if weights[smallest]*2 < weights[largest] + weights[smallest]:
                    location = locs[largest]
                else:
                    location = here
        # Move both inputs to the chosen location:
        inputs = [_ufunc_move_input(a, location, bshape) for a in inputs]
        # Execute computation:
        if location is here:
            return ufunc.__call__(inputs[0], inputs[1], **kwargs)
        else:
            if isinstance(location, numbers.Integral):
                # location is a single remote engine
                return call(ufunc.__call__, inputs[0], inputs[1], **kwargs)
            else:
                # location is a tuple (list of engine ids, distaxis) implying
                # that the moved inputs are now distributed arrays (or scalar)
                engine_ids, distaxis = location
                n = len(engine_ids)
                is_dist = tuple(isinstance(a, DistArray) for a in inputs)
                assert(is_dist[0] or is_dist[1])
                for i in 0, 1:
                    if is_dist[i]:
                        ndim = inputs[i].ndim
                        assert(inputs[i]._distaxis == distaxis)
                        assert(inputs[i]._n == n)
                def _remote_ucall(inputs, **kwargs):
                    """(Executed on a remote or local engine) call the ufunc"""
                    return ufunc.__call__(inputs[0], inputs[1], **kwargs)
                results = []
                kwargs = kwargs.copy()
                kwargs['block'] = False
                kwargs['prefer_local'] = False
                for j in range(n):
                    subinputs = tuple(inputs[i]._subarrays[j] if
                            is_dist[i] else inputs[i] for i in (0, 1))
                    results.append(call(_remote_ucall, subinputs, **kwargs))
                results = [convert_result(ar) for ar in results]
                return DistArray(results, distaxis)
    elif ufunc.nin > 2:
        raise Error(u'Distributing ufuncs with >2 inputs is not yet supported')
    else:
        raise Error(u'Distributed ufunc.%s() is not yet implemented' % method)

Response: Dispatches a ufunc to the local host or remote engine.


Instruction:
Can you generate the documentation for the following Python 3 function
def transpose(a, axes=None):
    if isinstance(a, np.ndarray):
        return np.transpose(a, axes)
    elif isinstance(a, RemoteArray):
        return a.transpose(*axes)
    elif isinstance(a, Remote):
        return _remote_to_array(a).transpose(*axes)
    elif isinstance(a, DistArray):
        if axes is None:
            axes = range(a.ndim - 1, -1, -1)
        axes = list(axes)
        if len(set(axes)) < len(axes):
            raise ValueError("repeated axis in transpose")
        if sorted(axes) != list(range(a.ndim)):
            raise ValueError("axes don't match array")
        distaxis = a._distaxis
        new_distaxis = axes.index(distaxis)
        new_subarrays = [ra.transpose(*axes) for ra in a._subarrays]
        return DistArray(new_subarrays, new_distaxis)
    else:
        return np.transpose(a, axes)

Response: Returns a view of the array with axes transposed.


Instruction:
Make a summary of the following Python 3 code
def rollaxis(a, axis, start=0):
    if isinstance(a, np.ndarray):
        return np.rollaxis(a, axis, start)
    if axis not in range(a.ndim):
        raise ValueError(
                'rollaxis: axis (%d) must be >=0 and < %d' % (axis, a.ndim))
    if start not in range(a.ndim + 1):
        raise ValueError(
                'rollaxis: start (%d) must be >=0 and < %d' % (axis, a.ndim+1))
    axes = list(range(a.ndim))
    axes.remove(axis)
    axes.insert(start, axis)
    return transpose(a, axes)

Response: Roll the specified axis backwards until it lies in a given position.


Instruction:
Create a Python 3 function to
insert a new axis corresponding to a given position in the array shape

Response:
def expand_dims(a, axis):
    """Insert a new axis, corresponding to a given position in the array shape

    Args:
      a (array_like): Input array.
      axis (int): Position (amongst axes) where new axis is to be inserted.
    """
    if hasattr(a, 'expand_dims') and hasattr(type(a), '__array_interface__'):
        return a.expand_dims(axis)
    else:
        return np.expand_dims(a, axis)


Instruction:
Create a Python 3 function for
concatenating a sequence of arrays together.

Response:
def concatenate(tup, axis=0):
    """Join a sequence of arrays together.
    Will aim to join `ndarray`, `RemoteArray`, and `DistArray` without moving
    their data, if they happen to be on different engines.

    Args:
      tup (sequence of array_like): Arrays to be concatenated. They must have
        the same shape, except in the dimension corresponding to `axis`.
      axis (int, optional): The axis along which the arrays will be joined.

    Returns:
      res: `ndarray`, if inputs were all local
           `RemoteArray`, if inputs were all on the same remote engine
           `DistArray`, if inputs were already scattered on different engines
    """
    from distob import engine
    if len(tup) is 0:
        raise ValueError('need at least one array to concatenate')
    first = tup[0]
    others = tup[1:]
    # allow subclasses to provide their own implementations of concatenate:
    if (hasattr(first, 'concatenate') and
            hasattr(type(first), '__array_interface__')):
        return first.concatenate(others, axis)
    # convert all arguments to arrays/RemoteArrays if they are not already:
    arrays = []
    for ar in tup:
        if isinstance(ar, DistArray):
            if axis == ar._distaxis:
                arrays.extend(ar._subarrays)
            else:
                # Since not yet implemented arrays distributed on more than
                # one axis, will fetch and re-scatter on the new axis:
                arrays.append(gather(ar))
        elif isinstance(ar, RemoteArray):
            arrays.append(ar)
        elif isinstance(ar, Remote):
            arrays.append(_remote_to_array(ar))
        elif hasattr(type(ar), '__array_interface__'):
            # then treat as a local ndarray
            arrays.append(ar)
        else:
            arrays.append(np.array(ar))
    if all(isinstance(ar, np.ndarray) for ar in arrays):
        return np.concatenate(arrays, axis)
    total_length = 0
    # validate dimensions are same, except for axis of concatenation:
    commonshape = list(arrays[0].shape)
    commonshape[axis] = None # ignore this axis for shape comparison
    for ar in arrays:
        total_length += ar.shape[axis]
        shp = list(ar.shape)
        shp[axis] = None
        if shp != commonshape:
            raise ValueError('incompatible shapes for concatenation')
    # set sensible target block size if splitting subarrays further:
    blocksize = ((total_length - 1) // engine.nengines) + 1
    rarrays = []
    for ar in arrays:
        if isinstance(ar, DistArray):
            rarrays.extend(ar._subarrays)
        elif isinstance(ar, RemoteArray):
            rarrays.append(ar)
        else:
            da = _scatter_ndarray(ar, axis, blocksize)
            for ra in da._subarrays:
                rarrays.append(ra)
            del da
    del arrays
    # At this point rarrays is a list of RemoteArray to be concatenated
    eid = rarrays[0]._id.engine
    if all(ra._id.engine == eid for ra in rarrays):
        # Arrays to be joined are all on the same engine
        if eid == engine.eid:
            # Arrays are all local
            return concatenate([gather(r) for r in rarrays], axis)
        else:
            return call(concatenate, rarrays, axis)
    else:
        # Arrays to be joined are on different engines.
        # TODO: consolidate any consecutive arrays already on same engine
        return DistArray(rarrays, axis)


Instruction:
How would you code a function in Python 3 to
stack arrays in sequence vertically ( row wise ) handling RemoteArray and DistArray without moving data.

Response:
def vstack(tup):
    """Stack arrays in sequence vertically (row wise),
    handling ``RemoteArray`` and ``DistArray`` without moving data.

    Args:
      tup (sequence of array_like)

    Returns:
      res: `ndarray`, if inputs were all local
           `RemoteArray`, if inputs were all on the same remote engine
           `DistArray`, if inputs were already scattered on different engines
    """
    # Follow numpy.vstack behavior for 1D arrays:
    arrays = list(tup)
    for i in range(len(arrays)):
        if arrays[i].ndim is 1:
            arrays[i] = arrays[i][np.newaxis, :]
    return concatenate(tup, axis=0)


Instruction:
Write a Python 3 script for
stacking arrays in sequence horizontally ( column wise ) handling RemoteArray and DistArray without moving data.

Response:
def hstack(tup):
    """Stack arrays in sequence horizontally (column wise),
    handling ``RemoteArray`` and ``DistArray`` without moving data.

    Args:
      tup (sequence of array_like)

    Returns:
      res: `ndarray`, if inputs were all local
           `RemoteArray`, if inputs were all on the same remote engine
           `DistArray`, if inputs were already scattered on different engines
    """
    # Follow numpy.hstack behavior for 1D arrays:
    if all(ar.ndim is 1 for ar in tup):
        return concatenate(tup, axis=0)
    else:
        return concatenate(tup, axis=1)


Instruction:
Can you create a Python 3 function that
stacks arrays in sequence depth wise.

Response:
def dstack(tup):
    """Stack arrays in sequence depth wise (along third dimension),
    handling ``RemoteArray`` and ``DistArray`` without moving data.

    Args:
      tup (sequence of array_like)

    Returns:
      res: `ndarray`, if inputs were all local
           `RemoteArray`, if inputs were all on the same remote engine
           `DistArray`, if inputs were already scattered on different engines
    """
    # Follow numpy.dstack behavior for 1D and 2D arrays:
    arrays = list(tup)
    for i in range(len(arrays)):
        if arrays[i].ndim is 1:
            arrays[i] = arrays[i][np.newaxis, :]
        if arrays[i].ndim is 2:
            arrays[i] = arrays[i][:, :, np.newaxis]
    return concatenate(arrays, axis=2)


Instruction:
Given the following Python 3 function, write the documentation
def _broadcast_shape(*args):
    #TODO: currently incorrect result if a Sequence is provided as an input
    shapes = [a.shape if hasattr(type(a), '__array_interface__')
              else () for a in args]
    ndim = max(len(sh) for sh in shapes) # new common ndim after broadcasting
    for i, sh in enumerate(shapes):
        if len(sh) < ndim:
            shapes[i] = (1,)*(ndim - len(sh)) + sh
    return tuple(max(sh[ax] for sh in shapes) for ax in range(ndim))

Response: Return the shape that would result from broadcasting the inputs


Instruction:
Can you tell what is the following Python 3 function doing
def mean(a, axis=None, dtype=None, out=None, keepdims=False):
    if (isinstance(a, np.ndarray) or
            isinstance(a, RemoteArray) or
            isinstance(a, DistArray)):
        return a.mean(axis=axis, dtype=dtype, out=out, keepdims=keepdims)
    else:
        return np.mean(a, axis=axis, dtype=dtype, out=out, keepdims=keepdims)

Response: Compute the arithmetic mean along the specified axis.


Instruction:
Create a Python 3 function to
force update of a local cached copy of the real object

Response:
def _fetch(self):
        """forces update of a local cached copy of the real object
        (regardless of the preference setting self.cache)"""
        if not self._obcache_current:
            from distob import engine
            ax = self._distaxis
            self._obcache = concatenate([ra._ob for ra in self._subarrays], ax)
            # let subarray obcaches and main obcache be views on same memory:
            for i in range(self._n):
                ix = [slice(None)] * self.ndim
                ix[ax] = slice(self._si[i], self._si[i+1])
                self._subarrays[i]._obcache = self._obcache[tuple(ix)]
            self._obcache_current = True
            # now prefer local processing:
            self.__engine_affinity__ = (
                    engine.eid, self.__engine_affinity__[1])


Instruction:
Can you generate the documentation for the following Python 3 function
def _tosubslices(self, sl):
        N = self.shape[self._distaxis]
        start, stop, step = sl.start, sl.stop, sl.step
        if step is None:
            step = 1
        ss = []
        ms = []
        if step > 0:
            if start is None:
                start = 0
            if stop is None:
                stop = N
            subs = range(0, self._n)
            for s in subs:
                low = self._si[s]
                high = self._si[s + 1]
                first = low + ((low - start) % step)
                last = high + ((high - start) % step)
                if start < high and stop > low and first < high:
                    ss.append(s)
                    substart = max(first, start) - low
                    substop = min(last, stop) - low
                    ms.append(slice(substart, substop, step))
        elif step < 0:
            if start is None:
                start = N - 1
            if stop is None:
                stop = -1
            subs = range(self._n - 1, -1, -1)
            for s in subs:
                low = self._si[s]
                high = self._si[s + 1]
                first = high + step + ((high - start) % step)
                last = low + step + ((low - start) % step)
                if start >= low and stop < high and first >= low:
                    ss.append(s)
                    substart = min(first, start) - low
                    substop = max(last + step, stop) - low
                    if substop < 0:
                        substop = None
                    ms.append(slice(substart, substop, step))
        else:
            raise ValueError('slice step cannot be zero')
        return ss, ms

Response: Maps a slice object for whole array to slice objects for subarrays.


Instruction:
Can you create a Python 3 function that
checks if the given shapes are all the same when ignoring axis ax.

Response:
def _valid_distaxis(shapes, ax):
        """`ax` is a valid candidate for a distributed axis if the given
        subarray shapes are all the same when ignoring axis `ax`"""
        compare_shapes = np.vstack(shapes)
        if ax < compare_shapes.shape[1]:
            compare_shapes[:, ax] = -1
        return np.count_nonzero(compare_shapes - compare_shapes[0]) == 0


Instruction:
Write a Python 3 function that can
return a new DistArray with the same shape as self but with the same dimensions.

Response:
def expand_dims(self, axis):
        """Insert a new axis, at a given position in the array shape
        Args:
          axis (int): Position (amongst axes) where new axis is to be inserted.
        """
        if axis == -1:
            axis = self.ndim
        if axis <= self._distaxis:
            subaxis = axis
            new_distaxis = self._distaxis + 1
        else:
            subaxis = axis - 1
            new_distaxis = self._distaxis
        new_subarrays = [expand_dims(ra, subaxis) for ra in self._subarrays]
        return DistArray(new_subarrays, new_distaxis)


Instruction:
Can you tell what is the following Python 3 function doing
def mean(self, axis=None, dtype=None, out=None, keepdims=False):
        if axis == -1:
            axis = self.ndim
        if axis is None:
            results = vectorize(mean)(self, axis, dtype, keepdims=False)
            weights = self._sublengths
            res = np.average(results, axis=None, weights=weights)
            if keepdims:
                for i in range(self.ndim):
                    res = expand_dims(res, res.ndim)
        elif axis == self._distaxis:
            results = vectorize(mean)(self, axis, dtype, keepdims=True)
            results = gather(results)
            # Average manually (np.average doesn't preserve ndarray subclasses)
            weights = (np.array(self._sublengths, dtype=np.float64) /
                       sum(self._sublengths))
            ix = [slice(None)] * self.ndim
            ix[axis] = 0
            res = results[ix] * weights[0]
            for i in range(1, self._n):
                ix[axis] = i
                res = res + results[ix] * weights[i]
            if keepdims:
                res = expand_dims(res, axis)
        else:
            res = vectorize(mean)(self, axis, dtype, keepdims=False)
            if keepdims:
                res = expand_dims(res, axis)
        if out is not None:
            out[:] = res
        return res

Response: Compute the arithmetic mean along the specified axis.


Instruction:
Write a Python 3 function that can
return True if successful False if failure

Response:
def run(*args, **kwargs):
    '''Returns True if successful, False if failure'''

    kwargs.setdefault('env', os.environ)
    kwargs.setdefault('shell', True)

    try:
        subprocess.check_call(' '.join(args), **kwargs)
        return True
    except subprocess.CalledProcessError:
        logger.debug('Error running: {}'.format(args))
        return False


Instruction:
Implement a Python 3 function for
returning a command to launch a subshell

Response:
def cmd():
    '''Return a command to launch a subshell'''

    if platform == 'win':
        return ['cmd.exe', '/K']

    elif platform == 'linux':
        ppid = os.getppid()
        ppid_cmdline_file = '/proc/{0}/cmdline'.format(ppid)
        try:
            with open(ppid_cmdline_file) as f:
                cmd = f.read()
            if cmd.endswith('\x00'):
                cmd = cmd[:-1]
            cmd = cmd.split('\x00')
            return cmd + [binpath('subshell.sh')]
        except:
            cmd = 'bash'

    else:
        cmd = 'bash'

    return [cmd, binpath('subshell.sh')]


Instruction:
Implement a function in Python 3 to
generate a prompt with a given prefix

Response:
def prompt(prefix=None, colored=True):
    '''Generate a prompt with a given prefix

    linux/osx: [prefix] user@host cwd $
          win: [prefix] cwd:
    '''

    if platform == 'win':
        return '[{0}] $P$G'.format(prefix)
    else:
        if colored:
            return (
                '[{0}] '  # White prefix
                '\\[\\033[01;32m\\]\\u@\\h\\[\\033[00m\\] '  # Green user@host
                '\\[\\033[01;34m\\]\\w $ \\[\\033[00m\\]'  # Blue cwd $
            ).format(prefix)
        return '[{0}] \\u@\\h \\w $ '.format(prefix)


Instruction:
Can you implement a function in Python 3 that
adds a file to the file repository.

Response:
def add_file(self, file, **kwargs):
        """Append a file to file repository.

        For file monitoring, monitor instance needs file.
        Please put the name of file to `file` argument.

        :param file: the name of file you want monitor.

        """

        if os.access(file, os.F_OK):

            if file in self.f_repository:
                raise DuplicationError("file already added.")

            self.f_repository.append(file)

        else:
            raise IOError("file not found.")


Instruction:
Can you generate a brief explanation for the following Python 3 code
def add_files(self, filelist, **kwargs):

        # check filelist is list type
        if not isinstance(filelist, list):
            raise TypeError("request the list type.")

        for file in filelist:
            self.add_file(file)

Response: Append files to the repository using this.


Instruction:
Here you have a function in Python 3, explain what it does
def monitor(self, sleep=5):

        manager = FileModificationObjectManager()

        timestamps = {}
        filebodies = {}

        # register original timestamp and filebody to dict
        for file in self.f_repository:
            timestamps[file] = self._get_mtime(file)
            filebodies[file] = open(file).read()

        while True:

            for file in self.f_repository:

                mtime = timestamps[file]
                fbody = filebodies[file]

                modified = self._check_modify(file, mtime, fbody)

                # file not modify -> continue
                if not modified:
                    continue

                # file modifies -> create the modification object

                new_mtime = self._get_mtime(file)
                new_fbody = open(file).read()

                obj = FileModificationObject(
                        file,
                        (mtime, new_mtime),
                        (fbody, new_fbody) )

                # overwrite new timestamp and filebody
                timestamps[file] = new_mtime
                filebodies[file] = new_fbody

                # append file modification object to manager
                manager.add_object(obj)

                # return new modification object
                yield obj

            time.sleep(sleep)

Response: Monitor file modification using timestamp and file body data and yield a FileModificationObject.


Instruction:
Write a Python 3 script for
initializing a Flask application for use with this extension.

Response:
def init_app(self, app):
        """Initialize a :class:`~flask.Flask` application for use with
        this extension.
        """
        self._jobs = []

        if not hasattr(app, 'extensions'):
            app.extensions = {}

        app.extensions['restpoints'] = self
        app.restpoints_instance = self
        app.add_url_rule('/ping', 'ping', ping)
        app.add_url_rule('/time', 'time', time)
        app.add_url_rule('/status', 'status', status(self._jobs))


Instruction:
Given the following Python 3 function, write the documentation
def add_status_job(self, job_func, name=None, timeout=3):
        job_name = job_func.__name__ if name is None else name
        job = (job_name, timeout, job_func)
        self._jobs.append(job)

Response: Adds a job to be included during status requests to the status endpoint.


Instruction:
Can you generate the documentation for the following Python 3 function
def _pipepager(text, cmd, color):
    import subprocess
    env = dict(os.environ)

    # If we're piping to less we might support colors under the
    # condition that
    cmd_detail = cmd.rsplit('/', 1)[-1].split()
    if color is None and cmd_detail[0] == 'less':
        less_flags = os.environ.get('LESS', '') + ' '.join(cmd_detail[1:])
        if not less_flags:
            env['LESS'] = '-R'
            color = True
        elif 'r' in less_flags or 'R' in less_flags:
            color = True

    if not color:
        text = strip_ansi(text)

    c = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE,
                         env=env)
    encoding = get_best_encoding(c.stdin)
    try:
        c.stdin.write(text.encode(encoding, 'replace'))
        c.stdin.close()
    except (IOError, KeyboardInterrupt):
        pass

    # Less doesn't respect ^C, but catches it for its own UI purposes (aborting
    # search or other commands inside less).
    #
    # That means when the user hits ^C, the parent process (click) terminates,
    # but less is still alive, paging the output and messing up the terminal.
    #
    # If the user wants to make the pager exit on ^C, they should set
    # `LESS='-K'`. It's not our decision to make.
    while True:
        try:
            c.wait()
        except KeyboardInterrupt:
            pass
        else:
            break

Response: Page through text by feeding it to another program.


Instruction:
Explain what the following Python 3 code does
def _get_funky(func):

    if isinstance(func, str):
        try:
            func = getattr(np, func)
        except:
            raise NameError("Nom de fonction non comprise")
    return func

Response: Renvoie un nom passé en paramètre un nom passé un paramètre un nom passé un paramètre un nom passé un paramètre un nom passé un paramètre


Instruction:
Here you have a function in Python 3, explain what it does
def profil_journalier(df, func='mean'):

    func = _get_funky(func)
    res = df.groupby(lambda x: x.hour).aggregate(func)
    return res

Response: Calculates du profil journalier un tableau un tableau de données


Instruction:
Can you generate a brief explanation for the following Python 3 code
def profil_hebdo(df, func='mean'):

    func = _get_funky(func)
    res = df.groupby(lambda x: x.weekday).aggregate(func)
    # On met des noms de jour à la place des numéros dans l'index
    res.index = [cal.day_name[i] for i in range(0,7)]
    return res

Response: A helper function to calculate a list of unique calendars in a DataFrame.


Instruction:
Here you have a function in Python 3, explain what it does
def profil_annuel(df, func='mean'):

    func = _get_funky(func)
    res = df.groupby(lambda x: x.month).aggregate(func)
    # On met des noms de mois à la place des numéros dans l'index
    res.index = [cal.month_name[i] for i in range(1,13)]
    return res

Response: A helper function to calculate un annuelle série temporelle


Instruction:
Can you implement a function in Python 3 that
formats un donnée du donnée du donnée et get_*.

Response:
def _format(noms):
    """
    Formate une donnée d'entrée pour être exploitable dans les fonctions liste_*
    et get_*.

    Paramètres:
    noms: chaîne de caractère, liste ou tuples de chaînes de caractères ou
    pandas.Series de chaînes de caractères.

    Retourne:
    Une chaînes de caractères dont chaque élément est séparé du suivant par les
    caractères ',' (simples quotes comprises)

    """
    if isinstance(noms, (list, tuple, pd.Series)):
        noms = ','.join(noms)
    noms = noms.replace(",", "','")
    return noms


Instruction:
Can you generate a brief explanation for the following Python 3 code
def date_range(debut, fin, freq):

    debut_dt = debut.replace(hour=0, minute=0, second=0, microsecond=0)
    fin_dt = fin.replace(hour=23, minute=59, second=0, microsecond=0)
    if freq in ('M', 'A'):  # Calle la fréquence sur le début de mois/année
        freq += 'S'
        debut_dt = debut_dt.replace(day=1, minute=0, second=0, microsecond=0)
        fin_dt = fin_dt.replace(day=1, minute=0, second=0, microsecond=0)
    dates_completes = pd.date_range(start=debut_dt, end=fin_dt, freq=freq)
    return dates_completes

Response: Return a list of dates in the order of the given date.


Instruction:
Write a Python 3 script for
connecting to the XAIR server.

Response:
def _connect(self):
        """
        Connexion à la base XAIR
        """

        try:
            # On passe par Oracle Instant Client avec le TNS ORA_FULL
            self.conn = cx_Oracle.connect(self._ORA_FULL)
            self.cursor = self.conn.cursor()
            print('XAIR: Connexion établie')
        except cx_Oracle.Error as e:
            print("Erreur: %s" % (e))
            raise cx_Oracle.Error('Echec de connexion')


Instruction:
Make a summary of the following Python 3 code
def liste_parametres(self, parametre=None):
        condition = ""
        if parametre:
            condition = "WHERE CCHIM='%s'" % parametre
        _sql = """SELECT CCHIM AS PARAMETRE,
        NCON AS LIBELLE,
        NOPOL AS CODE
        FROM NOM_MESURE %s ORDER BY CCHIM""" % condition
        return psql.read_sql(_sql, self.conn)

Response: Liste des parametres de la lista de la classe.


Instruction:
Here you have a function in Python 3, explain what it does
def liste_campagnes(self, campagne=None):

        condition = ""
        if campagne:
            condition = "WHERE NOM_COURT_CM='%s' """ % campagne
        _sql = """SELECT
        NOM_COURT_CM AS CAMPAGNE,
        IDENTIFIANT AS STATION,
        LIBELLE AS LIBELLE_CM,
        DATEDEB AS DEBUT,
        DATEFIN AS FIN
        FROM CAMPMES
        INNER JOIN CAMPMES_STATION USING (NOM_COURT_CM)
        INNER JOIN STATION USING (NOM_COURT_SIT)
        %s ORDER BY DATEDEB DESC""" % condition
        return psql.read_sql(_sql, self.conn)

Response: Liste des campagnes de mesure et des stations associées campagne. Si définie liste des stations associées campagne.


Instruction:
Can you generate the documentation for the following Python 3 function
def get_mesures(self, mes, debut=None, fin=None, freq='H', format=None,
                    dayfirst=False, brut=False):
        """
        Récupération des données de mesure.

        Paramètres:
        mes: Un nom de mesure ou plusieurs séparées par des virgules, une liste
            (list, tuple, pandas.Series) de noms
        debut: Chaine de caractère ou objet datetime décrivant la date de début.
            Défaut=date du jour
        fin: Chaine de caractère ou objet datetime décrivant la date de fin.
            Défaut=date de début
        freq: fréquence de temps. '15T' | 'H' | 'D' | 'M' | 'A' (15T pour quart-horaire)
        format: chaine de caractère décrivant le format des dates (ex:"%Y-%m-%d"
            pour debut/fin="2013-01-28"). Appeler pyair.date.strtime_help() pour
            obtenir la liste des codes possibles.
            Defaut="%Y-%m-%d"
        dayfirst: Si aucun format n'est fourni et que les dates sont des chaines
            de caractères, aide le décrypteur à transformer la date en objet datetime
            en spécifiant que les dates commencent par le jour (ex:11/09/2012
            pourrait être interpreté comme le 09 novembre si dayfirst=False)
        brut: si oui ou non renvoyer le dataframe brut, non invalidé, et les
            codes d'état des mesures
            Defaut=False

        Retourne:
        Un dataframe contenant toutes les mesures demandées.
        Si brut=True, renvoie le  dataframe des mesures brutes non invalidées et
        le dataframe des codes d'états.
        Le dataframe valide (net) peut être alors recalculé en faisant:
        brut, etats = xr.get_mesure(..., brut=True)
        invalides = etats_to_invalid(etats)
        net = brut.mask(invalides)

        """

        def create_index(index, freq):
            """
            Nouvel index [id, date] avec date formaté suivant le pas de temps voulu
            index: index de l'ancien dataframe, tel que [date à minuit, date à ajouter]

            """
            decalage = 1  # sert à compenser l'aberration des temps qui veut qu'on marque sur la fin d'une période (ex: à 24h, la pollution de 23 à minuit)
            if freq == 'T' or freq == '15T':
                f = pd.tseries.offsets.Minute
                decalage = 15
            if freq == 'H':
                f = pd.tseries.offsets.Hour
            if freq == 'D':
                f = pd.tseries.offsets.Day
            if freq == 'M':
                f = pd.tseries.offsets.MonthBegin
            if freq == 'A':
                f = pd.tseries.offsets.YearBegin
            else:
                f = pd.tseries.offsets.Hour
            new_index = [date + f(int(delta) - decalage) for date, delta in index]
            return new_index

        # Reformatage du champ des noms de mesure
        mes = _format(mes)

        # Analyse des champs dates
        debut = to_date(debut, dayfirst, format)
        if not fin:
            fin = debut
        else:
            fin = to_date(fin, dayfirst, format)

        # La freq de temps Q n'existe pas, on passe d'abord par une fréquence 15 minutes
        if freq in ('Q', 'T'):
            freq = '15T'

        # Sélection des champs et de la table en fonctions de la fréquence de temps souhaitée
        if freq == '15T':
            diviseur = 96
            champ_val = ','.join(['Q_M%02i AS "%i"' % (x, x * 15) for x in range(1, diviseur + 1)])
            champ_code = 'Q_ETATV'
            table = 'JOURNALIER'
        elif freq == 'H':
            diviseur = 24
            champ_val = ','.join(['H_M%02i AS "%i"' % (x, x) for x in range(1, diviseur + 1)])
            champ_code = 'H_ETAT'
            table = 'JOURNALIER'
        elif freq == 'D':
            diviseur = 1
            champ_val = 'J_M01 AS "1"'
            champ_code = 'J_ETAT'
            table = 'JOURNALIER'
        elif freq == 'M':
            diviseur = 12
            champ_val = ','.join(['M_M%02i AS "%i"' % (x, x) for x in range(1, diviseur + 1)])
            champ_code = 'M_ETAT'
            table = 'MOIS'
        elif freq == 'A':
            diviseur = 1
            champ_val = 'A_M01 AS "1"'
            champ_code = 'A_ETAT'
            table = 'MOIS'
        else:
            raise ValueError("freq doit être T, H, D, M ou A")

        if table == 'JOURNALIER':
            champ_date = 'J_DATE'
            debut_db = debut
            fin_db = fin
        else:
            champ_date = 'M_DATE'
            # Pour les freq='M' et 'A', la table contient toutes les valeurs sur une
            # année entière. Pour ne pas perturber la récupération si on passait des
            # dates en milieu d'année, on transforme les dates pour être calées en début
            # et en fin d'année. Le recadrage se fera plus loin dans le code, lors du reindex
            debut_db = debut.replace(month=1, day=1, hour=0, minute=0)
            fin_db = fin.replace(month=12, day=31, hour=23, minute=0)

        debut_db = debut_db.strftime("%Y-%m-%d")
        fin_db = fin_db.strftime("%Y-%m-%d")

        # Récupération des valeurs et codes d'états associés
        _sql = """SELECT
        IDENTIFIANT as "id",
        {champ_date} as "date",
        {champ_code} as "etat",
        {champ_val}
        FROM {table}
        INNER JOIN MESURE USING (NOM_COURT_MES)
        WHERE IDENTIFIANT IN ('{mes}')
        AND {champ_date} BETWEEN TO_DATE('{debut}', 'YYYY-MM-DD') AND TO_DATE('{fin}', 'YYYY-MM-DD')
        ORDER BY IDENTIFIANT, {champ_date} ASC""".format(champ_date=champ_date,
                                                         table=table,
                                                         champ_code=champ_code,
                                                         mes=mes,
                                                         champ_val=champ_val,
                                                         debut=debut_db,
                                                         fin=fin_db)
        ## TODO : A essayer quand la base sera en version 11g
        # _sql = """SELECT *
        # FROM ({selection})
        # UNPIVOT (IDENTIFIANT FOR VAL IN ({champ_as}))""".format(selection=_sql,
        # champ_date=champ_date,
        # champ_as=champ_as)

        # On recupere les valeurs depuis la freq dans une dataframe
        rep = psql.read_sql(_sql, self.conn)

        # On créait un multiindex pour manipuler plus facilement le dataframe
        df = rep.set_index(['id', 'date'])

        # Stack le dataframe pour mettre les colonnes en lignes, en supprimant la colonne des états
        # puis on unstack suivant l'id pour avoir les polluants en colonnes
        etats = df['etat']
        df = df.drop('etat', axis=1)
        df_stack = df.stack(dropna=False)
        df = df_stack.unstack('id')

        # Calcul d'un nouvel index avec les bonnes dates. L'index du df est
        # formé du champ date à minuit, et des noms des champs de valeurs
        # qui sont aliassés de 1 à 24 pour les heures, ... voir champ_val.
        # On aggrève alors ces 2 valeurs pour avoir des dates alignées qu'on utilise alors comme index final
        index = create_index(df.index, freq)
        df.reset_index(inplace=True, drop=True)
        df['date'] = index
        df = df.set_index(['date'])
        # Traitement des codes d'état
        # On concatène les codes d'état pour chaque polluant
        # etats = etats.sum(level=0)
        # etats = pd.DataFrame(zip(*etats.apply(list)))
        etats = etats.unstack('id')
        etats.fillna(value=MISSING_CODE * diviseur, inplace=True)
        etats = etats.sum(axis=0)
        etats = pd.DataFrame(list(zip(*etats.apply(list))))
        etats.index = df.index
        etats.columns = df.columns

        # Remplacement des valeurs aux dates manquantes par des NaN
        dates_completes = date_range(debut, fin, freq)
        df = df.reindex(dates_completes)
        etats = etats.reindex(dates_completes)

        # Invalidation par codes d'état
        # Pour chaque code d'état, regarde si oui ou non il est invalidant en le remplacant par un booléen
        invalid = etats_to_invalid(etats)

        if not brut:
            # dans le dataframe, masque toute valeur invalide par NaN
            dfn = df.mask(invalid)  # DataFrame net
            return dfn
        else:
            return df, etats

Response: Return a list of all the mesures in the current state.


Instruction:
How would you code a function in Python 3 to
return a list of manuelles of a specific site.

Response:
def get_manuelles(self, site, code_parametre, debut, fin, court=False):
        """
        Recupération des mesures manuelles (labo) pour un site

        site: numéro du site (voir fonction liste_sites_prelevement)
        code_parametre: code ISO du paramètre à rechercher (C6H6=V4)
        debut: date de début du premier prélèvement
        fin: date de fin du dernier prélèvement
        court: Renvoie un tableau au format court ou long (colonnes)

        """

        condition = "WHERE MESLA.NOPOL='%s' " % code_parametre
        condition += "AND SITMETH.NSIT=%s " % site
        condition += "AND PRELEV.DATE_DEB>=TO_DATE('%s', 'YYYY-MM-DD') " % debut
        condition += "AND PRELEV.DATE_FIN<=TO_DATE('%s', 'YYYY-MM-DD') " % fin
        if court == False:
            select = """SELECT
                        MESLA.LIBELLE AS MESURE,
                        METH.LIBELLE AS METHODE,
                        ANA.VALEUR AS VALEUR,
                        MESLA.UNITE AS UNITE,
                        ANA.CODE_QUALITE AS CODE_QUALITE,
                        ANA.DATE_ANA AS DATE_ANALYSE,
                        ANA.ID_LABO AS LABO,
                        PRELEV.DATE_DEB AS DEBUT,
                        PRELEV.DATE_FIN AS FIN,
                        ANA.COMMENTAIRE AS COMMENTAIRE,
                        SITE.LIBELLE AS SITE,
                        SITE.AXE AS ADRESSE,
                        COM.NOM_COMMUNE AS COMMUNE"""
        else:
            select = """SELECT
                        MESLA.LIBELLE AS MESURE,
                        ANA.VALEUR AS VALEUR,
                        MESLA.UNITE AS UNITE,
                        ANA.CODE_QUALITE AS CODE_QUALITE,
                        PRELEV.DATE_DEB AS DEBUT,
                        PRELEV.DATE_FIN AS FIN,
                        SITE.AXE AS ADRESSE,
                        COM.NOM_COMMUNE AS COMMUNE"""
        _sql = """%s
        FROM ANALYSE ANA
        INNER JOIN PRELEVEMENT PRELEV ON (ANA.CODE_PRELEV=PRELEV.CODE_PRELEV AND ANA.CODE_SMP=PRELEV.CODE_SMP)
        INNER JOIN MESURE_LABO MESLA ON (ANA.CODE_MES_LABO=MESLA.CODE_MES_LABO AND ANA.CODE_SMP=MESLA.CODE_SMP)
        INNER JOIN SITE_METH_PRELEV SITMETH ON (ANA.CODE_SMP=SITMETH.CODE_SMP)
        INNER JOIN METH_PRELEVEMENT METH ON (SITMETH.CODE_METH_P=METH.CODE_METH_P)
        INNER JOIN SITE_PRELEVEMENT SITE ON (SITE.NSIT=SITMETH.NSIT)
        INNER JOIN COMMUNE COM ON (COM.NINSEE=SITE.NINSEE)
        %s
        ORDER BY MESLA.NOPOL,MESLA.LIBELLE,PRELEV.DATE_DEB""" % (select, condition)
        return psql.read_sql(_sql, self.conn)


Instruction:
Can you create a Python 3 function that
returns a pandas. DataFrame with the indices of the entries in the result set.

Response:
def get_indices(self, res, debut, fin):
        """
        Récupération des indices ATMO pour un réseau donné.

        Paramètres:
        res : Nom du ou des réseaux à chercher (str, list, pandas.Series)
        debut: date de début, format YYYY-MM-JJ (str)
        fin: Date de fin, format YYYY-MM-JJ (str)

        """
        res = _format(res)

        _sql = """SELECT
        J_DATE AS "date",
        NOM_AGGLO AS "reseau",
        C_IND_CALCULE AS "indice"
        FROM RESULTAT_INDICE
        INNER JOIN GROUPE_ATMO USING (NOM_COURT_GRP)
        WHERE NOM_AGGLO IN ('%s')
        AND J_DATE BETWEEN TO_DATE('%s', 'YYYY-MM-DD') AND TO_DATE('%s', 'YYYY-MM-DD') """ % (res, debut, fin)
        rep = psql.read_sql(_sql, self.conn)
        df = rep.set_index(['reseau', 'date'])
        df = df['indice']
        df = df.unstack('reseau')
        dates_completes = date_range(to_date(debut), to_date(fin), freq='D')
        df = df.reindex(dates_completes)
        return df


Instruction:
Here you have a function in Python 3, explain what it does
def get_indices_et_ssi(self, reseau, debut, fin, complet=True):
        if complet:
            i_str = "c_ind_diffuse"
            ssi_str = "c_ss_indice"
        else:
            i_str = "p_ind_diffuse"
            ssi_str = "p_ss_indice"

        _sql = """SELECT
                g.nom_agglo as "reseau",
                i.j_date as "date",
                max(case when i.{0}>0 then i.{0} else 0 end) indice,
                max(case when n.cchim='NO2' then ssi.{1} else 0 end) no2,
                max(case when n.cchim='PM10' then ssi.{1} else 0 end) pm10,
                max(case when n.cchim='O3' then ssi.{1} else 0 end) o3,
                max(case when n.cchim='SO2' then ssi.{1} else 0 end) so2
        FROM resultat_indice i
        INNER JOIN resultat_ss_indice ssi ON (i.nom_court_grp=ssi.nom_court_grp AND i.j_date=ssi.j_date)
        INNER JOIN groupe_atmo g ON (i.nom_court_grp=g.nom_court_grp)
        INNER JOIN nom_mesure n ON (ssi.nopol=n.nopol)
        WHERE g.nom_agglo='{2}'
        AND i.j_date BETWEEN
        TO_DATE('{3}', 'YYYY-MM-DD') AND
        TO_DATE('{4}', 'YYYY-MM-DD')
        GROUP BY
        g.nom_agglo,
        i.j_date
        ORDER BY i.j_date""".format(i_str, ssi_str, reseau, debut, fin)
        df = psql.read_sql(_sql, self.conn)
        df = df.set_index(['reseau', 'date'])
        return df

Response: Renvoie l indice et les sous_indices


Instruction:
How would you implement a function in Python 3 that
attempts to run a global hook by name with args

Response:
def run_global_hook(hook_name, *args):
    '''Attempt to run a global hook by name with args'''

    hook_finder = HookFinder(get_global_hook_path())
    hook = hook_finder(hook_name)
    if hook:
        hook.run(*args)


Instruction:
Make a summary of the following Python 3 code
def status(jobs):
    def status_handler():
        endpoints = []
        stats = {"endpoints": None}

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=5)
        # This is basically calling the below within the executor:
        #
        #     >>> timeit(job[2], number=1)
        #
        # job is a tuple of (name, timeout, func) so the above is really:
        #     >>> timeit(func, number=1)
        #
        #gen = ((job, executor.submit(timeit, job[2], number=1)) for job in jobs)
        #for job, future in gen:
        for job, future in [(job, executor.submit(timeit, job[2], number=1)) for job in jobs]:
            name, timeout, _ = job
            endpoint = {"endpoint": name}
            try:
                data = future.result(timeout=timeout)
                endpoint["duration"] = data
            except concurrent.futures.TimeoutError:
                endpoint["error"] = "timeout exceeded"
            except Exception as ex:
                endpoint["error"] = str(ex)
            endpoints.append(endpoint)

        if len(endpoints) > 0:
            stats["endpoints"] = endpoints

        executor.shutdown(wait=False)
        return jsonify(**stats)
        # TODO: Look into potentially cleaning up jobs that have timed-out.
        #
        #       This could be done by changing jobs from a func to an object
        #       that implements `def interrupt(self):` which would be used
        #       to interrupt/stop/close/cleanup any resources.
    return status_handler

Response: Handler that calls each status job in a worker pool attempting to timeout.


Instruction:
Here you have a function in Python 3, explain what it does
def moyennes_glissantes(df, sur=8, rep=0.75):
    return pd.rolling_mean(df, window=sur, min_periods=rep * sur)

Response: Calculates the rolling mean of moyennes glissantes


Instruction:
How would you explain what the following Python 3 function does
def nombre_depassement(df, valeur, freq=None):

    dep = depassement(df, valeur)
    if freq is not None:
        dep = dep.resample(freq, how='sum')
    return dep.count()

Response: Return the nombre de dépassement d un valeur.


Instruction:
Here you have a function in Python 3, explain what it does
def aot40_vegetation(df, nb_an):

    return _aot(df.tshift(1), nb_an=nb_an, limite=80, mois_debut=5, mois_fin=7,
                heure_debut=8, heure_fin=19)

Response: Aot - 40 de la souce un AOT - 40 de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la souce de la sou


Instruction:
Can you generate the documentation for the following Python 3 function
def no2(df):

    polluant = "NO2"

    # Le DataFrame doit être en heure
    if not isinstance(df.index.freq, pdoffset.Hour):
        raise FreqException("df doit être en heure.")

    res = {"Seuil de RI en moyenne H: 200u": depassement(df, valeur=200),
           "Seuil d'Alerte sur 3H consécutives: 400u": consecutive(df, valeur=400, sur=3),
           "Seuil d'Alerte sur 3J consécutifs: 200u": consecutive(df.resample('D', how='max'), valeur=200, sur=3),
           "Valeur limite pour la santé humaine 18H/A: 200u": depassement(df, valeur=200),
           "Valeur limite pour la santé humaine en moyenne A: 40u": depassement(df.resample('A', how='mean'),
                                                                                valeur=40),
           "Objectif de qualité en moyenne A: 40u": depassement(df.resample('A', how='mean'), valeur=40),
           "Protection de la végétation en moyenne A: 30u": depassement(df.resample('A', how='mean'), valeur=30),
           }

    return polluant, res

Response: A helper function for calculating the non - no2 sans un terme d un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 sans un no2 s


Instruction:
Write a Python 3 script for
pouring les particules PM10

Response:
def pm10(df):
    """
    Calculs réglementaires pour les particules PM10

    Paramètres:
    df: DataFrame contenant les mesures, avec un index temporel
    (voir xair.get_mesure)

    Retourne:
    Une série de résultats dans un DataFrame :
    ******
    unité (u): µg/m3 (microgramme par mètre cube)

    Seuil de RI en moyenne J: 50u
    Seuil d'Alerte en moyenne J: 80u
    Valeur limite pour la santé humaine 35J/A: 50u
    Valeur limite pour la santé humaine en moyenne A: 40u
    Objectif de qualité en moyenne A: 30u

    Les résultats sont donnés en terme d'heure de dépassement

    """

    polluant = 'PM10'
    # Le DataFrame doit être en jour
    if not isinstance(df.index.freq, pdoffset.Day):
        raise FreqException("df doit être en jour.")

    res = {"Seuil de RI en moyenne J: 50u": depassement(df, valeur=50),
           "Seuil d'Alerte en moyenne J: 80u": depassement(df, valeur=80),
           "Valeur limite pour la santé humaine 35J/A: 50u": depassement(df, valeur=50),
           "Valeur limite pour la santé humaine en moyenne A: 40u": depassement(df.resample('A', how='mean'),
                                                                                valeur=40),
           "Objectif de qualité en moyenne A: 30u": depassement(df.resample('A', how='mean'), valeur=30),
           }

    return polluant, res


Instruction:
Given the following Python 3 function, write the documentation
def so2(df):

    polluant = 'SO2'

    # Le DataFrame doit être en heure
    if not isinstance(df.index.freq, pdoffset.Hour):
        raise FreqException("df doit être en heure.")

    res = {"Seuil de RI en moyenne H: 300u": depassement(df, valeur=300),
           "Seuil d'Alerte sur 3H consécutives: 500u": depassement(df, valeur=500),
           "Valeur limite pour la santé humaine 24H/A: 350u": depassement(df, valeur=350),
           "Valeur limite pour la santé humaine 3J/A: 125u": depassement(df.resample('D', how='mean'), valeur=125),
           "Objectif de qualité en moyenne A: 50u": depassement(df.resample('A', how='mean'), valeur=50),
           "Protection de la végétation en moyenne A: 20u": depassement(df.resample('A', how='mean'), valeur=20),
           "Protection de la végétation du 01/10 au 31/03: 20u": depassement(
               df[(df.index.month <= 3) | (df.index.month >= 10)], valeur=20),
           }

    return polluant, res

Response: A helper function for calculating the non - defaul sont donnés in a single sont donnés.


Instruction:
Can you tell what is the following Python 3 function doing
def c6h6(df):

    polluant = 'C6H6'

    # Le DataFrame doit être en heure
    if not isinstance(df.index.freq, pdoffset.Hour):
        raise FreqException("df doit être en heure.")

    res = {"Objectif de qualité en moyenne A: 2u": depassement(df.resample('A', how='mean'), valeur=2),
           "Valeur limite pour la santé humaine en moyenne A: 5u": depassement(df.resample('A', how='mean'), valeur=5),
           }

    return polluant, res

Response: C6H6 - > C6H6


Instruction:
How would you code a function in Python 3 to
print out synthes de la fonction renvoyant les valeurs de la fonction de calculées.

Response:
def print_synthese(fct, df):
    """
    Présente une synthèse des calculs réglementaires en fournissant les valeurs
    calculées suivant les réglementations définies dans chaque fonction de calcul
    et un tableau de nombre de dépassement.

    Paramètres:
    fct: fonction renvoyant les éléments calculées
    df: DataFrame de valeurs d'entrée à fournir à la fonction

    Retourne:
    Imprime sur l'écran les valeurs synthétisées

    """

    res_count = dict()

    polluant, res = fct(df)
    print("\nPour le polluant: %s" % polluant)

    print("\nValeurs mesurées suivant critères:")
    for k, v in res.items():
        comp = compresse(v)
        if not comp.empty:
            comp.index.name = k
            print(comp.to_string(na_rep='', float_format=lambda x: "%.0f" % x))
        else:
            print("\n%s: aucune valeur en dépassement" % k)
        res_count[k] = v.count()

    res_count = pd.DataFrame(res_count).T
    print("Nombre de dépassements des critères:\n")
    print(res_count)


Instruction:
How would you explain what the following Python 3 function does
def html_synthese(fct, df):

    html = str()
    res_count = dict()
    buf = StringIO()
    polluant, res = fct(df)
    html += '

Pour le polluant: {}

'.format(polluant)

    # On enregistre tous les résultats dans le buffer et on calcule la somme de chaque
    for k, v in res.items():
        buf.write("
")
        comp = compresse(v)
        if not comp.empty:
            comp.index.name = k
            comp.to_html(buf=buf,
                         sparsify=True,
                         na_rep="")
        else:
            buf.write(
                '

{}	Aucun dépassement

'.format(
                    k))
        buf.write("
")
        res_count[k] = v.count()

    res_count = pd.DataFrame(res_count).T
    res_count.index.name = "Nombre de dépassements des critères"
    html += "
"
    html += res_count.to_html(sparsify=True)
    html += "
"

    html += buf.getvalue()

    return html

Response: A helper function to format html synthese un synthant de un synthant.


Instruction:
Write a Python 3 function that can
show the max value of the max value of the last date in a DataFrame.

Response:
def show_max(df):
    """Pour chaque serie (colonne) d'un DataFrame, va rechercher la (les) valeur(s)
    et la (les) date(s) du (des) max.

    Paramètres:
    df: DataFrame de valeurs à calculer

    Retourne:
    Un DataFrame montrant pour chaque serie (colonne), les valeurs maxs aux dates
    d'apparition.
    """
    df = df.astype(pd.np.float)
    res = list()
    for c in df.columns:
        serie = df[c]
        res.append(serie.where(cond=serie == serie.max(), other=pd.np.nan).dropna())
    return pd.DataFrame(res).T


Instruction:
Here you have a function in Python 3, explain what it does
def taux_de_representativite(df):
    return (df.count().astype(pd.np.float) / df.shape[0] * 100).round(1)

Response: Calculate le taux de representativite


Instruction:
Can you implement a function in Python 3 that
validates all the entries in the environment cache.

Response:
def validate(self):
        '''Validate all the entries in the environment cache.'''

        for env in list(self):
            if not env.exists:
                self.remove(env)


Instruction:
Here you have a function in Python 3, explain what it does
def load(self):
        '''Load the environment cache from disk.'''

        if not os.path.exists(self.path):
            return

        with open(self.path, 'r') as f:
            env_data = yaml.load(f.read())

        if env_data:
            for env in env_data:
                self.add(VirtualEnvironment(env['root']))

Response: Load the environment cache from disk.


Instruction:
Here you have a function in Python 3, explain what it does
def save(self):
        '''Save the environment cache to disk.'''

        env_data = [dict(name=env.name, root=env.path) for env in self]
        encode = yaml.safe_dump(env_data, default_flow_style=False)

        with open(self.path, 'w') as f:
            f.write(encode)

Response: Save the environment cache to disk.


Instruction:
Given the following Python 3 function, write the documentation
def prompt(text, default=None, hide_input=False,
           confirmation_prompt=False, type=None,
           value_proc=None, prompt_suffix=': ',
           show_default=True, err=False):
    """Prompts a user for input.  This is a convenience function that can
    be used to prompt a user for input later.

    If the user aborts the input by sending a interrupt signal, this
    function will catch it and raise a :exc:`Abort` exception.

    .. versionadded:: 6.0
       Added unicode support for cmd.exe on Windows.

    .. versionadded:: 4.0
       Added the `err` parameter.

    :param text: the text to show for the prompt.
    :param default: the default value to use if no input happens.  If this
                    is not given it will prompt until it's aborted.
    :param hide_input: if this is set to true then the input value will
                       be hidden.
    :param confirmation_prompt: asks for confirmation for the value.
    :param type: the type to use to check the value against.
    :param value_proc: if this parameter is provided it's a function that
                       is invoked instead of the type conversion to
                       convert a value.
    :param prompt_suffix: a suffix that should be added to the prompt.
    :param show_default: shows or hides the default value in the prompt.
    :param err: if set to true the file defaults to ``stderr`` instead of
                ``stdout``, the same as with echo.
    """
    result = None

    def prompt_func(text):
        f = hide_input and hidden_prompt_func or visible_prompt_func
        try:
            # Write the prompt separately so that we get nice
            # coloring through colorama on Windows
            echo(text, nl=False, err=err)
            return f('')
        except (KeyboardInterrupt, EOFError):
            # getpass doesn't print a newline if the user aborts input with ^C.
            # Allegedly this behavior is inherited from getpass(3).
            # A doc bug has been filed at https://bugs.python.org/issue24711
            if hide_input:
                echo(None, err=err)
            raise Abort()

    if value_proc is None:
        value_proc = convert_type(type, default)

    prompt = _build_prompt(text, prompt_suffix, show_default, default)

    while 1:
        while 1:
            value = prompt_func(prompt)
            if value:
                break
            # If a default is set and used, then the confirmation
            # prompt is always skipped because that's the only thing
            # that really makes sense.
            elif default is not None:
                return default
        try:
            result = value_proc(value)
        except UsageError as e:
            echo('Error: %s' % e.message, err=err)
            continue
        if not confirmation_prompt:
            return result
        while 1:
            value2 = prompt_func('Repeat for confirmation: ')
            if value2:
                break
        if value == value2:
            return result
        echo('Error: the two entered values do not match', err=err)

Response: Prompts a user for input.


Instruction:
Make a summary of the following Python 3 code
def _remote_setup_engine(engine_id, nengines):
    if distob.engine is None:
        distob.engine = distob.ObjectEngine(engine_id, nengines)
    # TODO these imports should be unnecessary with improved deserialization
    import numpy as np
    from scipy import stats
    # TODO Using @ipyparallel.interactive still did not import to __main__
    #      so will do it this way for now.
    import __main__
    __main__.__dict__['np'] = np
    __main__.__dict__['stats'] = stats

Response: Create an objectEngine instance and set up the objectengine module.


Instruction:
Can you implement a function in Python 3 that
prepares all iPython engines for distributed object processing.

Response:
def setup_engines(client=None):
    """Prepare all iPython engines for distributed object processing.

    Args:
      client (ipyparallel.Client, optional): If None, will create a client
        using the default ipyparallel profile.
    """
    if not client:
        try:
            client = ipyparallel.Client()
        except:
            raise DistobClusterError(
                u"""Could not connect to an ipyparallel cluster. Make
                 sure a cluster is started (e.g. to use the CPUs of a
                 single computer, can type 'ipcluster start')""")
    eids = client.ids
    if not eids:
        raise DistobClusterError(
                u'No ipyparallel compute engines are available')
    nengines = len(eids)
    dv = client[eids]
    dv.use_dill()
    with dv.sync_imports(quiet=True):
        import distob
    # create global ObjectEngine distob.engine on each engine
    ars = []
    for i in eids:
        dv.targets = i
        ars.append(dv.apply_async(_remote_setup_engine, i, nengines))
    dv.wait(ars)
    for ar in ars:
        if not ar.successful():
            raise ar.r
    # create global ObjectHub distob.engine on the client host
    if distob.engine is None:
        distob.engine = ObjectHub(-1, client)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _process_args(args, kwargs, prefer_local=True, recurse=True):
    this_engine = distob.engine.eid
    local_args = []
    remote_args = []
    execloc = this_engine  # the chosen engine id for execution of the call
    for a in args:
        id = None
        if isinstance(a, Remote):
            id = a._ref.id
        elif isinstance(a, Ref):
            id = a.id
        elif isinstance(a, Id):
            id = a
        if id is not None:
            if id.engine is this_engine:
                local_args.append(distob.engine[id])
                remote_args.append(distob.engine[id])
            else:
                if (prefer_local and isinstance(a, Remote) and
                        a._obcache_current):
                    local_args.append(a._obcache)
                    remote_args.append(id)
                else:
                    # will choose remote execution
                    if execloc is not this_engine and id.engine is not execloc:
                        raise DistobValueError(
                            'two remote arguments are from different engines')
                    else:
                        execloc = id.engine
                        local_args.append(None)
                        remote_args.append(id)
        elif (isinstance(a, collections.Sequence) and
                not isinstance(a, string_types) and recurse):
            eid, ls, _ = _process_args(a, {}, prefer_local, recurse=False)
            if eid is not this_engine:
                if execloc is not this_engine and eid is not execloc:
                    raise DistobValueError(
                            'two remote arguments are from different engines')
                execloc = eid
            local_args.append(ls)
            remote_args.append(ls)
        else:
            # argument is an ordinary object
            local_args.append(a)
            remote_args.append(a)
    local_kwargs = dict()
    remote_kwargs = dict()
    for k, a in kwargs.items():
        id = None
        if isinstance(a, Remote):
            id = a._ref.id
        elif isinstance(a, Ref):
            id = a.id
        elif isinstance(a, Id):
            id = a
        if id is not None:
            if id.engine is this_engine:
                local_kwargs[k] = distob.engine[id]
                remote_kwargs[k] = distob.engine[id]
            else:
                if (prefer_local and isinstance(a, Remote) and
                        a._obcache_current):
                    local_kwargs[k] = a._obcache
                    remote_kwargs[k] = id
                else:
                    # will choose remote execution
                    if execloc is not this_engine and id.engine is not execloc:
                        raise DistobValueError(
                            'two remote arguments are from different engines')
                    else:
                        execloc = id.engine
                        local_kwargs[k] = None
                        remote_kwargs[k] = id
        elif (isinstance(a, collections.Sequence) and
                not isinstance(a, string_types) and recurse):
            eid, ls, _ = _process_args(a, {}, prefer_local, recurse=False)
            if eid is not this_engine:
                if execloc is not this_engine and eid is not execloc:
                    raise DistobValueError(
                            'two remote arguments are from different engines')
                execloc = eid
            local_kwargs[k] = ls
            remote_kwargs[k] = ls
        else:
            # argument is an ordinary object
            local_kwargs[k] = a
            remote_kwargs[k] = a
    if execloc is this_engine:
        return execloc, tuple(local_args), local_kwargs
    else:
        return execloc, tuple(remote_args), remote_kwargs

Response: Process the arguments and kwargs.


Instruction:
Given the following Python 3 function, write the documentation
def call(f, *args, **kwargs):
    this_engine = distob.engine.eid
    prefer_local = kwargs.pop('prefer_local', True)
    block = kwargs.pop('block', True)
    execloc, args, kwargs = _process_args(args, kwargs, prefer_local)
    if execloc is this_engine:
        r = f(*args, **kwargs)
    else:
        if False and prefer_local:
            # result cache disabled until issue mattja/distob#1 is fixed
            try:
                kwtuple = tuple((k, kwargs[k]) for k in sorted(kwargs.keys()))
                key = (f, args, kwtuple)
                r = _call_cache[key]
            except TypeError as te:
                if te.args[0][:10] == 'unhashable':
                    #print("unhashable. won't be able to cache")
                    r = _uncached_call(execloc, f, *args, **kwargs)
                else:
                    raise
            except KeyError:
                r = _uncached_call(execloc, f, *args, **kwargs)
                if block:
                    _call_cache[key] = r.r
        else:
            r = _uncached_call(execloc, f, *args, **kwargs)
    if block:
        return convert_result(r)
    else:
        return r

Response: Execute a function on the arguments locally or remotely as appropriate.


Instruction:
Can you generate the documentation for the following Python 3 function
def convert_result(r):
    if (isinstance(r, collections.Sequence) and
            not isinstance(r, string_types)):
        rs = []
        for subresult in r:
            rs.append(convert_result(subresult))
        return rs
    if isinstance(r, ipyparallel.AsyncResult):
        r = r.r
    if isinstance(r, Ref):
        RemoteClass = distob.engine.proxy_types[r.type]
        r = RemoteClass(r)
    return r

Response: Waits for and converts any AsyncResults.