Instruction:
Here you have a function in Python 3, explain what it does
def _add_method(self, effect, verb, resource, conditions):
        if verb != '*' and not hasattr(HttpVerb, verb):
            raise NameError('Invalid HTTP verb ' + verb +
                            '. Allowed verbs in HttpVerb class')
        resource_pattern = re.compile(self.path_regex)
        if not resource_pattern.match(resource):
            raise NameError('Invalid resource path: ' + resource +
                            '. Path should match ' + self.path_regex)

        if resource[:1] == '/':
            resource = resource[1:]

        resource_arn = ('arn:aws:execute-api:' +
                        self.region + ':' +
                        self.aws_account_id + ':' +
                        self.rest_api_id + '/' +
                        self.stage + '/' +
                        verb + '/' +
                        resource)

        if effect.lower() == 'allow':
            self.allowMethods.append({
                'resource_arn': resource_arn,
                'conditions': conditions
            })
        elif effect.lower() == 'deny':
            self.denyMethods.append({
                'resource_arn': resource_arn,
                'conditions': conditions
            })

Response: Adds a method to the internal list of allowed or denied methods.


Instruction:
Make a summary of the following Python 3 code
def _get_effect_statement(self, effect, methods):
        statements = []

        if len(methods) > 0:
            statement = self._get_empty_statement(effect)

            for method in methods:
                if (method['conditions'] is None or
                        len(method['conditions']) == 0):
                    statement['Resource'].append(method['resource_arn'])
                else:
                    cond_statement = self._get_empty_statement(effect)
                    cond_statement['Resource'].append(method['resource_arn'])
                    cond_statement['Condition'] = method['conditions']
                    statements.append(cond_statement)
            statements.append(statement)

        return statements

Response:
This function loops over an array of objects containing
        a resourceArn and conditions statement and generates the array of statements that are used to create the policy.


Instruction:
Write a Python 3 function that can
add an API Gateway method with a list of conditions to the allowed methods list.

Response:
def allow_method_with_conditions(self, verb, resource, conditions):
        """
        Adds an API Gateway method (Http verb + Resource path) to the
        list of allowed methods and includes a condition for the policy
        statement. More on AWS policy conditions here:
        http://docs.aws.amazon.com/IAM/latest/UserGuide/
        reference_policies_elements.html#Condition
        """
        self._add_method('Allow', verb, resource, conditions)


Instruction:
Can you tell what is the following Python 3 function doing
def deny_method_with_conditions(self, verb, resource, conditions):
        self._add_method('Deny', verb, resource, conditions)

Response:
Adds an API Gateway method to the list of denied methods and includes a condition for the policy
        statement.


Instruction:
Explain what the following Python 3 code does
def build(self):
        if ((self.allowMethods is None or len(self.allowMethods) == 0) and
                (self.denyMethods is None or len(self.denyMethods) == 0)):
            raise NameError('No statements defined for the policy')

        policy = {
            'principalId': self.principal_id,
            'policyDocument': {
                'Version': self.version,
                'Statement': []
            }
        }

        policy['policyDocument']['Statement'].extend(
            self._get_effect_statement('Allow', self.allowMethods))
        policy['policyDocument']['Statement'].extend(
            self._get_effect_statement('Deny', self.denyMethods))

        return policy

Response: Builds the policy document based on the internal lists of allow and deny conditions.


Instruction:
Given the following Python 3 function, write the documentation
def deref(self, data):

        # We have to make a deepcopy here to create a proper JSON
        # compatible object, otherwise `json.dumps` fails when it
        # hits jsonref.JsonRef objects.
        deref = copy.deepcopy(jsonref.JsonRef.replace_refs(data))

        # Write out JSON version because we might want this.
        self.write_template(deref, filename='swagger.json')

        return deref

Response: Convert to JSON version and write out the JSON version.


Instruction:
Can you tell what is the following Python 3 function doing
def check_pre_requirements(pre_requirements):
    pre_requirements = set(pre_requirements or [])
    pre_requirements.add('virtualenv')

    for requirement in pre_requirements:
        if not which(requirement):
            print_error('Requirement {0!r} is not found in system'.
                        format(requirement))
            return False

    return True

Response: Check all necessary system requirements to exist.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def config_to_args(config):
    result = []

    for key, value in iteritems(config):
        if value is False:
            continue

        key = '--{0}'.format(key.replace('_', '-'))

        if isinstance(value, (list, set, tuple)):
            for item in value:
                result.extend((key, smart_str(item)))
        elif value is not True:
            result.extend((key, smart_str(value)))
        else:
            result.append(key)

    return tuple(result)

Response: Convert config dict to list of arguments.


Instruction:
Can you write a function in Python 3 where it
creates virtual environment. :param env: Virtual environment name. :param args: Pass given arguments to ``virtualenv`` script. :param recerate: Recreate virtual environment? By default: False :param ignore_activated: Ignore already activated virtual environment and create new one. By default: False :param quiet: Do not output messages into terminal. By default: False

Response:
def create_env(env, args, recreate=False, ignore_activated=False, quiet=False):
    """Create virtual environment.

    :param env: Virtual environment name.
    :param args: Pass given arguments to ``virtualenv`` script.
    :param recerate: Recreate virtual environment? By default: False
    :param ignore_activated:
        Ignore already activated virtual environment and create new one. By
        default: False
    :param quiet: Do not output messages into terminal. By default: False
    """
    cmd = None
    result = True

    inside_env = hasattr(sys, 'real_prefix') or os.environ.get('VIRTUAL_ENV')
    env_exists = os.path.isdir(env)

    if not quiet:
        print_message('== Step 1. Create virtual environment ==')

    if (
        recreate or (not inside_env and not env_exists)
    ) or (
        ignore_activated and not env_exists
    ):
        cmd = ('virtualenv', ) + args + (env, )

    if not cmd and not quiet:
        if inside_env:
            message = 'Working inside of virtual environment, done...'
        else:
            message = 'Virtual environment {0!r} already created, done...'
        print_message(message.format(env))

    if cmd:
        with disable_error_handler():
            result = not run_cmd(cmd, echo=not quiet)

    if not quiet:
        print_message()

    return result


Instruction:
Can you generate the documentation for the following Python 3 function
def install(env, requirements, args, ignore_activated=False,
            install_dev_requirements=False, quiet=False):
    """Install library or project into virtual environment.

    :param env: Use given virtual environment name.
    :param requirements: Use given requirements file for pip.
    :param args: Pass given arguments to pip script.
    :param ignore_activated:
        Do not run pip inside already activated virtual environment. By
        default: False
    :param install_dev_requirements:
        When enabled install prefixed or suffixed dev requirements after
        original installation process completed. By default: False
    :param quiet: Do not output message to terminal. By default: False
    """
    if os.path.isfile(requirements):
        args += ('-r', requirements)
        label = 'project'
    else:
        args += ('-U', '-e', '.')
        label = 'library'

    # Attempt to install development requirements
    if install_dev_requirements:
        dev_requirements = None
        dirname = os.path.dirname(requirements)
        basename, ext = os.path.splitext(os.path.basename(requirements))

        # Possible dev requirements files:
        #
        # * <requirements>-dev.<ext>
        # * dev-<requirements>.<ext>
        # * <requirements>_dev.<ext>
        # * dev_<requirements>.<ext>
        # * <requirements>dev.<ext>
        # * dev<requirements>.<ext>
        #
        # Where <requirements> is basename of given requirements file to use
        # and <ext> is its extension.
        for delimiter in ('-', '_', ''):
            filename = os.path.join(
                dirname, ''.join((basename, delimiter, 'dev', ext))
            )
            if os.path.isfile(filename):
                dev_requirements = filename
                break

            filename = os.path.join(
                dirname, ''.join(('dev', delimiter, basename, ext))
            )
            if os.path.isfile(filename):
                dev_requirements = filename
                break

        # If at least one dev requirements file found, install dev requirements
        if dev_requirements:
            args += ('-r', dev_requirements)

    if not quiet:
        print_message('== Step 2. Install {0} =='.format(label))

    result = not pip_cmd(env,
                         ('install', ) + args,
                         ignore_activated,
                         echo=not quiet)

    if not quiet:
        print_message()

    return result

Response: Install library or project into virtual environment.


Instruction:
Can you tell what is the following Python 3 function doing
def iteritems(data, **kwargs):
    return iter(data.items(**kwargs)) if IS_PY3 else data.iteritems(**kwargs)

Response: Iterate over dict items.


Instruction:
Implement a Python 3 function for
iterating over dict keys.

Response:
def iterkeys(data, **kwargs):
    """Iterate over dict keys."""
    return iter(data.keys(**kwargs)) if IS_PY3 else data.iterkeys(**kwargs)


Instruction:
How would you implement a function in Python 3 that
parses command line arguments and return a parser instance.

Response:
def parse_args(args):
    """
    Parse args from command line by creating argument parser instance and
    process it.

    :param args: Command line arguments list.
    """
    from argparse import ArgumentParser

    description = ('Bootstrap Python projects and libraries with virtualenv '
                   'and pip.')
    parser = ArgumentParser(description=description)
    parser.add_argument('--version', action='version', version=__version__)

    parser.add_argument(
        '-c', '--config', default=DEFAULT_CONFIG,
        help='Path to config file. By default: {0}'.format(DEFAULT_CONFIG)
    )
    parser.add_argument(
        '-p', '--pre-requirements', default=[], nargs='+',
        help='List of pre-requirements to check, separated by space.'
    )
    parser.add_argument(
        '-e', '--env',
        help='Virtual environment name. By default: {0}'.
             format(CONFIG[__script__]['env'])
    )
    parser.add_argument(
        '-r', '--requirements',
        help='Path to requirements file. By default: {0}'.
             format(CONFIG[__script__]['requirements'])
    )
    parser.add_argument(
        '-d', '--install-dev-requirements', action='store_true', default=None,
        help='Install prefixed or suffixed "dev" requirements after '
             'installation of original requirements file or library completed '
             'without errors.'
    )
    parser.add_argument(
        '-C', '--hook', help='Execute this hook after bootstrap process.'
    )
    parser.add_argument(
        '--ignore-activated', action='store_true', default=None,
        help='Ignore pre-activated virtualenv, like on Travis CI.'
    )
    parser.add_argument(
        '--recreate', action='store_true', default=None,
        help='Recreate virtualenv on every run.'
    )
    parser.add_argument(
        '-q', '--quiet', action='store_true', default=None,
        help='Minimize output, show only error messages.'
    )

    return parser.parse_args(args)


Instruction:
Can you generate the documentation for the following Python 3 function
def pip_cmd(env, cmd, ignore_activated=False, **kwargs):
    cmd = tuple(cmd)
    dirname = safe_path(env)

    if not ignore_activated:
        activated_env = os.environ.get('VIRTUAL_ENV')

        if hasattr(sys, 'real_prefix'):
            dirname = sys.prefix
        elif activated_env:
            dirname = activated_env

    pip_path = os.path.join(dirname, 'Scripts' if IS_WINDOWS else 'bin', 'pip')

    if kwargs.pop('return_path', False):
        return pip_path

    if not os.path.isfile(pip_path):
        raise OSError('No pip found at {0!r}'.format(pip_path))

    # Disable pip version check in tests
    if BOOTSTRAPPER_TEST_KEY in os.environ and cmd[0] == 'install':
        cmd = list(cmd)
        cmd.insert(1, '--disable-pip-version-check')
        cmd = tuple(cmd)

    with disable_error_handler():
        return run_cmd((pip_path, ) + cmd, **kwargs)

Response: r Run pip command in given virtual environment.


Instruction:
Given the following Python 3 function, write the documentation
def prepare_args(config, bootstrap):
    config = copy.deepcopy(config)
    environ = dict(copy.deepcopy(os.environ))

    data = {'env': bootstrap['env'],
            'pip': pip_cmd(bootstrap['env'], '', return_path=True),
            'requirements': bootstrap['requirements']}
    environ.update(data)

    if isinstance(config, string_types):
        return config.format(**environ)

    for key, value in iteritems(config):
        if not isinstance(value, string_types):
            continue
        config[key] = value.format(**environ)

    return config_to_args(config)

Response: Convert config dict to command line args line.


Instruction:
Write a Python 3 script for
printing error message to stderr using ANSI - colors.

Response:
def print_error(message, wrap=True):
    """Print error message to stderr, using ANSI-colors.

    :param message: Message to print
    :param wrap:
        Wrap message into ``ERROR: <message>. Exit...`` template. By default:
        True
    """
    if wrap:
        message = 'ERROR: {0}. Exit...'.format(message.rstrip('.'))

    colorizer = (_color_wrap(colorama.Fore.RED)
                 if colorama
                 else lambda message: message)
    return print(colorizer(message), file=sys.stderr)


Instruction:
Explain what the following Python 3 code does
def print_message(message=None):
    kwargs = {'stdout': sys.stdout,
              'stderr': sys.stderr,
              'shell': True}
    return subprocess.call('echo "{0}"'.format(message or ''), **kwargs)

Response: Print message via subprocess. call function.


Instruction:
How would you code a function in Python 3 to
read and parse a config file.

Response:
def read_config(filename, args):
    """
    Read and parse configuration file. By default, ``filename`` is relative
    path to current work directory.

    If no config file found, default ``CONFIG`` would be used.

    :param filename: Read config from given filename.
    :param args: Parsed command line arguments.
    """
    # Initial vars
    config = defaultdict(dict)
    splitter = operator.methodcaller('split', ' ')

    converters = {
        __script__: {
            'env': safe_path,
            'pre_requirements': splitter,
        },
        'pip': {
            'allow_external': splitter,
            'allow_unverified': splitter,
        }
    }
    default = copy.deepcopy(CONFIG)
    sections = set(iterkeys(default))

    # Append download-cache for old pip versions
    if int(getattr(pip, '__version__', '1.x').split('.')[0]) < 6:
        default['pip']['download_cache'] = safe_path(os.path.expanduser(
            os.path.join('~', '.{0}'.format(__script__), 'pip-cache')
        ))

    # Expand user and environ vars in config filename
    is_default = filename == DEFAULT_CONFIG
    filename = os.path.expandvars(os.path.expanduser(filename))

    # Read config if it exists on disk
    if not is_default and not os.path.isfile(filename):
        print_error('Config file does not exist at {0!r}'.format(filename))
        return None

    parser = ConfigParser()

    try:
        parser.read(filename)
    except ConfigParserError:
        print_error('Cannot parse config file at {0!r}'.format(filename))
        return None

    # Apply config for each possible section
    for section in sections:
        if not parser.has_section(section):
            continue

        items = parser.items(section)

        # Make auto convert here for integers and boolean values
        for key, value in items:
            try:
                value = int(value)
            except (TypeError, ValueError):
                try:
                    value = bool(strtobool(value))
                except ValueError:
                    pass

            if section in converters and key in converters[section]:
                value = converters[section][key](value)

            config[section][key] = value

    # Update config with default values if necessary
    for section, data in iteritems(default):
        if section not in config:
            config[section] = data
        else:
            for key, value in iteritems(data):
                config[section].setdefault(key, value)

    # Update bootstrap config from parsed args
    keys = set((
        'env', 'hook', 'install_dev_requirements', 'ignore_activated',
        'pre_requirements', 'quiet', 'recreate', 'requirements'
    ))

    for key in keys:
        value = getattr(args, key)
        config[__script__].setdefault(key, value)

        if key == 'pre_requirements' and not value:
            continue

        if value is not None:
            config[__script__][key] = value

    return config


Instruction:
Can you tell what is the following Python 3 function doing
def run_cmd(cmd, echo=False, fail_silently=False, **kwargs):
    out, err = None, None

    if echo:
        cmd_str = cmd if isinstance(cmd, string_types) else ' '.join(cmd)
        kwargs['stdout'], kwargs['stderr'] = sys.stdout, sys.stderr
        print_message('$ {0}'.format(cmd_str))
    else:
        out, err = get_temp_streams()
        kwargs['stdout'], kwargs['stderr'] = out, err

    try:
        retcode = subprocess.call(cmd, **kwargs)
    except subprocess.CalledProcessError as err:
        if fail_silently:
            return False
        print_error(str(err) if IS_PY3 else unicode(err))  # noqa
    finally:
        if out:
            out.close()
        if err:
            err.close()

    if retcode and echo and not fail_silently:
        print_error('Command {0!r} returned non-zero exit status {1}'.
                    format(cmd_str, retcode))

    return retcode

Response: r Run given command with subprocess. call.


Instruction:
Write a Python 3 script to
run post - bootstrap hook if any.

Response:
def run_hook(hook, config, quiet=False):
    """Run post-bootstrap hook if any.

    :param hook: Hook to run.
    :param config: Configuration dict.
    :param quiet: Do not output messages to STDOUT/STDERR. By default: False
    """
    if not hook:
        return True

    if not quiet:
        print_message('== Step 3. Run post-bootstrap hook ==')

    result = not run_cmd(prepare_args(hook, config),
                         echo=not quiet,
                         fail_silently=True,
                         shell=True)

    if not quiet:
        print_message()

    return result


Instruction:
Can you generate a brief explanation for the following Python 3 code
def save_traceback(err):
    # Store logs to ~/.bootstrapper directory
    dirname = safe_path(os.path.expanduser(
        os.path.join('~', '.{0}'.format(__script__))
    ))

    # But ensure that directory exists
    if not os.path.isdir(dirname):
        os.mkdir(dirname)

    # Now we ready to put traceback to log file
    filename = os.path.join(dirname, '{0}.log'.format(__script__))

    with open(filename, 'a+') as handler:
        traceback.print_exc(file=handler)

    # And show colorized message
    message = ('User aborted workflow'
               if isinstance(err, KeyboardInterrupt)
               else 'Unexpected error catched')
    print_error(message)
    print_error('Full log stored to {0}'.format(filename), False)

    return True

Response: Save error traceback to bootstrapper log file.


Instruction:
Given the following Python 3 function, write the documentation
def smart_str(value, encoding='utf-8', errors='strict'):
    if not IS_PY3 and isinstance(value, unicode):  # noqa
        return value.encode(encoding, errors)
    return str(value)

Response: Convert Python object to string.


Instruction:
Implement a function in Python 3 to
copy srcfile in destdir with extension basename + get_extension

Response:
def copy_w_ext(srcfile, destdir, basename):
    """ Copy `srcfile` in `destdir` with name `basename + get_extension(srcfile)`.
    Add pluses to the destination path basename if a file with the same name already
    exists in `destdir`.

    Parameters
    ----------
    srcfile: str

    destdir: str

    basename:str

    Returns
    -------
    dstpath: str
    """

    ext = get_extension(op.basename(srcfile))

    dstpath = op.join(destdir, basename + ext)

    return copy_w_plus(srcfile, dstpath)


Instruction:
Here you have a function in Python 3, explain what it does
def copy_w_plus(src, dst):
    dst_ext = get_extension(dst)
    dst_pre = remove_ext   (dst)

    while op.exists(dst_pre + dst_ext):
        dst_pre += '+'

    shutil.copy(src, dst_pre + dst_ext)

    return dst_pre + dst_ext

Response: Copy file from src path to dst path.


Instruction:
Make a summary of the following Python 3 code
def get_abspath(folderpath):
    if not op.exists(folderpath):
        raise FolderNotFound(folderpath)

    return op.abspath(folderpath)

Response: Returns the absolute path of folderpath.


Instruction:
Can you create a Python 3 function that
returns the extension of the file.

Response:
def get_extension(filepath, check_if_exists=False, allowed_exts=ALLOWED_EXTS):
    """Return the extension of fpath.

    Parameters
    ----------
    fpath: string
    File name or path

    check_if_exists: bool

    allowed_exts: dict
    Dictionary of strings, where the key if the last part of a complex ('.' separated) extension
    and the value is the previous part.
    For example: for the '.nii.gz' extension I would have a dict as {'.gz': ['.nii',]}

    Returns
    -------
    str
    The extension of the file name or path
    """
    if check_if_exists:
        if not op.exists(filepath):
            raise IOError('File not found: ' + filepath)

    rest, ext = op.splitext(filepath)
    if ext in allowed_exts:
        alloweds = allowed_exts[ext]
        _, ext2 = op.splitext(rest)
        if ext2 in alloweds:
            ext = ext2 + ext

    return ext


Instruction:
How would you implement a function in Python 3 that
adds the extension ext to fpath if it doesn t already exist.

Response:
def add_extension_if_needed(filepath, ext, check_if_exists=False):
    """Add the extension ext to fpath if it doesn't have it.

    Parameters
    ----------
    filepath: str
    File name or path

    ext: str
    File extension

    check_if_exists: bool

    Returns
    -------
    File name or path with extension added, if needed.
    """
    if not filepath.endswith(ext):
        filepath += ext

    if check_if_exists:
        if not op.exists(filepath):
            raise IOError('File not found: ' + filepath)

    return filepath


Instruction:
How would you implement a function in Python 3 that
parses a file with a list of subjects.

Response:
def parse_subjects_list(filepath, datadir='', split=':', labelsf=None):
    """Parses a file with a list of: <subject_file>:<subject_class_label>.

    Parameters
    ----------
    filepath: str
    Path to file with a list of: <subject_file>:<subject_class_label>.
    Where ':' can be any split character

    datadir: str
    String to be path prefix of each line of the fname content,
    only in case the lines are relative file paths.

    split: str
    Split character for each line

    labelsf: str
    Path to file with a list of the labels if it is not included in
    fname. It will overwrite the labels from fname.

    Returns
    -------
    [labels, subjs] where labels is a list of labels and subjs a list of
    filepaths
    """
    labels = []
    subjs  = []

    if datadir:
        datadir += op.sep

    with open(filepath, 'r') as f:
        for s in f:
            line = s.strip().split(split)
            if len(line) == 2:
                labels.append(np.float(line[1]))
                subjf = line[0].strip()
            else:
                subjf = line.strip()

            if not op.isabs(subjf):
                subjs.append(datadir + subjf)
            else:
                subjs.append(subjf)

    if labelsf is not None:
        labels = np.loadtxt(labelsf)

    return [labels, subjs]


Instruction:
Write a Python 3 function for
creating a file where each line is a subject_file.

Response:
def create_subjects_file(filelist, labels, output_file, split=':'):
    """Creates a file where each line is <subject_file>:<subject_class_label>.

    Parameters
    ----------
    filelist: list of str
    List of filepaths

    labels: list of int, str or labels that can be transformed with str()
    List of labels

    output_file: str
    Output file path

    split: str
    Split character for each line

    """
    if len(filelist) != len(labels):
        raise ValueError('Expected `filelist` and `labels` to have the same length.'
                         'Got {} and {}.'.format(len(filelist), len(labels)))

    lines = []
    for i, subj in enumerate(filelist):
        lab  = labels[i]
        line = subj + split + str(lab)
        lines.append(line)

    lines = np.array(lines)
    np.savetxt(output_file, lines, fmt='%s')


Instruction:
Here you have a function in Python 3, explain what it does
def join_path_to_filelist(path, filelist):
    return [op.join(path, str(item)) for item in filelist]

Response: Joins path to each line in filelist


Instruction:
Can you create a Python 3 function that
deletes all files in filelist in folder

Response:
def remove_all(filelist, folder=''):
    """Deletes all files in filelist

    Parameters
    ----------
    filelist: list of str
        List of the file paths to be removed

    folder: str
        Path to be used as common directory for all file paths in filelist
    """
    if not folder:
        for f in filelist:
            os.remove(f)
    else:
        for f in filelist:
            os.remove(op.join(folder, f))


Instruction:
Make a summary of the following Python 3 code
def get_folder_subpath(path, folder_depth):
    if path[0] == op.sep:
        folder_depth += 1

    return op.sep.join(path.split(op.sep)[0:folder_depth])

Response: Returns a folder path of path with depth given by folder_dept


Instruction:
How would you implement a function in Python 3 that
returns a TemporaryDirectory object for the specified prefix and basepath.

Response:
def get_temp_dir(prefix=None, basepath=None):
    """
    Uses tempfile to create a TemporaryDirectory using
    the default arguments.
    The folder is created using tempfile.mkdtemp() function.

    Parameters
    ----------
    prefix: str
    Name prefix for the temporary folder.

    basepath: str
    Directory where the new folder must be created.
    The default directory is chosen from a platform-dependent
    list, but the user of the application can control the
    directory location by setting the TMPDIR, TEMP or TMP
    environment variables.

    Returns
    -------
    folder object
    """
    if basepath is None:
        return tempfile.TemporaryDirectory(dir=basepath)
    else:
        return tempfile.TemporaryDirectory(prefix=prefix, dir=basepath)


Instruction:
Implement a Python 3 function for
returning the length of the file using the wc GNU command.

Response:
def ux_file_len(filepath):
    """Returns the length of the file using the 'wc' GNU command

    Parameters
    ----------
    filepath: str

    Returns
    -------
    float
    """
    p = subprocess.Popen(['wc', '-l', filepath], stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE)
    result, err = p.communicate()

    if p.returncode != 0:
        raise IOError(err)

    l = result.strip()
    l = int(l.split()[0])
    return l


Instruction:
Can you write a function in Python 3 where it
merges two dictionaries. Values that evaluate to true take priority over falsy values. `dict_1` takes priority over `dict_2`.

Response:
def merge(dict_1, dict_2):
    """Merge two dictionaries.

    Values that evaluate to true take priority over falsy values.
    `dict_1` takes priority over `dict_2`.

    """
    return dict((str(key), dict_1.get(key) or dict_2.get(key))
                for key in set(dict_2) | set(dict_1))


Instruction:
Implement a Python 3 function for
returning a system path if it exists.

Response:
def get_sys_path(rcpath, app_name, section_name=None):
    """Return a folder path if it exists.

    First will check if it is an existing system path, if it is, will return it
    expanded and absoluted.

    If this fails will look for the rcpath variable in the app_name rcfiles or
    exclusively within the given section_name, if given.

    Parameters
    ----------
    rcpath: str
        Existing folder path or variable name in app_name rcfile with an
        existing one.

    section_name: str
        Name of a section in the app_name rcfile to look exclusively there for
        variable names.

    app_name: str
        Name of the application to look for rcfile configuration files.

    Returns
    -------
    sys_path: str
        A expanded absolute file or folder path if the path exists.

    Raises
    ------
    IOError if the proposed sys_path does not exist.
    """
    # first check if it is an existing path
    if op.exists(rcpath):
        return op.realpath(op.expanduser(rcpath))

    # look for the rcfile
    try:
        settings = rcfile(app_name, section_name)
    except:
        raise

    # look for the variable within the rcfile configutarions
    try:
        sys_path = op.expanduser(settings[rcpath])
    except KeyError:
        raise IOError('Could not find an existing variable with name {0} in'
                      ' section {1} of {2}rc config setup. Maybe it is a '
                      ' folder that could not be found.'.format(rcpath,
                                                                section_name,
                                                                app_name))
    # found the variable, now check if it is an existing path
    else:
        if not op.exists(sys_path):
            raise IOError('Could not find the path {3} indicated by the '
                          'variable {0} in section {1} of {2}rc config '
                          'setup.'.format(rcpath, section_name, app_name,
                                          sys_path))

        # expand the path and return
        return op.realpath(op.expanduser(sys_path))


Instruction:
How would you code a function in Python 3 to
read environment variables and config files and return them merged with the config files and the command line arguments.

Response:
def rcfile(appname, section=None, args={}, strip_dashes=True):
    """Read environment variables and config files and return them merged with
    predefined list of arguments.

    Parameters
    ----------
    appname: str
        Application name, used for config files and environment variable
        names.

    section: str
        Name of the section to be read. If this is not set: appname.

    args:
        arguments from command line (optparse, docopt, etc).

    strip_dashes: bool
        Strip dashes prefixing key names from args dict.

    Returns
    --------
    dict
        containing the merged variables of environment variables, config
        files and args.

    Raises
    ------
    IOError
        In case the return value is empty.

    Notes
    -----
    Environment variables are read if they start with appname in uppercase
    with underscore, for example:

        TEST_VAR=1

    Config files compatible with ConfigParser are read and the section name
    appname is read, example:

        [appname]
        var=1

    We can also have host-dependent configuration values, which have
    priority over the default appname values.

        [appname]
        var=1

        [appname:mylinux]
        var=3

    For boolean flags do not try to use: 'True' or 'False',
                                         'on' or 'off',
                                         '1' or '0'.
    Unless you are willing to parse this values by yourself.
    We recommend commenting the variables out with '#' if you want to set a
    flag to False and check if it is in the rcfile cfg dict, i.e.:

        flag_value = 'flag_variable' in cfg

    Files are read from: /etc/appname/config,
                         /etc/appfilerc,
                         ~/.config/appname/config,
                         ~/.config/appname,
                         ~/.appname/config,
                         ~/.appnamerc,
                         appnamerc,
                         .appnamerc,
                         appnamerc file found in 'path' folder variable in args,
                         .appnamerc file found in 'path' folder variable in args,
                         file provided by 'config' variable in args.

    Example
    -------
        args = rcfile(__name__, docopt(__doc__, version=__version__))
    """
    if strip_dashes:
        for k in args.keys():
            args[k.lstrip('-')] = args.pop(k)

    environ = get_environment(appname)

    if section is None:
        section = appname

    config = get_config(appname,
                        section,
                        args.get('config', ''),
                        args.get('path', ''))
    config = merge(merge(args, config), environ)

    if not config:
        raise IOError('Could not find any rcfile for application '
                      '{}.'.format(appname))

    return config


Instruction:
Write a Python 3 function that can
return the dictionary containing the rcfile section configuration and variable values.

Response:
def get_rcfile_section(app_name, section_name):
    """ Return the dictionary containing the rcfile section configuration
    variables.

    Parameters
    ----------
    section_name: str
        Name of the section in the rcfiles.

    app_name: str
        Name of the application to look for its rcfiles.

    Returns
    -------
    settings: dict
        Dict with variable values
    """
    try:
        settings = rcfile(app_name, section_name)
    except IOError:
        raise
    except:
        raise KeyError('Error looking for section {} in {} '
                       ' rcfiles.'.format(section_name, app_name))
    else:
        return settings


Instruction:
Make a summary of the following Python 3 code
def get_rcfile_variable_value(var_name, app_name, section_name=None):
    cfg = get_rcfile_section(app_name, section_name)

    if var_name in cfg:
        raise KeyError('Option {} not found in {} '
                       'section.'.format(var_name, section_name))

    return cfg[var_name]

Response: Returns the value of the variable in the section_name section of the rc file.


Instruction:
Can you implement a function in Python 3 that
finds the first section in the rcfiles where the first var_name is found in the app_name rcfiles.

Response:
def find_in_sections(var_name, app_name):
    """ Return the section and the value of the variable where the first
    var_name is found in the app_name rcfiles.

    Parameters
    ----------
    var_name: str
        Name of the variable to be searched for.

    app_name: str
        Name of the application to look for its rcfiles.

    Returns
    -------
    section_name: str
        Name of the section in the rcfiles where var_name was first found.

    var_value: str
        The value of the first variable with given var_name.
    """
    sections = get_sections(app_name)

    if not sections:
        raise ValueError('No sections found in {} rcfiles.'.format(app_name))

    for s in sections:
        try:
            var_value = get_rcfile_variable_value(var_name, section_name=s,
                                                  app_name=app_name)
        except:
            pass
        else:
            return s, var_value

    raise KeyError('No variable {} has been found in {} '
                   'rcfiles.'.format(var_name, app_name))


Instruction:
Explain what the following Python 3 code does
def filter_list(lst, pattern):
    if is_fnmatch_regex(pattern) and not is_regex(pattern):
        #use fnmatch
        log.info('Using fnmatch for {0}'.format(pattern))
        filst = fnmatch.filter(lst, pattern)

    else:
        #use re
        log.info('Using regex match for {0}'.format(pattern))
        filst = match_list(lst, pattern)

    if filst:
        filst.sort()

    return filst

Response: Filters the list using pattern.


Instruction:
Write a Python 3 script for
giving a nested dictionary adict and a path string returns its childen just below the path.

Response:
def get_subdict(adict, path, sep=os.sep):
    """
    Given a nested dictionary adict.
    This returns its childen just below the path.
    The path is a string composed of adict keys separated by sep.

    :param adict: nested dict

    :param path: str

    :param sep: str

    :return: dict or list or leaf of treemap

    """
    return reduce(adict.__class__.get, [p for p in op.split(sep) if p], adict)


Instruction:
Can you create a Python 3 function that
gives a nested dictionary this returns all its leave elements in a list.

Response:
def get_dict_leaves(data):
    """
    Given a nested dictionary, this returns all its leave elements in a list.

    :param adict:

    :return: list
    """
    result = []
    if isinstance(data, dict):
        for item in data.values():
            result.extend(get_dict_leaves(item))
    elif isinstance(data, list):
        result.extend(data)
    else:
        result.append(data)

    return result


Instruction:
How would you explain what the following Python 3 function does
def get_possible_paths(base_path, path_regex):
    if not path_regex:
        return []

    if len(path_regex) < 1:
        return []

    if path_regex[0] == os.sep:
        path_regex = path_regex[1:]

    rest_files = ''
    if os.sep in path_regex:
        #split by os.sep
        node_names = path_regex.partition(os.sep)
        first_node = node_names[0]
        rest_nodes = node_names[2]

        folder_names = filter_list(os.listdir(base_path), first_node)

        for nom in folder_names:
            new_base = op.join(base_path, nom)
            if op.isdir(new_base):
                rest_files = get_possible_paths(new_base, rest_nodes)
    else:
        rest_files = filter_list(os.listdir(base_path), path_regex)

    files = []
    if rest_files:
        files = [op.join(base_path, f) for f in rest_files]

    return files

Response: Returns a list of all possible paths within base_path.


Instruction:
Make a summary of the following Python 3 code
def _import_config(filepath):
        if not op.isfile(filepath):
            raise IOError('Data config file not found. '
                          'Got: {0}'.format(filepath))

        cfg = import_pyfile(filepath)

        if not hasattr(cfg, 'root_path'):
            raise KeyError('Config file root_path key not found.')

        if not hasattr(cfg, 'filetree'):
            raise KeyError('Config file filetree key not found.')

        return cfg.root_path, cfg.filetree

Response: Imports filetree and root_path variable values from the filepath.


Instruction:
Write a Python 3 script to
remove the nodes that match the pattern.

Response:
def remove_nodes(self, pattern, adict):
        """
        Remove the nodes that match the pattern.
        """
        mydict = self._filetree if adict is None else adict

        if isinstance(mydict, dict):
            for nom in mydict.keys():
                if isinstance(mydict[nom], dict):
                    matchs = filter_list(mydict[nom], pattern)
                    for nom in matchs:
                        mydict = self.remove_nodes(pattern, mydict[nom])
                        mydict.pop(nom)
                else:
                    mydict[nom] = filter_list(mydict[nom], pattern)
        else:
            matchs = set(filter_list(mydict, pattern))
            mydict = set(mydict) - matchs

        return mydict


Instruction:
Can you write a function in Python 3 where it
returns the number of nodes that match the pattern.

Response:
def count_node_match(self, pattern, adict=None):
        """
        Return the number of nodes that match the pattern.

        :param pattern:

        :param adict:
        :return: int
        """
        mydict = self._filetree if adict is None else adict

        k = 0
        if isinstance(mydict, dict):
            names = mydict.keys()
            k += len(filter_list(names, pattern))
            for nom in names:
                k += self.count_node_match(pattern, mydict[nom])
        else:
            k = len(filter_list(mydict, pattern))

        return k


Instruction:
How would you explain what the following Python 3 function does
def as_float_array(X, copy=True, force_all_finite=True):
    if isinstance(X, np.matrix) or (not isinstance(X, np.ndarray)
                                    and not sp.issparse(X)):
        return check_array(X, ['csr', 'csc', 'coo'], dtype=np.float64,
                           copy=copy, force_all_finite=force_all_finite,
                           ensure_2d=False)
    elif sp.issparse(X) and X.dtype in [np.float32, np.float64]:
        return X.copy() if copy else X
    elif X.dtype in [np.float32, np.float64]:  # is numpy array
        return X.copy('F' if X.flags['F_CONTIGUOUS'] else 'C') if copy else X
    else:
        return X.astype(np.float32 if X.dtype == np.int32 else np.float64)

Response: Converts an array - like to an array of floats


Instruction:
Create a Python 3 function to
return number of samples in array - like x.

Response:
def _num_samples(x):
    """Return number of samples in array-like x."""
    if not hasattr(x, '__len__') and not hasattr(x, 'shape'):
        if hasattr(x, '__array__'):
            x = np.asarray(x)
        else:
            raise TypeError("Expected sequence or array-like, got %r" % x)
    return x.shape[0] if hasattr(x, 'shape') else len(x)


Instruction:
Write a Python 3 function that can
check that all arrays have consistent length.

Response:
def check_consistent_length(*arrays):
    """Check that all arrays have consistent first dimensions.

    Checks whether all objects in arrays have the same shape or length.

    Parameters
    ----------
    arrays : list or tuple of input objects.
        Objects that will be checked for consistent length.
    """

    uniques = np.unique([_num_samples(X) for X in arrays if X is not None])
    if len(uniques) > 1:
        raise ValueError("Found arrays with inconsistent numbers of samples: %s"
                         % str(uniques))


Instruction:
Make a summary of the following Python 3 code
def indexable(*iterables):
    result = []
    for X in iterables:
        if sp.issparse(X):
            result.append(X.tocsr())
        elif hasattr(X, "__getitem__") or hasattr(X, "iloc"):
            result.append(X)
        elif X is None:
            result.append(X)
        else:
            result.append(np.array(X))
    check_consistent_length(*result)
    return result

Response: Make arrays indexable for cross - validation.


Instruction:
Implement a Python 3 function for
inputting validation on an array.

Response:
def check_array(array, accept_sparse=None, dtype=None, order=None, copy=False,
                force_all_finite=True, ensure_2d=True, allow_nd=False):
    """Input validation on an array, list, sparse matrix or similar.

    By default, the input is converted to an at least 2nd numpy array.

    Parameters
    ----------
    array : object
        Input object to check / convert.

    accept_sparse : string, list of string or None (default=None)
        String[s] representing allowed sparse matrix formats, such as 'csc',
        'csr', etc.  None means that sparse matrix input will raise an error.
        If the input is sparse but not in the allowed format, it will be
        converted to the first listed format.

    order : 'F', 'C' or None (default=None)
        Whether an array will be forced to be fortran or c-style.

    copy : boolean (default=False)
        Whether a forced copy will be triggered. If copy=False, a copy might
        be triggered by a conversion.

    force_all_finite : boolean (default=True)
        Whether to raise an error on np.inf and np.nan in X.

    ensure_2d : boolean (default=True)
        Whether to make X at least 2d.

    allow_nd : boolean (default=False)
        Whether to allow X.ndim > 2.

    Returns
    -------
    X_converted : object
        The converted and validated X.
    """
    if isinstance(accept_sparse, str):
        accept_sparse = [accept_sparse]

    if sp.issparse(array):
        array = _ensure_sparse_format(array, accept_sparse, dtype, order,
                                      copy, force_all_finite)
    else:
        if ensure_2d:
            array = np.atleast_2d(array)
        array = np.array(array, dtype=dtype, order=order, copy=copy)
        if not allow_nd and array.ndim >= 3:
            raise ValueError("Found array with dim %d. Expected <= 2" %
                             array.ndim)
        if force_all_finite:
            _assert_all_finite(array)

    return array


Instruction:
Here you have a function in Python 3, explain what it does
def check_X_y(X, y, accept_sparse=None, dtype=None, order=None, copy=False,
              force_all_finite=True, ensure_2d=True, allow_nd=False,
              multi_output=False):
    """Input validation for standard estimators.

    Checks X and y for consistent length, enforces X 2d and y 1d.
    Standard input checks are only applied to y. For multi-label y,
    set multi_ouput=True to allow 2d and sparse y.

    Parameters
    ----------
    X : nd-array, list or sparse matrix
        Input data.

    y : nd-array, list or sparse matrix
        Labels.

    accept_sparse : string, list of string or None (default=None)
        String[s] representing allowed sparse matrix formats, such as 'csc',
        'csr', etc.  None means that sparse matrix input will raise an error.
        If the input is sparse but not in the allowed format, it will be
        converted to the first listed format.

    order : 'F', 'C' or None (default=None)
        Whether an array will be forced to be fortran or c-style.

    copy : boolean (default=False)
        Whether a forced copy will be triggered. If copy=False, a copy might
        be triggered by a conversion.

    force_all_finite : boolean (default=True)
        Whether to raise an error on np.inf and np.nan in X.

    ensure_2d : boolean (default=True)
        Whether to make X at least 2d.

    allow_nd : boolean (default=False)
        Whether to allow X.ndim > 2.

    Returns
    -------
    X_converted : object
        The converted and validated X.
    """
    X = check_array(X, accept_sparse, dtype, order, copy, force_all_finite,
                    ensure_2d, allow_nd)
    if multi_output:
        y = check_array(y, 'csr', force_all_finite=True, ensure_2d=False)
    else:
        y = column_or_1d(y, warn=True)
        _assert_all_finite(y)

    check_consistent_length(X, y)

    return X, y

Response: Input validation for standard estimators.


Instruction:
Here you have a function in Python 3, explain what it does
def column_or_1d(y, warn=False):
    shape = np.shape(y)
    if len(shape) == 1:
        return np.ravel(y)
    if len(shape) == 2 and shape[1] == 1:
        if warn:
            warnings.warn("A column-vector y was passed when a 1d array was"
                          " expected. Please change the shape of y to "
                          "(n_samples, ), for example using ravel().",
                          DataConversionWarning, stacklevel=2)
        return np.ravel(y)

    raise ValueError("bad input shape {0}".format(shape))

Response: Ravel column or 1d numpy array.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def warn_if_not_float(X, estimator='This algorithm'):
    if not isinstance(estimator, str):
        estimator = estimator.__class__.__name__
    if X.dtype.kind != 'f':
        warnings.warn("%s assumes floating point values as input, "
                      "got %s" % (estimator, X.dtype))
        return True
    return False

Response: Warning utility function to check that data type is floating point.


Instruction:
How would you implement a function in Python 3 that
converts an arbitrary array to numpy. ndarray.

Response:
def as_ndarray(arr, copy=False, dtype=None, order='K'):
    """Convert an arbitrary array to numpy.ndarray.

    In the case of a memmap array, a copy is automatically made to break the
    link with the underlying file (whatever the value of the "copy" keyword).

    The purpose of this function is mainly to get rid of memmap objects, but
    it can be used for other purposes. In particular, combining copying and
    casting can lead to performance improvements in some cases, by avoiding
    unnecessary copies.

    If not specified, input array order is preserved, in all cases, even when
    a copy is requested.

    Caveat: this function does not copy during bool to/from 1-byte dtype
    conversions. This can lead to some surprising results in some rare cases.
    Example:

        a = numpy.asarray([0, 1, 2], dtype=numpy.int8)
        b = as_ndarray(a, dtype=bool)  # array([False, True, True], dtype=bool)
        c = as_ndarray(b, dtype=numpy.int8)  # array([0, 1, 2], dtype=numpy.int8)

    The usually expected result for the last line would be array([0, 1, 1])
    because True evaluates to 1. Since there is no copy made here, the original
    array is recovered.

    Parameters
    ----------
    arr: array-like
        input array. Any value accepted by numpy.asarray is valid.

    copy: bool
        if True, force a copy of the array. Always True when arr is a memmap.

    dtype: any numpy dtype
        dtype of the returned array. Performing copy and type conversion at the
        same time can in some cases avoid an additional copy.

    order: string
        gives the order of the returned array.
        Valid values are: "C", "F", "A", "K", None.
        default is "K". See ndarray.copy() for more information.

    Returns
    -------
    ret: np.ndarray
        Numpy array containing the same data as arr, always of class
        numpy.ndarray, and with no link to any underlying file.
    """
    if order not in ('C', 'F', 'A', 'K', None):
        raise ValueError("Invalid value for 'order': {}".format(str(order)))

    if isinstance(arr, np.memmap):
        if dtype is None:
            if order in ('K', 'A', None):
                ret = np.array(np.asarray(arr), copy=True)
            else:
                ret = np.array(np.asarray(arr), copy=True, order=order)
        else:
            if order in ('K', 'A', None):
                # always copy (even when dtype does not change)
                ret = np.asarray(arr).astype(dtype)
            else:
                # load data from disk without changing order
                # Changing order while reading through a memmap is incredibly
                # inefficient.
                ret = _asarray(np.array(arr, copy=True), dtype=dtype, order=order)

    elif isinstance(arr, np.ndarray):
        ret = _asarray(arr, dtype=dtype, order=order)
        # In the present cas, np.may_share_memory result is always reliable.
        if np.may_share_memory(ret, arr) and copy:
            # order-preserving copy
            ret = ret.T.copy().T if ret.flags['F_CONTIGUOUS'] else ret.copy()

    elif isinstance(arr, (list, tuple)):
        if order in ("A", "K"):
            ret = np.asarray(arr, dtype=dtype)
        else:
            ret = np.asarray(arr, dtype=dtype, order=order)

    else:
        raise ValueError("Type not handled: {}".format(arr.__class__))

    return ret


Instruction:
How would you code a function in Python 3 to
call FSL tools to apply transformations to a given atlas to a functional image. Given the transformation matrices. Parameters ---------- atlas_filepath: str Path to the 3D atlas volume file. anatbrain_filepath: str Path to the anatomical brain volume file (skull-stripped and registered to the same space as the atlas, e.g., MNI). meanfunc_filepath: str Path to the average functional image to be used as reference in the last applywarp step. atlas2anat_nonlin_xfm_filepath: str Path to the atlas to anatomical brain linear transformation .mat file. If you have the inverse transformation, i.e., anatomical brain to atlas, set is_atlas2anat_inverted to True. is_atlas2anat_inverted: bool If False will have to calculate the inverse atlas2anat transformation to apply the transformations. This step will be performed with FSL invwarp. anat2func_lin_xfm_filepath: str Path to the anatomical to functional .mat linear transformation file. atlasinanat_out_filepath: str Path to output file which will contain the 3D atlas in the subject anatomical space. atlasinfunc_out_filepath: str Path to output file which will contain the 3D atlas in the subject functional space. verbose: bool If verbose will show DEBUG log info. rewrite: bool If True will re-run all the commands overwriting any existing file. Otherwise will check if each file exists and if it does won't run the command. parallel: bool If True will launch the commands using ${FSLDIR}/fsl_sub to use the cluster infrastructure you have setup with FSL (SGE or HTCondor).

Response:
def xfm_atlas_to_functional(atlas_filepath, anatbrain_filepath, meanfunc_filepath,
                            atlas2anat_nonlin_xfm_filepath, is_atlas2anat_inverted,
                            anat2func_lin_xfm_filepath,
                            atlasinanat_out_filepath, atlasinfunc_out_filepath,
                            interp='nn', rewrite=True, parallel=False):
    """Call FSL tools to apply transformations to a given atlas to a functional image.
    Given the transformation matrices.

    Parameters
    ----------
    atlas_filepath: str
        Path to the 3D atlas volume file.

    anatbrain_filepath: str
        Path to the anatomical brain volume file (skull-stripped and registered to the same space as the atlas,
        e.g., MNI).

    meanfunc_filepath: str
        Path to the average functional image to be used as reference in the last applywarp step.

    atlas2anat_nonlin_xfm_filepath: str
        Path to the atlas to anatomical brain linear transformation .mat file.
        If you have the inverse transformation, i.e., anatomical brain to atlas, set is_atlas2anat_inverted to True.

    is_atlas2anat_inverted: bool
        If False will have to calculate the inverse atlas2anat transformation to apply the transformations.
        This step will be performed with FSL invwarp.

    anat2func_lin_xfm_filepath: str
        Path to the anatomical to functional .mat linear transformation file.

    atlasinanat_out_filepath: str
        Path to output file which will contain the 3D atlas in the subject anatomical space.

    atlasinfunc_out_filepath: str
        Path to output file which will contain the 3D atlas in the subject functional space.

    verbose: bool
        If verbose will show DEBUG log info.

    rewrite: bool
        If True will re-run all the commands overwriting any existing file. Otherwise will check if
        each file exists and if it does won't run the command.

    parallel: bool
        If True will launch the commands using ${FSLDIR}/fsl_sub to use the cluster infrastructure you have setup
        with FSL (SGE or HTCondor).
    """
    if is_atlas2anat_inverted:
        # I already have the inverted fields I need
        anat_to_mni_nl_inv = atlas2anat_nonlin_xfm_filepath
    else:
        # I am creating the inverted fields then...need output file path:
        output_dir         = op.abspath   (op.dirname(atlasinanat_out_filepath))
        ext                = get_extension(atlas2anat_nonlin_xfm_filepath)
        anat_to_mni_nl_inv = op.join(output_dir, remove_ext(op.basename(atlas2anat_nonlin_xfm_filepath)) + '_inv' + ext)

    # setup the commands to be called
    invwarp_cmd   = op.join('${FSLDIR}', 'bin', 'invwarp')
    applywarp_cmd = op.join('${FSLDIR}', 'bin', 'applywarp')
    fslsub_cmd    = op.join('${FSLDIR}', 'bin', 'fsl_sub')

    # add fsl_sub before the commands
    if parallel:
        invwarp_cmd   = fslsub_cmd + ' ' + invwarp_cmd
        applywarp_cmd = fslsub_cmd + ' ' + applywarp_cmd

    # create the inverse fields
    if rewrite or (not is_atlas2anat_inverted and not op.exists(anat_to_mni_nl_inv)):
        log.debug('Creating {}.\n'.format(anat_to_mni_nl_inv))
        cmd  = invwarp_cmd + ' '
        cmd += '-w {} '.format(atlas2anat_nonlin_xfm_filepath)
        cmd += '-o {} '.format(anat_to_mni_nl_inv)
        cmd += '-r {} '.format(anatbrain_filepath)
        log.debug('Running {}'.format(cmd))
        check_call(cmd)

    # transform the atlas to anatomical space
    if rewrite or not op.exists(atlasinanat_out_filepath):
        log.debug('Creating {}.\n'.format(atlasinanat_out_filepath))
        cmd  = applywarp_cmd + ' '
        cmd += '--in={}     '.format(atlas_filepath)
        cmd += '--ref={}    '.format(anatbrain_filepath)
        cmd += '--warp={}   '.format(anat_to_mni_nl_inv)
        cmd += '--interp={} '.format(interp)
        cmd += '--out={}    '.format(atlasinanat_out_filepath)
        log.debug('Running {}'.format(cmd))
        check_call(cmd)

    # transform the atlas to functional space
    if rewrite or not op.exists(atlasinfunc_out_filepath):
        log.debug('Creating {}.\n'.format(atlasinfunc_out_filepath))
        cmd  = applywarp_cmd + ' '
        cmd += '--in={}     '.format(atlasinanat_out_filepath)
        cmd += '--ref={}    '.format(meanfunc_filepath)
        cmd += '--premat={} '.format(anat2func_lin_xfm_filepath)
        cmd += '--interp={} '.format(interp)
        cmd += '--out={}    '.format(atlasinfunc_out_filepath)
        log.debug('Running {}'.format(cmd))
        check_call(cmd)


Instruction:
Can you tell what is the following Python 3 function doing
def fwhm2sigma(fwhm):
    fwhm = np.asarray(fwhm)
    return fwhm / np.sqrt(8 * np.log(2))

Response: Convert a FWHM value to sigma in a Gaussian kernel.


Instruction:
Write a Python 3 script to
convert a sigma in a Gaussian kernel to a FWHM value.

Response:
def sigma2fwhm(sigma):
    """Convert a sigma in a Gaussian kernel to a FWHM value.

    Parameters
    ----------
    sigma: float or numpy.array
       sigma value or values

    Returns
    -------
    fwhm: float or numpy.array
       fwhm values corresponding to `sigma` values
    """
    sigma = np.asarray(sigma)
    return np.sqrt(8 * np.log(2)) * sigma


Instruction:
Create a Python 3 function for
smoothing images with a Gaussian filter.

Response:
def _smooth_data_array(arr, affine, fwhm, copy=True):
    """Smooth images with a a Gaussian filter.

    Apply a Gaussian filter along the three first dimensions of arr.

    Parameters
    ----------
    arr: numpy.ndarray
        3D or 4D array, with image number as last dimension.

    affine: numpy.ndarray
        Image affine transformation matrix for image.

    fwhm: scalar, numpy.ndarray
        Smoothing kernel size, as Full-Width at Half Maximum (FWHM) in millimeters.
        If a scalar is given, kernel width is identical on all three directions.
        A numpy.ndarray must have 3 elements, giving the FWHM along each axis.

    copy: bool
        if True, will make a copy of the input array. Otherwise will directly smooth the input array.

    Returns
    -------
    smooth_arr: numpy.ndarray
    """

    if arr.dtype.kind == 'i':
        if arr.dtype == np.int64:
            arr = arr.astype(np.float64)
        else:
            arr = arr.astype(np.float32)
    if copy:
        arr = arr.copy()

    # Zeroe possible NaNs and Inf in the image.
    arr[np.logical_not(np.isfinite(arr))] = 0

    try:
        # Keep the 3D part of the affine.
        affine = affine[:3, :3]

        # Convert from FWHM in mm to a sigma.
        fwhm_sigma_ratio = np.sqrt(8 * np.log(2))
        vox_size         = np.sqrt(np.sum(affine ** 2, axis=0))
        sigma            = fwhm / (fwhm_sigma_ratio * vox_size)
        for n, s in enumerate(sigma):
            ndimage.gaussian_filter1d(arr, s, output=arr, axis=n)
    except:
        raise ValueError('Error smoothing the array.')
    else:
        return arr


Instruction:
Write a Python 3 function that can
smooth images using a Gaussian filter along the three first dimensions of each image in images.

Response:
def smooth_imgs(images, fwhm):
    """Smooth images using a Gaussian filter.

    Apply a Gaussian filter along the three first dimensions of each image in images.
    In all cases, non-finite values in input are zeroed.

    Parameters
    ----------
    imgs: str or img-like object or iterable of img-like objects
        See boyle.nifti.read.read_img
        Image(s) to smooth.

    fwhm: scalar or numpy.ndarray
        Smoothing kernel size, as Full-Width at Half Maximum (FWHM) in millimeters.
        If a scalar is given, kernel width is identical on all three directions.
        A numpy.ndarray must have 3 elements, giving the FWHM along each axis.

    Returns
    -------
    smooth_imgs: nibabel.Nifti1Image or list of.
        Smooth input image/s.
    """
    if fwhm <= 0:
        return images

    if not isinstance(images, string_types) and hasattr(images, '__iter__'):
        only_one = False
    else:
        only_one = True
        images = [images]

    result = []
    for img in images:
        img    = check_img(img)
        affine = img.get_affine()
        smooth = _smooth_data_array(img.get_data(), affine, fwhm=fwhm, copy=True)
        result.append(nib.Nifti1Image(smooth, affine))

    if only_one:
        return result[0]
    else:
        return result


Instruction:
Can you generate the documentation for the following Python 3 function
def _smooth_array(arr, affine, fwhm=None, ensure_finite=True, copy=True, **kwargs):

    if arr.dtype.kind == 'i':
        if arr.dtype == np.int64:
            arr = arr.astype(np.float64)
        else:
            # We don't need crazy precision
            arr = arr.astype(np.float32)
    if copy:
        arr = arr.copy()

    if ensure_finite:
        # SPM tends to put NaNs in the data outside the brain
        arr[np.logical_not(np.isfinite(arr))] = 0

    if fwhm == 'fast':
        arr = _fast_smooth_array(arr)
    elif fwhm is not None:
        # Keep only the scale part.
        affine = affine[:3, :3]

        # Convert from a FWHM to a sigma:
        fwhm_over_sigma_ratio = np.sqrt(8 * np.log(2))
        vox_size = np.sqrt(np.sum(affine ** 2, axis=0))
        sigma = fwhm / (fwhm_over_sigma_ratio * vox_size)
        for n, s in enumerate(sigma):
            ndimage.gaussian_filter1d(arr, s, output=arr, axis=n, **kwargs)

    return arr

Response: Smooth images by applying a Gaussian filter along the three first dimensions of arr.


Instruction:
Explain what the following Python 3 code does
def smooth_img(imgs, fwhm, **kwargs):

    # Use hasattr() instead of isinstance to workaround a Python 2.6/2.7 bug
    # See http://bugs.python.org/issue7624
    if hasattr(imgs, "__iter__") \
       and not isinstance(imgs, string_types):
        single_img = False
    else:
        single_img = True
        imgs = [imgs]

    ret = []
    for img in imgs:
        img = check_niimg(img)
        affine = img.get_affine()
        filtered = _smooth_array(img.get_data(), affine, fwhm=fwhm,
                                 ensure_finite=True, copy=True, **kwargs)
        ret.append(new_img_like(img, filtered, affine, copy_header=True))

    if single_img:
        return ret[0]
    else:
        return ret

Response: Smooth images by applying a Gaussian filter along the three first dimensions of arr.


Instruction:
Explain what the following Python 3 code does
def signed_session(self, session=None):

        if session:
            session = super(ClientCertAuthentication, self).signed_session(session)
        else:
            session = super(ClientCertAuthentication, self).signed_session()

        if self.cert is not None:
            session.cert = self.cert
        if self.ca_cert is not None:
            session.verify = self.ca_cert
        if self.no_verify:
            session.verify = False

        return session

Response:
Create requests. Session object with any required auth headers
        applied.


Instruction:
Can you generate the documentation for the following Python 3 function
def signed_session(self, session=None):

        from sfctl.config import (aad_metadata, aad_cache)

        if session:
            session = super(AdalAuthentication, self).signed_session(session)
        else:
            session = super(AdalAuthentication, self).signed_session()

        if self.no_verify:
            session.verify = False

        authority_uri, cluster_id, client_id = aad_metadata()
        existing_token, existing_cache = aad_cache()
        context = adal.AuthenticationContext(authority_uri,
                                             cache=existing_cache)
        new_token = context.acquire_token(cluster_id,
                                          existing_token['userId'], client_id)
        header = "{} {}".format("Bearer", new_token['accessToken'])
        session.headers['Authorization'] = header
        return session

Response: Create requests. Session object with AAD auth headers


Instruction:
Can you tell what is the following Python 3 function doing
def voxspace_to_mmspace(img):
    shape, affine = img.shape[:3], img.affine
    coords = np.array(np.meshgrid(*(range(i) for i in shape), indexing='ij'))
    coords = np.rollaxis(coords, 0, len(shape) + 1)
    mm_coords = nib.affines.apply_affine(affine, coords)

    return mm_coords

Response: Return a grid with coordinates in 3D physical space for img.


Instruction:
Implement a function in Python 3 to
convert a Voxel coordinates to a Triplet with real 3D world coordinates.

Response:
def voxcoord_to_mm(cm, i, j, k):
    '''
    Parameters
    ----------
    cm: nipy.core.reference.coordinate_map.CoordinateMap

    i, j, k: floats
        Voxel coordinates

    Returns
    -------
    Triplet with real 3D world coordinates

    '''
    try:
        mm = cm([i, j, k])
    except Exception as exc:
        raise Exception('Error on converting coordinates.') from exc
    else:
        return mm


Instruction:
Write a Python 3 script for
converting a coordinate map to a 3D voxel coordinates.

Response:
def mm_to_voxcoord(cm, x, y, z):
    '''
    Parameters
    ----------
    cm: nipy.core.reference.coordinate_map.CoordinateMap

    x, y, z: floats
        Physical coordinates

    Returns
    -------
    Triplet with 3D voxel coordinates
    '''
    try:
        vox = cm.inverse()([x, y, z])
    except Exception as exc:
        raise Exception('Error on converting coordinates') from exc
    else:
        return vox


Instruction:
Can you write a function in Python 3 where it
gets a 3D CoordinateMap from img.

Response:
def get_3D_coordmap(img):
    '''
    Gets a 3D CoordinateMap from img.

    Parameters
    ----------
    img: nib.Nifti1Image or nipy Image

    Returns
    -------
    nipy.core.reference.coordinate_map.CoordinateMap
    '''
    if isinstance(img, nib.Nifti1Image):
        img = nifti2nipy(img)

    if img.ndim == 4:
        from nipy.core.reference.coordinate_map import drop_io_dim
        cm = drop_io_dim(img.coordmap, 3)
    else:
        cm = img.coordmap

    return cm


Instruction:
Write a Python 3 script to
return the header and affine matrix from a Nifti file.

Response:
def get_img_info(image):
    """Return the header and affine matrix from a Nifti file.

    Parameters
    ----------
    image: img-like object or str
        Can either be:
        - a file path to a Nifti image
        - any object with get_data() and get_affine() methods, e.g., nibabel.Nifti1Image.
        If niimg is a string, consider it as a path to Nifti image and
        call nibabel.load on it. If it is an object, check if get_data()
        and get_affine() methods are present, raise TypeError otherwise.

    Returns
    -------
    hdr, aff
    """
    try:
        img = check_img(image)
    except Exception as exc:
        raise Exception('Error reading file {0}.'.format(repr_imgs(image))) from exc
    else:
        return img.get_header(), img.get_affine()


Instruction:
How would you code a function in Python 3 to
return the voxel matrix of the Nifti image.

Response:
def get_img_data(image, copy=True):
    """Return the voxel matrix of the Nifti file.
    If safe_mode will make a copy of the img before returning the data, so the input image is not modified.

    Parameters
    ----------
    image: img-like object or str
        Can either be:
        - a file path to a Nifti image
        - any object with get_data() and get_affine() methods, e.g., nibabel.Nifti1Image.
        If niimg is a string, consider it as a path to Nifti image and
        call nibabel.load on it. If it is an object, check if get_data()
        and get_affine() methods are present, raise TypeError otherwise.

    copy: bool
    If safe_mode will make a copy of the img before returning the data, so the input image is not modified.

    Returns
    -------
    array_like
    """
    try:
        img = check_img(image)
        if copy:
            return get_data(img)
        else:
            return img.get_data()
    except Exception as exc:
        raise Exception('Error when reading file {0}.'.format(repr_imgs(image))) from exc


Instruction:
Given the following Python 3 function, write the documentation
def load_nipy_img(nii_file):
    # delayed import because could not install nipy on Python 3 on OSX
    import nipy

    if not os.path.exists(nii_file):
        raise FileNotFound(nii_file)

    try:
        return nipy.load_image(nii_file)
    except Exception as exc:
        raise Exception('Reading file {0}.'.format(repr_imgs(nii_file))) from exc

Response: Read a Nifti file and return as nipy. Image


Instruction:
Given the following Python 3 function, write the documentation
def niftilist_to_array(img_filelist, outdtype=None):
    try:
        first_img = img_filelist[0]
        vol       = get_img_data(first_img)
    except IndexError as ie:
        raise Exception('Error getting the first item of img_filelis: {}'.format(repr_imgs(img_filelist[0]))) from ie

    if not outdtype:
        outdtype = vol.dtype

    outmat = np.zeros((len(img_filelist), np.prod(vol.shape)), dtype=outdtype)

    try:
        for i, img_file in enumerate(img_filelist):
            vol = get_img_data(img_file)
            outmat[i, :] = vol.flatten()
    except Exception as exc:
        raise Exception('Error on reading file {0}.'.format(img_file)) from exc

    return outmat, vol.shape

Response: This function takes a list of absolute paths to nifti files and creates a Numpy array containing the data of the volumes.


Instruction:
Can you tell what is the following Python 3 function doing
def _crop_img_to(image, slices, copy=True):

    img    = check_img(image)
    data   = img.get_data()
    affine = img.get_affine()

    cropped_data = data[slices]
    if copy:
        cropped_data   = cropped_data.copy()

    linear_part        = affine[:3, :3]
    old_origin         = affine[:3,  3]
    new_origin_voxel   = np.array([s.start for s in slices])
    new_origin         = old_origin + linear_part.dot(new_origin_voxel)

    new_affine         = np.eye(4)
    new_affine[:3, :3] = linear_part
    new_affine[:3,  3] = new_origin

    new_img = nib.Nifti1Image(cropped_data, new_affine)

    return new_img

Response: Crop an image to a smaller size and modify the affine accordingly.


Instruction:
Can you tell what is the following Python 3 function doing
def crop_img(image, rtol=1e-8, copy=True):

    img              = check_img(image)
    data             = img.get_data()
    infinity_norm    = max(-data.min(), data.max())
    passes_threshold = np.logical_or(data < -rtol * infinity_norm,
                                     data >  rtol * infinity_norm)

    if data.ndim == 4:
        passes_threshold = np.any(passes_threshold, axis=-1)

    coords = np.array(np.where(passes_threshold))
    start  = coords.min(axis=1)
    end    = coords.max(axis=1) + 1

    # pad with one voxel to avoid resampling problems
    start = np.maximum(start - 1, 0)
    end   = np.minimum(end   + 1, data.shape[:3])

    slices = [slice(s, e) for s, e in zip(start, end)]

    return _crop_img_to(img, slices, copy=copy)

Response: Crop an image as much as possible and return a new image with the same size.


Instruction:
Create a Python 3 function for
creating a new image of the same type as the reference image.

Response:
def new_img_like(ref_niimg, data, affine=None, copy_header=False):
    """Create a new image of the same class as the reference image

    Parameters
    ----------
    ref_niimg: image
        Reference image. The new image will be of the same type.

    data: numpy array
        Data to be stored in the image

    affine: 4x4 numpy array, optional
        Transformation matrix

    copy_header: boolean, optional
        Indicated if the header of the reference image should be used to
        create the new image

    Returns
    -------
    new_img: image
        A loaded image with the same type (and header) as the reference image.
    """
    # Hand-written loading code to avoid too much memory consumption
    if not (hasattr(ref_niimg, 'get_data')
              and hasattr(ref_niimg,'get_affine')):
        if isinstance(ref_niimg, _basestring):
            ref_niimg = nib.load(ref_niimg)
        elif operator.isSequenceType(ref_niimg):
            ref_niimg = nib.load(ref_niimg[0])
        else:
            raise TypeError(('The reference image should be a niimg, %r '
                            'was passed') % ref_niimg )

    if affine is None:
        affine = ref_niimg.get_affine()
    if data.dtype == bool:
        default_dtype = np.int8
        if (LooseVersion(nib.__version__) >= LooseVersion('1.2.0') and
                isinstance(ref_niimg, nib.freesurfer.mghformat.MGHImage)):
            default_dtype = np.uint8
        data = as_ndarray(data, dtype=default_dtype)
    header = None
    if copy_header:
        header = copy.copy(ref_niimg.get_header())
        header['scl_slope'] = 0.
        header['scl_inter'] = 0.
        header['glmax'] = 0.
        header['cal_max'] = np.max(data) if data.size > 0 else 0.
        header['cal_max'] = np.min(data) if data.size > 0 else 0.
    return ref_niimg.__class__(data, affine, header=header)


Instruction:
Write a Python 3 script to
save variables to HDF5 file.

Response:
def save_variables_to_hdf5(file_path, variables, mode='w', h5path='/'):
    """
    Parameters
    ----------
    file_path: str

    variables: dict
        Dictionary with objects. Object name -> object

    mode: str
        HDF5 file access mode
        See h5py documentation for details.
        r   Readonly, file must exist
        r+  Read/write, file must exist
        w   Create file, truncate if exists
        w-  Create file, fail if exists
        a   Read/write if exists, create otherwise (default)

    Notes
    -----
    It is recommended to use numpy arrays as objects.
    List or tuples of strings won't work, convert them into numpy.arrays before.
    """
    if not isinstance(variables, dict):
        raise ValueError('Expected `variables` to be a dict, got a {}.'.format(type(variables)))

    if not variables:
        raise ValueError('Expected `variables` to be a non-empty dict.')

    h5file  = h5py.File(file_path, mode=mode)
    h5group = h5file.require_group(h5path)

    for vn in variables:
        data = variables[vn]

        # fix for string numpy arrays
        if hasattr(data, 'dtype') and (data.dtype.type is np.string_ or data.dtype.type is np.unicode_):
            dt   = h5py.special_dtype(vlen=str)
            data = data.astype(dt)

        if isinstance(data, dict):
            for key in data:
                #h5group.create_dataset(str(key))
                #import ipdb
                #ipdb.set_trace()
                h5group[str(key)] = data[key]

        elif isinstance(data, list):
            for idx, item in enumerate(data):
                #h5group.create_dataset(str(idx))
                h5group[str(idx)] = item
        else:
            h5group[vn] = data

    h5file.close()


Instruction:
Can you tell what is the following Python 3 function doing
def get_h5file(file_path, mode='r'):
    if not op.exists(file_path):
        raise IOError('Could not find file {}.'.format(file_path))

    try:
        h5file = h5py.File(file_path, mode=mode)
    except:
        raise
    else:
        return h5file

Response: Returns the h5py. File given its file path.


Instruction:
Can you generate the documentation for the following Python 3 function
def extract_datasets(h5file, h5path='/'):
    if isinstance(h5file, str):
        _h5file = h5py.File(h5file, mode='r')
    else:
        _h5file = h5file

    _datasets = get_datasets(_h5file, h5path)
    datasets  = OrderedDict()
    try:
        for ds in _datasets:
            datasets[ds.name.split('/')[-1]] = ds[:]
    except:
        raise RuntimeError('Error reading datasets in {}/{}.'.format(_h5file.filename, h5path))
    finally:
        if isinstance(h5file, str):
            _h5file.close()

    return datasets

Response: Return all datasets contained in file_path in h5file in an OrderedDict.


Instruction:
Can you tell what is the following Python 3 function doing
def _get_node_names(h5file, h5path='/', node_type=h5py.Dataset):
    if isinstance(h5file, str):
        _h5file = get_h5file(h5file, mode='r')
    else:
        _h5file = h5file

    if not h5path.startswith('/'):
        h5path = '/' + h5path

    names = []
    try:
        h5group = _h5file.require_group(h5path)

        for node in _hdf5_walk(h5group, node_type=node_type):
            names.append(node.name)
    except:
        raise RuntimeError('Error getting node names from {}/{}.'.format(_h5file.filename, h5path))
    finally:
        if isinstance(h5file, str):
            _h5file.close()

    return names

Response: Get the node names within the HDF5 file h5path.


Instruction:
Can you tell what is the following Python 3 function doing
def mask(self, image):
        if image is None:
            self._mask = None

        try:
            mask = load_mask(image)
        except Exception as exc:
            raise Exception('Could not load mask image {}.'.format(image)) from exc
        else:
            self._mask = mask

Response:
setter
        - Sets the mask attribute of the object.


Instruction:
Implement a function in Python 3 to
check if two images are compatible with the current set of items.

Response:
def check_compatibility(self, one_img, another_img=None):
        """
        Parameters
        ----------
        one_img: str or img-like object.
            See NeuroImage constructor docstring.

        anoter_img: str or img-like object.
            See NeuroImage constructor docstring.
            If None will use the first image of self.images, if there is any.

        Raises
        ------
        NiftiFilesNotCompatible
            If one_img and another_img aren't compatible.

        ValueError
            If another_img is None and there are no other images in this set.
        """
        if another_img is None:
            if len(self.items) > 0:
                another_img = self.items[0]
            else:
                raise ValueError('self.items is empty, need an image to compare '
                                 'with {}'.format(repr_imgs(one_img)))

        try:
            if self.all_compatible:
                check_img_compatibility(one_img, another_img)
            if self.mask is not None:
                check_img_compatibility(one_img, self.mask, only_check_3d=True)
        except:
            raise


Instruction:
Write a Python 3 script to
set the labels of the related object.

Response:
def set_labels(self, labels):
        """
        Parameters
        ----------
        labels: list of int or str
            This list will be checked to have the same size as

        Raises
        ------
        ValueError
            if len(labels) != self.n_subjs
        """
        if not isinstance(labels, string_types) and len(labels) != self.n_subjs:
            raise ValueError('The number of given labels ({}) is not the same '
                             'as the number of subjects ({}).'.format(len(labels), self.n_subjs))

        self.labels = labels


Instruction:
Can you tell what is the following Python 3 function doing
def _load_images_and_labels(self, images, labels=None):
        if not isinstance(images, (list, tuple)):
            raise ValueError('Expected an iterable (list or tuple) of strings or img-like objects. '
                             'Got a {}.'.format(type(images)))

        if not len(images) > 0:
            raise ValueError('Expected an iterable (list or tuple) of strings or img-like objects '
                             'of size higher than 0. Got {} items.'.format(len(images)))

        if labels is not None and len(labels) != len(images):
            raise ValueError('Expected the same length for image set ({}) and '
                             'labels list ({}).'.format(len(images), len(labels)))

        first_file = images[0]
        if first_file:
            first_img = NeuroImage(first_file)
        else:
            raise('Error reading image {}.'.format(repr_imgs(first_file)))

        for idx, image in enumerate(images):
            try:
                img = NeuroImage(image)
                self.check_compatibility(img, first_img)
            except:
                log.exception('Error reading image {}.'.format(repr_imgs(image)))
                raise
            else:
                self.items.append(img)

        self.set_labels(labels)

Response: Read the images and labels into self. items and set the labels.


Instruction:
Can you write a function in Python 3 where it
returns numpy. ndarray with the masked or flatten image data and the relevant information.

Response:
def to_matrix(self, smooth_fwhm=0, outdtype=None):
        """Return numpy.ndarray with the masked or flatten image data and
           the relevant information (mask indices and volume shape).

        Parameters
        ----------
        smooth__fwhm: int
            Integer indicating the size of the FWHM Gaussian smoothing kernel
            to smooth the subject volumes before creating the data matrix

        outdtype: dtype
            Type of the elements of the array, if None will obtain the dtype from
            the first nifti file.

        Returns
        -------
        outmat, mask_indices, vol_shape

        outmat: Numpy array with shape N x prod(vol.shape)
                containing the N files as flat vectors.

        mask_indices: matrix with indices of the voxels in the mask

        vol_shape: Tuple with shape of the volumes, for reshaping.
        """
        if not self.all_compatible:
            raise ValueError("`self.all_compatible` must be True in order to use this function.")

        if not outdtype:
            outdtype = self.items[0].dtype

        # extract some info from the mask
        n_voxels     = None
        mask_indices = None
        mask_shape   = self.items[0].shape[:3]
        if self.has_mask:
            mask_arr     = self.mask.get_data()
            mask_indices = np.nonzero(mask_arr)
            mask_shape   = self.mask.shape
            n_voxels     = np.count_nonzero(mask_arr)

        # if the mask is empty will use the whole image
        if n_voxels is None:
            log.debug('Non-zero voxels have not been found in mask {}'.format(self.mask))
            n_voxels     = np.prod(mask_shape)
            mask_indices = None

        # get the shape of the flattened subject data
        ndims = self.items[0].ndim
        if ndims == 3:
            subj_flat_shape = (n_voxels, )
        elif ndims == 4:
            subj_flat_shape = (n_voxels, self.items[0].shape[3])
        else:
            raise NotImplementedError('The subject images have {} dimensions. '
                                      'Still have not implemented t_matrix for this shape.'.format(ndims))

        # create and fill the big matrix
        outmat = np.zeros((self.n_subjs, ) + subj_flat_shape, dtype=outdtype)
        try:
            for i, image in enumerate(self.items):
                if smooth_fwhm > 0:
                    image.fwhm = smooth_fwhm

                if self.has_mask:
                    image.set_mask(self.mask)

                outmat[i, :], _, _ = image.mask_and_flatten()
                image.clear_data()

        except Exception as exc:
            raise Exception('Error flattening file {0}'.format(image)) from exc
        else:
            return outmat, mask_indices, mask_shape


Instruction:
Can you generate the documentation for the following Python 3 function
def to_file(self, output_file, smooth_fwhm=0, outdtype=None):
        outmat, mask_indices, mask_shape = self.to_matrix(smooth_fwhm, outdtype)

        exporter = ExportData()
        content = {'data':         outmat,
                   'labels':       self.labels,
                   'mask_indices': mask_indices,
                   'mask_shape':   mask_shape, }

        if self.others:
            content.update(self.others)

        log.debug('Creating content in file {}.'.format(output_file))
        try:
            exporter.save_variables(output_file, content)
        except Exception as exc:
            raise Exception('Error saving variables to file {}.'.format(output_file)) from exc

Response: Save the Numpy array created from to_matrix function to the output_file.


Instruction:
Given the following Python 3 function, write the documentation
def _init_subj_data(self, subj_files):
        try:
            if isinstance(subj_files, list):
                self.from_list(subj_files)

            elif isinstance(subj_files, dict):
                self.from_dict(subj_files)
            else:
                raise ValueError('Could not recognize subj_files argument variable type.')
        except Exception as exc:
            raise Exception('Cannot read subj_files input argument.') from exc

Response: Initialize the object from a list of subj_files.


Instruction:
Can you generate the documentation for the following Python 3 function
def _load_image(file_path):
        if not os.path.exists(file_path):
            raise FileNotFound(file_path)

        try:
            nii_img           = load_nipy_img(file_path)
            nii_img.file_path = file_path
            return nii_img
        except Exception as exc:
            raise Exception('Reading file {0}.'.format(file_path)) from exc

Response: Loads a nipy. Image from a file and returns a nipy. Image object with a file_path member.


Instruction:
Write a Python 3 script for
smoothing the image using LinearFilter

Response:
def _smooth_img(nii_img, smooth_fwhm):
        """
        Parameters
        ----------
        nii_img: nipy.Image

        smooth_fwhm: float

        Returns
        -------
        smoothed nipy.Image
        """
        # delayed import because could not install nipy on Python 3 on OSX
        from   nipy.algorithms.kernel_smooth import LinearFilter

        if smooth_fwhm <= 0:
            return nii_img

        filter = LinearFilter(nii_img.coordmap, nii_img.shape)
        return filter.smooth(nii_img)


Instruction:
How would you implement a function in Python 3 that
loads items and labels from a dictionary of str files.

Response:
def from_dict(self, subj_files):
        """
        Parameters
        ----------
        subj_files: dict of str
            file_path -> int/str
        """
        for group_label in subj_files:
            try:
                group_files = subj_files[group_label]
                self.items.extend([self._load_image(get_abspath(imgf)) for imgf in group_files])

                self.labels.extend([group_label]*len(group_files))

            except Exception as exc:
                raise Exception('Error while reading files from '
                                'group {0}.'.format(group_label)) from exc


Instruction:
Make a summary of the following Python 3 code
def from_list(self, subj_files):
        for sf in subj_files:
            try:
                nii_img = self._load_image(get_abspath(sf))
                self.items.append(nii_img)
            except Exception as exc:
                raise Exception('Error while reading file {0}.'.format(sf)) from exc

Response: Reads a list of file paths and adds them to the internal list of items.


Instruction:
Here you have a function in Python 3, explain what it does
def set_labels(self, subj_labels):
        if len(subj_labels) != self.n_subjs:
            raise ValueError('The number of given labels is not the same as the number of subjects.')

        self.labels = subj_labels

Response: Sets the labels of the items in the log file.


Instruction:
Write a Python 3 script to
create a Numpy array with the data and return the relevant information.

Response:
def to_matrix(self, smooth_fwhm=0, outdtype=None):
        """Create a Numpy array with the data and return the relevant information (mask indices and volume shape).

        Parameters
        ----------
        smooth_fwhm: int
            Integer indicating the size of the FWHM Gaussian smoothing kernel
            to smooth the subject volumes before creating the data matrix

        outdtype: dtype
            Type of the elements of the array, if None will obtain the dtype from
            the first nifti file.

        Returns
        -------
        outmat, mask_indices, vol_shape

        outmat: Numpy array with shape N x prod(vol.shape)
                containing the N files as flat vectors.

        mask_indices: matrix with indices of the voxels in the mask

        vol_shape: Tuple with shape of the volumes, for reshaping.
        """

        vol = self.items[0].get_data()
        if not outdtype:
            outdtype = vol.dtype

        n_voxels     = None
        mask_indices = None
        mask_shape   = self.items[0].shape

        if self.has_mask:
            mask_arr     = get_img_data(self.mask_file)
            mask_indices = np.where(mask_arr > 0)
            mask_shape   = mask_arr.shape
            n_voxels     = np.count_nonzero(mask_arr)

        if n_voxels is None:
            log.debug('Non-zero voxels have not been found in mask {}'.format(self.mask_file))
            n_voxels = np.prod(vol.shape)

        outmat = np.zeros((self.n_subjs, n_voxels), dtype=outdtype)
        try:
            for i, nipy_img in enumerate(self.items):
                vol = self._smooth_img(nipy_img, smooth_fwhm).get_data()
                if self.has_mask is not None:
                    outmat[i, :] = vol[mask_indices]
                else:
                    outmat[i, :] = vol.flatten()
        except Exception as exc:
            raise Exception('Error when flattening file {0}'.format(nipy_img.file_path)) from exc
        else:
            return outmat, mask_indices, mask_shape


Instruction:
Create a Python 3 function for
writing msg to stderr and exits with return code

Response:
def die(msg, code=-1):
    """Writes msg to stderr and exits with return code"""
    sys.stderr.write(msg + "\n")
    sys.exit(code)


Instruction:
Explain what the following Python 3 code does
def check_call(cmd_args):
    p = subprocess.Popen(cmd_args, stdout=subprocess.PIPE)
    (output, err) = p.communicate()
    return output

Response: Calls the command in a list of command line arguments and returns the output of the command.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def call_command(cmd_name, args_strings):
    if not op.isabs(cmd_name):
        cmd_fullpath = which(cmd_name)
    else:
        cmd_fullpath = cmd_name

    try:
        cmd_line = [cmd_fullpath] + args_strings

        log.info('Calling: {}.'.format(cmd_line))
        retval = subprocess.check_call(cmd_line)
    except CalledProcessError as ce:
        log.exception("Error calling command {} with arguments: "
                      "{} \n With return code: {}".format(cmd_name, args_strings,
                                                          ce.returncode))
        raise
    else:
        return retval

Response: Call a command with arguments and returns its return value.


Instruction:
Implement a function in Python 3 to
submit cmd to HTCondor and calls subprocess. call.

Response:
def condor_call(cmd, shell=True):
    """
    Tries to submit cmd to HTCondor, if it does not succeed, it will
    be called with subprocess.call.

    Parameters
    ----------
    cmd: string
        Command to be submitted

    Returns
    -------
    """
    log.info(cmd)
    ret = condor_submit(cmd)
    if ret != 0:
        subprocess.call(cmd, shell=shell)


Instruction:
Make a summary of the following Python 3 code
def condor_submit(cmd):
    is_running = subprocess.call('condor_status', shell=True) == 0
    if not is_running:
        raise CalledProcessError('HTCondor is not running.')

    sub_cmd = 'condor_qsub -shell n -b y -r y -N ' \
              + cmd.split()[0] + ' -m n'

    log.info('Calling: ' + sub_cmd)

    return subprocess.call(sub_cmd + ' ' + cmd, shell=True)

Response: Submits cmd to HTCondor queue


Instruction:
Create a Python 3 function for
cleaning previously built package artifacts.

Response:
def clean(ctx):
    """Clean previously built package artifacts.
    """
    ctx.run(f'python setup.py clean')
    dist = ROOT.joinpath('dist')
    print(f'removing {dist}')
    shutil.rmtree(str(dist))


Instruction:
How would you explain what the following Python 3 function does
def upload(ctx, repo):
    artifacts = ' '.join(
        shlex.quote(str(n))
        for n in ROOT.joinpath('dist').glob('pipfile[-_]cli-*')
    )
    ctx.run(f'twine upload --repository="{repo}" {artifacts}')

Response: Upload the package to an index server.


Instruction:
Here you have a function in Python 3, explain what it does
def load_command_table(self, args): #pylint: disable=too-many-statements

        # Need an empty client for the select and upload operations
        with CommandSuperGroup(__name__, self,
                               'rcctl.custom_cluster#{}') as super_group:
            with super_group.group('cluster') as group:
                group.command('select', 'select')

        with CommandSuperGroup(__name__, self, 'rcctl.custom_reliablecollections#{}',
                               client_factory=client_create) as super_group:
            with super_group.group('dictionary') as group:
                group.command('query', 'query_reliabledictionary')
                group.command('execute', 'execute_reliabledictionary')
                group.command('schema', 'get_reliabledictionary_schema')
                group.command('list', 'get_reliabledictionary_list')
                group.command('type-schema', 'get_reliabledictionary_type_schema')

        with ArgumentsContext(self, 'dictionary') as ac:
            ac.argument('application_name', options_list=['--application-name', '-a'])
            ac.argument('service_name', options_list=['--service-name', '-s'])
            ac.argument('dictionary_name', options_list=['--dictionary-name', '-d'])
            ac.argument('output_file', options_list=['--output-file', '-out'])
            ac.argument('input_file', options_list=['--input-file', '-in'])
            ac.argument('query_string', options_list=['--query-string', '-q'])
            ac.argument('type_name', options_list=['--type-name', '-t'])

        return OrderedDict(self.command_table)

Response: Load all Service Fabric commands and return a dictionary of all commands that can be executed.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def open_volume_file(filepath):
    # check if the file exists
    if not op.exists(filepath):
        raise IOError('Could not find file {}.'.format(filepath))

    # define helper functions
    def open_nifti_file(filepath):
        return NiftiImage(filepath)

    def open_mhd_file(filepath):
        return MedicalImage(filepath)
        vol_data, hdr_data = load_raw_data_with_mhd(filepath)
        # TODO: convert vol_data and hdr_data into MedicalImage
        return vol_data, hdr_data

    def open_mha_file(filepath):
        raise NotImplementedError('This function has not been implemented yet.')

    # generic loader function
    def _load_file(filepath, loader):
        return loader(filepath)

    # file_extension -> file loader function
    filext_loader = {
                    'nii': open_nifti_file,
                    'mhd': open_mhd_file,
                    'mha': open_mha_file,
                    }

    # get extension of the `filepath`
    ext = get_extension(filepath)

    # find the loader from `ext`
    loader = None
    for e in filext_loader:
        if ext in e:
            loader = filext_loader[e]

    if loader is None:
        raise ValueError('Could not find a loader for file {}.'.format(filepath))

    return _load_file(filepath, loader)

Response: Open a nifti image and medical image file and return the data and header data.