Instruction:
How would you code a function in Python 3 to
copy entities between workspaces

Response:
def copy_entities(from_namespace, from_workspace, to_namespace,
                  to_workspace, etype, enames, link_existing_entities=False):
    """Copy entities between workspaces

    Args:
        from_namespace (str): project (namespace) to which source workspace belongs
        from_workspace (str): Source workspace name
        to_namespace (str): project (namespace) to which target workspace belongs
        to_workspace (str): Target workspace name
        etype (str): Entity type
        enames (list(str)): List of entity names to copy
        link_existing_entities (boolean): Link all soft conflicts to the entities that already exist.

    Swagger:
        https://api.firecloud.org/#!/Entities/copyEntities
    """

    uri = "workspaces/{0}/{1}/entities/copy".format(to_namespace, to_workspace)
    body = {
        "sourceWorkspace": {
            "namespace": from_namespace,
            "name": from_workspace
        },
        "entityType": etype,
        "entityNames": enames
    }

    return __post(uri, json=body, params={'linkExistingEntities':
                                          str(link_existing_entities).lower()})


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_entities(namespace, workspace, etype):
    uri = "workspaces/{0}/{1}/entities/{2}".format(namespace, workspace, etype)
    return __get(uri)

Response: List entities of given type in a workspace.


Instruction:
Create a Python 3 function to
get entities of given type in a workspace as TSV.

Response:
def get_entities_tsv(namespace, workspace, etype):
    """List entities of given type in a workspace as a TSV.

    Identical to get_entities(), but the response is a TSV.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        etype (str): Entity type

    Swagger:
        https://api.firecloud.org/#!/Entities/browserDownloadEntitiesTSV
    """
    uri = "workspaces/{0}/{1}/entities/{2}/tsv".format(namespace,
                                                workspace, etype)
    return __get(uri)


Instruction:
Can you generate the documentation for the following Python 3 function
def get_entity(namespace, workspace, etype, ename):
    uri = "workspaces/{0}/{1}/entities/{2}/{3}".format(namespace,
                                            workspace, etype, ename)
    return __get(uri)

Response:
Request entity information.

    Gets entity metadata and attributes.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        etype (str): Entity type
        ename (str): The entity's unique id

    Swagger:
        https://api.firecloud.org/#!/Entities/getEntity


Instruction:
Can you write a function in Python 3 where it
deletes entities in a workspace

Response:
def delete_entities(namespace, workspace, json_body):
    """Delete entities in a workspace.

    Note: This action is not reversible. Be careful!

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        json_body:
        [
          {
            "entityType": "string",
            "entityName": "string"
          }
        ]

    Swagger:
        https://api.firecloud.org/#!/Entities/deleteEntities
    """

    uri = "workspaces/{0}/{1}/entities/delete".format(namespace, workspace)
    return __post(uri, json=json_body)


Instruction:
Here you have a function in Python 3, explain what it does
def delete_entity_type(namespace, workspace, etype, ename):

    uri = "workspaces/{0}/{1}/entities/delete".format(namespace, workspace)
    if isinstance(ename, string_types):
        body = [{"entityType":etype, "entityName":ename}]
    elif isinstance(ename, Iterable):
        body = [{"entityType":etype, "entityName":i} for i in ename]

    return __post(uri, json=body)

Response: Delete entities in a workspace.


Instruction:
Write a Python 3 script for
paginating version of get_entities_with_type

Response:
def get_entities_query(namespace, workspace, etype, page=1,
                       page_size=100, sort_direction="asc",
                       filter_terms=None):
    """Paginated version of get_entities_with_type.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name

    Swagger:
        https://api.firecloud.org/#!/Entities/entityQuery

    """

    # Initial parameters for pagination
    params = {
        "page" : page,
        "pageSize" : page_size,
        "sortDirection" : sort_direction
    }
    if filter_terms:
        params['filterTerms'] = filter_terms

    uri = "workspaces/{0}/{1}/entityQuery/{2}".format(namespace,workspace,etype)
    return __get(uri, params=params)


Instruction:
Given the following Python 3 function, write the documentation
def update_entity(namespace, workspace, etype, ename, updates):
    headers = _fiss_agent_header({"Content-type":  "application/json"})
    uri = "{0}workspaces/{1}/{2}/entities/{3}/{4}".format(fcconfig.root_url,
                                            namespace, workspace, etype, ename)

    # FIXME: create __patch method, akin to __get, __delete etc
    return __SESSION.patch(uri, headers=headers, json=updates)

Response: Update the attributes of an entity in a workspace.


Instruction:
Implement a function in Python 3 to
list method configurations in workspace.

Response:
def list_workspace_configs(namespace, workspace, allRepos=False):
    """List method configurations in workspace.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/listWorkspaceMethodConfigs
        DUPLICATE: https://api.firecloud.org/#!/Workspaces/listWorkspaceMethodConfigs
    """
    uri = "workspaces/{0}/{1}/methodconfigs".format(namespace, workspace)
    return __get(uri, params={'allRepos': allRepos})


Instruction:
Write a Python 3 script to
create method configuration in workspace.

Response:
def create_workspace_config(namespace, workspace, body):
    """Create method configuration in workspace.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        body (json) : a filled-in JSON object for the new method config
                      (e.g. see return value of get_workspace_config)

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/postWorkspaceMethodConfig
        DUPLICATE: https://api.firecloud.org/#!/Workspaces/postWorkspaceMethodConfig
    """

    #json_body = {
    #    "namespace"      : mnamespace,
    #    "name"           : method,
    #    "rootEntityType" : root_etype,
    #    "inputs" : {},
    #    "outputs" : {},
    #    "prerequisites" : {}
    #}
    uri = "workspaces/{0}/{1}/methodconfigs".format(namespace, workspace)
    return __post(uri, json=body)


Instruction:
How would you explain what the following Python 3 function does
def delete_workspace_config(namespace, workspace, cnamespace, config):
    uri = "workspaces/{0}/{1}/method_configs/{2}/{3}".format(namespace,
                                            workspace, cnamespace, config)
    return __delete(uri)

Response: Delete method configuration in workspace.


Instruction:
Create a Python 3 function for
getting method configuration in workspace.

Response:
def get_workspace_config(namespace, workspace, cnamespace, config):
    """Get method configuration in workspace.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        cnamespace (str): Config namespace
        config (str): Config name

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/getWorkspaceMethodConfig
    """

    uri = "workspaces/{0}/{1}/method_configs/{2}/{3}".format(namespace,
                                            workspace, cnamespace, config)
    return __get(uri)


Instruction:
Can you write a function in Python 3 where it
adds or overwrite method configuration in workspace.

Response:
def overwrite_workspace_config(namespace, workspace, cnamespace, configname, body):
    """Add or overwrite method configuration in workspace.

    Args:
        namespace  (str): project to which workspace belongs
        workspace  (str): Workspace name
        cnamespace (str): Configuration namespace
        configname (str): Configuration name
        body      (json): new body (definition) of the method config

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/overwriteWorkspaceMethodConfig
    """
    headers = _fiss_agent_header({"Content-type": "application/json"})
    uri = "workspaces/{0}/{1}/method_configs/{2}/{3}".format(namespace,
                                        workspace, cnamespace, configname)
    return __put(uri, headers=headers, json=body)


Instruction:
How would you code a function in Python 3 to
update method configuration in workspace.

Response:
def update_workspace_config(namespace, workspace, cnamespace, configname, body):
    """Update method configuration in workspace.

    Args:
        namespace  (str): project to which workspace belongs
        workspace  (str): Workspace name
        cnamespace (str): Configuration namespace
        configname (str): Configuration name
        body      (json): new body (definition) of the method config

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/updateWorkspaceMethodConfig
    """
    uri = "workspaces/{0}/{1}/method_configs/{2}/{3}".format(namespace,
                                        workspace, cnamespace, configname)
    return __post(uri, json=body)


Instruction:
Write a Python 3 function for
getting syntax validation for a configuration.

Response:
def validate_config(namespace, workspace, cnamespace, config):
    """Get syntax validation for a configuration.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        cnamespace (str): Configuration namespace
        config (str): Configuration name

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/validate_method_configuration
    """
    uri = "workspaces/{0}/{1}/method_configs/{2}/{3}/validate".format(namespace,
                                                    workspace, cnamespace, config)
    return __get(uri)


Instruction:
Write a Python 3 function that can
rename a method configuration in a workspace.

Response:
def rename_workspace_config(namespace, workspace, cnamespace, config,
                                            new_namespace, new_name):
    """Rename a method configuration in a workspace.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        mnamespace (str): Config namespace
        config (str): Config name
        new_namespace (str): Updated config namespace
        new_name (str): Updated method name

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/renameWorkspaceMethodConfig
    """

    body = {
        "namespace" : new_namespace,
        "name"      : new_name,
        # I have no idea why this is required by FC, but it is...
        "workspaceName" : {
            "namespace" : namespace,
            "name"      : workspace
        }
    }
    uri = "workspaces/{0}/{1}/method_configs/{2}/{3}/rename".format(namespace,
                                                workspace, cnamespace, config)
    return __post(uri, json=body)


Instruction:
Given the following Python 3 function, write the documentation
def copy_config_from_repo(namespace, workspace, from_cnamespace,
                          from_config, from_snapshot_id, to_cnamespace,
                          to_config):
    """Copy a method config from the methods repository to a workspace.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        from_cnamespace (str): Source configuration namespace
        from_config (str): Source configuration name
        from_snapshot_id (int): Source configuration snapshot_id
        to_cnamespace (str): Target configuration namespace
        to_config (str): Target configuration name

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/copyFromMethodRepo
        DUPLICATE: https://api.firecloud.org/#!/Method_Repository/copyFromMethodRepo
    """

    body = {
        "configurationNamespace"  : from_cnamespace,
        "configurationName"       : from_config,
        "configurationSnapshotId" : from_snapshot_id,
        "destinationNamespace"    : to_cnamespace,
        "destinationName"         : to_config
    }
    uri = "workspaces/{0}/{1}/method_configs/copyFromMethodRepo".format(
                                                        namespace, workspace)
    return __post(uri, json=body)

Response: Copy a method config from the methods repository to a workspace.


Instruction:
Write a Python 3 function for
copying a method config from a workspace to the methods repository.

Response:
def copy_config_to_repo(namespace, workspace, from_cnamespace,
                        from_config, to_cnamespace, to_config):
    """Copy a method config from a workspace to the methods repository.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        from_cnamespace (str): Source configuration namespace
        from_config (str): Source configuration name
        to_cnamespace (str): Target configuration namespace
        to_config (str): Target configuration name

    Swagger:
        https://api.firecloud.org/#!/Method_Configurations/copyToMethodRepo
        DUPLICATE: https://api.firecloud.org/#!/Method_Repository/copyToMethodRepo
    """

    body = {
        "configurationNamespace" : to_cnamespace,
        "configurationName"      : to_config,
        "sourceNamespace"        : from_cnamespace,
        "sourceName"             : from_config
    }
    uri = "workspaces/{0}/{1}/method_configs/copyToMethodRepo".format(
                                                    namespace, workspace)
    return __post(uri, json=body)


Instruction:
Make a summary of the following Python 3 code
def list_repository_methods(namespace=None, name=None, snapshotId=None):
    params = {k:v for (k,v) in locals().items() if v is not None}
    return __get("methods", params=params)

Response: List all methods in the methods repository.


Instruction:
Can you generate the documentation for the following Python 3 function
def get_config_template(namespace, method, version):

    body = {
        "methodNamespace" : namespace,
        "methodName"      : method,
        "methodVersion"   : int(version)
    }
    return __post("template", json=body)

Response: Get the configuration template for a method.


Instruction:
Can you implement a function in Python 3 that
gets a description of the inputs and outputs for a method.

Response:
def get_inputs_outputs(namespace, method, snapshot_id):
    """Get a description of the inputs and outputs for a method.

    The method should exist in the methods repository.

    Args:
        namespace (str): Methods namespace
        method (str): method name
        snapshot_id (int): snapshot_id of the method

    Swagger:
        https://api.firecloud.org/#!/Method_Repository/getMethodIO
    """

    body = {
        "methodNamespace" : namespace,
        "methodName"      : method,
        "methodVersion"   : snapshot_id
    }
    return __post("inputsOutputs", json=body)


Instruction:
Create a Python 3 function for
getting a method configuration from the methods repository.

Response:
def get_repository_config(namespace, config, snapshot_id):
    """Get a method configuration from the methods repository.

    Args:
        namespace (str): Methods namespace
        config (str): config name
        snapshot_id (int): snapshot_id of the method

    Swagger:
        https://api.firecloud.org/#!/Method_Repository/getMethodRepositoryConfiguration
    """
    uri = "configurations/{0}/{1}/{2}".format(namespace, config, snapshot_id)
    return __get(uri)


Instruction:
Given the following Python 3 function, write the documentation
def get_repository_method(namespace, method, snapshot_id, wdl_only=False):
    uri = "methods/{0}/{1}/{2}?onlyPayload={3}".format(namespace, method,
                                                       snapshot_id,
                                                       str(wdl_only).lower())
    return __get(uri)

Response: Get a method definition from the method repository.


Instruction:
Write a Python 3 script to
create or update a workflow definition.

Response:
def update_repository_method(namespace, method, synopsis, wdl, doc=None,
                             comment=""):
    """Create/Update workflow definition.

    FireCloud will create a new snapshot_id for the given workflow.

    Args:
        namespace (str): Methods namespace
        method (str): method name
        synopsis (str): short (<80 char) description of method
        wdl (file): Workflow Description Language file
        doc (file): (Optional) Additional documentation
        comment (str): (Optional) Comment specific to this snapshot

    Swagger:
        https://api.firecloud.org/#!/Method_Repository/post_api_methods

    """
    with open(wdl, 'r') as wf:
        wdl_payload = wf.read()
    if doc is not None:
        with open (doc, 'r') as df:
            doc = df.read()

    body = {
        "namespace": namespace,
        "name": method,
        "entityType": "Workflow",
        "payload": wdl_payload,
        "documentation": doc,
        "synopsis": synopsis,
        "snapshotComment": comment
    }

    return __post("methods",
                  json={key: value for key, value in body.items() if value})


Instruction:
Can you tell what is the following Python 3 function doing
def delete_repository_method(namespace, name, snapshot_id):
    uri = "methods/{0}/{1}/{2}".format(namespace, name, snapshot_id)
    return __delete(uri)

Response: Redacts a method and all of its associated configurations in the methods repository.


Instruction:
Given the following Python 3 function, write the documentation
def get_repository_method_acl(namespace, method, snapshot_id):
    uri = "methods/{0}/{1}/{2}/permissions".format(namespace,method,snapshot_id)
    return __get(uri)

Response: Get permissions for a method in the methods repository.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def update_repository_method_acl(namespace, method, snapshot_id, acl_updates):

    uri = "methods/{0}/{1}/{2}/permissions".format(namespace,method,snapshot_id)
    return __post(uri, json=acl_updates)

Response: Set the permissions of a method in the Method Repository.


Instruction:
Explain what the following Python 3 code does
def get_repository_config_acl(namespace, config, snapshot_id):
    uri = "configurations/{0}/{1}/{2}/permissions".format(namespace,
                                                    config, snapshot_id)
    return __get(uri)

Response: Get permissions for a specific configuration in the methods repository.


Instruction:
Given the following Python 3 function, write the documentation
def update_repository_config_acl(namespace, config, snapshot_id, acl_updates):

    uri = "configurations/{0}/{1}/{2}/permissions".format(namespace,
                                                config, snapshot_id)
    return __post(uri, json=acl_updates)

Response: Set the permissions of a configuration in the methods repository.


Instruction:
Can you tell what is the following Python 3 function doing
def create_submission(wnamespace, workspace, cnamespace, config,
                      entity, etype, expression=None, use_callcache=True):
    """Submit job in FireCloud workspace.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        cnamespace (str): Method configuration namespace
        config (str): Method configuration name
        entity (str): Entity to submit job on. Should be the same type as
            the root entity type of the method config, unless an
            expression is used
        etype (str): Entity type of root_entity
        expression (str): Instead of using entity as the root entity,
            evaluate the root entity from this expression.
        use_callcache (bool): use call cache if applicable (default: true)

    Swagger:
        https://api.firecloud.org/#!/Submissions/createSubmission
    """

    uri = "workspaces/{0}/{1}/submissions".format(wnamespace, workspace)
    body = {
        "methodConfigurationNamespace" : cnamespace,
        "methodConfigurationName" : config,
         "entityType" : etype,
         "entityName" : entity,
         "useCallCache" : use_callcache
    }

    if expression:
        body['expression'] = expression

    return __post(uri, json=body)

Response: Submit a new job in FireCloud workspace.


Instruction:
Given the following Python 3 function, write the documentation
def abort_submission(namespace, workspace, submission_id):
    uri = "workspaces/{0}/{1}/submissions/{2}".format(namespace,
                                        workspace, submission_id)
    return __delete(uri)

Response: Abort running job in a workspace.


Instruction:
Write a Python 3 script to
request submission information. Args: namespace (str): project to which workspace belongs workspace (str): Workspace name submission_id (str): Submission's unique identifier Swagger: https://api.firecloud.org/#!/Submissions/monitorSubmission

Response:
def get_submission(namespace, workspace, submission_id):
    """Request submission information.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        submission_id (str): Submission's unique identifier

    Swagger:
        https://api.firecloud.org/#!/Submissions/monitorSubmission
    """
    uri = "workspaces/{0}/{1}/submissions/{2}".format(namespace,
                                            workspace, submission_id)
    return __get(uri)


Instruction:
Create a Python 3 function for
requesting the metadata for a workflow in a submission.

Response:
def get_workflow_metadata(namespace, workspace, submission_id, workflow_id):
    """Request the metadata for a workflow in a submission.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        submission_id (str): Submission's unique identifier
        workflow_id (str): Workflow's unique identifier.

    Swagger:
        https://api.firecloud.org/#!/Submissions/workflowMetadata
    """
    uri = "workspaces/{0}/{1}/submissions/{2}/workflows/{3}".format(namespace,
                                            workspace, submission_id, workflow_id)
    return __get(uri)


Instruction:
Explain what the following Python 3 code does
def get_workflow_outputs(namespace, workspace, submission_id, workflow_id):
    uri = "workspaces/{0}/{1}/".format(namespace, workspace)
    uri += "submissions/{0}/workflows/{1}/outputs".format(submission_id,
                                                            workflow_id)
    return __get(uri)

Response: Request the outputs for a workflow in a submission.


Instruction:
Given the following Python 3 function, write the documentation
def create_workspace(namespace, name, authorizationDomain="", attributes=None):

    if not attributes:
        attributes = dict()

    body = {
        "namespace": namespace,
        "name": name,
        "attributes": attributes
    }
    if authorizationDomain:
        authDomain = [{"membersGroupName": authorizationDomain}]
    else:
        authDomain = []

    body["authorizationDomain"] = authDomain

    return __post("workspaces", json=body)

Response: Create a new FireCloud Workspace.


Instruction:
Explain what the following Python 3 code does
def update_workspace_acl(namespace, workspace, acl_updates, invite_users_not_found=False):
    uri = "{0}workspaces/{1}/{2}/acl?inviteUsersNotFound={3}".format(fcconfig.root_url,
                                                namespace, workspace, str(invite_users_not_found).lower())
    headers = _fiss_agent_header({"Content-type":  "application/json"})
    # FIXME: create __patch method, akin to __get, __delete etc
    return __SESSION.patch(uri, headers=headers, data=json.dumps(acl_updates))

Response: Update the access control list for a workspace.


Instruction:
Write a Python 3 function for
cloning a FireCloud workspace.

Response:
def clone_workspace(from_namespace, from_workspace, to_namespace, to_workspace,
                    authorizationDomain=""):
    """Clone a FireCloud workspace.

    A clone is a shallow copy of a FireCloud workspace, enabling
    easy sharing of data, such as TCGA data, without duplication.

    Args:
        from_namespace (str):  project (namespace) to which source workspace belongs
        from_workspace (str): Source workspace's name
        to_namespace (str):  project to which target workspace belongs
        to_workspace (str): Target workspace's name
        authorizationDomain: (str) required authorization domains

    Swagger:
        https://api.firecloud.org/#!/Workspaces/cloneWorkspace
    """

    if authorizationDomain:
        if isinstance(authorizationDomain, string_types):
            authDomain = [{"membersGroupName": authorizationDomain}]
        else:
            authDomain = [{"membersGroupName": authDomain} for authDomain in authorizationDomain]
    else:
        authDomain = []

    body = {
        "namespace": to_namespace,
        "name": to_workspace,
        "attributes": dict(),
        "authorizationDomain": authDomain,
    }

    uri = "workspaces/{0}/{1}/clone".format(from_namespace, from_workspace)
    return __post(uri, json=body)


Instruction:
Implement a function in Python 3 to
update or remove workspace attributes.

Response:
def update_workspace_attributes(namespace, workspace, attrs):
    """Update or remove workspace attributes.

    Args:
        namespace (str): project to which workspace belongs
        workspace (str): Workspace name
        attrs (list(dict)): List of update operations for workspace attributes.
            Use the helper dictionary construction functions to create these:

            _attr_set()      : Set/Update attribute
            _attr_rem()     : Remove attribute
            _attr_ladd()    : Add list member to attribute
            _attr_lrem()    : Remove list member from attribute

    Swagger:
        https://api.firecloud.org/#!/Workspaces/updateAttributes
    """
    headers = _fiss_agent_header({"Content-type":  "application/json"})
    uri = "{0}workspaces/{1}/{2}/updateAttributes".format(fcconfig.root_url,
                                                        namespace, workspace)
    body = json.dumps(attrs)

    # FIXME: create __patch method, akin to __get, __delete etc
    return __SESSION.patch(uri, headers=headers, data=body)


Instruction:
Explain what the following Python 3 code does
def add_user_to_group(group, role, email):
    uri = "groups/{0}/{1}/{2}".format(group, role, email)
    return __put(uri)

Response: Adds a user to a group


Instruction:
Create a Python 3 function to
remove a user from a group

Response:
def remove_user_from_group(group, role, email):
    """Remove a user from a group the caller owns

    Args:
        group (str): Group name
        role (str) : Role of user for group; either 'member' or 'admin'
        email (str): Email of user or group to remove

    Swagger:
        https://api.firecloud.org/#!/Groups/removeUserFromGroup
    """
    uri = "groups/{0}/{1}/{2}".format(group, role, email)
    return __delete(uri)


Instruction:
Can you tell what is the following Python 3 function doing
def register_commands(self, subparsers):
        #print_("DEV PLUGIN: Loaded commands")
        prsr = subparsers.add_parser(
            'upload',  description='Copy the file or directory into the given')
        prsr.add_argument('workspace', help='Workspace name')
        prsr.add_argument('source', help='File or directory to upload')
        prsr.add_argument('-s', '--show', action='store_true',
                            help="Show the gsutil command, but don't run it")

        dest_help = 'Destination relative to the bucket root. '
        dest_help += 'If omitted the file will be placed in the root directory'
        prsr.add_argument('-d', '--destination', help=dest_help)

        prsr.set_defaults(func=upload)

Response: Register commands to a list of subparsers


Instruction:
How would you explain what the following Python 3 function does
def new(namespace, name, wdl, synopsis,
            documentation=None, api_url=fapi.PROD_API_ROOT):
        """Create new FireCloud method.

        If the namespace + name already exists, a new snapshot is created.

        Args:
            namespace (str): Method namespace for this method
            name (str): Method name
            wdl (file): WDL description
            synopsis (str): Short description of task
            documentation (file): Extra documentation for method
        """
        r = fapi.update_workflow(namespace, name, synopsis,
                                 wdl, documentation, api_url)
        fapi._check_response_code(r, 201)
        d = r.json()
        return Method(namespace, name, d["snapshotId"])

Response: Create a new FireCloud method.


Instruction:
Create a Python 3 function for
returning a method template for this method.

Response:
def template(self):
        """Return a method template for this method."""
        r = fapi.get_config_template(self.namespace, self.name,
                                        self.snapshot_id, self.api_url)
        fapi._check_response_code(r, 200)
        return r.json()


Instruction:
Can you generate the documentation for the following Python 3 function
def inputs_outputs(self):
        r = fapi.get_inputs_outputs(self.namespace, self.name,
                                    self.snapshot_id, self.api_url)
        fapi._check_response_code(r, 200)
        return r.json()

Response: Get information on method inputs & outputs.


Instruction:
How would you implement a function in Python 3 that
gets the access control list for this method.

Response:
def acl(self):
        """Get the access control list for this method."""
        r = fapi.get_repository_method_acl(
            self.namespace, self.name, self.snapshot_id, self.api_url)
        fapi._check_response_code(r, 200)
        return r.json()


Instruction:
Explain what the following Python 3 code does
def set_acl(self, role, users):
        acl_updates = [{"user": user, "role": role} for user in users]
        r = fapi.update_repository_method_acl(
            self.namespace, self.name, self.snapshot_id,
            acl_updates, self.api_url
        )
        fapi._check_response_code(r, 200)

Response: Set permissions for this method.


Instruction:
Explain what the following Python 3 code does
def __ensure_gcloud():
    if which('gcloud') is None:
        gcloud_path = os.path.join(os.path.expanduser('~'),
                                   'google-cloud-sdk', 'bin')
        env_path = os.getenv('PATH')
        if os.path.isdir(gcloud_path):
            if env_path is not None:
                os.environ['PATH'] = gcloud_path + os.pathsep + env_path
            else:
                os.environ['PATH'] = gcloud_path

Response: Ensure that the gcloud - sdk directory for the executables exists.


Instruction:
Write a Python 3 function that can
parse a set of named config files and store the result in a config dictionary.

Response:
def config_parse(files=None, config=None, config_profile=".fissconfig", **kwargs):
    '''
    Read initial configuration state, from named config files; store
    this state within a config dictionary (which may be nested) whose keys may
    also be referenced as attributes (safely, defaulting to None if unset).  A
    config object may be passed in, as a way of accumulating or overwriting
    configuration state; if one is NOT passed, the default config obj is used
    '''
    local_config = config
    config = __fcconfig

    cfgparser = configparser.SafeConfigParser()

    filenames = list()

    # Give personal/user followed by current working directory configuration the first say
    filenames.append(os.path.join(os.path.expanduser('~'), config_profile))

    filenames.append(os.path.join(os.getcwd(), config_profile))

    if files:
        if isinstance(files, string_types):
            filenames.append(files)
        elif isinstance(files, Iterable):
            for f in files:
                if isinstance(f, IOBase):
                    f = f.name
                filenames.append(f)

    cfgparser.read(filenames)

    # [DEFAULT] defines common variables for interpolation/substitution in
    # other sections, and are stored at the root level of the config object
    for keyval in cfgparser.items('DEFAULT'):
        #print("config_parse: adding config variable %s=%s" % (keyval[0], str(keyval[1])))
        __fcconfig[keyval[0]] = keyval[1]

    for section in cfgparser.sections():
        config[section] = attrdict()
        for option in cfgparser.options(section):
            # DEFAULT vars ALSO behave as though they were defined in every
            # section, but we purposely skip them here so that each section
            # reflects only the options explicitly defined in that section
            if not config[option]:
                config[section][option] = cfgparser.get(section, option)

    config.verbosity = int(config.verbosity)
    if not config.root_url.endswith('/'):
        config.root_url += '/'
    if os.path.isfile(config.credentials):
        os.environ[environment_vars.CREDENTIALS] = config.credentials

    # if local_config override options with passed options
    if local_config is not None:
        for key, value in local_config.items():
            config[key] = value

    # if any explict config options are passed override.
    for key, value in kwargs.items():
        config[key] = value

    return config


Instruction:
Create a Python 3 function to
create a new FireCloud workspace.

Response:
def new(namespace, name, protected=False,
            attributes=dict(), api_url=fapi.PROD_API_ROOT):
        """Create a new FireCloud workspace.

        Returns:
            Workspace: A new FireCloud workspace

        Raises:
            FireCloudServerError: API call failed.
        """
        r = fapi.create_workspace(namespace, name, protected, attributes, api_url)
        fapi._check_response_code(r, 201)
        return Workspace(namespace, name, api_url)


Instruction:
Write a Python 3 script for
reloading workspace metadata from firecloud.

Response:
def refresh(self):
        """Reload workspace metadata from firecloud.

        Workspace metadata is cached in the data attribute of a Workspace,
        and may become stale, requiring a refresh().
        """
        r = fapi.get_workspace(self.namespace, self.name, self.api_url)
        fapi._check_response_code(r, 200)
        self.data = r.json()
        return self


Instruction:
Create a Python 3 function to
delete the workspace from FireCloud.

Response:
def delete(self):
        """Delete the workspace from FireCloud.

        Note:
            This action cannot be undone. Be careful!
        """
        r = fapi.delete_workspace(self.namespace, self.name)
        fapi._check_response_code(r, 202)


Instruction:
Create a Python 3 function for
locking this Workspace. This causes the workspace to behave in a read-only way, regardless of access permissions.

Response:
def lock(self):
        """Lock this Workspace.

        This causes the workspace to behave in a read-only way,
        regardless of access permissions.
        """
        r = fapi.lock_workspace(self.namespace, self.name, self.api_url)
        fapi._check_response_code(r, 204)
        self.data['workspace']['isLocked'] = True
        return self


Instruction:
How would you implement a function in Python 3 that
sets the value of a workspace attribute.

Response:
def update_attribute(self, attr, value):
        """Set the value of a workspace attribute."""
        update = [fapi._attr_up(attr, value)]
        r = fapi.update_workspace_attributes(self.namespace, self.name,
                                             update, self.api_url)
        fapi._check_response_code(r, 200)


Instruction:
Make a summary of the following Python 3 code
def remove_attribute(self, attr):
        update = [fapi._attr_rem(attr)]
        r = fapi.update_workspace_attributes(self.namespace, self.name,
                                                update, self.api_url)
        self.data["workspace"]["attributes"].pop(attr, None)
        fapi._check_response_code(r, 200)

Response: Removes an attribute from a workspace.


Instruction:
Explain what the following Python 3 code does
def import_tsv(self, tsv_file):
        r = fapi.upload_entities_tsv(self.namespace, self.name,
                                     self.tsv_file, self.api_url)
        fapi._check_response_code(r, 201)

Response: Upload entity data to workspace from tsv loadfile.


Instruction:
Write a Python 3 function for
returning entity in this workspace.

Response:
def get_entity(self, etype, entity_id):
        """Return entity in this workspace.

        Args:
            etype (str): Entity type
            entity_id (str): Entity name/unique id
        """
        r = fapi.get_entity(self.namespace, self.name, etype,
                               entity_id, self.api_url)
        fapi._check_response_code(r, 200)
        dresp = r.json()
        return Entity(etype, entity_id, dresp['attributes'])


Instruction:
Create a Python 3 function to
delete an entity in this workspace.

Response:
def delete_entity(self, etype, entity_id):
        """Delete an entity in this workspace.

        Args:
            etype (str): Entity type
            entity_id (str): Entity name/unique id
        """
        r = fapi.delete_entity(self.namespace, self.name, etype,
                                  entity_id, self.api_url)
        fapi._check_response_code(r, 202)


Instruction:
Can you tell what is the following Python 3 function doing
def import_entities(self, entities):
        edata = Entity.create_payload(entities)
        r = fapi.upload_entities(self.namespace, self.name,
                                 edata, self.api_url)
        fapi._check_response_code(r, 201)

Response: Upload entities to firecloud.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def create_set(self, set_id, etype, entities):
        if etype not in {"sample", "pair", "participant"}:
            raise ValueError("Unsupported entity type:" + str(etype))

        payload = "membership:" + etype + "_set_id\t" + etype + "_id\n"

        for e in entities:
            if e.etype != etype:
                msg =  "Entity type '" + e.etype + "' does not match "
                msg += "set type '" + etype + "'"
                raise ValueError(msg)
            payload += set_id + '\t' + e.entity_id + '\n'

        r = fapi.upload_entities(self.namespace, self.name,
                                    payload, self.api_url)
        fapi._check_response_code(r, 201)

Response: Create a set of entities and upload to FireCloud.


Instruction:
Make a summary of the following Python 3 code
def submissions(self):
        r = fapi.get_submissions(self.namespace, self.name, self.api_url)
        fapi._check_response_code(r, 200)
        return r.json()

Response: List job submissions in workspace.


Instruction:
Make a summary of the following Python 3 code
def entity_types(self):
        r = fapi.get_entity_types(self.namespace, self.name, self.api_url)
        fapi._check_response_code(r, 200)
        return r.json().keys()

Response: List entity types in workspace.


Instruction:
Explain what the following Python 3 code does
def entities(self):
        r = fapi.get_entities_with_type(self.namespace,
                                        self.name, self.api_url)
        fapi._check_response_code(r, 200)
        edicts = r.json()
        return [Entity(e['entityType'], e['name'], e['attributes'])
                for e in edicts]

Response: List all entities in workspace.


Instruction:
Can you generate the documentation for the following Python 3 function
def __get_entities(self, etype):
        r = fapi.get_entities(self.namespace, self.name,
                              etype, self.api_url)
        fapi._check_response_code(r, 200)
        return [Entity(e['entityType'], e['name'], e['attributes'])
                for e in r.json()]

Response: Helper to get entities for a given type.


Instruction:
Can you create a Python 3 function that
copies entities from another workspace.

Response:
def copy_entities(self, from_namespace, from_workspace, etype, enames):
        """Copy entities from another workspace.

        Args:
            from_namespace (str): Source workspace namespace
            from_workspace (str): Source workspace name
            etype (str): Entity type
            enames (list(str)): List of entity names to copy
        """
        r = fapi.copy_entities(from_namespace, from_workspace,
                               self.namespace, self.name, etype, enames,
                               self.api_url)
        fapi._check_response_code(r, 201)


Instruction:
Explain what the following Python 3 code does
def configs(self):
        raise NotImplementedError
        r = fapi.get_configs(self.namespace, self.name, self.api_url)
        fapi._check_response_code(r, 200)
        cdata = r.json()
        configs = []
        for c in cdata:
            cnamespace = c['namespace']
            cname = c['name']
            root_etype = c['rootEntityType']
            method_namespace = c['methodRepoMethod']['methodNamespace']
            method_name = c['methodRepoMethod']['methodName']
            method_version = c['methodRepoMethod']['methodVersion']

Response: Get method configurations in a workspace.


Instruction:
How would you explain what the following Python 3 function does
def acl(self):
        r = fapi.get_workspace_acl(self.namespace, self.name, self.api_url)
        fapi._check_response_code(r, 200)
        return r.json()

Response: Get the access control list for this workspace.


Instruction:
Write a Python 3 function for
cloning this workspace. Args: to_namespace (str): Target workspace namespace to_name (str): Target workspace name

Response:
def clone(self, to_namespace, to_name):
        """Clone this workspace.

        Args:
            to_namespace (str): Target workspace namespace
            to_name (str): Target workspace name
        """
        r = fapi.clone_workspace(self.namespace, self.name,
                                 to_namespace, to_name, self.api_url)
        fapi._check_response_code(r, 201)
        return Workspace(to_namespace, to_name, self.api_url)


Instruction:
Can you tell what is the following Python 3 function doing
def supervise(project, workspace, namespace, workflow,
              sample_sets, recovery_file):
    """ Supervise submission of jobs from a Firehose-style workflow of workflows"""
    # Get arguments

    logging.info("Initializing FireCloud Supervisor...")
    logging.info("Saving recovery checkpoints to " + recovery_file)

    # Parse workflow description
    # these three objects must be saved in order to recover the supervisor

    args = {
        'project'  : project,
        'workspace': workspace,
        'namespace': namespace,
        'workflow' : workflow,
        'sample_sets': sample_sets
    }

    monitor_data, dependencies = init_supervisor_data(workflow, sample_sets)
    recovery_data = {
        'args' : args,
        'monitor_data' : monitor_data,
        'dependencies' : dependencies
    }

    # Monitor loop. Keep going until all nodes have been evaluated
    supervise_until_complete(monitor_data, dependencies, args, recovery_file)

Response: This function is used to run a Firehose - style workflow of jobs from a Firehose - style workflow of jobs.


Instruction:
Make a summary of the following Python 3 code
def init_supervisor_data(dotfile, sample_sets):
    with open(dotfile) as wf:
        graph_data = wf.read()

    graph = pydot.graph_from_dot_data(graph_data)[0]
    nodes = [n.get_name().strip('"') for n in graph.get_nodes()]

    monitor_data = dict()
    dependencies = {n:[] for n in nodes}

    # Initialize empty list of dependencies
    for n in nodes:
        monitor_data[n] = dict()
        for sset in sample_sets:
            monitor_data[n][sset] = {
                'state'        : "Not Started",
                'evaluated'    : False,
                'succeeded'    : False
            }

    edges = graph.get_edges()

    # Iterate over the edges, and get the dependency information for each node
    for e in edges:
        source = e.get_source().strip('"')
        dest = e.get_destination().strip('"')

        dep = e.get_attributes()
        dep['upstream_task'] = source

        dependencies[dest].append(dep)

    return monitor_data, dependencies

Response: Parse a DOT file and initialize the supervisor data structure


Instruction:
Write a Python 3 function that can
validate that all tasks in the supervisor are valid and that all permissions requirements are satisfied.

Response:
def validate_monitor_tasks(dependencies, args):
    """ Validate that all entries in the supervisor are valid task configurations and
        that all permissions requirements are satisfied.
    """
    # Make a list of all task configurations needed to supervise
    sup_configs = sorted(dependencies.keys())

    try:
        logging.info("Validating supervisor data...")
        # Make an api call to list task configurations in the workspace
        r = fapi.list_workspace_configs(args['project'], args['workspace'])
        fapi._check_response_code(r, 200)
        space_configs = r.json()

        # Make a dict for easy lookup later
        space_configs = { c["name"]: c for c in space_configs}

        # Also make an api call to list methods you have view permissions for
        r = fapi.list_repository_methods()
        fapi._check_response_code(r, 200)
        repo_methods = r.json()

        ## Put in a form that is more easily searchable: namespace/name:snapshot
        repo_methods = {m['namespace'] + '/' + m['name'] + ':' + str(m['snapshotId'])
                        for m in repo_methods if m['entityType'] == 'Workflow'}

        valid = True

        for config in sup_configs:
            # ensure config exists in the workspace
            if config not in space_configs:
                logging.error("No task configuration for "
                              + config + " found in " + args['project'] + "/"
                              + args['workspace'])
                valid = False
            else:
                # Check access permissions for the referenced method
                m = space_configs[config]['methodRepoMethod']
                ref_method = m['methodNamespace'] + "/" + m['methodName'] + ":" + str(m['methodVersion'])
                if ref_method not in repo_methods:
                    logging.error(config+ " -- You don't have permisson to run the referenced method: "
                                  + ref_method)
                    valid = False

    except Exception as e:
        logging.error("Exception occurred while validating supervisor: " + str(e))
        raise
        return False

    return valid


Instruction:
Can you create a Python 3 function that
recovers data from recovery_file and resume monitoring

Response:
def recover_and_supervise(recovery_file):
    """ Retrieve monitor data from recovery_file and resume monitoring """
    try:
        logging.info("Attempting to recover Supervisor data from " + recovery_file)
        with open(recovery_file) as rf:
            recovery_data = json.load(rf)
            monitor_data = recovery_data['monitor_data']
            dependencies = recovery_data['dependencies']
            args = recovery_data['args']

    except:
        logging.error("Could not recover monitor data, exiting...")
        return 1

    logging.info("Data successfully loaded, resuming Supervisor")
    supervise_until_complete(monitor_data, dependencies, args, recovery_file)


Instruction:
How would you explain what the following Python 3 function does
def supervise_until_complete(monitor_data, dependencies, args, recovery_file):
    project = args['project']
    workspace = args['workspace']
    namespace = args['namespace']
    sample_sets = args['sample_sets']
    recovery_data = {'args': args}

    if not validate_monitor_tasks(dependencies, args):
        logging.error("Errors found, aborting...")
        return

    while True:
        # There are 4 possible states for each node:
        #   1. Not Started -- In this state, check all the dependencies for the
        #         node (possibly 0). If all of them have been evaluated, and the
        #         satisfiedMode is met, start the task, change to "Running". if
        #         satisfiedMode is not met, change to "Evaluated"
        #
        #   2. Running -- Submitted in FC. Check the submission endpoint, and
        #         if it has completed, change to "Completed", set evaluated=True,
        #         and whether the task succeeded
        #         Otherwise, do nothing
        #
        #   3. Completed -- Job ran in FC and either succeeded or failed. Do nothing
        #   4. Evaluated -- All dependencies evaluated, but this task did not run
        #         do nothing

        # Keep a tab of the number of jobs in each category
        running = 0
        waiting = 0
        completed = 0

        # Get the submissions
        r = fapi.list_submissions(project, workspace)
        sub_list = r.json()
        #TODO: filter this list by submission time first?
        sub_lookup = {s["submissionId"]: s for s in sub_list}

        # Keys of dependencies is the list of tasks to run
        for n in dependencies:
            for sset in sample_sets:
                task_data = monitor_data[n][sset]
                if task_data['state'] == "Not Started":
                    # See if all of the dependencies have been evaluated
                    upstream_evaluated = True
                    for dep in dependencies[n]:
                        # Look up the status of the task
                        upstream_task_data = monitor_data[dep['upstream_task']][sset]
                        if not upstream_task_data.get('evaluated'):
                            upstream_evaluated = False

                    # if all of the dependencies have been evaluated, we can evaluate
                    # this node
                    if upstream_evaluated:
                        # Now check the satisfied Mode of all the dependencies
                        should_run = True
                        for dep in dependencies[n]:
                            upstream_task_data = monitor_data[dep['upstream_task']][sset]
                            mode = dep['satisfiedMode']

                            # Task must have succeeded for OnComplete
                            if mode == '"OnComplete"' and not upstream_task_data['succeeded']:
                                should_run = False
                            # 'Always' and 'Optional' run once the deps have been
                            # evaluated

                        if should_run:
                            # Submit the workflow to FC
                            fc_config = n
                            logging.info("Starting workflow " + fc_config + " on " + sset)

                            # How to handle errors at this step?
                            for retry in range(3):
                                r = fapi.create_submission(
                                    project, workspace, namespace, fc_config,
                                    sset, etype="sample_set", expression=None
                                )
                                if r.status_code == 201:
                                    task_data['submissionId'] = r.json()['submissionId']
                                    task_data['state'] = "Running"
                                    running += 1
                                    break
                                else:
                                    # There was an error, under certain circumstances retry
                                    logging.debug("Create_submission for " + fc_config
                                                  + "failed on " + sset + " with the following response:"
                                                  + r.content + "\nRetrying...")

                            else:
                                # None of the attempts above succeeded, log an error, mark as failed
                                logging.error("Maximum retries exceeded")
                                task_data['state'] = 'Completed'
                                task_data['evaluated'] = True
                                task_data['succeeded'] = False

                        else:
                            # This task will never be able to run, mark evaluated
                            task_data['state'] = "Evaluated"
                            task_data['evaluated'] = True
                            completed += 1
                    else:
                        waiting += 1

                elif task_data['state'] == "Running":
                    submission = sub_lookup[task_data['submissionId']]
                    status = submission['status']

                    if status == "Done":
                        # Look at the individual workflows to see if there were
                        # failures
                        logging.info("Workflow " + n + " completed for " + sset)
                        success = 'Failed' not in submission['workflowStatuses']
                        task_data['evaluated'] = True
                        task_data['succeeded'] = success
                        task_data['state'] = "Completed"

                        completed += 1
                    else:
                        # Submission isn't done, don't do anything
                        running += 1

                else:
                    # Either Completed or evaluated
                    completed += 1

                # Save the state of the monitor for recovery purposes
                # Have to do this for every workflow + sample_set so we don't lose track of any
                recovery_data['monitor_data'] = monitor_data
                recovery_data['dependencies'] = dependencies

                with open(recovery_file, 'w') as rf:
                    json.dump(recovery_data, rf)

        logging.info("{0} Waiting, {1} Running, {2} Completed".format(waiting, running, completed))

        # If all tasks have been evaluated, we are done
        if all(monitor_data[n][sset]['evaluated']
                      for n in monitor_data for sset in monitor_data[n]):
            logging.info("DONE.")
            break
        time.sleep(30)

Response: This function is a loop forever until all tasks are evaluated or completed.


Instruction:
How would you explain what the following Python 3 function does
def space_list(args):
    ''' List accessible workspaces, in TSV form: <namespace><TAB>workspace'''

    r = fapi.list_workspaces()
    fapi._check_response_code(r, 200)

    spaces = []
    project = args.project
    if project:
        project = re.compile('^' + project)

    for space in r.json():
        ns = space['workspace']['namespace']
        if project and not project.match(ns):
            continue
        ws = space['workspace']['name']
        spaces.append(ns + '\t' + ws)

    # Sort for easier downstream viewing, ignoring case
    return sorted(spaces, key=lambda s: s.lower())

Response: List accessible workspaces in TSV form


Instruction:
Can you write a function in Python 3 where it
determines if the named space exists in the given project ( namespace

Response:
def space_exists(args):
    """ Determine if the named space exists in the given project (namespace)"""
    # The return value is the INVERSE of UNIX exit status semantics, (where
    # 0 = good/true, 1 = bad/false), so to check existence in UNIX one would do
    #   if ! fissfc space_exists blah ; then
    #      ...
    #   fi
    try:
        r = fapi.get_workspace(args.project, args.workspace)
        fapi._check_response_code(r, 200)
        exists = True
    except FireCloudServerError as e:
        if e.code == 404:
            exists = False
        else:
            raise
    if fcconfig.verbosity:
        result = "DOES NOT" if not exists else "DOES"
        eprint('Space <%s> %s exist in project <%s>' %
                                (args.workspace, result, args.project))
    return exists


Instruction:
Write a Python 3 script for
creating a new workspace.

Response:
def space_new(args):
    """ Create a new workspace. """
    r = fapi.create_workspace(args.project, args.workspace,
                                 args.authdomain, dict())
    fapi._check_response_code(r, 201)
    if fcconfig.verbosity:
        eprint(r.content)
    return 0


Instruction:
Write a Python 3 script to
get metadata for a workspace.

Response:
def space_info(args):
    """ Get metadata for a workspace. """
    r = fapi.get_workspace(args.project, args.workspace)
    fapi._check_response_code(r, 200)
    return r.text


Instruction:
Can you tell what is the following Python 3 function doing
def space_acl(args):
    ''' Retrieve access control list for a workspace'''
    r = fapi.get_workspace_acl(args.project, args.workspace)
    fapi._check_response_code(r, 200)
    result = dict()
    for user, info in sorted(r.json()['acl'].items()):
        result[user] = info['accessLevel']
    return result

Response: Retrieve access control list for a workspace


Instruction:
Write a Python 3 script for
assigning an ACL role to list of users for a workspace

Response:
def space_set_acl(args):
    """ Assign an ACL role to list of users for a workspace """
    acl_updates = [{"email": user,
                   "accessLevel": args.role} for user in args.users]
    r = fapi.update_workspace_acl(args.project, args.workspace, acl_updates)
    fapi._check_response_code(r, 200)
    errors = r.json()['usersNotFound']

    if len(errors):
        eprint("Unable to assign role for unrecognized users:")
        for user in errors:
            eprint("\t{0}".format(user['email']))
        return 1

    if fcconfig.verbosity:
        print("Successfully updated {0} role(s)".format(len(acl_updates)))

    return 0


Instruction:
How would you implement a function in Python 3 that
searches for workspaces matching certain criteria

Response:
def space_search(args):
    """ Search for workspaces matching certain criteria """
    r = fapi.list_workspaces()
    fapi._check_response_code(r, 200)

    # Parse the JSON for workspace + namespace; then filter by
    # search terms: each term is treated as a regular expression
    workspaces = r.json()
    extra_terms = []
    if args.bucket:
        workspaces = [w for w in workspaces
                      if re.search(args.bucket, w['workspace']['bucketName'])]
        extra_terms.append('bucket')

    # FIXME: add more filter terms
    pretty_spaces = []
    for space in workspaces:
        ns = space['workspace']['namespace']
        ws = space['workspace']['name']
        pspace = ns + '/' + ws
        # Always show workspace storage id
        pspace += '\t' + space['workspace']['bucketName']
        pretty_spaces.append(pspace)

    # Sort for easier viewing, ignore case
    return sorted(pretty_spaces, key=lambda s: s.lower())


Instruction:
Write a Python 3 function that can
upload an entity loadfile.

Response:
def entity_import(args):
    """ Upload an entity loadfile. """
    project = args.project
    workspace = args.workspace
    chunk_size = args.chunk_size

    with open(args.tsvfile) as tsvf:
        headerline = tsvf.readline().strip()
        entity_data = [l.rstrip('\n') for l in tsvf]

    return _batch_load(project, workspace, headerline, entity_data, chunk_size)


Instruction:
Write a Python 3 function for
returning a list of lines that would suffice to reconstitute a container ( set ) entity if passed to entity_import.

Response:
def set_export(args):
    '''Return a list of lines in TSV form that would suffice to reconstitute a
       container (set) entity, if passed to entity_import.  The first line in
       the list is the header, and subsequent lines are the container members.
    '''

    r = fapi.get_entity(args.project, args.workspace, args.entity_type, args.entity)
    fapi._check_response_code(r, 200)
    set_type = args.entity_type
    set_name = args.entity
    member_type = set_type.split('_')[0]
    members = r.json()['attributes'][member_type+'s']['items']

    result = ["membership:{}_id\t{}_id".format(set_type, member_type)]
    result += ["%s\t%s" % (set_name, m['entityName']) for m in members ]
    return result


Instruction:
Can you generate the documentation for the following Python 3 function
def entity_types(args):
    r = fapi.list_entity_types(args.project, args.workspace)
    fapi._check_response_code(r, 200)
    return r.json().keys()

Response: List entity types in a workspace


Instruction:
How would you code a function in Python 3 to
list entities in a workspace.

Response:
def entity_list(args):
    """ List entities in a workspace. """
    r = fapi.get_entities_with_type(args.project, args.workspace)
    fapi._check_response_code(r, 200)
    return [ '{0}\t{1}'.format(e['entityType'], e['name']) for e in r.json() ]


Instruction:
Implement a function in Python 3 to
list participants within a container

Response:
def participant_list(args):
    ''' List participants within a container'''

    # Case 1: retrieve participants within a named data entity
    if args.entity_type and args.entity:
        # Edge case: caller asked for participant within participant (itself)
        if args.entity_type == 'participant':
            return [ args.entity.strip() ]
        # Otherwise retrieve the container entity
        r = fapi.get_entity(args.project, args.workspace, args.entity_type, args.entity)
        fapi._check_response_code(r, 200)
        participants = r.json()['attributes']["participants"]['items']
        return [ participant['entityName'] for participant in participants ]

    # Case 2: retrieve all participants within a workspace
    return __get_entities(args, "participant", page_size=2000)


Instruction:
Write a Python 3 script to
list the pairs within a container.

Response:
def pair_list(args):
    ''' List pairs within a container. '''

    # Case 1: retrieve pairs within a named data entity
    if args.entity_type and args.entity:
        # Edge case: caller asked for pair within a pair (itself)
        if args.entity_type == 'pair':
            return [ args.entity.strip() ]
        # Edge case: pairs for a participant, which has to be done hard way
        # by iteratiing over all samples (see firecloud/discussion/9648)
        elif args.entity_type == 'participant':
            entities = _entity_paginator(args.project, args.workspace,
                                     'pair', page_size=2000)
            return [ e['name'] for e in entities if
                     e['attributes']['participant']['entityName'] == args.entity]

        # Otherwise retrieve the container entity
        r = fapi.get_entity(args.project, args.workspace, args.entity_type, args.entity)
        fapi._check_response_code(r, 200)
        pairs = r.json()['attributes']["pairs"]['items']
        return [ pair['entityName'] for pair in pairs]

    # Case 2: retrieve all pairs within a workspace
    return __get_entities(args, "pair", page_size=2000)


Instruction:
Write a Python 3 script for
listing samples within a container.

Response:
def sample_list(args):
    ''' List samples within a container. '''

    # Case 1: retrieve samples within a named data entity
    if args.entity_type and args.entity:
        # Edge case: caller asked for samples within a sample (itself)
        if args.entity_type == 'sample':
            return [ args.entity.strip() ]
        # Edge case: samples for a participant, which has to be done hard way
        # by iteratiing over all samples (see firecloud/discussion/9648)
        elif args.entity_type == 'participant':
            samples = _entity_paginator(args.project, args.workspace,
                                     'sample', page_size=2000)
            return [ e['name'] for e in samples if
                     e['attributes']['participant']['entityName'] == args.entity]

        # Otherwise retrieve the container entity
        r = fapi.get_entity(args.project, args.workspace, args.entity_type, args.entity)
        fapi._check_response_code(r, 200)
        if args.entity_type == 'pair':
            pair = r.json()['attributes']
            samples = [ pair['case_sample'], pair['control_sample'] ]
        else:
            samples = r.json()['attributes']["samples"]['items']

        return [ sample['entityName'] for sample in samples ]

    # Case 2: retrieve all samples within a workspace
    return __get_entities(args, "sample", page_size=2000)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def entity_delete(args):

    msg = "WARNING: this will delete {0} {1} in {2}/{3}".format(
        args.entity_type, args.entity, args.project, args.workspace)

    if not (args.yes or _confirm_prompt(msg)):
        return

    json_body=[{"entityType": args.entity_type,
                "entityName": args.entity}]
    r = fapi.delete_entities(args.project, args.workspace, json_body)
    fapi._check_response_code(r, 204)
    if fcconfig.verbosity:
        print("Succesfully deleted " + args.type + " " + args.entity)

Response: Delete an entity in a workspace.


Instruction:
Can you implement a function in Python 3 that
submits a new workflow to the methods repository.

Response:
def meth_new(args):
    """ Submit a new workflow (or update) to the methods repository. """
    r = fapi.update_repository_method(args.namespace, args.method,
                                      args.synopsis, args.wdl, args.doc,
                                      args.comment)
    fapi._check_response_code(r, 201)
    if fcconfig.verbosity:
        print("Method %s installed to project %s" % (args.method,
                                                     args.namespace))
    return 0


Instruction:
Can you generate the documentation for the following Python 3 function
def meth_delete(args):
    message = "WARNING: this will delete workflow \n\t{0}/{1}:{2}".format(
                                    args.namespace, args.method, args.snapshot_id)
    if not args.yes and not _confirm_prompt(message):
        return

    r = fapi.delete_repository_method(args.namespace, args.method,
                                                    args.snapshot_id)
    fapi._check_response_code(r, 200)
    if fcconfig.verbosity:
        print("Method %s removed from project %s" % (args.method,args.namespace))
    return 0

Response: Remove a method from the method repository


Instruction:
Explain what the following Python 3 code does
def meth_wdl(args):
    ''' Retrieve WDL for given version of a repository method'''
    r = fapi.get_repository_method(args.namespace, args.method,
                                   args.snapshot_id, True)
    fapi._check_response_code(r, 200)
    return r.text

Response: Retrieve WDL for given version of a repository method


Instruction:
Can you generate a brief explanation for the following Python 3 code
def meth_acl(args):
    ''' Retrieve access control list for given version of a repository method'''
    r = fapi.get_repository_method_acl(args.namespace, args.method,
                                                    args.snapshot_id)
    fapi._check_response_code(r, 200)
    acls = sorted(r.json(), key=lambda k: k['user'])
    return map(lambda acl: '{0}\t{1}'.format(acl['user'], acl['role']), acls)

Response: Retrieve access control list for given version of a repository method


Instruction:
Explain what the following Python 3 code does
def meth_set_acl(args):
    acl_updates = [{"user": user, "role": args.role} \
                   for user in set(expand_fc_groups(args.users)) \
                   if user != fapi.whoami()]

    id = args.snapshot_id
    if not id:
        # get the latest snapshot_id for this method from the methods repo
        r = fapi.list_repository_methods(namespace=args.namespace,
                                         name=args.method)
        fapi._check_response_code(r, 200)
        versions = r.json()
        if len(versions) == 0:
            if fcconfig.verbosity:
                eprint("method {0}/{1} not found".format(args.namespace,
                                                         args.method))
            return 1
        latest = sorted(versions, key=lambda m: m['snapshotId'])[-1]
        id = latest['snapshotId']

    r = fapi.update_repository_method_acl(args.namespace, args.method, id,
                                          acl_updates)
    fapi._check_response_code(r, 200)
    if fcconfig.verbosity:
        print("Updated ACL for {0}/{1}:{2}".format(args.namespace, args.method,
                                                   id))
    return 0

Response: Assign an ACL role to a list of users for a workflow.


Instruction:
Can you tell what is the following Python 3 function doing
def expand_fc_groups(users):
    groups = None
    for user in users:
        fcgroup = None
        if '@' not in user:
            fcgroup = user
        elif user.lower().endswith('@firecloud.org'):
            if groups is None:
                r = fapi.get_groups()
                fapi._check_response_code(r, 200)
                groups = {group['groupEmail'].lower():group['groupName'] \
                          for group in r.json() if group['role'] == 'Admin'}
            if user.lower() not in groups:
                if fcconfig.verbosity:
                    eprint("You do not have access to the members of {}".format(user))
                yield user
                continue
            else:
                fcgroup = groups[user.lower()]
        else:
            yield user
            continue
        r = fapi.get_group(fcgroup)
        fapi._check_response_code(r, 200)
        fcgroup_data = r.json()
        for admin in fcgroup_data['adminsEmails']:
            yield admin
        for member in fcgroup_data['membersEmails']:
            yield member

Response: Given a list of users return all members of the group.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def meth_list(args):
    r = fapi.list_repository_methods(namespace=args.namespace,
                                     name=args.method,
                                     snapshotId=args.snapshot_id)
    fapi._check_response_code(r, 200)

    # Parse the JSON for the workspace + namespace
    methods = r.json()
    results = []
    for m in methods:
        ns = m['namespace']
        n = m['name']
        sn_id = m['snapshotId']
        results.append('{0}\t{1}\t{2}'.format(ns,n,sn_id))

    # Sort for easier viewing, ignore case
    return sorted(results, key=lambda s: s.lower())

Response: List workflows in the methods repository


Instruction:
Write a Python 3 script to
invoke a task ( method configuration ) on given entity in given space

Response:
def config_start(args):
    '''Invoke a task (method configuration), on given entity in given space'''

    # Try to use call caching (job avoidance)?  Flexibly accept range of answers
    cache = getattr(args, "cache", True)
    cache = cache is True or (cache.lower() in ["y", "true", "yes", "t", "1"])

    if not args.namespace:
        args.namespace = fcconfig.method_ns
    if not args.namespace:
        raise RuntimeError("namespace not provided, or configured by default")

    r = fapi.create_submission(args.project, args.workspace,args.namespace,
                            args.config, args.entity, args.entity_type,
                            args.expression, use_callcache=cache)
    fapi._check_response_code(r, 201)
    id = r.json()['submissionId']

    return ("Started {0}/{1} in {2}/{3}: id={4}".format(
        args.namespace, args.config, args.project, args.workspace, id)), id


Instruction:
Can you write a function in Python 3 where it
aborts a task by submission ID in given space

Response:
def config_stop(args):
    '''Abort a task (method configuration) by submission ID in given space'''

    r = fapi.abort_submission(args.project, args.workspace,
                              args.submission_id)
    fapi._check_response_code(r, 204)

    return ("Aborted {0} in {1}/{2}".format(args.submission_id,
                                            args.project,
                                            args.workspace))


Instruction:
Implement a function in Python 3 to
list the method configs in the methods repository or a workspace.

Response:
def config_list(args):
    """ List configuration(s) in the methods repository or a workspace. """
    verbose = fcconfig.verbosity
    if args.workspace:
        if verbose:
            print("Retrieving method configs from space {0}".format(args.workspace))
        if not args.project:
            eprint("No project given, and no default project configured")
            return 1
        r = fapi.list_workspace_configs(args.project, args.workspace)
        fapi._check_response_code(r, 200)
    else:
        if verbose:
            print("Retrieving method configs from method repository")
        r = fapi.list_repository_configs(namespace=args.namespace,
                                         name=args.config,
                                         snapshotId=args.snapshot_id)
        fapi._check_response_code(r, 200)

    # Parse the JSON for the workspace + namespace
    methods = r.json()
    results = []
    for m in methods:
        ns = m['namespace']
        if not ns:
            ns = '__EMPTYSTRING__'
        name = m['name']
        # Ugh: configs in workspaces look different from configs in methodRepo
        mver = m.get('methodRepoMethod', None)
        if mver:
            mver = mver.get('methodVersion', 'unknown')     # space config
        else:
            mver = m.get('snapshotId', 'unknown')           # repo  config
        results.append('{0}\t{1}\tsnapshotId:{2}'.format(ns, name, mver))

    # Sort for easier viewing, ignore case
    return sorted(results, key=lambda s: s.lower())


Instruction:
Implement a function in Python 3 to
retrieve access control list for a method configuration

Response:
def config_acl(args):
    ''' Retrieve access control list for a method configuration'''
    r = fapi.get_repository_config_acl(args.namespace, args.config,
                                                    args.snapshot_id)
    fapi._check_response_code(r, 200)
    acls = sorted(r.json(), key=lambda k: k['user'])
    return map(lambda acl: '{0}\t{1}'.format(acl['user'], acl['role']), acls)


Instruction:
Can you create a Python 3 function that
assigns an ACL role to a list of users for a config.

Response:
def config_set_acl(args):
    """ Assign an ACL role to a list of users for a config. """
    acl_updates = [{"user": user, "role": args.role} \
                   for user in set(expand_fc_groups(args.users)) \
                   if user != fapi.whoami()]

    id = args.snapshot_id
    if not id:
        # get the latest snapshot_id for this method from the methods repo
        r = fapi.list_repository_configs(namespace=args.namespace,
                                         name=args.config)
        fapi._check_response_code(r, 200)
        versions = r.json()
        if len(versions) == 0:
            if fcconfig.verbosity:
                eprint("Configuration {0}/{1} not found".format(args.namespace,
                                                                args.config))
            return 1
        latest = sorted(versions, key=lambda c: c['snapshotId'])[-1]
        id = latest['snapshotId']

    r = fapi.update_repository_config_acl(args.namespace, args.config, id,
                                          acl_updates)
    fapi._check_response_code(r, 200)
    if fcconfig.verbosity:
        print("Updated ACL for {0}/{1}:{2}".format(args.namespace, args.config,
                                                   id))
    return 0


Instruction:
Can you implement a function in Python 3 that
retrieves a method config from a workspace send stdout

Response:
def config_get(args):
    """ Retrieve a method config from a workspace, send stdout """
    r = fapi.get_workspace_config(args.project, args.workspace,
                                        args.namespace, args.config)
    fapi._check_response_code(r, 200)
    # Setting ensure_ascii to False ensures unicode string returns
    return json.dumps(r.json(), indent=4, separators=(',', ': '),
                      sort_keys=True, ensure_ascii=False)


Instruction:
How would you code a function in Python 3 to
retrieve the WDL for a method config in a workspace send stdout

Response:
def config_wdl(args):
    """ Retrieve the WDL for a method config in a workspace, send stdout """
    r = fapi.get_workspace_config(args.project, args.workspace,
                                  args.namespace, args.config)
    fapi._check_response_code(r, 200)

    method = r.json()["methodRepoMethod"]
    args.namespace   = method["methodNamespace"]
    args.method      = method["methodName"]
    args.snapshot_id = method["methodVersion"]

    return meth_wdl(args)


Instruction:
Create a Python 3 function for
comparing method configuration definitions across workspaces. Ignores methodConfigVersion if the verbose argument is not set

Response:
def config_diff(args):
    """Compare method configuration definitions across workspaces. Ignores
       methodConfigVersion if the verbose argument is not set"""
    config_1 = config_get(args).splitlines()
    args.project = args.Project
    args.workspace = args.Workspace
    cfg_1_name = args.config
    if args.Config is not None:
        args.config = args.Config
    if args.Namespace is not None:
        args.namespace = args.Namespace
    config_2 = config_get(args).splitlines()
    if not args.verbose:
        config_1 = skip_cfg_ver(config_1)
        config_2 = skip_cfg_ver(config_2)
    return list(unified_diff(config_1, config_2, cfg_1_name, args.config, lineterm=''))


Instruction:
Write a Python 3 script to
install a valid method configuration into a workspace in one of several several ways.

Response:
def config_put(args):
    '''Install a valid method configuration into a workspace, in one of several
       ways: from a JSON file containing a config definition (both file names
       and objects are supported); as a string representing the content of such
       a JSON file; or as a dict generated from such JSON content, e.g via
       json.loads(). Note that the CLI supports only string & filename input.'''

    config = args.config
    if os.path.isfile(config):
        with open(config, 'r') as fp:
            config = json.loads(fp.read())
    elif isinstance(config, str):
        config = json.loads(config)
    elif isinstance(config, dict):
        pass
    elif hasattr(config, "read"):
        config = json.loads(config.read())
    else:
        raise ValueError('Input method config must be filename, string or dict')

    r = fapi.create_workspace_config(args.project, args.workspace, config)
    fapi._check_response_code(r, [201])
    return True


Instruction:
Implement a function in Python 3 to
install a new method config into a workspace.

Response:
def config_new(args):
    '''Attempt to install a new method config into a workspace, by: generating
       a template from a versioned method in the methods repo, then launching
       a local editor (respecting the $EDITOR environment variable) to fill in
       the incomplete input/output fields.  Returns True if the config was
       successfully installed, otherwise False'''

    cfg = config_template(args)
    # Iteratively try to edit/install the config: exit iteration by EITHER
    #   Successful config_put() after editing
    #   Leaving config unchanged in editor, e.g. quitting out of VI with :q
    #   FIXME: put an small integer upper bound on the # of loops here
    while True:
        try:
            edited = fccore.edit_text(cfg)
            if edited == cfg:
                eprint("No edits made, method config not installed ...")
                break
            if __EDITME__ in edited:
                eprint("Edit is incomplete, method config not installed ...")
                time.sleep(1)
                continue
            args.config = cfg = edited
            config_put(args)
            return True
        except FireCloudServerError as fce:
            __pretty_print_fc_exception(fce)

    return False


Instruction:
Explain what the following Python 3 code does
def config_delete(args):
    r = fapi.delete_workspace_config(args.project, args.workspace,
                                        args.namespace, args.config)
    fapi._check_response_code(r, [200,204])
    return r.text if r.text else None

Response: Remove a method config from a workspace


Instruction:
Write a Python 3 function for
copying a method config to a new name space or namespace

Response:
def config_copy(args):
    """ Copy a method config to new name/space/namespace/project (or all 4) """
    if not (args.tospace or args.toname or args.toproject or args.tonamespace):
        raise RuntimeError('A new config name OR workspace OR project OR ' +
                           'namespace must be given (or all)')

    copy = fapi.get_workspace_config(args.fromproject, args.fromspace,
                                            args.namespace, args.config)
    fapi._check_response_code(copy, 200)

    copy = copy.json()
    if not args.toname:
        args.toname = args.config

    if not args.tonamespace:
        args.tonamespace = args.namespace

    if not args.toproject:
        args.toproject = args.fromproject

    if not args.tospace:
        args.tospace = args.fromspace

    copy['name'] = args.toname
    copy['namespace'] = args.tonamespace

    # Instantiate the copy
    r = fapi.overwrite_workspace_config(args.toproject, args.tospace,
                                        args.tonamespace, args.toname, copy)
    fapi._check_response_code(r, 200)

    if fcconfig.verbosity:
        print("Method %s/%s:%s copied to %s/%s:%s" % (
                args.fromproject, args.fromspace, args.config,
                args.toproject, args.tospace, args.toname))

    return 0


Instruction:
Given the following Python 3 function, write the documentation
def attr_get(args):
    '''Return a dict of attribute name/value pairs: if entity name & type
    are specified then attributes will be retrieved from that entity,
    otherwise workspace-level attributes will be returned.  By default all
    attributes attached to the given object will be returned, but a subset
    can be selected by specifying a list of attribute names; names which
    refer to a non-existent attribute will be silently ignored. By default
    a special __header__ entry is optionally added to the result. '''

    if args.entity_type and args.entity:
        r = fapi.get_entity(args.project, args.workspace, args.entity_type, args.entity)
        fapi._check_response_code(r, 200)
        attrs = r.json()['attributes']
        # It is wrong for the members of container objects to appear as metadata
        # (attributes) attached to the container, as it conflates fundamentally
        # different things: annotation vs membership. This can be seen by using
        # a suitcase model of reasoning: ATTRIBUTES are METADATA like the weight
        # & height of the suitcase, the shipping label and all of its passport
        # stamps; while MEMBERS are the actual CONTENTS INSIDE the suitcase.
        # This conflation also contradicts the design & docs (which make a clear
        # distinction between MEMBERSHIP and UPDATE loadfiles).  For this reason
        # a change has been requested of the FireCloud dev team (via forum), and
        # until it is implemented we will elide "list of members" attributes here
        # (but later may provide a way for users to request such, if wanted)
        for k in ["samples", "participants", "pairs"]:
            attrs.pop(k, None)
    else:
        r = fapi.get_workspace(args.project, args.workspace)
        fapi._check_response_code(r, 200)
        attrs = r.json()['workspace']['attributes']

    if args.attributes:         # return a subset of attributes, if requested
        attrs = {k:attrs[k] for k in set(attrs).intersection(args.attributes)}

    # If some attributes have been collected, return in appropriate format
    if attrs:
        if args.entity:                     # Entity attributes

            def textify(thing):
                if isinstance(thing, dict):
                    thing = thing.get("items", thing.get("entityName", "__UNKNOWN__"))
                return "{0}".format(thing)

            result = {args.entity : u'\t'.join(map(textify, attrs.values()))}
            # Add "hidden" header of attribute names, for downstream convenience
            object_id = u'entity:%s_id' % args.entity_type
            result['__header__'] = [object_id] + list(attrs.keys())
        else:
            result = attrs                  # Workspace attributes
    else:
        result = {}

    return result

Response: Return a dict of attribute name - value pairs for a given entity type or workspace - level object.


Instruction:
Create a Python 3 function to
retrieve names of all attributes attached to a given object either an entity type + name or workspace

Response:
def attr_list(args):
    '''Retrieve names of all attributes attached to a given object, either
       an entity (if entity type+name is provided) or workspace (if not)'''
    args.attributes = None
    result = attr_get(args)
    names = result.get("__header__",[])
    if names:
        names = names[1:]
    else:
        names = result.keys()
    return sorted(names)


Instruction:
Write a Python 3 script for
setting key = value attributes for a single entity or a single workspace

Response:
def attr_set(args):
    ''' Set key=value attributes: if entity name & type are specified then
    attributes will be set upon that entity, otherwise the attribute will
    be set at the workspace level'''

    if args.entity_type and args.entity:
        prompt = "Set {0}={1} for {2}:{3} in {4}/{5}?\n[Y\\n]: ".format(
                            args.attribute, args.value, args.entity_type,
                            args.entity, args.project, args.workspace)

        if not (args.yes or _confirm_prompt("", prompt)):
            return 0

        update = fapi._attr_set(args.attribute, args.value)
        r = fapi.update_entity(args.project, args.workspace, args.entity_type,
                                                        args.entity, [update])
        fapi._check_response_code(r, 200)
    else:
        prompt = "Set {0}={1} in {2}/{3}?\n[Y\\n]: ".format(
            args.attribute, args.value, args.project, args.workspace
        )

        if not (args.yes or _confirm_prompt("", prompt)):
            return 0

        update = fapi._attr_set(args.attribute, args.value)
        r = fapi.update_workspace_attributes(args.project, args.workspace,
                                                                [update])
        fapi._check_response_code(r, 200)
    return 0


Instruction:
Make a summary of the following Python 3 code
def attr_delete(args):
    ''' Delete key=value attributes: if entity name & type are specified then
    attributes will be deleted from that entity, otherwise the attribute will
    be removed from the workspace'''

    if args.entity_type and args.entities:
        # Since there is no attribute deletion endpoint, we must perform 2 steps
        # here: first we retrieve the entity_ids, and any foreign keys (e.g.
        # participant_id for sample_id); and then construct a loadfile which
        # specifies which entities are to have what attributes removed.  Note
        # that FireCloud uses the magic keyword __DELETE__ to indicate that
        # an attribute should be deleted from an entity.

        # Step 1: see what entities are present, and filter to those requested
        entities = _entity_paginator(args.project, args.workspace,
                                     args.entity_type,
                                     page_size=1000, filter_terms=None,
                                     sort_direction="asc")
        if args.entities:
            entities = [e for e in entities if e['name'] in args.entities]

        # Step 2: construct a loadfile to delete these attributes
        attrs = sorted(args.attributes)
        etype = args.entity_type

        entity_data = []
        for entity_dict in entities:
            name = entity_dict['name']
            line = name
            # TODO: Fix other types?
            if etype == "sample":
                line += "\t" + entity_dict['attributes']['participant']['entityName']
            for attr in attrs:
                line += "\t__DELETE__"
            # Improve performance by only updating records that have changed
            entity_data.append(line)

        entity_header = ["entity:" + etype + "_id"]
        if etype == "sample":
            entity_header.append("participant_id")
        entity_header = '\t'.join(entity_header + list(attrs))

        # Remove attributes from an entity
        message = "WARNING: this will delete these attributes:\n\n" + \
                  ','.join(args.attributes) + "\n\n"
        if args.entities:
            message += 'on these {0}s:\n\n'.format(args.entity_type) + \
                       ', '.join(args.entities)
        else:
            message += 'on all {0}s'.format(args.entity_type)
        message += "\n\nin workspace {0}/{1}\n".format(args.project, args.workspace)
        if not args.yes and not _confirm_prompt(message):
            return 0

        # TODO: reconcile with other batch updates
        # Chunk the entities into batches of 500, and upload to FC
        if args.verbose:
            print("Batching " + str(len(entity_data)) + " updates to Firecloud...")
        chunk_len = 500
        total = int(len(entity_data) / chunk_len) + 1
        batch = 0
        for i in range(0, len(entity_data), chunk_len):
            batch += 1
            if args.verbose:
                print("Updating samples {0}-{1}, batch {2}/{3}".format(
                    i+1, min(i+chunk_len, len(entity_data)), batch, total
                ))
            this_data = entity_header + '\n' + '\n'.join(entity_data[i:i+chunk_len])

            # Now push the entity data back to firecloud
            r = fapi.upload_entities(args.project, args.workspace, this_data)
            fapi._check_response_code(r, 200)
    else:
        message = "WARNING: this will delete the following attributes in " + \
                  "{0}/{1}\n\t".format(args.project, args.workspace) + \
                  "\n\t".join(args.attributes)

        if not (args.yes or _confirm_prompt(message)):
            return 0

        updates = [fapi._attr_rem(a) for a in args.attributes]
        r = fapi.update_workspace_attributes(args.project, args.workspace,
                                             updates)
        fapi._check_response_code(r, 200)

    return 0

Response: Delete attributes from an entity in a workspace.


Instruction:
Given the following Python 3 function, write the documentation
def attr_copy(args):
    if not args.to_workspace:
        args.to_workspace = args.workspace
    if not args.to_project:
        args.to_project = args.project
    if (args.project == args.to_project
        and args.workspace == args.to_workspace):
        eprint("destination project and namespace must differ from"
               " source workspace")
        return 1

    # First get the workspace attributes of the source workspace
    r = fapi.get_workspace(args.project, args.workspace)
    fapi._check_response_code(r, 200)

    # Parse the attributes
    workspace_attrs = r.json()['workspace']['attributes']

    # If we passed attributes, only use those
    if args.attributes:
        workspace_attrs = {k:v for k, v in iteritems(workspace_attrs)
                           if k in args.attributes}

    if len(workspace_attrs) == 0:
        print("No workspace attributes defined in {0}/{1}".format(
            args.project, args.workspace))
        return 1

    message = "This will copy the following workspace attributes to {0}/{1}\n"
    message = message.format(args.to_project, args.to_workspace)
    for k, v in sorted(iteritems(workspace_attrs)):
        message += '\t{0}\t{1}\n'.format(k, v)

    if not args.yes and not _confirm_prompt(message):
        return 0

    # make the attributes into updates
    updates = [fapi._attr_set(k,v) for k,v in iteritems(workspace_attrs)]
    r = fapi.update_workspace_attributes(args.to_project, args.to_workspace,
                                                                    updates)
    fapi._check_response_code(r, 200)
    return 0

Response: Copy workspace attributes between workspaces.


Instruction:
Can you generate the documentation for the following Python 3 function
def attr_fill_null(args):
    NULL_SENTINEL = "gs://broad-institute-gdac/GDAC_FC_NULL"
    attrs = args.attributes

    if not attrs:
        print("Error: provide at least one attribute to set")
        return 1

    if 'participant' in attrs or 'samples' in attrs:
        print("Error: can't assign null to samples or participant")
        return 1

    # Set entity attributes
    if args.entity_type is not None:
        print("Collecting entity data...")
        # Get existing attributes
        entities = _entity_paginator(args.project, args.workspace,
                                     args.entity_type,
                                     page_size=1000, filter_terms=None,
                                     sort_direction="asc")

        # samples need participant_id as well
        #TODO: This may need more fixing for other types
        orig_attrs = list(attrs)
        if args.entity_type == "sample":
            attrs.insert(0, "participant_id")

        header = "entity:" + args.entity_type + "_id\t" + "\t".join(attrs)
        # Book keep the number of updates for each attribute
        attr_update_counts = {a : 0 for a in orig_attrs}

        # construct new entity data by inserting null sentinel, and counting
        # the number of updates
        entity_data = []
        for entity_dict in entities:
            name = entity_dict['name']
            etype = entity_dict['entityType']
            e_attrs = entity_dict['attributes']
            line = name
            altered = False
            for attr in attrs:
                if attr == "participant_id":
                    line += "\t" + e_attrs['participant']['entityName']
                    continue # This attribute is never updated by fill_null
                if attr not in e_attrs:
                    altered = True
                    attr_update_counts[attr] += 1
                line += "\t" + str(e_attrs.get(attr, NULL_SENTINEL))
            # Improve performance by only updating records that have changed
            if altered:
                entity_data.append(line)

        # Check to see if all entities are being set to null for any attributes
        # This is usually a mistake, so warn the user
        num_entities = len(entities)
        prompt = "Continue? [Y\\n]: "
        for attr in orig_attrs:
            if num_entities == attr_update_counts[attr]:
                message = "WARNING: no {0}s with attribute '{1}'\n".format(
                    args.entity_type, attr
                )
                if not args.yes and not _confirm_prompt(message, prompt):
                    return

        # check to see if no sentinels are necessary
        if not any(c != 0 for c in itervalues(attr_update_counts)):
            print("No null sentinels required, exiting...")
            return 0

        if args.to_loadfile:
            print("Saving loadfile to " + args.to_loadfile)
            with open(args.to_loadfile, "w") as f:
                f.write(header + '\n')
                f.write("\n".join(entity_data))
            return 0

        updates_table = "     count attribute\n"
        for attr in sorted(attr_update_counts):
            count = attr_update_counts[attr]
            updates_table += "{0:>10} {1}\n".format(count, attr)

        message = "WARNING: This will insert null sentinels for " \
                  "these attributes:\n" + updates_table
        if not args.yes and not _confirm_prompt(message):
            return 0

        # Chunk the entities into batches of 500, and upload to FC
        print("Batching " + str(len(entity_data)) + " updates to Firecloud...")
        chunk_len = 500
        total = int(len(entity_data) / chunk_len) + 1
        batch = 0
        for i in range(0, len(entity_data), chunk_len):
            batch += 1
            print("Updating samples {0}-{1}, batch {2}/{3}".format(
                i+1, min(i+chunk_len, len(entity_data)), batch, total
            ))
            this_data = header + '\n' + '\n'.join(entity_data[i:i+chunk_len])

            # Now push the entity data back to firecloud
            r = fapi.upload_entities(args.project, args.workspace, this_data)
            fapi._check_response_code(r, 200)

        return 0
    else:
        # TODO: set workspace attributes
        print("attr_fill_null requires an entity type")
        return 1

Response: Assign the null sentinel value for all entities which do not have a value for all attributes.