Instruction:
Write a Python 3 script to
start a multi - process server with the given context manager.

Response:
def start_server(worker_actxmgr: AsyncServerContextManager,
                 main_ctxmgr: Optional[ServerMainContextManager] = None,
                 extra_procs: Iterable[Callable] = tuple(),
                 stop_signals: Iterable[signal.Signals] = (
                     signal.SIGINT,
                     signal.SIGTERM),
                 num_workers: int = 1,
                 use_threading: bool = False,
                 args: Iterable[Any] = tuple()):
    '''
    Starts a multi-process server where each process has their own individual
    asyncio event loop.  Their lifecycles are automantically managed -- if the
    main program receives one of the signals specified in ``stop_signals`` it
    will initiate the shutdown routines on each worker that stops the event
    loop gracefully.

    Args:
        worker_actxmgr: An asynchronous context manager that dicates the
                        initialization and shutdown steps of each worker.
                        It should accept the following three arguments:

                        * **loop**: the asyncio event loop created and set
                          by aiotools
                        * **pidx**: the 0-based index of the worker
                          (use this for per-worker logging)
                        * **args**: a concatenated tuple of values yielded by
                          **main_ctxmgr** and the user-defined arguments in
                          **args**.

                        aiotools automatically installs an interruption handler
                        that calls ``loop.stop()`` to the given event loop,
                        regardless of using either threading or
                        multiprocessing.

        main_ctxmgr: An optional context manager that performs global
                     initialization and shutdown steps of the whole program.
                     It may yield one or more values to be passed to worker
                     processes along with **args** passed to this function.
                     There is no arguments passed to those functions since
                     you can directly access ``sys.argv`` to parse command
                     line arguments and/or read user configurations.

        extra_procs: An iterable of functions that consist of extra processes
                     whose lifecycles are synchronized with other workers.

                     You should write the shutdown steps of them differently
                     depending on the value of **use_threading** argument.

                     If it is ``False`` (default), they will get
                     a :class:`BaseException` depending on the received stop signal
                     number, either :class:`KeyboardInterrupt` (for SIGINT),
                     :class:`SystemExit` (for SIGTERM), or
                     :class:`InterruptedBySignal` (otherwise).

                     If it is ``True``, they should check their **intr_event**
                     argument periodically because there is no way to install
                     signal handlers in Python threads (only the main thread
                     can install signal handlers).

                     It should accept the following three arguments:

                     * **intr_event**: :class:`threading.Event` object that
                       signals the interruption of the main thread (only
                       available when **use_threading** is ``True``; otherwise
                       it is set to ``None``)
                     * **pidx**: same to **worker_actxmgr** argument
                     * **args**: same to **worker_actxmgr** argument

        stop_signals: A list of UNIX signals that the main program to
                      recognize as termination signals.

        num_workers: The number of children workers.

        use_threading: Use :mod:`threading` instead of :mod:`multiprocessing`.
                       In this case, the GIL may become the performance
                       bottleneck.  Set this ``True`` only when you know what
                       you are going to do.  Note that this changes the way
                       to write user-defined functions passed as **extra_procs**.

        args: The user-defined arguments passed to workers and extra
              processes.  If **main_ctxmgr** yields one or more values,
              they are *prepended* to this user arguments when passed to
              workers and extra processes.

    Returns:
        None

    .. versionchanged:: 0.3.2

       The name of argument **num_proc** is changed to **num_workers**.
       Even if **num_workers** is 1, a child is created instead of
       doing everything at the main thread.

    .. versionadded:: 0.3.2

       The argument ``extra_procs`` and ``main_ctxmgr``.

    .. versionadded:: 0.4.0

       Now supports use of threading instead of multiprocessing via
       **use_threading** option.

    .. versionchanged:: 0.8.0

       Now **worker_actxmgr** must be an instance of
       :class:`AsyncServerContextManager` or async generators decorated by
       ``@aiotools.server``.

       Now **main_ctxmgr** must be an instance of :class:`ServerMainContextManager`
       or plain generators decorated by ``@aiotools.main``.

       The usage is same to asynchronous context managers, but optionally you can
       distinguish the received stop signal by retrieving the return value of the
       ``yield`` statement.

       In **extra_procs** in non-threaded mode, stop signals are converted into
       either one of :class:`KeyboardInterrupt`, :class:`SystemExit`, or
       :class:`InterruptedBySignal` exception.
    '''

    @_main_ctxmgr
    def noop_main_ctxmgr():
        yield

    def create_child(*args, **kwargs):
        if use_threading:
            return threading.Thread(*args, **kwargs)
        else:
            return mp.Process(*args, **kwargs)

    assert stop_signals

    if main_ctxmgr is None:
        main_ctxmgr = noop_main_ctxmgr

    children = []
    _children_ctxs.clear()
    _children_loops.clear()
    intr_event = threading.Event()
    sigblock_mask = frozenset(stop_signals)
    main_ctx = main_ctxmgr()

    # temporarily block signals and register signal handlers to mainloop
    signal.pthread_sigmask(signal.SIG_BLOCK, sigblock_mask)

    old_loop = asyncio.get_event_loop()
    mainloop = asyncio.new_event_loop()
    asyncio.set_event_loop(mainloop)

    # to make subprocess working in child threads
    try:
        asyncio.get_child_watcher()
    except NotImplementedError:
        pass  # for uvloop

    # build a main-to-worker interrupt channel using signals
    def handle_stop_signal(signum):
        main_ctx.yield_return = signum
        if use_threading:
            with _children_lock:
                for c in _children_ctxs:
                    c.yield_return = signum
                for l in _children_loops:
                    l.call_soon_threadsafe(l.stop)
            intr_event.set()
        else:
            os.killpg(0, signum)
        mainloop.stop()

    for signum in stop_signals:
        mainloop.add_signal_handler(
            signum,
            functools.partial(handle_stop_signal, signum))

    # build a reliable worker-to-main interrupt channel using a pipe
    # (workers have no idea whether the main interrupt is enabled/disabled)
    def handle_child_interrupt(fd):
        child_idx = struct.unpack('i', os.read(fd, 4))[0]  # noqa
        log.debug(f'Child {child_idx} has interrupted the main program.')
        # self-interrupt to initiate the main-to-worker interrupts
        signal.pthread_sigmask(signal.SIG_UNBLOCK, {signal.SIGINT})
        os.kill(0, signal.SIGINT)

    if use_threading:
        child_intr_pipe = os.pipe()
        rfd = child_intr_pipe[0]
    else:
        child_intr_pipe = mp.Pipe()
        rfd = child_intr_pipe[0].fileno()
    mainloop.add_reader(rfd, handle_child_interrupt, rfd)

    # start
    with main_ctx as main_args:

        # retrieve args generated by the user-defined main
        if main_args is None:
            main_args = tuple()
        if not isinstance(main_args, tuple):
            main_args = (main_args, )

        # spawn managed async workers
        for i in range(num_workers):
            p = create_child(target=_worker_main, daemon=True,
                             args=(worker_actxmgr, use_threading, stop_signals,
                                   child_intr_pipe[1], i,
                                   main_args + args))
            p.start()
            children.append(p)

        # spawn extra workers
        for i, f in enumerate(extra_procs):
            p = create_child(target=_extra_main, daemon=True,
                             args=(f, use_threading, stop_signals,
                                   intr_event, num_workers + i,
                                   main_args + args))
            p.start()
            children.append(p)
        try:
            # unblock the stop signals for user/external interrupts.
            signal.pthread_sigmask(signal.SIG_UNBLOCK, sigblock_mask)

            # run!
            mainloop.run_forever()

            # if interrupted, wait for workers to finish.
            for child in children:
                child.join()
        finally:
            mainloop.close()
            asyncio.set_event_loop(old_loop)


Instruction:
How would you code a function in Python 3 to
check if the lock is acquired

Response:
def is_acquired(self):
        """Check if the lock is acquired"""
        values = self.client.get(self.key)
        return six.b(self._uuid) in values


Instruction:
Can you tell what is the following Python 3 function doing
def client(host='localhost', port=2379,
           ca_cert=None, cert_key=None, cert_cert=None,
           timeout=None, protocol="http"):
    """Return an instance of an Etcd3Client."""
    return Etcd3Client(host=host,
                       port=port,
                       ca_cert=ca_cert,
                       cert_key=cert_key,
                       cert_cert=cert_cert,
                       timeout=timeout,
                       protocol=protocol)

Response: Return an instance of an Etcd3Client.


Instruction:
How would you implement a function in Python 3 that
constructs a full url to the v3alpha API given a specific path

Response:
def get_url(self, path):
        """Construct a full url to the v3alpha API given a specific path

        :param path:
        :return: url
        """
        host = ('[' + self.host + ']' if (self.host.find(':') != -1)
                else self.host)
        base_url = self.protocol + '://' + host + ':' + str(self.port)
        return base_url + '/v3alpha/' + path.lstrip("/")


Instruction:
Explain what the following Python 3 code does
def lease(self, ttl=DEFAULT_TIMEOUT):
        result = self.post(self.get_url("/lease/grant"),
                           json={"TTL": ttl, "ID": 0})
        return Lease(int(result['ID']), client=self)

Response: Create a Lease object given a timeout.


Instruction:
Can you implement a function in Python 3 that
creates a Lock object given an ID and a TTL

Response:
def lock(self, id=str(uuid.uuid4()), ttl=DEFAULT_TIMEOUT):
        """Create a Lock object given an ID and timeout

        :param id: ID for the lock, creates a new uuid if not provided
        :param ttl: timeout
        :return: Lock object
        """
        return Lock(id, ttl=ttl, client=self)


Instruction:
Make a summary of the following Python 3 code
def create(self, key, value):
        base64_key = _encode(key)
        base64_value = _encode(value)
        txn = {
            'compare': [{
                'key': base64_key,
                'result': 'EQUAL',
                'target': 'CREATE',
                'create_revision': 0
            }],
            'success': [{
                'request_put': {
                    'key': base64_key,
                    'value': base64_value,
                }
            }],
            'failure': []
        }
        result = self.transaction(txn)
        if 'succeeded' in result:
            return result['succeeded']
        return False

Response: Atomically create the given key only if the key doesn t exist.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def put(self, key, value, lease=None):
        payload = {
            "key": _encode(key),
            "value": _encode(value)
        }
        if lease:
            payload['lease'] = lease.id
        self.post(self.get_url("/kv/put"), json=payload)
        return True

Response: Put the given key into the key - value store.


Instruction:
Make a summary of the following Python 3 code
def get(self, key, metadata=False, sort_order=None,
            sort_target=None, **kwargs):
        """Range gets the keys in the range from the key-value store.

        :param key:
        :param metadata:
        :param sort_order: 'ascend' or 'descend' or None
        :param sort_target: 'key' or 'version' or 'create' or 'mod' or 'value'
        :param kwargs:
        :return:
        """
        try:
            order = 0
            if sort_order:
                order = _SORT_ORDER.index(sort_order)
        except ValueError:
            raise ValueError('sort_order must be one of "ascend" or "descend"')

        try:
            target = 0
            if sort_target:
                target = _SORT_TARGET.index(sort_target)
        except ValueError:
            raise ValueError('sort_target must be one of "key", '
                             '"version", "create", "mod" or "value"')

        payload = {
            "key": _encode(key),
            "sort_order": order,
            "sort_target": target,
        }
        payload.update(kwargs)
        result = self.post(self.get_url("/kv/range"),
                           json=payload)
        if 'kvs' not in result:
            return []

        if metadata:
            def value_with_metadata(item):
                item['key'] = _decode(item['key'])
                value = _decode(item.pop('value'))
                return value, item

            return [value_with_metadata(item) for item in result['kvs']]
        else:
            return [_decode(item['value']) for item in result['kvs']]

Response: Get the keys in the key - value store.


Instruction:
Create a Python 3 function for
getting all keys currently stored in etcd.

Response:
def get_all(self, sort_order=None, sort_target='key'):
        """Get all keys currently stored in etcd.

        :returns: sequence of (value, metadata) tuples
        """
        return self.get(
            key=_encode(b'\0'),
            metadata=True,
            sort_order=sort_order,
            sort_target=sort_target,
            range_end=_encode(b'\0'),
        )


Instruction:
Here you have a function in Python 3, explain what it does
def get_prefix(self, key_prefix, sort_order=None, sort_target=None):
        return self.get(key_prefix,
                        metadata=True,
                        range_end=_encode(_increment_last_byte(key_prefix)),
                        sort_order=sort_order,
                        sort_target=sort_target)

Response: Get a range of keys with a prefix.


Instruction:
Can you implement a function in Python 3 that
deletes the given key - value store range.

Response:
def delete(self, key, **kwargs):
        """DeleteRange deletes the given range from the key-value store.

        A delete request increments the revision of the key-value store and
        generates a delete event in the event history for every deleted key.

        :param key:
        :param kwargs:
        :return:
        """
        payload = {
            "key": _encode(key),
        }
        payload.update(kwargs)

        result = self.post(self.get_url("/kv/deleterange"),
                           json=payload)
        if 'deleted' in result:
            return True
        return False


Instruction:
Can you tell what is the following Python 3 function doing
def delete_prefix(self, key_prefix):
        return self.delete(
            key_prefix, range_end=_encode(_increment_last_byte(key_prefix)))

Response: Delete a range of keys with a prefix in etcd.


Instruction:
Can you tell what is the following Python 3 function doing
def transaction(self, txn):
        return self.post(self.get_url("/kv/txn"),
                         data=json.dumps(txn))

Response: Processes multiple requests in a single transaction.


Instruction:
Can you tell what is the following Python 3 function doing
def watch(self, key, **kwargs):
        event_queue = queue.Queue()

        def callback(event):
            event_queue.put(event)

        w = watch.Watcher(self, key, callback, **kwargs)
        canceled = threading.Event()

        def cancel():
            canceled.set()
            event_queue.put(None)
            w.stop()

        def iterator():
            while not canceled.is_set():
                event = event_queue.get()
                if event is None:
                    canceled.set()
                if not canceled.is_set():
                    yield event

        return iterator(), cancel

Response: Watch a key for changes and return an iterator of events and cancel.


Instruction:
Can you generate the documentation for the following Python 3 function
def watch_prefix(self, key_prefix, **kwargs):
        kwargs['range_end'] = \
            _increment_last_byte(key_prefix)
        return self.watch(key_prefix, **kwargs)

Response: The same as watch but watches a range of keys with a prefix.


Instruction:
Given the following Python 3 function, write the documentation
def watch_once(self, key, timeout=None, **kwargs):
        event_queue = queue.Queue()

        def callback(event):
            event_queue.put(event)

        w = watch.Watcher(self, key, callback, **kwargs)
        try:
            return event_queue.get(timeout=timeout)
        except queue.Empty:
            raise exceptions.WatchTimedOut()
        finally:
            w.stop()

Response: Watch a key and stops after the first event.


Instruction:
How would you code a function in Python 3 to
watch a range of keys with a prefix.

Response:
def watch_prefix_once(self, key_prefix, timeout=None, **kwargs):
        """Watches a range of keys with a prefix, similar to watch_once"""
        kwargs['range_end'] = \
            _increment_last_byte(key_prefix)
        return self.watch_once(key_prefix, timeout=timeout, **kwargs)


Instruction:
Can you implement a function in Python 3 that
schedules a timer with the given callable and the interval in seconds.

Response:
def create_timer(cb: Callable[[float], None], interval: float,
                 delay_policy: TimerDelayPolicy = TimerDelayPolicy.DEFAULT,
                 loop: Optional[asyncio.BaseEventLoop] = None) -> asyncio.Task:
    '''
    Schedule a timer with the given callable and the interval in seconds.
    The interval value is also passed to the callable.
    If the callable takes longer than the timer interval, all accumulated
    callable's tasks will be cancelled when the timer is cancelled.

    Args:
        cb: TODO - fill argument descriptions

    Returns:
        You can stop the timer by cancelling the returned task.
    '''
    if not loop:
        loop = asyncio.get_event_loop()

    async def _timer():
        fired_tasks = []
        try:
            while True:
                if delay_policy == TimerDelayPolicy.CANCEL:
                    for t in fired_tasks:
                        if not t.done():
                            t.cancel()
                            await t
                    fired_tasks.clear()
                else:
                    fired_tasks[:] = [t for t in fired_tasks if not t.done()]
                t = loop.create_task(cb(interval=interval))
                fired_tasks.append(t)
                await asyncio.sleep(interval)
        except asyncio.CancelledError:
            for t in fired_tasks:
                t.cancel()
            await asyncio.gather(*fired_tasks)

    return loop.create_task(_timer())


Instruction:
How would you explain what the following Python 3 function does
def ttl(self):
        result = self.client.post(self.client.get_url("/kv/lease/timetolive"),
                                  json={"ID": self.id})
        return int(result['TTL'])

Response: LeaseTimeToLive retrieves lease information.


Instruction:
Implement a Python 3 function for
getting the keys associated with this lease.

Response:
def keys(self):
        """Get the keys associated with this lease.

        :return:
        """
        result = self.client.post(self.client.get_url("/kv/lease/timetolive"),
                                  json={"ID": self.id,
                                        "keys": True})
        keys = result['keys'] if 'keys' in result else []
        return [_decode(key) for key in keys]


Instruction:
Given the following Python 3 function, write the documentation
def apartial(coro, *args, **kwargs):
    '''
    Wraps a coroutine function with pre-defined arguments (including keyword
    arguments).  It is an asynchronous version of :func:`functools.partial`.
    '''

    @functools.wraps(coro)
    async def wrapped(*cargs, **ckwargs):
        return await coro(*args, *cargs, **kwargs, **ckwargs)

    return wrapped

Response: A coroutine function that returns a single object.


Instruction:
Can you tell what is the following Python 3 function doing
def _get(self, version, method, url_or_urls, **kwargs):
        if not url_or_urls:
            raise ValueError('%s requires a url or a list of urls given: %s' %
                             (method.title(), url_or_urls))

        # a flag we can use instead of calling isinstance() all the time
        multi = isinstance(url_or_urls, list)

        # throw an error early for too many URLs
        if multi and len(url_or_urls) > 20:
            raise ValueError('Embedly accepts only 20 urls at a time. Url '
                             'Count:%s' % len(url_or_urls))

        query = ''

        key = kwargs.get('key', self.key)

        # make sure that a key was set on the client or passed in
        if not key:
            raise ValueError('Requires a key. None given: %s' % key)

        kwargs['key'] = key

        query += urlencode(kwargs)

        if multi:
            query += '&urls=%s&' % ','.join([quote(url) for url in url_or_urls])
        else:
            query += '&url=%s' % quote(url_or_urls)

        url = 'http://api.embed.ly/%s/%s?%s' % (version, method, query)

        http = httplib2.Http(timeout=self.timeout)

        headers = {'User-Agent': self.user_agent,
                   'Connection': 'close'}

        resp, content = http.request(url, headers=headers)

        if resp['status'] == '200':
            data = json.loads(content.decode('utf-8'))

            if kwargs.get('raw', False):
                data['raw'] = content
        else:
            data = {'type': 'error',
                    'error': True,
                    'error_code': int(resp['status'])}

        if multi:
            return map(lambda url, data: Url(data, method, url),
                       url_or_urls, data)

        return Url(data, method, url_or_urls)

Response: _get makes the actual call to api. embed. ly


Instruction:
Can you tell what is the following Python 3 function doing
def python_2_unicode_compatible(klass):
    if sys.version_info[0] == 2:
        if '__str__' not in klass.__dict__:
            raise ValueError("@python_2_unicode_compatible cannot be applied "
                             "to %s because it doesn't define __str__()." %
                             klass.__name__)
        klass.__unicode__ = klass.__str__
        klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
    return klass

Response: A class decorator that can be used to make a class unicode compatible.


Instruction:
Create a Python 3 function for
encoding the given data using base - 64

Response:
def _encode(data):
    """Encode the given data using base-64

    :param data:
    :return: base-64 encoded string
    """
    if not isinstance(data, bytes_types):
        data = six.b(str(data))
    return base64.b64encode(data).decode("utf-8")


Instruction:
Given the following Python 3 function, write the documentation
def _decode(data):
    if not isinstance(data, bytes_types):
        data = six.b(str(data))
    return base64.b64decode(data.decode("utf-8"))

Response: Decode the base - 64 encoded string


Instruction:
Here you have a function in Python 3, explain what it does
def _increment_last_byte(data):
    if not isinstance(data, bytes_types):
        if isinstance(data, six.string_types):
            data = data.encode('utf-8')
        else:
            data = six.b(str(data))
    s = bytearray(data)
    s[-1] = s[-1] + 1
    return bytes(s)

Response: Get the last byte in the array and increment it


Instruction:
How would you explain what the following Python 3 function does
def metadata(self):
        self.metadata_path = os.path.join(self.path, 'metadata.rb')
        if not os.path.isfile(self.metadata_path):
            raise ValueError("Cookbook needs metadata.rb, %s"
                             % self.metadata_path)

        if not self._metadata:
            self._metadata = MetadataRb(open(self.metadata_path, 'r+'))

        return self._metadata

Response: Return dict representation of this cookbook s metadata. rb.


Instruction:
Make a summary of the following Python 3 code
def berksfile(self):
        self.berks_path = os.path.join(self.path, 'Berksfile')
        if not self._berksfile:
            if not os.path.isfile(self.berks_path):
                raise ValueError("No Berksfile found at %s"
                                 % self.berks_path)
            self._berksfile = Berksfile(open(self.berks_path, 'r+'))
        return self._berksfile

Response: Return this cookbook s Berksfile instance.


Instruction:
Make a summary of the following Python 3 code
def from_dict(cls, dictionary):
        cookbooks = set()
        # put these in order
        groups = [cookbooks]

        for key, val in dictionary.items():
            if key == 'depends':
                cookbooks.update({cls.depends_statement(cbn, meta)
                                  for cbn, meta in val.items()})

        body = ''
        for group in groups:
            if group:
                body += '\n'
            body += '\n'.join(group)
        return cls.from_string(body)

Response: Create a MetadataRb instance from a dictionary.


Instruction:
Create a Python 3 function for
returning a valid Ruby depends statement for the metadata. rb file.

Response:
def depends_statement(cookbook_name, metadata=None):
        """Return a valid Ruby 'depends' statement for the metadata.rb file."""
        line = "depends '%s'" % cookbook_name
        if metadata:
            if not isinstance(metadata, dict):
                raise TypeError("Stencil dependency options for %s "
                                "should be a dict of options, not %s."
                                % (cookbook_name, metadata))
            if metadata:
                line = "%s '%s'" % (line, "', '".join(metadata))
        return line


Instruction:
Write a Python 3 function that can
parse the metadata. rb into a dict.

Response:
def parse(self):
        """Parse the metadata.rb into a dict."""
        data = utils.ruby_lines(self.readlines())
        data = [tuple(j.strip() for j in line.split(None, 1))
                for line in data]
        depends = {}
        for line in data:
            if not len(line) == 2:
                continue
            key, value = line
            if key == 'depends':
                value = value.split(',')
                lib = utils.ruby_strip(value[0])
                detail = [utils.ruby_strip(j) for j in value[1:]]
                depends[lib] = detail
        datamap = {key: utils.ruby_strip(val) for key, val in data}
        if depends:
            datamap['depends'] = depends
        self.seek(0)
        return datamap


Instruction:
Create a Python 3 function for
merging two MetadataRbs into this one.

Response:
def merge(self, other):
        """Add requirements from 'other' metadata.rb into this one."""
        if not isinstance(other, MetadataRb):
            raise TypeError("MetadataRb to merge should be a 'MetadataRb' "
                            "instance, not %s.", type(other))
        current = self.to_dict()
        new = other.to_dict()

        # compare and gather cookbook dependencies
        meta_writelines = ['%s\n' % self.depends_statement(cbn, meta)
                           for cbn, meta in new.get('depends', {}).items()
                           if cbn not in current.get('depends', {})]

        self.write_statements(meta_writelines)
        return self.to_dict()


Instruction:
Can you implement a function in Python 3 that
parses this Berksfile into a dict.

Response:
def parse(self):
        """Parse this Berksfile into a dict."""
        self.flush()
        self.seek(0)
        data = utils.ruby_lines(self.readlines())
        data = [tuple(j.strip() for j in line.split(None, 1))
                for line in data]
        datamap = {}
        for line in data:
            if len(line) == 1:
                datamap[line[0]] = True
            elif len(line) == 2:
                key, value = line
                if key == 'cookbook':
                    datamap.setdefault('cookbook', {})
                    value = [utils.ruby_strip(v) for v in value.split(',')]
                    lib, detail = value[0], value[1:]
                    datamap['cookbook'].setdefault(lib, {})
                    # if there is additional dependency data but its
                    # not the ruby hash, its the version constraint
                    if detail and not any("".join(detail).startswith(o)
                                          for o in self.berks_options):
                        constraint, detail = detail[0], detail[1:]
                        datamap['cookbook'][lib]['constraint'] = constraint
                    if detail:
                        for deet in detail:
                            opt, val = [
                                utils.ruby_strip(i)
                                for i in deet.split(':', 1)
                            ]
                            if not any(opt == o for o in self.berks_options):
                                raise ValueError(
                                    "Cookbook detail '%s' does not specify "
                                    "one of '%s'" % (opt, self.berks_options))
                            else:
                                datamap['cookbook'][lib][opt.strip(':')] = (
                                    utils.ruby_strip(val))
                elif key == 'source':
                    datamap.setdefault(key, [])
                    datamap[key].append(utils.ruby_strip(value))
                elif key:
                    datamap[key] = utils.ruby_strip(value)
        self.seek(0)
        return datamap


Instruction:
Explain what the following Python 3 code does
def from_dict(cls, dictionary):
        cookbooks = set()
        sources = set()
        other = set()
        # put these in order
        groups = [sources, cookbooks, other]

        for key, val in dictionary.items():
            if key == 'cookbook':
                cookbooks.update({cls.cookbook_statement(cbn, meta)
                                  for cbn, meta in val.items()})
            elif key == 'source':
                sources.update({"source '%s'" % src for src in val})
            elif key == 'metadata':
                other.add('metadata')

        body = ''
        for group in groups:
            if group:
                body += '\n'
            body += '\n'.join(group)
        return cls.from_string(body)

Response: Create a Berksfile instance from a dictionary.


Instruction:
Can you write a function in Python 3 where it
returns a valid Ruby cookbook statement for the Berksfile.

Response:
def cookbook_statement(cookbook_name, metadata=None):
        """Return a valid Ruby 'cookbook' statement for the Berksfile."""
        line = "cookbook '%s'" % cookbook_name
        if metadata:
            if not isinstance(metadata, dict):
                raise TypeError("Berksfile dependency hash for %s "
                                "should be a dict of options, not %s."
                                % (cookbook_name, metadata))
            # not like the others...
            if 'constraint' in metadata:
                line += ", '%s'" % metadata.pop('constraint')
            for opt, spec in metadata.items():
                line += ", %s: '%s'" % (opt, spec)
        return line


Instruction:
Given the following Python 3 function, write the documentation
def merge(self, other):
        if not isinstance(other, Berksfile):
            raise TypeError("Berksfile to merge should be a 'Berksfile' "
                            "instance, not %s.", type(other))
        current = self.to_dict()
        new = other.to_dict()

        # compare and gather cookbook dependencies
        berks_writelines = ['%s\n' % self.cookbook_statement(cbn, meta)
                            for cbn, meta in new.get('cookbook', {}).items()
                            if cbn not in current.get('cookbook', {})]

        # compare and gather 'source' requirements
        berks_writelines.extend(["source '%s'\n" % src for src
                                 in new.get('source', [])
                                 if src not in current.get('source', [])])

        self.write_statements(berks_writelines)
        return self.to_dict()

Response: Merge two Berksfiles into this one.


Instruction:
Implement a Python 3 function for
returning a Stencil instance given a stencil name.

Response:
def get_stencil(self, stencil_name, **options):
        """Return a Stencil instance given a stencil name."""
        if stencil_name not in self.manifest.get('stencils', {}):
            raise ValueError("Stencil '%s' not declared in StencilSet "
                             "manifest." % stencil_name)
        stencil = copy.deepcopy(self.manifest)
        allstencils = stencil.pop('stencils')
        stencil.pop('default_stencil', None)
        override = allstencils[stencil_name]
        utils.deepupdate(stencil, override)

        # merge options, prefer **options (probably user-supplied)
        for opt, data in stencil.get('options', {}).items():
            if opt not in options:
                options[opt] = data.get('default', '')
        stencil['options'] = options

        name = stencil['options'].get('name')
        files = stencil['files'].copy()
        for fil, templ in files.items():
            if '<NAME>' in fil:
                # check for the option b/c there are
                # cases in which it may not exist
                if not name:
                    raise ValueError("Stencil does not include a name option")

                stencil['files'].pop(fil)
                fil = fil.replace('<NAME>', name)
                stencil['files'][fil] = templ

        return stencil


Instruction:
How would you explain what the following Python 3 function does
def _determine_selected_stencil(stencil_set, stencil_definition):
    if 'stencil' not in stencil_definition:
        selected_stencil_name = stencil_set.manifest.get('default_stencil')
    else:
        selected_stencil_name = stencil_definition.get('stencil')

    if not selected_stencil_name:
        raise ValueError("No stencil name, within stencil set %s, specified."
                         % stencil_definition['name'])

    return selected_stencil_name

Response: Determine the name of the selected stencil within a set.


Instruction:
Write a Python 3 function for
building a map of variables for this generated cookbook and stencil.

Response:
def _build_template_map(cookbook, cookbook_name, stencil):
    """Build a map of variables for this generated cookbook and stencil.

    Get template variables from stencil option values, adding the default ones
    like cookbook and cookbook year.
    """
    template_map = {
        'cookbook': {"name": cookbook_name},
        'options': stencil['options']
    }

    # Cookbooks may not yet have metadata, so we pass an empty dict if so
    try:
        template_map['cookbook'] = cookbook.metadata.to_dict().copy()
    except ValueError:
        # ValueError may be returned if this cookbook does not yet have any
        # metadata.rb written by a stencil. This is okay, as everyone should
        # be using the base stencil first, and then we'll try to call
        # cookbook.metadata again in this method later down.
        pass

    template_map['cookbook']['year'] = datetime.datetime.now().year
    return template_map


Instruction:
Explain what the following Python 3 code does
def _render_binaries(files, written_files):
    for source_path, target_path in files.items():
        needdir = os.path.dirname(target_path)
        assert needdir, "Target should have valid parent dir"
        try:
            os.makedirs(needdir)
        except OSError as err:
            if err.errno != errno.EEXIST:
                raise

        if os.path.isfile(target_path):
            if target_path in written_files:
                LOG.warning("Previous stencil has already written file %s.",
                            target_path)
            else:
                print("Skipping existing file %s" % target_path)
                LOG.info("Skipping existing file %s", target_path)
                continue

        print("Writing rendered file %s" % target_path)
        LOG.info("Writing rendered file %s", target_path)

        shutil.copy(source_path, target_path)
        if os.path.exists(target_path):
            written_files.append(target_path)

Response: Write binary contents from filetable into files.


Instruction:
How would you implement a function in Python 3 that
renders all the templates in the filetable into a list of files.

Response:
def _render_templates(files, filetable, written_files, force, open_mode='w'):
    """Write template contents from filetable into files.

    Using filetable for the rendered templates, and the list of files, render
    all the templates into actual files on disk, forcing to overwrite the file
    as appropriate, and using the given open mode for the file.
    """
    for tpl_path, content in filetable:
        target_path = files[tpl_path]
        needdir = os.path.dirname(target_path)
        assert needdir, "Target should have valid parent dir"
        try:
            os.makedirs(needdir)
        except OSError as err:
            if err.errno != errno.EEXIST:
                raise

        if os.path.isfile(target_path):
            if force:
                LOG.warning("Forcing overwrite of existing file %s.",
                            target_path)
            elif target_path in written_files:
                LOG.warning("Previous stencil has already written file %s.",
                            target_path)
            else:
                print("Skipping existing file %s" % target_path)
                LOG.info("Skipping existing file %s", target_path)
                continue

        with open(target_path, open_mode) as newfile:
            print("Writing rendered file %s" % target_path)
            LOG.info("Writing rendered file %s", target_path)
            newfile.write(content)
            written_files.append(target_path)


Instruction:
Explain what the following Python 3 code does
def build_cookbook(build_config, templatepack_path,
                   cookbooks_home, force=False):
    """Build a cookbook from a fastfood.json file.

    Can build on an existing cookbook, otherwise this will
    create a new cookbook for you based on your templatepack.
    """
    with open(build_config) as cfg:
        cfg = json.load(cfg)

    cookbook_name = cfg['name']
    template_pack = pack.TemplatePack(templatepack_path)

    written_files = []
    cookbook = create_new_cookbook(cookbook_name, cookbooks_home)

    for stencil_definition in cfg['stencils']:

        selected_stencil_set_name = stencil_definition.get('stencil_set')
        stencil_set = template_pack.load_stencil_set(selected_stencil_set_name)

        selected_stencil_name = _determine_selected_stencil(
            stencil_set,
            stencil_definition
        )

        stencil = stencil_set.get_stencil(selected_stencil_name,
                                          **stencil_definition)

        updated_cookbook = process_stencil(
            cookbook,
            cookbook_name,  # in case no metadata.rb yet
            template_pack,
            force,
            stencil_set,
            stencil,
            written_files
        )

    return written_files, updated_cookbook

Response: Build a new cookbook from a fastfood. json file.


Instruction:
Make a summary of the following Python 3 code
def process_stencil(cookbook, cookbook_name, template_pack,
                    force_argument, stencil_set, stencil, written_files):
    """Process the stencil requested, writing any missing files as needed.

    The stencil named 'stencilset_name' should be one of
    templatepack's stencils.
    """
    # force can be passed on the command line or forced in a stencil's options
    force = force_argument or stencil['options'].get('force', False)

    stencil['files'] = stencil.get('files') or {}
    files = {
        # files.keys() are template paths, files.values() are target paths
        # {path to template: rendered target path, ... }
        os.path.join(stencil_set.path, tpl): os.path.join(cookbook.path, tgt)
        for tgt, tpl in stencil['files'].items()
    }

    stencil['partials'] = stencil.get('partials') or {}
    partials = {
        # files.keys() are template paths, files.values() are target paths
        # {path to template: rendered target path, ... }
        os.path.join(stencil_set.path, tpl): os.path.join(cookbook.path, tgt)
        for tgt, tpl in stencil['partials'].items()
    }

    stencil['binaries'] = stencil.get('binaries') or {}
    binaries = {
        # files.keys() are binary paths, files.values() are target paths
        # {path to binary: rendered target path, ... }
        os.path.join(stencil_set.path, tpl): os.path.join(cookbook.path, tgt)
        for tgt, tpl in stencil['binaries'].items()
    }

    template_map = _build_template_map(cookbook, cookbook_name, stencil)

    filetable = templating.render_templates(*files.keys(), **template_map)
    _render_templates(files, filetable, written_files, force)

    parttable = templating.render_templates(*partials.keys(), **template_map)
    _render_templates(partials, parttable, written_files, force, open_mode='a')

    # no templating needed for binaries, just pass off to the copy method
    _render_binaries(binaries, written_files)

    # merge metadata.rb dependencies
    stencil_metadata_deps = {'depends': stencil.get('dependencies', {})}
    stencil_metadata = book.MetadataRb.from_dict(stencil_metadata_deps)
    cookbook.metadata.merge(stencil_metadata)

    # merge Berksfile dependencies
    stencil_berks_deps = {'cookbook': stencil.get('berks_dependencies', {})}
    stencil_berks = book.Berksfile.from_dict(stencil_berks_deps)
    cookbook.berksfile.merge(stencil_berks)

    return cookbook

Response: Processes the stencil requested writing any missing files as needed.


Instruction:
Create a Python 3 function to
create a new cookbook.

Response:
def create_new_cookbook(cookbook_name, cookbooks_home):
    """Create a new cookbook.

    :param cookbook_name: Name of the new cookbook.
    :param cookbooks_home: Target dir for new cookbook.
    """
    cookbooks_home = utils.normalize_path(cookbooks_home)

    if not os.path.exists(cookbooks_home):
        raise ValueError("Target cookbook dir %s does not exist."
                         % os.path.relpath(cookbooks_home))

    target_dir = os.path.join(cookbooks_home, cookbook_name)
    LOG.debug("Creating dir -> %s", target_dir)
    try:
        os.makedirs(target_dir)
    except OSError as err:
        if err.errno != errno.EEXIST:
            raise
        else:
            LOG.info("Skipping existing directory %s", target_dir)

    cookbook_path = os.path.join(cookbooks_home, cookbook_name)
    cookbook = book.CookBook(cookbook_path)

    return cookbook


Instruction:
Create a Python 3 function to
tidy up lines from a file honor comment lines.

Response:
def ruby_lines(text):
    """Tidy up lines from a file, honor # comments.

    Does not honor ruby block comments (yet).
    """
    if isinstance(text, basestring):
        text = text.splitlines()
    elif not isinstance(text, list):
        raise TypeError("text should be a list or a string, not %s"
                        % type(text))
    return [l.strip() for l in text if l.strip() and not
            l.strip().startswith('#')]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def deepupdate(original, update, levels=5):
    if not isinstance(update, dict):
        update = dict(update)
    if not levels > 0:
        original.update(update)
    else:
        for key, val in update.items():
            if isinstance(original.get(key), dict):
                # might need a force=True to override this
                if not isinstance(val, dict):
                    raise TypeError("Trying to update dict %s with "
                                    "non-dict %s" % (original[key], val))
                deepupdate(original[key], val, levels=levels-1)
            else:
                original.update({key: val})

Response: Update original dict with items from iterable update.


Instruction:
Explain what the following Python 3 code does
def write_statements(self, statements):
        self.seek(0)
        original_content_lines = self.readlines()
        new_content_lines = copy.copy(original_content_lines)
        # ignore blanks and sort statements to be written
        statements = sorted([stmnt for stmnt in statements if stmnt])

        # find all the insert points for each statement
        uniqs = {stmnt.split(None, 1)[0] for stmnt in statements}
        insert_locations = {}
        for line in reversed(original_content_lines):
            if not uniqs:
                break
            if not line:
                continue
            for word in uniqs.copy():
                if line.startswith(word):
                    index = original_content_lines.index(line) + 1
                    insert_locations[word] = index
                    uniqs.remove(word)

        for statement in statements:
            print("writing to %s : %s" % (self, statement))
            startswith = statement.split(None, 1)[0]
            # insert new statement with similar OR at the end of the file
            new_content_lines.insert(
                insert_locations.get(startswith, len(new_content_lines)),
                statement)

        if new_content_lines != original_content_lines:
            self.seek(0)
            self.writelines(new_content_lines)
            self.flush()

Response: Insert the statements into the file neatly.


Instruction:
Here you have a function in Python 3, explain what it does
def _fastfood_build(args):
    written_files, cookbook = food.build_cookbook(
        args.config_file, args.template_pack,
        args.cookbooks, args.force)

    if len(written_files) > 0:
        print("%s: %s files written" % (cookbook,
                                        len(written_files)))
    else:
        print("%s up to date" % cookbook)

    return written_files, cookbook

Response: Run on fastfood build.


Instruction:
Write a Python 3 script for
running on fastfood list.

Response:
def _fastfood_list(args):
    """Run on `fastfood list`."""
    template_pack = pack.TemplatePack(args.template_pack)
    if args.stencil_set:
        stencil_set = template_pack.load_stencil_set(args.stencil_set)
        print("Available Stencils for %s:" % args.stencil_set)
        for stencil in stencil_set.stencils:
            print("  %s" % stencil)
    else:
        print('Available Stencil Sets:')
        for name, vals in template_pack.stencil_sets.items():
            print("  %12s - %12s" % (name, vals['help']))


Instruction:
Can you write a function in Python 3 where it
runs on fastfood show.

Response:
def _fastfood_show(args):
    """Run on `fastfood show`."""
    template_pack = pack.TemplatePack(args.template_pack)
    if args.stencil_set:
        stencil_set = template_pack.load_stencil_set(args.stencil_set)
        print("Stencil Set %s:" % args.stencil_set)
        print('  Stencils:')
        for stencil in stencil_set.stencils:
            print("    %s" % stencil)
        print('  Options:')
        for opt, vals in stencil_set.manifest['options'].items():
            print("    %s - %s" % (opt, vals['help']))


Instruction:
Can you tell what is the following Python 3 function doing
def _release_info():
    pypi_url = 'http://pypi.python.org/pypi/fastfood/json'
    headers = {
        'Accept': 'application/json',
    }
    request = urllib.Request(pypi_url, headers=headers)
    response = urllib.urlopen(request).read().decode('utf_8')
    data = json.loads(response)
    return data

Response: Check latest fastfood release info from PyPI.


Instruction:
Here you have a function in Python 3, explain what it does
def getenv(option_name, default=None):
    env = "%s_%s" % (NAMESPACE.upper(), option_name.upper())
    return os.environ.get(env, default)

Response: Return the option from the environment in the FASTFOOD namespace.


Instruction:
Given the following Python 3 function, write the documentation
def main(argv=None):
    # pylint: disable=missing-docstring
    import argparse
    import traceback

    class HelpfulParser(argparse.ArgumentParser):
        def error(self, message, print_help=False):
            if 'too few arguments' in message:
                sys.argv.insert(0, os.path.basename(sys.argv.pop(0)))
                message = ("%s. Try getting help with `%s -h`"
                           % (message, " ".join(sys.argv)))
            if print_help:
                self.print_help()
            sys.stderr.write('\nerror: %s\n' % message)
            sys.exit(2)

    parser = HelpfulParser(
        prog=NAMESPACE,
        description=__doc__.splitlines()[0],
        epilog="\n".join(__doc__.splitlines()[1:]),
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        )

    version_string = 'version %s' % fastfood.__version__
    parser.description = '%s ( %s )' % (parser.description, version_string)

    # version_group = subparsers.add_group()
    version_group = parser.add_argument_group(
        title='version info',
        description='Use these arguments to get version info.')

    vers_arg = version_group.add_argument(
        '-V', '--version', action='version',
        help="Return the current fastfood version.",
        version='%s %s' % (parser.prog, version_string))

    class LatestVersionAction(vers_arg.__class__):
        def __call__(self, prsr, *args, **kw):
            info = _release_info()
            vers = info['info']['version']
            release = info['releases'][vers][0]
            uploaded = datetime.strptime(
                release['upload_time'], '%Y-%m-%dT%H:%M:%S')
            sym = EXCLAIM if vers != fastfood.__version__ else CHECK
            message = u"{}  fastfood version {} uploaded {}\n"
            message = message.format(sym, vers, uploaded.ctime())
            prsr.exit(message=message)

    version_group.add_argument(
        '-L', '--latest', action=LatestVersionAction,
        help="Lookup the latest relase from PyPI.")

    verbose = parser.add_mutually_exclusive_group()
    verbose.add_argument('-v', dest='loglevel', action='store_const',
                         const=logging.INFO,
                         help="Set log-level to INFO.")
    verbose.add_argument('-vv', dest='loglevel', action='store_const',
                         const=logging.DEBUG,
                         help="Set log-level to DEBUG.")
    parser.set_defaults(loglevel=logging.WARNING)
    home = os.getenv('HOME') or os.path.expanduser('~') or os.getcwd()
    parser.add_argument(
        '--template-pack', help='template pack location',
        default=getenv(
            'template_pack', os.path.join(home, '.fastfood')))
    parser.add_argument(
        '--cookbooks', help='cookbooks directory',
        default=getenv(
            'cookbooks', os.path.join(home, 'cookbooks')))

    subparsers = parser.add_subparsers(
        dest='_subparsers', title='fastfood commands',
        description='operations...',
        help='...')

    #
    # `fastfood list`
    #
    list_parser = subparsers.add_parser(
        'list', help='List available stencils',
        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    list_parser.add_argument('stencil_set', nargs='?',
                             help="Stencil set to list stencils from")
    list_parser.set_defaults(func=_fastfood_list)

    #
    # `fastfood show <stencil_set>`
    #
    show_parser = subparsers.add_parser(
        'show', help='Show stencil set information',
        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    show_parser.add_argument('stencil_set',
                             help="Stencil set to list stencils from")
    show_parser.set_defaults(func=_fastfood_show)

    #
    # `fastfood build`
    #
    build_parser = subparsers.add_parser(
        'build', help='Create or update a cookbook using a config',
        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    build_parser.add_argument('config_file',
                              help="JSON config file")
    build_parser.add_argument('--force', '-f', action='store_true',
                              default=False, help="Overwrite existing files.")

    build_parser.set_defaults(func=_fastfood_build)

    setattr(_LOCAL, 'argparser', parser)
    if not argv:
        argv = None
    args = parser.parse_args(args=argv)
    if hasattr(args, 'options'):
        args.options = {k: v for k, v in args.options}

    logging.basicConfig(level=args.loglevel)

    try:
        args.func(args)
    except exc.FastfoodError as err:
        title = exc.get_friendly_title(err)
        print('%s  %s: %s' % (RED_X, title, str(err)),
              file=sys.stderr)
        sys.stderr.flush()
        sys.exit(1)
    except Exception as err:
        print('%s  Unexpected error. Please report this traceback.'
              % INTERROBANG,
              file=sys.stderr)
        traceback.print_exc()
        # todo: tracack in -v or -vv mode?
        sys.stderr.flush()
        sys.exit(1)
    except KeyboardInterrupt:
        sys.exit("\nStahp")

Response: This is the fastfood command line interface.


Instruction:
How would you implement a function in Python 3 that
turns class instance or name into an eyeball - friendly title.

Response:
def get_friendly_title(err):
    """Turn class, instance, or name (str) into an eyeball-friendly title.

    E.g. FastfoodStencilSetNotListed --> 'Stencil Set Not Listed'
    """
    if isinstance(err, basestring):
        string = err
    else:
        try:
            string = err.__name__
        except AttributeError:
            string = err.__class__.__name__
    split = _SPLITCASE_RE.findall(string)
    if not split:
        split.append(string)
    if len(split) > 1 and split[0] == 'Fastfood':
        split.pop(0)

    return " ".join(split)


Instruction:
Here you have a function in Python 3, explain what it does
def _validate(self, key, cls=None):
        if key not in self.manifest:
            raise ValueError("Manifest %s requires '%s'."
                             % (self.manifest_path, key))
        if cls:
            if not isinstance(self.manifest[key], cls):
                raise TypeError("Manifest value '%s' should be %s, not %s"
                                % (key, cls, type(self.manifest[key])))

Response: Verify the manifest schema.


Instruction:
Can you write a function in Python 3 where it
lists of stencil sets.

Response:
def stencil_sets(self):
        """List of stencil sets."""
        if not self._stencil_sets:
            self._stencil_sets = self.manifest['stencil_sets']
        return self._stencil_sets


Instruction:
Can you write a function in Python 3 where it
loads a Stencil Set from this template pack.

Response:
def load_stencil_set(self, stencilset_name):
        """Return the Stencil Set from this template pack."""
        if stencilset_name not in self._stencil_sets:
            if stencilset_name not in self.manifest['stencil_sets'].keys():
                raise exc.FastfoodStencilSetNotListed(
                    "Stencil set '%s' not listed in %s under stencil_sets."
                    % (stencilset_name, self.manifest_path))
            stencil_path = os.path.join(
                self.path, 'stencils', stencilset_name)
            self._stencil_sets[stencilset_name] = (
                stencil_module.StencilSet(stencil_path))
        return self._stencil_sets[stencilset_name]


Instruction:
Given the following Python 3 function, write the documentation
def qstring(option):
    if (re.match(NODE_ATTR_RE, option) is None and
            re.match(CHEF_CONST_RE, option) is None):
        return "'%s'" % option
    else:
        return option

Response: Custom quoting method for jinja.


Instruction:
Can you generate the documentation for the following Python 3 function
def render_templates_generator(*files, **template_map):
    for path in files:
        if not os.path.isfile(path):
            raise ValueError("Template file %s not found"
                             % os.path.relpath(path))
        else:
            try:
                with codecs.open(path, encoding='utf-8') as f:
                    text = f.read()
                template = JINJA_ENV.from_string(text)
            except jinja2.TemplateSyntaxError as err:
                msg = ("Error rendering jinja2 template for file %s "
                       "on line %s. Error: %s"
                       % (path, err.lineno, err.message))
                raise type(err)(
                    msg, err.lineno, filename=os.path.basename(path))

            result = template.render(**template_map)
            if not result.endswith('\n'):
                result += '\n'
            yield path, result

Response: Yields the contents of the given file and returns a generator that yields the contents of the file.


Instruction:
Can you tell what is the following Python 3 function doing
def generate(self, model, outfolder):
        _logger.info('Generating code to {!r}.'.format(outfolder))

        for task in self.tasks:
            for element in task.filtered_elements(model):
                task.run(element, outfolder)

Response: Generate code for given model.


Instruction:
How would you explain what the following Python 3 function does
def run(self, element, outfolder):
        filepath = self.relative_path_for_element(element)
        if outfolder and not os.path.isabs(filepath):
            filepath = os.path.join(outfolder, filepath)

        _logger.debug('{!r} --> {!r}'.format(element, filepath))

        self.ensure_folder(filepath)
        self.generate_file(element, filepath)

Response: Apply this task to model element.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def create_template_context(self, element, **kwargs):
        context = dict(element=element, **kwargs)
        if self.global_context:
            context.update(**self.global_context)
        return context

Response: Create a dictionary that can be used to create a template context.


Instruction:
How would you explain what the following Python 3 function does
def _validated(self, value):
        if value is None:
            return None
        if isinstance(value, bson.ObjectId):
            return value
        try:
            return bson.ObjectId(value)
        except (ValueError, AttributeError):
            self.fail('invalid_object_id')

Response: Format the value or raise a : exc : ValidationException.


Instruction:
How would you explain what the following Python 3 function does
def create_environment(self, **kwargs):
        return jinja2.Environment(
            loader=jinja2.FileSystemLoader(self.templates_path),
            **kwargs
        )

Response: Create a Jinja2 Environment object.


Instruction:
Can you create a Python 3 function that
returns array of pairs of adjacent elements in a.

Response:
def pairs(a):
    """Return array of pairs of adjacent elements in a.

    >>> pairs([1, 2, 3, 4])
    array([[1, 2],
           [2, 3],
           [3, 4]])

    """
    a = np.asarray(a)
    return as_strided(a, shape=(a.size - 1, 2), strides=a.strides * 2)


Instruction:
How would you explain what the following Python 3 function does
def transcript_sort_key(transcript):
    return (
        -len(transcript.protein_sequence),
        -len(transcript.sequence),
        transcript.name
    )

Response: Returns the key used to sort transcripts in a sequence.


Instruction:
Make a summary of the following Python 3 code
def best_transcript(transcripts):
    assert len(transcripts) > 0
    sorted_list = sorted(transcripts, key=transcript_sort_key)
    return sorted_list[0]

Response: Given a set of coding transcripts choose the best transcript


Instruction:
Explain what the following Python 3 code does
def predict_epitopes_from_args(args):
    mhc_model = mhc_binding_predictor_from_args(args)
    variants = variant_collection_from_args(args)
    gene_expression_dict = rna_gene_expression_dict_from_args(args)
    transcript_expression_dict = rna_transcript_expression_dict_from_args(args)

    predictor = TopiaryPredictor(
        mhc_model=mhc_model,
        padding_around_mutation=args.padding_around_mutation,
        ic50_cutoff=args.ic50_cutoff,
        percentile_cutoff=args.percentile_cutoff,
        min_transcript_expression=args.rna_min_transcript_expression,
        min_gene_expression=args.rna_min_gene_expression,
        only_novel_epitopes=args.only_novel_epitopes,
        raise_on_error=not args.skip_variant_errors)
    return predictor.predict_from_variants(
        variants=variants,
        transcript_expression_dict=transcript_expression_dict,
        gene_expression_dict=gene_expression_dict)

Response: Predicts epitopes from the given commandline arguments.


Instruction:
Explain what the following Python 3 code does
def get_random_word(dictionary, min_word_length=3, max_word_length=8):

    while True:
        # Choose a random word
        word = choice(dictionary)
        # Stop looping as soon as we have a valid candidate
        if len(word) >= min_word_length and len(word) <= max_word_length:
            break

    return word

Response: Returns a random word from the dictionary


Instruction:
Can you create a Python 3 function that
generates a random password from the ISO.

Response:
def pw(min_word_length=3, max_word_length=8, max_int_value=1000, number_of_elements=4, no_special_characters=False):
    """
        Generate a password
    """

    # Set the position of the integer
    int_position = set_int_position(number_of_elements)

    # Load dictionary
    dictionary = load_dictionary()

    password = ''
    for i in range(number_of_elements):
        # Add word or integer
        if i == int_position:
            password += str(get_random_int(max_int_value))
        else:
            password += get_random_word(dictionary,
                                        min_word_length,
                                        max_word_length).title()

        # Add separator
        if i != number_of_elements - 1:
            password += get_random_separator(no_special_characters)

    return password


Instruction:
Can you tell what is the following Python 3 function doing
def protein_subsequences_around_mutations(effects, padding_around_mutation):
    protein_subsequences = {}
    protein_subsequence_start_offsets = {}
    for effect in effects:
        protein_sequence = effect.mutant_protein_sequence
        # some effects will lack a mutant protein sequence since
        # they are either silent or unpredictable
        if protein_sequence:
            mutation_start = effect.aa_mutation_start_offset
            mutation_end = effect.aa_mutation_end_offset
            seq_start_offset = max(
                0,
                mutation_start - padding_around_mutation)
            # some pseudogenes have stop codons in the reference sequence,
            # if we try to use them for epitope prediction we should trim
            # the sequence to not include the stop character '*'
            first_stop_codon_index = protein_sequence.find("*")
            if first_stop_codon_index < 0:
                first_stop_codon_index = len(protein_sequence)

            seq_end_offset = min(
                first_stop_codon_index,
                mutation_end + padding_around_mutation)
            subsequence = protein_sequence[seq_start_offset:seq_end_offset]
            protein_subsequences[effect] = subsequence
            protein_subsequence_start_offsets[effect] = seq_start_offset
    return protein_subsequences, protein_subsequence_start_offsets

Response: Given a list of effects get a dictionary of subsequences and a dictionary of subsequence start offsets.


Instruction:
Can you tell what is the following Python 3 function doing
def check_padding_around_mutation(given_padding, epitope_lengths):
    min_required_padding = max(epitope_lengths) - 1
    if not given_padding:
        return min_required_padding
    else:
        require_integer(given_padding, "Padding around mutation")
        if given_padding < min_required_padding:
            raise ValueError(
                "Padding around mutation %d cannot be less than %d "
                "for epitope lengths %s" % (
                    given_padding,
                    min_required_padding,
                    epitope_lengths))
        return given_padding

Response: Checks that the given padding is less than the minimum required padding around the mutation.


Instruction:
Write a Python 3 script for
returning the interval of the mutation in the peptide.

Response:
def peptide_mutation_interval(
        peptide_start_in_protein,
        peptide_length,
        mutation_start_in_protein,
        mutation_end_in_protein):
    """
    Half-open interval of mutated residues in the peptide, determined
    from the mutation interval in the original protein sequence.

    Parameters
    ----------
    peptide_start_in_protein : int
        Position of the first peptide residue within the protein
        (starting from 0)

    peptide_length : int

    mutation_start_in_protein : int
        Position of the first mutated residue starting from 0. In the case of a
        deletion, the position where the first residue had been.

    mutation_end_in_protein : int
        Position of the last mutated residue in the mutant protein. In the case
        of a deletion, this is equal to the mutation_start_in_protein.
    )
    """
    if peptide_start_in_protein > mutation_end_in_protein:
        raise ValueError("Peptide starts after mutation")
    elif peptide_start_in_protein + peptide_length < mutation_start_in_protein:
        raise ValueError("Peptide ends before mutation")

    # need a half-open start/end interval
    peptide_mutation_start_offset = min(
        peptide_length,
        max(0, mutation_start_in_protein - peptide_start_in_protein))
    peptide_mutation_end_offset = min(
        peptide_length,
        max(0, mutation_end_in_protein - peptide_start_in_protein))
    return (peptide_mutation_start_offset, peptide_mutation_end_offset)


Instruction:
Implement a function in Python 3 to
create an Indicator object from a dictionary.

Response:
def from_dict(cls, indicator):
        """
        Create an indicator object from a dictionary.

        :param indicator: The dictionary.
        :return: The indicator object.
        """

        tags = indicator.get('tags')
        if tags is not None:
            tags = [Tag.from_dict(tag) for tag in tags]

        return Indicator(value=indicator.get('value'),
                         type=indicator.get('indicatorType'),
                         priority_level=indicator.get('priorityLevel'),
                         correlation_count=indicator.get('correlationCount'),
                         whitelisted=indicator.get('whitelisted'),
                         weight=indicator.get('weight'),
                         reason=indicator.get('reason'),
                         first_seen=indicator.get('firstSeen'),
                         last_seen=indicator.get('lastSeen'),
                         sightings=indicator.get('sightings'),
                         source=indicator.get('source'),
                         notes=indicator.get('notes'),
                         tags=tags,
                         enclave_ids=indicator.get('enclaveIds'))


Instruction:
Create a Python 3 function to
return a dictionary representation of the object.

Response:
def to_dict(self, remove_nones=False):
        """
        Creates a dictionary representation of the indicator.

        :param remove_nones: Whether ``None`` values should be filtered out of the dictionary.  Defaults to ``False``.
        :return: A dictionary representation of the indicator.
        """

        if remove_nones:
            return super().to_dict(remove_nones=True)

        tags = None
        if self.tags is not None:
            tags = [tag.to_dict(remove_nones=remove_nones) for tag in self.tags]

        return {
            'value': self.value,
            'indicatorType': self.type,
            'priorityLevel': self.priority_level,
            'correlationCount': self.correlation_count,
            'whitelisted': self.whitelisted,
            'weight': self.weight,
            'reason': self.reason,
            'firstSeen': self.first_seen,
            'lastSeen': self.last_seen,
            'source': self.source,
            'notes': self.notes,
            'tags': tags,
            'enclaveIds': self.enclave_ids
        }


Instruction:
Can you tell what is the following Python 3 function doing
def filter_false_positive(df, process_time):
    result = []
    track = []
    count = 0
    for o in df['alerts']:
        count += 1
        if 'closedState' in o:
            if o['closedState'] != 'False Positive':
                if 'distinguishers' in o:
                    try:
                        if 'virus' in o['distinguishers']:
                            if o['distinguishers']['virus'] != 'fetestevent':
                                result.append(o)
                            else:
                                track.append(o)  # Track fetestevents that are skipped
                        else:
                            result.append(o)
                    except TypeError:
                        result.append(o)
                else:
                    result.append(o)
            else:
                track.append(o)  # Track false positives that are skipped
        elif 'distinguishers' in o:
            try:
                if 'virus' in o['distinguishers']:
                    if o['distinguishers']['virus'] != 'fetestevent':
                        result.append(o)
                    else:
                        track.append(o)  # Track fetestevents that are skipped
                else:
                    result.append(o)
            except TypeError:
                result.append(o)

    trackfile = open('tracking_fetest_' + process_time + '.txt', 'w')
    numskip = 1
    for item in track:
        trackfile.write("\n\n**** {:d}: Display ID {} ****\n\n{}".format(numskip, item['displayId'], item))
        numskip += 1
    return result

Response: method that takes in FireEye ( FE ) alerts and filters FE - tests and False Positives


Instruction:
Here you have a function in Python 3, explain what it does
def process_alert(file_name):

    processed_line = open(file_name, 'r').read()
    char_pos = processed_line.find("}")
    new_line = "{" + processed_line[char_pos + 2:]
    return new_line

Response: A function that removes the alerts property from the FireEye alert and transforms the data into a JSON ready format


Instruction:
Here you have a function in Python 3, explain what it does
def apply_filter(
        filter_fn,
        collection,
        result_fn=None,
        filter_name="",
        collection_name=""):
    """
    Apply filter to effect collection and print number of dropped elements

    Parameters
    ----------
    """
    n_before = len(collection)
    filtered = [x for x in collection if filter_fn(x)]
    n_after = len(filtered)
    if not collection_name:
        collection_name = collection.__class__.__name__
    logging.info(
        "%s filtering removed %d/%d entries of %s",
        filter_name,
        (n_before - n_after),
        n_before,
        collection_name)
    return result_fn(filtered) if result_fn else collection.__class__(filtered)

Response: Apply filter to effect collection and print number of dropped elements


Instruction:
Here you have a function in Python 3, explain what it does
def filter_silent_and_noncoding_effects(effects):
    return apply_filter(
        filter_fn=lambda effect: isinstance(effect, NonsilentCodingMutation),
        collection=effects,
        result_fn=effects.clone_with_new_elements,
        filter_name="Silent mutation")

Response: Filter out non - coding variants which result in modified proteins.


Instruction:
Make a summary of the following Python 3 code
def apply_variant_expression_filters(
        variants,
        gene_expression_dict,
        gene_expression_threshold,
        transcript_expression_dict,
        transcript_expression_threshold):
    """
    Filter a collection of variants by gene and transcript expression thresholds

    Parameters
    ----------
    variants : varcode.VariantCollection

    gene_expression_dict : dict

    gene_expression_threshold : float

    transcript_expression_dict : dict

    transcript_expression_threshold : float
    """
    if gene_expression_dict:
        variants = apply_filter(
            lambda variant: any(
                gene_expression_dict.get(gene_id, 0.0) >=
                gene_expression_threshold
                for gene_id in variant.gene_ids
            ),
            variants,
            result_fn=variants.clone_with_new_elements,
            filter_name="Variant gene expression (min=%0.4f)" % gene_expression_threshold)
    if transcript_expression_dict:
        variants = apply_filter(
            lambda variant: any(
                transcript_expression_dict.get(transcript_id, 0.0) >=
                transcript_expression_threshold
                for transcript_id in variant.transcript_ids
            ),
            variants,
            result_fn=variants.clone_with_new_elements,
            filter_name=(
                "Variant transcript expression (min=%0.4f)" % (
                    transcript_expression_threshold,)))
    return variants

Response: Filter a collection of variants by gene and transcript expression thresholds


Instruction:
How would you explain what the following Python 3 function does
def apply_effect_expression_filters(
        effects,
        gene_expression_dict,
        gene_expression_threshold,
        transcript_expression_dict,
        transcript_expression_threshold):
    """
    Filter collection of varcode effects by given gene
    and transcript expression thresholds.

    Parameters
    ----------
    effects : varcode.EffectCollection

    gene_expression_dict : dict

    gene_expression_threshold : float

    transcript_expression_dict : dict

    transcript_expression_threshold : float
    """
    if gene_expression_dict:
        effects = apply_filter(
            lambda effect: (
                gene_expression_dict.get(effect.gene_id, 0.0) >=
                gene_expression_threshold),
            effects,
            result_fn=effects.clone_with_new_elements,
            filter_name="Effect gene expression (min = %0.4f)" % gene_expression_threshold)

    if transcript_expression_dict:
        effects = apply_filter(
            lambda effect: (
                transcript_expression_dict.get(effect.transcript_id, 0.0) >=
                transcript_expression_threshold
            ),
            effects,
            result_fn=effects.clone_with_new_elements,
            filter_name=(
                "Effect transcript expression (min=%0.4f)" % (
                    transcript_expression_threshold,)))
    return effects

Response: Filter collection of varcode effects by given gene and transcript expression thresholds.


Instruction:
Make a summary of the following Python 3 code
def submit_indicators(self, indicators, enclave_ids=None, tags=None):

        if enclave_ids is None:
            enclave_ids = self.enclave_ids

        if tags is not None:
            tags = [tag.to_dict() for tag in tags]

        body = {
            "enclaveIds": enclave_ids,
            "content": [indicator.to_dict() for indicator in indicators],
            "tags": tags
        }
        self._client.post("indicators", data=json.dumps(body))

Response: Submit indicators to the specified enclave.


Instruction:
Here you have a function in Python 3, explain what it does
def get_indicators(self, from_time=None, to_time=None, enclave_ids=None,
                       included_tag_ids=None, excluded_tag_ids=None,
                       start_page=0, page_size=None):
        """
        Creates a generator from the |get_indicators_page| method that returns each successive indicator as an
        |Indicator| object containing values for the 'value' and 'type' attributes only; all
        other |Indicator| object attributes will contain Null values.

        :param int from_time: start of time window in milliseconds since epoch (defaults to 7 days ago).
        :param int to_time: end of time window in milliseconds since epoch (defaults to current time).
        :param list(string) enclave_ids: a list of enclave IDs from which to get indicators from.
        :param list(string) included_tag_ids: only indicators containing ALL of these tag GUIDs will be returned.
        :param list(string) excluded_tag_ids: only indicators containing NONE of these tags GUIDs be returned.
        :param int start_page: see 'page_size' explanation.
        :param int page_size: Passing the integer 1000 as the argument to this parameter should result in your script
        making fewer API calls because it returns the largest quantity of indicators with each API call.  An API call
        has to be made to fetch each |Page|.
        :return: A generator of |Indicator| objects containing values for the "value" and "type" attributes only.
        All other attributes of the |Indicator| object will contain Null values.

        """
        indicators_page_generator = self._get_indicators_page_generator(
            from_time=from_time,
            to_time=to_time,
            enclave_ids=enclave_ids,
            included_tag_ids=included_tag_ids,
            excluded_tag_ids=excluded_tag_ids,
            page_number=start_page,
            page_size=page_size
        )

        indicators_generator = Page.get_generator(page_generator=indicators_page_generator)

        return indicators_generator

Response: Returns a generator that returns each successive indicator for the given time range.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _get_indicators_page_generator(self, from_time=None, to_time=None, page_number=0, page_size=None,
                                       enclave_ids=None, included_tag_ids=None, excluded_tag_ids=None):
        """
        Creates a generator from the |get_indicators_page| method that returns each successive page.

        :param int from_time: start of time window in milliseconds since epoch (defaults to 7 days ago)
        :param int to_time: end of time window in milliseconds since epoch (defaults to current time)
        :param int page_number: the page number
        :param int page_size: the page size
        :param list(string) enclave_ids: a list of enclave IDs to filter by
        :param list(string) included_tag_ids: only indicators containing ALL of these tags will be returned
        :param list(string) excluded_tag_ids: only indicators containing NONE of these tags will be returned
        :return: a |Page| of |Indicator| objects
        """

        get_page = functools.partial(
            self.get_indicators_page,
            from_time=from_time,
            to_time=to_time,
            page_number=page_number,
            page_size=page_size,
            enclave_ids=enclave_ids,
            included_tag_ids=included_tag_ids,
            excluded_tag_ids=excluded_tag_ids
        )
        return Page.get_page_generator(get_page, page_number, page_size)

Response: Creates a generator from the get_indicators_page method that returns each successive page.


Instruction:
Can you implement a function in Python 3 that
gets a page of indicators matching the provided filters.

Response:
def get_indicators_page(self, from_time=None, to_time=None, page_number=None, page_size=None,
                            enclave_ids=None, included_tag_ids=None, excluded_tag_ids=None):
        """
        Get a page of indicators matching the provided filters.

        :param int from_time: start of time window in milliseconds since epoch (defaults to 7 days ago)
        :param int to_time: end of time window in milliseconds since epoch (defaults to current time)
        :param int page_number: the page number
        :param int page_size: the page size
        :param list(string) enclave_ids: a list of enclave IDs to filter by
        :param list(string) included_tag_ids: only indicators containing ALL of these tags will be returned
        :param list(string) excluded_tag_ids: only indicators containing NONE of these tags will be returned
        :return: a |Page| of indicators
        """

        params = {
            'from': from_time,
            'to': to_time,
            'pageSize': page_size,
            'pageNumber': page_number,
            'enclaveIds': enclave_ids,
            'tagIds': included_tag_ids,
            'excludedTagIds': excluded_tag_ids
        }

        resp = self._client.get("indicators", params=params)

        page_of_indicators = Page.from_dict(resp.json(), content_type=Indicator)

        return page_of_indicators


Instruction:
Write a Python 3 script for
searching for indicators in the specified enclave.

Response:
def search_indicators(self, search_term=None,
                          enclave_ids=None,
                          from_time=None,
                          to_time=None,
                          indicator_types=None,
                          tags=None,
                          excluded_tags=None):
        """
        Uses the |search_indicators_page| method to create a generator that returns each successive indicator.

        :param str search_term: The term to search for.  If empty, no search term will be applied.  Otherwise, must
            be at least 3 characters.
        :param list(str) enclave_ids: list of enclave ids used to restrict indicators to specific enclaves (optional - by
            default indicators from all of user's enclaves are returned)
        :param int from_time: start of time window in milliseconds since epoch (optional)
        :param int to_time: end of time window in milliseconds since epoch (optional)
        :param list(str) indicator_types: a list of indicator types to filter by (optional)
        :param list(str) tags: Name (or list of names) of tag(s) to filter indicators by.  Only indicators containing
            ALL of these tags will be returned. (optional)
        :param list(str) excluded_tags: Indicators containing ANY of these tags will be excluded from the results.
        :return: The generator.
        """

        return Page.get_generator(page_generator=self._search_indicators_page_generator(search_term, enclave_ids,
                                                                                from_time, to_time,
                                                                                indicator_types, tags,
                                                                                excluded_tags))


Instruction:
Here you have a function in Python 3, explain what it does
def _search_indicators_page_generator(self, search_term=None,
                                          enclave_ids=None,
                                          from_time=None,
                                          to_time=None,
                                          indicator_types=None,
                                          tags=None,
                                          excluded_tags=None,
                                          start_page=0,
                                          page_size=None):
        """
        Creates a generator from the |search_indicators_page| method that returns each successive page.

        :param str search_term: The term to search for.  If empty, no search term will be applied.  Otherwise, must
            be at least 3 characters.
        :param list(str) enclave_ids: list of enclave ids used to restrict indicators to specific enclaves (optional - by
            default indicators from all of user's enclaves are returned)
        :param int from_time: start of time window in milliseconds since epoch (optional)
        :param int to_time: end of time window in milliseconds since epoch (optional)
        :param list(str) indicator_types: a list of indicator types to filter by (optional)
        :param list(str) tags: Name (or list of names) of tag(s) to filter indicators by.  Only indicators containing
            ALL of these tags will be returned. (optional)
        :param list(str) excluded_tags: Indicators containing ANY of these tags will be excluded from the results.
        :param int start_page: The page to start on.
        :param page_size: The size of each page.
        :return: The generator.
        """

        get_page = functools.partial(self.search_indicators_page, search_term, enclave_ids,
                                     from_time, to_time, indicator_types, tags, excluded_tags)
        return Page.get_page_generator(get_page, start_page, page_size)

Response: Creates a generator from the |search_indicators_page| method that returns each successive page.


Instruction:
Given the following Python 3 function, write the documentation
def search_indicators_page(self, search_term=None,
                               enclave_ids=None,
                               from_time=None,
                               to_time=None,
                               indicator_types=None,
                               tags=None,
                               excluded_tags=None,
                               page_size=None,
                               page_number=None):
        """
        Search for indicators containing a search term.

        :param str search_term: The term to search for.  If empty, no search term will be applied.  Otherwise, must
            be at least 3 characters.
        :param list(str) enclave_ids: list of enclave ids used to restrict to indicators found in reports in specific
            enclaves (optional - by default reports from all of the user's enclaves are used)
        :param int from_time: start of time window in milliseconds since epoch (optional)
        :param int to_time: end of time window in milliseconds since epoch (optional)
        :param list(str) indicator_types: a list of indicator types to filter by (optional)
        :param list(str) tags: Name (or list of names) of tag(s) to filter indicators by.  Only indicators containing
            ALL of these tags will be returned. (optional)
        :param list(str) excluded_tags: Indicators containing ANY of these tags will be excluded from the results.
        :param int page_number: the page number to get.
        :param int page_size: the size of the page to be returned.
        :return: a |Page| of |Indicator| objects.
        """

        body = {
            'searchTerm': search_term
        }

        params = {
            'enclaveIds': enclave_ids,
            'from': from_time,
            'to': to_time,
            'entityTypes': indicator_types,
            'tags': tags,
            'excludedTags': excluded_tags,
            'pageSize': page_size,
            'pageNumber': page_number
        }

        resp = self._client.post("indicators/search", params=params, data=json.dumps(body))

        return Page.from_dict(resp.json(), content_type=Indicator)

Response: Search for indicators containing a search term.


Instruction:
How would you explain what the following Python 3 function does
def get_related_indicators(self, indicators=None, enclave_ids=None):

        return Page.get_generator(page_generator=self._get_related_indicators_page_generator(indicators, enclave_ids))

Response: Uses the |get_related_indicators_page| method to create a generator that returns each successive report.


Instruction:
Create a Python 3 function for
returning the metadata associated with a single indicator.

Response:
def get_indicator_metadata(self, value):
        """
        Provide metadata associated with a single indicators, including value, indicatorType, noteCount,
        sightings, lastSeen, enclaveIds, and tags. The metadata is determined based on the enclaves the user making the
        request has READ access to.

        :param value: an indicator value to query.
        :return: A dict containing three fields: 'indicator' (an |Indicator| object), 'tags' (a list of |Tag|
            objects), and 'enclaveIds' (a list of enclave IDs that the indicator was found in).

        .. warning:: This method is deprecated.  Please use |get_indicators_metadata| instead.
        """

        result = self.get_indicators_metadata([Indicator(value=value)])
        if len(result) > 0:
            indicator = result[0]
            return {
                'indicator': indicator,
                'tags': indicator.tags,
                'enclaveIds': indicator.enclave_ids
            }
        else:
            return None


Instruction:
Make a summary of the following Python 3 code
def get_indicators_metadata(self, indicators):

        data = [{
            'value': i.value,
            'indicatorType': i.type
        } for i in indicators]

        resp = self._client.post("indicators/metadata", data=json.dumps(data))

        return [Indicator.from_dict(x) for x in resp.json()]

Response: This method returns the metadata associated with a list of indicators.


Instruction:
Can you generate the documentation for the following Python 3 function
def get_indicator_details(self, indicators, enclave_ids=None):

        # if the indicators parameter is a string, make it a singleton
        if isinstance(indicators, string_types):
            indicators = [indicators]

        params = {
            'enclaveIds': enclave_ids,
            'indicatorValues': indicators
        }
        resp = self._client.get("indicators/details", params=params)

        return [Indicator.from_dict(indicator) for indicator in resp.json()]

Response: This method returns details for all indicators in the specified enclave.


Instruction:
Can you create a Python 3 function that
adds a list of terms to the user s company s whitelist.

Response:
def add_terms_to_whitelist(self, terms):
        """
        Add a list of terms to the user's company's whitelist.

        :param terms: The list of terms to whitelist.
        :return: The list of extracted |Indicator| objects that were whitelisted.
        """

        resp = self._client.post("whitelist", json=terms)
        return [Indicator.from_dict(indicator) for indicator in resp.json()]


Instruction:
Explain what the following Python 3 code does
def delete_indicator_from_whitelist(self, indicator):

        params = indicator.to_dict()
        self._client.delete("whitelist", params=params)

Response: Delete an indicator from the user s company s whitelist.


Instruction:
Write a Python 3 script to
return a list of indicators that are trending in the community.

Response:
def get_community_trends(self, indicator_type=None, days_back=None):
        """
        Find indicators that are trending in the community.

        :param indicator_type: A type of indicator to filter by.  If ``None``, will get all types of indicators except
            for MALWARE and CVEs (this convention is for parity with the corresponding view on the Dashboard).
        :param days_back: The number of days back to search.  Any integer between 1 and 30 is allowed.
        :return: A list of |Indicator| objects.
        """

        params = {
            'type': indicator_type,
            'daysBack': days_back
        }

        resp = self._client.get("indicators/community-trending", params=params)
        body = resp.json()

        # parse items in response as indicators
        return [Indicator.from_dict(indicator) for indicator in body]


Instruction:
Can you generate a brief explanation for the following Python 3 code
def get_whitelist_page(self, page_number=None, page_size=None):

        params = {
            'pageNumber': page_number,
            'pageSize': page_size
        }
        resp = self._client.get("whitelist", params=params)
        return Page.from_dict(resp.json(), content_type=Indicator)

Response: Gets a paginated list of indicators that the user s company has whitelisted.


Instruction:
How would you implement a function in Python 3 that
finds all reports that contain any of the given indicators and returns correlated indicators from those reports.

Response:
def get_related_indicators_page(self, indicators=None, enclave_ids=None, page_size=None, page_number=None):
        """
        Finds all reports that contain any of the given indicators and returns correlated indicators from those reports.

        :param indicators: list of indicator values to search for
        :param enclave_ids: list of IDs of enclaves to search in
        :param page_size: number of results per page
        :param page_number: page to start returning results on
        :return: A |Page| of |Report| objects.
        """

        params = {
            'indicators': indicators,
            'enclaveIds': enclave_ids,
            'pageNumber': page_number,
            'pageSize': page_size
        }

        resp = self._client.get("indicators/related", params=params)

        return Page.from_dict(resp.json(), content_type=Indicator)


Instruction:
Here you have a function in Python 3, explain what it does
def _get_indicators_for_report_page_generator(self, report_id, start_page=0, page_size=None):

        get_page = functools.partial(self.get_indicators_for_report_page, report_id=report_id)
        return Page.get_page_generator(get_page, start_page, page_size)

Response: Creates a generator that returns each successive page.


Instruction:
How would you explain what the following Python 3 function does
def _get_related_indicators_page_generator(self, indicators=None, enclave_ids=None, start_page=0, page_size=None):

        get_page = functools.partial(self.get_related_indicators_page, indicators, enclave_ids)
        return Page.get_page_generator(get_page, start_page, page_size)

Response:
Creates a generator from the |get_related_indicators_page| method that returns each
        successive page.


Instruction:
Write a Python 3 script for
creating a generator from the |get_whitelist_page| method that returns each successive page.

Response:
def _get_whitelist_page_generator(self, start_page=0, page_size=None):
        """
        Creates a generator from the |get_whitelist_page| method that returns each successive page.

        :param int start_page: The page to start on.
        :param int page_size: The size of each page.
        :return: The generator.
        """

        return Page.get_page_generator(self.get_whitelist_page, start_page, page_size)


Instruction:
Write a Python 3 function that can
display the system as points.

Response:
def points(self, size=1.0, highlight=None, colorlist=None, opacity=1.0):
        """Display the system as points.

        :param float size: the size of the points.

        """
        if colorlist is None:
            colorlist = [get_atom_color(t) for t in self.topology['atom_types']]
        if highlight is not None:
            if isinstance(highlight, int):
                colorlist[highlight] = 0xff0000
            if isinstance(highlight, (list, np.ndarray)):
                for i in highlight:
                    colorlist[i] = 0xff0000

        sizes = [size] * len(self.topology['atom_types'])

        points = self.add_representation('points', {'coordinates': self.coordinates.astype('float32'),
                                                    'colors': colorlist,
                                                    'sizes': sizes,
                                                    'opacity': opacity})
        # Update closure
        def update(self=self, points=points):
            self.update_representation(points, {'coordinates': self.coordinates.astype('float32')})

        self.update_callbacks.append(update)
        self.autozoom(self.coordinates)


Instruction:
Given the following Python 3 function, write the documentation
def labels(self, text=None, coordinates=None, colorlist=None, sizes=None, fonts=None, opacity=1.0):
        '''Display atomic labels for the system'''
        if coordinates is None:
            coordinates=self.coordinates
        l=len(coordinates)
        if text is None:
            if len(self.topology.get('atom_types'))==l:
                text=[self.topology['atom_types'][i]+str(i+1) for i in range(l)]
            else:
                text=[str(i+1) for i in range(l)]

        text_representation = self.add_representation('text', {'coordinates':   coordinates,
                                                               'text':        text,
                                                               'colors':      colorlist,
                                                               'sizes':       sizes,
                                                               'fonts':       fonts,
                                                               'opacity':     opacity})
        def update(self=self, text_representation=text_representation):
            self.update_representation(text_representation, {'coordinates': coordinates})

        self.update_callbacks.append(update)

Response: Display atomic labels for the system


Instruction:
Can you write a function in Python 3 where it
removes all atomic labels from the system

Response:
def remove_labels(self):
        '''Remove all atomic labels from the system'''
        for rep_id in self.representations.keys():
            if self.representations[rep_id]['rep_type']=='text' and rep_id not in self._axes_reps:
                self.remove_representation(rep_id)