Instruction:
Can you generate a brief explanation for the following Python 3 code
def _mdiff(fromlines, tolines, context=None, linejunk=None,
           charjunk=IS_CHARACTER_JUNK):
    r"""Returns generator yielding marked up from/to side by side differences.

    Arguments:
    fromlines -- list of text lines to compared to tolines
    tolines -- list of text lines to be compared to fromlines
    context -- number of context lines to display on each side of difference,
               if None, all from/to text lines will be generated.
    linejunk -- passed on to ndiff (see ndiff documentation)
    charjunk -- passed on to ndiff (see ndiff documentation)

    This function returns an iterator which returns a tuple:
    (from line tuple, to line tuple, boolean flag)

    from/to line tuple -- (line num, line text)
        line num -- integer or None (to indicate a context separation)
        line text -- original line text with following markers inserted:
            '\0+' -- marks start of added text
            '\0-' -- marks start of deleted text
            '\0^' -- marks start of changed text
            '\1' -- marks end of added/deleted/changed text

    boolean flag -- None indicates context separation, True indicates
        either "from" or "to" line contains a change, otherwise False.

    This function/iterator was originally developed to generate side by side
    file difference for making HTML pages (see HtmlDiff class for example
    usage).

    Note, this function utilizes the ndiff function to generate the side by
    side difference markup.  Optional ndiff arguments may be passed to this
    function and they in turn will be passed to ndiff.
    """
    import re

    # regular expression for finding intraline change indices
    change_re = re.compile('(\++|\-+|\^+)')

    # create the difference iterator to generate the differences
    diff_lines_iterator = ndiff(fromlines,tolines,linejunk,charjunk)

    def _make_line(lines, format_key, side, num_lines=[0,0]):
        """Returns line of text with user's change markup and line formatting.

        lines -- list of lines from the ndiff generator to produce a line of
                 text from.  When producing the line of text to return, the
                 lines used are removed from this list.
        format_key -- '+' return first line in list with "add" markup around
                          the entire line.
                      '-' return first line in list with "delete" markup around
                          the entire line.
                      '?' return first line in list with add/delete/change
                          intraline markup (indices obtained from second line)
                      None return first line in list with no markup
        side -- indice into the num_lines list (0=from,1=to)
        num_lines -- from/to current line number.  This is NOT intended to be a
                     passed parameter.  It is present as a keyword argument to
                     maintain memory of the current line numbers between calls
                     of this function.

        Note, this function is purposefully not defined at the module scope so
        that data it needs from its parent function (within whose context it
        is defined) does not need to be of module scope.
        """
        num_lines[side] += 1
        # Handle case where no user markup is to be added, just return line of
        # text with user's line format to allow for usage of the line number.
        if format_key is None:
            return (num_lines[side],lines.pop(0)[2:])
        # Handle case of intraline changes
        if format_key == '?':
            text, markers = lines.pop(0), lines.pop(0)
            # find intraline changes (store change type and indices in tuples)
            sub_info = []
            def record_sub_info(match_object,sub_info=sub_info):
                sub_info.append([match_object.group(1)[0],match_object.span()])
                return match_object.group(1)
            change_re.sub(record_sub_info,markers)
            # process each tuple inserting our special marks that won't be
            # noticed by an xml/html escaper.
            for key,(begin,end) in sub_info[::-1]:
                text = text[0:begin]+'\0'+key+text[begin:end]+'\1'+text[end:]
            text = text[2:]
        # Handle case of add/delete entire line
        else:
            text = lines.pop(0)[2:]
            # if line of text is just a newline, insert a space so there is
            # something for the user to highlight and see.
            if not text:
                text = ' '
            # insert marks that won't be noticed by an xml/html escaper.
            text = '\0' + format_key + text + '\1'
        # Return line of text, first allow user's line formatter to do its
        # thing (such as adding the line number) then replace the special
        # marks with what the user's change markup.
        return (num_lines[side],text)

    def _line_iterator():
        """Yields from/to lines of text with a change indication.

        This function is an iterator.  It itself pulls lines from a
        differencing iterator, processes them and yields them.  When it can
        it yields both a "from" and a "to" line, otherwise it will yield one
        or the other.  In addition to yielding the lines of from/to text, a
        boolean flag is yielded to indicate if the text line(s) have
        differences in them.

        Note, this function is purposefully not defined at the module scope so
        that data it needs from its parent function (within whose context it
        is defined) does not need to be of module scope.
        """
        lines = []
        num_blanks_pending, num_blanks_to_yield = 0, 0
        while True:
            # Load up next 4 lines so we can look ahead, create strings which
            # are a concatenation of the first character of each of the 4 lines
            # so we can do some very readable comparisons.
            while len(lines) < 4:
                try:
                    lines.append(diff_lines_iterator.next())
                except StopIteration:
                    lines.append('X')
            s = ''.join([line[0] for line in lines])
            if s.startswith('X'):
                # When no more lines, pump out any remaining blank lines so the
                # corresponding add/delete lines get a matching blank line so
                # all line pairs get yielded at the next level.
                num_blanks_to_yield = num_blanks_pending
            elif s.startswith('-?+?'):
                # simple intraline change
                yield _make_line(lines,'?',0), _make_line(lines,'?',1), True
                continue
            elif s.startswith('--++'):
                # in delete block, add block coming: we do NOT want to get
                # caught up on blank lines yet, just process the delete line
                num_blanks_pending -= 1
                yield _make_line(lines,'-',0), None, True
                continue
            elif s.startswith(('--?+', '--+', '- ')):
                # in delete block and see an intraline change or unchanged line
                # coming: yield the delete line and then blanks
                from_line,to_line = _make_line(lines,'-',0), None
                num_blanks_to_yield,num_blanks_pending = num_blanks_pending-1,0
            elif s.startswith('-+?'):
                # intraline change
                yield _make_line(lines,None,0), _make_line(lines,'?',1), True
                continue
            elif s.startswith('-?+'):
                # intraline change
                yield _make_line(lines,'?',0), _make_line(lines,None,1), True
                continue
            elif s.startswith('-'):
                # delete FROM line
                num_blanks_pending -= 1
                yield _make_line(lines,'-',0), None, True
                continue
            elif s.startswith('+--'):
                # in add block, delete block coming: we do NOT want to get
                # caught up on blank lines yet, just process the add line
                num_blanks_pending += 1
                yield None, _make_line(lines,'+',1), True
                continue
            elif s.startswith(('+ ', '+-')):
                # will be leaving an add block: yield blanks then add line
                from_line, to_line = None, _make_line(lines,'+',1)
                num_blanks_to_yield,num_blanks_pending = num_blanks_pending+1,0
            elif s.startswith('+'):
                # inside an add block, yield the add line
                num_blanks_pending += 1
                yield None, _make_line(lines,'+',1), True
                continue
            elif s.startswith(' '):
                # unchanged text, yield it to both sides
                yield _make_line(lines[:],None,0),_make_line(lines,None,1),False
                continue
            # Catch up on the blank lines so when we yield the next from/to
            # pair, they are lined up.
            while(num_blanks_to_yield < 0):
                num_blanks_to_yield += 1
                yield None,('','\n'),True
            while(num_blanks_to_yield > 0):
                num_blanks_to_yield -= 1
                yield ('','\n'),None,True
            if s.startswith('X'):
                raise StopIteration
            else:
                yield from_line,to_line,True

    def _line_pair_iterator():
        """Yields from/to lines of text with a change indication.

        This function is an iterator.  It itself pulls lines from the line
        iterator.  Its difference from that iterator is that this function
        always yields a pair of from/to text lines (with the change
        indication).  If necessary it will collect single from/to lines
        until it has a matching pair from/to pair to yield.

        Note, this function is purposefully not defined at the module scope so
        that data it needs from its parent function (within whose context it
        is defined) does not need to be of module scope.
        """
        line_iterator = _line_iterator()
        fromlines,tolines=[],[]
        while True:
            # Collecting lines of text until we have a from/to pair
            while (len(fromlines)==0 or len(tolines)==0):
                from_line, to_line, found_diff =line_iterator.next()
                if from_line is not None:
                    fromlines.append((from_line,found_diff))
                if to_line is not None:
                    tolines.append((to_line,found_diff))
            # Once we have a pair, remove them from the collection and yield it
            from_line, fromDiff = fromlines.pop(0)
            to_line, to_diff = tolines.pop(0)
            yield (from_line,to_line,fromDiff or to_diff)

    # Handle case where user does not want context differencing, just yield
    # them up without doing anything else with them.
    line_pair_iterator = _line_pair_iterator()
    if context is None:
        while True:
            yield line_pair_iterator.next()
    # Handle case where user wants context differencing.  We must do some
    # storage of lines until we know for sure that they are to be yielded.
    else:
        context += 1
        lines_to_write = 0
        while True:
            # Store lines up until we find a difference, note use of a
            # circular queue because we only need to keep around what
            # we need for context.
            index, contextLines = 0, [None]*(context)
            found_diff = False
            while(found_diff is False):
                from_line, to_line, found_diff = line_pair_iterator.next()
                i = index % context
                contextLines[i] = (from_line, to_line, found_diff)
                index += 1
            # Yield lines that we have collected so far, but first yield
            # the user's separator.
            if index > context:
                yield None, None, None
                lines_to_write = context
            else:
                lines_to_write = index
                index = 0
            while(lines_to_write):
                i = index % context
                index += 1
                yield contextLines[i]
                lines_to_write -= 1
            # Now yield the context lines after the change
            lines_to_write = context-1
            while(lines_to_write):
                from_line, to_line, found_diff = line_pair_iterator.next()
                # If another change within the context, extend the context
                if found_diff:
                    lines_to_write = context-1
                else:
                    lines_to_write -= 1
                yield from_line, to_line, found_diff

Response: r Returns an iterator that generates the diff between two lists of lines.


Instruction:
Write a Python 3 script to
make a new Match object from a sequence or iterable

Response:
def _make(cls, iterable, new=tuple.__new__, len=len):
        'Make a new Match object from a sequence or iterable'
        result = new(cls, iterable)
        if len(result) != 3:
            raise TypeError('Expected 3 arguments, got %d' % len(result))
        return result


Instruction:
Explain what the following Python 3 code does
def set_seq1(self, a):

        if a is self.a:
            return
        self.a = a
        self.matching_blocks = self.opcodes = None

Response: Set the first sequence to be compared.


Instruction:
Can you create a Python 3 function that
sets the second sequence to be compared.

Response:
def set_seq2(self, b):
        """Set the second sequence to be compared.

        The first sequence to be compared is not changed.

        >>> s = SequenceMatcher(None, "abcd", "bcde")
        >>> s.ratio()
        0.75
        >>> s.set_seq2("abcd")
        >>> s.ratio()
        1.0
        >>>

        SequenceMatcher computes and caches detailed information about the
        second sequence, so if you want to compare one sequence S against
        many sequences, use .set_seq2(S) once and call .set_seq1(x)
        repeatedly for each of the other sequences.

        See also set_seqs() and set_seq1().
        """

        if b is self.b:
            return
        self.b = b
        self.matching_blocks = self.opcodes = None
        self.fullbcount = None
        self.__chain_b()


Instruction:
Explain what the following Python 3 code does
def find_longest_match(self, alo, ahi, blo, bhi):

        # CAUTION:  stripping common prefix or suffix would be incorrect.
        # E.g.,
        #    ab
        #    acab
        # Longest matching block is "ab", but if common prefix is
        # stripped, it's "a" (tied with "b").  UNIX(tm) diff does so
        # strip, so ends up claiming that ab is changed to acab by
        # inserting "ca" in the middle.  That's minimal but unintuitive:
        # "it's obvious" that someone inserted "ac" at the front.
        # Windiff ends up at the same place as diff, but by pairing up
        # the unique 'b's and then matching the first two 'a's.

        a, b, b2j, isbjunk = self.a, self.b, self.b2j, self.isbjunk
        besti, bestj, bestsize = alo, blo, 0
        # find longest junk-free match
        # during an iteration of the loop, j2len[j] = length of longest
        # junk-free match ending with a[i-1] and b[j]
        j2len = {}
        nothing = []
        for i in xrange(alo, ahi):
            # look at all instances of a[i] in b; note that because
            # b2j has no junk keys, the loop is skipped if a[i] is junk
            j2lenget = j2len.get
            newj2len = {}
            for j in b2j.get(a[i], nothing):
                # a[i] matches b[j]
                if j < blo:
                    continue
                if j >= bhi:
                    break
                k = newj2len[j] = j2lenget(j-1, 0) + 1
                if k > bestsize:
                    besti, bestj, bestsize = i-k+1, j-k+1, k
            j2len = newj2len

        # Extend the best by non-junk elements on each end.  In particular,
        # "popular" non-junk elements aren't in b2j, which greatly speeds
        # the inner loop above, but also means "the best" match so far
        # doesn't contain any junk *or* popular non-junk elements.
        while besti > alo and bestj > blo and \
              not isbjunk(b[bestj-1]) and \
              a[besti-1] == b[bestj-1]:
            besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
        while besti+bestsize < ahi and bestj+bestsize < bhi and \
              not isbjunk(b[bestj+bestsize]) and \
              a[besti+bestsize] == b[bestj+bestsize]:
            bestsize += 1

        # Now that we have a wholly interesting match (albeit possibly
        # empty!), we may as well suck up the matching junk on each
        # side of it too.  Can't think of a good reason not to, and it
        # saves post-processing the (possibly considerable) expense of
        # figuring out what to do with it.  In the case of an empty
        # interesting match, this is clearly the right thing to do,
        # because no other kind of match is possible in the regions.
        while besti > alo and bestj > blo and \
              isbjunk(b[bestj-1]) and \
              a[besti-1] == b[bestj-1]:
            besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
        while besti+bestsize < ahi and bestj+bestsize < bhi and \
              isbjunk(b[bestj+bestsize]) and \
              a[besti+bestsize] == b[bestj+bestsize]:
            bestsize = bestsize + 1

        return Match(besti, bestj, bestsize)

Response: Find the longest matching block in a and b.


Instruction:
Implement a function in Python 3 to
return a list of triples describing matching subsequences.

Response:
def get_matching_blocks(self):
        """Return list of triples describing matching subsequences.

        Each triple is of the form (i, j, n), and means that
        a[i:i+n] == b[j:j+n].  The triples are monotonically increasing in
        i and in j.  New in Python 2.5, it's also guaranteed that if
        (i, j, n) and (i', j', n') are adjacent triples in the list, and
        the second is not the last triple in the list, then i+n != i' or
        j+n != j'.  IOW, adjacent triples never describe adjacent equal
        blocks.

        The last triple is a dummy, (len(a), len(b), 0), and is the only
        triple with n==0.

        >>> s = SequenceMatcher(None, "abxcd", "abcd")
        >>> s.get_matching_blocks()
        [Match(a=0, b=0, size=2), Match(a=3, b=2, size=2), Match(a=5, b=4, size=0)]
        """

        if self.matching_blocks is not None:
            return self.matching_blocks
        la, lb = len(self.a), len(self.b)

        # This is most naturally expressed as a recursive algorithm, but
        # at least one user bumped into extreme use cases that exceeded
        # the recursion limit on their box.  So, now we maintain a list
        # ('queue`) of blocks we still need to look at, and append partial
        # results to `matching_blocks` in a loop; the matches are sorted
        # at the end.
        queue = [(0, la, 0, lb)]
        matching_blocks = []
        while queue:
            alo, ahi, blo, bhi = queue.pop()
            i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi)
            # a[alo:i] vs b[blo:j] unknown
            # a[i:i+k] same as b[j:j+k]
            # a[i+k:ahi] vs b[j+k:bhi] unknown
            if k:   # if k is 0, there was no matching block
                matching_blocks.append(x)
                if alo < i and blo < j:
                    queue.append((alo, i, blo, j))
                if i+k < ahi and j+k < bhi:
                    queue.append((i+k, ahi, j+k, bhi))
        matching_blocks.sort()

        # It's possible that we have adjacent equal blocks in the
        # matching_blocks list now.  Starting with 2.5, this code was added
        # to collapse them.
        i1 = j1 = k1 = 0
        non_adjacent = []
        for i2, j2, k2 in matching_blocks:
            # Is this block adjacent to i1, j1, k1?
            if i1 + k1 == i2 and j1 + k1 == j2:
                # Yes, so collapse them -- this just increases the length of
                # the first block by the length of the second, and the first
                # block so lengthened remains the block to compare against.
                k1 += k2
            else:
                # Not adjacent.  Remember the first block (k1==0 means it's
                # the dummy we started with), and make the second block the
                # new block to compare against.
                if k1:
                    non_adjacent.append((i1, j1, k1))
                i1, j1, k1 = i2, j2, k2
        if k1:
            non_adjacent.append((i1, j1, k1))

        non_adjacent.append( (la, lb, 0) )
        self.matching_blocks = map(Match._make, non_adjacent)
        return self.matching_blocks


Instruction:
Make a summary of the following Python 3 code
def get_opcodes(self):

        if self.opcodes is not None:
            return self.opcodes
        i = j = 0
        self.opcodes = answer = []
        for ai, bj, size in self.get_matching_blocks():
            # invariant:  we've pumped out correct diffs to change
            # a[:i] into b[:j], and the next matching block is
            # a[ai:ai+size] == b[bj:bj+size].  So we need to pump
            # out a diff to change a[i:ai] into b[j:bj], pump out
            # the matching block, and move (i,j) beyond the match
            tag = ''
            if i < ai and j < bj:
                tag = 'replace'
            elif i < ai:
                tag = 'delete'
            elif j < bj:
                tag = 'insert'
            if tag:
                answer.append( (tag, i, ai, j, bj) )
            i, j = ai+size, bj+size
            # the list of matching blocks is terminated by a
            # sentinel with size 0
            if size:
                answer.append( ('equal', ai, i, bj, j) )
        return answer

Response: Return a list of 5 - tuples describing how to turn a into b.


Instruction:
Given the following Python 3 function, write the documentation
def get_grouped_opcodes(self, n=3):

        codes = self.get_opcodes()
        if not codes:
            codes = [("equal", 0, 1, 0, 1)]
        # Fixup leading and trailing groups if they show no changes.
        if codes[0][0] == 'equal':
            tag, i1, i2, j1, j2 = codes[0]
            codes[0] = tag, max(i1, i2-n), i2, max(j1, j2-n), j2
        if codes[-1][0] == 'equal':
            tag, i1, i2, j1, j2 = codes[-1]
            codes[-1] = tag, i1, min(i2, i1+n), j1, min(j2, j1+n)

        nn = n + n
        group = []
        for tag, i1, i2, j1, j2 in codes:
            # End the current group and start a new one whenever
            # there is a large range with no changes.
            if tag == 'equal' and i2-i1 > nn:
                group.append((tag, i1, min(i2, i1+n), j1, min(j2, j1+n)))
                yield group
                group = []
                i1, j1 = max(i1, i2-n), max(j1, j2-n)
            group.append((tag, i1, i2, j1 ,j2))
        if group and not (len(group)==1 and group[0][0] == 'equal'):
            yield group

Response: Return a generator of grouped change clusters.


Instruction:
How would you implement a function in Python 3 that
returns a measure of the sequences s similarity.

Response:
def ratio(self):
        """Return a measure of the sequences' similarity (float in [0,1]).

        Where T is the total number of elements in both sequences, and
        M is the number of matches, this is 2.0*M / T.
        Note that this is 1 if the sequences are identical, and 0 if
        they have nothing in common.

        .ratio() is expensive to compute if you haven't already computed
        .get_matching_blocks() or .get_opcodes(), in which case you may
        want to try .quick_ratio() or .real_quick_ratio() first to get an
        upper bound.

        >>> s = SequenceMatcher(None, "abcd", "bcde")
        >>> s.ratio()
        0.75
        >>> s.quick_ratio()
        0.75
        >>> s.real_quick_ratio()
        1.0
        """

        matches = reduce(lambda sum, triple: sum + triple[-1],
                         self.get_matching_blocks(), 0)
        return _calculate_ratio(matches, len(self.a) + len(self.b))


Instruction:
Given the following Python 3 function, write the documentation
def quick_ratio(self):

        # viewing a and b as multisets, set matches to the cardinality
        # of their intersection; this counts the number of matches
        # without regard to order, so is clearly an upper bound
        if self.fullbcount is None:
            self.fullbcount = fullbcount = {}
            for elt in self.b:
                fullbcount[elt] = fullbcount.get(elt, 0) + 1
        fullbcount = self.fullbcount
        # avail[x] is the number of times x appears in 'b' less the
        # number of times we've seen it in 'a' so far ... kinda
        avail = {}
        availhas, matches = avail.__contains__, 0
        for elt in self.a:
            if availhas(elt):
                numb = avail[elt]
            else:
                numb = fullbcount.get(elt, 0)
            avail[elt] = numb - 1
            if numb > 0:
                matches = matches + 1
        return _calculate_ratio(matches, len(self.a) + len(self.b))

Response: Return an upper bound on ratio() relatively quickly.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def real_quick_ratio(self):

        la, lb = len(self.a), len(self.b)
        # can't have more matches than the number of elements in the
        # shorter sequence
        return _calculate_ratio(min(la, lb), la + lb)

Response: Return an upper bound on ratio() very quickly.


Instruction:
Can you generate the documentation for the following Python 3 function
def _dump(self, tag, x, lo, hi):
        for i in xrange(lo, hi):
            yield '%s %s' % (tag, x[i])

Response: Generate comparison results for a same - tagged range.


Instruction:
Make a summary of the following Python 3 code
def make_file(self,fromlines,tolines,fromdesc='',todesc='',context=False,
                  numlines=5):
        """Returns HTML file of side by side comparison with change highlights

        Arguments:
        fromlines -- list of "from" lines
        tolines -- list of "to" lines
        fromdesc -- "from" file column header string
        todesc -- "to" file column header string
        context -- set to True for contextual differences (defaults to False
            which shows full differences).
        numlines -- number of context lines.  When context is set True,
            controls number of lines displayed before and after the change.
            When context is False, controls the number of lines to place
            the "next" link anchors before the next change (so click of
            "next" link jumps to just before the change).
        """

        return self._file_template % dict(
            styles = self._styles,
            legend = self._legend,
            table = self.make_table(fromlines,tolines,fromdesc,todesc,
                                    context=context,numlines=numlines))

Response: Returns HTML file of side by side comparison with change highlights


Instruction:
How would you code a function in Python 3 to
return fromlines and tolines with tabs expanded and newlines removed.

Response:
def _tab_newline_replace(self,fromlines,tolines):
        """Returns from/to line lists with tabs expanded and newlines removed.

        Instead of tab characters being replaced by the number of spaces
        needed to fill in to the next tab stop, this function will fill
        the space with tab characters.  This is done so that the difference
        algorithms can identify changes in a file when tabs are replaced by
        spaces and vice versa.  At the end of the HTML generation, the tab
        characters will be replaced with a nonbreakable space.
        """
        def expand_tabs(line):
            # hide real spaces
            line = line.replace(' ','\0')
            # expand tabs into spaces
            line = line.expandtabs(self._tabsize)
            # replace spaces from expanded tabs back into tab characters
            # (we'll replace them with markup after we do differencing)
            line = line.replace(' ','\t')
            return line.replace('\0',' ').rstrip('\n')
        fromlines = [expand_tabs(line) for line in fromlines]
        tolines = [expand_tabs(line) for line in tolines]
        return fromlines,tolines


Instruction:
Explain what the following Python 3 code does
def _split_line(self,data_list,line_num,text):
        # if blank line or context separator, just add it to the output list
        if not line_num:
            data_list.append((line_num,text))
            return

        # if line text doesn't need wrapping, just add it to the output list
        size = len(text)
        max = self._wrapcolumn
        if (size <= max) or ((size -(text.count('\0')*3)) <= max):
            data_list.append((line_num,text))
            return

        # scan text looking for the wrap point, keeping track if the wrap
        # point is inside markers
        i = 0
        n = 0
        mark = ''
        while n < max and i < size:
            if text[i] == '\0':
                i += 1
                mark = text[i]
                i += 1
            elif text[i] == '\1':
                i += 1
                mark = ''
            else:
                i += 1
                n += 1

        # wrap point is inside text, break it up into separate lines
        line1 = text[:i]
        line2 = text[i:]

        # if wrap point is inside markers, place end marker at end of first
        # line and start marker at beginning of second line because each
        # line will have its own table tag markup around it.
        if mark:
            line1 = line1 + '\1'
            line2 = '\0' + mark + line2

        # tack on first line onto the output list
        data_list.append((line_num,line1))

        # use this routine again to wrap the remaining text
        self._split_line(data_list,'>',line2)

Response: This function splits the input text into separate lines at the specified line number.


Instruction:
Write a Python 3 function for
returning an iterator that splits the mdiff text lines into fromdata todata and flag.

Response:
def _line_wrapper(self,diffs):
        """Returns iterator that splits (wraps) mdiff text lines"""

        # pull from/to data and flags from mdiff iterator
        for fromdata,todata,flag in diffs:
            # check for context separators and pass them through
            if flag is None:
                yield fromdata,todata,flag
                continue
            (fromline,fromtext),(toline,totext) = fromdata,todata
            # for each from/to line split it at the wrap column to form
            # list of text lines.
            fromlist,tolist = [],[]
            self._split_line(fromlist,fromline,fromtext)
            self._split_line(tolist,toline,totext)
            # yield from/to line in pairs inserting blank lines as
            # necessary when one side has more wrapped lines
            while fromlist or tolist:
                if fromlist:
                    fromdata = fromlist.pop(0)
                else:
                    fromdata = ('',' ')
                if tolist:
                    todata = tolist.pop(0)
                else:
                    todata = ('',' ')
                yield fromdata,todata,flag


Instruction:
Can you generate the documentation for the following Python 3 function
def _collect_lines(self,diffs):

        fromlist,tolist,flaglist = [],[],[]
        # pull from/to data and flags from mdiff style iterator
        for fromdata,todata,flag in diffs:
            try:
                # store HTML markup of the lines into the lists
                fromlist.append(self._format_line(0,flag,*fromdata))
                tolist.append(self._format_line(1,flag,*todata))
            except TypeError:
                # exceptions occur for lines where context separators go
                fromlist.append(None)
                tolist.append(None)
            flaglist.append(flag)
        return fromlist,tolist,flaglist

Response:
Collects the lines from the mdiff output into separate lists
COOKIENAME and
COOKIENAME into a list of lists
COOKIENAME and a list of lists
COOKIENAME.


Instruction:
Write a Python 3 script for
creating unique anchor prefixes for the current page.

Response:
def _make_prefix(self):
        """Create unique anchor prefixes"""

        # Generate a unique anchor prefix so multiple tables
        # can exist on the same HTML page without conflicts.
        fromprefix = "from%d_" % HtmlDiff._default_prefix
        toprefix = "to%d_" % HtmlDiff._default_prefix
        HtmlDiff._default_prefix += 1
        # store prefixes so line format method has access
        self._prefix = [fromprefix,toprefix]


Instruction:
Can you write a function in Python 3 where it
converts the flags in fromlist to tolist.

Response:
def _convert_flags(self,fromlist,tolist,flaglist,context,numlines):
        """Makes list of "next" links"""

        # all anchor names will be generated using the unique "to" prefix
        toprefix = self._prefix[1]

        # process change flags, generating middle column of next anchors/links
        next_id = ['']*len(flaglist)
        next_href = ['']*len(flaglist)
        num_chg, in_change = 0, False
        last = 0
        for i,flag in enumerate(flaglist):
            if flag:
                if not in_change:
                    in_change = True
                    last = i
                    # at the beginning of a change, drop an anchor a few lines
                    # (the context lines) before the change for the previous
                    # link
                    i = max([0,i-numlines])
                    next_id[i] = ' id="difflib_chg_%s_%d"' % (toprefix,num_chg)
                    # at the beginning of a change, drop a link to the next
                    # change
                    num_chg += 1
                    next_href[last] = 'n' % (
                         toprefix,num_chg)
            else:
                in_change = False
        # check for cases where there is no content to avoid exceptions
        if not flaglist:
            flaglist = [False]
            next_id = ['']
            next_href = ['']
            last = 0
            if context:
                fromlist = ['	 No Differences Found 	']
                tolist = fromlist
            else:
                fromlist = tolist = ['	 Empty File 	']
        # if not a change on first line, drop a link
        if not flaglist[0]:
            next_href[0] = 'f' % toprefix
        # redo the last link to link to the top
        next_href[last] = 't' % (toprefix)

        return fromlist,tolist,flaglist,next_href,next_id


Instruction:
How would you implement a function in Python 3 that
returns an HTML table of side by side comparison with change highlights.

Response:
def make_table(self,fromlines,tolines,fromdesc='',todesc='',context=False,
                   numlines=5):
        """Returns HTML table of side by side comparison with change highlights

        Arguments:
        fromlines -- list of "from" lines
        tolines -- list of "to" lines
        fromdesc -- "from" file column header string
        todesc -- "to" file column header string
        context -- set to True for contextual differences (defaults to False
            which shows full differences).
        numlines -- number of context lines.  When context is set True,
            controls number of lines displayed before and after the change.
            When context is False, controls the number of lines to place
            the "next" link anchors before the next change (so click of
            "next" link jumps to just before the change).
        """

        # make unique anchor prefixes so that multiple tables may exist
        # on the same page without conflict.
        self._make_prefix()

        # change tabs to spaces before it gets more difficult after we insert
        # markup
        fromlines,tolines = self._tab_newline_replace(fromlines,tolines)

        # create diffs iterator which generates side by side from/to data
        if context:
            context_lines = numlines
        else:
            context_lines = None
        diffs = _mdiff(fromlines,tolines,context_lines,linejunk=self._linejunk,
                      charjunk=self._charjunk)

        # set up iterator to wrap lines that exceed desired width
        if self._wrapcolumn:
            diffs = self._line_wrapper(diffs)

        # collect up from/to lines and flags into lists (also format the lines)
        fromlist,tolist,flaglist = self._collect_lines(diffs)

        # process change flags, generating middle column of next anchors/links
        fromlist,tolist,flaglist,next_href,next_id = self._convert_flags(
            fromlist,tolist,flaglist,context,numlines)

        s = []
        fmt = '            <tr><td class="diff_next"%s>%s</td>%s' + \
              '<td class="diff_next">%s</td>%s</tr>\n'
        for i in range(len(flaglist)):
            if flaglist[i] is None:
                # mdiff yields None on separator lines skip the bogus ones
                # generated for the first line
                if i > 0:
                    s.append('        </tbody>        \n        <tbody>\n')
            else:
                s.append( fmt % (next_id[i],next_href[i],fromlist[i],
                                           next_href[i],tolist[i]))
        if fromdesc or todesc:
            header_row = '<thead><tr>%s%s%s%s</tr></thead>' % (
                '<th class="diff_next"><br /></th>',
                '<th colspan="2" class="diff_header">%s</th>' % fromdesc,
                '<th class="diff_next"><br /></th>',
                '<th colspan="2" class="diff_header">%s</th>' % todesc)
        else:
            header_row = ''

        table = self._table_template % dict(
            data_rows=''.join(s),
            header_row=header_row,
            prefix=self._prefix[1])

        return table.replace('\0+','<span class="diff_add">'). \
                     replace('\0-','<span class="diff_sub">'). \
                     replace('\0^','<span class="diff_chg">'). \
                     replace('\1','</span>'). \
                     replace('\t','&nbsp;')


Instruction:
Write a Python 3 function that can
create and return a benchmark that runs work_func p times in parallel.

Response:
def _MakeParallelBenchmark(p, work_func, *args):
  """Create and return a benchmark that runs work_func p times in parallel."""
  def Benchmark(b):  # pylint: disable=missing-docstring
    e = threading.Event()
    def Target():
      e.wait()
      for _ in xrange(b.N / p):
        work_func(*args)
    threads = []
    for _ in xrange(p):
      t = threading.Thread(target=Target)
      t.start()
      threads.append(t)
    b.ResetTimer()
    e.set()
    for t in threads:
      t.join()
  return Benchmark


Instruction:
Write a Python 3 script for
returning an inline function with the given body.

Response:
def visit_function_inline(self, node):
    """Returns an GeneratedExpr for a function with the given body."""
    # First pass collects the names of locals used in this function. Do this in
    # a separate pass so that we know whether to resolve a name as a local or a
    # global during the second pass.
    func_visitor = block.FunctionBlockVisitor(node)
    for child in node.body:
      func_visitor.visit(child)
    func_block = block.FunctionBlock(self.block, node.name, func_visitor.vars,
                                     func_visitor.is_generator)
    visitor = StatementVisitor(func_block, self.future_node)
    # Indent so that the function body is aligned with the goto labels.
    with visitor.writer.indent_block():
      visitor._visit_each(node.body)  # pylint: disable=protected-access

    result = self.block.alloc_temp()
    with self.block.alloc_temp('[]πg.Param') as func_args:
      args = node.args
      argc = len(args.args)
      self.writer.write('{} = make([]πg.Param, {})'.format(
          func_args.expr, argc))
      # The list of defaults only contains args for which a default value is
      # specified so pad it with None to make it the same length as args.
      defaults = [None] * (argc - len(args.defaults)) + args.defaults
      for i, (a, d) in enumerate(zip(args.args, defaults)):
        with self.visit_expr(d) if d else expr.nil_expr as default:
          tmpl = '$args[$i] = πg.Param{Name: $name, Def: $default}'
          self.writer.write_tmpl(tmpl, args=func_args.expr, i=i,
                                 name=util.go_str(a.arg), default=default.expr)
      flags = []
      if args.vararg:
        flags.append('πg.CodeFlagVarArg')
      if args.kwarg:
        flags.append('πg.CodeFlagKWArg')
      # The function object gets written to a temporary writer because we need
      # it as an expression that we subsequently bind to some variable.
      self.writer.write_tmpl(
          '$result = πg.NewFunction(πg.NewCode($name, $filename, $args, '
          '$flags, func(πF *πg.Frame, πArgs []*πg.Object) '
          '(*πg.Object, *πg.BaseException) {',
          result=result.name, name=util.go_str(node.name),
          filename=util.go_str(self.block.root.filename), args=func_args.expr,
          flags=' | '.join(flags) if flags else 0)
      with self.writer.indent_block():
        for var in func_block.vars.values():
          if var.type != block.Var.TYPE_GLOBAL:
            fmt = 'var {0} *πg.Object = {1}; _ = {0}'
            self.writer.write(fmt.format(
                util.adjust_local_name(var.name), var.init_expr))
        self.writer.write_temp_decls(func_block)
        self.writer.write('var πR *πg.Object; _ = πR')
        self.writer.write('var πE *πg.BaseException; _ = πE')
        if func_block.is_generator:
          self.writer.write(
              'return πg.NewGenerator(πF, func(πSent *πg.Object) '
              '(*πg.Object, *πg.BaseException) {')
          with self.writer.indent_block():
            self.writer.write_block(func_block, visitor.writer.getvalue())
            self.writer.write('return nil, πE')
          self.writer.write('}).ToObject(), nil')
        else:
          self.writer.write_block(func_block, visitor.writer.getvalue())
          self.writer.write(textwrap.dedent("""\
              if πE != nil {
              \tπR = nil
              } else if πR == nil {
              \tπR = πg.None
              }
              return πR, πE"""))
      self.writer.write('}), πF.Globals()).ToObject()')
    return result


Instruction:
Write a Python 3 script for
importing a module and binds it to a variable.

Response:
def _import_and_bind(self, imp):
    """Generates code that imports a module and binds it to a variable.

    Args:
      imp: Import object representing an import of the form "import x.y.z" or
          "from x.y import z". Expects only a single binding.
    """
    # Acquire handles to the Code objects in each Go package and call
    # ImportModule to initialize all modules.
    with self.block.alloc_temp() as mod, \
        self.block.alloc_temp('[]*πg.Object') as mod_slice:
      self.writer.write_checked_call2(
          mod_slice, 'πg.ImportModule(πF, {})', util.go_str(imp.name))

      # Bind the imported modules or members to variables in the current scope.
      for binding in imp.bindings:
        if binding.bind_type == imputil.Import.MODULE:
          self.writer.write('{} = {}[{}]'.format(
              mod.name, mod_slice.expr, binding.value))
          self.block.bind_var(self.writer, binding.alias, mod.expr)
        else:
          self.writer.write('{} = {}[{}]'.format(
              mod.name, mod_slice.expr, imp.name.count('.')))
          # Binding a member of the imported module.
          with self.block.alloc_temp() as member:
            self.writer.write_checked_call2(
                member, 'πg.GetAttr(πF, {}, {}, nil)',
                mod.expr, self.block.root.intern(binding.value))
            self.block.bind_var(self.writer, binding.alias, member.expr)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _write_except_dispatcher(self, exc, tb, handlers):
    handler_labels = []
    for i, except_node in enumerate(handlers):
      handler_labels.append(self.block.genlabel())
      if except_node.type:
        with self.visit_expr(except_node.type) as type_,\
            self.block.alloc_temp('bool') as is_inst:
          self.writer.write_checked_call2(
              is_inst, 'πg.IsInstance(πF, {}.ToObject(), {})', exc, type_.expr)
          self.writer.write_tmpl(textwrap.dedent("""\
              if $is_inst {
              \tgoto Label$label
              }"""), is_inst=is_inst.expr, label=handler_labels[-1])
      else:
        # This is a bare except. It should be the last handler.
        if i != len(handlers) - 1:
          msg = "default 'except:' must be last"
          raise util.ParseError(except_node, msg)
        self.writer.write('goto Label{}'.format(handler_labels[-1]))
    if handlers[-1].type:
      # There's no bare except, so the fallback is to re-raise.
      self.writer.write(
          'πE = πF.Raise({}.ToObject(), nil, {}.ToObject())'.format(exc, tb))
      self.writer.write('continue')
    return handler_labels

Response: Writes a dispatcher for an except statement.


Instruction:
Can you implement a function in Python 3 that
lists directory contents using cache.

Response:
def listdir(path):
    """List directory contents, using cache."""
    try:
        cached_mtime, list = cache[path]
        del cache[path]
    except KeyError:
        cached_mtime, list = -1, []
    mtime = os.stat(path).st_mtime
    if mtime != cached_mtime:
        list = os.listdir(path)
        list.sort()
    cache[path] = mtime, list
    return list


Instruction:
Explain what the following Python 3 code does
def annotate(head, list):
    for i in range(len(list)):
        if os.path.isdir(os.path.join(head, list[i])):
            list[i] = list[i] + '/'

Response: Add '/' suffixes to directories.


Instruction:
How would you implement a function in Python 3 that
formats a Python o into a pretty - printed representation.

Response:
def pformat(o, indent=1, width=80, depth=None):
    """Format a Python o into a pretty-printed representation."""
    return PrettyPrinter(indent=indent, width=width, depth=depth).pformat(o)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def format(self, o, context, maxlevels, level):
        return _safe_repr(o, context, maxlevels, level)

Response: Format o for a specific context returning a string containing the representation of o and a flag indicating whether the representation is readable or not.


Instruction:
Given the following Python 3 function, write the documentation
def Eps(value=None, loc=None):
    @llrule(loc, lambda parser: [])
    def rule(parser):
        return value
    return rule

Response: A rule that accepts no tokens and returns value.


Instruction:
Here you have a function in Python 3, explain what it does
def Loc(kind, loc=None):
    @llrule(loc, lambda parser: [kind])
    def rule(parser):
        result = parser._accept(kind)
        if result is unmatched:
            return result
        return result.loc
    return rule

Response: A rule that accepts a token of kind and returns its location.


Instruction:
Make a summary of the following Python 3 code
def Expect(inner_rule, loc=None):
    @llrule(loc, inner_rule.expected)
    def rule(parser):
        result = inner_rule(parser)
        if result is unmatched:
            expected = reduce(list.__add__, [rule.expected(parser) for rule in parser._errrules])
            expected = list(sorted(set(expected)))

            if len(expected) > 1:
                expected = " or ".join([", ".join(expected[0:-1]), expected[-1]])
            elif len(expected) == 1:
                expected = expected[0]
            else:
                expected = "(impossible)"

            error_tok = parser._tokens[parser._errindex]
            error = diagnostic.Diagnostic(
                "fatal", "unexpected {actual}: expected {expected}",
                {"actual": error_tok.kind, "expected": expected},
                error_tok.loc)
            parser.diagnostic_engine.process(error)
        return result
    return rule

Response: A rule that executes inner_rule and emits a diagnostic error if it returns None.


Instruction:
How would you explain what the following Python 3 function does
def SeqN(n, *inner_rules, **kwargs):
    @action(Seq(*inner_rules), loc=kwargs.get("loc", None))
    def rule(parser, *values):
        return values[n]
    return rule

Response:
A rule that accepts a sequence of tokens satisfying rules and returns
    the value returned by rule number n.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def Alt(*inner_rules, **kwargs):
    loc = kwargs.get("loc", None)
    expected = lambda parser: reduce(list.__add__, map(lambda x: x.expected(parser), inner_rules))
    if loc is not None:
        @llrule(loc, expected, cases=len(inner_rules))
        def rule(parser):
            data = parser._save()
            for idx, inner_rule in enumerate(inner_rules):
                result = inner_rule(parser)
                if result is unmatched:
                    parser._restore(data, rule=inner_rule)
                else:
                    rule.covered[idx] = True
                    return result
            return unmatched
    else:
        @llrule(loc, expected, cases=len(inner_rules))
        def rule(parser):
            data = parser._save()
            for inner_rule in inner_rules:
                result = inner_rule(parser)
                if result is unmatched:
                    parser._restore(data, rule=inner_rule)
                else:
                    return result
            return unmatched
    return rule

Response:
A rule that expects a sequence of tokens satisfying one of rules in sequence
    and returns the return
    value of that rule.


Instruction:
How would you explain what the following Python 3 function does
def Newline(loc=None):
    @llrule(loc, lambda parser: ["newline"])
    def rule(parser):
        result = parser._accept("newline")
        if result is unmatched:
            return result
        return []
    return rule

Response: A rule that accepts token of kind newline and returns an empty list.


Instruction:
Here you have a function in Python 3, explain what it does
def single_input(self, body):
        loc = None
        if body != []:
            loc = body[0].loc
        return ast.Interactive(body=body, loc=loc)

Response: single_input is a wrapper around ast. Interactive


Instruction:
Make a summary of the following Python 3 code
def file_input(parser, body):
        body = reduce(list.__add__, body, [])
        loc = None
        if body != []:
            loc = body[0].loc
        return ast.Module(body=body, loc=loc)

Response: file_input - Parses the file input and returns a Module object


Instruction:
Can you tell what is the following Python 3 function doing
def eval_input(self, expr):
        return ast.Expression(body=[expr], loc=expr.loc)

Response: eval_input: testlist NEWLINE * ENDMARKER


Instruction:
Here you have a function in Python 3, explain what it does
def decorated(self, decorators, classfuncdef):
        classfuncdef.at_locs = list(map(lambda x: x[0], decorators))
        classfuncdef.decorator_list = list(map(lambda x: x[1], decorators))
        classfuncdef.loc = classfuncdef.loc.join(decorators[0][0])
        return classfuncdef

Response: decorated - sets the at_locs and decorators of the classfuncdef


Instruction:
Given the following Python 3 function, write the documentation
def funcdef__26(self, def_loc, ident_tok, args, colon_loc, suite):
        return ast.FunctionDef(name=ident_tok.value, args=args, returns=None,
                               body=suite, decorator_list=[],
                               at_locs=[], keyword_loc=def_loc, name_loc=ident_tok.loc,
                               colon_loc=colon_loc, arrow_loc=None,
                               loc=def_loc.join(suite[-1].loc))

Response: (2.6, 2.7) funcdef: 'def' NAME parameters ':' suite


Instruction:
Can you tell what is the following Python 3 function doing
def varargslist__26(self, fparams, args):
        for fparam, default_opt in fparams:
            if default_opt:
                equals_loc, default = default_opt
                args.equals_locs.append(equals_loc)
                args.defaults.append(default)
            elif len(args.defaults) > 0:
                error = diagnostic.Diagnostic(
                    "fatal", "non-default argument follows default argument", {},
                    fparam.loc, [args.args[-1].loc.join(args.defaults[-1].loc)])
                self.diagnostic_engine.process(error)

            args.args.append(fparam)

        def fparam_loc(fparam, default_opt):
            if default_opt:
                equals_loc, default = default_opt
                return fparam.loc.join(default.loc)
            else:
                return fparam.loc

        if args.loc is None:
            args.loc = fparam_loc(*fparams[0]).join(fparam_loc(*fparams[-1]))
        elif len(fparams) > 0:
            args.loc = args.loc.join(fparam_loc(*fparams[0]))

        return args

Response: 2. 6. 1. 2. 3. 2. 3. 2. 2.


Instruction:
Write a Python 3 script to
name [ : test ]

Response:
def tfpdef(self, ident_tok, annotation_opt):
        """(3.0-) tfpdef: NAME [':' test]"""
        if annotation_opt:
            colon_loc, annotation = annotation_opt
            return self._arg(ident_tok, colon_loc, annotation)
        return self._arg(ident_tok)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def expr_stmt(self, lhs, rhs):
        if isinstance(rhs, ast.AugAssign):
            if isinstance(lhs, ast.Tuple) or isinstance(lhs, ast.List):
                error = diagnostic.Diagnostic(
                    "fatal", "illegal expression for augmented assignment", {},
                    rhs.op.loc, [lhs.loc])
                self.diagnostic_engine.process(error)
            else:
                rhs.target = self._assignable(lhs)
                rhs.loc = rhs.target.loc.join(rhs.value.loc)
                return rhs
        elif rhs is not None:
            rhs.targets = list(map(self._assignable, [lhs] + rhs.targets))
            rhs.loc = lhs.loc.join(rhs.value.loc)
            return rhs
        else:
            return ast.Expr(value=lhs, loc=lhs.loc)

Response: Return an expression statement that assigns lhs and rhs to the current object.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def del_stmt(self, stmt_loc, exprs):
        # Python uses exprlist here, but does *not* obey the usual
        # tuple-wrapping semantics, so we embed the rule directly.
        """del_stmt: 'del' exprlist"""
        return ast.Delete(targets=[self._assignable(expr, is_delete=True) for expr in exprs],
                          loc=stmt_loc.join(exprs[-1].loc), keyword_loc=stmt_loc)

Response: Return an ast. Delete statement for the given list of expressions.


Instruction:
Can you create a Python 3 function that
returns statement for the cache entry.

Response:
def return_stmt(self, stmt_loc, values):
        """return_stmt: 'return' [testlist]"""
        loc = stmt_loc
        if values:
            loc = loc.join(values.loc)
        return ast.Return(value=values,
                          loc=loc, keyword_loc=stmt_loc)


Instruction:
Write a Python 3 script to
yield the given expression.

Response:
def yield_stmt(self, expr):
        """yield_stmt: yield_expr"""
        return ast.Expr(value=expr, loc=expr.loc)


Instruction:
Make a summary of the following Python 3 code
def raise_stmt__30(self, raise_loc, exc_opt):
        exc = from_loc = cause = None
        loc = raise_loc
        if exc_opt:
            exc, cause_opt = exc_opt
            loc = loc.join(exc.loc)
            if cause_opt:
                from_loc, cause = cause_opt
                loc = loc.join(cause.loc)
        return ast.Raise(exc=exc, inst=None, tback=None, cause=cause,
                         keyword_loc=raise_loc, from_loc=from_loc, loc=loc)

Response: raise statement for 3. 0


Instruction:
Can you generate the documentation for the following Python 3 function
def import_name(self, import_loc, names):
        return ast.Import(names=names,
                          keyword_loc=import_loc, loc=import_loc.join(names[-1].loc))

Response: get_import_name - returns an ast. Import node


Instruction:
Explain what the following Python 3 code does
def import_from(self, from_loc, module_name, import_loc, names):
        (dots_loc, dots_count), dotted_name_opt = module_name
        module_loc = module = None
        if dotted_name_opt:
            module_loc, module = dotted_name_opt
        lparen_loc, names, rparen_loc = names
        loc = from_loc.join(names[-1].loc)
        if rparen_loc:
            loc = loc.join(rparen_loc)

        if module == "__future__":
            self.add_flags([x.name for x in names])

        return ast.ImportFrom(names=names, module=module, level=dots_count,
                              keyword_loc=from_loc, dots_loc=dots_loc, module_loc=module_loc,
                              import_loc=import_loc, lparen_loc=lparen_loc, rparen_loc=rparen_loc,
                              loc=loc)

Response: A function to create an ast. ImportFrom node.


Instruction:
Can you implement a function in Python 3 that
parses a global statement

Response:
def global_stmt(self, global_loc, names):
        """global_stmt: 'global' NAME (',' NAME)*"""
        return ast.Global(names=list(map(lambda x: x.value, names)),
                          name_locs=list(map(lambda x: x.loc, names)),
                          keyword_loc=global_loc, loc=global_loc.join(names[-1].loc))


Instruction:
Can you tell what is the following Python 3 function doing
def exec_stmt(self, exec_loc, body, in_opt):
        in_loc, globals, locals = None, None, None
        loc = exec_loc.join(body.loc)
        if in_opt:
            in_loc, globals, locals = in_opt
            if locals:
                loc = loc.join(locals.loc)
            else:
                loc = loc.join(globals.loc)
        return ast.Exec(body=body, locals=locals, globals=globals,
                        loc=loc, keyword_loc=exec_loc, in_loc=in_loc)

Response: exec_stmt is a statement that returns an ast. Exec instance.


Instruction:
Can you tell what is the following Python 3 function doing
def nonlocal_stmt(self, nonlocal_loc, names):
        return ast.Nonlocal(names=list(map(lambda x: x.value, names)),
                            name_locs=list(map(lambda x: x.loc, names)),
                            keyword_loc=nonlocal_loc, loc=nonlocal_loc.join(names[-1].loc))

Response: Return an ast. Nonlocal node for the given names.


Instruction:
Write a Python 3 function for
returning an ast. Assert node for the assert statement.

Response:
def assert_stmt(self, assert_loc, test, msg):
        """assert_stmt: 'assert' test [',' test]"""
        loc = assert_loc.join(test.loc)
        if msg:
            loc = loc.join(msg.loc)
        return ast.Assert(test=test, msg=msg,
                          loc=loc, keyword_loc=assert_loc)


Instruction:
Make a summary of the following Python 3 code
def if_stmt(self, if_loc, test, if_colon_loc, body, elifs, else_opt):
        stmt = ast.If(orelse=[],
                      else_loc=None, else_colon_loc=None)

        if else_opt:
            stmt.else_loc, stmt.else_colon_loc, stmt.orelse = else_opt

        for elif_ in reversed(elifs):
            stmt.keyword_loc, stmt.test, stmt.if_colon_loc, stmt.body = elif_
            stmt.loc = stmt.keyword_loc.join(stmt.body[-1].loc)
            if stmt.orelse:
                stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)
            stmt = ast.If(orelse=[stmt],
                          else_loc=None, else_colon_loc=None)

        stmt.keyword_loc, stmt.test, stmt.if_colon_loc, stmt.body = \
            if_loc, test, if_colon_loc, body
        stmt.loc = stmt.keyword_loc.join(stmt.body[-1].loc)
        if stmt.orelse:
            stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)
        return stmt

Response: Parse an if statement.


Instruction:
Explain what the following Python 3 code does
def while_stmt(self, while_loc, test, while_colon_loc, body, else_opt):
        stmt = ast.While(test=test, body=body, orelse=[],
                         keyword_loc=while_loc, while_colon_loc=while_colon_loc,
                         else_loc=None, else_colon_loc=None,
                         loc=while_loc.join(body[-1].loc))
        if else_opt:
            stmt.else_loc, stmt.else_colon_loc, stmt.orelse = else_opt
            stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)

        return stmt

Response: parse a while statement


Instruction:
Explain what the following Python 3 code does
def for_stmt(self, for_loc, target, in_loc, iter, for_colon_loc, body, else_opt):
        stmt = ast.For(target=self._assignable(target), iter=iter, body=body, orelse=[],
                       keyword_loc=for_loc, in_loc=in_loc, for_colon_loc=for_colon_loc,
                       else_loc=None, else_colon_loc=None,
                       loc=for_loc.join(body[-1].loc))
        if else_opt:
            stmt.else_loc, stmt.else_colon_loc, stmt.orelse = else_opt
            stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)

        return stmt

Response: for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]


Instruction:
Explain what the following Python 3 code does
def try_stmt(self, try_loc, try_colon_loc, body, stmt):
        stmt.keyword_loc, stmt.try_colon_loc, stmt.body = \
            try_loc, try_colon_loc, body
        stmt.loc = stmt.loc.join(try_loc)
        return stmt

Response: Parse a try statement.


Instruction:
Explain what the following Python 3 code does
def with_stmt__26(self, with_loc, context, with_var, colon_loc, body):
        if with_var:
            as_loc, optional_vars = with_var
            item = ast.withitem(context_expr=context, optional_vars=optional_vars,
                                as_loc=as_loc, loc=context.loc.join(optional_vars.loc))
        else:
            item = ast.withitem(context_expr=context, optional_vars=None,
                                as_loc=None, loc=context.loc)
        return ast.With(items=[item], body=body,
                        keyword_loc=with_loc, colon_loc=colon_loc,
                        loc=with_loc.join(body[-1].loc))

Response: With statement 2. 6. 3. 1. 2.


Instruction:
Make a summary of the following Python 3 code
def with_stmt__27(self, with_loc, items, colon_loc, body):
        return ast.With(items=items, body=body,
                        keyword_loc=with_loc, colon_loc=colon_loc,
                        loc=with_loc.join(body[-1].loc))

Response: With statement for base class 2. 7. 3. 1


Instruction:
Make a summary of the following Python 3 code
def old_lambdef(self, lambda_loc, args_opt, colon_loc, body):
        if args_opt is None:
            args_opt = self._arguments()
            args_opt.loc = colon_loc.begin()
        return ast.Lambda(args=args_opt, body=body,
                          lambda_loc=lambda_loc, colon_loc=colon_loc,
                          loc=lambda_loc.join(body.loc))

Response: 2. 6. 1. 2


Instruction:
Explain what the following Python 3 code does
def comparison(self, lhs, rhs):
        if len(rhs) > 0:
            return ast.Compare(left=lhs, ops=list(map(lambda x: x[0], rhs)),
                               comparators=list(map(lambda x: x[1], rhs)),
                               loc=lhs.loc.join(rhs[-1][1].loc))
        else:
            return lhs

Response: Compare two sets of entries.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def star_expr__30(self, star_opt, expr):
        if star_opt:
            return ast.Starred(value=expr, ctx=None,
                               star_loc=star_opt, loc=expr.loc.join(star_opt))
        return expr

Response: (3.0, 3.1) star_expr: ['*'] expr


Instruction:
Can you tell what is the following Python 3 function doing
def power(self, atom, trailers, factor_opt):
        for trailer in trailers:
            if isinstance(trailer, ast.Attribute) or isinstance(trailer, ast.Subscript):
                trailer.value = atom
            elif isinstance(trailer, ast.Call):
                trailer.func = atom
            trailer.loc = atom.loc.join(trailer.loc)
            atom = trailer
        if factor_opt:
            op_loc, factor = factor_opt
            return ast.BinOp(left=atom, op=ast.Pow(loc=op_loc), right=factor,
                             loc=atom.loc.join(factor.loc))
        return atom

Response: power atom trailer * [ ** factor ]


Instruction:
Write a Python 3 function that can
get subscriptlist - returns subscript list

Response:
def subscriptlist(self, subscripts):
        """subscriptlist: subscript (',' subscript)* [',']"""
        if len(subscripts) == 1:
            return ast.Subscript(slice=subscripts[0], ctx=None, loc=None)
        elif all([isinstance(x, ast.Index) for x in subscripts]):
            elts  = [x.value for x in subscripts]
            loc   = subscripts[0].loc.join(subscripts[-1].loc)
            index = ast.Index(value=ast.Tuple(elts=elts, ctx=None,
                                              begin_loc=None, end_loc=None, loc=loc),
                              loc=loc)
            return ast.Subscript(slice=index, ctx=None, loc=None)
        else:
            extslice = ast.ExtSlice(dims=subscripts,
                                    loc=subscripts[0].loc.join(subscripts[-1].loc))
            return ast.Subscript(slice=extslice, ctx=None, loc=None)


Instruction:
Can you generate a brief explanation for the following Python 3 code
def dictmaker(self, elts):
        return ast.Dict(keys=list(map(lambda x: x[0], elts)),
                        values=list(map(lambda x: x[2], elts)),
                        colon_locs=list(map(lambda x: x[1], elts)),
                        loc=None)

Response: (2.6) dictmaker: test ':' test (',' test ':' test)* [',']


Instruction:
Explain what the following Python 3 code does
def arglist(self, args, call):
        for arg in args:
            if isinstance(arg, ast.keyword):
                call.keywords.append(arg)
            elif len(call.keywords) > 0:
                error = diagnostic.Diagnostic(
                    "fatal", "non-keyword arg after keyword arg", {},
                    arg.loc, [call.keywords[-1].loc])
                self.diagnostic_engine.process(error)
            else:
                call.args.append(arg)
        return call

Response: add arglist to call


Instruction:
Implement a function in Python 3 to
yield expression 2. 6. 3. 2

Response:
def yield_expr__26(self, yield_loc, exprs):
        """(2.6, 2.7, 3.0, 3.1, 3.2) yield_expr: 'yield' [testlist]"""
        if exprs is not None:
            return ast.Yield(value=exprs,
                             yield_loc=yield_loc, loc=yield_loc.join(exprs.loc))
        else:
            return ast.Yield(value=None,
                             yield_loc=yield_loc, loc=yield_loc)


Instruction:
Can you generate the documentation for the following Python 3 function
def urlparse(url, scheme='', allow_fragments=True):
    tuple = urlsplit(url, scheme, allow_fragments)
    scheme, netloc, url, query, fragment = tuple
    if scheme in uses_params and ';' in url:
        url, params = _splitparams(url)
    else:
        params = ''
    return ParseResult(scheme, netloc, url, params, query, fragment)

Response: Parse a URL into 6 components.


Instruction:
Can you generate the documentation for the following Python 3 function
def urlsplit(url, scheme='', allow_fragments=True):
    allow_fragments = bool(allow_fragments)
    key = url, scheme, allow_fragments, type(url), type(scheme)
    cached = _parse_cache.get(key, None)
    if cached:
        return cached
    if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth
        clear_cache()
    netloc = query = fragment = ''
    i = url.find(':')
    if i > 0:
        if url[:i] == 'http': # optimize the common case
            scheme = url[:i].lower()
            url = url[i+1:]
            if url[:2] == '//':
                netloc, url = _splitnetloc(url, 2)
                if (('[' in netloc and ']' not in netloc) or
                        (']' in netloc and '[' not in netloc)):
                    raise ValueError("Invalid IPv6 URL")
            if allow_fragments and '#' in url:
                url, fragment = url.split('#', 1)
            if '?' in url:
                url, query = url.split('?', 1)
            v = SplitResult(scheme, netloc, url, query, fragment)
            _parse_cache[key] = v
            return v
        for c in url[:i]:
            if c not in scheme_chars:
                break
        else:
            # make sure "url" is not actually a port number (in which case
            # "scheme" is really part of the path)
            rest = url[i+1:]
            if not rest or any(c not in '0123456789' for c in rest):
                # not a port number
                scheme, url = url[:i].lower(), rest

    if url[:2] == '//':
        netloc, url = _splitnetloc(url, 2)
        if (('[' in netloc and ']' not in netloc) or
                (']' in netloc and '[' not in netloc)):
            raise ValueError("Invalid IPv6 URL")
    if allow_fragments and '#' in url:
        url, fragment = url.split('#', 1)
    if '?' in url:
        url, query = url.split('?', 1)
    v = SplitResult(scheme, netloc, url, query, fragment)
    _parse_cache[key] = v
    return v

Response: Parse a URL into 5 components.


Instruction:
Create a Python 3 function for
putting a parsed URL back together.

Response:
def urlunparse(data):
    """Put a parsed URL back together again.  This may result in a
    slightly different, but equivalent URL, if the URL that was parsed
    originally had redundant delimiters, e.g. a ? with an empty query
    (the draft states that these are equivalent)."""
    scheme, netloc, url, params, query, fragment = data
    if params:
        url = "%s;%s" % (url, params)
    return urlunsplit((scheme, netloc, url, query, fragment))


Instruction:
Can you tell what is the following Python 3 function doing
def urlunsplit(data):
    scheme, netloc, url, query, fragment = data
    if netloc or (scheme and scheme in uses_netloc and url[:2] != '//'):
        if url and url[:1] != '/': url = '/' + url
        url = '//' + (netloc or '') + url
    if scheme:
        url = scheme + ':' + url
    if query:
        url = url + '?' + query
    if fragment:
        url = url + '#' + fragment
    return url

Response: Combine the elements of a tuple as returned by urlsplit into a complete URL as a string.


Instruction:
Implement a Python 3 function for
joining a base URL and a possibly relative URL to form an absolute URL interpretation of the latter.

Response:
def urljoin(base, url, allow_fragments=True):
    """Join a base URL and a possibly relative URL to form an absolute
    interpretation of the latter."""
    if not base:
        return url
    if not url:
        return base
    bscheme, bnetloc, bpath, bparams, bquery, bfragment = \
            urlparse(base, '', allow_fragments)
    scheme, netloc, path, params, query, fragment = \
            urlparse(url, bscheme, allow_fragments)
    if scheme != bscheme or scheme not in uses_relative:
        return url
    if scheme in uses_netloc:
        if netloc:
            return urlunparse((scheme, netloc, path,
                               params, query, fragment))
        netloc = bnetloc
    if path[:1] == '/':
        return urlunparse((scheme, netloc, path,
                           params, query, fragment))
    if not path and not params:
        path = bpath
        params = bparams
        if not query:
            query = bquery
        return urlunparse((scheme, netloc, path,
                           params, query, fragment))
    segments = bpath.split('/')[:-1] + path.split('/')
    # XXX The stuff below is bogus in various ways...
    if segments[-1] == '.':
        segments[-1] = ''
    while '.' in segments:
        segments.remove('.')
    while 1:
        i = 1
        n = len(segments) - 1
        while i < n:
            if (segments[i] == '..'
                and segments[i-1] not in ('', '..')):
                del segments[i-1:i+1]
                break
            i = i+1
        else:
            break
    if segments == ['', '..']:
        segments[-1] = ''
    elif len(segments) >= 2 and segments[-1] == '..':
        segments[-2:] = ['']
    return urlunparse((scheme, netloc, '/'.join(segments),
                       params, query, fragment))


Instruction:
Create a Python 3 function for
removing any existing fragment from the URL.

Response:
def urldefrag(url):
    """Removes any existing fragment from URL.

    Returns a tuple of the defragmented URL and the fragment.  If
    the URL contained no fragments, the second element is the
    empty string.
    """
    if '#' in url:
        s, n, p, a, q, frag = urlparse(url)
        defrag = urlunparse((s, n, p, a, q, ''))
        return defrag, frag
    else:
        return url, ''


Instruction:
Write a Python 3 function for
parsing a query string given as a string argument.

Response:
def parse_qs(qs, keep_blank_values=0, strict_parsing=0):
    """Parse a query given as a string argument.

        Arguments:

        qs: percent-encoded query string to be parsed

        keep_blank_values: flag indicating whether blank values in
            percent-encoded queries should be treated as blank strings.
            A true value indicates that blanks should be retained as
            blank strings.  The default false value indicates that
            blank values are to be ignored and treated as if they were
            not included.

        strict_parsing: flag indicating what to do with parsing errors.
            If false (the default), errors are silently ignored.
            If true, errors raise a ValueError exception.
    """
    dict = {}
    for name, value in parse_qsl(qs, keep_blank_values, strict_parsing):
        if name in dict:
            dict[name].append(value)
        else:
            dict[name] = [value]
    return dict


Instruction:
Can you implement a function in Python 3 that
parses a percent - encoded query string into a list of items.

Response:
def parse_qsl(qs, keep_blank_values=0, strict_parsing=0):
    """Parse a query given as a string argument.

    Arguments:

    qs: percent-encoded query string to be parsed

    keep_blank_values: flag indicating whether blank values in
        percent-encoded queries should be treated as blank strings.  A
        true value indicates that blanks should be retained as blank
        strings.  The default false value indicates that blank values
        are to be ignored and treated as if they were  not included.

    strict_parsing: flag indicating what to do with parsing errors. If
        false (the default), errors are silently ignored. If true,
        errors raise a ValueError exception.

    Returns a list, as G-d intended.
    """
    pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')]
    r = []
    for name_value in pairs:
        if not name_value and not strict_parsing:
            continue
        nv = name_value.split('=', 1)
        if len(nv) != 2:
            if strict_parsing:
                raise ValueError, "bad query field: %r" % (name_value,)
            # Handle case of a control-name with no equal sign
            if keep_blank_values:
                nv.append('')
            else:
                continue
        if len(nv[1]) or keep_blank_values:
            name = unquote(nv[0].replace('+', ' '))
            value = unquote(nv[1].replace('+', ' '))
            r.append((name, value))

    return r


Instruction:
Can you generate the documentation for the following Python 3 function
def _replace(_self, **kwds):
        'Return a new SplitResult object replacing specified fields with new values'
        result = _self._make(map(kwds.pop, ('scheme', 'netloc', 'path', 'query', 'fragment'), _self))
        if kwds:
            raise ValueError('Got unexpected field names: %r' % kwds.keys())
        return result

Response: Return a new SplitResult object replacing specified fields with new values


Instruction:
Can you implement a function in Python 3 that
gets the lines for a file from the cache.

Response:
def getlines(filename, module_globals=None):
    """Get the lines for a file from the cache.
    Update the cache if it doesn't contain an entry for this file already."""

    if filename in cache:
        return cache[filename][2]

    try:
        return updatecache(filename, module_globals)
    except MemoryError:
        clearcache()
        return []


Instruction:
Make a summary of the following Python 3 code
def checkcache(filename=None):

    if filename is None:
        filenames = cache.keys()
    else:
        if filename in cache:
            filenames = [filename]
        else:
            return

    for filename in filenames:
        size, mtime, lines, fullname = cache[filename]
        if mtime is None:
            continue   # no-op for files loaded via a __loader__
        try:
            stat = os.stat(fullname)
        except os.error:
            del cache[filename]
            continue
        if size != stat.st_size or mtime != stat.st_mtime:
            del cache[filename]

Response: Discard cache entries that are out of date.


Instruction:
Can you tell what is the following Python 3 function doing
def updatecache(filename, module_globals=None):

    if filename in cache:
        del cache[filename]
    if not filename or (filename.startswith('<') and filename.endswith('>')):
        return []

    fullname = filename
    try:
        stat = os.stat(fullname)
    except OSError:
        basename = filename

        # Try for a __loader__, if available
        if module_globals and '__loader__' in module_globals:
            name = module_globals.get('__name__')
            loader = module_globals['__loader__']
            get_source = getattr(loader, 'get_source', None)

            if name and get_source:
                try:
                    data = get_source(name)
                except (ImportError, IOError):
                    pass
                else:
                    if data is None:
                        # No luck, the PEP302 loader cannot find the source
                        # for this module.
                        return []
                    cache[filename] = (
                        len(data), None,
                        [line+'\n' for line in data.splitlines()], fullname
                    )
                    return cache[filename][2]

        # Try looking through the module search path, which is only useful
        # when handling a relative filename.
        if os.path.isabs(filename):
            return []

        for dirname in sys.path:
            # When using imputil, sys.path may contain things other than
            # strings; ignore them when it happens.
            try:
                fullname = os.path.join(dirname, basename)
            except (TypeError, AttributeError):
                # Not sufficiently string-like to do anything useful with.
                continue
            try:
                stat = os.stat(fullname)
                break
            except os.error:
                pass
        else:
            return []
    try:
        with open(fullname, 'rU') as fp:
            lines = fp.readlines()
    except IOError:
        return []
    if lines and not lines[-1].endswith('\n'):
        lines[-1] += '\n'
    size, mtime = stat.st_size, stat.st_mtime
    cache[filename] = size, mtime, lines, fullname
    return lines

Response: Update a cache entry and return its list of lines.


Instruction:
Explain what the following Python 3 code does
def isfile(path):
    try:
        st = os.stat(path)
    except os.error:
        return False
    return stat.S_ISREG(st.st_mode)

Response: Test whether a path is a regular file


Instruction:
Create a Python 3 function for
returning True if the pathname refers to an existing directory.

Response:
def isdir(s):
    """Return true if the pathname refers to an existing directory."""
    try:
        st = os.stat(s)
    except os.error:
        return False
    return stat.S_ISDIR(st.st_mode)


Instruction:
Can you tell what is the following Python 3 function doing
def commonprefix(m):
    "Given a list of pathnames, returns the longest common leading component"
    if not m: return ''
    s1 = min(m)
    s2 = max(m)
    for i, c in enumerate(s1):
        if c != s2[i]:
            return s1[:i]
    return s1

Response: Given a list of pathnames returns the longest common leading component


Instruction:
Can you tell what is the following Python 3 function doing
def _splitext(p, sep, altsep, extsep):

    sepIndex = p.rfind(sep)
    if altsep:
        altsepIndex = p.rfind(altsep)
        sepIndex = max(sepIndex, altsepIndex)

    dotIndex = p.rfind(extsep)
    if dotIndex > sepIndex:
        # skip all leading dots
        filenameIndex = sepIndex + 1
        while filenameIndex < dotIndex:
            if p[filenameIndex] != extsep:
                return p[:dotIndex], p[dotIndex:]
            filenameIndex += 1

    return p, ''

Response: Split the extension from a pathname.


Instruction:
How would you implement a function in Python 3 that
wraps a single paragraph of text returning a list of wrapped lines.

Response:
def wrap(text, width=70, **kwargs):
    """Wrap a single paragraph of text, returning a list of wrapped lines.

    Reformat the single paragraph in 'text' so it fits in lines of no
    more than 'width' columns, and return a list of wrapped lines.  By
    default, tabs in 'text' are expanded with string.expandtabs(), and
    all other whitespace characters (including newline) are converted to
    space.  See TextWrapper class for available keyword args to customize
    wrapping behaviour.
    """
    w = TextWrapper(width=width, **kwargs)
    return w.wrap(text)


Instruction:
Can you generate the documentation for the following Python 3 function
def fill(text, width=70, **kwargs):
    w = TextWrapper(width=width, **kwargs)
    return w.fill(text)

Response: Fill a single paragraph of text returning a new string.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def dedent(text):
    # Look for the longest leading string of spaces and tabs common to
    # all lines.
    margin = None
    text = _whitespace_only_re.sub('', text)
    indents = _leading_whitespace_re.findall(text)
    for indent in indents:
        if margin is None:
            margin = indent

        # Current line more deeply indented than previous winner:
        # no change (previous winner is still on top).
        elif indent.startswith(margin):
            pass

        # Current line consistent with and no deeper than previous winner:
        # it's the new winner.
        elif margin.startswith(indent):
            margin = indent

        # Find the largest common whitespace between current line and previous
        # winner.
        else:
            for i, (x, y) in enumerate(zip(margin, indent)):
                if x != y:
                    margin = margin[:i]
                    break
            else:
                margin = margin[:len(indent)]

    # sanity check (testing/debugging only)
    if 0 and margin:
        for line in text.split("\n"):
            assert not line or line.startswith(margin), \
                   "line = %r, margin = %r" % (line, margin)

    if margin:
        text = re.sub(r'(?m)^' + margin, '', text)
    return text

Response: Remove any common leading whitespace from every line in text.


Instruction:
Can you tell what is the following Python 3 function doing
def _split(self, text):
        if isinstance(text, _unicode):
            if self.break_on_hyphens:
                pat = self.wordsep_re_uni
            else:
                pat = self.wordsep_simple_re_uni
        else:
            if self.break_on_hyphens:
                pat = self.wordsep_re
            else:
                pat = self.wordsep_simple_re
        chunks = pat.split(text)
        # chunks = filter(None, chunks)  # remove empty chunks
        chunks = [x for x in chunks if x is not None]
        return chunks

Response: Split the text into individual words.


Instruction:
Write a Python 3 script to
fix for sentence endings buried in chunks.

Response:
def _fix_sentence_endings(self, chunks):
        """_fix_sentence_endings(chunks : [string])

        Correct for sentence endings buried in 'chunks'.  Eg. when the
        original text contains "... foo.\\nBar ...", munge_whitespace()
        and split() will convert that to [..., "foo.", " ", "Bar", ...]
        which has one too few spaces; this method simply changes the one
        space to two.
        """
        i = 0
        patsearch = self.sentence_end_re.search
        while i < len(chunks)-1:
            if chunks[i+1] == " " and patsearch(chunks[i]):
                chunks[i+1] = "  "
                i += 2
            else:
                i += 1


Instruction:
Write a Python 3 function that can
handle a long word.

Response:
def _handle_long_word(self, reversed_chunks, cur_line, cur_len, width):
        """_handle_long_word(chunks : [string],
                             cur_line : [string],
                             cur_len : int, width : int)

        Handle a chunk of text (most likely a word, not whitespace) that
        is too long to fit in any line.
        """
        # Figure out when indent is larger than the specified width, and make
        # sure at least one character is stripped off on every pass
        if width < 1:
            space_left = 1
        else:
            space_left = width - cur_len

        # If we're allowed to break long words, then do so: put as much
        # of the next chunk onto the current line as will fit.
        if self.break_long_words:
            cur_line.append(reversed_chunks[-1][:space_left])
            reversed_chunks[-1] = reversed_chunks[-1][space_left:]

        # Otherwise, we have to preserve the long word intact.  Only add
        # it to the current line if there's nothing already there --
        # that minimizes how much we violate the width constraint.
        elif not cur_line:
            cur_line.append(reversed_chunks.pop())


Instruction:
Make a summary of the following Python 3 code
def _wrap_chunks(self, chunks):
        lines = []
        if self.width <= 0:
            raise ValueError("invalid width %r (must be > 0)" % self.width)

        # Arrange in reverse order so items can be efficiently popped
        # from a stack of chucks.
        chunks.reverse()

        while chunks:

            # Start the list of chunks that will make up the current line.
            # cur_len is just the length of all the chunks in cur_line.
            cur_line = []
            cur_len = 0

            # Figure out which static string will prefix this line.
            if lines:
                indent = self.subsequent_indent
            else:
                indent = self.initial_indent

            # Maximum width for this line.
            width = self.width - len(indent)

            # First chunk on line is whitespace -- drop it, unless this
            # is the very beginning of the text (ie. no lines started yet).
            if self.drop_whitespace and chunks[-1].strip() == '' and lines:
                # del chunks[-1]
                chunks.pop()

            while chunks:
                l = len(chunks[-1])

                # Can at least squeeze this chunk onto the current line.
                if cur_len + l <= width:
                    cur_line.append(chunks.pop())
                    cur_len += l

                # Nope, this line is full.
                else:
                    break

            # The current line is full, and the next chunk is too big to
            # fit on *any* line (not just this one).
            if chunks and len(chunks[-1]) > width:
                self._handle_long_word(chunks, cur_line, cur_len, width)

            # If the last chunk on this line is all whitespace, drop it.
            if self.drop_whitespace and cur_line and cur_line[-1].strip() == '':
                # del cur_line[-1]
                cur_line.pop()

            # Convert current line back to a string and store it in list
            # of all lines (return value).
            if cur_line:
                lines.append(indent + ''.join(cur_line))

        return lines

Response: Wrap a sequence of text chunks into a list of lines of length self. width.


Instruction:
Can you generate the documentation for the following Python 3 function
def wrap(self, text):
        text = self._munge_whitespace(text)
        chunks = self._split(text)
        if self.fix_sentence_endings:
            self._fix_sentence_endings(chunks)
        return self._wrap_chunks(chunks)

Response: Wraps the given text into a list of lines of the specified width.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def _long2bytesBigEndian(n, blocksize=0):

    # After much testing, this algorithm was deemed to be the fastest.
    s = b''
    pack = struct.pack
    while n > 0:
        s = pack('>I', n & 0xffffffff) + s
        n = n >> 32

    # Strip off leading zeros.
    for i in range(len(s)):
        if s[i] != '\000':
            break
    else:
        # Only happens when n == 0.
        s = '\000'
        i = 0

    s = s[i:]

    # Add back some pad bytes. This could be done more efficiently
    # w.r.t. the de-padding being done above, but sigh...
    if blocksize > 0 and len(s) % blocksize:
        s = (blocksize - len(s) % blocksize) * '\000' + s

    return s

Response: Convert a long integer to a byte string in big - endian byte order.


Instruction:
How would you code a function in Python 3 to
transform a list of characters into a list of longs.

Response:
def _bytelist2longBigEndian(list):
    "Transform a list of characters into a list of longs."

    imax = len(list) // 4
    hl = [0] * imax

    j = 0
    i = 0
    while i < imax:
        b0 = ord(list[j]) << 24
        b1 = ord(list[j+1]) << 16
        b2 = ord(list[j+2]) << 8
        b3 = ord(list[j+3])
        hl[i] = b0 | b1 | b2 | b3
        i = i+1
        j = j+4

    return hl


Instruction:
Implement a Python 3 function for
initializing the message - digest and set all fields to zero.

Response:
def init(self):
        "Initialize the message-digest and set all fields to zero."

        self.length = 0
        self.input = []

        # Initial 160 bit message digest (5 times 32 bit).
        self.H0 = 0x67452301
        self.H1 = 0xEFCDAB89
        self.H2 = 0x98BADCFE
        self.H3 = 0x10325476
        self.H4 = 0xC3D2E1F0


Instruction:
Can you create a Python 3 function that
terminates the message - digest computation and return digest.

Response:
def digest(self):
        """Terminate the message-digest computation and return digest.

        Return the digest of the strings passed to the update()
        method so far. This is a 16-byte string which may contain
        non-ASCII characters, including null bytes.
        """

        H0 = self.H0
        H1 = self.H1
        H2 = self.H2
        H3 = self.H3
        H4 = self.H4
        input = [] + self.input
        count = [] + self.count

        index = (self.count[1] >> 3) & 0x3f

        if index < 56:
            padLen = 56 - index
        else:
            padLen = 120 - index

        padding = ['\200'] + ['\000'] * 63
        self.update(padding[:padLen])

        # Append length (before padding).
        bits = _bytelist2longBigEndian(self.input[:56]) + count

        self._transform(bits)

        # Store state in digest.
        digest = _long2bytesBigEndian(self.H0, 4) + \
                 _long2bytesBigEndian(self.H1, 4) + \
                 _long2bytesBigEndian(self.H2, 4) + \
                 _long2bytesBigEndian(self.H3, 4) + \
                 _long2bytesBigEndian(self.H4, 4)

        self.H0 = H0
        self.H1 = H1
        self.H2 = H2
        self.H3 = H3
        self.H4 = H4
        self.input = input
        self.count = count

        return digest


Instruction:
How would you code a function in Python 3 to
return the next token in the queue.

Response:
def next(self, eof_token=False):
        """
        Returns token at ``offset`` as a :class:`Token` and advances ``offset``
        to point past the end of the token, where the token has:

        - *range* which is a :class:`pythonparser.source.Range` that includes
          the token but not surrounding whitespace,
        - *kind* which is a string containing one of Python keywords or operators,
          ``newline``, ``float``, ``int``, ``complex``, ``strbegin``,
          ``strdata``, ``strend``, ``ident``, ``indent``, ``dedent`` or ``eof``
          (if ``eof_token`` is True).
        - *value* which is the flags as lowercase string if *kind* is ``strbegin``,
          the string contents if *kind* is ``strdata``,
          the numeric value if *kind* is ``float``, ``int`` or ``complex``,
          the identifier if *kind* is ``ident`` and ``None`` in any other case.

        :param eof_token: if true, will return a token with kind ``eof``
            when the input is exhausted; if false, will raise ``StopIteration``.
        """
        if len(self.queue) == 0:
            self._refill(eof_token)

        return self.queue.pop(0)


Instruction:
How would you explain what the following Python 3 function does
def enterabs(self, time, priority, action, argument):
        event = Event(time, priority, action, argument)
        heapq.heappush(self._queue, event)
        return event

Response: Enter a new event at an absolute time.


Instruction:
Given the following Python 3 function, write the documentation
def run(self):
        # localize variable access to minimize overhead
        # and to improve thread safety
        q = self._queue
        delayfunc = self.delayfunc
        timefunc = self.timefunc
        pop = heapq.heappop
        while q:
            # TODO: modified part of grumpy version.
            checked_event = q[0]
            time, priority, action, argument = checked_event.get_fields()
            now = timefunc()
            if now < time:
                delayfunc(time - now)
            else:
                event = pop(q)
                # Verify that the event was not removed or altered
                # by another thread after we last looked at q[0].
                if event is checked_event:
                    action(*argument)
                    delayfunc(0)   # Let other threads run
                else:
                    heapq.heappush(q, event)

Response: Execute events until the queue is empty and the scheduler is running.


Instruction:
Write a Python 3 script for
registering a new dialect.

Response:
def register_dialect(name, dialect=None, **kwargs):
    """Create a mapping from a string name to a dialect class.
    dialect = csv.register_dialect(name, dialect)"""
    if not isinstance(name, basestring):
        raise TypeError("dialect name must be a string or unicode")

    dialect = _call_dialect(dialect, kwargs)
    _dialects[name] = dialect


Instruction:
Can you generate a brief explanation for the following Python 3 code
def field_size_limit(limit=undefined):

    global _field_limit
    old_limit = _field_limit

    if limit is not undefined:
        if not isinstance(limit, (int, long)):
            raise TypeError("int expected, got %s" %
                            (limit.__class__.__name__,))
        _field_limit = limit

    return old_limit

Response:
Sets the upper limit on parsed fields.
    Returns old limit


Instruction:
Can you write a function in Python 3 where it
shallows copy operation on arbitrary Python objects.

Response:
def copy(x):
    """Shallow copy operation on arbitrary Python objects.

    See the module's __doc__ string for more info.
    """

    cls = type(x)

    copier = _copy_dispatch.get(cls)
    if copier:
        return copier(x)

    copier = getattr(cls, "__copy__", None)
    if copier:
        return copier(x)

    reductor = dispatch_table.get(cls)
    if reductor:
        rv = reductor(x)
    else:
        reductor = getattr(x, "__reduce_ex__", None)
        if reductor:
            rv = reductor(2)
        else:
            reductor = getattr(x, "__reduce__", None)
            if reductor:
                rv = reductor()
            else:
                raise Error("un(shallow)copyable object of type %s" % cls)

    return _reconstruct(x, rv, 0)


Instruction:
Explain what the following Python 3 code does
def deepcopy(x, memo=None, _nil=[]):

    if memo is None:
        memo = {}

    d = id(x)
    y = memo.get(d, _nil)
    if y is not _nil:
        return y

    cls = type(x)

    copier = _deepcopy_dispatch.get(cls)
    if copier:
        y = copier(x, memo)
    else:
        try:
            issc = issubclass(cls, type)
        except TypeError: # cls is not a class (old Boost; see SF #502085)
            issc = 0
        if issc:
            y = _deepcopy_atomic(x, memo)
        else:
            copier = getattr(x, "__deepcopy__", None)
            if copier:
                y = copier(memo)
            else:
                reductor = dispatch_table.get(cls)
                if reductor:
                    rv = reductor(x)
                else:
                    reductor = getattr(x, "__reduce_ex__", None)
                    if reductor:
                        rv = reductor(2)
                    else:
                        reductor = getattr(x, "__reduce__", None)
                        if reductor:
                            rv = reductor()
                        else:
                            raise Error(
                                "un(deep)copyable object of type %s" % cls)
                y = _reconstruct(x, rv, 1, memo)

    memo[d] = y
    _keep_alive(x, memo) # Make sure x lives at least as long as d
    return y

Response: Deep copy operation on arbitrary Python objects.


Instruction:
How would you code a function in Python 3 to
keep a reference to the object x in the memo.

Response:
def _keep_alive(x, memo):
    """Keeps a reference to the object x in the memo.

    Because we remember objects by their id, we have
    to assure that possibly temporary objects are kept
    alive by referencing them.
    We store a reference at the id of the memo, which should
    normally not be used unless someone tries to deepcopy
    the memo itself...
    """
    try:
        memo[id(memo)].append(x)
    except KeyError:
        # aha, this is the first one :-)
        memo[id(memo)]=[x]