Instruction:
Write a Python 3 function that can
check for printf related issues.

Response:
def CheckPrintf(filename, clean_lines, linenum, error):
  """Check for printf related issues.

  Args:
    filename: The name of the current file.
    clean_lines: A CleansedLines instance containing the file.
    linenum: The number of the line to check.
    error: The function to call with any errors found.
  """
  line = clean_lines.elided[linenum]

  # When snprintf is used, the second argument shouldn't be a literal.
  match = Search(r'snprintf\s*\(([^,]*),\s*([0-9]*)\s*,', line)
  if match and match.group(2) != '0':
    # If 2nd arg is zero, snprintf is used to calculate size.
    error(filename, linenum, 'runtime/printf', 3,
          'If you can, use sizeof(%s) instead of %s as the 2nd arg '
          'to snprintf.' % (match.group(1), match.group(2)))

  # Check if some verboten C functions are being used.
  if Search(r'\bsprintf\s*\(', line):
    error(filename, linenum, 'runtime/printf', 5,
          'Never use sprintf. Use snprintf instead.')
  match = Search(r'\b(strcpy|strcat)\s*\(', line)
  if match:
    error(filename, linenum, 'runtime/printf', 4,
          'Almost always, snprintf is better than %s' % match.group(1))


Instruction:
Here you have a function in Python 3, explain what it does
def IsDerivedFunction(clean_lines, linenum):
  # Scan back a few lines for start of current function
  for i in xrange(linenum, max(-1, linenum - 10), -1):
    match = Match(r'^([^()]*\w+)\(', clean_lines.elided[i])
    if match:
      # Look for "override" after the matching closing parenthesis
      line, _, closing_paren = CloseExpression(
          clean_lines, i, len(match.group(1)))
      return (closing_paren >= 0 and
              Search(r'\boverride\b', line[closing_paren:]))
  return False

Response: Checks if the current line contains an inherited function.


Instruction:
Implement a Python 3 function for
checking if the current line contains an out - of - line method definition.

Response:
def IsOutOfLineMethodDefinition(clean_lines, linenum):
  """Check if current line contains an out-of-line method definition.

  Args:
    clean_lines: A CleansedLines instance containing the file.
    linenum: The number of the line to check.
  Returns:
    True if current line contains an out-of-line method definition.
  """
  # Scan back a few lines for start of current function
  for i in xrange(linenum, max(-1, linenum - 10), -1):
    if Match(r'^([^()]*\w+)\(', clean_lines.elided[i]):
      return Match(r'^[^()]*\w+::\w+\(', clean_lines.elided[i]) is not None
  return False


Instruction:
Can you tell what is the following Python 3 function doing
def IsInitializerList(clean_lines, linenum):
  for i in xrange(linenum, 1, -1):
    line = clean_lines.elided[i]
    if i == linenum:
      remove_function_body = Match(r'^(.*)\{\s*$', line)
      if remove_function_body:
        line = remove_function_body.group(1)

    if Search(r'\s:\s*\w+[({]', line):
      # A lone colon tend to indicate the start of a constructor
      # initializer list.  It could also be a ternary operator, which
      # also tend to appear in constructor initializer lists as
      # opposed to parameter lists.
      return True
    if Search(r'\}\s*,\s*$', line):
      # A closing brace followed by a comma is probably the end of a
      # brace-initialized member in constructor initializer list.
      return True
    if Search(r'[{};]\s*$', line):
      # Found one of the following:
      # - A closing brace or semicolon, probably the end of the previous
      #   function.
      # - An opening brace, probably the start of current class or namespace.
      #
      # Current line is probably not inside an initializer list since
      # we saw one of those things without seeing the starting colon.
      return False

  # Got to the beginning of the file without seeing the start of
  # constructor initializer list.
  return False

Response: Checks if current line is inside constructor initializer list.


Instruction:
Explain what the following Python 3 code does
def CheckForNonConstReference(filename, clean_lines, linenum,
                              nesting_state, error):
  """Check for non-const references.

  Separate from CheckLanguage since it scans backwards from current
  line, instead of scanning forward.

  Args:
    filename: The name of the current file.
    clean_lines: A CleansedLines instance containing the file.
    linenum: The number of the line to check.
    nesting_state: A NestingState instance which maintains information about
                   the current stack of nested blocks being parsed.
    error: The function to call with any errors found.
  """
  # Do nothing if there is no '&' on current line.
  line = clean_lines.elided[linenum]
  if '&' not in line:
    return

  # If a function is inherited, current function doesn't have much of
  # a choice, so any non-const references should not be blamed on
  # derived function.
  if IsDerivedFunction(clean_lines, linenum):
    return

  # Don't warn on out-of-line method definitions, as we would warn on the
  # in-line declaration, if it isn't marked with 'override'.
  if IsOutOfLineMethodDefinition(clean_lines, linenum):
    return

  # Long type names may be broken across multiple lines, usually in one
  # of these forms:
  #   LongType
  #       ::LongTypeContinued &identifier
  #   LongType::
  #       LongTypeContinued &identifier
  #   LongType<
  #       ...>::LongTypeContinued &identifier
  #
  # If we detected a type split across two lines, join the previous
  # line to current line so that we can match const references
  # accordingly.
  #
  # Note that this only scans back one line, since scanning back
  # arbitrary number of lines would be expensive.  If you have a type
  # that spans more than 2 lines, please use a typedef.
  if linenum > 1:
    previous = None
    if Match(r'\s*::(?:[\w<>]|::)+\s*&\s*\S', line):
      # previous_line\n + ::current_line
      previous = Search(r'\b((?:const\s*)?(?:[\w<>]|::)+[\w<>])\s*$',
                        clean_lines.elided[linenum - 1])
    elif Match(r'\s*[a-zA-Z_]([\w<>]|::)+\s*&\s*\S', line):
      # previous_line::\n + current_line
      previous = Search(r'\b((?:const\s*)?(?:[\w<>]|::)+::)\s*$',
                        clean_lines.elided[linenum - 1])
    if previous:
      line = previous.group(1) + line.lstrip()
    else:
      # Check for templated parameter that is split across multiple lines
      endpos = line.rfind('>')
      if endpos > -1:
        (_, startline, startpos) = ReverseCloseExpression(
            clean_lines, linenum, endpos)
        if startpos > -1 and startline < linenum:
          # Found the matching < on an earlier line, collect all
          # pieces up to current line.
          line = ''
          for i in xrange(startline, linenum + 1):
            line += clean_lines.elided[i].strip()

  # Check for non-const references in function parameters.  A single '&' may
  # found in the following places:
  #   inside expression: binary & for bitwise AND
  #   inside expression: unary & for taking the address of something
  #   inside declarators: reference parameter
  # We will exclude the first two cases by checking that we are not inside a
  # function body, including one that was just introduced by a trailing '{'.
  # TODO(unknown): Doesn't account for 'catch(Exception& e)' [rare].
  if (nesting_state.previous_stack_top and
      not (isinstance(nesting_state.previous_stack_top, _ClassInfo) or
           isinstance(nesting_state.previous_stack_top, _NamespaceInfo))):
    # Not at toplevel, not within a class, and not within a namespace
    return

  # Avoid initializer lists.  We only need to scan back from the
  # current line for something that starts with ':'.
  #
  # We don't need to check the current line, since the '&' would
  # appear inside the second set of parentheses on the current line as
  # opposed to the first set.
  if linenum > 0:
    for i in xrange(linenum - 1, max(0, linenum - 10), -1):
      previous_line = clean_lines.elided[i]
      if not Search(r'[),]\s*$', previous_line):
        break
      if Match(r'^\s*:\s+\S', previous_line):
        return

  # Avoid preprocessors
  if Search(r'\\\s*$', line):
    return

  # Avoid constructor initializer lists
  if IsInitializerList(clean_lines, linenum):
    return

  # We allow non-const references in a few standard places, like functions
  # called "swap()" or iostream operators like "<<" or ">>".  Do not check
  # those function parameters.
  #
  # We also accept & in static_assert, which looks like a function but
  # it's actually a declaration expression.
  whitelisted_functions = (r'(?:[sS]wap(?:<\w:+>)?|'
                           r'operator\s*[<>][<>]|'
                           r'static_assert|COMPILE_ASSERT'
                           r')\s*\(')
  if Search(whitelisted_functions, line):
    return
  elif not Search(r'\S+\([^)]*$', line):
    # Don't see a whitelisted function on this line.  Actually we
    # didn't see any function name on this line, so this is likely a
    # multi-line parameter list.  Try a bit harder to catch this case.
    for i in xrange(2):
      if (linenum > i and
          Search(whitelisted_functions, clean_lines.elided[linenum - i - 1])):
        return

  decls = ReplaceAll(r'{[^}]*}', ' ', line)  # exclude function body
  for parameter in re.findall(_RE_PATTERN_REF_PARAM, decls):
    if (not Match(_RE_PATTERN_CONST_REF_PARAM, parameter) and
        not Match(_RE_PATTERN_REF_STREAM_PARAM, parameter)):
      error(filename, linenum, 'runtime/references', 2,
            'Is this a non-const reference? '
            'If so, make const or use a pointer: ' +
            ReplaceAll(' *<', '<', parameter))

Response: Checks if a given line is a non - constant reference.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def CheckCasts(filename, clean_lines, linenum, error):
  line = clean_lines.elided[linenum]

  # Check to see if they're using an conversion function cast.
  # I just try to capture the most common basic types, though there are more.
  # Parameterless conversion functions, such as bool(), are allowed as they are
  # probably a member operator declaration or default constructor.
  match = Search(
      r'(\bnew\s+(?:const\s+)?|\S<\s*(?:const\s+)?)?\b'
      r'(int|float|double|bool|char|int32|uint32|int64|uint64)'
      r'(\([^)].*)', line)
  expecting_function = ExpectingFunctionArgs(clean_lines, linenum)
  if match and not expecting_function:
    matched_type = match.group(2)

    # matched_new_or_template is used to silence two false positives:
    # - New operators
    # - Template arguments with function types
    #
    # For template arguments, we match on types immediately following
    # an opening bracket without any spaces.  This is a fast way to
    # silence the common case where the function type is the first
    # template argument.  False negative with less-than comparison is
    # avoided because those operators are usually followed by a space.
    #
    #   function<double(double)>   // bracket + no space = false positive
    #   value < double(42)         // bracket + space = true positive
    matched_new_or_template = match.group(1)

    # Avoid arrays by looking for brackets that come after the closing
    # parenthesis.
    if Match(r'\([^()]+\)\s*\[', match.group(3)):
      return

    # Other things to ignore:
    # - Function pointers
    # - Casts to pointer types
    # - Placement new
    # - Alias declarations
    matched_funcptr = match.group(3)
    if (matched_new_or_template is None and
        not (matched_funcptr and
             (Match(r'\((?:[^() ]+::\s*\*\s*)?[^() ]+\)\s*\(',
                    matched_funcptr) or
              matched_funcptr.startswith('(*)'))) and
        not Match(r'\s*using\s+\S+\s*=\s*' + matched_type, line) and
        not Search(r'new\(\S+\)\s*' + matched_type, line)):
      error(filename, linenum, 'readability/casting', 4,
            'Using deprecated casting style.  '
            'Use static_cast<%s>(...) instead' %
            matched_type)

  if not expecting_function:
    CheckCStyleCast(filename, clean_lines, linenum, 'static_cast',
                    r'\((int|float|double|bool|char|u?int(16|32|64))\)', error)

  # This doesn't catch all cases. Consider (const char * const)"hello".
  #
  # (char *) "foo" should always be a const_cast (reinterpret_cast won't
  # compile).
  if CheckCStyleCast(filename, clean_lines, linenum, 'const_cast',
                     r'\((char\s?\*+\s?)\)\s*"', error):
    pass
  else:
    # Check pointer casts for other than string constants
    CheckCStyleCast(filename, clean_lines, linenum, 'reinterpret_cast',
                    r'\((\w+\s?\*+\s?)\)', error)

  # In addition, we look for people taking the address of a cast.  This
  # is dangerous -- casts can assign to temporaries, so the pointer doesn't
  # point where you think.
  #
  # Some non-identifier character is required before the '&' for the
  # expression to be recognized as a cast.  These are casts:
  #   expression = &static_cast<int*>(temporary());
  #   function(&(int*)(temporary()));
  #
  # This is not a cast:
  #   reference_type&(int* function_param);
  match = Search(
      r'(?:[^\w]&\(([^)*][^)]*)\)[\w(])|'
      r'(?:[^\w]&(static|dynamic|down|reinterpret)_cast\b)', line)
  if match:
    # Try a better error message when the & is bound to something
    # dereferenced by the casted pointer, as opposed to the casted
    # pointer itself.
    parenthesis_error = False
    match = Match(r'^(.*&(?:static|dynamic|down|reinterpret)_cast\b)<', line)
    if match:
      _, y1, x1 = CloseExpression(clean_lines, linenum, len(match.group(1)))
      if x1 >= 0 and clean_lines.elided[y1][x1] == '(':
        _, y2, x2 = CloseExpression(clean_lines, y1, x1)
        if x2 >= 0:
          extended_line = clean_lines.elided[y2][x2:]
          if y2 < clean_lines.NumLines() - 1:
            extended_line += clean_lines.elided[y2 + 1]
          if Match(r'\s*(?:->|\[)', extended_line):
            parenthesis_error = True

    if parenthesis_error:
      error(filename, linenum, 'readability/casting', 4,
            ('Are you taking an address of something dereferenced '
             'from a cast?  Wrapping the dereferenced expression in '
             'parentheses will make the binding more obvious'))
    else:
      error(filename, linenum, 'runtime/casting', 4,
            ('Are you taking an address of a cast?  '
             'This is dangerous: could be a temp var.  '
             'Take the address before doing the cast, rather than after'))

Response: Checks if the current file contains a valid conversion function cast.


Instruction:
Can you create a Python 3 function that
checks if a C - style cast is found.

Response:
def CheckCStyleCast(filename, clean_lines, linenum, cast_type, pattern, error):
  """Checks for a C-style cast by looking for the pattern.

  Args:
    filename: The name of the current file.
    clean_lines: A CleansedLines instance containing the file.
    linenum: The number of the line to check.
    cast_type: The string for the C++ cast to recommend.  This is either
      reinterpret_cast, static_cast, or const_cast, depending.
    pattern: The regular expression used to find C-style casts.
    error: The function to call with any errors found.

  Returns:
    True if an error was emitted.
    False otherwise.
  """
  line = clean_lines.elided[linenum]
  match = Search(pattern, line)
  if not match:
    return False

  # Exclude lines with keywords that tend to look like casts
  context = line[0:match.start(1) - 1]
  if Match(r'.*\b(?:sizeof|alignof|alignas|[_A-Z][_A-Z0-9]*)\s*$', context):
    return False

  # Try expanding current context to see if we one level of
  # parentheses inside a macro.
  if linenum > 0:
    for i in xrange(linenum - 1, max(0, linenum - 5), -1):
      context = clean_lines.elided[i] + context
  if Match(r'.*\b[_A-Z][_A-Z0-9]*\s*\((?:\([^()]*\)|[^()])*$', context):
    return False

  # operator++(int) and operator--(int)
  if context.endswith(' operator++') or context.endswith(' operator--'):
    return False

  # A single unnamed argument for a function tends to look like old style cast.
  # If we see those, don't issue warnings for deprecated casts.
  remainder = line[match.end(0):]
  if Match(r'^\s*(?:;|const\b|throw\b|final\b|override\b|[=>{),]|->)',
           remainder):
    return False

  # At this point, all that should be left is actual casts.
  error(filename, linenum, 'readability/casting', 4,
        'Using C-style cast.  Use %s<%s>(...) instead' %
        (cast_type, match.group(1)))

  return True


Instruction:
Make a summary of the following Python 3 code
def ExpectingFunctionArgs(clean_lines, linenum):
  line = clean_lines.elided[linenum]
  return (Match(r'^\s*MOCK_(CONST_)?METHOD\d+(_T)?\(', line) or
          (linenum >= 2 and
           (Match(r'^\s*MOCK_(?:CONST_)?METHOD\d+(?:_T)?\((?:\S+,)?\s*$',
                  clean_lines.elided[linenum - 1]) or
            Match(r'^\s*MOCK_(?:CONST_)?METHOD\d+(?:_T)?\(\s*$',
                  clean_lines.elided[linenum - 2]) or
            Search(r'\bstd::m?function\s*\<\s*$',
                   clean_lines.elided[linenum - 1]))))

Response: Checks whether the line at linenum is inside a function type argument list.


Instruction:
How would you implement a function in Python 3 that
checks if two files belong to the same module.

Response:
def FilesBelongToSameModule(filename_cc, filename_h):
  """Check if these two filenames belong to the same module.

  The concept of a 'module' here is a as follows:
  foo.h, foo-inl.h, foo.cc, foo_test.cc and foo_unittest.cc belong to the
  same 'module' if they are in the same directory.
  some/path/public/xyzzy and some/path/internal/xyzzy are also considered
  to belong to the same module here.

  If the filename_cc contains a longer path than the filename_h, for example,
  '/absolute/path/to/base/sysinfo.cc', and this file would include
  'base/sysinfo.h', this function also produces the prefix needed to open the
  header. This is used by the caller of this function to more robustly open the
  header file. We don't have access to the real include paths in this context,
  so we need this guesswork here.

  Known bugs: tools/base/bar.cc and base/bar.h belong to the same module
  according to this implementation. Because of this, this function gives
  some false positives. This should be sufficiently rare in practice.

  Args:
    filename_cc: is the path for the source (e.g. .cc) file
    filename_h: is the path for the header path

  Returns:
    Tuple with a bool and a string:
    bool: True if filename_cc and filename_h belong to the same module.
    string: the additional prefix needed to open the header file.
  """
  fileinfo_cc = FileInfo(filename_cc)
  if not fileinfo_cc.Extension().lstrip('.') in GetNonHeaderExtensions():
    return (False, '')

  fileinfo_h = FileInfo(filename_h)
  if not fileinfo_h.Extension().lstrip('.') in GetHeaderExtensions():
    return (False, '')

  filename_cc = filename_cc[:-(len(fileinfo_cc.Extension()))]
  matched_test_suffix = Search(_TEST_FILE_SUFFIX, fileinfo_cc.BaseName())
  if matched_test_suffix:
    filename_cc = filename_cc[:-len(matched_test_suffix.group(1))]

  filename_cc = filename_cc.replace('/public/', '/')
  filename_cc = filename_cc.replace('/internal/', '/')

  filename_h = filename_h[:-(len(fileinfo_h.Extension()))]
  if filename_h.endswith('-inl'):
    filename_h = filename_h[:-len('-inl')]
  filename_h = filename_h.replace('/public/', '/')
  filename_h = filename_h.replace('/internal/', '/')

  files_belong_to_same_module = filename_cc.endswith(filename_h)
  common_path = ''
  if files_belong_to_same_module:
    common_path = filename_cc[:-len(filename_h)]
  return files_belong_to_same_module, common_path


Instruction:
Given the following Python 3 function, write the documentation
def UpdateIncludeState(filename, include_dict, io=codecs):
  headerfile = None
  try:
    headerfile = io.open(filename, 'r', 'utf8', 'replace')
  except IOError:
    return False
  linenum = 0
  for line in headerfile:
    linenum += 1
    clean_line = CleanseComments(line)
    match = _RE_PATTERN_INCLUDE.search(clean_line)
    if match:
      include = match.group(2)
      include_dict.setdefault(include, linenum)
  return True

Response: Updates the include_dict with the lines found in the file.


Instruction:
Make a summary of the following Python 3 code
def CheckMakePairUsesDeduction(filename, clean_lines, linenum, error):
  line = clean_lines.elided[linenum]
  match = _RE_PATTERN_EXPLICIT_MAKEPAIR.search(line)
  if match:
    error(filename, linenum, 'build/explicit_make_pair',
          4,  # 4 = high confidence
          'For C++11-compatibility, omit template arguments from make_pair'
          ' OR use pair directly OR if appropriate, construct a pair directly')

Response: Checks that make_pair s template arguments are deduced.


Instruction:
Explain what the following Python 3 code does
def CheckRedundantVirtual(filename, clean_lines, linenum, error):
  # Look for "virtual" on current line.
  line = clean_lines.elided[linenum]
  virtual = Match(r'^(.*)(\bvirtual\b)(.*)$', line)
  if not virtual: return

  # Ignore "virtual" keywords that are near access-specifiers.  These
  # are only used in class base-specifier and do not apply to member
  # functions.
  if (Search(r'\b(public|protected|private)\s+$', virtual.group(1)) or
      Match(r'^\s+(public|protected|private)\b', virtual.group(3))):
    return

  # Ignore the "virtual" keyword from virtual base classes.  Usually
  # there is a column on the same line in these cases (virtual base
  # classes are rare in google3 because multiple inheritance is rare).
  if Match(r'^.*[^:]:[^:].*$', line): return

  # Look for the next opening parenthesis.  This is the start of the
  # parameter list (possibly on the next line shortly after virtual).
  # TODO(unknown): doesn't work if there are virtual functions with
  # decltype() or other things that use parentheses, but csearch suggests
  # that this is rare.
  end_col = -1
  end_line = -1
  start_col = len(virtual.group(2))
  for start_line in xrange(linenum, min(linenum + 3, clean_lines.NumLines())):
    line = clean_lines.elided[start_line][start_col:]
    parameter_list = Match(r'^([^(]*)\(', line)
    if parameter_list:
      # Match parentheses to find the end of the parameter list
      (_, end_line, end_col) = CloseExpression(
          clean_lines, start_line, start_col + len(parameter_list.group(1)))
      break
    start_col = 0

  if end_col < 0:
    return  # Couldn't find end of parameter list, give up

  # Look for "override" or "final" after the parameter list
  # (possibly on the next few lines).
  for i in xrange(end_line, min(end_line + 3, clean_lines.NumLines())):
    line = clean_lines.elided[i][end_col:]
    match = Search(r'\b(override|final)\b', line)
    if match:
      error(filename, linenum, 'readability/inheritance', 4,
            ('"virtual" is redundant since function is '
             'already declared as "%s"' % match.group(1)))

    # Set end_col to check whole lines after we are done with the
    # first line.
    end_col = 0
    if Search(r'[^\w]\s*$', line):
      break

Response: Checks if a line contains a redundant virtual function - specifier.


Instruction:
Write a Python 3 function that can
check if a line contains a redundant override or final virt - specifier.

Response:
def CheckRedundantOverrideOrFinal(filename, clean_lines, linenum, error):
  """Check if line contains a redundant "override" or "final" virt-specifier.

  Args:
    filename: The name of the current file.
    clean_lines: A CleansedLines instance containing the file.
    linenum: The number of the line to check.
    error: The function to call with any errors found.
  """
  # Look for closing parenthesis nearby.  We need one to confirm where
  # the declarator ends and where the virt-specifier starts to avoid
  # false positives.
  line = clean_lines.elided[linenum]
  declarator_end = line.rfind(')')
  if declarator_end >= 0:
    fragment = line[declarator_end:]
  else:
    if linenum > 1 and clean_lines.elided[linenum - 1].rfind(')') >= 0:
      fragment = line
    else:
      return

  # Check that at most one of "override" or "final" is present, not both
  if Search(r'\boverride\b', fragment) and Search(r'\bfinal\b', fragment):
    error(filename, linenum, 'readability/inheritance', 4,
          ('"override" is redundant since function is '
           'already declared as "final"'))


Instruction:
Create a Python 3 function to
check that the new block is in a namespace.

Response:
def IsBlockInNameSpace(nesting_state, is_forward_declaration):
  """Checks that the new block is directly in a namespace.

  Args:
    nesting_state: The _NestingState object that contains info about our state.
    is_forward_declaration: If the class is a forward declared class.
  Returns:
    Whether or not the new block is directly in a namespace.
  """
  if is_forward_declaration:
    return len(nesting_state.stack) >= 1 and (
      isinstance(nesting_state.stack[-1], _NamespaceInfo))

  return (len(nesting_state.stack) > 1 and
          nesting_state.stack[-1].check_namespace_indentation and
          isinstance(nesting_state.stack[-2], _NamespaceInfo))


Instruction:
Given the following Python 3 function, write the documentation
def ShouldCheckNamespaceIndentation(nesting_state, is_namespace_indent_item,
                                    raw_lines_no_comments, linenum):
  """This method determines if we should apply our namespace indentation check.

  Args:
    nesting_state: The current nesting state.
    is_namespace_indent_item: If we just put a new class on the stack, True.
      If the top of the stack is not a class, or we did not recently
      add the class, False.
    raw_lines_no_comments: The lines without the comments.
    linenum: The current line number we are processing.

  Returns:
    True if we should apply our namespace indentation check. Currently, it
    only works for classes and namespaces inside of a namespace.
  """

  is_forward_declaration = IsForwardClassDeclaration(raw_lines_no_comments,
                                                     linenum)

  if not (is_namespace_indent_item or is_forward_declaration):
    return False

  # If we are in a macro, we do not want to check the namespace indentation.
  if IsMacroDefinition(raw_lines_no_comments, linenum):
    return False

  return IsBlockInNameSpace(nesting_state, is_forward_declaration)

Response: This function determines if we should apply our namespace indentation check.


Instruction:
Write a Python 3 function for
flagging those C ++ 14 features that we restrict.

Response:
def FlagCxx14Features(filename, clean_lines, linenum, error):
  """Flag those C++14 features that we restrict.

  Args:
    filename: The name of the current file.
    clean_lines: A CleansedLines instance containing the file.
    linenum: The number of the line to check.
    error: The function to call with any errors found.
  """
  line = clean_lines.elided[linenum]

  include = Match(r'\s*#\s*include\s+[<"]([^<"]+)[">]', line)

  # Flag unapproved C++14 headers.
  if include and include.group(1) in ('scoped_allocator', 'shared_mutex'):
    error(filename, linenum, 'build/c++14', 5,
          ('<%s> is an unapproved C++14 header.') % include.group(1))


Instruction:
How would you explain what the following Python 3 function does
def ProcessFileData(filename, file_extension, lines, error,
                    extra_check_functions=None):
  """Performs lint checks and reports any errors to the given error function.

  Args:
    filename: Filename of the file that is being processed.
    file_extension: The extension (dot not included) of the file.
    lines: An array of strings, each representing a line of the file, with the
           last element being empty if the file is terminated with a newline.
    error: A callable to which errors are reported, which takes 4 arguments:
           filename, line number, error level, and message
    extra_check_functions: An array of additional check functions that will be
                           run on each source line. Each function takes 4
                           arguments: filename, clean_lines, line, error
  """
  lines = (['// marker so line numbers and indices both start at 1'] + lines +
           ['// marker so line numbers end in a known way'])

  include_state = _IncludeState()
  function_state = _FunctionState()
  nesting_state = NestingState()

  ResetNolintSuppressions()

  CheckForCopyright(filename, lines, error)
  ProcessGlobalSuppresions(lines)
  RemoveMultiLineComments(filename, lines, error)
  clean_lines = CleansedLines(lines)

  if file_extension in GetHeaderExtensions():
    CheckForHeaderGuard(filename, clean_lines, error)

  for line in range(clean_lines.NumLines()):
    ProcessLine(filename, file_extension, clean_lines, line,
                include_state, function_state, nesting_state, error,
                extra_check_functions)
    FlagCxx11Features(filename, clean_lines, line, error)
  nesting_state.CheckCompletedBlocks(filename, error)

  CheckForIncludeWhatYouUse(filename, clean_lines, include_state, error)

  # Check that the .cc file has included its header if it exists.
  if _IsSourceExtension(file_extension):
    CheckHeaderFileIncluded(filename, include_state, error)

  # We check here rather than inside ProcessLine so that we see raw
  # lines rather than "cleaned" lines.
  CheckForBadCharacters(filename, lines, error)

  CheckForNewlineAtEOF(filename, lines, error)

Response: Processes the file data and checks for errors.


Instruction:
How would you implement a function in Python 3 that
loads the configuration files and processes the config overrides.

Response:
def ProcessConfigOverrides(filename):
  """ Loads the configuration files and processes the config overrides.

  Args:
    filename: The name of the file being processed by the linter.

  Returns:
    False if the current |filename| should not be processed further.
  """

  abs_filename = os.path.abspath(filename)
  cfg_filters = []
  keep_looking = True
  while keep_looking:
    abs_path, base_name = os.path.split(abs_filename)
    if not base_name:
      break  # Reached the root directory.

    cfg_file = os.path.join(abs_path, "CPPLINT.cfg")
    abs_filename = abs_path
    if not os.path.isfile(cfg_file):
      continue

    try:
      with open(cfg_file) as file_handle:
        for line in file_handle:
          line, _, _ = line.partition('#')  # Remove comments.
          if not line.strip():
            continue

          name, _, val = line.partition('=')
          name = name.strip()
          val = val.strip()
          if name == 'set noparent':
            keep_looking = False
          elif name == 'filter':
            cfg_filters.append(val)
          elif name == 'exclude_files':
            # When matching exclude_files pattern, use the base_name of
            # the current file name or the directory name we are processing.
            # For example, if we are checking for lint errors in /foo/bar/baz.cc
            # and we found the .cfg file at /foo/CPPLINT.cfg, then the config
            # file's "exclude_files" filter is meant to be checked against "bar"
            # and not "baz" nor "bar/baz.cc".
            if base_name:
              pattern = re.compile(val)
              if pattern.match(base_name):
                _cpplint_state.PrintInfo('Ignoring "%s": file excluded by '
                    '"%s". File path component "%s" matches pattern "%s"\n' %
                    (filename, cfg_file, base_name, val))
                return False
          elif name == 'linelength':
            global _line_length
            try:
                _line_length = int(val)
            except ValueError:
                _cpplint_state.PrintError('Line length must be numeric.')
          elif name == 'extensions':
              global _valid_extensions
              try:
                  extensions = [ext.strip() for ext in val.split(',')]
                  _valid_extensions = set(extensions)
              except ValueError:
                  sys.stderr.write('Extensions should be a comma-separated list of values;'
                                   'for example: extensions=hpp,cpp\n'
                                   'This could not be parsed: "%s"' % (val,))
          elif name == 'headers':
              global _header_extensions
              try:
                  extensions = [ext.strip() for ext in val.split(',')]
                  _header_extensions = set(extensions)
              except ValueError:
                  sys.stderr.write('Extensions should be a comma-separated list of values;'
                                   'for example: extensions=hpp,cpp\n'
                                   'This could not be parsed: "%s"' % (val,))
          elif name == 'root':
            global _root
            _root = val
          else:
            _cpplint_state.PrintError(
                'Invalid configuration option (%s) in file %s\n' %
                (name, cfg_file))

    except IOError:
      _cpplint_state.PrintError(
          "Skipping config file '%s': Can't open for reading\n" % cfg_file)
      keep_looking = False

  # Apply all the accumulated filters in reverse order (top-level directory
  # config options having the least priority).
  for cfg_filter in reversed(cfg_filters):
     _AddFilters(cfg_filter)

  return True


Instruction:
How would you explain what the following Python 3 function does
def ProcessFile(filename, vlevel, extra_check_functions=None):

  _SetVerboseLevel(vlevel)
  _BackupFilters()

  if not ProcessConfigOverrides(filename):
    _RestoreFilters()
    return

  lf_lines = []
  crlf_lines = []
  try:
    # Support the UNIX convention of using "-" for stdin.  Note that
    # we are not opening the file with universal newline support
    # (which codecs doesn't support anyway), so the resulting lines do
    # contain trailing '\r' characters if we are reading a file that
    # has CRLF endings.
    # If after the split a trailing '\r' is present, it is removed
    # below.
    if filename == '-':
      lines = codecs.StreamReaderWriter(sys.stdin,
                                        codecs.getreader('utf8'),
                                        codecs.getwriter('utf8'),
                                        'replace').read().split('\n')
    else:
      lines = codecs.open(filename, 'r', 'utf8', 'replace').read().split('\n')

    # Remove trailing '\r'.
    # The -1 accounts for the extra trailing blank line we get from split()
    for linenum in range(len(lines) - 1):
      if lines[linenum].endswith('\r'):
        lines[linenum] = lines[linenum].rstrip('\r')
        crlf_lines.append(linenum + 1)
      else:
        lf_lines.append(linenum + 1)

  except IOError:
    _cpplint_state.PrintError(
        "Skipping input '%s': Can't open for reading\n" % filename)
    _RestoreFilters()
    return

  # Note, if no dot is found, this will give the entire filename as the ext.
  file_extension = filename[filename.rfind('.') + 1:]

  # When reading from stdin, the extension is unknown, so no cpplint tests
  # should rely on the extension.
  if filename != '-' and file_extension not in GetAllExtensions():
    # bazel 0.5.1> uses four distinct generated files that gives a warning
    # we suppress the warning for these files
    bazel_gen_files = set([
        "external/local_config_cc/libtool",
        "external/local_config_cc/make_hashed_objlist.py",
        "external/local_config_cc/wrapped_ar",
        "external/local_config_cc/wrapped_clang",
        "external/local_config_cc/xcrunwrapper.sh",
    ])
    if not filename in bazel_gen_files:
       _cpplint_state.PrintError('Ignoring %s; not a valid file name '
                                 '(%s)\n' % (filename, ', '.join(GetAllExtensions())))
  else:
    ProcessFileData(filename, file_extension, lines, Error,
                    extra_check_functions)

    # If end-of-line sequences are a mix of LF and CR-LF, issue
    # warnings on the lines with CR.
    #
    # Don't issue any warnings if all lines are uniformly LF or CR-LF,
    # since critique can handle these just fine, and the style guide
    # doesn't dictate a particular end of line sequence.
    #
    # We can't depend on os.linesep to determine what the desired
    # end-of-line sequence should be, since that will return the
    # server-side end-of-line sequence.
    if lf_lines and crlf_lines:
      # Warn on every line with CR.  An alternative approach might be to
      # check whether the file is mostly CRLF or just LF, and warn on the
      # minority, we bias toward LF here since most tools prefer LF.
      for linenum in crlf_lines:
        Error(filename, linenum, 'whitespace/newline', 1,
              'Unexpected \\r (^M) found; better to use only \\n')

  _RestoreFilters()

Response: Processes a single file.


Instruction:
How would you code a function in Python 3 to
print a list of all the error - categories used by error messages.

Response:
def PrintCategories():
  """Prints a list of all the error-categories used by error messages.

  These are the categories used to filter messages via --filter.
  """
  sys.stderr.write(''.join('  %s\n' % cat for cat in _ERROR_CATEGORIES))
  sys.exit(0)


Instruction:
Given the following Python 3 function, write the documentation
def ParseArguments(args):
  try:
    (opts, filenames) = getopt.getopt(args, '', ['help', 'output=', 'verbose=',
                                                 'counting=',
                                                 'filter=',
                                                 'root=',
                                                 'repository=',
                                                 'linelength=',
                                                 'extensions=',
                                                 'exclude=',
                                                 'headers=',
                                                 'quiet',
                                                 'recursive'])
  except getopt.GetoptError:
    PrintUsage('Invalid arguments.')

  verbosity = _VerboseLevel()
  output_format = _OutputFormat()
  filters = ''
  counting_style = ''
  recursive = False

  for (opt, val) in opts:
    if opt == '--help':
      PrintUsage(None)
    elif opt == '--output':
      if val not in ('emacs', 'vs7', 'eclipse', 'junit'):
        PrintUsage('The only allowed output formats are emacs, vs7, eclipse '
                   'and junit.')
      output_format = val
    elif opt == '--verbose':
      verbosity = int(val)
    elif opt == '--filter':
      filters = val
      if not filters:
        PrintCategories()
    elif opt == '--counting':
      if val not in ('total', 'toplevel', 'detailed'):
        PrintUsage('Valid counting options are total, toplevel, and detailed')
      counting_style = val
    elif opt == '--root':
      global _root
      _root = val
    elif opt == '--repository':
      global _repository
      _repository = val
    elif opt == '--linelength':
      global _line_length
      try:
        _line_length = int(val)
      except ValueError:
        PrintUsage('Line length must be digits.')
    elif opt == '--exclude':
      global _excludes
      if not _excludes:
        _excludes = set()
      _excludes.update(glob.glob(val))
    elif opt == '--extensions':
      global _valid_extensions
      try:
        _valid_extensions = set(val.split(','))
      except ValueError:
          PrintUsage('Extensions must be comma seperated list.')
    elif opt == '--headers':
      global _header_extensions
      try:
          _header_extensions = set(val.split(','))
      except ValueError:
        PrintUsage('Extensions must be comma seperated list.')
    elif opt == '--recursive':
      recursive = True
    elif opt == '--quiet':
      global _quiet
      _quiet = True

  if not filenames:
    PrintUsage('No files were specified.')

  if recursive:
    filenames = _ExpandDirectories(filenames)

  if _excludes:
    filenames = _FilterExcludedFiles(filenames)

  _SetOutputFormat(output_format)
  _SetVerboseLevel(verbosity)
  _SetFilters(filters)
  _SetCountingStyle(counting_style)

  return filenames

Response: Parses the command line arguments and sets the output format and verbosity level as side - effectss.


Instruction:
Explain what the following Python 3 code does
def _ExpandDirectories(filenames):
  expanded = set()
  for filename in filenames:
      if not os.path.isdir(filename):
        expanded.add(filename)
        continue

      for root, _, files in os.walk(filename):
        for loopfile in files:
          fullname = os.path.join(root, loopfile)
          if fullname.startswith('.' + os.path.sep):
            fullname = fullname[len('.' + os.path.sep):]
          expanded.add(fullname)

  filtered = []
  for filename in expanded:
      if os.path.splitext(filename)[1][1:] in GetAllExtensions():
          filtered.append(filename)

  return filtered

Response: Searches a list of filenames and replaces directories with their full names.


Instruction:
Write a Python 3 script for
filtering out files listed in the exclude command line switch.

Response:
def _FilterExcludedFiles(filenames):
  """Filters out files listed in the --exclude command line switch. File paths
  in the switch are evaluated relative to the current working directory
  """
  exclude_paths = [os.path.abspath(f) for f in _excludes]
  return [f for f in filenames if os.path.abspath(f) not in exclude_paths]


Instruction:
Write a Python 3 function that can
return the line number of the previous occurrence of a header.

Response:
def FindHeader(self, header):
    """Check if a header has already been included.

    Args:
      header: header to check.
    Returns:
      Line number of previous occurrence, or -1 if the header has not
      been seen before.
    """
    for section_list in self.include_list:
      for f in section_list:
        if f[0] == header:
          return f[1]
    return -1


Instruction:
Can you implement a function in Python 3 that
resets the section checking for preprocessor directive.

Response:
def ResetSection(self, directive):
    """Reset section checking for preprocessor directive.

    Args:
      directive: preprocessor directive (e.g. "if", "else").
    """
    # The name of the current section.
    self._section = self._INITIAL_SECTION
    # The path of last found header.
    self._last_header = ''

    # Update list of includes.  Note that we never pop from the
    # include list.
    if directive in ('if', 'ifdef', 'ifndef'):
      self.include_list.append([])
    elif directive in ('else', 'elif'):
      self.include_list[-1] = []


Instruction:
Given the following Python 3 function, write the documentation
def IsInAlphabeticalOrder(self, clean_lines, linenum, header_path):
    # If previous section is different from current section, _last_header will
    # be reset to empty string, so it's always less than current header.
    #
    # If previous line was a blank line, assume that the headers are
    # intentionally sorted the way they are.
    if (self._last_header > header_path and
        Match(r'^\s*#\s*include\b', clean_lines.elided[linenum - 1])):
      return False
    return True

Response: Checks if a header is in alphabetical order.


Instruction:
Can you tell what is the following Python 3 function doing
def CheckNextIncludeOrder(self, header_type):
    error_message = ('Found %s after %s' %
                     (self._TYPE_NAMES[header_type],
                      self._SECTION_NAMES[self._section]))

    last_section = self._section

    if header_type == _C_SYS_HEADER:
      if self._section <= self._C_SECTION:
        self._section = self._C_SECTION
      else:
        self._last_header = ''
        return error_message
    elif header_type == _CPP_SYS_HEADER:
      if self._section <= self._CPP_SECTION:
        self._section = self._CPP_SECTION
      else:
        self._last_header = ''
        return error_message
    elif header_type == _LIKELY_MY_HEADER:
      if self._section <= self._MY_H_SECTION:
        self._section = self._MY_H_SECTION
      else:
        self._section = self._OTHER_H_SECTION
    elif header_type == _POSSIBLE_MY_HEADER:
      if self._section <= self._MY_H_SECTION:
        self._section = self._MY_H_SECTION
      else:
        # This will always be the fallback because we're not sure
        # enough that the header is associated with this file.
        self._section = self._OTHER_H_SECTION
    else:
      assert header_type == _OTHER_HEADER
      self._section = self._OTHER_H_SECTION

    if last_section != self._section:
      self._last_header = ''

    return ''

Response: Checks if the next include order of the given header type is in the right order.


Instruction:
How would you implement a function in Python 3 that
sets the module s verbosity.

Response:
def SetVerboseLevel(self, level):
    """Sets the module's verbosity, and returns the previous setting."""
    last_verbose_level = self.verbose_level
    self.verbose_level = level
    return last_verbose_level


Instruction:
Write a Python 3 script to
add more filters to the existing list of error - message filters.

Response:
def AddFilters(self, filters):
    """ Adds more filters to the existing list of error-message filters. """
    for filt in filters.split(','):
      clean_filt = filt.strip()
      if clean_filt:
        self.filters.append(clean_filt)
    for filt in self.filters:
      if not (filt.startswith('+') or filt.startswith('-')):
        raise ValueError('Every filter in --filters must start with + or -'
                         ' (%s does not)' % filt)


Instruction:
Write a Python 3 script to
increment the module s error statistic.

Response:
def IncrementErrorCount(self, category):
    """Bumps the module's error statistic."""
    self.error_count += 1
    if self.counting in ('toplevel', 'detailed'):
      if self.counting != 'detailed':
        category = category.split('/')[0]
      if category not in self.errors_by_category:
        self.errors_by_category[category] = 0
      self.errors_by_category[category] += 1


Instruction:
Can you generate the documentation for the following Python 3 function
def PrintErrorCounts(self):
    for category, count in sorted(iteritems(self.errors_by_category)):
      self.PrintInfo('Category \'%s\' errors found: %d\n' %
                       (category, count))
    if self.error_count > 0:
      self.PrintInfo('Total errors found: %d\n' % self.error_count)

Response: Print a summary of errors by category and the total.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def Begin(self, function_name):
    self.in_a_function = True
    self.lines_in_function = 0
    self.current_function = function_name

Response: Starts analyzing the function body.


Instruction:
How would you explain what the following Python 3 function does
def RepositoryName(self):
    fullname = self.FullName()

    if os.path.exists(fullname):
      project_dir = os.path.dirname(fullname)

      # If the user specified a repository path, it exists, and the file is
      # contained in it, use the specified repository path
      if _repository:
        repo = FileInfo(_repository).FullName()
        root_dir = project_dir
        while os.path.exists(root_dir):
          # allow case insensitive compare on Windows
          if os.path.normcase(root_dir) == os.path.normcase(repo):
            return os.path.relpath(fullname, root_dir).replace('\\', '/')
          one_up_dir = os.path.dirname(root_dir)
          if one_up_dir == root_dir:
            break
          root_dir = one_up_dir

      if os.path.exists(os.path.join(project_dir, ".svn")):
        # If there's a .svn file in the current directory, we recursively look
        # up the directory tree for the top of the SVN checkout
        root_dir = project_dir
        one_up_dir = os.path.dirname(root_dir)
        while os.path.exists(os.path.join(one_up_dir, ".svn")):
          root_dir = os.path.dirname(root_dir)
          one_up_dir = os.path.dirname(one_up_dir)

        prefix = os.path.commonprefix([root_dir, project_dir])
        return fullname[len(prefix) + 1:]

      # Not SVN <= 1.6? Try to find a git, hg, or svn top level directory by
      # searching up from the current path.
      root_dir = current_dir = os.path.dirname(fullname)
      while current_dir != os.path.dirname(current_dir):
        if (os.path.exists(os.path.join(current_dir, ".git")) or
            os.path.exists(os.path.join(current_dir, ".hg")) or
            os.path.exists(os.path.join(current_dir, ".svn"))):
          root_dir = current_dir
        current_dir = os.path.dirname(current_dir)

      if (os.path.exists(os.path.join(root_dir, ".git")) or
          os.path.exists(os.path.join(root_dir, ".hg")) or
          os.path.exists(os.path.join(root_dir, ".svn"))):
        prefix = os.path.commonprefix([root_dir, project_dir])
        return fullname[len(prefix) + 1:]

    # Don't know what to do; header guard warnings may be wrong...
    return fullname

Response: r Returns the full path name of the current repository.


Instruction:
Write a Python 3 function that can
split the file into the directory basename and extension.

Response:
def Split(self):
    """Splits the file into the directory, basename, and extension.

    For 'chrome/browser/browser.cc', Split() would
    return ('chrome/browser', 'browser', '.cc')

    Returns:
      A tuple of (directory, basename, extension).
    """

    googlename = self.RepositoryName()
    project, rest = os.path.split(googlename)
    return (project,) + os.path.splitext(rest)


Instruction:
Here you have a function in Python 3, explain what it does
def _CollapseStrings(elided):
    if _RE_PATTERN_INCLUDE.match(elided):
      return elided

    # Remove escaped characters first to make quote/single quote collapsing
    # basic.  Things that look like escaped characters shouldn't occur
    # outside of strings and chars.
    elided = _RE_PATTERN_CLEANSE_LINE_ESCAPES.sub('', elided)

    # Replace quoted strings and digit separators.  Both single quotes
    # and double quotes are processed in the same loop, otherwise
    # nested quotes wouldn't work.
    collapsed = ''
    while True:
      # Find the first quote character
      match = Match(r'^([^\'"]*)([\'"])(.*)$', elided)
      if not match:
        collapsed += elided
        break
      head, quote, tail = match.groups()

      if quote == '"':
        # Collapse double quoted strings
        second_quote = tail.find('"')
        if second_quote >= 0:
          collapsed += head + '""'
          elided = tail[second_quote + 1:]
        else:
          # Unmatched double quote, don't bother processing the rest
          # of the line since this is probably a multiline string.
          collapsed += elided
          break
      else:
        # Found single quote, check nearby text to eliminate digit separators.
        #
        # There is no special handling for floating point here, because
        # the integer/fractional/exponent parts would all be parsed
        # correctly as long as there are digits on both sides of the
        # separator.  So we are fine as long as we don't see something
        # like "0.'3" (gcc 4.9.0 will not allow this literal).
        if Search(r'\b(?:0[bBxX]?|[1-9])[0-9a-fA-F]*$', head):
          match_literal = Match(r'^((?:\'?[0-9a-zA-Z_])*)(.*)$', "'" + tail)
          collapsed += head + match_literal.group(1).replace("'", '')
          elided = match_literal.group(2)
        else:
          second_quote = tail.find('\'')
          if second_quote >= 0:
            collapsed += head + "''"
            elided = tail[second_quote + 1:]
          else:
            # Unmatched single quote
            collapsed += elided
            break

    return collapsed

Response: Collapse a string and chars on a line to a simple "" or '' blocks.


Instruction:
Can you write a function in Python 3 where it
checks if the end of a namespace comment is correct.

Response:
def CheckEnd(self, filename, clean_lines, linenum, error):
    """Check end of namespace comments."""
    line = clean_lines.raw_lines[linenum]

    # Check how many lines is enclosed in this namespace.  Don't issue
    # warning for missing namespace comments if there aren't enough
    # lines.  However, do apply checks if there is already an end of
    # namespace comment and it's incorrect.
    #
    # TODO(unknown): We always want to check end of namespace comments
    # if a namespace is large, but sometimes we also want to apply the
    # check if a short namespace contained nontrivial things (something
    # other than forward declarations).  There is currently no logic on
    # deciding what these nontrivial things are, so this check is
    # triggered by namespace size only, which works most of the time.
    if (linenum - self.starting_linenum < 10
        and not Match(r'^\s*};*\s*(//|/\*).*\bnamespace\b', line)):
      return

    # Look for matching comment at end of namespace.
    #
    # Note that we accept C style "/* */" comments for terminating
    # namespaces, so that code that terminate namespaces inside
    # preprocessor macros can be cpplint clean.
    #
    # We also accept stuff like "// end of namespace <name>." with the
    # period at the end.
    #
    # Besides these, we don't accept anything else, otherwise we might
    # get false negatives when existing comment is a substring of the
    # expected namespace.
    if self.name:
      # Named namespace
      if not Match((r'^\s*};*\s*(//|/\*).*\bnamespace\s+' +
                    re.escape(self.name) + r'[\*/\.\\\s]*$'),
                   line):
        error(filename, linenum, 'readability/namespace', 5,
              'Namespace should be terminated with "// namespace %s"' %
              self.name)
    else:
      # Anonymous namespace
      if not Match(r'^\s*};*\s*(//|/\*).*\bnamespace[\*/\.\\\s]*$', line):
        # If "// namespace anonymous" or "// anonymous namespace (more text)",
        # mention "// anonymous namespace" as an acceptable form
        if Match(r'^\s*}.*\b(namespace anonymous|anonymous namespace)\b', line):
          error(filename, linenum, 'readability/namespace', 5,
                'Anonymous namespace should be terminated with "// namespace"'
                ' or "// anonymous namespace"')
        else:
          error(filename, linenum, 'readability/namespace', 5,
                'Anonymous namespace should be terminated with "// namespace"')


Instruction:
Create a Python 3 function to
check if the current position is inside template argument list.

Response:
def InTemplateArgumentList(self, clean_lines, linenum, pos):
    """Check if current position is inside template argument list.

    Args:
      clean_lines: A CleansedLines instance containing the file.
      linenum: The number of the line to check.
      pos: position just after the suspected template argument.
    Returns:
      True if (linenum, pos) is inside template arguments.
    """
    while linenum < clean_lines.NumLines():
      # Find the earliest character that might indicate a template argument
      line = clean_lines.elided[linenum]
      match = Match(r'^[^{};=\[\]\.<>]*(.)', line[pos:])
      if not match:
        linenum += 1
        pos = 0
        continue
      token = match.group(1)
      pos += len(match.group(0))

      # These things do not look like template argument list:
      #   class Suspect {
      #   class Suspect x; }
      if token in ('{', '}', ';'): return False

      # These things look like template argument list:
      #   template <class Suspect>
      #   template <class Suspect = default_value>
      #   template <class Suspect[]>
      #   template <class Suspect...>
      if token in ('>', '=', '[', ']', '.'): return True

      # Check if token is an unmatched '<'.
      # If not, move on to the next character.
      if token != '<':
        pos += 1
        if pos >= len(line):
          linenum += 1
          pos = 0
        continue

      # We can't be sure if we just find a single '<', and need to
      # find the matching '>'.
      (_, end_line, end_pos) = CloseExpression(clean_lines, linenum, pos - 1)
      if end_pos < 0:
        # Not sure if template argument list or syntax error in file
        return False
      linenum = end_line
      pos = end_pos
    return False


Instruction:
Implement a function in Python 3 to
update the preprocessor stack.

Response:
def UpdatePreprocessor(self, line):
    """Update preprocessor stack.

    We need to handle preprocessors due to classes like this:
      #ifdef SWIG
      struct ResultDetailsPageElementExtensionPoint {
      #else
      struct ResultDetailsPageElementExtensionPoint : public Extension {
      #endif

    We make the following assumptions (good enough for most files):
    - Preprocessor condition evaluates to true from #if up to first
      #else/#elif/#endif.

    - Preprocessor condition evaluates to false from #else/#elif up
      to #endif.  We still perform lint checks on these lines, but
      these do not affect nesting stack.

    Args:
      line: current line to check.
    """
    if Match(r'^\s*#\s*(if|ifdef|ifndef)\b', line):
      # Beginning of #if block, save the nesting stack here.  The saved
      # stack will allow us to restore the parsing state in the #else case.
      self.pp_stack.append(_PreprocessorInfo(copy.deepcopy(self.stack)))
    elif Match(r'^\s*#\s*(else|elif)\b', line):
      # Beginning of #else block
      if self.pp_stack:
        if not self.pp_stack[-1].seen_else:
          # This is the first #else or #elif block.  Remember the
          # whole nesting stack up to this point.  This is what we
          # keep after the #endif.
          self.pp_stack[-1].seen_else = True
          self.pp_stack[-1].stack_before_else = copy.deepcopy(self.stack)

        # Restore the stack to how it was before the #if
        self.stack = copy.deepcopy(self.pp_stack[-1].stack_before_if)
      else:
        # TODO(unknown): unexpected #else, issue warning?
        pass
    elif Match(r'^\s*#\s*endif\b', line):
      # End of #if or #else blocks.
      if self.pp_stack:
        # If we saw an #else, we will need to restore the nesting
        # stack to its former state before the #else, otherwise we
        # will just continue from where we left off.
        if self.pp_stack[-1].seen_else:
          # Here we can just use a shallow copy since we are the last
          # reference to it.
          self.stack = self.pp_stack[-1].stack_before_else
        # Drop the corresponding #if
        self.pp_stack.pop()
      else:
        # TODO(unknown): unexpected #endif, issue warning?
        pass


Instruction:
Here you have a function in Python 3, explain what it does
def Update(self, filename, clean_lines, linenum, error):
    line = clean_lines.elided[linenum]

    # Remember top of the previous nesting stack.
    #
    # The stack is always pushed/popped and not modified in place, so
    # we can just do a shallow copy instead of copy.deepcopy.  Using
    # deepcopy would slow down cpplint by ~28%.
    if self.stack:
      self.previous_stack_top = self.stack[-1]
    else:
      self.previous_stack_top = None

    # Update pp_stack
    self.UpdatePreprocessor(line)

    # Count parentheses.  This is to avoid adding struct arguments to
    # the nesting stack.
    if self.stack:
      inner_block = self.stack[-1]
      depth_change = line.count('(') - line.count(')')
      inner_block.open_parentheses += depth_change

      # Also check if we are starting or ending an inline assembly block.
      if inner_block.inline_asm in (_NO_ASM, _END_ASM):
        if (depth_change != 0 and
            inner_block.open_parentheses == 1 and
            _MATCH_ASM.match(line)):
          # Enter assembly block
          inner_block.inline_asm = _INSIDE_ASM
        else:
          # Not entering assembly block.  If previous line was _END_ASM,
          # we will now shift to _NO_ASM state.
          inner_block.inline_asm = _NO_ASM
      elif (inner_block.inline_asm == _INSIDE_ASM and
            inner_block.open_parentheses == 0):
        # Exit assembly block
        inner_block.inline_asm = _END_ASM

    # Consume namespace declaration at the beginning of the line.  Do
    # this in a loop so that we catch same line declarations like this:
    #   namespace proto2 { namespace bridge { class MessageSet; } }
    while True:
      # Match start of namespace.  The "\b\s*" below catches namespace
      # declarations even if it weren't followed by a whitespace, this
      # is so that we don't confuse our namespace checker.  The
      # missing spaces will be flagged by CheckSpacing.
      namespace_decl_match = Match(r'^\s*namespace\b\s*([:\w]+)?(.*)$', line)
      if not namespace_decl_match:
        break

      new_namespace = _NamespaceInfo(namespace_decl_match.group(1), linenum)
      self.stack.append(new_namespace)

      line = namespace_decl_match.group(2)
      if line.find('{') != -1:
        new_namespace.seen_open_brace = True
        line = line[line.find('{') + 1:]

    # Look for a class declaration in whatever is left of the line
    # after parsing namespaces.  The regexp accounts for decorated classes
    # such as in:
    #   class LOCKABLE API Object {
    #   };
    class_decl_match = Match(
        r'^(\s*(?:template\s*<[\w\s<>,:=]*>\s*)?'
        r'(class|struct)\s+(?:[A-Z_]+\s+)*(\w+(?:::\w+)*))'
        r'(.*)$', line)
    if (class_decl_match and
        (not self.stack or self.stack[-1].open_parentheses == 0)):
      # We do not want to accept classes that are actually template arguments:
      #   template <class Ignore1,
      #             class Ignore2 = Default<Args>,
      #             template <Args> class Ignore3>
      #   void Function() {};
      #
      # To avoid template argument cases, we scan forward and look for
      # an unmatched '>'.  If we see one, assume we are inside a
      # template argument list.
      end_declaration = len(class_decl_match.group(1))
      if not self.InTemplateArgumentList(clean_lines, linenum, end_declaration):
        self.stack.append(_ClassInfo(
            class_decl_match.group(3), class_decl_match.group(2),
            clean_lines, linenum))
        line = class_decl_match.group(4)

    # If we have not yet seen the opening brace for the innermost block,
    # run checks here.
    if not self.SeenOpenBrace():
      self.stack[-1].CheckBegin(filename, clean_lines, linenum, error)

    # Update access control if we are inside a class/struct
    if self.stack and isinstance(self.stack[-1], _ClassInfo):
      classinfo = self.stack[-1]
      access_match = Match(
          r'^(.*)\b(public|private|protected|signals)(\s+(?:slots\s*)?)?'
          r':(?:[^:]|$)',
          line)
      if access_match:
        classinfo.access = access_match.group(2)

        # Check that access keywords are indented +1 space.  Skip this
        # check if the keywords are not preceded by whitespaces.
        indent = access_match.group(1)
        if (len(indent) != classinfo.class_indent + 1 and
            Match(r'^\s*$', indent)):
          if classinfo.is_struct:
            parent = 'struct ' + classinfo.name
          else:
            parent = 'class ' + classinfo.name
          slots = ''
          if access_match.group(3):
            slots = access_match.group(3)
          error(filename, linenum, 'whitespace/indent', 3,
                '%s%s: should be indented +1 space inside %s' % (
                    access_match.group(2), slots, parent))

    # Consume braces or semicolons from what's left of the line
    while True:
      # Match first brace, semicolon, or closed parenthesis.
      matched = Match(r'^[^{;)}]*([{;)}])(.*)$', line)
      if not matched:
        break

      token = matched.group(1)
      if token == '{':
        # If namespace or class hasn't seen a opening brace yet, mark
        # namespace/class head as complete.  Push a new block onto the
        # stack otherwise.
        if not self.SeenOpenBrace():
          self.stack[-1].seen_open_brace = True
        elif Match(r'^extern\s*"[^"]*"\s*\{', line):
          self.stack.append(_ExternCInfo(linenum))
        else:
          self.stack.append(_BlockInfo(linenum, True))
          if _MATCH_ASM.match(line):
            self.stack[-1].inline_asm = _BLOCK_ASM

      elif token == ';' or token == ')':
        # If we haven't seen an opening brace yet, but we already saw
        # a semicolon, this is probably a forward declaration.  Pop
        # the stack for these.
        #
        # Similarly, if we haven't seen an opening brace yet, but we
        # already saw a closing parenthesis, then these are probably
        # function arguments with extra "class" or "struct" keywords.
        # Also pop these stack for these.
        if not self.SeenOpenBrace():
          self.stack.pop()
      else:  # token == '}'
        # Perform end of block checks and pop the stack.
        if self.stack:
          self.stack[-1].CheckEnd(filename, clean_lines, linenum, error)
          self.stack.pop()
      line = matched.group(2)

Response: Updates the nesting state with the current line.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def InnermostClass(self):
    for i in range(len(self.stack), 0, -1):
      classinfo = self.stack[i - 1]
      if isinstance(classinfo, _ClassInfo):
        return classinfo
    return None

Response: Returns the _ClassInfo object if we are inside a class.


Instruction:
Create a Python 3 function for
checking that all classes and namespaces have been completely parsed.

Response:
def CheckCompletedBlocks(self, filename, error):
    """Checks that all classes and namespaces have been completely parsed.

    Call this when all lines in a file have been processed.
    Args:
      filename: The name of the current file.
      error: The function to call with any errors found.
    """
    # Note: This test can result in false positives if #ifdef constructs
    # get in the way of brace matching. See the testBuildClass test in
    # cpplint_unittest.py for an example of this.
    for obj in self.stack:
      if isinstance(obj, _ClassInfo):
        error(filename, obj.starting_linenum, 'build/class', 5,
              'Failed to find complete declaration of class %s' %
              obj.name)
      elif isinstance(obj, _NamespaceInfo):
        error(filename, obj.starting_linenum, 'build/namespaces', 5,
              'Failed to find complete declaration of namespace %s' %
              obj.name)


Instruction:
Write a Python 3 script for
returning a new Streamlet by applying map_function to each element of this Streamlet.

Response:
def map(self, map_function):
    """Return a new Streamlet by applying map_function to each element of this Streamlet.
    """
    from heronpy.streamlet.impl.mapbolt import MapStreamlet
    map_streamlet = MapStreamlet(map_function, self)
    self._add_child(map_streamlet)
    return map_streamlet


Instruction:
Write a Python 3 function that can
return a new Streamlet by applying map_function to each element of this Streamlet and flattening the result.

Response:
def flat_map(self, flatmap_function):
    """Return a new Streamlet by applying map_function to each element of this Streamlet
       and flattening the result
    """
    from heronpy.streamlet.impl.flatmapbolt import FlatMapStreamlet
    fm_streamlet = FlatMapStreamlet(flatmap_function, self)
    self._add_child(fm_streamlet)
    return fm_streamlet


Instruction:
Explain what the following Python 3 code does
def filter(self, filter_function):
    from heronpy.streamlet.impl.filterbolt import FilterStreamlet
    filter_streamlet = FilterStreamlet(filter_function, self)
    self._add_child(filter_streamlet)
    return filter_streamlet

Response: Return a new Streamlet containing only the elements that satisfy filter_function


Instruction:
Can you create a Python 3 function that
returns a new Streamlet containing all elements of this streamlet but having num_partitions partitions.

Response:
def repartition(self, num_partitions, repartition_function=None):
    """Return a new Streamlet containing all elements of the this streamlet but having
    num_partitions partitions. Note that this is different from num_partitions(n) in
    that new streamlet will be created by the repartition call.
    If repartiton_function is not None, it is used to decide which parititons
    (from 0 to num_partitions -1), it should route each element to.
    It could also return a list of partitions if it wants to send it to multiple
    partitions.
    """
    from heronpy.streamlet.impl.repartitionbolt import RepartitionStreamlet
    if repartition_function is None:
      repartition_function = lambda x: x
    repartition_streamlet = RepartitionStreamlet(num_partitions, repartition_function, self)
    self._add_child(repartition_streamlet)
    return repartition_streamlet


Instruction:
Can you generate a brief explanation for the following Python 3 code
def clone(self, num_clones):
    retval = []
    for i in range(num_clones):
      retval.append(self.repartition(self.get_num_partitions()))
    return retval

Response:
Return num_clones number of streamlets each containing all elements
    of the current streamlet


Instruction:
Create a Python 3 function to
return a new Streamlet in which each element of this Streamlet is collected by a window defined by window_config and then reduced using reduce_function.

Response:
def reduce_by_window(self, window_config, reduce_function):
    """Return a new Streamlet in which each element of this Streamlet are collected
      over a window defined by window_config and then reduced using the reduce_function
      reduce_function takes two element at one time and reduces them to one element that
      is used in the subsequent operations.
    """
    from heronpy.streamlet.impl.reducebywindowbolt import ReduceByWindowStreamlet
    reduce_streamlet = ReduceByWindowStreamlet(window_config, reduce_function, self)
    self._add_child(reduce_streamlet)
    return reduce_streamlet


Instruction:
Given the following Python 3 function, write the documentation
def union(self, other_streamlet):
    from heronpy.streamlet.impl.unionbolt import UnionStreamlet
    union_streamlet = UnionStreamlet(self, other_streamlet)
    self._add_child(union_streamlet)
    other_streamlet._add_child(union_streamlet)
    return union_streamlet

Response: Returns a new Streamlet that consists of elements of both this and other_streamlet.


Instruction:
Can you generate the documentation for the following Python 3 function
def transform(self, transform_operator):
    from heronpy.streamlet.impl.transformbolt import TransformStreamlet
    transform_streamlet = TransformStreamlet(transform_operator, self)
    self._add_child(transform_streamlet)
    return transform_streamlet

Response: Returns a new Streamlet by applying the transform_operator on each element of this Streamlet.


Instruction:
Can you create a Python 3 function that
logs all elements of this streamlet. This returns nothing.

Response:
def log(self):
    """Logs all elements of this streamlet. This returns nothing
    """
    from heronpy.streamlet.impl.logbolt import LogStreamlet
    log_streamlet = LogStreamlet(self)
    self._add_child(log_streamlet)
    return


Instruction:
Create a Python 3 function to
call consume_function for each element of this streamlet. This function returns nothing.

Response:
def consume(self, consume_function):
    """Calls consume_function for each element of this streamlet. This function returns nothing
    """
    from heronpy.streamlet.impl.consumebolt import ConsumeStreamlet
    consume_streamlet = ConsumeStreamlet(consume_function, self)
    self._add_child(consume_streamlet)
    return


Instruction:
Implement a function in Python 3 to
return a new Streamlet by joining this streamlet with the given join_streamlet

Response:
def join(self, join_streamlet, window_config, join_function):
    """Return a new Streamlet by joining join_streamlet with this streamlet
    """
    from heronpy.streamlet.impl.joinbolt import JoinStreamlet, JoinBolt
    join_streamlet_result = JoinStreamlet(JoinBolt.INNER, window_config,
                                          join_function, self, join_streamlet)
    self._add_child(join_streamlet_result)
    join_streamlet._add_child(join_streamlet_result)
    return join_streamlet_result


Instruction:
Can you create a Python 3 function that
returns a new Streamlet by outer right join with this streamlet

Response:
def outer_right_join(self, join_streamlet, window_config, join_function):
    """Return a new Streamlet by outer right join_streamlet with this streamlet
    """
    from heronpy.streamlet.impl.joinbolt import JoinStreamlet, JoinBolt
    join_streamlet_result = JoinStreamlet(JoinBolt.OUTER_RIGHT, window_config,
                                          join_function, self, join_streamlet)
    self._add_child(join_streamlet_result)
    join_streamlet._add_child(join_streamlet_result)
    return join_streamlet_result


Instruction:
Given the following Python 3 function, write the documentation
def outer_left_join(self, join_streamlet, window_config, join_function):
    from heronpy.streamlet.impl.joinbolt import JoinStreamlet, JoinBolt
    join_streamlet_result = JoinStreamlet(JoinBolt.OUTER_LEFT, window_config,
                                          join_function, self, join_streamlet)
    self._add_child(join_streamlet_result)
    join_streamlet._add_child(join_streamlet_result)
    return join_streamlet_result

Response: Return a new Streamlet by left join_streamlet with this streamlet


Instruction:
Can you create a Python 3 function that
returns a new Streamlet by outer join_streamlet with this streamlet

Response:
def outer_join(self, join_streamlet, window_config, join_function):
    """Return a new Streamlet by outer join_streamlet with this streamlet
    """
    from heronpy.streamlet.impl.joinbolt import JoinStreamlet, JoinBolt

    join_streamlet_result = JoinStreamlet(JoinBolt.OUTER, window_config,
                                          join_function, self, join_streamlet)
    self._add_child(join_streamlet_result)
    join_streamlet._add_child(join_streamlet_result)
    return join_streamlet_result


Instruction:
Given the following Python 3 function, write the documentation
def reduce_by_key_and_window(self, window_config, reduce_function):
    from heronpy.streamlet.impl.reducebykeyandwindowbolt import ReduceByKeyAndWindowStreamlet
    reduce_streamlet = ReduceByKeyAndWindowStreamlet(window_config, reduce_function, self)
    self._add_child(reduce_streamlet)
    return reduce_streamlet

Response: Return a new Streamlet in which each key value pair of this Streamlet is collected by the reduce_function and the time_window is calculated over the time_window.


Instruction:
Make a summary of the following Python 3 code
def get(self):
    clusters = self.get_arguments(constants.PARAM_CLUSTER)
    environs = self.get_arguments(constants.PARAM_ENVIRON)
    topology_names = self.get_arguments(constants.PARAM_TOPOLOGY)

    ret = {}

    if len(topology_names) > 1:
      if not clusters:
        message = "Missing argument" + constants.PARAM_CLUSTER
        self.write_error_response(message)
        return

      if not environs:
        message = "Missing argument" + constants.PARAM_ENVIRON
        self.write_error_response(message)
        return

    ret = {}
    topologies = self.tracker.topologies
    for topology in topologies:
      cluster = topology.cluster
      environ = topology.environ
      topology_name = topology.name
      if not cluster or not environ:
        continue

      # This cluster is not asked for.
      if clusters and cluster not in clusters:
        continue

      # This environ is not asked for.
      if environs and environ not in environs:
        continue

      if topology_names and topology_name not in topology_names:
        continue

      if cluster not in ret:
        ret[cluster] = {}
      if environ not in ret[cluster]:
        ret[cluster][environ] = {}
      ret[cluster][environ][topology_name] = topology.get_machines()

    self.write_success_response(ret)

Response: get method returns a list of dicts


Instruction:
Implement a function in Python 3 to
stream and handle the output of a process

Response:
def _async_stream_process_output(process, stream_fn, handler):
  """ Stream and handle the output of a process
  :param process: the process to stream the output for
  :param stream_fn: the function that applies handler to process
  :param handler: a function that will be called for each log line
  :return: None
  """
  logging_thread = Thread(target=stream_fn, args=(process, handler, ))

  # Setting the logging thread as a daemon thread will allow it to exit with the program
  # rather than blocking the exit waiting for it to be handled manually.
  logging_thread.daemon = True
  logging_thread.start()

  return logging_thread


Instruction:
Write a Python 3 function for
getting the topology jars Returns the first file in the directory that matches the pattern.

Response:
def pick(dirname, pattern):
  '''
  Get the topology jars
  :param dirname:
  :param pattern:
  :return:
  '''
  file_list = fnmatch.filter(os.listdir(dirname), pattern)
  return file_list[0] if file_list else None


Instruction:
Given the following Python 3 function, write the documentation
def get(self):
    try:
      cluster = self.get_argument_cluster()
      role = self.get_argument_role()
      environ = self.get_argument_environ()
      topology_name = self.get_argument_topology()
      component = self.get_argument_component()
      metric_names = self.get_required_arguments_metricnames()
      start_time = self.get_argument_starttime()
      end_time = self.get_argument_endtime()
      self.validateInterval(start_time, end_time)
      instances = self.get_arguments(constants.PARAM_INSTANCE)

      topology = self.tracker.getTopologyByClusterRoleEnvironAndName(
          cluster, role, environ, topology_name)
      metrics = yield tornado.gen.Task(metricstimeline.getMetricsTimeline,
                                       topology.tmaster, component, metric_names,
                                       instances, int(start_time), int(end_time))
      self.write_success_response(metrics)
    except Exception as e:
      Log.debug(traceback.format_exc())
      self.write_error_response(e)

Response: get method - gets the metrics from the server


Instruction:
Given the following Python 3 function, write the documentation
def create_parser():
  help_epilog = '''Getting more help:
  heron-explorer help <command>     Disply help and options for <command>\n
  For detailed documentation, go to http://heronstreaming.io'''

  parser = argparse.ArgumentParser(
      prog='heron-explorer',
      epilog=help_epilog,
      formatter_class=SubcommandHelpFormatter,
      add_help=False)

  # sub-commands
  subparsers = parser.add_subparsers(
      title="Available commands",
      metavar='<command> <options>')

  # subparser for subcommands related to clusters
  clusters.create_parser(subparsers)

  # subparser for subcommands related to logical plan
  logicalplan.create_parser(subparsers)

  # subparser for subcommands related to physical plan
  physicalplan.create_parser(subparsers)

  # subparser for subcommands related to displaying info
  topologies.create_parser(subparsers)

  # subparser for help subcommand
  help.create_parser(subparsers)

  # subparser for version subcommand
  version.create_parser(subparsers)

  return parser

Response: create parser for all available commands


Instruction:
Here you have a function in Python 3, explain what it does
def expand_args(command):

    # Prepare arguments.
    if isinstance(command, (str, unicode)):
        splitter = shlex.shlex(command.encode('utf-8'))
        splitter.whitespace = '|'
        splitter.whitespace_split = True
        command = []

        while True:
            token = splitter.get_token()
            if token:
                command.append(token)
            else:
                break

        command = list(map(shlex.split, command))

    return command

Response: Parses command strings and returns a Popen - ready list.


Instruction:
Can you generate the documentation for the following Python 3 function
def run(command, data=None, timeout=None, kill_timeout=None, env=None, cwd=None):

    command = expand_args(command)

    history = []
    for c in command:

        if len(history):
            # due to broken pipe problems pass only first 10 KiB
            data = history[-1].std_out[0:10*1024]

        cmd = Command(c)
        try:
            out, err = cmd.run(data, timeout, kill_timeout, env, cwd)
            status_code = cmd.returncode
        except OSError as e:
            out, err = '', u"\n".join([e.strerror, traceback.format_exc()])
            status_code = 127

        r = Response(process=cmd)

        r.command = c
        r.std_out = out
        r.std_err = err
        r.status_code = status_code

        history.append(r)

    r = history.pop()
    r.history = history

    return r

Response: Executes a given commmand and returns Response.


Instruction:
Can you implement a function in Python 3 that
spawns a new process from the given command.

Response:
def connect(command, data=None, env=None, cwd=None):
    """Spawns a new process from the given command."""

    # TODO: support piped commands
    command_str = expand_args(command).pop()
    environ = dict(os.environ)
    environ.update(env or {})

    process = subprocess.Popen(command_str,
        universal_newlines=True,
        shell=False,
        env=environ,
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        bufsize=0,
        cwd=cwd,
    )

    return ConnectedCommand(process=process)


Instruction:
Can you create a Python 3 function that
sends a line to the standard input.

Response:
def send(self, str, end='\n'):
        """Sends a line to std_in."""
        return self._process.stdin.write(str+end)


Instruction:
Can you generate the documentation for the following Python 3 function
def bracket_split(source, brackets=('()', '{}', '[]'), strip=False):
    starts = [e[0] for e in brackets]
    in_bracket = 0
    n = 0
    last = 0
    while n < len(source):
        e = source[n]
        if not in_bracket and e in starts:
            in_bracket = 1
            start = n
            b_start, b_end = brackets[starts.index(e)]
        elif in_bracket:
            if e == b_start:
                in_bracket += 1
            elif e == b_end:
                in_bracket -= 1
                if not in_bracket:
                    if source[last:start]:
                        yield source[last:start]
                    last = n + 1
                    yield source[start + strip:n + 1 - strip]
        n += 1
    if source[last:]:
        yield source[last:]

Response: split a string into several lists of strings


Instruction:
Can you create a Python 3 function that
converts Py type to PyJs type.

Response:
def Js(val, Clamped=False):
    '''Converts Py type to PyJs type'''
    if isinstance(val, PyJs):
        return val
    elif val is None:
        return undefined
    elif isinstance(val, basestring):
        return PyJsString(val, StringPrototype)
    elif isinstance(val, bool):
        return true if val else false
    elif isinstance(val, float) or isinstance(val, int) or isinstance(
            val, long) or (NUMPY_AVAILABLE and isinstance(
                val,
                (numpy.int8, numpy.uint8, numpy.int16, numpy.uint16,
                 numpy.int32, numpy.uint32, numpy.float32, numpy.float64))):
        # This is supposed to speed things up. may not be the case
        if val in NUM_BANK:
            return NUM_BANK[val]
        return PyJsNumber(float(val), NumberPrototype)
    elif isinstance(val, FunctionType):
        return PyJsFunction(val, FunctionPrototype)
    #elif isinstance(val, ModuleType):
    #    mod = {}
    #    for name in dir(val):
    #        value = getattr(val, name)
    #        if isinstance(value, ModuleType):
    #            continue  # prevent recursive module conversion
    #        try:
    #            jsval = HJs(value)
    #        except RuntimeError:
    #            print 'Could not convert %s to PyJs object!' % name
    #            continue
    #        mod[name] = jsval
    #    return Js(mod)
    #elif isintance(val, ClassType):

    elif isinstance(val, dict):  # convert to object
        temp = PyJsObject({}, ObjectPrototype)
        for k, v in six.iteritems(val):
            temp.put(Js(k), Js(v))
        return temp
    elif isinstance(val, (list, tuple)):  #Convert to array
        return PyJsArray(val, ArrayPrototype)
    # convert to typedarray
    elif isinstance(val, JsObjectWrapper):
        return val.__dict__['_obj']
    elif NUMPY_AVAILABLE and isinstance(val, numpy.ndarray):
        if val.dtype == numpy.int8:
            return PyJsInt8Array(val, Int8ArrayPrototype)
        elif val.dtype == numpy.uint8 and not Clamped:
            return PyJsUint8Array(val, Uint8ArrayPrototype)
        elif val.dtype == numpy.uint8 and Clamped:
            return PyJsUint8ClampedArray(val, Uint8ClampedArrayPrototype)
        elif val.dtype == numpy.int16:
            return PyJsInt16Array(val, Int16ArrayPrototype)
        elif val.dtype == numpy.uint16:
            return PyJsUint16Array(val, Uint16ArrayPrototype)

        elif val.dtype == numpy.int32:
            return PyJsInt32Array(val, Int32ArrayPrototype)
        elif val.dtype == numpy.uint32:
            return PyJsUint16Array(val, Uint32ArrayPrototype)

        elif val.dtype == numpy.float32:
            return PyJsFloat32Array(val, Float32ArrayPrototype)
        elif val.dtype == numpy.float64:
            return PyJsFloat64Array(val, Float64ArrayPrototype)
    else:  # try to convert to js object
        return py_wrap(val)


Instruction:
Can you tell what is the following Python 3 function doing
def PyJsStrictEq(a, b):
    '''a===b'''
    tx, ty = Type(a), Type(b)
    if tx != ty:
        return false
    if tx == 'Undefined' or tx == 'Null':
        return true
    if a.is_primitive():  #string bool and number case
        return Js(a.value == b.value)
    if a.Class == b.Class == 'PyObjectWrapper':
        return Js(a.obj == b.obj)
    return Js(a is b)

Response: PyJsStrictEq returns a boolean if a == b otherwise false.


Instruction:
Can you generate the documentation for the following Python 3 function
def put(self, prop, val, op=None):  #external use!
        '''Just like in js: self.prop op= val
           for example when op is '+' it will be self.prop+=val
           op can be either None for simple assignment or one of:
           * / % + - << >> & ^ |'''
        if self.Class == 'Undefined' or self.Class == 'Null':
            raise MakeError('TypeError',
                            'Undefined and null dont have properties!')
        if not isinstance(prop, basestring):
            prop = prop.to_string().value
        if NUMPY_AVAILABLE and prop.isdigit():
            if self.Class == 'Int8Array':
                val = Js(numpy.int8(val.to_number().value))
            elif self.Class == 'Uint8Array':
                val = Js(numpy.uint8(val.to_number().value))
            elif self.Class == 'Uint8ClampedArray':
                if val < Js(numpy.uint8(0)):
                    val = Js(numpy.uint8(0))
                elif val > Js(numpy.uint8(255)):
                    val = Js(numpy.uint8(255))
                else:
                    val = Js(numpy.uint8(val.to_number().value))
            elif self.Class == 'Int16Array':
                val = Js(numpy.int16(val.to_number().value))
            elif self.Class == 'Uint16Array':
                val = Js(numpy.uint16(val.to_number().value))
            elif self.Class == 'Int32Array':
                val = Js(numpy.int32(val.to_number().value))
            elif self.Class == 'Uint32Array':
                val = Js(numpy.uint32(val.to_number().value))
            elif self.Class == 'Float32Array':
                val = Js(numpy.float32(val.to_number().value))
            elif self.Class == 'Float64Array':
                val = Js(numpy.float64(val.to_number().value))
            if isinstance(self.buff, numpy.ndarray):
                self.buff[int(prop)] = int(val.to_number().value)
        #we need to set the value to the incremented one
        if op is not None:
            val = getattr(self.get(prop), OP_METHODS[op])(val)
        if not self.can_put(prop):
            return val
        own_desc = self.get_own_property(prop)
        if is_data_descriptor(own_desc):
            if self.Class in [
                    'Array', 'Int8Array', 'Uint8Array', 'Uint8ClampedArray',
                    'Int16Array', 'Uint16Array', 'Int32Array', 'Uint32Array',
                    'Float32Array', 'Float64Array'
            ]:
                self.define_own_property(prop, {'value': val})
            else:
                self.own[prop]['value'] = val
            return val
        desc = self.get_property(prop)
        if is_accessor_descriptor(desc):
            desc['set'].call(self, (val, ))
        else:
            new = {
                'value': val,
                'writable': True,
                'configurable': True,
                'enumerable': True
            }
            if self.Class in [
                    'Array', 'Int8Array', 'Uint8Array', 'Uint8ClampedArray',
                    'Int16Array', 'Uint16Array', 'Int32Array', 'Uint32Array',
                    'Float32Array', 'Float64Array'
            ]:
                self.define_own_property(prop, new)
            else:
                self.own[prop] = new
        return val

Response: This method will put a property into the object.


Instruction:
Create a Python 3 function to
return the result of JS == compare. result is PyJs type: bool

Response:
def abstract_equality_comparison(self, other):
        ''' returns the result of JS == compare.
           result is PyJs type: bool'''
        tx, ty = self.TYPE, other.TYPE
        if tx == ty:
            if tx == 'Undefined' or tx == 'Null':
                return true
            if tx == 'Number' or tx == 'String' or tx == 'Boolean':
                return Js(self.value == other.value)
            return Js(self is other)  # Object
        elif (tx == 'Undefined' and ty == 'Null') or (ty == 'Undefined'
                                                      and tx == 'Null'):
            return true
        elif tx == 'Number' and ty == 'String':
            return self.abstract_equality_comparison(other.to_number())
        elif tx == 'String' and ty == 'Number':
            return self.to_number().abstract_equality_comparison(other)
        elif tx == 'Boolean':
            return self.to_number().abstract_equality_comparison(other)
        elif ty == 'Boolean':
            return self.abstract_equality_comparison(other.to_number())
        elif (tx == 'String' or tx == 'Number') and other.is_object():
            return self.abstract_equality_comparison(other.to_primitive())
        elif (ty == 'String' or ty == 'Number') and self.is_object():
            return self.to_primitive().abstract_equality_comparison(other)
        else:
            return false


Instruction:
Write a Python 3 function that can
call a property prop as a method.

Response:
def callprop(self, prop, *args):
        '''Call a property prop as a method (this will be self).

        NOTE: dont pass this and arguments here, these will be added
        automatically!'''
        if not isinstance(prop, basestring):
            prop = prop.to_string().value
        cand = self.get(prop)
        if not cand.is_callable():
            raise MakeError('TypeError',
                            '%s is not a function' % cand.typeof())
        return cand.call(self, args)


Instruction:
Explain what the following Python 3 code does
def _set_name(self, name):
        '''name is py type'''
        if self.own.get('name'):
            self.func_name = name
            self.own['name']['value'] = Js(name)

Response: name is py type


Instruction:
Can you create a Python 3 function that
calls this function and returns a result

Response:
def call(self, this, args=()):
        '''Calls this function and returns a result
        (converted to PyJs type so func can return python types)

        this must be a PyJs object and args must be a python tuple of PyJs objects.

        arguments object is passed automatically and will be equal to Js(args)
        (tuple converted to arguments object).You dont need to worry about number
        of arguments you provide if you supply less then missing ones will be set
        to undefined (but not present in arguments object).
        And if you supply too much then excess will not be passed
        (but they will be present in arguments object).
        '''
        if not hasattr(args, '__iter__'):  #get rid of it later
            args = (args, )
        args = tuple(Js(e) for e in args)  # this wont be needed later

        arguments = PyJsArguments(
            args, self)  # tuple will be converted to arguments object.
        arglen = self.argcount  #function expects this number of args.
        if len(args) > arglen:
            args = args[0:arglen]
        elif len(args) < arglen:
            args += (undefined, ) * (arglen - len(args))
        args += this, arguments  #append extra params to the arg list
        try:
            return Js(self.code(*args))
        except NotImplementedError:
            raise
        except RuntimeError as e:  # maximum recursion
            raise MakeError(
                'RangeError', e.message if
                not isinstance(e, NotImplementedError) else 'Not implemented!')


Instruction:
Can you write a function in Python 3 where it
removes functions and returns new source and 2 dicts.

Response:
def remove_functions(source, all_inline=False):
    """removes functions and returns new source, and 2 dicts.
        first dict with removed hoisted(global) functions and second with replaced inline functions"""
    global INLINE_COUNT
    inline = {}
    hoisted = {}
    n = 0
    limit = len(source) - 9  # 8 is length of 'function'
    res = ''
    last = 0
    while n < limit:
        if n and source[n - 1] in IDENTIFIER_PART:
            n += 1
            continue
        if source[n:n + 8] == 'function' and source[n +
                                                    8] not in IDENTIFIER_PART:
            if source[:n].rstrip().endswith(
                    '.'):  # allow function as a property name :)
                n += 1
                continue
            if source[n + 8:].lstrip().startswith(
                    ':'):  # allow functions inside objects...
                n += 1
                continue
            entered = n
            res += source[last:n]
            name = ''
            n = pass_white(source, n + 8)
            if source[n] in IDENTIFIER_START:  # hoisted function
                name, n = parse_identifier(source, n)
            args, n = pass_bracket(source, n, '()')
            if not args:
                raise SyntaxError('Function misses bracket with argnames ()')
            args = args.strip('() \n')
            args = tuple(parse_identifier(e, 0)[0]
                         for e in argsplit(args)) if args else ()
            if len(args) - len(set(args)):
                # I know its legal in JS but python does not allow duplicate argnames
                # I will not work around it
                raise SyntaxError(
                    'Function has duplicate argument names. Its not legal in this implementation. Sorry.'
                )
            block, n = pass_bracket(source, n, '{}')
            if not block:
                raise SyntaxError(
                    'Function does not have any code block to execute')
            mixed = False  # named function expression flag
            if name and not all_inline:
                # Here I will distinguish between named function expression (mixed) and a function statement
                before = source[:entered].rstrip()
                if any(endswith_keyword(before, e) for e in PRE_EXP_STARTS):
                    #print 'Ended ith keyword'
                    mixed = True
                elif before and before[-1] not in PRE_ALLOWED and not before[
                        -2:] in INCREMENTS:
                    #print 'Ended with'+repr(before[-1]), before[-1]=='}'
                    mixed = True
                else:
                    #print 'FUNCTION STATEMENT'
                    #its a function statement.
                    # todo remove fucking label if present!
                    hoisted[name] = block, args
            if not name or mixed or all_inline:  # its a function expression (can be both named and not named)
                #print 'FUNCTION EXPRESSION'
                INLINE_COUNT += 1
                iname = INLINE_NAME % INLINE_COUNT  # inline name
                res += ' ' + iname
                inline['%s@%s' % (
                    iname, name
                )] = block, args  #here added real name at the end because it has to be added to the func scope
            last = n
        else:
            n += 1
    res += source[last:]
    return res, hoisted, inline


Instruction:
Can you generate a brief explanation for the following Python 3 code
def ConstructArray(self, py_arr):
        ''' note py_arr elems are NOT converted to PyJs types!'''
        arr = self.NewArray(len(py_arr))
        arr._init(py_arr)
        return arr

Response: constructs a new array from a Python array


Instruction:
How would you explain what the following Python 3 function does
def ConstructObject(self, py_obj):
        ''' note py_obj items are NOT converted to PyJs types! '''
        obj = self.NewObject()
        for k, v in py_obj.items():
            obj.put(unicode(k), v)
        return obj

Response: constructs a new object from a dictionary


Instruction:
Write a Python 3 function that can
add an op_code with specified args to the tape.

Response:
def emit(self, op_code, *args):
        ''' Adds op_code with specified args to tape '''
        self.tape.append(OP_CODES[op_code](*args))


Instruction:
Can you generate the documentation for the following Python 3 function
def compile(self, start_loc=0):
        ''' Records locations of labels and compiles the code '''
        self.label_locs = {} if self.label_locs is None else self.label_locs
        loc = start_loc
        while loc < len(self.tape):
            if type(self.tape[loc]) == LABEL:
                self.label_locs[self.tape[loc].num] = loc
                del self.tape[loc]
                continue
            loc += 1
        self.compiled = True

Response: Records locations of labels and compiles the code.