Instruction:
Can you generate a brief explanation for the following Python 3 code
def addGrammaticalFeatsAndRoots( sentence, foundChains ):
    ''' Täiendab leitud verbiahelaid, lisades iga ahela kylge selle s6nade lemmad (ROOT
        v2ljad morf analyysist) ning morfoloogilised tunnused (POSTAG+FORM: eraldajaks
        '_' ning kui on mitu varianti, siis tuuakse k6ik variandid, eraldajaks '/');
        Atribuudid ROOTS ja MORPH sisaldavad tunnuste loetelusid (iga ahela liikme jaoks
        yks tunnus:
         Nt.
           ** 'püüab kodeerida' puhul tuleb MORPH väärtuseks ['V_b', 'V_da'] ning
              ROOTS väärtuseks ['püüd', 'kodeeri'];
           ** 'on tulnud' puhul tuleb MORPH väärtuseks ['V_vad/b', 'V_nud'] ning
              ROOTS väärtuseks ['ole', 'tule'];

         Lisaks leiatakse ahela p6hiverbi (esimese verbi) grammatilised tunnused:
           ** aeg (TENSE):       present, imperfect, perfect, pluperfect, past, ??
           ** k6neviis (MOOD):   indic, imper, condit, quotat, ??
           ** tegumood (VOICE):  personal, impersonal, ??
    '''
    _indicPresent     = ['n','d','b','me','te','vad']
    _indicImperfect   = ['sin', 'sid', 's', 'sime', 'site', 'sid']
    _imperatPlural    = ['gem', 'ge', 'gu']
    _conditPreesens   = ['ksin', 'ksid', 'ks', 'ksime', 'ksite', 'ksid']
    _conditPreteerium = ['nuksin', 'nuksid', 'nuks', 'nuksime', 'nuksite', 'nuksid']
    for i in range(len(foundChains)):
        matchObj1 = foundChains[i]
        roots = []
        grammFeats = []
        grammPosAndForms = []
        #
        # 1) Leiame kogu ahela morfoloogilised tunnused ja lemmad
        #
        for j in range(len( matchObj1[PHRASE] )):
            wid = matchObj1[PHRASE][j]
            token = [token for token in sentence if token[WORD_ID]==wid][0]
            analysisIDs = matchObj1[ANALYSIS_IDS][j]
            analyses = [ token[ANALYSIS][k] for k in range(len( token[ANALYSIS] )) if k in analysisIDs ]
            pos  = set( [a[POSTAG] for a in analyses] )
            form = set( [a[FORM] for a in analyses] )
            root = [a[ROOT] for a in analyses][0]
            grammatical = ("/".join(list(pos))) + '_' + ("/".join(list(form)))
            grammPosAndForms.append( (pos, form) )
            #  Yhtlustame m6ningaid mustreid (st kohendame nende morf analyysi selliseks, nagu
            # mustri poolt on eeldatud) ...
            if root == 'ei' and len(matchObj1[PHRASE])>1:
                grammatical = 'V_neg'
            if matchObj1[PATTERN][j] == '&':
                grammatical = 'J_'
            roots.append( root )
            grammFeats.append( grammatical )
        matchObj1[ROOTS] = roots
        matchObj1[MORPH] = grammFeats
        #
        # 2) Leiame eelneva põhjal ahela põhiverbi tunnused:  grammatilise aja (tense),
        #      kõneviisi (mood),  tegumoe (voice)
        #
        tense = "??"
        mood  = "??"
        voice = "??"
        if matchObj1[POLARITY] == 'POS':
            #
            #   Jaatuse tunnused
            #
            (pos, form) = grammPosAndForms[0]
            if 'V' in pos:
                #
                #  Indikatiiv e kindel kõneviis
                #
                if len(form.intersection( _indicPresent )) > 0:
                    tense = "present"
                    mood  = "indic"
                    voice = "personal"
                elif len(form.intersection( _indicImperfect )) > 0:
                    tense = "imperfect"
                    mood = "indic"
                    voice = "personal"
                elif 'takse' in form:
                    tense = "present"
                    mood  = "indic"
                    voice = "impersonal"
                elif 'ti' in form:
                    tense = "imperfect"
                    mood  = "indic"
                    voice = "impersonal"
                #
                #  Imperatiiv e käskiv kõneviis
                #
                elif 'o' in form or 'gu' in form:
                    tense = "present"
                    mood  = "imper"
                    voice = "personal"
                elif len(form.intersection( _imperatPlural )) > 0:
                    tense = "present"
                    mood  = "imper"
                    voice = "personal"
                elif 'tagu' in form:
                    tense = "present"
                    mood  = "imper"
                    voice = "impersonal"
                #
                #  Konditsionaal e tingiv kõneviis
                #
                elif len(form.intersection( _conditPreesens )) > 0:
                    tense = "present"
                    mood  = "condit"
                    voice = "personal"
                elif 'taks' in form:
                    tense = "present"
                    mood  = "condit"
                    voice = "impersonal"
                elif len(form.intersection( _conditPreteerium )) > 0:
                    tense = "past"
                    mood  = "condit"
                    voice = "personal"
                elif 'tuks' in form:
                    tense = "past"
                    mood  = "condit"
                    voice = "impersonal"
                #
                #  Kvotatiiv e kaudne kõneviis
                #
                elif 'vat' in form:
                    tense = "present"
                    mood  = "quotat"
                    voice = "personal"
                elif 'tavat' in form:
                    tense = "present"
                    mood  = "quotat"
                    voice = "impersonal"
                elif 'nuvat' in form:
                    tense = "past"
                    mood  = "quotat"
                    voice = "personal"
                elif 'tuvat' in form:
                    tense = "past"
                    mood  = "quotat"
                    voice = "impersonal"
                #
                #  Liitaeg: olema + nud (personaal), olema + tud (impersonaal)
                #
                if len(matchObj1[PATTERN]) > 1 and matchObj1[PATTERN][0] == 'ole':
                    # Kindla kõneviisi liitaeg
                    if mood == "indic" and (grammFeats[1] == "V_nud" or grammFeats[1] == "V_tud"):
                        if tense == "present":
                            #  Täisminevik
                            tense = "perfect"
                            if grammFeats[1] == "V_tud":
                                voice = "impersonal"
                        elif tense == "imperfect":
                            #  Enneminevik
                            tense = "pluperfect"
                            if grammFeats[1] == "V_tud":
                                voice = "impersonal"
                    # Tingiva ja kaudse kõneviisi liitaeg (nn üldminevik)
                    elif mood in ["quotat", "condit"] and tense == "present" and \
                         voice == "personal":
                            if grammFeats[1] == "V_nud":
                                tense = "past"
                            elif grammFeats[1] == "V_tud":
                                if mood == "quotat":
                                    tense = "past"
                                    voice = "impersonal"
                                else:
                                    # tingiv + tud jääb esialgu lahtiseks
                                    tense = "??"
                                    voice = "??"
        elif matchObj1[POLARITY] == 'NEG':
            #
            #   Eituse tunnused
            #
            if len(matchObj1[PATTERN]) > 1 and \
               (matchObj1[PATTERN][0] == 'ei' or matchObj1[PATTERN][0] == 'ega'):
                (pos, form) = grammPosAndForms[1]
                # Indikatiiv
                if 'o' in form or 'neg o' in form:
                    tense = "present"
                    mood  = "indic"
                    voice = "personal"
                elif 'ta' in form:
                    tense = "present"
                    mood  = "indic"
                    voice = "impersonal"
                elif 'nud' in form:
                    tense = "imperfect"
                    mood  = "indic"
                    voice = "personal"
                elif 'tud' in form:
                    tense = "imperfect"
                    mood  = "indic"
                    voice = "impersonal"
                # Konditsionaal
                elif 'ks' in form:
                    tense = "present"
                    mood  = "condit"
                    voice = "personal"
                elif 'taks' in form:
                    tense = "present"
                    mood  = "condit"
                    voice = "impersonal"
                elif 'nuks' in form:
                    tense = "past"
                    mood  = "condit"
                    voice = "personal"
                elif 'tuks' in form:
                    tense = "past"
                    mood  = "condit"
                    voice = "impersonal"
                # Kvotatiiv
                elif 'vat' in form:
                    tense = "present"
                    mood  = "quotat"
                    voice = "personal"
                elif 'tavat' in form:
                    tense = "present"
                    mood  = "quotat"
                    voice = "impersonal"
                elif 'nuvat' in form:
                    tense = "past"
                    mood  = "quotat"
                    voice = "personal"
                elif 'tuvat' in form:
                    tense = "past"
                    mood  = "quotat"
                    voice = "impersonal"
                #
                #  Liitaeg: ei + olema + nud (personaal), ei + olema + tud (impersonaal)
                #
                if len(matchObj1[PATTERN]) > 2 and matchObj1[PATTERN][1] == 'ole':
                    # Kindla kõneviisi liitaeg
                    if mood == "indic" and (grammFeats[2] == "V_nud" or grammFeats[2] == "V_tud"):
                        if tense == "present":
                            #  Täisminevik
                            tense = "perfect"
                            if grammFeats[2] == "V_tud":
                                voice = "impersonal"
                        elif tense == "imperfect":
                            #  Enneminevik
                            tense = "pluperfect"
                            if grammFeats[2] == "V_tud":
                                voice = "impersonal"
                    # Tingiva ja kaudse kõneviisi liitaeg (nn üldminevik)
                    elif mood in ["quotat", "condit"] and tense == "present" and \
                         voice == "personal":
                            if grammFeats[2] == "V_nud":
                                tense = "past"
                            elif grammFeats[2] == "V_tud":
                                if mood == "quotat":
                                    tense = "past"
                                    voice = "impersonal"
                                else:
                                    # tingiv + tud jääb esialgu lahtiseks
                                    tense = "??"
                                    voice = "??"
            elif len(matchObj1[PATTERN]) > 1 and matchObj1[PATTERN][0] == 'ära':
                (pos, form) = grammPosAndForms[1]
                # Imperatiiv
                if 'tagu' in form:
                    tense = "present"
                    mood  = "imper"
                    voice = "impersonal"
                else:
                    tense = "present"
                    mood  = "imper"
                    voice = "personal"
            elif matchObj1[PATTERN][0] == 'pole':
                (pos, form) = grammPosAndForms[0]
                # Indikatiiv
                if 'neg o' in form:
                    tense = "present"
                    mood  = "indic"
                    voice = "personal"
                elif 'neg nud' in form:
                    tense = "imperfect"
                    mood  = "indic"
                    voice = "personal"
                elif 'neg tud' in form:
                    tense = "imperfect"
                    mood  = "indic"
                    voice = "impersonal"
                # Konditsionaal
                elif 'neg ks' in form:
                    tense = "present"
                    mood  = "condit"
                    voice = "personal"
                elif 'neg nuks' in form:
                    tense = "past"
                    mood  = "condit"
                    voice = "personal"
                # Kvotatiiv
                elif 'neg vat' in form:
                    tense = "present"
                    mood  = "quotat"
                    voice = "personal"
                #
                #  Liitaeg: pole + nud (personaal), pole + tud (impersonaal)
                #
                if len(matchObj1[PATTERN]) > 1:
                    # Kindla kõneviisi liitaeg
                    if mood == "indic" and (grammFeats[1] == "V_nud" or grammFeats[1] == "V_tud"):
                        if tense == "present":
                            #  Täisminevik
                            tense = "perfect"
                            if grammFeats[1] == "V_tud":
                                voice = "impersonal"
                        elif tense == "imperfect":
                            #  Enneminevik
                            tense = "pluperfect"
                            if grammFeats[1] == "V_tud":
                                voice = "impersonal"
                    # Tingiva ja kaudse kõneviisi liitaeg (nn üldminevik)
                    elif mood in ["quotat", "condit"] and tense == "present" and \
                         voice == "personal":
                            if grammFeats[1] == "V_nud":
                                tense = "past"
                            elif grammFeats[1] == "V_tud":
                                if mood == "quotat":
                                    tense = "past"
                                    voice = "impersonal"
                                else:
                                    # tingiv + tud jääb esialgu lahtiseks
                                    tense = "??"
                                    voice = "??"
        matchObj1[MOOD] = mood
        matchObj1[TENSE] = tense
        matchObj1[VOICE] = voice

Response: Add grammatical feats and roots to the sentence.


Instruction:
Write a Python 3 function for
detecting verb chains from a given sentence.

Response:
def detectVerbChainsFromSent( self, sentence, **kwargs):
        ''' Detect verb chains from given sentence.

        Parameters
        ----------
        sentence:  list of dict
            A list of sentence words, each word in form of a dictionary containing
            morphological analysis and clause boundary annotations (must have CLAUSE_IDX);

        Keyword parameters
        ------------------
        expand2ndTime: boolean
            If True, regular verb chains (chains not ending with 'olema') are expanded twice.
            (default: False)
        breakOnPunctuation: boolean
            If True, expansion of regular verb chains will be broken in case of intervening punctuation.
            (default: False)
        removeSingleAraEi: boolean
            if True, verb chains consisting of a single word, 'ära' or 'ei', will be removed.
            (default: True)
        removeOverlapping: boolean
            If True, overlapping verb chains will be removed.
            (default: True)

        Returns
        -------
        list of dict
            List of detected verb chains, each verb chain has following attributes (keys):
             PHRASE      -- list of int : indexes pointing to elements in sentence that belong
                                            to the chain;
             PATTERN     -- list of str : for each word in phrase, marks whether it is 'ega', 'ei',
                                            'ära', 'pole', 'ole', '&' (conjunction: ja/ning/ega/või)
                                            'verb' (verb different than 'ole') or 'nom/adv';
             ANALYSIS_IDS -- list of (list of int) : for each word in phrase, points to index(es) of
                                                      morphological analyses that correspond to words
                                                      in the verb chains;
             ROOTS       -- list of str : for each word in phrase, lists its corresponding ROOT
                                            value from the morphological analysis; e.g. for the verb
                                            chain 'püüab kodeerida', the ROOT will be ['püüd',
                                            'kodeeri'];
             MORPH       -- list of str : for each word in phrase, lists its part-of-speech value
                                            and morphological form (in one string, separated by '_',
                                            and multiple variants of the pos/form separated by '/');
                                            e.g. for the verb chain 'on tulnud', the MORPH value
                                            will be ['V_vad/b', 'V_nud'];
             OTHER_VERBS  -- bool : whether there are other verbs in the context, potentially being
                                     part of the verb chain; if this is True, it is uncertain whether
                                     the chain is complete or not;

             POLARITY     -- 'POS', 'NEG' or '??' : grammatical polarity of the verb chain; Negative
                                                    polarity indicates that the verb phrase begins
                                                    with 'ei', 'ega', 'ära' or 'pole';
             TENSE        -- tense of the main verb:  'present', 'imperfect', 'perfect',
                                                              'pluperfect', 'past', '??';
             MOOD         -- mood of the main verb:   'indic', 'imper', 'condit', 'quotat', '??';
             VOICE        -- voice of the main verb:  'personal', 'impersonal', '??';

        '''
        # 0) Parse given arguments
        expand2ndTime = False
        removeOverlapping  = True
        removeSingleAraEi  = True
        breakOnPunctuation = False
        for argName, argVal in kwargs.items():
            if argName == 'expand2ndTime':
                expand2ndTime = bool(argVal)
            elif argName == 'removeOverlapping':
                removeOverlapping = bool(argVal)
            elif argName == 'removeSingleAraEi':
                removeSingleAraEi = bool(argVal)
            elif argName == 'breakOnPunctuation':
                breakOnPunctuation = bool(argVal)
            else:
                raise Exception(' Unsupported argument given: '+argName)

        # 1) Preprocessing
        sentence = addWordIDs( sentence )
        clauses  = getClausesByClauseIDs( sentence )

        # 2) Extract predicate-centric verb chains within each clause
        allDetectedVerbChains = []
        for clauseID in clauses:
            clause   = clauses[clauseID]

            # 2.1) Extract predicate-centric verb chains within each clause
            detectedBasicChains = _extractBasicPredicateFromClause(clause, clauseID)
            allDetectedVerbChains.extend( detectedBasicChains )

            # 2.2) Extract 'saama' + 'tud' verb phrases (typically rare)
            _expandSaamaWithTud( clause, clauseID, allDetectedVerbChains )

            # 2.3) Extend 'olema' chains with 'nud/tud/mas/mata' verbs (if possible)
            _expandOlemaVerbChains( clause, clauseID, allDetectedVerbChains )

            # 2.4) Expand non-olema verb chains inside the clause where possible (verb+verb chains)
            _expandVerbChainsBySubcat( clause, clauseID, allDetectedVerbChains, self.verbInfSubcatLexicon, False, breakOnPunctuation)

            # 2.5) Determine for which verb chains the context should be clear
            #    (no additional verbs can be added to the phrase)
            _determineVerbChainContextualAmbiguity( clause, clauseID, allDetectedVerbChains)

            # 2.6) Expand non-olema verb chains inside the clause 2nd time (verb+verb+verb chains)
            #      (Note that while verb+verb+verb+verb+...  chains are also possible, three verbs
            #       seems to be a critical length: longer chains are rare and thus making longer
            #       chains will likely lead to errors);
            if expand2ndTime:
                _expandVerbChainsBySubcat( clause, clauseID, allDetectedVerbChains, self.verbInfSubcatLexicon, False, breakOnPunctuation)

        # 3) Extract 'ega' negations (considering the whole sentence context)
        expandableEgaFound = _extractEgaNegFromSent( sentence, clauses, allDetectedVerbChains )

        if expandableEgaFound:
            for clauseID in clauses:
                clause = clauses[clauseID]
                # 3.1) Expand non-olema 'ega' verb chains inside the clause, if possible;
                _expandVerbChainsBySubcat( clause, clauseID, allDetectedVerbChains, self.verbInfSubcatLexicon, False, breakOnPunctuation)

            #_debugPrint(' | '+getJsonAsTextString(sentence, markTokens = [ verbObj[PHRASE] for verbObj in allDetectedVerbChains ]))

        # 4) Extend chains with nom/adv + Vinf relations
        if self.verbNomAdvVinfExtender:
            addGrammaticalFeatsAndRoots( sentence, allDetectedVerbChains )
            for clauseID in clauses:
                clause = clauses[clauseID]
                expansionPerformed = \
                    self.verbNomAdvVinfExtender.extendChainsInClause( clause, clauseID, allDetectedVerbChains )
                if expansionPerformed:
                    _determineVerbChainContextualAmbiguity( clause, clauseID, allDetectedVerbChains)

        # ) Remove redundant and overlapping verb phrases
        removeRedundantVerbChains( allDetectedVerbChains, removeOverlapping = removeOverlapping, removeSingleAraAndEi = removeSingleAraEi )

        # ) Add grammatical features (in the end)
        addGrammaticalFeatsAndRoots( sentence, allDetectedVerbChains )

        return allDetectedVerbChains


Instruction:
Write a Python 3 script for
finding balanced expression in text.

Response:
def findBalanced(text, openDelim, closeDelim):
    """
    Assuming that text contains a properly balanced expression
    :param openDelim: as opening delimiters and
    :param closeDelim: as closing delimiters.
    :return: an iterator producing pairs (start, end) of start and end
    positions in text containing a balanced expression.
    """
    openPat = '|'.join([re.escape(x) for x in openDelim])
    # pattern for delimiters expected after each opening delimiter
    afterPat = {o: re.compile(openPat+'|'+c, re.DOTALL) for o,c in zip(openDelim, closeDelim)}
    stack = []
    start = 0
    cur = 0
    end = len(text)
    startSet = False
    startPat = re.compile(openPat)
    nextPat = startPat
    while True:
        next = nextPat.search(text, cur)
        if not next:
            return
        if not startSet:
            start = next.start()
            startSet = True
        delim = next.group(0)
        if delim in openDelim:
            stack.append(delim)
            nextPat = afterPat[delim]
        else:
            opening = stack.pop()
            # assert opening == openDelim[closeDelim.index(next.group(0))]
            if stack:
                nextPat = afterPat[stack[-1]]
            else:
                yield start, next.end()
                nextPat = startPat
                start = next.end()
                startSet = False
        cur = next.end()


Instruction:
Can you create a Python 3 function that
forces conversion of given string to unicode type.

Response:
def as_unicode(s, encoding='utf-8'):
    """Force conversion of given string to unicode type.
    Unicode is ``str`` type for Python 3.x and ``unicode`` for Python 2.x .

    If the string is already in unicode, then no conversion is done and the same string is returned.

    Parameters
    ----------
    s: str or bytes (Python3), str or unicode (Python2)
        The string to convert to unicode.
    encoding: str
        The encoding of the input string (default: utf-8)

    Raises
    ------
    ValueError
        In case an input of invalid type was passed to the function.

    Returns
    -------
    ``str`` for Python3 or ``unicode`` for Python 2.
    """
    if isinstance(s, six.text_type):
        return s
    elif isinstance(s, six.binary_type):
        return s.decode(encoding)
    else:
        raise ValueError('Can only convert types {0} and {1}'.format(six.text_type, six.binary_type))


Instruction:
How would you explain what the following Python 3 function does
def as_binary(s, encoding='utf-8'):
    if isinstance(s, six.text_type):
        return s.encode(encoding)
    elif isinstance(s, six.binary_type):
        # make sure the binary is in required encoding
        return s.decode(encoding).encode(encoding)
    else:
        raise ValueError('Can only convert types {0} and {1}'.format(six.text_type, six.binary_type))

Response: Force conversion of given string to binary type.


Instruction:
Create a Python 3 function to
function for listing filenames with given prefix and suffix in the root directory.

Response:
def get_filenames(root, prefix=u'', suffix=u''):
    """Function for listing filenames with given prefix and suffix in the root directory.

    Parameters
    ----------

    prefix: str
        The prefix of the required files.
    suffix: str
        The suffix of the required files

    Returns
    -------
    list of str
        List of filenames matching the prefix and suffix criteria.
    """
    return [fnm for fnm in os.listdir(root) if fnm.startswith(prefix) and fnm.endswith(suffix)]


Instruction:
Can you generate the documentation for the following Python 3 function
def tag(self, text):
        if self.search_method == 'ahocorasick':
            events = self._find_keywords_ahocorasick(text.text)
        elif self.search_method == 'naive':
            events = self._find_keywords_naive(text.text)

        events = self._resolve_conflicts(events)
        if self.mapping:
            for item in events:
                item['type'] = self.map[
                    text.text[item['start']:item['end']]
                ]

        if self.return_layer:
            return events
        else:
            text[self.layer_name] = events

Response: Returns list of keywords in text.


Instruction:
Implement a Python 3 function for
returning a list of regex_matches in text.

Response:
def tag(self, text):
        """Retrieves list of regex_matches in text.

        Parameters
        ----------
        text: Text
            The estnltk text object to search for events.

        Returns
        -------
        list of matches
        """
        matches = self._match(text.text)
        matches = self._resolve_conflicts(matches)

        if self.return_layer:
            return matches
        else:
            text[self.layer_name] = matches


Instruction:
Can you tell what is the following Python 3 function doing
def tag(self, text):
        if self.search_method == 'ahocorasick':
            events = self._find_events_ahocorasick(text.text)
        elif self.search_method == 'naive':
            events = self._find_events_naive(text.text)

        events = self._resolve_conflicts(events)

        self._event_intervals(events, text)

        if self.return_layer:
            return events
        else:
            text[self.layer_name] = events

Response: Returns the list of events in the text.


Instruction:
Create a Python 3 function for
extracting the lemmas from the found match

Response:
def __extract_lemmas(self, doc, m, phrase):
        """
        :param sent: sentence from which the match was found
        :param m: the found match
        :phrase: name of the phrase
        :return: tuple of the lemmas in the match
        """

        ph_start = m['start']
        ph_end = m['end']
        start_index = None
        for ind, word in enumerate(doc['words']):
            if word['start'] == ph_start:
                start_index = ind
                break
        end_index = None
        for ind, word in enumerate(doc['words']):
            if word['end'] == ph_end:
                end_index = ind
                break
        if start_index is not None and end_index is not None:

            lem = []

            for i in doc['words'][start_index:end_index + 1]:
                word_lem = []
                for idx, j in enumerate(i['analysis']):
                    if i['analysis'][idx]['partofspeech'] in ['A', 'D', 'C', 'J']:
                        if i['analysis'][idx]['lemma'] not in word_lem:
                            word_lem.append(i['analysis'][idx]['lemma'])
                word_lem_str = '|'.join(word_lem)
                lem.append(word_lem_str)

        else:
            raise Exception('Something went really wrong')
        return lem


Instruction:
Can you tell what is the following Python 3 function doing
def yield_json_corpus(fnm):
    with codecs.open(fnm, 'rb', 'ascii') as f:
        line = f.readline()
        while line != '':
            yield Text(json.loads(line))
            line = f.readline()

Response: Function to read a JSON corpus from a file.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def write_json_corpus(documents, fnm):
    with codecs.open(fnm, 'wb', 'ascii') as f:
        for document in documents:
            f.write(json.dumps(document) + '\n')
    return documents

Response: Writes a list of Text instances as JSON corpus on disk.


Instruction:
Here you have a function in Python 3, explain what it does
def read_document(fnm):
    with codecs.open(fnm, 'rb', 'ascii') as f:
        return Text(json.loads(f.read()))

Response: Read a document that is stored in a text file as JSON.


Instruction:
Create a Python 3 function for
writing a Text document to file.

Response:
def write_document(doc, fnm):
    """Write a Text document to file.

    Parameters
    ----------
    doc: Text
        The document to save.
    fnm: str
        The filename to save the document
    """
    with codecs.open(fnm, 'wb', 'ascii') as f:
        f.write(json.dumps(doc, indent=2))


Instruction:
Can you create a Python 3 function that
adds a relation to the sourceSynset.

Response:
def addRelation(sourceSynset,relationName,targetSynset):
    """
    Adds relation with name <relationName> to
    <targetSynset>.

    """
    if not isinstance(sourceSynset, Synset):
        raise TypeError("sourceSynset not Synset instance")
    elif not isinstance(targetSynset, Synset):
        raise TypeError("targetSynset not Synset instance")
    elif relationName not in RELATION_NAMES:
        raise TypeError("relationName not in RELATION_NAMES")
    else:
        sourceSynset.addRelation(
            Relation(relationName,targetSynset)
            )
        return sourceSynset


Instruction:
Create a Python 3 function to
append a feature to the internal list of features.

Response:
def addFeature(self, feature):
        '''Appends Feature'''

        if isinstance(feature, Feature):
            self.features.append(feature)
        else:
            raise TypeError(
                'feature Type should be Feature, not %s' % type(feature))


Instruction:
Here you have a function in Python 3, explain what it does
def addSourceId(self, value):
        '''Adds SourceId to External_Info
        '''
        if isinstance(value, Source_Id):
            self.source_ids.append(value)
        else:
            raise (TypeError,
                   'source_id Type should be Source_Id, not %s' % type(source_id))

Response: Adds SourceId to External_Info


Instruction:
Create a Python 3 function to
add SourceId to External_Info

Response:
def addCorpusId(self, value):
        '''Adds SourceId to External_Info
        '''
        if isinstance(value, Corpus_Id):
            self.corpus_ids.append(value)
        else:
            raise (TypeError,
                   'source_id Type should be Source_Id, not %s' % type(source_id))


Instruction:
Can you write a function in Python 3 where it
parses the line of an ewn file into the properties self. levelNumber self. DRN self. fieldValue and self. noQuotes.

Response:
def parse_line(self,iStr):
        """Parses ewn file line
        """
        self.levelNumber = None
        self.DRN = None
        self.fieldTag = None
        self.fieldValue = None
        self.noQuotes = None
        if iStr and not(iStr.strip().startswith('#')):
            iList = iStr.strip().split(' ')
            self.levelNumber = int(iList.pop(0))
            if iList[0].startswith('@') and self.levelNumber != 3:
                self.DRN = int(iList.pop(0).strip('@'))
            else:
                self.DRN = None
            self.fieldTag = iList.pop(0)
            if iList and (
                iList[0].startswith('"') or
                iList[0].startswith('@')
                ):
                fv = ' '.join(iList)
                self.fieldValue = fv[1:-1]
            elif iList:
                if len(iList) == 1:
                    self.fieldValue = iList.pop(0)
                else:
                    self.fieldValue = ' '.join(iList)
                try:
                    self.fieldValue = int(self.fieldValue)
                except ValueError:
                    self.noQuotes = True


Instruction:
Implement a function in Python 3 to
parse a Synset from a file.

Response:
def parse_synset(self, offset=None, debug=False):
        """Parses Synset from file
        """
        if False:
            pass
        else:
            # WORD_INSTANCE
            def _word_instance():
                _synset(True)

            # WORD_MEANING
            def _synset(pn=False):
                if not pn:
                    self.synset = Synset()
                    self.pn = False
                else:
                    self.synset = WordInstance()
                    self.pn = True
                if self.DRN:
                    self.synset.number = self.DRN
                self.targetType = None

            def _variants():
                self.synset.variants = Variants()

            def _literal():
                a = Variant()
                self.synset.variants.append(a)
                self.synset.variants[-1].literal = self.fieldValue

            def _target_literal():
                    self.target_synset.variants.append(Variant())
                    self.target_synset.variants[-1].literal = self.fieldValue

            def _sense():
                self.synset.variants[-1].sense = self.fieldValue

            def _status():
                self.noQuotes = True
                try:
                    self.synset.variants[-1].status = as_unicode(self.fieldValue)
                except:
                    self.synset.variants[-1].status = as_unicode(str(self.fieldValue))
                self.noQuotes = False

            def _target_sense():
                self.target_synset.variants[-1].sense = self.fieldValue
                if self.targetType == 'internal':
                    self.synset.internalLinks[
                        -1].target_concept = self.target_synset
                elif self.targetType == 'ili':
                    self.synset.eqLinks[-1].target_concept = self.target_synset
                elif self.targetType == 'pv':
                    self.synset.propertyValues[-1].value = self.target_synset
                else:
                    print ('BOOOOOOOOO!!') # Error TODO

            def _gloss():
                self.synset.variants[-1].gloss = self.fieldValue
                self.synset.definition = self.fieldValue                            # ADDED BY KOM

            def _translations():
                self.synset.variants[-1].translations = Translations()

            def _translation():
                self.synset.variants[-1].translations.append(
                    Translation(
                        language=self.fieldValue.split(':')[0],
                        translation_value = self.fieldValue.split(':')[1])
                    )

            def _examples():
                self.synset.variants[-1].examples = Examples()
            def _usage_labels():
                self.synset.variants[-1].usage_labels = Usage_Labels()

            def _external_info():
                self.synset.variants[-1].externalInfo = External_Info()

            def _example():
                self.synset.variants[-1].examples.append(
                    Example(self.fieldValue)
                    )

            def _usage_label():
                self.synset.variants[
                    -1].usage_labels.append(
                    Usage_Label(name=self.fieldValue)
                    )

            def _usage_label_value():
                self.synset.variants[
                    -1].usage_labels[-1].usage_label_value = self.fieldValue

            def _source_id():
                if self.targetType == 'internal':
                    self.synset.internalLinks[-1].source_id = self.fieldValue
                    # self.synset.internalLinks[-1].source_ids.append(
                    #     Relation_Source_Id(number=self.fieldValue))
                elif self.targetType == 'ili':
                    self.synset.eqLinks[-1].source_id = self.fieldValue
                    # self.synset.eqLinks[-1].source_ids.append(
                    #     Relation_Source_Id(number=self.fieldValue))

                else:
                    if self.synset.variants[-1].external_info:
                        self.synset.variants[
                            -1].external_info.source_ids.append(
                            Source_Id(number=self.fieldValue)
                            )
                    else:
                        self.synset.variants[-1].external_info = External_Info()
                        self.synset.variants[
                            -1].external_info.source_ids.append(
                            Source_Id(number=self.fieldValue)
                            )

            def _corpus_id():
                if self.targetType == 'internal': # not needed
                    self.synset.internalLinks[-1].corpus_ids.append(
                        Relation_Corpus_Id(number=self.fieldValue))
                else:
                    if self.synset.variants[-1].external_info:
                        self.synset.variants[
                            -1].external_info.corpus_ids.append(
                            Corpus_Id(number=self.fieldValue)
                            )
                    else:
                        self.synset.variants[-1].external_info = External_Info()
                        self.synset.variants[
                            -1].external_info.corpus_ids.append(
                            Corpus_Id(number=self.fieldValue)
                            )

            def _frequency():
                self.synset.variants[
                    -1].external_info.corpus_ids[-1].frequency = self.fieldValue

            def _text_key():
                self.synset.variants[
                    -1].external_info.source_ids[-1].text_key = self.fieldValue

            def _number_key():
                self.synset.variants[
                    -1].external_info.source_ids[
                    -1].number_key = self.fieldValue
            def _pos():
                self.synset.pos = self.fieldValue

            # INTERNAL_LINKS
            def _target_concept():
                self.target_synset = Synset()
                self.target_synset.variants = Variants()
                if self.levelNumber == 3: # and self.fieldValue:
                    self.target_synset.number = int(self.fieldValue or 0)

            def _target_pos():
                self.target_synset.pos = self.fieldValue

            def _internal_links():
                self.synset.internalLinks = InternalLinks()
                self.targetType = 'internal'

            def _relation():
                if self.targetType == 'internal':
                    self.synset.internalLinks.append(Relation())
                    self.synset.internalLinks[-1].name = self.fieldValue
                elif self.targetType == 'ili':
                    self.synset.eqLinks.append(EqLink())
                    self.synset.eqLinks[-1].name = self.fieldValue
                else:
                    print ('BOOOOOOOOO!!') # Error TODO

            def _features():
                if self.targetType == 'internal':
                    self.synset.internalLinks[-1].features = Features()
                else:
                    self.synset.variants[-1].features = Features()
                    self.synset.variants[-1].features.append(Feature())

            def _feature():
                self.synset.variants[-1].features[-1].name = self.fieldValue

            def _feature_value():
                self.synset.variants[
                    -1].features[-1].featureValue = self.fieldValue

            def _reversed():
                self.synset.internalLinks[-1].features.append(Feature())
                self.synset.internalLinks[-1].features[-1].name = self.fieldTag
                self.synset.internalLinks[-1].features[-1].featureValue = True

            def _variant_to_variant():
                self.synset.internalLinks[-1].features.append(Feature())
                self.synset.internalLinks[-1].features[-1].name = self.fieldTag

            def _source_variant():
                self.variant_to_variant_source = self.fieldValue

            def _target_variant():
                self.variant_to_variant_target = self.fieldValue
                self.synset.internalLinks[
                    -1].features[-1].featureValue = (
                    self.variant_to_variant_source,
                    self.variant_to_variant_target)

            # EQ_LINKS
            def _eq_links():
                self.synset.eqLinks = EqLinks()
                self.targetType = 'ili'

            def _wn_offset():
                self.target_synset.wordnet_offset = self.fieldValue
                self.synset.eqLinks[-1].target_concept = self.target_synset

            def _add_on_id():
                self.target_synset.add_on_id = self.fieldValue
                self.synset.eqLinks[-1].target_concept = self.target_synset

            # PROPERTIES
            def _properties():
                self.synset.properties = Properties()

            def _name():
                if self.pn:
                    self.synset.propertyValues.append(
                        PropertyValue(name=self.fieldValue))
                else:
                    self.synset.properties.append(Property(self.fieldValue))

            # PROPERTY_VALUES
            def _property_values():
                self.synset.propertyValues = PropertyValues()

            def _property_value():
                self.synset.propertyValues[-1].value = self.fieldValue
                self.targetType = 'pv'

            def _property_wm():
                pass

            rulez = {
                (0,'WORD_MEANING'): _synset,
                (0,'WORD_INSTANCE'): _word_instance,
                (1,'PART_OF_SPEECH'): _pos,
                (1,'VARIANTS'): _variants,
                (2,'LITERAL'): _literal,
                (3,'SENSE'): _sense,
                (3,'STATUS'): _status,
                (3,'DEFINITION'): _gloss,
                (3,'EXAMPLES'): _examples,
                (3,'USAGE_LABELS'): _usage_labels,
                (4,'USAGE_LABEL'): _usage_label,
                (5,'USAGE_LABEL_VALUE'): _usage_label_value,
                (4,'EXAMPLE'): _example,
                (3,'TRANSLATIONS'): _translations,
                (4,'TRANSLATION'): _translation,
                (3,'EXTERNAL_INFO'): _external_info,
                (4,'SOURCE_ID'): _source_id,
                (4,'CORPUS_ID'): _corpus_id,
                (5,'FREQUENCY'): _frequency,
                (5,'TEXT_KEY'): _text_key,
                (5,'NUMBER_KEY'): _number_key,
                (1,'INTERNAL_LINKS'): _internal_links,
                (2,'RELATION'): _relation,
                (3,'TARGET_CONCEPT'): _target_concept,
                (4,'PART_OF_SPEECH'): _target_pos,
                (4,'LITERAL'): _target_literal,
                (5,'SENSE'): _target_sense,
                (3,'FEATURES'): _features,
                (4,'FEATURE'): _feature,
                (5,'FEATURE_VALUE'): _feature_value,
                (4,'REVERSED'): _reversed,
                (4,'VARIANT_TO_VARIANT'): _variant_to_variant,
                (5,'SOURCE_VARIANT'): _source_variant,
                (5,'TARGET_VARIANT'): _target_variant,
                (3,'SOURCE_ID'): _source_id,
                (1,'EQ_LINKS'): _eq_links,
                (2,'EQ_RELATION'): _relation,
                (3,'TARGET_ILI'): _target_concept,
                (4,'WORDNET_OFFSET'): _wn_offset,
                (4,'ADD_ON_ID'): _add_on_id,
                (1,'PROPERTIES'): _properties,
                (1,'PROPERTY_VALUES'): _property_values,
                (2,'NAME'): _name,
                (3,'VALUE'): _property_value,
                (3,'VALUE_AS_TEXT'): _property_value,
                (3,'VALUE_AS_WORD_MEANING'): _target_concept,
                }

            if not offset:
                offset = self.milestone
            else:
                self.milestone=offset
            if self.file:
                self.file.seek(offset,0)

            line = 'X'
            ili = False
            var = False
            while line.strip():

                offset = self.file.tell()
                self.file.seek(offset,0)

                line = as_unicode(self.file.readline(), self.encoding).strip()
                if debug:
                    print (line.encode('utf-8'))

                self.parse_line(line)
                self.noQuotes = None

                select = (self.levelNumber,self.fieldTag)

                if select in rulez.keys():
                    rulez[select]()
                else:
                    if line:
                        print (self.synset.polarisText)
                        raise ParseError("No parsing rule for '%s'" % line)
        return self.synset


Instruction:
Can you generate a brief explanation for the following Python 3 code
def parse_wordnet(self,debug=False):
        '''Parses wordnet from
        <self.file>
        '''
        synList = []
        self.milestone = 0 # to start from beginning of file
        while self.milestone < os.path.getsize(self.fileName) - 5:
            if debug:
                print ('self.milestone', self.milestone)
            a = self.parse_synset(offset=self.milestone)
            synList.append(a)
            self.milestone = self.file.tell()
        return synList

Response:
Parses wordnet from
        <self. file >


Instruction:
Can you create a Python 3 function that
appends one Translation to translations AttributeNames

Response:
def addTranslation(self,translation):
        '''Appends one Translation to translations
        '''
        if isinstance(translation, Translation):
            self.translations.append(translation)
        else:
            raise(TranslationError,
                   'translation Type should be Translation, not %s' % type(
                    translation)
                   )


Instruction:
Create a Python 3 function for
appending one VariantFeature to variantFeatures AttributeNames

Response:
def addVariantFeature(self,variantFeature):
        '''Appends one VariantFeature to variantFeatures
        '''
        if isinstance(variantFeature, Feature):
            self.features.append(variantFeature)
        else:
            raise(TypeError,
                  'variantFeature Type should be Feature, not %s' % type(
                    variantFeature)
                  )


Instruction:
Given the following Python 3 function, write the documentation
def addUsage_Label(self,usage_label):
        '''Appends one Usage_Label to usage_labels
        '''
        if isinstance(usage_label, Usage_Label):
            self.usage_labels.append(usage_label)
        else:
            raise (Usage_LabelError,
                   'usage_label Type should be Usage_Label, not %s' % type(
                    usage_label)
                   )

Response: Appends one Usage_Label to usage_labels


Instruction:
Can you implement a function in Python 3 that
appends one Example to examples AttributeNames

Response:
def addExample(self,example):
        '''Appends one Example to examples
        '''
        if isinstance(example, Example):
            self.examples.append(example)
        else:
            raise (ExampleError,
                   'example Type should be Example, not %s' % type(example)
                   )


Instruction:
Write a Python 3 script to
return the first variant of Variants AttributeNames Read - only

Response:
def firstVariant():
        """first variant of Variants
        Read-only

        """
        def fget(self):
            if self.variants:
                return self.variants[0]
            else:
                variant = Variant()
                return variant

        return locals()


Instruction:
How would you explain what the following Python 3 function does
def literals():
        '''Returns a list of literals
        in the Synset
        read-only
        '''
        def fget(self):
            if self.variants:
                return map(lambda x: x.literal,
                           self.variants)
            else:
                return None

        return locals()

Response:
Returns a list of literals
        in the Synset
        read - only


Instruction:
Explain what the following Python 3 code does
def addVariantOld(self,
                      literal='',
                      sense=0,
                      gloss='',
                      examples=[]):
        '''Appends variant
        sth to do that it would be possible
        to add Variant object

        '''
        var = Variant(literal=literal,
                      sense=sense,
                      gloss=gloss,
                      examples=examples)
        self.variants.append(var)

Response: Appends a Variant object to the internal list of variants.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def addVariant(self,variant):
        '''Appends one Variant to variants

        '''
        if isinstance(variant, Variant):
            self.variants.append(variant)
        else:
            raise (VariantError,
                   'variant Type should be Variant, not %s' % type(variant))

Response: Appends one Variant to the internal list of variants


Instruction:
Make a summary of the following Python 3 code
def addInternalLink(self, link):
        '''Appends InternalLink

        '''
        if isinstance(link, InternalLink):
            self.internalLinks.append(link)
        else:
            raise InternalLinkError(
                'link Type should be InternalLink, not %s' % type(link))

Response: Appends an internal link to the internalLinks list.


Instruction:
Create a Python 3 function for
appending a relation to the internalLinks list

Response:
def addRelation(self, link):
        '''Appends Relation

        '''
        if isinstance(link, Relation):
            self.internalLinks.append(link)
        else:
            raise TypeError(
                'link Type should be InternalLink, not %s' % type(link))


Instruction:
How would you explain what the following Python 3 function does
def addEqLink(self, link):
        '''Appends EqLink

        '''
        if isinstance(link, EqLink):
            self.eqLinks.append(link)
        else:
            raise TypeError(
                'link Type should be InternalLink, not %s' % type(link))

Response: Appends an internal link to the internal link list


Instruction:
Write a Python 3 script to
return list of named Relations. <name > may be string or list.

Response:
def named_relations(self, name, neg=False):
        '''Returns list of named Relations.

        <name> may be string or list.
        '''
        if self.internalLinks and not neg:

            if isinstance(name, six.string_types):
                return filter(lambda x: x.name == name,
                              self.internalLinks)

            elif isinstance(name, list):
                return filter(lambda x: x.name in name,
                              self.internalLinks)
            else:
                return None #should rise error

        elif self.internalLinks and neg:

            if isinstance(name, six.string_types):
                return filter(lambda x: x.name != name,
                              self.internalLinks)

            elif isinstance(name, list):
                return filter(lambda x: x.name not in name,
                              self.internalLinks)
            else:
                return None #should rise error

        else:
            return []


Instruction:
Explain what the following Python 3 code does
def named_eq_relations(self, name, neg=False):
        '''Returns list of named eqLinks.

        <name> may be string or list.
        '''
        if self.eqLinks and not neg:

            if isinstance(name, six.string_types):
                return filter(lambda x: x.relation.name == name,
                              self.eqLinks)

            elif isinstance(name, list):
                return filter(lambda x: x.relation.name in name,
                              self.eqLinks)
            else:
                return None #should rise error

        elif self.eqLinks and neg:

            if isinstance(name, six.string_types):
                return filter(lambda x: x.relation.name != name,
                              self.eqLinks)

            elif isinstance(name, list):
                return filter(lambda x: x.relation.name not in name,
                              self.eqLinks)
            else:
                return None #should rise error
        else:
            return None

Response:
Returns list of named eqLinks.

            <name > may be string or list.
            <name > may be list.


Instruction:
Create a Python 3 function to
parse synset from file <fileName > from offset offset

Response:
def parse(self,fileName,offset):
        '''Parses synset from file <fileName>
        from offset <offset>
        '''
        p = Parser()
        p.file = open(fileName, 'rb')
        a = p.parse_synset(offset=offset)
        p.file.close()
        self.__dict__.update(a.__dict__)


Instruction:
Write a Python 3 script for
appending synset to Polaris IO file <fileName >

Response:
def write(self,fileName):
        '''Appends synset to Polaris IO file <fileName>
        '''
        f = open(fileName, 'ab')
        f.write('%s%s' % (self.polarisText,
                          Synset.linebreak)
                )
        f.close()


Instruction:
Implement a Python 3 function for
fixing out - of - the - sentence links in the given sentence.

Response:
def _fix_out_of_sentence_links( alignments, sent_start, sent_end ):
    ''' Fixes out-of-the-sentence links in the given sentence.
        The sentence is a sublist of *alignments*, starting from
        *sent_start* and ending one token before *sent_end*;
    '''
    sent_len = sent_end - sent_start
    j = sent_start
    while j < sent_start + sent_len:
        for rel_id, rel in enumerate( alignments[j][PARSER_OUT] ):
            if int( rel[1] ) >= sent_len:
                # If the link points out-of-the-sentence, fix
                # the link so that it points inside the sentence
                # boundaries:
                wid = j - sent_start
                if sent_len == 1:
                    # a single word becomes a root
                    rel[1] = -1
                elif wid-1 > -1:
                    # word at the middle/end is linked to the previous
                    rel[1] = wid - 1
                elif wid-1 == -1:
                    # word at the beginning is linked to the next
                    rel[1] = wid + 1
                alignments[j][PARSER_OUT][rel_id] = rel
        j += 1


Instruction:
How would you implement a function in Python 3 that
normalizes dependency syntactic information in a given list of alignments.

Response:
def normalise_alignments( alignments, data_type=VISLCG3_DATA, **kwargs ):
    ''' Normalises dependency syntactic information in the given list of alignments.
        *) Translates tree node indices from the syntax format (indices starting
           from 1), to EstNLTK format (indices starting from 0);
        *) Removes redundant information (morphological analyses) and keeps only
           syntactic information, in the most compact format;
        *) Brings MaltParser and VISLCG3 info into common format;

        Expects that the list of alignments contains dicts, where each dict has
        following attributes (at minimum):
          'start'   -- start index of the word in Text;
          'end'     -- end index of the word in Text;
          'sent_id' -- index of the sentence in Text, starting from 0;
          'parser_out' -- list of analyses from the output of the syntactic parser;
        Assumes that dicts are listed in the order of words appearance in the text;
        ( basically, assumes the output of methods align_CONLL_with_Text() and
          align_cg3_with_Text() )

        Returns the input list (alignments), where old analysis lines ('parser_out') have
        been replaced with the new compact form of analyses (if keep_old == False), or where
        old analysis lines ('parser_out') have been replaced the new compact form of analyses,
        and the old analysis lines are preserved under a separate key: 'init_parser_out' (if
        keep_old == True);

        In the compact list of analyses, each item has the following structure:
           [ syntactic_label, index_of_the_head ]
         *) syntactic_label
                surface syntactic label of the word, e.g. '@SUBJ', '@OBJ', '@ADVL'
         *) index_of_the_head
                index of the head; -1 if the current token is root;

        Parameters
        -----------
        alignments : list of items
            A list of dicts, where each item/dict has following attributes:
            'start', 'end', 'sent_id', 'parser_out'

        data_type : str
            Type of data in list_of_analysis_lines; Possible types: 'vislcg3'
            (default), and 'conll';

        rep_miss_w_dummy : bool
            Optional argument specifying whether missing analyses should be replaced
            with dummy analyses ( in the form ['xxx', link_to_self] ); If False,
            an Exception is raised in case of a missing analysis;
            Default:True

        fix_selfrefs : bool
            Optional argument specifying  whether  self-references  in  syntactic
            dependencies should be fixed;
            A self-reference link is firstly re-oriented as a link to the previous word
            in the sentence, and if the previous word does  not  exist,  the  link  is
            re-oriented to the next word in the sentence; If the self-linked word is
            the only word in the sentence, it is made the root of the sentence;
            Default:True

        fix_out_of_sent : bool
            Optional argument specifying whether references pointing out of the sentence
            (the parent index exceeds the sentence boundaries) should be fixed;
            The logic used in fixing out-of-sentence links is the same as the logic for
            fix_selfrefs;
            Default:False

        keep_old : bool
            Optional argument specifying  whether the old analysis lines should be
            preserved after overwriting 'parser_out' with new analysis lines;
            If True, each dict will be augmented with key 'init_parser_out' which
            contains the initial/old analysis lines;
            Default:False

        mark_root : bool
            Optional argument specifying whether the root node in the dependency tree
            (the node pointing to -1) should be assigned the label 'ROOT' (regardless
            its current label).
            This might be required, if one wants to make MaltParser's and VISLCG3 out-
            puts more similar, as MaltParser currently uses 'ROOT' labels, while VISLCG3
            does not;
            Default:False

        (Example text: 'Millega pitsat tellida ? Hea küsimus .')
        Example input (VISLC3):
        -----------------------
        {'end': 7, 'sent_id': 0, 'start': 0, 'parser_out': ['\t"mis" Lga P inter rel sg kom @NN> @ADVL #1->3\r']}
        {'end': 14, 'sent_id': 0, 'start': 8, 'parser_out': ['\t"pitsa" Lt S com sg part @OBJ #2->3\r']}
        {'end': 22, 'sent_id': 0, 'start': 15, 'parser_out': ['\t"telli" Lda V main inf @IMV #3->0\r']}
        {'end': 23, 'sent_id': 0, 'start': 22, 'parser_out': ['\t"?" Z Int CLB #4->4\r']}
        {'end': 27, 'sent_id': 1, 'start': 24, 'parser_out': ['\t"hea" L0 A pos sg nom @AN> #1->2\r']}
        {'end': 35, 'sent_id': 1, 'start': 28, 'parser_out': ['\t"küsimus" L0 S com sg nom @SUBJ #2->0\r']}
        {'end': 36, 'sent_id': 1, 'start': 35, 'parser_out': ['\t"." Z Fst CLB #3->3\r']}

        Example output:
        ---------------
        {'sent_id': 0, 'start': 0, 'end': 7, 'parser_out': [['@NN>', 2], ['@ADVL', 2]]}
        {'sent_id': 0, 'start': 8, 'end': 14, 'parser_out': [['@OBJ', 2]]}
        {'sent_id': 0, 'start': 15, 'end': 22, 'parser_out': [['@IMV', -1]]}
        {'sent_id': 0, 'start': 22, 'end': 23, 'parser_out': [['xxx', 2]]}
        {'sent_id': 1, 'start': 24, 'end': 27, 'parser_out': [['@AN>', 1]]}
        {'sent_id': 1, 'start': 28, 'end': 35, 'parser_out': [['@SUBJ', -1]]}
        {'sent_id': 1, 'start': 35, 'end': 36, 'parser_out': [['xxx', 1]]}

    '''
    if not isinstance( alignments, list ):
        raise Exception('(!) Unexpected type of input argument! Expected a list of strings.')
    if data_type.lower() == VISLCG3_DATA:
       data_type = VISLCG3_DATA
    elif data_type.lower() == CONLL_DATA:
       data_type = CONLL_DATA
    else:
       raise Exception('(!) Unexpected type of data: ', data_type)
    keep_old         = False
    rep_miss_w_dummy = True
    mark_root        = False
    fix_selfrefs     = True
    fix_out_of_sent  = False
    for argName, argVal in kwargs.items():
        if argName in ['selfrefs', 'fix_selfrefs'] and argVal in [True, False]:
           #  Fix self-references
           fix_selfrefs = argVal
        if argName in ['keep_old'] and argVal in [True, False]:
           #  After the normalisation, keep also the original analyses;
           keep_old = argVal
        if argName in ['rep_miss_w_dummy', 'rep_miss'] and argVal in [True, False]:
           #  Replace missing analyses with dummy analyses;
           rep_miss_w_dummy = argVal
        if argName in ['mark_root', 'root'] and argVal in [True, False]:
           #  Mark the root node in the syntactic tree with the label ROOT;
           mark_root = argVal
        if argName in ['fix_out_of_sent']:
           #  Fix links pointing out of the sentence;
           fix_out_of_sent = bool(argVal)
    # Iterate over the alignments and normalise information
    prev_sent_id = -1
    wordID       = 0
    sentStart    = -1
    for i in range(len(alignments)):
        alignment = alignments[i]
        if prev_sent_id != alignment[SENT_ID]:
            # Detect and fix out-of-the-sentence links in the previous sentence (if required)
            if fix_out_of_sent and sentStart > -1:
                _fix_out_of_sentence_links( alignments, sentStart, i )
            # Start of a new sentence: reset word id
            wordID = 0
            sentStart = i
        # 1) Extract syntactic information
        foundRelations = []
        if data_type == VISLCG3_DATA:
            # *****************  VISLCG3 format
            for line in alignment[PARSER_OUT]:
                # Extract info from VISLCG3 format analysis:
                sfuncs  = pat_cg3_surface_rel.findall( line )
                deprels = pat_cg3_dep_rel.findall( line )
                # If sfuncs is empty, generate an empty syntactic function (e.g. for
                # punctuation)
                sfuncs = ['xxx'] if not sfuncs else sfuncs
                # Generate all pairs of labels vs dependency
                for func in sfuncs:
                    for (relS,relT) in deprels:
                        relS = int(relS)-1
                        relT = int(relT)-1
                        foundRelations.append( [func, relT] )
        elif data_type == CONLL_DATA:
            # *****************  CONLL format
            for line in alignment[PARSER_OUT]:
                parts = line.split('\t')
                if len(parts) != 10:
                    raise Exception('(!) Unexpected line format for CONLL data:', line)
                relT = int( parts[6] ) - 1
                func = parts[7]
                foundRelations.append( [func, relT] )
        # Handle missing relations (VISLCG3 specific problem)
        if not foundRelations:
            # If no alignments were found (probably due to an error in analysis)
            if rep_miss_w_dummy:
                # Replace missing analysis with a dummy analysis, with dep link
                # pointing to self;
                foundRelations.append( ['xxx', wordID] )
            else:
                raise Exception('(!) Analysis missing for the word nr.', alignment[0])
        # Fix self references ( if requested )
        if fix_selfrefs:
            for r in range(len(foundRelations)):
                if foundRelations[r][1] == wordID:
                    # Make it to point to the previous word in the sentence,
                    # and if the previous one does not exist, make it to point
                    # to the next word;
                    foundRelations[r][1] = \
                        wordID-1 if wordID-1 > -1 else wordID+1
                    # If the self-linked token is the only token in the sentence,
                    # mark it as the root of the sentence:
                    if wordID-1 == -1 and (i+1 == len(alignments) or \
                       alignments[i][SENT_ID] != alignments[i+1][SENT_ID]):
                        foundRelations[r][1] = -1
        # Mark the root node in the syntactic tree with the label ROOT ( if requested )
        if mark_root:
            for r in range(len(foundRelations)):
                if foundRelations[r][1] == -1:
                    foundRelations[r][0] = 'ROOT'
        # 2) Replace existing syntactic info with more compact info
        if not keep_old:
            # Overwrite old info
            alignment[PARSER_OUT] = foundRelations
        else:
            # or preserve the initial information, and add new compact information
            alignment[INIT_PARSER_OUT] = alignment[PARSER_OUT]
            alignment[PARSER_OUT]      = foundRelations
        alignments[i] = alignment
        prev_sent_id = alignment[SENT_ID]
        # Increase word id
        wordID += 1
    # Detect and fix out-of-the-sentence links in the last sentence (if required)
    if fix_out_of_sent and sentStart > -1:
        _fix_out_of_sentence_links( alignments, sentStart, len(alignments) )
    return alignments


Instruction:
Create a Python 3 function for
reading the output of VISLCG3 syntactic analysis from a given file and returns as a Text object.

Response:
def read_text_from_cg3_file( file_name, layer_name=LAYER_VISLCG3, **kwargs ):
    ''' Reads the output of VISLCG3 syntactic analysis from given file, and
        returns as a Text object.

        The Text object has been tokenized for paragraphs, sentences, words, and it
        contains syntactic analyses aligned with word spans, in the layer *layer_name*
        (by default: LAYER_VISLCG3);

        Attached syntactic analyses are in the format as is the output of
          utils.normalise_alignments();

        Note: when loading data from  https://github.com/EstSyntax/EDT  corpus,
        it  is  advisable  to  add  flags:  clean_up=True,  fix_sent_tags=True,
        fix_out_of_sent=True  in order to ensure that well-formed data will be
        read from the corpus;

        Parameters
        -----------
        file_name : str
            Name of the input file; Should contain syntactically analysed text,
            following the format of the output of VISLCG3 syntactic analyser;

        clean_up : bool
            Optional argument specifying whether the vislcg3_syntax.cleanup_lines()
            should be applied in the lines of syntactic analyses read from the
            file;
            Default: False

        layer_name : str
            Name of the Text's layer in which syntactic analyses are stored;
            Defaults to 'vislcg3_syntax';

            For other parameters, see optional parameters of the methods:

             utils.normalise_alignments():          "rep_miss_w_dummy", "fix_selfrefs",
                                                    "keep_old", "mark_root";
             vislcg3_syntax.align_cg3_with_Text():  "check_tokens", "add_word_ids";
             vislcg3_syntax.cleanup_lines():        "remove_caps", "remove_clo",
                                                    "double_quotes", "fix_sent_tags"

    '''
    clean_up = False
    for argName, argVal in kwargs.items():
        if argName in ['clean_up', 'cleanup'] and argVal in [True, False]:
           #  Clean up lines
           clean_up = argVal
    # 1) Load vislcg3 analysed text from file
    cg3_lines = []
    in_f = codecs.open(file_name, mode='r', encoding='utf-8')
    for line in in_f:
        # Skip comment lines
        if line.startswith('#'):
            continue
        cg3_lines.append( line.rstrip() )
    in_f.close()
    # Clean up lines of syntactic analyses (if requested)
    if clean_up:
        cg3_lines = cleanup_lines( cg3_lines, **kwargs )

    # 2) Extract sentences and word tokens
    sentences = []
    sentence  = []
    for i, line in enumerate( cg3_lines ):
        if line == '"<s>"':
            if sentence:
                print('(!) Sentence begins before previous ends at line: '+str(i), \
                      file=sys.stderr)
            sentence  = []
        elif pat_double_quoted.match( line ) and line != '"<s>"' and line != '"</s>"':
            token_match = pat_cg3_word_token.match( line )
            if token_match:
                line = token_match.group(1)
            else:
                raise Exception('(!) Unexpected token format: ', line)
            sentence.append( line )
        elif line == '"</s>"':
            if not sentence:
                print('(!) Empty sentence at line: '+str(i), \
                      file=sys.stderr)
            # (!) Use double space instead of single space in order to distinguish
            #     word-tokenizing space from the single space in the multiwords
            #     (e.g. 'Rio de Janeiro' as a single word);
            sentences.append( '  '.join(sentence) )
            sentence = []

    # 3) Construct the estnltk's Text
    kwargs4text = {
      # Use custom tokenization utils in order to preserve exactly the same
      # tokenization as was in the input;
      "word_tokenizer": RegexpTokenizer("  ", gaps=True),
      "sentence_tokenizer": LineTokenizer()
    }
    from estnltk.text import Text
    text = Text( '\n'.join(sentences), **kwargs4text )
    # Tokenize up to the words layer
    text.tokenize_words()

    # 4) Align syntactic analyses with the Text
    alignments = align_cg3_with_Text( cg3_lines, text, **kwargs )
    normalise_alignments( alignments, data_type=VISLCG3_DATA, **kwargs )
    # Attach alignments to the text
    text[ layer_name ] = alignments
    return text


Instruction:
Explain what the following Python 3 code does
def read_text_from_conll_file( file_name, layer_name=LAYER_CONLL, **kwargs ):
    ''' Reads the CONLL format syntactic analysis from given file, and returns as
        a Text object.

        The Text object has been tokenized for paragraphs, sentences, words, and it
        contains syntactic analyses aligned with word spans, in the layer *layer_name*
        (by default: LAYER_CONLL);

        Attached syntactic analyses are in the format as is the output of
          utils.normalise_alignments();

        Parameters
        -----------
        file_name : str
            Name of the input file; Should contain syntactically analysed text,
            following the CONLL format;

        layer_name : str
            Name of the Text's layer in which syntactic analyses are stored;
            Defaults to 'conll_syntax';

            For other parameters, see optional parameters of the methods:

             utils.normalise_alignments():          "rep_miss_w_dummy", "fix_selfrefs",
                                                    "keep_old", "mark_root";
             maltparser_support.align_CONLL_with_Text():  "check_tokens", "add_word_ids";

    '''
    # 1) Load conll analysed text from file
    conll_lines = []
    in_f = codecs.open(file_name, mode='r', encoding='utf-8')
    for line in in_f:
        # Skip comment lines
        if line.startswith('#'):
            continue
        conll_lines.append( line.rstrip() )
    in_f.close()

    # 2) Extract sentences and word tokens
    sentences = []
    sentence  = []
    for i, line in enumerate( conll_lines ):
        if len(line) > 0 and '\t' in line:
            features = line.split('\t')
            if len(features) != 10:
                raise Exception(' In file '+in_file+', line '+str(i)+\
                                ' with unexpected format: "'+line+'" ')
            word_id = features[0]
            token   = features[1]
            sentence.append( token )
        elif len(line)==0 or re.match('^\s+$', line):
            # End of a sentence
            if sentence:
               # (!) Use double space instead of single space in order to distinguish
               #     word-tokenizing space from the single space in the multiwords
               #     (e.g. 'Rio de Janeiro' as a single word);
               sentences.append( '  '.join(sentence) )
            sentence = []
    if sentence:
        sentences.append( '  '.join(sentence) )

    # 3) Construct the estnltk's Text
    kwargs4text = {
      # Use custom tokenization utils in order to preserve exactly the same
      # tokenization as was in the input;
      "word_tokenizer": RegexpTokenizer("  ", gaps=True),
      "sentence_tokenizer": LineTokenizer()
    }
    from estnltk.text import Text
    text = Text( '\n'.join(sentences), **kwargs4text )
    # Tokenize up to the words layer
    text.tokenize_words()

    # 4) Align syntactic analyses with the Text
    alignments = align_CONLL_with_Text( conll_lines, text, None, **kwargs )
    normalise_alignments( alignments, data_type=CONLL_DATA, **kwargs )
    # Attach alignments to the text
    text[ layer_name ] = alignments
    return text

Response: Reads the CONLL format syntactic analysis from a file and returns as a Text object.


Instruction:
Write a Python 3 function that can
give a list of word tokens and a list of dependency syntactic relations build a list of trees from the given sentence.

Response:
def build_trees_from_sentence( sentence, syntactic_relations, layer=LAYER_VISLCG3, \
                               sentence_id=0, **kwargs ):
    ''' Given a sentence ( a list of EstNLTK's word tokens ), and a list of
        dependency syntactic relations ( output of normalise_alignments() ),
        builds trees ( estnltk.syntax.utils.Tree objects ) from the sentence,
        and returns as a list of Trees (roots of trees).

        Note that  there  is  one-to-many  correspondence  between  EstNLTK's
        sentences and dependency syntactic trees, so the resulting list can
        contain more than one tree (root);
    '''
    trees_of_sentence = []
    nodes = [ -1 ]
    while( len(nodes) > 0 ):
        node = nodes.pop(0)
        # Find tokens in the sentence that take this node as their parent
        for i, syntax_token in enumerate( syntactic_relations ):
            parents = [ o[1] for o in syntax_token[PARSER_OUT] ]
            # There should be only one parent node; If there is more than one, take the
            # first node;
            parent = parents[0]
            if parent == node:
                labels  = [ o[0] for o in syntax_token[PARSER_OUT] ]
                estnltk_token = sentence[i]
                tree1 = Tree( estnltk_token, i, sentence_id, labels, parser=layer )
                if INIT_PARSER_OUT in syntax_token:
                    tree1.parser_output = syntax_token[INIT_PARSER_OUT]
                tree1.syntax_token = syntax_token
                if parent == -1:
                    # Add the root node
                    trees_of_sentence.append( tree1 )
                elif parent == i:
                    # If, for some strange reason, the node is unnormalised and is still
                    # linked to itself, add it as a singleton tree
                    trees_of_sentence.append( tree1 )
                else:
                    # For each root node, attempt to add the child
                    for root_node in trees_of_sentence:
                        root_node.add_child_to_subtree( parent, tree1 )
                if parent != i:
                   # Add the current node as a future parent to be examined
                   nodes.append( i )
    return trees_of_sentence


Instruction:
Implement a function in Python 3 to
give a text object and the name of the layer where dependency syntactic Relations are stored builds trees from all the sentences of the text and returns as a list of Trees.

Response:
def build_trees_from_text( text, layer, **kwargs ):
    ''' Given a text object and the name of the layer where dependency syntactic
        relations are stored, builds trees ( estnltk.syntax.utils.Tree objects )
        from all the sentences of the text and returns as a list of Trees.

        Uses the method  build_trees_from_sentence()  for acquiring trees of each
        sentence;

        Note that there is one-to-many correspondence between EstNLTK's sentences
        and dependency syntactic trees: one sentence can evoke multiple trees;
    '''
    from estnltk.text import Text
    assert isinstance(text, Text), \
           '(!) Unexpected text argument! Should be Estnltk\'s Text object.'
    assert layer in text, \
           '(!) The layer '+str(layer)+' is missing from the input text.'
    text_sentences = list( text.divide( layer=WORDS, by=SENTENCES ) )
    all_sentence_trees = []  # Collected sentence trees
    prev_sent_id       = -1
    #  (!) Note: if the Text object has been split into smaller Texts with split_by(),
    #      SENT_ID-s still refer to old text, and thus are not useful as indices
    #      anymore;
    #      Therefore, we also use another variable -- norm_prev_sent_id -- that always
    #      counts sentences starting from 0, and use  SENT_ID / prev_sent_id  only for
    #      deciding whether one sentence ends and another begins;
    norm_prev_sent_id  = -1
    current_sentence   = []
    k = 0
    while k < len( text[layer] ):
        node_desc = text[layer][k]
        if prev_sent_id != node_desc[SENT_ID] and current_sentence:
            norm_prev_sent_id += 1
            # If the index of the sentence has changed, and we have collected a sentence,
            # then build tree(s) from this sentence
            assert norm_prev_sent_id<len(text_sentences), '(!) Sentence with the index '+str(norm_prev_sent_id)+\
                                                          ' not found from the input text.'
            sentence = text_sentences[norm_prev_sent_id]
            trees_of_sentence = \
                build_trees_from_sentence( sentence, current_sentence, layer, sentence_id=norm_prev_sent_id, \
                                           **kwargs )
            # Record trees constructed from this sentence
            all_sentence_trees.extend( trees_of_sentence )
            # Reset the sentence collector
            current_sentence = []
        # Collect sentence
        current_sentence.append( node_desc )
        prev_sent_id = node_desc[SENT_ID]
        k += 1
    if current_sentence:
        norm_prev_sent_id += 1
        assert norm_prev_sent_id<len(text_sentences), '(!) Sentence with the index '+str(norm_prev_sent_id)+\
                                                      ' not found from the input text.'
        sentence = text_sentences[norm_prev_sent_id]
        # If we have collected a sentence, then build tree(s) from this sentence
        trees_of_sentence = \
            build_trees_from_sentence( sentence, current_sentence, layer, sentence_id=norm_prev_sent_id, \
                                       **kwargs )
        # Record trees constructed from this sentence
        all_sentence_trees.extend( trees_of_sentence )
    return all_sentence_trees


Instruction:
Given the following Python 3 function, write the documentation
def add_child_to_self( self, tree ):
        ''' Adds given *tree* as a child of the current tree. '''
        assert isinstance(tree, Tree), \
               '(!) Unexpected type of argument for '+argName+'! Should be Tree.'
        if (not self.children):
            self.children = []
        tree.parent = self
        self.children.append(tree)

Response: Adds given tree as a child of the current tree.


Instruction:
Given the following Python 3 function, write the documentation
def add_child_to_subtree( self, parent_word_id, tree ):
        ''' Searches for the tree with *parent_word_id* from the current subtree
            (from this tree and from all of its subtrees). If the parent tree is
            found, attaches the given *tree* as its child. If the parent tree is
            not found, the current tree is not changed.
        '''
        if (self.word_id == parent_word_id):
            self.add_child_to_self( tree )
        elif (self.children):
            for child in self.children:
                child.add_child_to_subtree(parent_word_id, tree)

Response:
Searches for the tree with parent_word_id from the current subtree
            and adds it as a child of the given tree.


Instruction:
Make a summary of the following Python 3 code
def get_root( self, **kwargs ):
        ''' Returns this tree if it has no parents, or, alternatively, moves
            up via the parent links of this tree until reaching the tree with
            no parents, and returnes the parentless tree as the root.
        '''
        if self.parent == None:
            return self
        else:
            return self.parent.get_root( **kwargs )

Response: Returns this tree if it has no parents or returns the parentless tree if it has no parents.


Instruction:
Here you have a function in Python 3, explain what it does
def _satisfies_conditions( self, tree_node, **kwargs ):
        ''' Check whether given *tree_node* satisfies the conditions given
            as arguments in *kwargs*.

            By default (if no conditions are given in *kwargs*), returns
            True.

            If there are multiple conditions listed  (e.g.  'label_regexp'
            and 'word_template'),  *True*  is returned only when the node
            satisfies all the conditions.

            Following conditions are supported:
            -----------------------------------
            label : str
                Syntactic label (e.g. '@SUBJ', '@OBJ' etc.) that the node
                must have within its analysis; If the node does not have the
                label, the node will be discarded;

            label_regexp : str
                A regular expression pattern (as string) describing the
                syntactic label (e.g. '@SUBJ', '@OBJ' etc.) that the node
                must have within its analysis;
                If none of the node's labels matches the pattern, the node
                will be discarded;

            word_template : estnltk.mw_verbs.utils.WordTemplate
                A WordTemplate describing morphological constraints imposed
                to the word of the node;
                If the word's morphological features do not match the template,
                the node will be discarded;

        '''
        matches = []
        # A) Check syntactic label by matching a string
        syntactic_label = kwargs.get('label', None)
        if syntactic_label:
            matches.append( bool(tree_node.labels and syntactic_label in tree_node.labels) )

        # B) Check syntactic label by matching a regular expression
        synt_label_regexp = kwargs.get('label_regexp', None)
        if synt_label_regexp:
            if isinstance(synt_label_regexp, basestring):
                # Compile the regexp (if it hasn't been compiled yet)
                synt_label_regexp = re.compile(synt_label_regexp)
                kwargs['label_regexp'] = synt_label_regexp
            if isinstance(synt_label_regexp, RE_TYPE):
                # Apply the pre-compiled regexp
                if tree_node.labels:
                    matches.append( any([synt_label_regexp.match(label) != None for label in tree_node.labels]) )
                else:
                    matches.append( False )

        # C) Check whether the word token of the node matches a word template
        word_template = kwargs.get('word_template', None)
        if word_template:
            if isinstance(word_template, WordTemplate):
                matches.append( word_template.matches( tree_node.token ) )
            else:
                raise Exception('(!) Unexpected word_template. Should be from class WordTemplate.')
        return len(matches) == 0 or all(matches)

Response: Returns True if given tree_node satisfies the conditions given by kwargs.


Instruction:
Here you have a function in Python 3, explain what it does
def get_children( self, **kwargs ):
        ''' Recursively collects and returns all subtrees of given tree (if no
            arguments are given), or, alternatively, collects and returns subtrees
            satisfying some specific criteria (pre-specified in the arguments);

            Parameters
            -----------
            depth_limit : int
                Specifies how deep into the subtrees of this tree the search goes;
                Examples:
                 depth_limit=2 -- children of this node, and also children's
                                  direct children are considered as collectibles;
                 depth_limit=1 -- only children of this node are considered;
                 depth_limit=0 -- the end of search (only this node is considered);
                Default: unbounded ( the search is not limited by depth )

            include_self : bool
                Specifies whether this tree should also be included as a collectible
                subtree. If this tree is includes, it still must satisfy all the
                criteria before it is included in the collection;
                Default: False

            sorted : bool
                Specifies returned trees should be sorted in the ascending order of
                word_ids (basically: by the order of words in the text);
                If sorting is not applied, there is no guarantee that resulting trees
                follow the order of words in text;
                Default: False

            Following parameters can be used to set conditions for subtrees:
            -----------------------------------------------------------------
            label : str
                Syntactic label (e.g. '@SUBJ', '@OBJ' etc.) that the node
                must have within its analysis; If the node does not have the
                label, the node will be discarded;

            label_regexp : str
                A regular expression pattern (as string) describing the
                syntactic label (e.g. '@SUBJ', '@OBJ' etc.) that the node
                must have within its analysis;
                If none of the node's labels matches the pattern, the node
                will be discarded;

            word_template : estnltk.mw_verbs.utils.WordTemplate
                A WordTemplate describing morphological constraints imposed
                to the word of the node;
                If the word's morphological features do not match the template,
                the node will be discarded;

        '''
        depth_limit  = kwargs.get('depth_limit', 922337203685477580) # Just a nice big number to
                                                                     # assure that by default,
                                                                     # there is no depth limit ...
        include_self = kwargs.get('include_self', False)
        sorted_by_word_ids = kwargs.get('sorted', False)
        subtrees = []
        if include_self:
            if self._satisfies_conditions( self, **kwargs ):
                subtrees.append( self )
        if depth_limit >= 1 and self.children:
            # 1) Add children of given tree
            for child in self.children:
                if self._satisfies_conditions( child, **kwargs ):
                    subtrees.append(child)
            # 2) Collect children of given tree's children
            kwargs['include_self'] = False
            kwargs['depth_limit']  = depth_limit - 1
            for child in self.children:
                childs_results = child.get_children( **kwargs )
                if childs_results:
                    subtrees.extend(childs_results)
        if sorted_by_word_ids:
            # Sort by word_id-s, in ascending order
            subtrees = sorted(subtrees, key=lambda x: x.word_id)
        return subtrees

Response: Returns a list of all children of a node.


Instruction:
Can you generate the documentation for the following Python 3 function
def as_dependencygraph( self, keep_dummy_root=False, add_morph=True ):
        ''' Returns this tree as NLTK's DependencyGraph object.

            Note that this method constructs 'zero_based' graph,
            where counting of the words starts from 0 and the
            root index is -1 (not 0, as in Malt-TAB format);

            Parameters
            -----------
            add_morph : bool
                Specifies whether the morphological information
                (information about word lemmas, part-of-speech, and
                features) should be added to graph nodes.
                Note that even if **add_morph==True**, morphological
                information is only added if it is available via
                estnltk's layer  token['analysis'];
                Default: True
            keep_dummy_root : bool
                Specifies whether the graph should include a dummy
                TOP / ROOT node, which does not refer to any word,
                and yet is the topmost node of the tree.
                If the dummy root node is not used, then the root
                node is the word node headed by -1;
                Default: False

            For more information about NLTK's DependencyGraph, see:
             http://www.nltk.org/_modules/nltk/parse/dependencygraph.html
        '''
        from nltk.parse.dependencygraph import DependencyGraph
        graph = DependencyGraph( zero_based = True )
        all_tree_nodes = [self] + self.get_children()
        #
        # 0) Fix the root
        #
        if keep_dummy_root:
            #  Note: we have to re-construct  the root node manually,
            #  as DependencyGraph's current interface seems to provide
            #  no easy/convenient means for fixing the root node;
            graph.nodes[-1] = graph.nodes[0]
            graph.nodes[-1].update( { 'address': -1 } )
            graph.root = graph.nodes[-1]
        del graph.nodes[0]
        #
        # 1) Update / Add nodes of the graph
        #
        for child in all_tree_nodes:
            rel  = 'xxx' if not child.labels else '|'.join(child.labels)
            address = child.word_id
            word    = child.text
            graph.nodes[address].update(
            {
                'address': address,
                'word':  child.text,
                'rel':   rel,
            } )
            if not keep_dummy_root and child == self:
                # If we do not keep the dummy root node, set this tree
                # as the root node
                graph.root = graph.nodes[address]
            if add_morph and child.morph:
                # Add morphological information, if possible
                lemmas  = set([analysis[LEMMA] for analysis in child.morph])
                postags = set([analysis[POSTAG] for analysis in child.morph])
                feats   = set([analysis[FORM] for analysis in child.morph])
                lemma  = ('|'.join( list(lemmas)  )).replace(' ','_')
                postag = ('|'.join( list(postags) )).replace(' ','_')
                feats  = ('|'.join( list(feats) )).replace(' ','_')
                graph.nodes[address].update(
                {
                    'tag  ': postag,
                    'ctag' : postag,
                    'feats': feats,
                    'lemma': lemma
                } )

        #
        # 2) Update / Add arcs of the graph
        #
        for child in all_tree_nodes:
            #  Connect children of given word
            deps = [] if not child.children else [c.word_id for c in child.children]
            head_address = child.word_id
            for dep in deps:
                graph.add_arc( head_address, dep )
            if child.parent == None and keep_dummy_root:
                graph.add_arc( -1, head_address )
            #  Connect the parent of given node
            head = -1 if not child.parent else child.parent.word_id
            graph.nodes[head_address].update(
            {
                'head':  head,
            } )
        return graph

Response: Returns this tree as a DependencyGraph object.


Instruction:
Make a summary of the following Python 3 code
def get_tree_depth( self ):
        ''' Finds depth of this tree. '''
        if (self.children):
            depth = 1
            childDepths = []
            for child in self.children:
                childDepths.append( child.get_tree_depth() )
            return depth + max(childDepths)
        else:
            return 0

Response: Returns the depth of this tree.


Instruction:
Explain what the following Python 3 code does
def debug_print_tree( self, spacing='' ):
        ''' *Debug only* method for outputting the tree. '''
        print (spacing+" "+str(self.word_id)+" "+str(self.text))
        if (self.children):
            spacing=spacing+"  "
            for child in self.children:
                child.debug_print_tree(spacing)

Response: Debug only method for outputting the tree.


Instruction:
How would you explain what the following Python 3 function does
def addRule(self, field, regExpPattern):
        '''Adds new rule for checking whether a value of the field matches given regular
           expression regExpPattern;

           Parameters
           ----------
           field: str
                keyword, e.g. 'partofspeech', 'root', 'text' etc
           regExpPattern: str
                a regular expression that the value of the field must match (using method
                re.match( regExpPattern, token[field]) ).
        '''
        compiled = re.compile( regExpPattern )
        if field in self.analysisFields:
            if self.analysisRules == None:
                self.analysisRules = dict()
            self.analysisRules[field] = compiled
        else:
            if self.otherRules == None:
                self.otherRules = dict()
            self.otherRules[field] = compiled

Response: Adds a new rule for checking whether a value of the field matches given regular expression regExpPattern.


Instruction:
Write a Python 3 function for
determining whether the given token is in the WordTemplate.

Response:
def matches(self, tokenJson):
        '''Determines whether given token (tokenJson) satisfies all the rules listed
           in the WordTemplate. If the rules describe tokenJson[ANALYSIS], it is
           required that at least one item in the list tokenJson[ANALYSIS] satisfies
           all the rules (but it is not required that all the items should satisfy).
           Returns a boolean value.

           Parameters
           ----------
           tokenJson: pyvabamorf's analysis of a single word token;
        '''
        if self.otherRules != None:
            otherMatches = []
            for field in self.otherRules:
                match = field in tokenJson and ((self.otherRules[field]).match(tokenJson[field]) != None)
                otherMatches.append( match )
            if not otherMatches or not all(otherMatches):
                return False
            elif self.analysisRules == None and all(otherMatches):
                return True
        if self.analysisRules != None:
            assert ANALYSIS in tokenJson, "No ANALYSIS found within token: "+str(tokenJson)
            totalMatches = []
            for analysis in tokenJson[ANALYSIS]:
                # Check whether this analysis satisfies all the rules
                # (if not, discard the analysis)
                matches = []
                for field in self.analysisRules:
                    value = analysis[field] if field in analysis else ""
                    match = (self.analysisRules[field]).match(value) != None
                    matches.append( match )
                    if not match:
                        break
                totalMatches.append( all(matches) )
            #  Return True iff there was at least one analysis that
            # satisfied all the rules;
            return any(totalMatches)
        return False


Instruction:
Can you tell what is the following Python 3 function doing
def matchingAnalyseIndexes(self, tokenJson):
        '''Determines whether given token (tokenJson) satisfies all the rules listed
           in the WordTemplate and returns a list of analyse indexes that correspond
           to tokenJson[ANALYSIS] elements that are matching all the rules.
           An empty list is returned if none of the analyses match (all the rules),
           or (!) if none of the rules are describing the ANALYSIS part of the
           token;

           Parameters
           ----------
           tokenJson: pyvabamorf's analysis of a single word token;
        '''
        matchingResults = self.matchingAnalyses(tokenJson)
        if matchingResults:
            indexes = [ tokenJson[ANALYSIS].index(analysis) for analysis in matchingResults ]
            return indexes
        return matchingResults

Response: Determines whether given token is in the WordTemplate and returns a list of analyse indexes that correspond to the analysis of the given token. Returns an empty list if no analysis of the given token is found.