Instruction:
Write a Python 3 script to
return the score of the match between the desired language D and supported language S.

Response:
def tag_match_score(desired: {str, Language}, supported: {str, Language}) -> int:
    """
    Return a number from 0 to 100 indicating the strength of match between the
    language the user desires, D, and a supported language, S. Higher numbers
    are better. A reasonable cutoff for not messing with your users is to
    only accept scores of 75 or more.

    A score of 100 means the languages are the same, possibly after normalizing
    and filling in likely values.

    >>> tag_match_score('en', 'en')
    100
    >>> tag_match_score('en', 'en-US')
    100
    >>> tag_match_score('zh-Hant', 'zh-TW')
    100
    >>> tag_match_score('ru-Cyrl', 'ru')
    100
    >>> # Serbo-Croatian is a politically contentious idea, but in practice
    >>> # it's considered equivalent to Serbian in Latin characters.
    >>> tag_match_score('sh', 'sr-Latn')
    100

    A score of 92 to 97 indicates a regional difference.

    >>> tag_match_score('zh-HK', 'zh-MO')   # Chinese is similar in Hong Kong and Macao
    97
    >>> tag_match_score('en-AU', 'en-GB')   # Australian English is similar to British English
    96
    >>> tag_match_score('en-IN', 'en-GB')   # Indian English is also similar to British English
    96
    >>> tag_match_score('es-PR', 'es-419')  # Peruvian Spanish is Latin American Spanish
    96
    >>> tag_match_score('en-US', 'en-GB')   # American and British English are somewhat different
    94
    >>> tag_match_score('es-MX', 'es-ES')   # Mexican Spanish is different from Spanish Spanish
    92
    >>> # Serbian has two scripts, and people might prefer one but understand both
    >>> tag_match_score('sr-Latn', 'sr-Cyrl')
    95
    >>> # European Portuguese is different from the most common form (Brazilian Portuguese)
    >>> tag_match_score('pt', 'pt-PT')
    92

    A score of 86 to 90 indicates that people who use the desired language
    are demographically likely to understand the supported language, even if
    the languages themselves are unrelated. There are many languages that have
    a one-way connection of this kind to English or French.

    >>> tag_match_score('ta', 'en')  # Tamil to English
    86
    >>> tag_match_score('mg', 'fr')  # Malagasy to French
    86

    Sometimes it's more straightforward than that: people who use the desired
    language are demographically likely to understand the supported language
    because it's demographically relevant and highly related.

    >>> tag_match_score('af', 'nl')  # Afrikaans to Dutch
    86
    >>> tag_match_score('ms', 'id')  # Malay to Indonesian
    86
    >>> tag_match_score('nn', 'nb')  # Nynorsk to Norwegian Bokmål
    90
    >>> tag_match_score('nb', 'da')  # Norwegian Bokmål to Danish
    88

    A score of 80 to 85 indicates a particularly contentious difference in
    script, where people who understand one script can learn the other but
    probably won't be happy with it. This specifically applies to Chinese.

    >>> tag_match_score('zh-Hans', 'zh-Hant')
    85
    >>> tag_match_score('zh-CN', 'zh-HK')
    85
    >>> tag_match_score('zh-CN', 'zh-TW')
    85
    >>> tag_match_score('zh-Hant', 'zh-Hans')
    81
    >>> tag_match_score('zh-TW', 'zh-CN')
    81

    When the supported script is a different one than desired, this is usually
    a major difference with score of 60 or less.

    >>> tag_match_score('ja', 'ja-Latn-US-hepburn')
    56

    >>> # You can read the Shavian script, right?
    >>> tag_match_score('en', 'en-Shaw')
    56

    When there is no indication the supported language will be understood, the
    score will be 20 or less, to a minimum of 0.

    >>> tag_match_score('es', 'fr')   # Spanish and French are different.
    16
    >>> tag_match_score('en', 'ta')   # English speakers generally do not know Tamil.
    0

    CLDR doesn't take into account which languages are considered part of a
    common 'macrolanguage'. We have this data, so we can use it in matching.
    If two languages have no other rule that would allow them to match, but
    share a macrolanguage, they'll get a match score of 20 less than what
    they would get if the language matched.

    >>> tag_match_score('arz', 'ar')   # Egyptian Arabic to Standard Arabic
    80
    >>> tag_match_score('arz', 'ary')  # Egyptian Arabic to Moroccan Arabic
    76

    Here's an example that has script, region, and language differences, but
    a macrolanguage in common.

    Written Chinese is usually presumed to be Mandarin Chinese, but colloquial
    Cantonese can be written as well. When it is, it probably has region,
    script, and language differences from the usual mainland Chinese. But it is
    still part of the 'Chinese' macrolanguage, so there is more similarity
    than, say, comparing Mandarin to Hindi.

    >>> tag_match_score('yue', 'zh')
    36

    Comparing Swiss German ('gsw') to standardized German ('de') shows how
    these scores can be asymmetrical. Swiss German speakers will understand
    German, so the score in that direction is 92. Most German speakers find
    Swiss German unintelligible, and CLDR in fact assigns this a score of 16.

    This seems a little bit extreme, but the asymmetry is certainly there. And
    if your text is tagged as 'gsw', it must be that way for a reason.

    >>> tag_match_score('gsw', 'de')
    92
    >>> tag_match_score('de', 'gsw')
    16
    """
    desired_ld = Language.get(desired)
    supported_ld = Language.get(supported)
    return desired_ld.match_score(supported_ld)


Instruction:
Can you write a function in Python 3 where it
returns the best - matching language code for the given language.

Response:
def best_match(desired_language: {str, Language}, supported_languages: list,
               min_score: int=75) -> (str, int):
    """
    You have software that supports any of the `supported_languages`. You want
    to use `desired_language`. This function lets you choose the right language,
    even if there isn't an exact match.

    Returns:

    - The best-matching language code, which will be one of the
      `supported_languages` or 'und'
    - The score of the match, from 0 to 100

    `min_score` sets the minimum match score. If all languages match with a lower
    score than that, the result will be 'und' with a score of 0.

    When there is a tie for the best matching language, the first one in the
    tie will be used.

    Setting `min_score` lower will enable more things to match, at the cost
    of possibly mis-handling data or upsetting users. Read the documentation
    for :func:`tag_match_score` to understand what the numbers mean.

    >>> best_match('fr', ['de', 'en', 'fr'])
    ('fr', 100)
    >>> best_match('sh', ['hr', 'bs', 'sr-Latn', 'sr-Cyrl'])
    ('sr-Latn', 100)
    >>> best_match('zh-CN', ['zh-Hant', 'zh-Hans', 'gan', 'nan'])
    ('zh-Hans', 100)
    >>> best_match('zh-CN', ['cmn-Hant', 'cmn-Hans', 'gan', 'nan'])
    ('cmn-Hans', 100)
    >>> best_match('pt', ['pt-BR', 'pt-PT'])
    ('pt-BR', 100)
    >>> best_match('en-AU', ['en-GB', 'en-US'])
    ('en-GB', 96)
    >>> best_match('es-MX', ['es-ES', 'es-419', 'en-US'])
    ('es-419', 96)
    >>> best_match('es-MX', ['es-PU', 'es-AR', 'es-PY'])
    ('es-PU', 95)
    >>> best_match('es-MX', ['es-AR', 'es-PU', 'es-PY'])
    ('es-AR', 95)
    >>> best_match('zsm', ['id', 'mhp'])
    ('id', 86)
    >>> best_match('eu', ['el', 'en', 'es'])
    ('es', 90)
    >>> best_match('eu', ['el', 'en', 'es'], min_score=92)
    ('und', 0)
    """
    # Quickly return if the desired language is directly supported
    if desired_language in supported_languages:
        return desired_language, 100

    # Reduce the desired language to a standard form that could also match
    desired_language = standardize_tag(desired_language)
    if desired_language in supported_languages:
        return desired_language, 100

    match_scores = [
        (supported, tag_match_score(desired_language, supported))
        for supported in supported_languages
    ]
    match_scores = [
        (supported, score) for (supported, score) in match_scores
        if score >= min_score
    ] + [('und', 0)]

    match_scores.sort(key=lambda item: -item[1])
    return match_scores[0]


Instruction:
How would you explain what the following Python 3 function does
def make(cls, language=None, extlangs=None, script=None,
             region=None, variants=None, extensions=None, private=None):
        """
        Create a Language object by giving any subset of its attributes.

        If this value has been created before, return the existing value.
        """
        values = (language, tuple(extlangs or ()), script, region,
                  tuple(variants or ()), tuple(extensions or ()), private)
        if values in cls._INSTANCES:
            return cls._INSTANCES[values]

        instance = cls(
            language=language, extlangs=extlangs,
            script=script, region=region, variants=variants,
            extensions=extensions, private=private
        )
        cls._INSTANCES[values] = instance
        return instance

Response: Create a Language object by giving any subset of its attributes.


Instruction:
Can you create a Python 3 function that
returns a new Language object from a language tag string.

Response:
def get(tag: {str, 'Language'}, normalize=True) -> 'Language':
        """
        Create a Language object from a language tag string.

        If normalize=True, non-standard or overlong tags will be replaced as
        they're interpreted. This is recommended.

        Here are several examples of language codes, which are also test cases.
        Most language codes are straightforward, but these examples will get
        pretty obscure toward the end.

        >>> Language.get('en-US')
        Language.make(language='en', region='US')

        >>> Language.get('zh-Hant')
        Language.make(language='zh', script='Hant')

        >>> Language.get('und')
        Language.make()

        This function is idempotent, in case you already have a Language object:

        >>> Language.get(Language.get('en-us'))
        Language.make(language='en', region='US')

        The non-code 'root' is sometimes used to represent the lack of any
        language information, similar to 'und'.

        >>> Language.get('root')
        Language.make()

        By default, getting a Language object will automatically convert
        deprecated tags:

        >>> Language.get('iw')
        Language.make(language='he')

        >>> Language.get('in')
        Language.make(language='id')

        One type of deprecated tag that should be replaced is for sign
        languages, which used to all be coded as regional variants of a
        fictitious global sign language called 'sgn'. Of course, there is no
        global sign language, so sign languages now have their own language
        codes.

        >>> Language.get('sgn-US')
        Language.make(language='ase')

        >>> Language.get('sgn-US', normalize=False)
        Language.make(language='sgn', region='US')

        'en-gb-oed' is a tag that's grandfathered into the standard because it
        has been used to mean "spell-check this with Oxford English Dictionary
        spelling", but that tag has the wrong shape. We interpret this as the
        new standardized tag 'en-gb-oxendict', unless asked not to normalize.

        >>> Language.get('en-gb-oed')
        Language.make(language='en', region='GB', variants=['oxendict'])

        >>> Language.get('en-gb-oed', normalize=False)
        Language.make(language='en-gb-oed')

        'zh-min-nan' is another oddly-formed tag, used to represent the
        Southern Min language, which includes Taiwanese as a regional form. It
        now has its own language code.

        >>> Language.get('zh-min-nan')
        Language.make(language='nan')

        There's not much we can do with the vague tag 'zh-min':

        >>> Language.get('zh-min')
        Language.make(language='zh-min')

        Occasionally Wiktionary will use 'extlang' tags in strange ways, such
        as using the tag 'und-ibe' for some unspecified Iberian language.

        >>> Language.get('und-ibe')
        Language.make(extlangs=['ibe'])

        Here's an example of replacing multiple deprecated tags.

        The language tag 'sh' (Serbo-Croatian) ended up being politically
        problematic, and different standards took different steps to address
        this. The IANA made it into a macrolanguage that contains 'sr', 'hr',
        and 'bs'. Unicode further decided that it's a legacy tag that should
        be interpreted as 'sr-Latn', which the language matching rules say
        is mutually intelligible with all those languages.

        We complicate the example by adding on the region tag 'QU', an old
        provisional tag for the European Union, which is now standardized as
        'EU'.

        >>> Language.get('sh-QU')
        Language.make(language='sr', script='Latn', region='EU')
        """
        if isinstance(tag, Language):
            if not normalize:
                # shortcut: we have the tag already
                return tag

            # We might need to normalize this tag. Convert it back into a
            # string tag, to cover all the edge cases of normalization in a
            # way that we've already solved.
            tag = tag.to_tag()

        if (tag, normalize) in Language._PARSE_CACHE:
            return Language._PARSE_CACHE[tag, normalize]

        data = {}
        # if the complete tag appears as something to normalize, do the
        # normalization right away. Smash case when checking, because the
        # case normalization that comes from parse_tag() hasn't been applied
        # yet.
        tag_lower = tag.lower()
        if normalize and tag_lower in LANGUAGE_REPLACEMENTS:
            tag = LANGUAGE_REPLACEMENTS[tag_lower]

        components = parse_tag(tag)

        for typ, value in components:
            if typ == 'extlang' and normalize and 'language' in data:
                # smash extlangs when possible
                minitag = '%s-%s' % (data['language'], value)
                norm = LANGUAGE_REPLACEMENTS.get(minitag.lower())
                if norm is not None:
                    data.update(
                        Language.get(norm, normalize).to_dict()
                    )
                else:
                    data.setdefault('extlangs', []).append(value)
            elif typ in {'extlang', 'variant', 'extension'}:
                data.setdefault(typ + 's', []).append(value)
            elif typ == 'language':
                if value == 'und':
                    pass
                elif normalize:
                    replacement = LANGUAGE_REPLACEMENTS.get(value.lower())
                    if replacement is not None:
                        # parse the replacement if necessary -- this helps with
                        # Serbian and Moldovan
                        data.update(
                            Language.get(replacement, normalize).to_dict()
                        )
                    else:
                        data['language'] = value
                else:
                    data['language'] = value
            elif typ == 'region':
                if normalize:
                    data['region'] = REGION_REPLACEMENTS.get(value.lower(), value)
                else:
                    data['region'] = value
            elif typ == 'grandfathered':
                # If we got here, we got a grandfathered tag but we were asked
                # not to normalize it, or the CLDR data doesn't know how to
                # normalize it. The best we can do is set the entire tag as the
                # language.
                data['language'] = value
            else:
                data[typ] = value

        result = Language.make(**data)
        Language._PARSE_CACHE[tag, normalize] = result
        return result


Instruction:
Explain what the following Python 3 code does
def to_tag(self) -> str:
        if self._str_tag is not None:
            return self._str_tag
        subtags = ['und']
        if self.language:
            subtags[0] = self.language
        if self.extlangs:
            for extlang in sorted(self.extlangs):
                subtags.append(extlang)
        if self.script:
            subtags.append(self.script)
        if self.region:
            subtags.append(self.region)
        if self.variants:
            for variant in sorted(self.variants):
                subtags.append(variant)
        if self.extensions:
            for ext in self.extensions:
                subtags.append(ext)
        if self.private:
            subtags.append(self.private)
        self._str_tag = '-'.join(subtags)
        return self._str_tag

Response: Convert a Language object to a standard language tag as a string.


Instruction:
Implement a Python 3 function for
simplifying the script of the language.

Response:
def simplify_script(self) -> 'Language':
        """
        Remove the script from some parsed language data, if the script is
        redundant with the language.

        >>> Language.make(language='en', script='Latn').simplify_script()
        Language.make(language='en')

        >>> Language.make(language='yi', script='Latn').simplify_script()
        Language.make(language='yi', script='Latn')

        >>> Language.make(language='yi', script='Hebr').simplify_script()
        Language.make(language='yi')
        """
        if self._simplified is not None:
            return self._simplified

        if self.language and self.script:
            if DEFAULT_SCRIPTS.get(self.language) == self.script:
                result = self.update_dict({'script': None})
                self._simplified = result
                return self._simplified

        self._simplified = self
        return self._simplified


Instruction:
Implement a function in Python 3 to
assume the script for the current language.

Response:
def assume_script(self) -> 'Language':
        """
        Fill in the script if it's missing, and if it can be assumed from the
        language subtag. This is the opposite of `simplify_script`.

        >>> Language.make(language='en').assume_script()
        Language.make(language='en', script='Latn')

        >>> Language.make(language='yi').assume_script()
        Language.make(language='yi', script='Hebr')

        >>> Language.make(language='yi', script='Latn').assume_script()
        Language.make(language='yi', script='Latn')

        This fills in nothing when the script cannot be assumed -- such as when
        the language has multiple scripts, or it has no standard orthography:

        >>> Language.make(language='sr').assume_script()
        Language.make(language='sr')

        >>> Language.make(language='eee').assume_script()
        Language.make(language='eee')

        It also dosn't fill anything in when the language is unspecified.

        >>> Language.make(region='US').assume_script()
        Language.make(region='US')
        """
        if self._assumed is not None:
            return self._assumed
        if self.language and not self.script:
            try:
                self._assumed = self.update_dict({'script': DEFAULT_SCRIPTS[self.language]})
            except KeyError:
                self._assumed = self
        else:
            self._assumed = self
        return self._assumed


Instruction:
Write a Python 3 script to
return a new Language object with the same language as the current one.

Response:
def prefer_macrolanguage(self) -> 'Language':
        """
        BCP 47 doesn't specify what to do with macrolanguages and the languages
        they contain. The Unicode CLDR, on the other hand, says that when a
        macrolanguage has a dominant standardized language, the macrolanguage
        code should be used for that language. For example, Mandarin Chinese
        is 'zh', not 'cmn', according to Unicode, and Malay is 'ms', not 'zsm'.

        This isn't a rule you'd want to follow in all cases -- for example, you may
        want to be able to specifically say that 'ms' (the Malay macrolanguage)
        contains both 'zsm' (Standard Malay) and 'id' (Indonesian). But applying
        this rule helps when interoperating with the Unicode CLDR.

        So, applying `prefer_macrolanguage` to a Language object will
        return a new object, replacing the language with the macrolanguage if
        it is the dominant language within that macrolanguage. It will leave
        non-dominant languages that have macrolanguages alone.

        >>> Language.get('arb').prefer_macrolanguage()
        Language.make(language='ar')

        >>> Language.get('cmn-Hant').prefer_macrolanguage()
        Language.make(language='zh', script='Hant')

        >>> Language.get('yue-Hant').prefer_macrolanguage()
        Language.make(language='yue', script='Hant')
        """
        if self._macrolanguage is not None:
            return self._macrolanguage
        language = self.language or 'und'
        if language in NORMALIZED_MACROLANGUAGES:
            self._macrolanguage = self.update_dict({
                'language': NORMALIZED_MACROLANGUAGES[language]
            })
        else:
            self._macrolanguage = self
        return self._macrolanguage


Instruction:
Can you generate a brief explanation for the following Python 3 code
def broaden(self) -> 'List[Language]':
        if self._broader is not None:
            return self._broader
        self._broader = [self]
        seen = set(self.to_tag())
        for keyset in self.BROADER_KEYSETS:
            filtered = self._filter_attributes(keyset)
            tag = filtered.to_tag()
            if tag not in seen:
                self._broader.append(filtered)
                seen.add(tag)
        return self._broader

Response: Iterate through increasingly general versions of this parsed language tag and return a list of languages that can be used to find the broader version of that language tag.


Instruction:
Here you have a function in Python 3, explain what it does
def maximize(self) -> 'Language':
        if self._filled is not None:
            return self._filled

        for broader in self.broaden():
            tag = broader.to_tag()
            if tag in LIKELY_SUBTAGS:
                result = Language.get(LIKELY_SUBTAGS[tag], normalize=False)
                result = result.update(self)
                self._filled = result
                return result

        raise RuntimeError(
            "Couldn't fill in likely values. This represents a problem with "
            "the LIKELY_SUBTAGS data."
        )

Response: Returns a new language tag with the most likely values.


Instruction:
Here you have a function in Python 3, explain what it does
def match_score(self, supported: 'Language') -> int:
        if supported == self:
            return 100

        desired_complete = self.prefer_macrolanguage().maximize()
        supported_complete = supported.prefer_macrolanguage().maximize()

        desired_triple = (desired_complete.language, desired_complete.script, desired_complete.region)
        supported_triple = (supported_complete.language, supported_complete.script, supported_complete.region)

        return 100 - raw_distance(desired_triple, supported_triple)

Response: This method returns the score of matching the language with the given language.


Instruction:
Write a Python 3 function that can
get the name of the language in a natural language.

Response:
def language_name(self, language=DEFAULT_LANGUAGE, min_score: int=75) -> str:
        """
        Give the name of the language (not the entire tag, just the language part)
        in a natural language. The target language can be given as a string or
        another Language object.

        By default, things are named in English:

        >>> Language.get('fr').language_name()
        'French'
        >>> Language.get('el').language_name()
        'Greek'

        But you can ask for language names in numerous other languages:

        >>> Language.get('fr').language_name('fr')
        'français'
        >>> Language.get('el').language_name('fr')
        'grec'

        Why does everyone get Slovak and Slovenian confused? Let's ask them.

        >>> Language.get('sl').language_name('sl')
        'slovenščina'
        >>> Language.get('sk').language_name('sk')
        'slovenčina'
        >>> Language.get('sl').language_name('sk')
        'slovinčina'
        >>> Language.get('sk').language_name('sl')
        'slovaščina'
        """
        return self._get_name('language', language, min_score)


Instruction:
Create a Python 3 function to
return the name of the autonym language.

Response:
def autonym(self, min_score: int=95) -> str:
        """
        Give the name of this language *in* this language.

        >>> Language.get('fr').autonym()
        'français'
        >>> Language.get('es').autonym()
        'español'
        >>> Language.get('ja').autonym()
        '日本語'

        This doesn't give the name of the region or script, but in some cases
        the language can name itself in multiple scripts:

        >>> Language.get('sr-Latn').autonym()
        'srpski'
        >>> Language.get('sr-Cyrl').autonym()
        'српски'
        >>> Language.get('pa').autonym()
        'ਪੰਜਾਬੀ'
        >>> Language.get('pa-Arab').autonym()
        'پنجابی'

        This only works for language codes that CLDR has locale data for. You
        can't ask for the autonym of 'ja-Latn' and get 'nihongo'.
        """
        return self.language_name(language=self, min_score=min_score)


Instruction:
How would you explain what the following Python 3 function does
def script_name(self, language=DEFAULT_LANGUAGE, min_score: int=75) -> str:
        return self._get_name('script', language, min_score)

Response: Get the script part of the language tag in a natural language.


Instruction:
How would you code a function in Python 3 to
get the name of the region part of the language tag.

Response:
def region_name(self, language=DEFAULT_LANGUAGE, min_score: int=75) -> str:
        """
        Describe the region part of the language tag in a natural language.
        """
        return self._get_name('region', language, min_score)


Instruction:
Write a Python 3 script to
describe each of the variant parts of the language tag in a natural taxonomy.

Response:
def variant_names(self, language=DEFAULT_LANGUAGE, min_score: int=75) -> list:
        """
        Describe each of the variant parts of the language tag in a natural
        language.
        """
        names = []
        for variant in self.variants:
            var_names = code_to_names('variant', variant)
            names.append(self._best_name(var_names, language, min_score))
        return names


Instruction:
Can you implement a function in Python 3 that
returns a dictionary that describes a given language tag in a specified natural language.

Response:
def describe(self, language=DEFAULT_LANGUAGE, min_score: int=75) -> dict:
        """
        Return a dictionary that describes a given language tag in a specified
        natural language.

        See `language_name` and related methods for more specific versions of this.

        The desired `language` will in fact be matched against the available
        options using the matching technique that this module provides.  We can
        illustrate many aspects of this by asking for a description of Shavian
        script (a script devised by author George Bernard Shaw), and where you
        might find it, in various languages.

        >>> from pprint import pprint
        >>> shaw = Language.make(script='Shaw').maximize()
        >>> pprint(shaw.describe('en'))
        {'language': 'English', 'region': 'United Kingdom', 'script': 'Shavian'}

        >>> pprint(shaw.describe('fr'))
        {'language': 'anglais', 'region': 'Royaume-Uni', 'script': 'shavien'}

        >>> pprint(shaw.describe('es'))
        {'language': 'inglés', 'region': 'Reino Unido', 'script': 'shaviano'}

        >>> pprint(shaw.describe('pt'))
        {'language': 'inglês', 'region': 'Reino Unido', 'script': 'shaviano'}

        >>> pprint(shaw.describe('uk'))
        {'language': 'англійська', 'region': 'Велика Британія', 'script': 'шоу'}

        >>> pprint(shaw.describe('arb'))
        {'language': 'الإنجليزية', 'region': 'المملكة المتحدة', 'script': 'الشواني'}

        >>> pprint(shaw.describe('th'))
        {'language': 'อังกฤษ', 'region': 'สหราชอาณาจักร', 'script': 'ซอเวียน'}

        >>> pprint(shaw.describe('zh-Hans'))
        {'language': '英语', 'region': '英国', 'script': '萧伯纳式文'}

        >>> pprint(shaw.describe('zh-Hant'))
        {'language': '英文', 'region': '英國', 'script': '簫柏納字符'}

        >>> pprint(shaw.describe('ja'))
        {'language': '英語', 'region': 'イギリス', 'script': 'ショー文字'}

        When we don't have a localization for the language, we fall back on
        'und', which just shows the language codes.

        >>> pprint(shaw.describe('lol'))
        {'language': 'en', 'region': 'GB', 'script': 'Shaw'}

        Wait, is that a real language?

        >>> pprint(Language.get('lol').maximize().describe())
        {'language': 'Mongo', 'region': 'Congo - Kinshasa', 'script': 'Latin'}
        """
        names = {}
        if self.language:
            names['language'] = self.language_name(language, min_score)
        if self.script:
            names['script'] = self.script_name(language, min_score)
        if self.region:
            names['region'] = self.region_name(language, min_score)
        if self.variants:
            names['variants'] = self.variant_names(language, min_score)
        return names


Instruction:
Write a Python 3 script for
finding the subtag of a particular tagtype that has the given name.

Response:
def find_name(tagtype: str, name: str, language: {str, 'Language', None}=None):
        """
        Find the subtag of a particular `tagtype` that has the given `name`.

        The default language, "und", will allow matching names in any language,
        so you can get the code 'fr' by looking up "French", "Français", or
        "francés".

        Occasionally, names are ambiguous in a way that can be resolved by
        specifying what name the language is supposed to be in. For example,
        there is a language named 'Malayo' in English, but it's different from
        the language named 'Malayo' in Spanish (which is Malay). Specifying the
        language will look up the name in a trie that is only in that language.

        In a previous version, we thought we were going to deprecate the
        `language` parameter, as there weren't significant cases of conflicts
        in names of things between languages. Well, we got more data, and
        conflicts in names are everywhere.

        Specifying the language that the name should be in is still not
        required, but it will help to make sure that names can be
        round-tripped.

        >>> Language.find_name('language', 'francés')
        Language.make(language='fr')

        >>> Language.find_name('region', 'United Kingdom')
        Language.make(region='GB')

        >>> Language.find_name('script', 'Arabic')
        Language.make(script='Arab')

        >>> Language.find_name('language', 'norsk bokmål')
        Language.make(language='nb')

        >>> Language.find_name('language', 'norsk')
        Language.make(language='no')

        >>> Language.find_name('language', 'norsk', 'en')
        Traceback (most recent call last):
            ...
        LookupError: Can't find any language named 'norsk'

        >>> Language.find_name('language', 'norsk', 'no')
        Language.make(language='no')

        >>> Language.find_name('language', 'malayo', 'en')
        Language.make(language='mbp')

        >>> Language.find_name('language', 'malayo', 'es')
        Language.make(language='ms')

        Some langauge names resolve to more than a language. For example,
        the name 'Brazilian Portuguese' resolves to a language and a region,
        and 'Simplified Chinese' resolves to a language and a script. In these
        cases, a Language object with multiple subtags will be returned.

        >>> Language.find_name('language', 'Brazilian Portuguese', 'en')
        Language.make(language='pt', region='BR')

        >>> Language.find_name('language', 'Simplified Chinese', 'en')
        Language.make(language='zh', script='Hans')

        A small amount of fuzzy matching is supported: if the name can be
        shortened to match a single language name, you get that language.
        This allows, for example, "Hakka dialect" to match "Hakka".

        >>> Language.find_name('language', 'Hakka dialect')
        Language.make(language='hak')
        """

        # No matter what form of language we got, normalize it to a single
        # language subtag
        if isinstance(language, Language):
            language = language.language
        elif isinstance(language, str):
            language = get(language).language
        if language is None:
            language = 'und'

        code = name_to_code(tagtype, name, language)
        if code is None:
            raise LookupError("Can't find any %s named %r" % (tagtype, name))
        if '-' in code:
            return Language.get(code)
        else:
            data = {tagtype: code}
            return Language.make(**data)


Instruction:
Given the following Python 3 function, write the documentation
def to_dict(self):
        if self._dict is not None:
            return self._dict

        result = {}
        for key in self.ATTRIBUTES:
            value = getattr(self, key)
            if value:
                result[key] = value
        self._dict = result
        return result

Response: Returns a dictionary of the attributes of this Language object which can be useful for constructing a similar object.


Instruction:
Here you have a function in Python 3, explain what it does
def update(self, other: 'Language') -> 'Language':
        return Language.make(
            language=other.language or self.language,
            extlangs=other.extlangs or self.extlangs,
            script=other.script or self.script,
            region=other.region or self.region,
            variants=other.variants or self.variants,
            extensions=other.extensions or self.extensions,
            private=other.private or self.private
        )

Response: Update this Language with the fields of the other Language.


Instruction:
Implement a Python 3 function for
updating the attributes of this Language from a dictionary.

Response:
def update_dict(self, newdata: dict) -> 'Language':
        """
        Update the attributes of this Language from a dictionary.
        """
        return Language.make(
            language=newdata.get('language', self.language),
            extlangs=newdata.get('extlangs', self.extlangs),
            script=newdata.get('script', self.script),
            region=newdata.get('region', self.region),
            variants=newdata.get('variants', self.variants),
            extensions=newdata.get('extensions', self.extensions),
            private=newdata.get('private', self.private)
        )


Instruction:
Can you create a Python 3 function that
selects a subset of keys from a dictionary.

Response:
def _filter_keys(d: dict, keys: set) -> dict:
        """
        Select a subset of keys from a dictionary.
        """
        return {key: d[key] for key in keys if key in d}


Instruction:
Given the following Python 3 function, write the documentation
def _filter_attributes(self, keyset):
        filtered = self._filter_keys(self.to_dict(), keyset)
        return Language.make(**filtered)

Response: Returns a copy of this object with a subset of its attributes set.


Instruction:
How would you explain what the following Python 3 function does
def _searchable_form(self) -> 'Language':
        if self._searchable is not None:
            return self._searchable

        self._searchable = self._filter_attributes(
            {'language', 'script', 'region'}
        ).simplify_script().prefer_macrolanguage()
        return self._searchable

Response: Convert a parsed language tag so that the information it contains is in the most likely form.


Instruction:
Given the following Python 3 function, write the documentation
def resolve_name(key, vals, debug=False):
    max_priority = max([val[2] for val in vals])
    val_count = Counter([val[1] for val in vals if val[2] == max_priority])
    if len(val_count) == 1:
        unanimous = val_count.most_common(1)
        return unanimous[0][0]

    for pkey in val_count:
        if pkey in AMBIGUOUS_PREFERENCES:
            others = set(val_count)
            others.remove(pkey)
            if others == others & AMBIGUOUS_PREFERENCES[pkey]:
                if debug:
                    print("Resolved: {} -> {}".format(key, pkey))
                return pkey

    # In debug mode, show which languages vote for which name
    if debug and max_priority >= 0:
        votes = defaultdict(list)
        for voter, val, prio in vals:
            if prio == max_priority:
                votes[val].append(voter)

        print("{}:".format(key))
        for val, voters in sorted(votes.items()):
            print("\t{}: {}".format(val, ' '.join(voters)))

    # Don't use names that remain ambiguous
    return None

Response: Given a name and a number of possible values find the name that can be resolved to.


Instruction:
Make a summary of the following Python 3 code
def read_cldr_names(path, language, category):
    filename = data_filename('{}/{}/{}.json'.format(path, language, category))
    fulldata = json.load(open(filename, encoding='utf-8'))
    data = fulldata['main'][language]['localeDisplayNames'][category]
    return data

Response: Read CLDR s names for things in a particular language and category.


Instruction:
Can you write a function in Python 3 where it
parses the IANA subtag registry file and yield a dictionary of information for each subtag it describes.

Response:
def parse_file(file):
    """
    Take an open file containing the IANA subtag registry, and yield a
    dictionary of information for each subtag it describes.
    """
    lines = []
    for line in file:
        line = line.rstrip('\n')
        if line == '%%':
            # This is a separator between items. Parse the data we've
            # collected and yield the result.
            yield from parse_item(lines)
            lines.clear()
        elif line.startswith('  '):
            # This is a continuation line. Concatenate it to the previous
            # line, including one of the spaces.
            lines[-1] += line[1:]
        else:
            lines.append(line)
    yield from parse_item(lines)


Instruction:
Can you tell what is the following Python 3 function doing
def parse_item(lines):
    info = {}
    for line in lines:
        key, value = line.split(': ', 1)
        if key in LIST_KEYS:
            info.setdefault(key, []).append(value)
        else:
            assert key not in info
            info[key] = value

    if 'Subtag' in info or 'Tag' in info:
        yield info

Response: Given the lines that form a subtag entry parse the data they contain.


Instruction:
Explain what the following Python 3 code does
def _make_simple_distances():
    distances = {}
    def sym(desired, supported, strength):
        "Define a symmetric distance between languages."
        desired_t = tuple(desired.split('-'))
        supported_t = tuple(supported.split('-'))
        distances[desired_t, supported_t] = strength
        distances[supported_t, desired_t] = strength

    def one(desired, supported, strength):
        "Define a one-way distance between languages."
        desired_t = tuple(desired.split('-'))
        supported_t = tuple(supported.split('-'))
        distances[desired_t, supported_t] = strength

    def ok(desired, supported):
        "Define the most common type of link: a one-way distance of 10."
        one(desired, supported, 10)

    sym('no', 'nb', 1)
    sym('hr', 'bs', 4)
    sym('sh', 'bs', 4)
    sym('sr', 'bs', 4)
    sym('sh', 'hr', 4)
    sym('sr', 'hr', 4)
    sym('sh', 'sr', 4)
    sym('ssy', 'aa', 4)
    one('gsw', 'de', 4)
    one('lb', 'de', 4)
    sym('da', 'no', 8)
    sym('da', 'nb', 8)
    ok('ab', 'ru')
    ok('ach', 'en')
    ok('af', 'nl')
    ok('ak', 'en')
    ok('ay', 'es')
    ok('az', 'ru')
    ok('az-Latn', 'ru-Cyrl')
    ok('be', 'ru')
    ok('bem', 'en')
    ok('bh', 'hi')
    ok('bn', 'en')
    ok('bn-Beng', 'en-Latn')
    ok('br', 'fr')
    ok('ceb', 'fil')
    ok('chr', 'en')
    ok('ckb', 'ar')
    ok('co', 'fr')
    ok('crs', 'fr')
    ok('cy', 'en')
    ok('ee', 'en')
    ok('eo', 'en')
    ok('et', 'fi')
    ok('eu', 'es')
    ok('fo', 'da')
    ok('fy', 'nl')
    ok('ga', 'en')
    ok('gaa', 'en')
    ok('gd', 'en')
    ok('gl', 'es')
    ok('gn', 'es')
    ok('gu', 'hi')
    ok('ha', 'en')
    ok('haw', 'en')
    ok('ht', 'fr')
    ok('hy', 'ru')
    ok('hy-Armn', 'ru-Cyrl')
    ok('ia', 'en')
    ok('ig', 'en')
    ok('is', 'en')
    ok('jv', 'id')
    ok('ka-Geor', 'en-Latn')
    ok('ka', 'en')
    ok('kg', 'fr')
    ok('kk', 'ru')
    ok('km', 'en')
    ok('km-Khmr', 'en-Latn')
    ok('kn', 'en')
    ok('kn-Knda', 'en-Latn')
    ok('kri', 'en')
    ok('ku', 'tr')
    ok('ky', 'ru')
    ok('la', 'it')
    ok('lg', 'en')
    ok('ln', 'fr')
    ok('lo', 'en')
    ok('lo-Laoo', 'en-Latn')
    ok('loz', 'en')
    ok('lua', 'fr')
    ok('mfe', 'en')
    ok('mg', 'fr')
    ok('mi', 'en')
    ok('mk', 'bg')
    ok('ml', 'en')
    ok('ml-Mlym', 'en-Latn')
    ok('mn', 'ru')
    ok('mr', 'hi')
    ok('ms', 'id')
    ok('mt', 'en')
    ok('my', 'en')
    ok('my-Mymr', 'en-Latn')
    ok('ne', 'en')
    ok('ne-Deva', 'en-Latn')
    sym('nn', 'nb', 10)
    ok('nn', 'no')
    ok('nso', 'en')
    ok('ny', 'en')
    ok('nyn', 'en')
    ok('oc', 'fr')
    ok('om', 'en')
    ok('or', 'en')
    ok('or-Orya', 'en-Latn')
    ok('pa', 'en')
    ok('pa-Guru', 'en-Latn')
    ok('pcm', 'en')
    ok('ps', 'en')
    ok('ps-Arab', 'en-Latn')
    ok('qu', 'es')
    ok('rm', 'de')
    ok('rn', 'en')
    ok('rw', 'fr')
    ok('sa', 'hi')
    ok('sd', 'en')
    ok('sd-Arab', 'en-Latn')
    ok('si', 'en')
    ok('si-Sinh', 'en-Latn')
    ok('sn', 'en')
    ok('so', 'en')
    ok('sq', 'en')
    ok('st', 'en')
    ok('su', 'id')
    ok('sw', 'en')
    ok('ta', 'en')
    ok('ta-Taml', 'en-Latn')
    ok('te', 'en')
    ok('te-Telu', 'en-Latn')
    ok('tg', 'ru')
    ok('ti', 'en')
    ok('ti-Ethi', 'en-Latn')
    ok('tk', 'ru')
    ok('tk-Latn', 'ru-Cyrl')
    ok('tlh', 'en')
    ok('tn', 'en')
    ok('to', 'en')
    ok('tt', 'ru')
    ok('tum', 'en')
    ok('ug', 'zh')
    ok('ur', 'en')
    ok('ur-Arab', 'en-Latn')
    ok('uz', 'ru')
    ok('uz-Latn', 'ru-Cyrl')
    ok('wo', 'fr')
    ok('xh', 'en')
    ok('yi', 'en')
    ok('yi-Hebr', 'en-Latn')
    ok('yo', 'en')
    ok('zu', 'en')
    sym('sr-Latn', 'sr-Cyrl', 5)
    one('zh-Hans', 'zh-Hant', 15)
    one('zh-Hant', 'zh-Hans', 19)
    sym('zh-Hant-HK', 'zh-Hant-MO', 3)

    return distances

Response: This function creates a simple structure of the base class for the base class.


Instruction:
Given the following Python 3 function, write the documentation
def normalize_name(name):
    name = name.casefold()
    name = name.replace("’", "'")
    name = name.replace("-", " ")
    name = name.replace("(", "")
    name = name.replace(")", "")
    name = name.replace(",", "")
    return name.strip()

Response: Normalizes a language - code component name.


Instruction:
Can you generate the documentation for the following Python 3 function
def load_trie(filename):
    trie = marisa_trie.BytesTrie()
    # marisa_trie raises warnings that make no sense. Ignore them.
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        trie.load(filename)
    return trie

Response: Load a BytesTrie from the marisa_trie on - disk format.


Instruction:
How would you implement a function in Python 3 that
gets a language script or region from a name in some language.

Response:
def name_to_code(category, name, language: str='und'):
    """
    Get a language, script, or region by its name in some language.

    The language here must be a string representing a language subtag only.
    The `Language.find` method can handle other representations of a language
    and normalize them to this form.

    The default language, "und", will allow matching names in any language,
    so you can get the code 'fr' by looking up "French", "Français", or
    "francés".

    A small amount of fuzzy matching is supported: if the name can be
    shortened or lengthened to match a single language name, you get that
    language. This allows, for example, "Hakka Chinese" to match "Hakka".

    Occasionally, names are ambiguous in a way that can be resolved by
    specifying what name the language is supposed to be in. For example,
    there is a language named 'Malayo' in English, but it's different from
    the language named 'Malayo' in Spanish (which is Malay). Specifying the
    language will look up the name in a trie that is only in that language.
    """
    assert '/' not in language, "Language codes cannot contain slashes"
    assert '-' not in language, "This code should be reduced to a language subtag only"
    trie_name = '{}/name_to_{}'.format(language, category)
    if trie_name not in TRIES:
        TRIES[trie_name] = load_trie(data_filename('trie/{}.marisa'.format(trie_name)))

    trie = TRIES[trie_name]
    lookup = normalize_name(name)
    if lookup in trie:
        return get_trie_value(trie, lookup)
    else:
        # Is this a language plus extra junk? Maybe it has "...isch", "... language",
        # or "... Chinese" attached to it, for example.
        prefixes = trie.prefixes(lookup)
        if prefixes and len(prefixes[-1]) >= 4:
            return get_trie_value(trie, prefixes[-1])
        else:
            return None


Instruction:
Create a Python 3 function to
give a code for a language script or region get a dictionary of its names in various languages.

Response:
def code_to_names(category, code):
    """
    Given the code for a language, script, or region, get a dictionary of its
    names in various languages.
    """
    trie_name = '{}_to_name'.format(category)
    if trie_name not in TRIES:
        TRIES[trie_name] = load_trie(data_filename('trie/{}.marisa'.format(trie_name)))

    trie = TRIES[trie_name]
    lookup = code.lower() + '@'
    possible_keys = trie.keys(lookup)
    names = {}
    for key in possible_keys:
        target_language = key.split('@')[1]
        names[target_language] = get_trie_value(trie, key)
    return names


Instruction:
Can you tell what is the following Python 3 function doing
def unicode2encode(text, charmap):
    '''
    charmap : dictionary which has both encode as key, unicode as value
    '''
    if isinstance(text, (list, tuple)):
        unitxt = ''
        for line in text:
            for val,key in charmap.items():
                if key in line:
                    line = line.replace(key, val)
                # end of if val in text:
            unitxt += line
        # end of for line in text:
        return unitxt
    elif isinstance(text, str):
        for val,key in charmap.items():
            if key in text:
                text = text.replace(key, val)
        return text

Response: converts unicode to unicode


Instruction:
How would you explain what the following Python 3 function does
def unicode2auto(unicode_text, encode_text):

    _all_unique_encodes_, _all_common_encodes_ = _get_unique_common_encodes()
    # get unique word which falls under any one of available encodes from
    # user passed text lines
    unique_chars = _get_unique_ch(encode_text, _all_common_encodes_)
    # count common encode chars
    clen = len(_all_common_encodes_)
    msg = "Sorry, couldn't find encode :-(\n"
    msg += 'Need more words to find unique encode out side of %d ' % clen
    msg += 'common compound characters'
    if not unique_chars:
        print(msg)
        return ''
    # end of if not unique_chars:

    for encode_name, encode_keys in _all_unique_encodes_:
        if not len(encode_keys): continue
        for ch in encode_keys:
            # check either encode char is presnent in word
            if ch in unique_chars:
                # found encode
                print(("Found encode : ", encode_name))
                encode = _all_encodes_[encode_name]
                return unicode2encode(unicode_text, encode)
            # end of if ch in unique_chars:
        # end of ifor ch in encode_keys:
    else:
        print(msg)
        return ''

Response: This function will convert unicode text into other word components by auto - finds the encode of the word and returns the new word.


Instruction:
Explain what the following Python 3 code does
def print_huffman_code_cwl(code,p,v):
    cwl = 0.0
    for k,_v in code.items():
        print(u"%s -> %s"%(k,_v))
        cwl += p[v.index(k)]*len(_v)
    print(u"cwl = %g"%cwl)
    return cwl,code.values()

Response: print the code map for huffman code p v is probability map


Instruction:
Implement a Python 3 function for
implementing Daegmar - Levenshtein edit distance algorithm

Response:
def edit_distance(wordA,wordB):
    """" Implements Daegmar-Levenshtein edit distance algorithm:
         Ref: https://en.wikipedia.org/wiki/Edit_distance
         Ref: https://en.wikipedia.org/wiki/Levenshtein_distance"""
    if not type(wordA) is list:
        lettersA = tamil.utf8.get_letters(wordA)
    else:
        lettersA = wordA

    if not type(wordB) is list:
        lettersB = tamil.utf8.get_letters(wordB)
    else:
        lettersB = wordB
    n_A = len(lettersA)
    n_B = len(lettersB)
    dist_AB = [[0 for i in range(0,n_B+1)] for i in range(0,(n_A+1))]
    # Target prefix reached by insertion
    for j in range(1,n_B+1):
        dist_AB[0][j] = j
    for i in range(1,n_A+1):
        dist_AB[i][0] = i
    for j in range(1,n_B+1):
        for i in range(1,n_A+1):
            if (lettersA[i-1] == lettersB[j-1]):
                new_dist = dist_AB[i-1][j-1]
            else:
                new_dist = min( [dist_AB[i-1][j]+1, dist_AB[i][j-1]+1, dist_AB[i-1][j-1]+1] ) #del, ins, or sub
            dist_AB[i][j] = new_dist
    return dist_AB[-1][-1]


Instruction:
How would you code a function in Python 3 to
calculate edit - distance between two words in the language tree.

Response:
def Dice_coeff(wordA,wordB):
    """
        # Calculate edit-distance - Implements the Dice coefficent
        # Ref: https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient
        # distance should be between 0 - 1.0. can be used as a similarity match
    """
    if not type(wordA) is list:
        lettersA = tamil.utf8.get_letters(wordA)
    else:
        lettersA = wordA

    if not type(wordB) is list:
        lettersB = tamil.utf8.get_letters(wordB)
    else:
        lettersB = wordB
    n_A = len(lettersA)
    n_B = len(lettersB)
    # OK only if unique - set(lettersA).intersection(set(lettersB))
    n_AB = len( list( filter( lambda cmnL: cmnL in lettersB, lettersA) ) )
    return (2.0*n_AB)/(n_A+n_B)


Instruction:
Here you have a function in Python 3, explain what it does
def generate_splits(self):
        L = len(self.letters)-1
        for idx,letter in enumerate(self.letters):
            if not( letter in tamil.utf8.grantha_uyirmei_letters):
                continue
            muthal = idx == 0 and u"" or u"".join(self.letters[0:idx])
            meethi = idx == L and u"" or u"".join(self.letters[idx+1:])
            mei,uyir = tamil.utf8.splitMeiUyir(letter)
            muthal = muthal + mei
            meethi = uyir + meethi
            self.results.append([muthal,meethi])
        return len(self.results) > 0

Response: Generate the splits of the muthal mei and uyir.


Instruction:
Create a Python 3 function for
generating a list of patterns for the given word.

Response:
def oridam_generate_patterns(word_in,cm,ed=1,level=0,pos=0,candidates=None):
    """ ed = 1 by default, pos - internal variable for algorithm """
    alternates = cm.get(word_in[pos],[])
    if not candidates:
        candidates = []
    assert ed <= len(word_in), 'edit distance has to be comparable to word size [ins/del not explored]'
    if (pos >len(word_in)) or ed == 0:
        return candidates
    pfx = ''
    sfx = ''
    curr_candidates = []
    for p in range(0,pos):
        pfx = pfx + word_in[p]
    for p in range(pos+1,len(word_in)):
        sfx = sfx + word_in[p]
    for alt in alternates:
        word_alt = pfx + alt + sfx
        if not (word_alt in candidates):
            candidates.append( word_alt )
            curr_candidates.append( word_alt )
    for n_pos in range(pos,len(word_in)):
        # already what we have ' candidates ' of this round are edit-distance 1
        for word in curr_candidates:
            oridam_generate_patterns(word,cm,ed-1,level+1,n_pos,candidates)
    if level == 0:
        #candidates.append(word_in)
        for n_pos in range(pos,len(word_in)):
            oridam_generate_patterns(word_in,cm,ed, level+1,n_pos,candidates)
    return candidates


Instruction:
Given the following Python 3 function, write the documentation
def to_unicode_repr( _letter ):
    # Python 2-3 compatible
    return u"u'"+ u"".join( [ u"\\u%04x"%ord(l) for l in _letter ] ) + u"'"

Response: Convert a unicode character to a unicode string.


Instruction:
Can you tell what is the following Python 3 function doing
def uyirmei_constructed( mei_idx, uyir_idx):
    idx,idy = mei_idx,uyir_idx
    assert ( idy >= 0 and idy < uyir_len() )
    assert ( idx >= 0 and idx < 6+mei_len() )
    return grantha_agaram_letters[mei_idx]+accent_symbols[uyir_idx]

Response: construct uyirmei letter give mei index and uyir index


Instruction:
Implement a function in Python 3 to
return True if the string contains any English letters in the string.

Response:
def has_english( word_in ):
    """ return True if word_in has any English letters in the string"""
    return not all_tamil(word_in) and len(word_in) > 0 and any([l in word_in for l in string.ascii_letters])


Instruction:
Explain what the following Python 3 code does
def all_tamil( word_in ):
    if isinstance(word_in,list):
        word = word_in
    else:
        word = get_letters( word_in )
    return all( [(letter in tamil_letters) for letter in word] )

Response: predicate checks if all letters of the input word are Tamil letters


Instruction:
Can you generate the documentation for the following Python 3 function
def reverse_word( word ):
    op = get_letters( word )
    op.reverse()
    return u"".join(op)

Response: reverse a Tamil word according to letters not unicode - points


Instruction:
Implement a Python 3 function for
returning a list of all the letters present in the word

Response:
def get_letters( word ):
    """ splits the word into a character-list of tamil/english
    characters present in the stream """
    ta_letters = list()
    not_empty = False
    WLEN,idx = len(word),0
    while (idx < WLEN):
        c = word[idx]
        #print(idx,hex(ord(c)),len(ta_letters))
        if c in uyir_letter_set or c == ayudha_letter:
            ta_letters.append(c)
            not_empty = True
        elif c in grantha_agaram_set:
            ta_letters.append(c)
            not_empty = True
        elif c in accent_symbol_set:
            if not not_empty:
                # odd situation
                ta_letters.append(c)
                not_empty = True
            else:
                #print("Merge/accent")
                ta_letters[-1] += c
        else:
            if ord(c) < 256 or not (is_tamil_unicode(c)):
                ta_letters.append( c )
            else:
                if not_empty:
                    #print("Merge/??")
                    ta_letters[-1]+= c
                else:
                    ta_letters.append(c)
                    not_empty = True
        idx = idx + 1
    return ta_letters


Instruction:
Write a Python 3 script for
returning an iterable of all letters in the word

Response:
def get_letters_iterable( word ):
    """ splits the word into a character-list of tamil/english
    characters present in the stream """
    WLEN,idx = len(word),0

    while (idx < WLEN):
        c = word[idx]
        #print(idx,hex(ord(c)),len(ta_letters))
        if c in uyir_letter_set or c == ayudha_letter:
            idx = idx + 1
            yield c
        elif c in grantha_agaram_set:
            if idx + 1 < WLEN and word[idx+1] in all_symbol_set:
                c2 = word[idx+1]
                idx = idx + 2
                yield (c + c2)
            else:
                idx = idx + 1
                yield c
        else:
            idx = idx + 1
            yield c
    return


Instruction:
Write a Python 3 script to
yield a list of words from a list of UTF - 8 letters

Response:
def get_words_iterable( letters, tamil_only=False ):
    """ given a list of UTF-8 letters section them into words, grouping them at spaces """

    # correct algorithm for get-tamil-words
    buf = []
    for idx,let in enumerate(letters):
        if not let.isspace():
            if istamil(let) or (not tamil_only):
                buf.append( let )
        else:
            if len(buf) > 0:
                yield  u"".join( buf )
                buf = []
    if len(buf) > 0:
        yield u"".join(buf)


Instruction:
Write a Python 3 function for
reversing a Tamil word according to letters not unicode - points

Response:
def get_tamil_words( letters ):
    """ reverse a Tamil word according to letters, not unicode-points """
    if not isinstance(letters,list):
        raise Exception("metehod needs to be used with list generated from 'tamil.utf8.get_letters(...)'")
    return [word for word in get_words_iterable( letters, tamil_only = True )]


Instruction:
Write a Python 3 function for
comparing words in Tamil lexicographic order

Response:
def compare_words_lexicographic( word_a, word_b ):
    """ compare words in Tamil lexicographic order """
    # sanity check for words to be all Tamil
    if ( not all_tamil(word_a) ) or (not all_tamil(word_b)) :
        #print("## ")
        #print(word_a)
        #print(word_b)
        #print("Both operands need to be Tamil words")
        pass
    La = len(word_a)
    Lb = len(word_b)
    all_TA_letters = u"".join(tamil_letters)
    for itr in range(0,min(La,Lb)):
            pos1 = all_TA_letters.find( word_a[itr] )
            pos2 = all_TA_letters.find( word_b[itr] )

            if pos1 != pos2 :
                    #print  not( pos1 > pos2), pos1, pos2
                    return cmp(pos1, pos2)

    # result depends on if La is shorter than Lb, or 0 if La == Lb  i.e. cmp
    return cmp(La,Lb)


Instruction:
How would you explain what the following Python 3 function does
def word_intersection( word_a, word_b ):
    positions = []
    word_a_letters = get_letters( word_a )
    word_b_letters = get_letters( word_b )
    for idx,wa in enumerate(word_a_letters):
        for idy,wb in enumerate(word_b_letters):
            if ( wa == wb ):
                positions.append( (idx, idy) )
    return positions

Response: return a list of tuples where word_a and word_b intersect


Instruction:
How would you explain what the following Python 3 function does
def joinMeiUyir(mei_char, uyir_char):
    if not mei_char: return uyir_char
    if not uyir_char: return mei_char

    if not isinstance(mei_char, PYTHON3 and str or unicode):
        raise ValueError(u"Passed input mei character '%s' must be unicode, not just string" % mei_char)
    if not isinstance(uyir_char, PYTHON3 and str or unicode) and uyir_char != None:
        raise ValueError(u"Passed input uyir character '%s' must be unicode, not just string" % uyir_char)
    if mei_char not in grantha_mei_letters:
        raise ValueError(u"Passed input character '%s' is not a tamil mei character" % mei_char)
    if uyir_char not in uyir_letters:
        raise ValueError(u"Passed input character '%s' is not a tamil uyir character" % uyir_char)
    if uyir_char:
        uyiridx = uyir_letters.index(uyir_char)
    else:
        return mei_char
    meiidx = grantha_mei_letters.index(mei_char)
    # calculate uyirmei index
    uyirmeiidx = meiidx*12 + uyiridx
    return grantha_uyirmei_letters[uyirmeiidx]

Response: This function joins mei character and uyir character and returns as


Instruction:
Can you create a Python 3 function that
expands uyir or mei - letter range etc.

Response:
def expand_tamil(start,end):
    """ expand uyir or mei-letter range etc.
        i.e. அ-ஔ  gets converted to அ,ஆ,இ,ஈ,உ,ஊ,எ,ஏ,ஐ,ஒ,ஓ,ஔ etc.
    """
    # few sequences
    for seq in [utf8.uyir_letters, utf8.grantha_mei_letters, \
                utf8.grantha_agaram_letters]:
        if is_containing_seq(start,end,seq):
            return expand_sequence(start,end,seq)
    # all Tamil letters
    seq = utf8.grantha_uyirmei_letters
    if is_containing_seq(start,end,seq):
        return expand_sequence(start,end,seq)

    raise Exception("Cannot understand sequence [%s-%s]"%(start,end))


Instruction:
Can you tell what is the following Python 3 function doing
def make_pattern( patt, flags=0 ):
    # print('input',len(patt))
    patt_letters = utf8.get_letters( patt )
    patt_out = list()
    idx = 0
    # print('output',len(patt_letters))
    patt = [None,None]
    prev = None
    LEN_PATT = len(patt_letters)
    while( idx < LEN_PATT ):
        if utf8.istamil( patt_letters[idx] ) and ( prev == '-' or ((idx+1) < LEN_PATT and patt_letters[idx+1] == u'-') ):
            if (idx+1) < LEN_PATT and patt_letters[idx+1] == u'-':
                patt[0] = patt_letters[idx]
                idx = idx + 2
                prev = "-"
            elif prev == '-':
                patt[1] = patt_letters[idx]
                patt_out.extend( expand_tamil( patt[0], patt[1]) )
                idx = idx + 1
                prev = patt_letters[idx]
            continue
        patt_out.extend( patt_letters[idx] )
        prev = patt_letters[idx]
        idx = idx + 1
    opattern = u"".join( patt_out )
    compile_regexp = re.compile( opattern, flags )
    return (compile_regexp,opattern)

Response: returns a compile regular expression object


Instruction:
Make a summary of the following Python 3 code
def frequency_model( self ):
        prev_letter = None
        # use a generator in corpus
        prev_letter = list(self.corpus.next_tamil_letter())[0]
        for next_letter in self.corpus.next_tamil_letter():
            # update frequency from corpus
            key = prev_letter+next_letter
            val = self.bigram.get(key,None)
            prev_letter = next_letter
            if not val:
                self.bigram[key] = 0
            self.bigram[key] += 1

        return

Response: build a letter frequency model for Tamil letters from a corpus


Instruction:
Create a Python 3 function for
converting a byte - ASCII encoded string into equivalent Unicode string

Response:
def convert_to_unicode( tscii_input ):
    """ convert a byte-ASCII encoded string into equivalent Unicode string
    in the UTF-8 notation."""
    output = list()
    prev = None
    prev2x = None
    # need a look ahead of 2 tokens atleast
    for char in tscii_input:
        ## print "%2x"%ord(char) # debugging
        if ord(char) < 128 :
            # base-ASCII copy to output
            output.append( char )
            prev = None
            prev2x = None

        elif ord(char) in ISCII_DIRECT_LOOKUP:
            if ( prev in ISCII_PRE_MODIFIER ):
                curr_char = [ISCII[ord(char)],ISCII[prev]]
            else:
                # we are direct lookup char
                curr_char = [ISCII[ord(char)]]
                char = None

            output.extend( curr_char )

        elif ( (ord(char) in ISCII_POST_MODIFIER) ):

            if ( (prev in ISCII_DIRECT_LOOKUP) and
                 (prev2x in ISCII_PRE_MODIFIER) ):
                if len(output) >= 2:
                    del output[-1] #we are reducing this token to something new
                    del output[-2]
                elif len(output)==1:
                    del output[-1]
                else:
                    # nothing to delete here..
                    pass
                output.extend( [ISCII[prev], ISCII[prev2x]] )
            else:
                print("Warning: malformed ISCII encoded file; skipping characters")

            prev = None
            char = None
        else:
            # pass - must be one of the pre/post modifiers
            pass

        prev2x = prev
        if char:
            prev = ord(char)
    return u"".join(output)


Instruction:
Write a Python 3 script for
computing the intersection graph into the dictionary of words and their intersections

Response:
def compute( self ):
        # compute the intersection graph into @xsections dictionary
        wordlist = self.wordlist
        """ build a dictionary of words, and their intersections """
        xsections = {}
        for i in range(len(wordlist)):
            word_i = wordlist[i]
            for j in range(len(wordlist)):
                word_j = wordlist[j]
                if i == j:
                    # force self-intersection to be 0
                    if not xsections.get(word_i,None):
                        xsections[word_i] = ['']
                    else:
                        xsections[word_i].extend([''])
                    continue
                # optimize for, i > j, info is calculated already
                if i > j:
                    xsec_counts = xsections[word_j][i]
                else:
                    xsec_counts = tamil.utf8.word_intersection( word_i, word_j )
                if not xsections.get(word_i,None):
                    xsections[word_i] =  [xsec_counts]
                else:
                    xsections[word_i].extend( [ xsec_counts ] )
        self.xsections = xsections


Instruction:
Implement a function in Python 3 to
check if word is in sequence

Response:
def in_sequence( word, ref_set, ref_reason, freq_threshold = 2 ):
        """ ignore ctx information right now. If repetition/match length >= @freq_threshold then we flag-it """
        chars = get_letters(word)
        flag = True #no error assumed
        reason = None #no reason
        freq_count = 0
        for char in chars:
            if char in ref_set:
                freq_count += 1
                if freq_count >= freq_threshold:
                    flag = False
                    break
                continue
            freq_count = 0 # continue loop
        if not flag:
            reason = ref_reason
        return flag,reason


Instruction:
How would you explain what the following Python 3 function does
def apply(self, word, ctx=None):
        return Sequential.in_sequence(word,AdjacentVowels.uyir_letters,AdjacentVowels.reason)

Response: apply the rule to word


Instruction:
Implement a Python 3 function for
applying the rule to word

Response:
def apply(self, word, ctx=None):
        """ ignore ctx information right now """
        flag,reason = Sequential.in_sequence(word,AdjacentConsonants.mei_letters,AdjacentConsonants.reason,self.freq_threshold)
        if flag:
            flag,reason = Sequential.in_sequence(word,AdjacentConsonants.agaram_letters,AdjacentConsonants.reason,self.freq_threshold)
        return flag,reason


Instruction:
Can you generate the documentation for the following Python 3 function
def apply(self,word,ctx=None):
        chars = get_letters(word)
        flag = True #no error assumed
        reason = None #no reason
        prev_letter = None
        for char in chars:
            if prev_letter == char:
                flag = False
                break
            prev_letter = char # continue loop
        if not flag:
            reason = RepeatedLetters.reason
        return flag,reason

Response: check if the word is in the list of letters and return the flag and reason


Instruction:
Can you generate a brief explanation for the following Python 3 code
def apply(self, word, ctx=None):
        chars = get_letters(word)
        flag = True #no error assumed
        reason = None #no reason
        prev_char = None

        for char in chars:
            rule1,rule2,rule3 = False,False,False
            # rule 1 : uyir followed by kombugal
            rule1 = (char[-1] in utf8.accent_symbols) and (char[0] in utf8.uyir_letters)
            if not rule1:
                # rule 2 : two pullis adjacent to each other
                rule2 = len(char) >= 2 and (char[-1] == utf8.pulli_symbols[0]) and (char[-2] == char[-1] )
                if not rule2:
                    # rule 3 : none of the accent symbols repeat
                    # exclusions to rule 3 : non-standard Unicode encoding of periya kombu / siriya kombu with thunai kaal
                    rule3 =  len(char) >= 2 and (char[-1] in utf8.accent_symbols) and (char[-2] in utf8.accent_symbols) \
                    and not( char[-1] == u"ா" and char[-2] in [u"ெ",u"ே"])

            if rule1 or rule2 or rule3:
                flag = False
                reason = BadIME.reason
                break
            prev_char = char # continue loop
        #print([flag,reason])
        return flag,reason

Response: check if the word is in the list of possible characters


Instruction:
Implement a function in Python 3 to
set the current string.

Response:
def set_current(self, value):
        '''
        Set the self.current string.
        '''
        self.current = value
        self.cursor = 0
        self.limit = len(self.current)
        self.limit_backward = 0
        self.bra = self.cursor
        self.ket = self.limit


Instruction:
Given the following Python 3 function, write the documentation
def replace_s(self, c_bra, c_ket, s):
        '''
        to replace chars between c_bra and c_ket in self.current by the
        chars in s.

        @type c_bra int
        @type c_ket int
        @type s: string
        '''
        adjustment = len(s) - (c_ket - c_bra)
        self.current = self.current[0:c_bra] + s + self.current[c_ket:]
        self.limit += adjustment
        if self.cursor >= c_ket:
            self.cursor += adjustment
        elif self.cursor > c_bra:
            self.cursor = c_bra
        return adjustment

Response:
replace_s is used to replace the current string by the characters between c_bra and c_ket in self. current by the the
            chars in s.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def slice_to(self, s):
        '''
        Copy the slice into the supplied StringBuffer

        @type s: string
        '''
        result = ''
        if self.slice_check():
            result = self.current[self.bra:self.ket]
        return result

Response: Copy the slice into the supplied StringBuffer


Instruction:
Can you tell what is the following Python 3 function doing
def isTamilPredicate(word):
        for c in word:
            if unicodedata.name(c).split()[0] != u'TAMIL' :
                return False
        return True

Response: is Tamil word : boolean True/False


Instruction:
Explain what the following Python 3 code does
def cleanupPunct( tweet ):
        tweet = ''.join( map( lambda c: (unicodedata.name(c).split()[0] in [u'TAMIL',u'LATIN']) and c or u' ', tweet) )
        return tweet

Response: Remove punctuation from a tweet.


Instruction:
Write a Python 3 function that can
word needs to all be in the same tamil language

Response:
def getTamilWords( tweet ):
        """" word needs to all be in the same tamil language """
        tweet = TamilTweetParser.cleanupPunct( tweet );
        nonETwords = filter( lambda x: len(x) > 0 , re.split(r'\s+',tweet) );#|"+|\'+|#+
        tamilWords = filter( TamilTweetParser.isTamilPredicate, nonETwords );
        return tamilWords


Instruction:
Write a Python 3 script for
converting a byte - ASCII encoded string into equivalent Unicode string

Response:
def convert_to_unicode( tscii_input ):
    """ convert a byte-ASCII encoded string into equivalent Unicode string
    in the UTF-8 notation."""
    output = list()
    prev = None
    prev2x = None
    # need a look ahead of 2 tokens atleast
    for char in tscii_input:
        ## print "%2x"%ord(char) # debugging
        if ord(char) < 128 :
            # base-ASCII copy to output
            output.append( char )
            prev = None
            prev2x = None

        elif ord(char) in TSCII_DIRECT_LOOKUP:
            if ( prev in TSCII_PRE_MODIFIER ):
                curr_char = [TSCII[ord(char)],TSCII[prev]]
            else:
                # we are direct lookup char
                curr_char = [TSCII[ord(char)]]
                char = None

            output.extend( curr_char )

        elif ( (ord(char) in TSCII_POST_MODIFIER) ):

            if ( (prev in TSCII_DIRECT_LOOKUP) and
                 (prev2x in TSCII_PRE_MODIFIER) ):
                if len(output) >= 2:
                    del output[-1] #we are reducing this token to something new
                    del output[-2]
                elif len(output)==1:
                    del output[-1]
                else:
                    # nothing to delete here..
                    pass
                output.extend( [TSCII[prev], TSCII[prev2x]] )
            else:
                print("Warning: malformed TSCII encoded file; skipping characters")

            prev = None
            char = None
        else:
            # pass - must be one of the pre/post modifiers
            pass

        prev2x = prev
        if char:
            prev = ord(char)
    return u"".join(output)


Instruction:
Write a Python 3 script to
return the first unique character in the input text.

Response:
def _get_unique_ch(text, all_common_encodes):
    """
        text : encode sample strings

        returns unique word / characters from input text encode strings.
    """

    unique_chars = ''
    if isinstance(text, str):
        text = text.split("\n")
    elif isinstance(text, (list, tuple)):
        pass

    special_chars = ['.', ',', ';', ':','', ' ', '\r', '\t', '=', '\n']
    for line in text:
        for word in line.split(' '):
            if ( not PYTHON3 ): word = word.decode( 'utf-8')
            for ch in all_common_encodes:
                if ch in word: word = word.replace(ch, '')
            # end of for ch in _all_common_encodes_:
            # if len of word is zero, then go for another word
            if not word: continue

            for ch in word:
                if ch.isdigit() or ch in special_chars:
                    # remove special common chars
                    word = word.replace(ch, '')
                    continue
                # end of if ch.isdigit() or ...:
                # Whola, got unique chars from user passed text
                return word
            # end of for ch in word:
        # end of for word in line.split(' '):
    # end of for line in text:
    return ''


Instruction:
Given the following Python 3 function, write the documentation
def _get_unique_common_encodes():

    _all_unique_encodes_ = []
    _all_unicode_encodes_ = {}
    _all_common_encodes_ = set([])
    _all_common_encodes_single_char_ = set([])

    for name, encode in _all_encodes_.items():
        encode_utf8 = set([PYTHON3 and ch or ch.decode( 'utf-8') for ch in encode.keys()])
        _all_unicode_encodes_[name] = encode_utf8
    _all_unique_encodes_full_ =_all_unicode_encodes_.copy()

    for supname, super_encode in _all_unicode_encodes_.items():
        for subname, sub_encode in _all_unicode_encodes_.items():
            if supname == subname: continue
            # get unique of super_encode among other encodings
            super_encode = super_encode - sub_encode
        # get common for all over encodings
        common = _all_unique_encodes_full_[supname] - super_encode
        # merge common to all encodings common
        _all_common_encodes_ = _all_common_encodes_.union(common)
        # store super_encode's unique keys with its name
        _all_unique_encodes_.append((supname, super_encode))
    for ch in _all_common_encodes_:
        # collect single common chars
        if len(ch) == 1: _all_common_encodes_single_char_.add(ch)
    # end of for ch in _all_common_encodes_:

    # remove single common char from compound common chars
    _all_common_encodes_ -= _all_common_encodes_single_char_

    if __WRITE_CHARS_TXT:
        # write common compound characters of all encodes
        f = open('all.encodes.common.chars.txt', 'w')
        for ch in _all_common_encodes_:
            ch = ch.encode('utf-8')
            for encode_keys in _all_encodes_.values():
                if ch in encode_keys:
                    uni = encode_keys[ch]
                    break
                # end of if ch in encode_keys:
            # end of for encode_keys in _all_encodes_.values():
            f.write(ch + '  =>  ' + uni + '\n')
        # end of for ch in _all_common_encodes_:
        f.close()
        # write unique compound characters of all encodes
        for encode_name, encode_keys in _all_unique_encodes_:
            f = open(encode_name + '.unique.chars.txt', 'w')
            for ch in encode_keys:
                ch = ch.encode('utf-8')
                uni = _all_encodes_[encode_name][ch]
                f.write(ch + '  =>  ' + uni + '\n')
            # end of for ch in encode_keys:
            f.close()
        # end of for encode_name, encode_keys in _all_unique_encodes_:
    # end of if __WRITE_CHARS_TXT:

    return (_all_unique_encodes_, _all_common_encodes_)

Response: Returns a tuple of unique and common encodings.


Instruction:
Can you generate the documentation for the following Python 3 function
def auto2unicode(text):

    _all_unique_encodes_, _all_common_encodes_ = _get_unique_common_encodes()
    # get unique word which falls under any one of available encodes from
    # user passed text lines
    unique_chars = _get_unique_ch(text, _all_common_encodes_)
    # count common encode chars
    clen = len(_all_common_encodes_)
    msg = "Sorry, couldn't find encode :-(\n"
    msg += 'Need more words to find unique encode out side of %d ' % clen
    msg += 'common compound characters'
    if not unique_chars:
        print(msg)
        return ''
    # end of if not unique_chars:

    for encode_name, encode_keys in _all_unique_encodes_:
        if not len(encode_keys): continue
        for ch in encode_keys:
            # check either encode char is presnent in word
            if ch in unique_chars:
                # found encode
                print(("Found encode : ", encode_name))
                encode = _all_encodes_[encode_name]
                return encode2unicode(text, encode)
            # end of if ch in unique_chars:
        # end of ifor ch in encode_keys:
    else:
        print(msg)
        return ''

Response: This function tries to identify encode in available encodings and converts text into unicode string.


Instruction:
How would you code a function in Python 3 to
build a letter frequency model for Tamil letters from a corpus

Response:
def frequency_model( self ):
        """ build a letter frequency model for Tamil letters from a corpus """
        # use a generator in corpus
        for next_letter in self.corpus.next_tamil_letter():
            # update frequency from corpus
            self.letter[next_letter] = self.letter[next_letter] + 1


Instruction:
Implement a function in Python 3 to
build a Tamil bigram letter model

Response:
def language_model(self,verbose=True):
        """ builds a Tamil bigram letter model """
        # use a generator in corpus
        prev = None
        for next_letter in self.corpus.next_tamil_letter():
            # update frequency from corpus
            if prev:
                self.letter2[prev][next_letter] += 1
                if ( verbose ) :
                    print(prev)
                    print(next_letter)
                    print( self.letter2[prev][next_letter] )
            prev = next_letter #update always
        return


Instruction:
Given the following Python 3 function, write the documentation
def language_model(self,verbose=True):
        # use a generator in corpus
        p2 = None
        p1 = None
        for next_letter in self.corpus.next_tamil_letter():
            # update frequency from corpus
            if p2:
                trig = p2+p1+next_letter
                self.letter3[trig] = 1 + self.letter3.get(trig,0)
            p2 = p1
            p1 = next_letter #update always
        return

Response: builds a Tamil bigram letter model


Instruction:
Create a Python 3 function for
converting a number into a tamil string

Response:
def num2tamilstr( *args ):
    """ work till one lakh crore - i.e 1e5*1e7 = 1e12.
        turn number into a numeral, Indian style. Fractions upto 1e-30"""
    number = args[0]
    if len(args) < 2:
        filenames = []
    else:
        filenames = args[1]
    if len(args) ==3:
        tensSpecial = args[2]
    else:
        tensSpecial='BASIC'
    if not any( filter( lambda T: isinstance( number, T), [str,unicode,int, long, float]) ) or isinstance(number,complex):
        raise Exception('num2tamilstr input has to be a long or integer or float')
    if float(number) > long(1e12):
        raise Exception('num2tamilstr input is too large')
    if float(number) < 0:
        return u"- "+num2tamilstr( -number )

    units = (u'பூஜ்ஜியம்', u'ஒன்று', u'இரண்டு', u'மூன்று', u'நான்கு', u'ஐந்து', u'ஆறு', u'ஏழு', u'எட்டு', u'ஒன்பது', u'பத்து') # 0-10
    units_suffix = (u'பூஜ்ஜியம்', u'தொன்று', u'திரண்டு', u'மூன்று', u'நான்கு', u'தைந்து', u'தாறு', u'தேழு', u'தெட்டு', u'தொன்பது', u'பத்து') # 0-10
    units_suffix_nine = (u'பூஜ்ஜியம்', u'றொன்று', u'றிரண்டு', u'மூன்று', u'நான்கு', u'றைந்து', u'றாறு', u'றேழு', u'றெட்டு', u'றொன்பது', u'பத்து') # 0-10
    tween = [1.0,2.0,5.0,6.0,7.0,8.0,9.0]
    teens = (u'பதினொன்று', u'பனிரண்டு', u'பதிமூன்று', u'பதினான்கு', u'பதினைந்து',u'பதினாறு', u'பதினேழு', u'பதினெட்டு', u'பத்தொன்பது') # 11-19
    tens = (u'பத்து', u'இருபது', u'முப்பது', u'நாற்பது', u'ஐம்பது',u'அறுபது', u'எழுபது', u'எண்பது', u'தொன்னூறு') # 10-90
    tens_full_prefix = (u'இருபத்து', u'முப்பத்து', u'நாற்பத்து', u'ஐம்பத்து', u'அறுபத்து', u'எழுபத்து', u'எண்பத்து', u'தொன்னூற்று') # 10+-90+
    tens_prefix = (u'இருபத்', u'முப்பத்', u'நாற்பத்', u'ஐம்பத்', u'அறுபத்', u'எழுபத்', u'எண்பத்', u'தொன்னூற்') # 10+-90+
    hundreds = ( u'நூறு', u'இருநூறு', u'முன்னூறு', u'நாநூறு',u'ஐநூறு', u'அறுநூறு', u'எழுநூறு', u'எண்ணூறு', u'தொள்ளாயிரம்') #100 - 900
    hundreds_suffix = (u'நூற்றி', u'இருநூற்று', u'முன்னூற்று', u'நாநூற்று', u'ஐநூற்று', u'அறுநூற்று', u'எழுநூற்று', u'எண்ணூற்று',u'தொள்ளாயிரத்து') #100+ - 900+

    one_thousand_prefix = u'ஓர்'
    thousands = (u'ஆயிரம்',u'ஆயிரத்து')

    one_prefix = u'ஒரு'
    lakh = (u'இலட்சம்',u'இலட்சத்து')
    crore = (u'கோடி',u'கோடியே')

    pulli = u'புள்ளி'
    n_one = 1.0
    n_ten = 10.0
    n_hundred = 100.0
    n_thousand = 1000.0
    n_lakh = 100.0*n_thousand
    n_crore = (100.0*n_lakh)
    # handle fractional parts
    if float(number) > 0.0 and float(number) < 1.0:
        rval = []
        rval.append(pulli)
        filenames.append( 'pulli' )
        number_str = str(number).replace('0.','')
        for digit in number_str:
            filenames.append( "units_%d"%int(digit))
            rval.append( units[int(digit)] )
        return u' '.join(rval)

    if isinstance(number,str) or isinstance(number,unicode):
        result = u""
        number = number.strip()
        assert(len(args) == 1)
        assert(len(number) > 0)
        is_negative = number[0] == "-"
        if  is_negative:
            number = number[1:]
        frac_part = u""
        if number.find(".") >= 0:
            rat_part,frac_part = number.split(".")
            frac_part = num2tamilstr(u"0."+frac_part)
        else:
            rat_part = number
        if len(rat_part) > 0:
            result = num2tamilstr(float(rat_part))
        result = result +u" "+ frac_part
        return is_negative and "-" + result.strip() or result.strip()

    suffix_base = { n_crore: crore, n_lakh : lakh, n_thousand : thousands}
    suffix_file_map = {n_crore: "crore", n_lakh : "lakh", n_thousand : "thousands"}

    file_map = {n_crore :["one_prefix","crore_0"],
                n_lakh : ["one_prefix","lakh_0"],
                n_thousand :  ["one_thousand_prefix", "thousands_0"],
               n_hundred : ["hundreds_0"], #special
               n_ten : ["units_10"],
               n_one : ["units_1"]}

    num_map = {n_crore : [one_prefix,crore[0]],
               n_lakh  : [one_prefix,lakh[0]],
               n_thousand : [one_thousand_prefix, thousands[0]],
               n_hundred : [hundreds[0]], #special
               n_ten : [units[10]],
               n_one : [units[1]]}

    all_bases = [n_crore, n_lakh, n_thousand, n_hundred, n_ten,n_one]
    allowed_bases = list(filter( lambda base: number >= base, all_bases ))
    if len(allowed_bases) >= 1:
        n_base = allowed_bases[0]
        if number == n_base:
            if tensSpecial=='BASIC':
                filenames.extend(file_map[n_base])
                return u" ".join(num_map[n_base])
            elif tensSpecial=='NINES':
                filenames.extend(file_map[n_base])
                return units_suffix_nine[long(number%10)]
            else:
                filenames.extend(file_map[n_base])
                return units_suffix[long(number%10)]
        quotient_number = long( number/n_base )
        residue_number = number - n_base*quotient_number
        #print number, n_base, quotient_number, residue_number, tensSpecial
        if n_base == n_one:
            if isinstance(number,float):
                int_part = long(number%10)
                frac = number - float(int_part)
                filenames.append("units_%d"%int_part)
                if abs(frac) > 1e-30:
                    if tensSpecial=='BASIC':
                        return units[int_part]+u' ' + num2tamilstr(frac,filenames)
                    elif tensSpecial=='NINES':
                        return units_suffix_nine[int_part]+u' ' + num2tamilstr(frac,filenames)
                    else:
                        return units_suffix[int_part]+u' ' + num2tamilstr(frac,filenames)

                else:
                    if tensSpecial=='BASIC':
                        return units[int_part]
                    elif tensSpecial=='NINES':
                        return units_suffix_nine[int_part]
                    else:
                        return units_suffix[int_part]
            else:
                if tensSpecial=='BASIC':
                    filenames.append("units_%d"%number)
                    return units[number]
                elif tensSpecial=='NINES':
                    filenames.append("units_%d"%number)
                    return units_suffix_nine[number]
                else:
                    filenames.append("units_%d"%number)
                    return units_suffix[number]

        elif n_base == n_ten:
            if residue_number < 1.0:
                filenames.append("tens_%d"%(quotient_number-1))
                if residue_number == 0.0:
                    return tens[quotient_number-1]
                #else: //seems not reachable.
                #    numeral = tens[quotient_number-1]
            elif number < 20:
                filenames.append("teens_%d"%(number-10))
                residue_number = math.fmod(number,1)
                teen_number = int(math.floor(number - 10))
                if residue_number > 1e-30:
                    return teens[teen_number-1] +u' ' + num2tamilstr(residue_number,filenames)
                else:
                    return teens[teen_number-1]+u' '
            if residue_number < 1.0:
                filenames.append( "tens_%d"%(quotient_number-1) )
                numeral = tens[quotient_number-1]+u' '
            else:
                if residue_number in tween:
                    filenames.append( "tens_prefix_%d"%(quotient_number-2) )
                    numeral = tens_prefix[quotient_number-2]
                    tensSpecial='SPECIAL'
                    if (quotient_number==9):
                        tensSpecial = 'NINES'
                else:
                    filenames.append( "tens_prefix_%d"%(quotient_number-2) )
                    numeral = tens_full_prefix[quotient_number-2]+u' '
        elif n_base == n_hundred:
            if residue_number == 0:
                filenames.append("hundreds_%d"%(quotient_number-1))
                return hundreds[quotient_number-1]+u' '
            if residue_number < 1.0:
                filenames.append( "hundreds_%d"%(quotient_number-1) )
                numeral = hundreds[quotient_number-1]+u' '
            else:
                filenames.append("hundreds_suffix_%d"%(quotient_number-1))
                numeral = hundreds_suffix[quotient_number-1]+u' '
        else:
            if ( quotient_number == 1 ):
                if n_base == n_thousand:
                    filenames.append("one_thousand_prefix")
                    numeral = one_thousand_prefix
                else:
                    filenames.append("one_prefix")
                    numeral = one_prefix
            else:
                numeral = num2tamilstr( quotient_number, filenames )
        if n_base >= n_thousand:
            suffix = suffix_base[n_base][long(residue_number >= 1)]
            suffix_filename = "%s_%d"%(suffix_file_map[n_base],long(residue_number >= 1))
            if residue_number == 0:
                filenames.append(suffix_filename)
                return numeral + u' ' + suffix+u' '
            filenames.append(suffix_filename)
            numeral = numeral + u' ' + suffix+u' '
        residue_numeral = num2tamilstr( residue_number, filenames, tensSpecial)
        #return numeral+u' '+residue_numeral
        return numeral+residue_numeral
    # number has to be zero
    filenames.append("units_0")
    return units[0]


Instruction:
Can you tell what is the following Python 3 function doing
def num2tamilstr_american( *args ):
    number = args[0]
    """ work till 1000 trillion  - 1 - i.e  = 1e12*1e3 - 1.
        turn number into a numeral, American style. Fractions upto 1e-30. """

    if not any( filter( lambda T: isinstance( number, T), [int, str, unicode, long, float]) ) or isinstance(number,complex):
        raise Exception('num2tamilstr_american input has to be long or integer')
    if float(number) >= long(1e15):
        raise Exception('num2tamilstr input is too large')
    if float(number) < 0:
        return u"- "+num2tamilstr_american( -float(number) )

    units = (u'பூஜ்ஜியம்', u'ஒன்று', u'இரண்டு', u'மூன்று', u'நான்கு', u'ஐந்து', u'ஆறு', u'ஏழு', u'எட்டு', u'ஒன்பது', u'பத்து') # 0-10
    hundreds = ( u'நூறு', u'இருநூறு', u'முன்னூறு', u'நாநூறு',u'ஐநூறு', u'அறுநூறு', u'எழுநூறு', u'எண்ணூறு', u'தொள்ளாயிரம்') #100 - 900

    one_thousand_prefix = u'ஓர்'
    thousands = (u'ஆயிரம்',u'ஆயிரத்து')

    one_prefix = u'ஒரு'
    mil = u'மில்லியன்'
    million = (mil,mil)

    bil = u'பில்லியன்'
    billion = (bil,bil)

    tril = u'டிரில்லியன்'
    trillion = (tril,tril)

    n_one = 1
    n_ten = 10
    n_hundred = 100
    n_thousand = 1000
    n_million = 1000*n_thousand
    n_billion = long(1000*n_million)
    n_trillion = long(1000*n_billion)

    suffix_base = { n_trillion: trillion,
                    n_billion : billion,
                    n_million : million,
                    n_thousand : thousands}

    num_map = {n_trillion : [one_prefix,trillion[0]],
               n_billion : [one_prefix,billion[0]],
               n_million  : [one_prefix,million[0]],
               n_thousand : [one_thousand_prefix, thousands[0]],
               n_hundred : [hundreds[0]], #special
               n_ten : [units[10]],
               n_one : [units[1]]}

    all_bases = [n_trillion,n_billion, n_million, n_thousand, n_hundred, n_ten,n_one]
    allowed_bases = list(filter( lambda base: float(number) >= base, all_bases ))

    # handle fractional parts
    if float(number) > 0.0 and float(number) <= 1000.0:
        return num2tamilstr(number)

    if isinstance(number,str) or isinstance(number,unicode):
        result = u""
        number = number.strip()
        assert(len(args) == 1)
        assert(len(number) > 0)
        is_negative = number[0] == "-"
        if  is_negative:
            number = number[1:]
        frac_part = u""
        if number.find(".") >= 0:
            rat_part,frac_part = number.split(".")
            frac_part = num2tamilstr_american(u"0."+frac_part)
        else:
            rat_part = number
        if len(rat_part) > 0:
            result = num2tamilstr_american(float(rat_part))
        result = result +u" "+ frac_part
        return result.strip()

    if len(allowed_bases) >= 1:
        n_base = allowed_bases[0]
        if number == n_base:
            return u" ".join(num_map[n_base])
        quotient_number = long( number/n_base )
        residue_number = number - n_base*quotient_number

        if n_base < n_thousand:
            raise Exception("This can never happen")
        else:
            if ( quotient_number == 1 ):
                if n_base == n_thousand:
                    numeral = one_thousand_prefix+u' '
                else:
                    numeral = one_prefix+u' '
            else:
                numeral = num2tamilstr( quotient_number )
        if n_base >= n_thousand:
            suffix = suffix_base[n_base][long(residue_number >= 1)]
            if residue_number == 0:
                return numeral + u' ' + suffix
            numeral = numeral + u' ' + suffix
        residue_numeral = num2tamilstr_american( residue_number )
        return numeral+u' '+residue_numeral

    # number has to be zero
    return units[0]

Response: Convert a number into a numeral American style.


Instruction:
Explain what the following Python 3 code does
def get_tamil_words_iterable( letters ):
        #punctuations = u'-,+,/,*,>,<,_,],[,{,},(,)'.split(',')+[',']
        #isspace_or_tamil = lambda x:  not x in punctuations  and tamil.utf8.istamil(x)

        # correct algorithm for get-tamil-words
        buf = []
        for idx,let in enumerate(letters):
            if tamil.utf8.istamil( let ):
                buf.append( let )
            else:
                if len(buf) > 0:
                    yield  u"".join( buf )
                    buf = []
        if len(buf) > 0:
            yield u"".join(buf)

Response: given a list of UTF - 8 letters section them into words grouping them at spaces


Instruction:
Create a Python 3 function for
checking if structure of file is as expected and return dictionary linking names to run_IDs.

Response:
def validate_split_runs_file(split_runs_file):
    """Check if structure of file is as expected and return dictionary linking names to run_IDs."""
    try:
        content = [l.strip() for l in split_runs_file.readlines()]
        if content[0].upper().split('\t') == ['NAME', 'RUN_ID']:
            return {c.split('\t')[1]: c.split('\t')[0] for c in content[1:] if c}
        else:
            sys.exit("ERROR: Mandatory header of --split_runs tsv file not found: 'NAME', 'RUN_ID'")
            logging.error("Mandatory header of --split_runs tsv file not found: 'NAME', 'RUN_ID'")
    except IndexError:
        sys.exit("ERROR: Format of --split_runs tab separated file not as expected")
        logging.error("ERROR: Format of --split_runs tab separated file not as expected")


Instruction:
Write a Python 3 function for
changing the dataset identifiers based on the names in the dictionary.

Response:
def change_identifiers(datadf, split_dict):
    """Change the dataset identifiers based on the names in the dictionary."""
    for rid, name in split_dict.items():
        datadf.loc[datadf["runIDs"] == rid, "dataset"] = name


Instruction:
Given the following Python 3 function, write the documentation
def make_report(plots, path):
    '''
    Creates a fat html report based on the previously created files
    plots is a list of Plot objects defined by a path and title
    statsfile is the file to which the stats have been saved,
    which is parsed to a table (rather dodgy)
    '''
    logging.info("Writing html report.")
    html_head = """<!DOCTYPE html>

        <meta charset="UTF-8">

            <title>NanoComp Report</title>
        """
    html_content = ["\n\n
NanoComp report
"]
    html_content.append("
Summary statistics
")
    html_content.append(utils.stats2html(path + "NanoStats.txt"))
    html_content.append('\n
\n
\n
\n
')
    html_content.append("
Plots
")
    for plot in plots:
        html_content.append("\n
" + plot.title + "
\n" + plot.encode())
        html_content.append('\n
\n
\n
\n
')
    html_body = '\n'.join(html_content) + ""
    html_str = html_head + html_body
    with open(path + "NanoComp-report.html", "w") as html_file:
        html_file.write(html_str)
    return path + "NanoComp-report.html"

Response: Creates a fat html report based on the previously created files and plots.


Instruction:
Write a Python 3 function for
returning an HTTP response object.

Response:
def _http_response(self, url, method, data=None, **kwargs):
        """url -> full target url
           method -> method from requests
           data -> request body
           kwargs -> url formatting args
        """
        header = {'content-type': 'application/json'}

        if data:
            data = json.dumps(data)
        path = url.format(**kwargs)
        logger.debug("%s %s", method.__name__.upper(), path)
        response = method(self.host + path,
                          data=data, headers=header, **self.method_kwargs)
        logger.debug("%s %s", response.status_code, response.reason)
        response.raise_for_status()

        return response


Instruction:
Can you generate the documentation for the following Python 3 function
def search(self, q=''):
        if q:
            q = '?q=' + q
        return self._http_call('/v1/search' + q, get)

Response: GET / v1 / search


Instruction:
Write a Python 3 script for
getting the image layer

Response:
def get_images_layer(self, image_id):
        """GET /v1/images/{image_id}/layer"""
        return self._http_call(self.IMAGE_LAYER, get, image_id=image_id)


Instruction:
Can you generate the documentation for the following Python 3 function
def put_images_layer(self, image_id, data):
        return self._http_call(self.IMAGE_LAYER, put,
                               image_id=image_id, data=data)

Response: PUT the image layer


Instruction:
Can you generate a brief explanation for the following Python 3 code
def put_image_layer(self, image_id, data):
        return self._http_call(self.IMAGE_JSON, put,
                               data=data, image_id=image_id)

Response: PUT an image layer


Instruction:
Implement a function in Python 3 to
get the image layer for the given image_id.

Response:
def get_image_layer(self, image_id):
        """GET /v1/images/(image_id)/json"""
        return self._http_call(self.IMAGE_JSON, get, image_id=image_id)