| --- |
| library_name: transformers |
| language: |
| - en |
| base_model: |
| - huawei-noah/TinyBERT_General_4L_312D |
| pipeline_tag: token-classification |
| --- |
| |
| # Model Description |
| Keyphrase extraction is a technique in text analysis where you extract the keyphrases from a paragraph. |
|
|
| The **tinyBert-keyword** model is a fine-tuned version of the huawei-noah/TinyBERT_General_4L_312D model, tailored specifically for Keyphrase extraction. |
| |
| **huawei-noah/TinyBERT_General_4L_312D** is a distilled version of BERT, specifically designed to be smaller and faster for general NLP tasks. |
| |
| - **Finetuned from:** huawei-noah/TinyBERT_General_4L_312D |
|
|
|
|
| ## How to use |
| ```python |
| import torch |
| device = 'cuda' if torch.cuda.is_available() else 'cpu' |
| ``` |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForTokenClassification |
| import difflib |
| |
| tokenizer = AutoTokenizer.from_pretrained("nirusanan/tinyBert-keyword") |
| model = AutoModelForTokenClassification.from_pretrained("nirusanan/tinyBert-keyword").to(device) |
| ``` |
|
|
| ```python |
| text = """ |
| Computer Vision: VLMs are trained on large datasets of images, videos, or other visual data. They use deep neural networks to extract features and represent the visual information. |
| Natural Language Processing (NLP): VLMs are also trained on large datasets of text, which enables them to understand and generate natural language. |
| Cross-modal Interaction: The combination of computer vision and NLP allows the VLM to interact and process both visual and textual data in a unified manner. |
| Types of Vision Language Models: |
| |
| Visual-Bert: Visual-BERT (Bilinear Pooling for Visual Question Answering) is a popular VLM that uses a combination of visual feature extractors and language models. |
| LXMERT: LXMERT (Large Scale Instance and Instance-Specific Multimodal Representation Learning) is a VLM designed for visual reasoning and question answering tasks. |
| VL-BERT: VL-BERT (Visual Large Language Bert) is a VLM that uses a transformer-based architecture to model visual and textual representations. |
| """ |
| ``` |
|
|
| ```python |
| id2label = model.config.id2label |
| |
| tokenized = tokenizer( |
| text, |
| padding=True, |
| truncation=True, |
| return_offsets_mapping=True, |
| return_tensors="pt" |
| ) |
| |
| input_ids = tokenized["input_ids"].to(device) |
| attention_mask = tokenized["attention_mask"].to(device) |
| outputs = model(input_ids=input_ids, attention_mask=attention_mask) |
| predictions = torch.argmax(outputs.logits, dim=2) |
| |
| tokens = tokenizer.convert_ids_to_tokens(input_ids[0]) |
| token_predictions = [id2label[pred.item()] for pred in predictions[0]] |
| ``` |
|
|
| ```python |
| entities = [] |
| current_entity = None |
| |
| for idx, (token, pred) in enumerate(zip(tokens, token_predictions)): |
| if pred.startswith("B-"): |
| if current_entity: |
| entities.append(current_entity) |
| current_entity = {"type": pred[2:], "start": idx, "text": token} |
| elif pred.startswith("I-") and current_entity: |
| current_entity["text"] += f" {token}" |
| elif current_entity: |
| entities.append(current_entity) |
| current_entity = None |
| |
| if current_entity: |
| entities.append(current_entity) |
| ``` |
|
|
| ```python |
| keywords = [] |
| for i in entities: |
| keywords.append(i['text']) |
| ``` |
|
|
| ```python |
| def clean_keyword(keyword): |
| return keyword.replace(" ##", "") |
| |
| def find_closest_word(keyword, word_positions): |
| keyword_cleaned = clean_keyword(keyword) |
| best_match = None |
| best_score = float('inf') |
| |
| for pos, word in word_positions.items(): |
| score = difflib.SequenceMatcher(None, keyword_cleaned, word).ratio() |
| if score > 0.8 and (best_match is None or score > best_score): |
| best_match = word |
| best_score = score |
| |
| return best_match or keyword_cleaned |
| ``` |
|
|
| ```python |
| words = text.split() |
| word_positions = {i: word.strip(".,") for i, word in enumerate(words)} |
| |
| cleaned_keywords = [] |
| for keyword in keywords: |
| closest_word = find_closest_word(keyword, word_positions) |
| cleaned_keywords.append({'text': closest_word}) |
| ``` |
|
|
| ```python |
| unique_keywords = {} |
| for item in cleaned_keywords: |
| text = item['text'].lower() |
| if text not in unique_keywords: |
| unique_keywords[text] = item |
| |
| cleaned_keywords_unique = list(unique_keywords.values()) |
| |
| if len(cleaned_keywords_unique) > 5: |
| final_keywords = cleaned_keywords_unique[:5] |
| else: |
| final_keywords = cleaned_keywords_unique |
| |
| text_values = [item['text'] for item in final_keywords] |
| text_values |
| ``` |
