Spaces:
Runtime error
Runtime error
Dmitry Kirsanov commited on
Commit ·
a9fdece
1
Parent(s): eb2c877
pre-chew into sentences to make it easier for gpt
Browse files- requirements.txt +2 -1
- understand.py +8 -4
requirements.txt
CHANGED
|
@@ -4,4 +4,5 @@ bs4
|
|
| 4 |
transformers
|
| 5 |
anthropic
|
| 6 |
google-generativeai
|
| 7 |
-
tiktoken
|
|
|
|
|
|
| 4 |
transformers
|
| 5 |
anthropic
|
| 6 |
google-generativeai
|
| 7 |
+
tiktoken
|
| 8 |
+
nltk
|
understand.py
CHANGED
|
@@ -1,4 +1,5 @@
|
|
| 1 |
import re
|
|
|
|
| 2 |
|
| 3 |
import parse
|
| 4 |
import llm
|
|
@@ -190,6 +191,9 @@ Output the line "ACTORS:" followed by your list of actors, one per line.
|
|
| 190 |
|
| 191 |
|
| 192 |
def annotate_actors(self):
|
|
|
|
|
|
|
|
|
|
| 193 |
ret = self.query(
|
| 194 |
f"""Below is a document inside <document></document>. It is a {self.values.get("document type")}.
|
| 195 |
|
|
@@ -202,17 +206,17 @@ A reader of this document has compiled a list of actors (parties) mentioned or i
|
|
| 202 |
|
| 203 |
{newline.join(["- " + x for x in self.values.get("actors, generalized")])}
|
| 204 |
|
| 205 |
-
Your goal is to annotate each sentence of this document with its intended audience
|
| 206 |
|
| 207 |
-
For example, suppose you have a document describing the game of Sudoku, and your parties are "Players" and "Creators". Here are example sentences with audience markup:
|
| 208 |
|
| 209 |
<example>
|
| 210 |
-
Sudoku is a popular logic puzzle with numbers
|
| 211 |
</example>
|
| 212 |
|
| 213 |
Output just the document with the audience markup after each sentence, but otherwise unchanged:
|
| 214 |
|
| 215 |
-
<document>{
|
| 216 |
annotated = self.parse_document(ret)
|
| 217 |
self.values.put("FAD with actors", annotated)
|
| 218 |
|
|
|
|
| 1 |
import re
|
| 2 |
+
from nltk.tokenize import sent_tokenize
|
| 3 |
|
| 4 |
import parse
|
| 5 |
import llm
|
|
|
|
| 191 |
|
| 192 |
|
| 193 |
def annotate_actors(self):
|
| 194 |
+
sents = sent_tokenize(self.values.get("FAD"))
|
| 195 |
+
markup = "".join([f'<sentence actors="">{x}</sentence>' for x in sents])
|
| 196 |
+
|
| 197 |
ret = self.query(
|
| 198 |
f"""Below is a document inside <document></document>. It is a {self.values.get("document type")}.
|
| 199 |
|
|
|
|
| 206 |
|
| 207 |
{newline.join(["- " + x for x in self.values.get("actors, generalized")])}
|
| 208 |
|
| 209 |
+
Your goal is to annotate each sentence of this document, inside a <sentence></sentence>, with its intended audience by picking one or more parties from this list. For each sentence, ask yourself: which of the parties from the list would want to know this sentence's information? Use only the parties from the list.
|
| 210 |
|
| 211 |
+
Add the relevant partes, separated by commas, inside the "actors" atribute of each <sentence> tag. For example, suppose you have a document describing the game of Sudoku, and your parties are "Players" and "Creators". Here are example sentences with audience markup:
|
| 212 |
|
| 213 |
<example>
|
| 214 |
+
<sentence actors="Creators,Players">Sudoku is a popular logic puzzle with numbers with simple rules that even beginners can easily master.</sentence> <sentence actors="Creators">A presented puzzle with more than one possible solution would usually be considered defective.</sentence> <sentence actors="Players">If you have completed finding all positions for a number, draw an X over that number in the list outside.</sentence>
|
| 215 |
</example>
|
| 216 |
|
| 217 |
Output just the document with the audience markup after each sentence, but otherwise unchanged:
|
| 218 |
|
| 219 |
+
<document>{markup}</document>""")
|
| 220 |
annotated = self.parse_document(ret)
|
| 221 |
self.values.put("FAD with actors", annotated)
|
| 222 |
|