Dmitry Kirsanov commited on
Commit
93bd36d
·
1 Parent(s): 7f13e9d

more hand-holding

Browse files
Files changed (1) hide show
  1. understand.py +19 -7
understand.py CHANGED
@@ -1,4 +1,5 @@
1
  import re
 
2
  from nltk.tokenize import sent_tokenize
3
  import nltk
4
  nltk.download('punkt')
@@ -144,7 +145,7 @@ Here's the document:
144
  set2 = set(two_attempts[1][what['plural']])
145
  jackard[what['plural']] = len(set1.intersection(set2)) / len(set1.union(set2))
146
  if set1 != set2:
147
- print(f"===Two attempts of {what['plural']} are different, jackard similarity: {jackard[what['plural']]}")
148
  ret = self.query(
149
  f"""Below is a document inside <document></document>. It is a {self.values.get("document type")}.
150
 
@@ -240,22 +241,33 @@ Output just the document with the audience markup sentence, but otherwise unchan
240
  self.values.put("FAD with audience, diff", diff)
241
 
242
  def split_document(self):
 
 
 
243
  ret = self.query(
244
- f"""Below is a document inside <document></document>. It is a {self.values.get("document type")}.
245
 
246
  Summary of the document:
247
  {self.values.get("summary")}
248
 
249
  Read the entire document carefully. Remember everything you know about the concepts it touches.
250
 
251
- Each sentence of the document is marked up with an audience attribute that indicates who is the intended audience of this sentence. Here's the complete list of audiences:
252
 
253
- {newline.join(["- " + x for x in self.values.get("actors, generalized")])}
 
 
 
 
 
 
 
 
254
 
255
- Your goal is to select all sentences that include "{self.values.get("actors, generalized")[0]}" as the audience and rewrite them into a connected structured text that makes sense for this audience. You can rearrange the selected sentences, add other sentences from the original document, or write your own connecting sentences. Output the result as plain text, omitting all <sentence> tags.
256
 
257
- <document>{self.values.get("FAD with audience")}</document>""")
258
- self.values.put(f"""FAD for {self.values.get("actors, generalized")[0]}""", ret)
259
 
260
 
261
  def ingest(self, filename):
 
1
  import re
2
+ import xml.etree.ElementTree as ET
3
  from nltk.tokenize import sent_tokenize
4
  import nltk
5
  nltk.download('punkt')
 
145
  set2 = set(two_attempts[1][what['plural']])
146
  jackard[what['plural']] = len(set1.intersection(set2)) / len(set1.union(set2))
147
  if set1 != set2:
148
+ print(f"===Two attempts of {what['plural']} are different, jaccard similarity: {jackard[what['plural']]}")
149
  ret = self.query(
150
  f"""Below is a document inside <document></document>. It is a {self.values.get("document type")}.
151
 
 
241
  self.values.put("FAD with audience, diff", diff)
242
 
243
  def split_document(self):
244
+ audience = self.values.get("actors, generalized")[0]
245
+ derivative_string = f"<document>{self.values.get("FAD with audience")}</document>"
246
+ derivative = newline.join(ET.fromstring(derivative_string).fiindall(f"//sentence[contains(@audience,'{audience}')]/text()"))
247
  ret = self.query(
248
+ f"""Below is a document inside <document>...</document>. It is a {self.values.get("document type")}.
249
 
250
  Summary of the document:
251
  {self.values.get("summary")}
252
 
253
  Read the entire document carefully. Remember everything you know about the concepts it touches.
254
 
255
+ An editor was tasked with creative a derivative document that is addressed to {audience}. The editor compiled all sentences from the document that have information relevant for {audience}. These sentences are dumped below the main document inside <derivative-document for="{audience}">...</derivative-document>.
256
 
257
+ Your goal is to edit the derivative document for consistency. Rearrange its sentences in a way that makes sense for a reader. If necessary, write, or adapt from the original <document>, connecting sentences.
258
+
259
+ Here's the otiginal document:
260
+
261
+ <document>{self.values.get("FAD")}</document>
262
+
263
+ Here's the derivative document that you need to edit:
264
+
265
+ <derivative-document for="{audience}">{derivative}</derivative-document>
266
 
267
+ Output just your revision inside <derivative-document>...</derivative-document>.
268
 
269
+ """)
270
+ self.values.put(f"""FAD for {self.values.get("actors, generalized")[0]}""", self.parse_document(ret))
271
 
272
 
273
  def ingest(self, filename):