### What problem does this PR solve?
#2295
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
fix parsing spaces in russian language PDFs (#1987) (#2427)
### What problem does this PR solve?
[#1987](https://github.com/infiniflow/ragflow/issues/1987)
When scanning PDF files character by character, the parser excluded
spaces if the string did not match regex. Text from [Russian
documents](https://github.com/user-attachments/files/16659706/dogovor_oferta.pdf)
needs spaces, but it does not match the regex because it uses different
alphabet. That's why PDFs were parsed incorrectly and were almost
unusable as source. Fixed that by adding Russian alphabet to regex.
There might be problems with other languages that use different
alphabets. I additionally tested [PDF in
Spanish](https://www.scusd.edu/sites/main/files/file-attachments/howtohelpyourchildsucceedinschoolspanish.pdf?1338307816)
and old [a-zA-Z...] regex parses it correctly with spaces.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fix docx parser line bug (#1715)
H+1: Kevin Hu
dda4c86
### What problem does this PR solve?
#1704
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
fix: When parsing the bold content in PDF, the result is duplicated. (#1729)
### What problem does this PR solve?
_fix: When parsing the bold content in PDF, the result is duplicated._
the detail: [When using OCR to recognize Chinese titles, the structure
appears to be
duplicated](https://github.com/infiniflow/ragflow/issues/1718)
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fix pdfparser content confusion (#1700)
H
eec0415
### What problem does this PR solve?
#1407 #1656
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
pypdf2 to pypdf (#1684)
Kevin Hu
10534c3
### What problem does this PR solve?
pypdf and PyPDF2 possible Infinite Loop when a comment isn't followed by
a character #59
### Type of change
- [x] Refactoring
fix bug about divided by zero (#1482)
Kevin Hu
4e6516f
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix: Delete hardcode (#1464)
Yuhao Tsui
789efbc
### What problem does this PR solve?
After checking the language of the pdf, the line will hardcode the
language into Chinese
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix pdf_paser char content confusion (#1462)
H
1164cba
### What problem does this PR solve?
#1407
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix pdf_parser content confusion (#1458)
H
ece4f03
### What problem does this PR solve?
#1407
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fix occasional errors in pdf table recognition (#1277)
### What problem does this PR solve?
Fix occasional errors in pdf table recognition
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
add self-rag (#1070)
KevinHuSh
a49657b
### What problem does this PR solve?
#1069
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
- Update readme
- Add license
### Type of change
- [x] Documentation Update
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
fix bug in pdf parser (#986)
KevinHuSh
eefeab4
### What problem does this PR solve?
#963
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix #917 #915 (#946)
KevinHuSh
c61bcde
### What problem does this PR solve?
#917
#915
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fixbug for computing 'not concating feature' (#896)
### What problem does this PR solve?
When pdfparser call `_naive_vertical_merge` method，there is a "not
concating feature " value by computing difference between `b` and `b_`'s
layoutno ,but actually is `b` and `b`. I think it's a bug, so fix it.
Please check again.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix coordinate error (#686)
KevinHuSh
d684fd5
### What problem does this PR solve?
#683
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
remove PyMuPDF (#618)
KevinHuSh
5db8a67
### What problem does this PR solve?
#613
### Type of change
- [x] Other (please describe):
refine code (#595)
KevinHuSh
cfd6ece
### What problem does this PR solve?
### Type of change
- [x] Refactoring
fix exception in pdf parser (#584)
KevinHuSh
e319829
### What problem does this PR solve?
#451
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
refactor code (#583)
KevinHuSh
2d09c38
### What problem does this PR solve?
### Type of change
- [x] Refactoring
Refactor (#537)
KevinHuSh
3069c36
### What problem does this PR solve?
### Type of change
- [x] Refactoring
enlarge docker memory usage (#501)
KevinHuSh
3cefaa0
### What problem does this PR solve?
### Type of change
- [x] Refactoring
fix divide by zero bug (#447)
KevinHuSh
8f4f7c1
### What problem does this PR solve?
#445
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Bug fix pdf parse index out of range (#440)
加帆
f77c02e
### What problem does this PR solve?
fix a bug comes when parse some pdf file #436
### Type of change
- [☑️ ] Bug Fix (non-breaking change which fixes an issue)
rm page number exception for pdf parser (#424)
KevinHuSh
0e9dd76
### What problem does this PR solve?
#423
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
make sure the models will not be load twice (#422)
KevinHuSh
c9a1362
### What problem does this PR solve?
#381
### Type of change
- [x] Refactoring
let's load model from local (#163)
KevinHuSh
8ee4f9f
apply pep8 formalize (#155)
KevinHuSh
79ada0b
support snapshot download from local (#153)
KevinHuSh
8f39e7a
* support snapshot download from local
* let snapshot download from local
fix plainPdf bugs (#152)
KevinHuSh
328b4c9
refine page ranges (#147)
KevinHuSh
7f98e24
add use layout or not option (#145)
KevinHuSh
b085dec
* add use layout or not option
* trival
refine manual parser (#140)
KevinHuSh
004756c
refine for English corpus (#135)
KevinHuSh
08bab63
fix github account login issue (#132)
KevinHuSh
89444d3
refine manul parser (#131)
KevinHuSh
7d85666
add dockerfile for cuda envirement. Refine table search strategy, (#123)
KevinHuSh
9fe9fc4
refine pdf parser, add time zone to userinfo (#112)
KevinHuSh
e34cb81
fix task cancling bug (#98)
KevinHuSh
279ca43
change callback strategy, add timezone to docker (#96)
KevinHuSh
b83edb4
fix position extraction bug (#93)
KevinHuSh
4e03dc3
* fix position extraction bug
* remove delimiter for naive parser
fix table desc bugs, add positions to chunks (#91)
KevinHuSh
64a0633
solve task execution issues (#90)
KevinHuSh
bcb7249
resolve the issue of naive parser (#87)
KevinHuSh
ce95639
fix user login issue (#85)
KevinHuSh
f666f56
refine admin initialization (#75)
KevinHuSh
4c52eb9
rename vision, add layour and tsr recognizer (#70)
KevinHuSh
62e78ef
* rename vision, add layour and tsr recognizer
* trivial fixing
