### What problem does this PR solve?
#2295
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
fix parsing spaces in russian language PDFs (#1987) (#2427)
### What problem does this PR solve?
[#1987](https://github.com/infiniflow/ragflow/issues/1987)
When scanning PDF files character by character, the parser excluded
spaces if the string did not match regex. Text from [Russian
documents](https://github.com/user-attachments/files/16659706/dogovor_oferta.pdf)
needs spaces, but it does not match the regex because it uses different
alphabet. That's why PDFs were parsed incorrectly and were almost
unusable as source. Fixed that by adding Russian alphabet to regex.
There might be problems with other languages that use different
alphabets. I additionally tested [PDF in
Spanish](https://www.scusd.edu/sites/main/files/file-attachments/howtohelpyourchildsucceedinschoolspanish.pdf?1338307816)
and old [a-zA-Z...] regex parses it correctly with spaces.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
optimize text parser (#2144)
Kevin Hu
13b285d
### What problem does this PR solve?
### Type of change
- [x] Performance Improvement
Format file format from Windows/dos to Unix (#1949)
### What problem does this PR solve?
Related source file is in Windows/DOS format, they are format to Unix
format.
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
fix parameter error (#1925)
Kevin Hu
1f75d02
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
add sql to naive parser (#1908)
Kevin Hu
ef2a724
### What problem does this PR solve?
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
add support for eml file parser (#1768)
黄腾+2: aopstudio, Kevin Hu
6ed07a9
### What problem does this PR solve?
add support for eml file parser
#1363
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Fix docx parser line bug (#1715)
H+1: Kevin Hu
dda4c86
### What problem does this PR solve?
#1704
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
fix: When parsing the bold content in PDF, the result is duplicated. (#1729)
### What problem does this PR solve?
_fix: When parsing the bold content in PDF, the result is duplicated._
the detail: [When using OCR to recognize Chinese titles, the structure
appears to be
duplicated](https://github.com/infiniflow/ragflow/issues/1718)
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fix pdfparser content confusion (#1700)
H
eec0415
### What problem does this PR solve?
#1407 #1656
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
pypdf2 to pypdf (#1684)
Kevin Hu
10534c3
### What problem does this PR solve?
pypdf and PyPDF2 possible Infinite Loop when a comment isn't followed by
a character #59
### Type of change
- [x] Refactoring
fix bug of ragflowdocxpparser (#1642)
Kevin Hu
fe5404c
### What problem does this PR solve?
#1627
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix generate error (#1590)
Kevin Hu
b5e86a6
### What problem does this PR solve?
#1550 #1210
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix bug about divided by zero (#1482)
Kevin Hu
4e6516f
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix: Delete hardcode (#1464)
Yuhao Tsui
789efbc
### What problem does this PR solve?
After checking the language of the pdf, the line will hardcode the
language into Chinese
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix pdf_paser char content confusion (#1462)
H
1164cba
### What problem does this PR solve?
#1407
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix pdf_parser content confusion (#1458)
H
ece4f03
### What problem does this PR solve?
#1407
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Support table for markdown file in general parser (#1278)
### What problem does this PR solve?
Support extracting table for markdown file in general parser
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Fix occasional errors in pdf table recognition (#1277)
### What problem does this PR solve?
Fix occasional errors in pdf table recognition
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
feat: support json file.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
fix too long query exception (#1195)
KevinHuSh
9aa975e
### What problem does this PR solve?
#1161
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
add self-rag (#1070)
KevinHuSh
a49657b
### What problem does this PR solve?
#1069
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
- Update readme
- Add license
### Type of change
- [x] Documentation Update
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
fix bug in pdf parser (#986)
KevinHuSh
eefeab4
### What problem does this PR solve?
#963
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix create dialog bug (#982)
KevinHuSh
3abc590
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
When pdfparser call `_naive_vertical_merge` method，there is a "not
concating feature " value by computing difference between `b` and `b_`'s
layoutno ,but actually is `b` and `b`. I think it's a bug, so fix it.
Please check again.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
fix minio helth bug (#850)
KevinHuSh
b837154
### What problem does this PR solve?
#643
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Split Excel file into different chunks (#847)
GYH
366c531
### What problem does this PR solve?
Split Excel into different chunk
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
fix coordinate error (#686)
KevinHuSh
d684fd5
### What problem does this PR solve?
#683
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
refine text decode (#657)
KevinHuSh
b5b25b4
### What problem does this PR solve?
#651
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
remove PyMuPDF (#618)
KevinHuSh
5db8a67
### What problem does this PR solve?
#613
### Type of change
- [x] Other (please describe):
refine code (#595)
KevinHuSh
cfd6ece
### What problem does this PR solve?
### Type of change
- [x] Refactoring
fix exception in pdf parser (#584)
KevinHuSh
e319829
### What problem does this PR solve?
#451
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
refactor code (#583)
KevinHuSh
2d09c38
### What problem does this PR solve?
### Type of change
- [x] Refactoring
Refactor (#537)
KevinHuSh
3069c36
### What problem does this PR solve?
### Type of change
- [x] Refactoring
enlarge docker memory usage (#501)
KevinHuSh
3cefaa0
### What problem does this PR solve?
### Type of change
- [x] Refactoring
Fit a lot of encodings for text file. (#458)
KevinHuSh
d54aa01
### What problem does this PR solve?
#384
### Type of change
- [x] Performance Improvement
fix divide by zero bug (#447)
KevinHuSh
8f4f7c1
### What problem does this PR solve?
#445
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Bug fix pdf parse index out of range (#440)
加帆
f77c02e
### What problem does this PR solve?
fix a bug comes when parse some pdf file #436
### Type of change
- [☑️ ] Bug Fix (non-breaking change which fixes an issue)
rm page number exception for pdf parser (#424)
KevinHuSh
0e9dd76
### What problem does this PR solve?
#423
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
make sure the models will not be load twice (#422)
KevinHuSh
c9a1362
### What problem does this PR solve?
#381
### Type of change
- [x] Refactoring
build ragflow image from scratch (#376)
jie yang
67dea7a
### What problem does this PR solve?
issue: #205
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
To avoid assertion while no rows in excel (#197)
KevinHuSh
a505adc
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/196)]
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Breaking Change (fix or feature that could cause existing
functionality not to work as expected)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Test cases
- [ ] Python SDK impacted, Need to update PyPI
- [ ] Other (please describe):
let's load model from local (#163)
KevinHuSh
8ee4f9f
apply pep8 formalize (#155)
KevinHuSh
79ada0b
support snapshot download from local (#153)
KevinHuSh
8f39e7a
* support snapshot download from local
* let snapshot download from local
黄腾 +2: aopstudio, Kevin Hu ### What problem does this PR solve? add support for eml file parser #1363 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Zhedong Cen <cenzhedong2@126.com> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>6ed07a9
