Commit History
-
Fix t_ocr.py for PNG image. (#4625)
Kevin Hu### What problem does this PR solve? #4586 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)ae761d0 -
deepdoc use GPU if possible (#4618)
### What problem does this PR solve? deepdoc use GPU if possible ### Type of change - [x] Refactoringad4b91a
-
Remove usage of eval() from postprocess.py (#4571)
Mathias PanzenböckRemove usage of `eval()` from postprocess.py ### What problem does this PR solve? The use of `eval()` is a potential security risk. While the use of `eval()` is guarded and thus not a security risk normally, `assert`s aren't run if `-O` or `-OO` is passed to the interpreter, and as such then the guard would not apply. In any case there is no reason to use `eval()` here at all. ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] Other (please describe): Potential security fix if somehow the passed `modul_name` could be user controlled.93f905e
-
Update comments (#4569)
### What problem does this PR solve? Add license statement. ### Type of change - [x] Refactoring Signed-off-by: Jin Hai <haijin.chn@gmail.com>fa82d94
-
Remove use of eval() from recognizer.py (#4480)
Mathias Panzenböck`eval(op_type)` -> `getattr(operators, op_type)` ### What problem does this PR solve? Using `eval()` can lead to code injections and is entirely unnecessary here. ### Type of change - [x] Other (please describe): Best practice code improvement, preventing the possibility of code injection.1b54e4d -
Remove use of eval() from ocr.py (#4481)
Mathias Panzenböck`eval(op_name)` -> `getattr(operators, op_name)` ### What problem does this PR solve? Using `eval()` can lead to code injections and is entirely unnecessary here. ### Type of change - [x] Other (please describe): Best practice code improvement, preventing the possibility of code injection.ba2fa9c
-
Accelerate titles' embeddings. (#4492)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Performance Improvementb07ff91 -
Refactor. (#4487)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Refactoring83ee116
-
fix:t_recognizer TypeError: 'super' object is not callable (#4404)
Zhi-Qiang You +2: youzhiqiang, Kevin Hu### What problem does this PR solve? [Bug]: layout recognizer failed for wrong boxes class type #4230 (https://github.com/infiniflow/ragflow/issues/4230) ### Type of change - [✅ ] Bug Fix (non-breaking change which fixes an issue) --------- Co-authored-by: youzhiqiang <zhiqiang.you@aminer.com> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>b57c1d6
-
Fix t_recognizer issue. (#4387)
Kevin Hu### What problem does this PR solve? #4230 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)957cf28
-
Fix param error. (#4355)
Kevin Hu### What problem does this PR solve? #4230 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)2820402
-
Fix t_recognizer.py after model updating. (#4330)
Kevin Hu### What problem does this PR solve? #4230 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)1955ace
-
Catch the exception while parsing pptx. (#4202)
Kevin Hu### What problem does this PR solve? #4189 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)74b229a
-
Add Llama3.3 (#4174)
Kevin Hu### What problem does this PR solve? #4168 ### Type of change - [x] New Feature (non-breaking change which adds functionality)84758ca
-
fix chunk method "Table" losing content when the Excel file has multi… (#4123)
…ple sheets ### What problem does this PR solve? discussed in https://github.com/infiniflow/ragflow/pull/4102 - In excel_parser.py, `total` means the total number of rows in Excel, but it return in the first iterate, that lead to the wrong `to_page` - In table.py, it when Excel file has multiple sheets, it will be divided into multiple parts, every part size is 3000, `data` may be empty, because it has recorded in the last iterate. ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)75faa66
-
Upgrades Document Layout Analysis model. (#4054)
Kevin Hu### What problem does this PR solve? #4052 ### Type of change - [x] New Feature (non-breaking change which adds functionality)0129457
-
Fix json file parse (#4004)
### What problem does this PR solve? Fix json file parsing ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) --------- Signed-off-by: jinhai <haijin.chn@gmail.com>a11ffc5
-
Add back beartype (#3967)
### What problem does this PR solve? Add back beartype ### Type of change - [x] Refactoring63cb972
-
Added static check at PR CI (#3921)
### What problem does this PR solve? Added static check at PR CI ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] Refactoringfe9b6b3 -
Fix errors detected by Ruff (#3918)
### What problem does this PR solve? Fix errors detected by Ruff ### Type of change - [x] Refactoring0404a52
-
Fix parsing JSON file error (#3829)
### What problem does this PR solve? Close issue: #3828 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) Signed-off-by: jinhai <haijin.chn@gmail.com>43bceb7 -
Fix: page_chars attribute does not exist in some formats of PDF (#3796)
### What problem does this PR solve? In #3335 someone suggested to upgrade pdfplumber==0.11.1, but that didn't solve it. It's actually the special formatting in some of the pdfs that triggers the problem. ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)3c857ed
-
Fix out of boundary. (#3786)
Kevin Hu### What problem does this PR solve? #3769 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)1275b47
-
Edit chunk shall update instead of insert it (#3709)
### What problem does this PR solve? Edit chunk shall update instead of insert it. Close #3679 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)1b2aab6
-
Always open text file for write with UTF-8 (#3688)
### What problem does this PR solve? Always open text file for write with UTF-8. Close #932 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)9a73da2
-
Added kb_id filter to knn. Fix #3458 (#3513)
### What problem does this PR solve? Added kb_id filter to knn. Fix #3458 - [x] Bug Fix (non-breaking change which fixes an issue)aebd986
-
Introduced beartype (#3460)
### What problem does this PR solve? Introduced [beartype](https://github.com/beartype/beartype) for runtime type-checking. ### Type of change - [x] New Feature (non-breaking change which adds functionality)eae0334
-
Move settings initialization after module init phase (#3438)
### What problem does this PR solve? 1. Module init won't connect database any more. 2. Config in settings need to be used with settings.CONFIG_NAME ### Type of change - [x] Refactoring Signed-off-by: jinhai <haijin.chn@gmail.com>6101699
-
Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve? Use consistent log file names, introduced initLogger ### Type of change - [ ] Bug Fix (non-breaking change which fixes an issue) - [ ] New Feature (non-breaking change which adds functionality) - [ ] Documentation Update - [x] Refactoring - [ ] Performance Improvement - [ ] Other (please describe):8bc2fc9 -
fix bug about fetching knowledge graph (#3394)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)6a9fa6b
-
Rework logging (#3358)
Unified all log files into one. ### What problem does this PR solve? Unified all log files into one. ### Type of change - [x] Refactoring22fe41e
-
Fix:#3230 When parsing a docx file using the Book parsing method, to_page is always -1, resulting in a block count of 0 even if parsing is successful (#3249)
kuschzzp +1: Kevin Hu### What problem does this PR solve? When parsing a docx file using the Book parsing method, to_page is always -1, resulting in a block count of 0 even if parsing is successful Fix:#3230 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>ff43695
-
search between multiple indiices for team function (#3079)
Kevin Hu### What problem does this PR solve? #2834 ### Type of change - [x] New Feature (non-breaking change which adds functionality)95863fc
-
bigger resolution for OCR (#2919)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Performance Improvement7b6220c
-
fix: torch dependency start error (#2777)
### What problem does this PR solve? when use slim image, remove ```torch``` denpendency. ### Type of change - [✓] Bug Fix (non-breaking change which fixes an issue) --------- Co-authored-by: chongchuanbing <chongchuanbing@gmail.com> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>0de98c4
-
chore: update operators.py (#2724)
### What problem does this PR solve? substract -> subtract ### Type of change - [ ] Bug Fix (non-breaking change which fixes an issue) - [ ] New Feature (non-breaking change which adds functionality) - [ ] Documentation Update - [x] Refactoring - [ ] Performance Improvement - [ ] Other (please describe):b2a5c0f
-
trival (#2650)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)03a3e60 -
rename get_txt to get_text (#2649)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)d624a4e -
Add get_txt function (#2639)
yqkcn +1: Kevin Hu### What problem does this PR solve? Add get_txt function to reduce duplicate code ### Type of change - [x] Refactoring --------- Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>82adb1c
-
Fix: renrank_model and pdf_parser bugs | Update: session API (#2601)
liuhua +1: liuhua### What problem does this PR solve? Fix: renrank_model and pdf_parser bugs | Update: session API #2575 #2559 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] Refactoring --------- Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>678763e
-
add lighten control (#2567)
Kevin Hu### What problem does this PR solve? #2295 ### Type of change - [ ] Bug Fix (non-breaking change which fixes an issue) - [x] New Feature (non-breaking change which adds functionality) - [ ] Documentation Update - [ ] Refactoring - [ ] Performance Improvement - [ ] Other (please describe):dbcbb17
-
fix parsing spaces in russian language PDFs (#1987) (#2427)
### What problem does this PR solve? [#1987](https://github.com/infiniflow/ragflow/issues/1987) When scanning PDF files character by character, the parser excluded spaces if the string did not match regex. Text from [Russian documents](https://github.com/user-attachments/files/16659706/dogovor_oferta.pdf) needs spaces, but it does not match the regex because it uses different alphabet. That's why PDFs were parsed incorrectly and were almost unusable as source. Fixed that by adding Russian alphabet to regex. There might be problems with other languages that use different alphabets. I additionally tested [PDF in Spanish](https://www.scusd.edu/sites/main/files/file-attachments/howtohelpyourchildsucceedinschoolspanish.pdf?1338307816) and old [a-zA-Z...] regex parses it correctly with spaces. ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)bac5213
-
optimize text parser (#2144)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Performance Improvement13b285d
-
Format file format from Windows/dos to Unix (#1949)
### What problem does this PR solve? Related source file is in Windows/DOS format, they are format to Unix format. ### Type of change - [x] Refactoring Signed-off-by: Jin Hai <haijin.chn@gmail.com>aeb6dbc
-
fix parameter error (#1925)
Kevin Hu### What problem does this PR solve? ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)1f75d02
-
add sql to naive parser (#1908)
Kevin Hu### What problem does this PR solve? ### Type of change - [ ] Bug Fix (non-breaking change which fixes an issue) - [x] New Feature (non-breaking change which adds functionality)ef2a724
-
add support for eml file parser (#1768)
黄腾 +2: aopstudio, Kevin Hu ### What problem does this PR solve? add support for eml file parser #1363 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Zhedong Cen <cenzhedong2@126.com> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>6ed07a9
-
Fix docx parser line bug (#1715)
H +1: Kevin Hu### What problem does this PR solve? #1704 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) --------- Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>dda4c86 -
fix: When parsing the bold content in PDF, the result is duplicated. (#1729)
### What problem does this PR solve? _fix: When parsing the bold content in PDF, the result is duplicated._ the detail: [When using OCR to recognize Chinese titles, the structure appears to be duplicated](https://github.com/infiniflow/ragflow/issues/1718) ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)971f83c
-
Fix pdfparser error (#1707)
H### What problem does this PR solve? ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue)ea41e57