Rain-Bus/daily-paper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27 Commits 1 Branch 0 Tags
29fb20828e4bef1dabb1a3683f77f1bbcb864ea3
Commit Graph

7 Commits

Author SHA1 Message Date
Rain-Bus 21f16e6756 feat: refactor summarizer and PDF extraction pipeline 
- Split summarizer into summary_generator and summary_persister modules
- Refactor pdf_image_extractor to two-phase pipeline with PicoDet layout detection
- Add layout_detector service for PicoDet-S_layout_3cls integration
- Add exceptions module with ConflictError and NotFoundError
- Improve admin dashboard with better statistics and task management
- Add design review document with system optimization suggestions
- Add new tests for crawler, pdf_downloader, pipeline, and summary_utils
- Update dependencies and configuration
- Clean up dead code and improve error handling
 2026-06-13 13:16:47 +08:00
Rain-Bus 32978b3fc5 feat: add admin dashboard, pipeline service, lightbox, and update dependencies 2026-06-09 09:32:10 +08:00
Rain-Bus 4072a05460 chore: update project config, services, templates and styling 2026-06-06 00:49:45 +08:00
Rain-Bus 904eec392e feat: overhaul UI styling, improve templates, enhance services and tests 2026-06-06 00:38:56 +08:00
Rain-Bus 85c4cfb9e8 refactor: restructure services and add image/pdf extraction utilities 
- Add image_extractor, pdf_downloader, pi_client, trends services
- Add shared utils module
- Refactor summarizer, embedder, routes for cleaner separation
- Update tests to match new service structure
 2026-06-06 00:00:55 +08:00
Rain-Bus ba9afa212c feat: add compare, trends routes, embedder service, and phase5 tests 2026-06-05 23:32:06 +08:00
Rain-Bus 2cfd1a8a9f feat: add admin crawl, cleanup, delete, logs endpoints with scheduler and tests 
- Add POST /admin/crawl with TaskLock-based reentrancy guard
- Add POST /admin/cleanup (tmp files older than 24h) with CrawlLog
- Add POST /admin/delete with date range and 'DELETE' confirm token
- Add GET /admin/logs (paginated CrawlLog + DataDeleteJob viewer)
- Add app/services/cleaner.py (cleanup_tmp, delete_papers_by_date_range)
- Add app/services/scheduler.py (APScheduler daily crawl/cleanup jobs)
- Wire scheduler startup/shutdown hooks in app/main.py
- Add admin nav link in base.html and APP_HOST security warning
- Add apscheduler>=3.10 dependency
- Add tests/test_admin_phase4.py covering the new endpoints
 2026-06-05 23:07:45 +08:00